GNU bug report logs -
#44494
etags.el xref-location-marker does not handle TAGS references to .el.gz files
Previous Next
Reported by: Pierre Rouleau <prouleau001 <at> gmail.com>
Date: Fri, 6 Nov 2020 23:24:02 UTC
Severity: normal
Tags: confirmed, patch
Merged with 2807
Done: Stefan Monnier <monnier <at> iro.umontreal.ca>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 44494 in the body.
You can then email your comments to 44494 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Fri, 06 Nov 2020 23:24:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Pierre Rouleau <prouleau001 <at> gmail.com>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Fri, 06 Nov 2020 23:24:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
This problem was detected in emacs 26.3, but is also present in emacs 27.1,
according to the code posted inside
https://github.com/emacs-mirror/emacs/blob/master/test/manual/etags/el-src/emacs/lisp/progmodes/etags.el#L2139
Problem description follows:
----------------------------BUG DESCRIPTION [ ------------------------------
The current version of etags distributed with emacs 27.0 is capable of
parsing compressed files like something.el.gz but unfortunately the
reference inside the generated TAGS file uses the name "something.el"
instead of the complete file name "something.el.gz". The same is true
for the other compressed files (.bz2, .xz and .lzma).
When trying to use the etags xref back-end to move point to a definition
inside such a file, the xref-find-definitions command will fail because the
TAGS file identifies the .el file instead of the .el.gz file.
xref-find-definitions fails because it only checks if the uncompressed
file exists, it does not try to see if the compressed file exists.
The issue is inside etags.el xref-location-marker method.
The current code is:
(cl-defmethod xref-location-marker ((l xref-etags-location))
(with-slots (tag-info file) l
(let ((buffer (find-file-noselect file)))
(with-current-buffer buffer
(save-excursion
(etags-goto-tag-location tag-info)
(point-marker))))))
One could consider that the issue is inside the etags utility.
An alternative would be to provide Emacs with the ability to check
for the presence of the .el file and if it is not present look for the
equivalent compressed files.
Here's a proposal for a solution:
(defun etags-file-or-compressed-file-for (fname)
"Return the valid file name for FNAME.
Check if FNAME is an existing file name, if not
try FNAME appended with the following compression extensions:
- \".gz\", the extension of compressed files created by gzip
- \".bz2\", the extension for compressed files created by bzip2
- \".xz\", the extension for compressed files created by xz
- \".lzma\", the extension for compressed files created by xz.
Return the file that exists or nil if nothing found."
(let ((fpath nil))
(cl-dolist (ext '(""
".gz"
".bz2"
".xz"
".lzma"))
(setq fpath (concat fname ext))
(when (file-exists-p fpath)
(cl-return fpath)))))
(cl-defmethod xref-location-marker ((l xref-etags-location))
(with-slots (tag-info file) l
(let (buffer
(fname (pel-file-or-compressed-file-for file)))
(if fname
(setq buffer (find-file-noselect fname))
(user-error "file %s (or .gz, .bz2, .xz, .lzma) does not exist"
file))
(with-current-buffer buffer
(save-excursion
(etags-goto-tag-location tag-info)
(point-marker))))))
------------------------------ END OF DESCRIPTION ]
--------------------------------
In GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.6.0)
of 2019-08-30 built on Mojave.local
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Configured using:
'configure --disable-dependency-tracking --disable-silent-rules
--enable-locallisppath=/usr/local/share/emacs/site-lisp
--infodir=/usr/local/Cellar/emacs/26.3/share/info/emacs
--prefix=/usr/local/Cellar/emacs/26.3 --with-gnutls --without-x
--with-xml2 --without-dbus --with-modules --without-ns
--without-imagemagick'
Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB MODULES THREADS
Important settings:
value of $LANG: en_CA.UTF-8
locale-coding-system: utf-8-unix
Major mode: Lisp Interaction
Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
electric-indent-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
line-number-mode: t
transient-mark-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail tool-bar rmail-loaddefs mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
regexp-opt rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
term/xterm xterm time-date elec-pair mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads kqueue
multi-tty make-network-process emacs)
Memory information:
((conses 16 96177 5831)
(symbols 48 19813 1)
(miscs 40 33 96)
(strings 32 28253 1011)
(string-bytes 1 748198)
(vectors 16 11977)
(vector-slots 8 455846 6474)
(floats 8 48 566)
(intervals 56 197 0)
(buffers 992 11))
;; ---------------------------
Thank you,
/Pierre Rouleau
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 02:23:02 GMT)
Full text and
rfc822 format available.
Message #8 received at 44494 <at> debbugs.gnu.org (full text, mbox):
On 07.11.2020 01:22, Pierre Rouleau wrote:
> This problem was detected in emacs 26.3, but is also present in emacs
> 27.1, according to the code posted inside
> https://github.com/emacs-mirror/emacs/blob/master/test/manual/etags/el-src/emacs/lisp/progmodes/etags.el#L2139
Hi!
Do you see the same problem with 'M-x find-tag'?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 03:32:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 44494 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Fri, Nov 6, 2020 at 9:22 PM Dmitry Gutov <dgutov <at> yandex.ru> wrote:
> On 07.11.2020 01:22, Pierre Rouleau wrote:
> > This problem was detected in emacs 26.3, but is also present in emacs
> > 27.1, according to the code posted inside
> >
> https://github.com/emacs-mirror/emacs/blob/master/test/manual/etags/el-src/emacs/lisp/progmodes/etags.el#L2139
>
> Hi!
>
> Do you see the same problem with 'M-x find-tag'?
>
Short answer: yes
Longer answer: you can try it on Emacs lib files.
For example, I created a TAGS file that contains the following:
define-globalized-minor-mode global-prettify-symbols-mode^?247,10355
(define-derived-mode prog-mode ^?251,10485
^L
/usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el,1014
(defvar cc-bytecomp-unbound-variables ^?76,2968
(defvar cc-bytecomp-original-functions ^?77,3011
(defvar cc-bytecomp-original-properties ^?78,3055
(defvar cc-bytecomp-loaded-files ^?79,3100
(defvar cc-bytecomp-environment-set ^?86,3302
(defmacro cc-bytecomp-debug-msg ^?88,3344
(defun cc-bytecomp-compiling-or-loading ^?93,3432
(defsubst cc-bytecomp-is-compiling ^?134,4714
(defsubst cc-bytecomp-is-loading ^?138,4857
(defun cc-bytecomp-setup-environment ^?143,5065
(defun cc-bytecomp-restore-environment ^?191,6703
(defun cc-bytecomp-load ^?256,8749
(defmacro cc-require ^?293,10222
(defmacro cc-conditional-require ^?305,10617
(defmacro cc-conditional-require-after-load ^?318,11068
(defmacro cc-provide ^?333,11627
(defmacro cc-load ^?340,11887
(defmacro cc-require-when-compile ^?351,12266
(defmacro cc-external-require ^?362,12703
(defmacro cc-bytecomp-defvar ^?371,13055
(defmacro cc-bytecomp-defun ^?392,13857
(defmacro cc-bytecomp-put ^?419,14990
(defmacro cc-bytecomp-boundp ^?437,15739
(defmacro cc-bytecomp-fboundp ^?447,16140
^L
/usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/make-mode.el,6494
(defgroup makefile ^?95,3661
(defface makefile-space^?101,3839
(defface makefile-targets^?107,4026
(defface makefile-shell^?114,4302
Then with the file
/usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-cmds.el.gz
in a buffer and cc-bytecomp not loaded trying both
M-x xref-find-definitions cc-require
and
M-x find-tag cc-require
I get:
Rerun etags: ‘^(defmacro cc-require ’ not found in
/usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el
If I update the etags.el with my suggested code, M-x xref-find-definition
works but find-tag still does not.
Since find-tag is obsolete since emacs 25.1 I did not look into it.
I did not look into the etags utility itself yet. This looks like a
consistency problem.
I would think that the etags utility should generate a reference for el.gz
files listing
the real file name. On the other hand having the fix in both places both
in etags utility
and in emacs etags.el would reduce probability of errors.
--
/Pierre
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 07:20:01 GMT)
Full text and
rfc822 format available.
Message #14 received at 44494 <at> debbugs.gnu.org (full text, mbox):
> From: Pierre Rouleau <prouleau001 <at> gmail.com>
> Date: Fri, 6 Nov 2020 18:22:46 -0500
>
> One could consider that the issue is inside the etags utility.
I don't think this is the best alternative. TAGS tables are supposed
to provide information in a way that doesn't require re-running etags
each time some change in the sources is made. The way things are now,
compressing or decompressing a file doesn't require re-running etags,
which is a Good Thing.
Handling this in the code which accesses and interprets TAGS sounds
like a better alternative.
> Here's a proposal for a solution:
>
> (defun etags-file-or-compressed-file-for (fname)
> "Return the valid file name for FNAME.
> Check if FNAME is an existing file name, if not
> try FNAME appended with the following compression extensions:
> - \".gz\", the extension of compressed files created by gzip
> - \".bz2\", the extension for compressed files created by bzip2
> - \".xz\", the extension for compressed files created by xz
> - \".lzma\", the extension for compressed files created by xz.
>
> Return the file that exists or nil if nothing found."
> (let ((fpath nil))
> (cl-dolist (ext '(""
> ".gz"
> ".bz2"
> ".xz"
> ".lzma"))
> (setq fpath (concat fname ext))
> (when (file-exists-p fpath)
> (cl-return fpath)))))
We already have in etags.el support for finding compressed files, see
tag-find-file-of-tag-noselect. Can't that be reused? If not, let's
at least reuse tags-compression-info-list, okay?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 08:01:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 44494 <at> debbugs.gnu.org (full text, mbox):
> From: Pierre Rouleau <prouleau001 <at> gmail.com>
> Date: Fri, 6 Nov 2020 22:31:02 -0500
> Cc: 44494 <at> debbugs.gnu.org
>
> Do you see the same problem with 'M-x find-tag'?
>
> Short answer: yes
>
> Longer answer: you can try it on Emacs lib files.
>
> For example, I created a TAGS file that contains the following:
>
> define-globalized-minor-mode global-prettify-symbols-mode^?247,10355
> (define-derived-mode prog-mode ^?251,10485
> ^L
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el,1014
> (defvar cc-bytecomp-unbound-variables ^?76,2968
> (defvar cc-bytecomp-original-functions ^?77,3011
> (defvar cc-bytecomp-original-properties ^?78,3055
> (defvar cc-bytecomp-loaded-files ^?79,3100
> (defvar cc-bytecomp-environment-set ^?86,3302
> (defmacro cc-bytecomp-debug-msg ^?88,3344
> (defun cc-bytecomp-compiling-or-loading ^?93,3432
> (defsubst cc-bytecomp-is-compiling ^?134,4714
> (defsubst cc-bytecomp-is-loading ^?138,4857
> (defun cc-bytecomp-setup-environment ^?143,5065
> (defun cc-bytecomp-restore-environment ^?191,6703
> (defun cc-bytecomp-load ^?256,8749
> (defmacro cc-require ^?293,10222
> (defmacro cc-conditional-require ^?305,10617
> (defmacro cc-conditional-require-after-load ^?318,11068
> (defmacro cc-provide ^?333,11627
> (defmacro cc-load ^?340,11887
> (defmacro cc-require-when-compile ^?351,12266
> (defmacro cc-external-require ^?362,12703
> (defmacro cc-bytecomp-defvar ^?371,13055
> (defmacro cc-bytecomp-defun ^?392,13857
> (defmacro cc-bytecomp-put ^?419,14990
> (defmacro cc-bytecomp-boundp ^?437,15739
> (defmacro cc-bytecomp-fboundp ^?447,16140
> ^L
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/make-mode.el,6494
> (defgroup makefile ^?95,3661
> (defface makefile-space^?101,3839
> (defface makefile-targets^?107,4026
> (defface makefile-shell^?114,4302
>
> Then with the file /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-cmds.el.gz in a buffer
> and cc-bytecomp not loaded trying both
>
> M-x xref-find-definitions cc-require
>
> and
>
> M-x find-tag cc-require
>
> I get:
>
> Rerun etags: ‘^(defmacro cc-require ’ not found in
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el
I cannot reproduce this: find-tag works in this situation, at least in
the current emacs-27 branch and in stock Emacs 27.1. Which doesn't
surprise me, since etags.el already has code that handles compressed
files.
Moreover, M-. (which uses xref) works as well. So I'm no longer sure
I understand what is the problem you are seeing. If you see this in
Emacs 26, please retry in Emacs 27, and let's take this from there.
FTR, the steps I used for reproducing were slightly different:
. "make TAGS" in the top-level directory of the Emacs source tree
. gzip lisp/abbrevs.el
. emacs -Q
. C-x C-f lisp/simple.el
. C-u M-. kill-all-abbrevs RET
And for find-tag, replace the last 2 commands with
. M-x visit-tags-table RET RET
. M-x find-tag RET kill-all-abbrevs RET
Both of these work and show abbrevs.el.gz at the correct line.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 14:16:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 44494 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sat, Nov 7, 2020 at 3:00 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> > From: Pierre Rouleau <prouleau001 <at> gmail.com>
> > Date: Fri, 6 Nov 2020 22:31:02 -0500
> > Cc: 44494 <at> debbugs.gnu.org
> >
> > Do you see the same problem with 'M-x find-tag'?
> >
> > Short answer: yes
> >
> > Longer answer: you can try it on Emacs lib files.
> >
> > For example, I created a TAGS file that contains the following:
> >
> > define-globalized-minor-mode global-prettify-symbols-mode^?247,10355
> > (define-derived-mode prog-mode ^?251,10485
> > ^L
> >
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el,1014
> > (defvar cc-bytecomp-unbound-variables ^?76,2968
> > (defvar cc-bytecomp-original-functions ^?77,3011
> > (defvar cc-bytecomp-original-properties ^?78,3055
> > (defvar cc-bytecomp-loaded-files ^?79,3100
> > (defvar cc-bytecomp-environment-set ^?86,3302
> > (defmacro cc-bytecomp-debug-msg ^?88,3344
> > (defun cc-bytecomp-compiling-or-loading ^?93,3432
> > (defsubst cc-bytecomp-is-compiling ^?134,4714
> > (defsubst cc-bytecomp-is-loading ^?138,4857
> > (defun cc-bytecomp-setup-environment ^?143,5065
> > (defun cc-bytecomp-restore-environment ^?191,6703
> > (defun cc-bytecomp-load ^?256,8749
> > (defmacro cc-require ^?293,10222
> > (defmacro cc-conditional-require ^?305,10617
> > (defmacro cc-conditional-require-after-load ^?318,11068
> > (defmacro cc-provide ^?333,11627
> > (defmacro cc-load ^?340,11887
> > (defmacro cc-require-when-compile ^?351,12266
> > (defmacro cc-external-require ^?362,12703
> > (defmacro cc-bytecomp-defvar ^?371,13055
> > (defmacro cc-bytecomp-defun ^?392,13857
> > (defmacro cc-bytecomp-put ^?419,14990
> > (defmacro cc-bytecomp-boundp ^?437,15739
> > (defmacro cc-bytecomp-fboundp ^?447,16140
> > ^L
> >
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/make-mode.el,6494
> > (defgroup makefile ^?95,3661
> > (defface makefile-space^?101,3839
> > (defface makefile-targets^?107,4026
> > (defface makefile-shell^?114,4302
> >
> > Then with the file
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-cmds.el.gz
> in a buffer
> > and cc-bytecomp not loaded trying both
> >
> > M-x xref-find-definitions cc-require
> >
> > and
> >
> > M-x find-tag cc-require
> >
> > I get:
> >
> > Rerun etags: ‘^(defmacro cc-require ’ not found in
> >
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmodes/cc-bytecomp.el
>
> I cannot reproduce this: find-tag works in this situation, at least in
> the current emacs-27 branch and in stock Emacs 27.1. Which doesn't
> surprise me, since etags.el already has code that handles compressed
> files.
>
> Moreover, M-. (which uses xref) works as well. So I'm no longer sure
> I understand what is the problem you are seeing. If you see this in
> Emacs 26, please retry in Emacs 27, and let's take this from there.
>
> FTR, the steps I used for reproducing were slightly different:
>
> . "make TAGS" in the top-level directory of the Emacs source tree
> . gzip lisp/abbrevs.el
> . emacs -Q
> . C-x C-f lisp/simple.el
> . C-u M-. kill-all-abbrevs RET
>
> And for find-tag, replace the last 2 commands with
>
> . M-x visit-tags-table RET RET
> . M-x find-tag RET kill-all-abbrevs RET
>
> Both of these work and show abbrevs.el.gz at the correct line.
>
Hi, I did try your test, both on emacs 26.3 and 27.1,
and for the exact scenario you described, it works if you search for
kill-all-abbrevs.
I should have better describe the failure scenario.
- Problem occurs in emacs 26.3 and 27.1.
- I search for cc-require , a defmacro, not a defun, and not something
that is already loaded.
- on my system all files emacs/lisp directory are .el.gz file, however,
I do not think that the fact they are all compressed is significant.
With both emacs 26.3 and 27.1, iI do:
. emacs -Q
. C-x C-f lisp/simple.el
. C-u M-x cc-require
==> message: No definitions found for cc-require
At this point:
- the xref backend is the default for .el files; etags.
- xref-backend-functions is set to (elisp--xref-backend t), a local setting.
- the abbrev feature is loaded, as M-: (featurep 'abbrev) returns t.
- So looking for kill-all-abbrevs using the elisp--xref-backend works, my
understanding is that it works because the file that holds the definition
is loaded.
I wanted to be able to access definitions from code that might not have
already been loaded so I thought I would use the etags--xref-backend
instead.
With emacs 26.3 and 27.1 with all lisp/simple files compressed,
I do (and get approximately same results, shown below):
. emacs -Q
. C-x C-f lisp/simple.el.gz
. M-x xref-etags-mode
. C-u M-x cc-require
emacs==> prompts Visit tags table (default TAGS): ....
me ====> I select the TAGS file where all definitions are stored and hit RET
- emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
/usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\
es/cc-bytecomp.el
- emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
/usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el
If I look inside the TAGS file and search for cc-require,
I see it listed under the cc-bytecomp.el file.
If at this point, I load the 2 functions I wrote and try again,
it succeeds in finding cc-require.
To recap you need to try searching for something that is not already
loaded and use the etags xref backend with a file that contains the
definition
of what one is searching and that is located inside a compressed file.
Let me know if you want me to try other scenarios.
Thanks!
/Pierre Rouleau
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 14:23:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 44494 <at> debbugs.gnu.org (full text, mbox):
> From: Pierre Rouleau <prouleau001 <at> gmail.com>
> Date: Sat, 7 Nov 2020 09:15:12 -0500
> Cc: dgutov <at> yandex.ru, 44494 <at> debbugs.gnu.org
>
> With both emacs 26.3 and 27.1, iI do:
> . emacs -Q
> . C-x C-f lisp/simple.el
> . C-u M-x cc-require
> ==> message: No definitions found for cc-require
>
> At this point:
> - the xref backend is the default for .el files; etags.
> - xref-backend-functions is set to (elisp--xref-backend t), a local setting.
> - the abbrev feature is loaded, as M-: (featurep 'abbrev) returns t.
> - So looking for kill-all-abbrevs using the elisp--xref-backend works, my
> understanding is that it works because the file that holds the definition
> is loaded.
How does this explain the fact that find-tag also worked for me? Does
it work for you?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 14:38:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 44494 <at> debbugs.gnu.org (full text, mbox):
> From: Pierre Rouleau <prouleau001 <at> gmail.com>
> Date: Sat, 7 Nov 2020 09:15:12 -0500
> Cc: dgutov <at> yandex.ru, 44494 <at> debbugs.gnu.org
>
> To recap you need to try searching for something that is not already
> loaded and use the etags xref backend with a file that contains the definition
> of what one is searching and that is located inside a compressed file.
OK, I've now tried this with paren.el, which is not loaded in "emacs
-Q". I can confirm that M-. fails to find functions in paren.el (I
tried show-paren-function, FTR), even if I use xref-etags-mode, but
"M-x find-tag" succeeds.
So I think we should try to understand why find-tag does work in this
case, and see how to make xref-find-definitions do the same. Could
you perhaps do that?
> . emacs -Q
> . C-x C-f lisp/simple.el.gz
> . M-x xref-etags-mode
> . C-u M-x cc-require
> emacs==> prompts Visit tags table (default TAGS): ....
> me ====> I select the TAGS file where all definitions are stored and hit RET
> - emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
> /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\
> es/cc-bytecomp.el
> - emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
> /usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el
Note that at this point, you have an empty cc-bytecomp.el buffer.
Which I think gives a clue as to where the problem lies.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 14:49:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 44494 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sat, Nov 7, 2020 at 9:37 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> > From: Pierre Rouleau <prouleau001 <at> gmail.com>
> > Date: Sat, 7 Nov 2020 09:15:12 -0500
> > Cc: dgutov <at> yandex.ru, 44494 <at> debbugs.gnu.org
> >
> > To recap you need to try searching for something that is not already
> > loaded and use the etags xref backend with a file that contains the
> definition
> > of what one is searching and that is located inside a compressed file.
>
> OK, I've now tried this with paren.el, which is not loaded in "emacs
> -Q". I can confirm that M-. fails to find functions in paren.el (I
> tried show-paren-function, FTR), even if I use xref-etags-mode, but
> "M-x find-tag" succeeds.
>
> So I think we should try to understand why find-tag does work in this
> case, and see how to make xref-find-definitions do the same. Could
> you perhaps do that?
>
> > . emacs -Q
> > . C-x C-f lisp/simple.el.gz
> > . M-x xref-etags-mode
> > . C-u M-x cc-require
> > emacs==> prompts Visit tags table (default TAGS): ....
> > me ====> I select the TAGS file where all definitions are stored and hit
> RET
> > - emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
> > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\
> > es/cc-bytecomp.el
> > - emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
> >
> /usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el
>
> Note that at this point, you have an empty cc-bytecomp.el buffer.
> Which I think gives a clue as to where the problem lies.
>
You are correct, I tried it with find-tag in emacs 26.3 and 27.1 and
find-tag cc-require
does find it, even with the xref-backend-functions set to its default of
(elisp--xref-backend t).
It fails with xref-find-definitions but works with find-tag.
I agree there's a need to see what differs there.
Thanks
--
/Pierre
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 15:40:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 44494 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sat, Nov 7, 2020 at 9:48 AM Pierre Rouleau <prouleau001 <at> gmail.com> wrote:
> On Sat, Nov 7, 2020 at 9:37 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
>
>> > From: Pierre Rouleau <prouleau001 <at> gmail.com>
>> > Date: Sat, 7 Nov 2020 09:15:12 -0500
>> > Cc: dgutov <at> yandex.ru, 44494 <at> debbugs.gnu.org
>> >
>> > To recap you need to try searching for something that is not already
>> > loaded and use the etags xref backend with a file that contains the
>> definition
>> > of what one is searching and that is located inside a compressed file.
>>
>> OK, I've now tried this with paren.el, which is not loaded in "emacs
>> -Q". I can confirm that M-. fails to find functions in paren.el (I
>> tried show-paren-function, FTR), even if I use xref-etags-mode, but
>> "M-x find-tag" succeeds.
>>
>> So I think we should try to understand why find-tag does work in this
>> case, and see how to make xref-find-definitions do the same. Could
>> you perhaps do that?
>>
>> > . emacs -Q
>> > . C-x C-f lisp/simple.el.gz
>> > . M-x xref-etags-mode
>> > . C-u M-x cc-require
>> > emacs==> prompts Visit tags table (default TAGS): ....
>> > me ====> I select the TAGS file where all definitions are stored and
>> hit RET
>> > - emacs 26.3 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
>> > /usr/local/Cellar/emacs/26.3/share/emacs/26.3/lisp/progmod\
>> > es/cc-bytecomp.el
>> > - emacs 27.1 ==> Rerun etags: ‘^(defmacro cc-require ’ not found in
>> >
>> /usr/local/Cellar/emacs/27.1/share/emacs/27.1/lisp/progmodes/cc-bytecomp.el
>>
>> Note that at this point, you have an empty cc-bytecomp.el buffer.
>> Which I think gives a clue as to where the problem lies.
>>
>
> You are correct, I tried it with find-tag in emacs 26.3 and 27.1 and
> find-tag cc-require
> does find it, even with the xref-backend-functions set to its default of
> (elisp--xref-backend t).
>
> It fails with xref-find-definitions but works with find-tag.
>
> I agree there's a need to see what differs there.
>
> Thanks
>
> --
> /Pierre
>
One difference is that when using find-tag is using code from etags.el
exclusively:
- find-tag-noselect
. - find-tag-in-order , which tries different predicates and the one that
succeeds is tag-implicit-name-match-p
. - it identifies the cc-bytecomp.el.gz
.- calls etags-goto-tag-location
When using xref-find-definition the xref backend is used. It's not the
same code.
The xref backend code for elisp does not find it. The backend code for
etags does not find it either.
It tries to open cc-bytecomp.el as its the file name it gets from the TAGS
file.
It detects the file not being present and reports it as missing, assuming
the file have been removed.
To me the 2 sets of code look very different.
--
/Pierre
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 15:53:01 GMT)
Full text and
rfc822 format available.
Message #35 received at 44494 <at> debbugs.gnu.org (full text, mbox):
> From: Pierre Rouleau <prouleau001 <at> gmail.com>
> Date: Sat, 7 Nov 2020 10:39:41 -0500
> Cc: dgutov <at> yandex.ru, 44494 <at> debbugs.gnu.org
>
> One difference is that when using find-tag is using code from etags.el exclusively:
> - find-tag-noselect
> . - find-tag-in-order , which tries different predicates and the one that succeeds is
> tag-implicit-name-match-p
> . - it identifies the cc-bytecomp.el.gz
> .- calls etags-goto-tag-location
The Xref etags backend also uses tag-implicit-name-match-p:
(defvar etags-xref-find-definitions-tag-order '(tag-exact-match-p
tag-implicit-name-match-p)
"Tag order used in `xref-backend-definitions' to look for definitions.
If you want `xref-find-definitions' to find the tagged files by their
file name, add `tag-partial-file-name-match-p' to the list value.")
> When using xref-find-definition the xref backend is used. It's not the same code.
> The xref backend code for elisp does not find it. The backend code for etags does not find it either.
> It tries to open cc-bytecomp.el as its the file name it gets from the TAGS file.
> It detects the file not being present and reports it as missing, assuming the file have been removed.
>
> To me the 2 sets of code look very different.
They share some of the code, at least when xref-etags-mode is used.
So it sounds like some information found by tag-implicit-name-match-p
doesn't get handed back to Xref?
For the Xref's own ELisp backend, we will probably need to code
something in xref.el.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sat, 07 Nov 2020 16:11:02 GMT)
Full text and
rfc822 format available.
Message #38 received at 44494 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sat, Nov 7, 2020 at 10:52 AM Eli Zaretskii <eliz <at> gnu.org> wrote:
> > From: Pierre Rouleau <prouleau001 <at> gmail.com>
> > Date: Sat, 7 Nov 2020 10:39:41 -0500
> > Cc: dgutov <at> yandex.ru, 44494 <at> debbugs.gnu.org
> >
> > One difference is that when using find-tag is using code from etags.el
> exclusively:
> > - find-tag-noselect
> > . - find-tag-in-order , which tries different predicates and the one
> that succeeds is
> > tag-implicit-name-match-p
> > . - it identifies the cc-bytecomp.el.gz
> > .- calls etags-goto-tag-location
>
> The Xref etags backend also uses tag-implicit-name-match-p:
>
> (defvar etags-xref-find-definitions-tag-order '(tag-exact-match-p
>
> tag-implicit-name-match-p)
> "Tag order used in `xref-backend-definitions' to look for definitions.
>
> If you want `xref-find-definitions' to find the tagged files by their
> file name, add `tag-partial-file-name-match-p' to the list value.")
>
> > When using xref-find-definition the xref backend is used. It's not the
> same code.
> > The xref backend code for elisp does not find it. The backend code for
> etags does not find it either.
> > It tries to open cc-bytecomp.el as its the file name it gets from the
> TAGS file.
> > It detects the file not being present and reports it as missing,
> assuming the file have been removed.
> >
> > To me the 2 sets of code look very different.
>
> They share some of the code, at least when xref-etags-mode is used.
> So it sounds like some information found by tag-implicit-name-match-p
> doesn't get handed back to Xref?
>
> For the Xref's own ELisp backend, we will probably need to code
> something in xref.el.
>
Sorry, I meant that find-tag-in-order is able to find the reference in the
TAGS file when
it tries the tag-implicit-name-match-p predicate. find-tag-in-order
returns a marker that identifies the .el.gz file.
I agree that for the ELisp backend something probably needs to be done to
support it.
But I also think that something must also be done for the etags xref
backend.
--
/Pierre
[Message part 2 (text/html, inline)]
Forcibly Merged 2807 44494.
Request was from
Lars Ingebrigtsen <larsi <at> gnus.org>
to
control <at> debbugs.gnu.org.
(Thu, 12 May 2022 16:24:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Thu, 12 Oct 2023 12:40:03 GMT)
Full text and
rfc822 format available.
Message #43 received at 44494 <at> debbugs.gnu.org (full text, mbox):
My way of reproducing Bug#2807 is:
In emacs repo directory:
make tags
make install
emacs -Q
M-x visit-tags-table RET /path/to/where/installed/TAGS/ended-up
M-x tags-search RET tags-search
While it doesn't error out with:
`next-file: Opening input file: no such file or directory,
It says: All files processed
without finding tags-search.
Checking messages I see:
Scanning file /usr/local/share/emacs/30.0.50/lisp/cus-start.el...
Scanning file /usr/local/share/emacs/30.0.50/lisp/international/emoji.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/fontset.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/isearch-x.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-brackets.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-category.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-combining.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-comment.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-confusable.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-decimal.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-decomposition.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-digit.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-lowercase.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-mirrored.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-name.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-numeric.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-old-name.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-special-lowercase.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-special-titlecase.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-special-uppercase.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-titlecase.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/uni-uppercase.el...
Scanning file /usr/local/share/emacs/30.0.50/lisp/international/utf-7.el...
Scanning file /usr/local/share/emacs/30.0.50/lisp/org/ox-ascii.el...
Scanning file /usr/local/share/emacs/30.0.50/lisp/org/ob-matlab.el...
Scanning file /usr/local/share/emacs/30.0.50/lisp/mail/blessmail.el...
Scanning file /usr/local/share/emacs/30.0.50/lisp/loadup.el...
Scanning file /usr/local/share/emacs/30.0.50/lisp/leim/quail/ZOZY.el...
Scanning file /usr/local/share/emacs/30.0.50/lisp/leim/ja-dic/ja-dic.el...
Scanning file
/usr/local/share/emacs/30.0.50/lisp/international/eucjp-ms.el...
Which says it's not scanning every file. And something is off. It
couldn't be scanning cus-start.el, because I only have cus-start.elc and
cus-start.el.gz in that directory. It seems to me that it's scanning
cus-load.el, which is not byte-compiled. I think that the behavior
change with regards to the OP reproducer is:
commit df1dbaf121703aebae83d2725b7aed8b961f2913
Author: Dmitry Gutov <dgutov <at> yandex.ru>
Date: Fri Jul 30 14:58:25 2021 +0200
Make fileloop skip missing files
* lisp/fileloop.el (fileloop-next-file): If a file doesn't exist,
skip to the next one (bug#44979).
For reproducing Bug#44494, I follow the same steps, but instead of
executing tags-search I do:
(require 'xref)
M-x xref-etags-mode
C-u M-. tags-search
And get:
user-error: Rerun etags: ‘^(defun tags-search ’ not found in
/usr/local/share/emacs/30.0.50/lisp/progmodes/etags.el
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Thu, 12 Oct 2023 12:46:02 GMT)
Full text and
rfc822 format available.
Message #46 received at 44494 <at> debbugs.gnu.org (full text, mbox):
On 12/10/2023 15:39, Mauro Aranda wrote:
> For reproducing Bug#44494, I follow the same steps, but instead of
> executing tags-search I do:
> (require 'xref)
> M-x xref-etags-mode
> C-u M-. tags-search
tags-search is not an Xref command. So whether you load 'xref' and turn
on xref-etags-mode, or not, should have no effect on how 'tags-search'
works.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Thu, 12 Oct 2023 12:48:02 GMT)
Full text and
rfc822 format available.
Message #49 received at 44494 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
tags 2807 patch
quit
I did some debugging, and while internally etags.el knows how to handle
compressed files, it doesn't pass good enough information to external
tools like fileloop (Bug#2807) and xref (Bug#44494).
I attach a patch to fix both bugs, Bug#2807 and Bug#44494. It reuses
tags-compression-info-list to pass the correct filename to the tools
mentioned.
[0001-Fix-searching-for-tags-in-compressed-files.patch (text/x-patch, attachment)]
Added tag(s) patch.
Request was from
Mauro Aranda <maurooaranda <at> gmail.com>
to
control <at> debbugs.gnu.org.
(Thu, 12 Oct 2023 12:48:03 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Thu, 12 Oct 2023 12:52:02 GMT)
Full text and
rfc822 format available.
Message #54 received at 44494 <at> debbugs.gnu.org (full text, mbox):
On 12/10/23 09:44, Dmitry Gutov wrote:
> On 12/10/2023 15:39, Mauro Aranda wrote:
>> For reproducing Bug#44494, I follow the same steps, but instead of
>> executing tags-search I do:
>> (require 'xref)
>> M-x xref-etags-mode
>> C-u M-. tags-search
>
> tags-search is not an Xref command. So whether you load 'xref' and
turn on xref-etags-mode, or not, should have no effect on how
'tags-search' works.
I'm not running tags-search in recipe for Bug#44494. I'm asking
xref-find-definitions (M-.) to find the definition of tags-search.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Thu, 12 Oct 2023 14:29:02 GMT)
Full text and
rfc822 format available.
Message #57 received at 44494 <at> debbugs.gnu.org (full text, mbox):
> +(defun etags--ensure-file (file)
> + "Ensure FILE can be visited.
> +
> +FILE should be an expanded file name.
> +This function tries to locate FILE, possibly adding it a suffix
> +present in `tags-compression-info-list'. If the file can't be found,
> +signals an error.
> +Else, returns the filename that can be visited for sure."
> + (let ((f (locate-file file nil tags-compression-info-list)))
> + (unless f
> + (signal 'file-missing (list "Cannot locate file in TAGS" file)))
> + f))
The patch looks pretty good, but other parts of the code use
check `auto-compression-mode` before using `tags-compression-info-list`,
so we should probably do the same here.
As other comments mention in the file, this arrangement is suboptimal
because the search for compressed filenames should probably be moved to
jka-compr's code (e.g. using `jka-compr-compression-info-list` rather
than `tags-compression-info-list`).
Historical side note: jka-compr used to have the ability to do what we
want here "transparently" (it changed things like `find-file-noselect`
to look for compressed versions of the file, among other things).
IIRC it was removed when it got integrated into Emacs (don't know why
but I assumed it was too hackish/ugly/costly/brittle).
We should arguably re-add this feature, tho maybe not transparent,
i.e. let packages who need that request that feature explicitly (like
here).
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Thu, 12 Oct 2023 15:06:02 GMT)
Full text and
rfc822 format available.
Message #60 received at 44494 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 12/10/23 11:28, Stefan Monnier via Bug reports for GNU Emacs, the
Swiss army knife of text editors wrote:
>> +(defun etags--ensure-file (file)
>> + "Ensure FILE can be visited.
>> +
>> +FILE should be an expanded file name.
>> +This function tries to locate FILE, possibly adding it a suffix
>> +present in `tags-compression-info-list'. If the file can't be found,
>> +signals an error.
>> +Else, returns the filename that can be visited for sure."
>> + (let ((f (locate-file file nil tags-compression-info-list)))
>> + (unless f
>> + (signal 'file-missing (list "Cannot locate file in TAGS" file)))
>> + f))
>
> The patch looks pretty good, but other parts of the code use
> check `auto-compression-mode` before using `tags-compression-info-list`,
> so we should probably do the same here.
Thank you! Yes, I should've added that check. I attach a patch that
incorporates the check.
> As other comments mention in the file, this arrangement is suboptimal
> because the search for compressed filenames should probably be moved to
> jka-compr's code (e.g. using `jka-compr-compression-info-list` rather
> than `tags-compression-info-list`).
Yes, but I opted to go with this simpler patch for a 24 year-old bug
report.
> Historical side note: jka-compr used to have the ability to do what we
> want here "transparently" (it changed things like `find-file-noselect`
> to look for compressed versions of the file, among other things).
> IIRC it was removed when it got integrated into Emacs (don't know why
> but I assumed it was too hackish/ugly/costly/brittle).
>
> We should arguably re-add this feature, tho maybe not transparent,
> i.e. let packages who need that request that feature explicitly (like
> here).
>
That's good to know, thank you. And I agree that the feature would be
good to have, but I hope that in the meantime we can go with the updated
patch.
[0001-Fix-searching-for-tags-in-compressed-files.patch (text/x-patch, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Thu, 12 Oct 2023 15:49:02 GMT)
Full text and
rfc822 format available.
Message #63 received at 44494 <at> debbugs.gnu.org (full text, mbox):
On 12/10/2023 15:51, Mauro Aranda wrote:
> I'm not running tags-search in recipe for Bug#44494. I'm asking
> xref-find-definitions (M-.) to find the definition of tags-search.
Ah, thanks. Your latest patch looks good to me.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#44494; Package
emacs.
(Sun, 15 Oct 2023 04:14:03 GMT)
Full text and
rfc822 format available.
Message #66 received at 44494-done <at> debbugs.gnu.org (full text, mbox):
> That's good to know, thank you. And I agree that the feature would be
> good to have, but I hope that in the meantime we can go with the updated
> patch.
Yes, of course, I pushed it to `master`, thanks.
Still hoping for Someone™ to try and move that code to jka-compr :-)
Stefan
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Sun, 12 Nov 2023 12:24:06 GMT)
Full text and
rfc822 format available.
This bug report was last modified 163 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.