GNU bug report logs - #79027
xref/pp changes in latest versions?

Previous Next

To reply to this bug, email your comments to 79027 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: xref/pp changes in latest versions?
Date: Wed, 16 Jul 2025 00:04:32 +0200

[Message part 1 (text/plain, inline)]

Hi,

I am experiencing an error in a CI job in GitHub in some packages I
currently maintain (verilog-ext and vhdl-ext, available on MELPA). The
error only happens in the `snapshot` build, cached from a recent Emacs
version (https://github.com/purcell/nix-emacs-ci).

In both of these packages (verilog-ext and vhdl-ext) I have created an
`xref` backend based on tree-sitter. To test it there is an ERT test suite
that writes the xref items to a text file with `(pp-default-function
#'pp-29)` and compares it with a reference file created previously using
the same function.

This test works well with all Emacs versions from 29.1 to 30.1 and with
release-snapshot. However with the latest versions it seems something in
`pp` or in `xref` might have changed that I am missing and maybe I should
update to. Is there any idea of what could it be?

I attach the log of GitHub actions failing job in case it could be useful.

Thanks in advance!

[Message part 2 (text/html, inline)]

[xref_issue.log (text/x-log, attachment)]

Message #8 received at 79027 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>, "79027 <at> debbugs.gnu.org"
 <79027 <at> debbugs.gnu.org>
Subject: RE: [External] : bug#79027: xref/pp changes in latest versions?
Date: Wed, 16 Jul 2025 00:27:01 +0000

> This test works well with all Emacs versions from
> 29.1 to 30.1 and with release-snapshot. However
> with the latest versions it seems something in `pp`
> or in `xref` might have changed that I am missing
> and maybe I should update to. Is there any idea of
> what could it be?

I wonder if this might be related to bug #78810.
Maybe not, since you say that the problem you see
doesn't exist in v29.1 to v30.1 (using `pp-29'), and
bug #78810 points to problems that start with v30.1.

Still, I wonder.  You say there's no problem with the
use of `(pp-default-function #'pp-29)'.  And standard
library `bookmark.el' was changed to use `pp-28' as a
workaround for the changed def of `pp':

(let ((pp-default-function #'pp-28))
  (dolist (i bookmark-alist) (pp i (current-buffer))))

But maybe you're seeing another problem with the
changed def of `pp' (or something from `xref'...).

I doubt this helps.  But maybe there's a connection
involving the changed definition of `pp'.

Message #11 received at 79027 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
Cc: 79027 <at> debbugs.gnu.org
Subject: Re: bug#79027: xref/pp changes in latest versions?
Date: Wed, 16 Jul 2025 14:39:17 +0300

> From: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
> Date: Wed, 16 Jul 2025 00:04:32 +0200
> 
> I am experiencing an error in a CI job in GitHub in some packages I currently maintain (verilog-ext and
> vhdl-ext, available on MELPA). The error only happens in the `snapshot` build, cached from a recent Emacs
> version (https://github.com/purcell/nix-emacs-ci).
> 
> In both of these packages (verilog-ext and vhdl-ext) I have created an `xref` backend based on tree-sitter.
> To test it there is an ERT test suite that writes the xref items to a text file with `(pp-default-function #'pp-29)`
> and compares it with a reference file created previously using the same function.
> 
> This test works well with all Emacs versions from 29.1 to 30.1 and with release-snapshot. However with the
> latest versions it seems something in `pp` or in `xref` might have changed that I am missing and maybe I
> should update to. Is there any idea of what could it be?
> 
> I attach the log of GitHub actions failing job in case it could be useful.

Judging by the log, the failures are due to differences in text
properties, and specifically to the differences in the 'fontified'
properties.  Why is that significant for your test?  The 'fontified'
property is used internally by JIT font-lock feature of Emacs, so it
shouldn't matter to any Lisp program, unless the program is somehow
related to font-lock machinery.

For that matter, how about telling more about what
test-hdl-files-equal does and what is its purpose and main ideas of
implementation?  It's hard to tell anything intelligent without some
additional information.

Message #14 received at 79027 <at> debbugs.gnu.org (full text, mbox):

From: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79027 <at> debbugs.gnu.org
Subject: Re: bug#79027: xref/pp changes in latest versions?
Date: Wed, 16 Jul 2025 14:54:29 +0200

[Message part 1 (text/plain, inline)]

Thanks for the quick response.

The test compares a reference text file generated beforehand using `xref`
and `pp` (e.g.
https://github.com/gmlarumbe/verilog-ext/blob/main/test/ref/xref/ucontroller.xref.defs.el)
with a text file generated during the test.

Reference files and current test files are generated using the function
`test-hdl-process-file-fn`:
- Line 84 @
https://github.com/gmlarumbe/test-hdl/blob/86ed7f9dcb0d0697e6b7ce0a64d82b7245cfade9/test-hdl-common.el


This particular xref test compares a list of xref objects gathered from a
Verilog file using the following test function:
(cl-defun verilog-ext-test-xref-fn (&key refs type)
  (let (ret-val)
    (dolist (ref refs (nreverse ret-val))
      (push (verilog-ext-xref--find-symbol ref type) ret-val))))

The function `verilog-ext-xref--find-symbol` returns a list of `xref-items`
with their respective `xref-file-locations` from a hash-table created
previously by parsing the file using tree-sitter.
- Line 40 @
https://github.com/gmlarumbe/verilog-ext/blob/main/verilog-ext-xref.el

This list is pretty printed to a file using `pp-29` as
`pp-default-function` for compatibility in tests between Emacs 29 and 30.

Use of font-lock to have a list of fontified xref matches is customizable.
In this case the files are parsed and `font-lock-ensure` is run. This did
not seem to be a problem and the behaviour is the same for versions from
29.1 to 30.1.

I tried to override the existing definitions on `pp.el` and `xref.el` so
that all Emacs versions in CI used the same code. However the `snapshot`
build still fails.

El mié, 16 jul 2025 a las 13:39, Eli Zaretskii (<eliz <at> gnu.org>) escribió:

> > From: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
> > Date: Wed, 16 Jul 2025 00:04:32 +0200
> >
> > I am experiencing an error in a CI job in GitHub in some packages I
> currently maintain (verilog-ext and
> > vhdl-ext, available on MELPA). The error only happens in the `snapshot`
> build, cached from a recent Emacs
> > version (https://github.com/purcell/nix-emacs-ci).
> >
> > In both of these packages (verilog-ext and vhdl-ext) I have created an
> `xref` backend based on tree-sitter.
> > To test it there is an ERT test suite that writes the xref items to a
> text file with `(pp-default-function #'pp-29)`
> > and compares it with a reference file created previously using the same
> function.
> >
> > This test works well with all Emacs versions from 29.1 to 30.1 and with
> release-snapshot. However with the
> > latest versions it seems something in `pp` or in `xref` might have
> changed that I am missing and maybe I
> > should update to. Is there any idea of what could it be?
> >
> > I attach the log of GitHub actions failing job in case it could be
> useful.
>
> Judging by the log, the failures are due to differences in text
> properties, and specifically to the differences in the 'fontified'
> properties.  Why is that significant for your test?  The 'fontified'
> property is used internally by JIT font-lock feature of Emacs, so it
> shouldn't matter to any Lisp program, unless the program is somehow
> related to font-lock machinery.
>
> For that matter, how about telling more about what
> test-hdl-files-equal does and what is its purpose and main ideas of
> implementation?  It's hard to tell anything intelligent without some
> additional information.
>

[Message part 2 (text/html, inline)]

Message #17 received at 79027 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
Cc: 79027 <at> debbugs.gnu.org
Subject: Re: bug#79027: xref/pp changes in latest versions?
Date: Wed, 16 Jul 2025 16:28:46 +0300

> From: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
> Date: Wed, 16 Jul 2025 14:54:29 +0200
> Cc: 79027 <at> debbugs.gnu.org
> 
> The test compares a reference text file generated beforehand using `xref` and `pp` (e.g.
> https://github.com/gmlarumbe/verilog-ext/blob/main/test/ref/xref/ucontroller.xref.defs.el) with a text file
> generated during the test.
> 
> Reference files and current test files are generated using the function `test-hdl-process-file-fn`:
> - Line 84 @
> https://github.com/gmlarumbe/test-hdl/blob/86ed7f9dcb0d0697e6b7ce0a64d82b7245cfade9/test-hdl-common.el
> 
> 
> This particular xref test compares a list of xref objects gathered from a Verilog file using the following test
> function:
> (cl-defun verilog-ext-test-xref-fn (&key refs type)
>   (let (ret-val)
>     (dolist (ref refs (nreverse ret-val))
>       (push (verilog-ext-xref--find-symbol ref type) ret-val))))
> 
> The function `verilog-ext-xref--find-symbol` returns a list of `xref-items` with their respective
> `xref-file-locations` from a hash-table created previously by parsing the file using tree-sitter.
> - Line 40 @ https://github.com/gmlarumbe/verilog-ext/blob/main/verilog-ext-xref.el
> 
> This list is pretty printed to a file using `pp-29` as `pp-default-function` for compatibility in tests between
> Emacs 29 and 30.
> 
> Use of font-lock to have a list of fontified xref matches is customizable. In this case the files are parsed and
> `font-lock-ensure` is run. This did not seem to be a problem and the behaviour is the same for versions from
> 29.1 to 30.1.
> 
> I tried to override the existing definitions on `pp.el` and `xref.el` so that all Emacs versions in CI used the
> same code. However the `snapshot` build still fails.

Why can't you ignore the differences in the 'fontified' property?  It
should be completely uninteresting for your test to pay attention to
that property.  E.g., run the text through remove-text-properties to
remove the 'fontified' property, and then compare.

Message #20 received at 79027 <at> debbugs.gnu.org (full text, mbox):

From: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 79027 <at> debbugs.gnu.org
Subject: Re: bug#79027: xref/pp changes in latest versions?
Date: Wed, 16 Jul 2025 15:57:56 +0200

[Message part 1 (text/plain, inline)]

I guess it was the default behaviour of the package to take them into
account but it should not be a problem at all to ignore them for testing.

I just tried removing text properties and text file comparison now works.

Thanks a lot for your quick help!

El mié, 16 jul 2025 a las 15:29, Eli Zaretskii (<eliz <at> gnu.org>) escribió:

> > From: Gonzalo Larumbe <gonzalomlarumbe <at> gmail.com>
> > Date: Wed, 16 Jul 2025 14:54:29 +0200
> > Cc: 79027 <at> debbugs.gnu.org
> >
> > The test compares a reference text file generated beforehand using
> `xref` and `pp` (e.g.
> >
> https://github.com/gmlarumbe/verilog-ext/blob/main/test/ref/xref/ucontroller.xref.defs.el)
> with a text file
> > generated during the test.
> >
> > Reference files and current test files are generated using the function
> `test-hdl-process-file-fn`:
> > - Line 84 @
> >
> https://github.com/gmlarumbe/test-hdl/blob/86ed7f9dcb0d0697e6b7ce0a64d82b7245cfade9/test-hdl-common.el
> >
> >
> > This particular xref test compares a list of xref objects gathered from
> a Verilog file using the following test
> > function:
> > (cl-defun verilog-ext-test-xref-fn (&key refs type)
> >   (let (ret-val)
> >     (dolist (ref refs (nreverse ret-val))
> >       (push (verilog-ext-xref--find-symbol ref type) ret-val))))
> >
> > The function `verilog-ext-xref--find-symbol` returns a list of
> `xref-items` with their respective
> > `xref-file-locations` from a hash-table created previously by parsing
> the file using tree-sitter.
> > - Line 40 @
> https://github.com/gmlarumbe/verilog-ext/blob/main/verilog-ext-xref.el
> >
> > This list is pretty printed to a file using `pp-29` as
> `pp-default-function` for compatibility in tests between
> > Emacs 29 and 30.
> >
> > Use of font-lock to have a list of fontified xref matches is
> customizable. In this case the files are parsed and
> > `font-lock-ensure` is run. This did not seem to be a problem and the
> behaviour is the same for versions from
> > 29.1 to 30.1.
> >
> > I tried to override the existing definitions on `pp.el` and `xref.el` so
> that all Emacs versions in CI used the
> > same code. However the `snapshot` build still fails.
>
> Why can't you ignore the differences in the 'fontified' property?  It
> should be completely uninteresting for your test to pay attention to
> that property.  E.g., run the text through remove-text-properties to
> remove the 'fontified' property, and then compare.
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.