GNU bug report logs - #80016
[PATCH] Load lisp sub-directories in addition to the lisp-dir

Previous Next

To reply to this bug, email your comments to 80016 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Aritro Sen <1.sen.aritro <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Cc: Eli Zaretskii <eliz <at> gnu.org>, Philip Kaludercic <philipk <at> posteo.net>
Subject: [PATCH] Load lisp sub-directories in addition to the lisp-dir
Date: Tue, 16 Dec 2025 02:18:00 +0530

[Message part 1 (text/plain, inline)]

Tags: patch

This patch implements the feature request described in bug #79874. It
introduces the keyword lisp-subdirs to use-package-vc. This allows
subdirectories of the PKG-DIR to be added to the Emacs load-path when
installing packages directly via VC (e.g., Git), in addition to the
PKG-DIR or lisp-dir itself. The subdirectory paths must be specified
relative to lisp-dir (which defaults to PKG-DIR).

This patch depends on bug#80015 being solved. This patch is written 
based on the discussions in bug#79873.

[0001-use-package-lisp-subdirs.patch (text/x-patch, attachment)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philip Kaludercic <philipk <at> posteo.net>
To: Aritro Sen <1.sen.aritro <at> gmail.com>
Cc: bug-gnu-emacs <at> gnu.org, Eli Zaretskii <eliz <at> gnu.org>
Subject: Re: [PATCH] Load lisp sub-directories in addition to the lisp-dir
Date: Thu, 18 Dec 2025 09:21:09 +0000

Aritro Sen <1.sen.aritro <at> gmail.com> writes:

> Tags: patch
>
> This patch implements the feature request described in bug #79874. It
> introduces the keyword lisp-subdirs to use-package-vc. This allows
> subdirectories of the PKG-DIR to be added to the Emacs load-path when
> installing packages directly via VC (e.g., Git), in addition to the
> PKG-DIR or lisp-dir itself. The subdirectory paths must be specified
> relative to lisp-dir (which defaults to PKG-DIR).
>
> This patch depends on bug#80015 being solved. This patch is written
> based on the discussions in bug#79873.
>
> From 70748db9a20b0c9649ba54f457e4a4ce547d62f1 Mon Sep 17 00:00:00 2001
> From: Aritro Sen <1.sen.aritro <at> gmail.com>
> Date: Tue, 16 Dec 2025 01:10:12 +0530
> Subject: [PATCH] Load lisp-subdirs in addition to pkg-dir
>
> ---
>  doc/emacs/package.texi               |  8 ++++++
>  lisp/emacs-lisp/package-vc.el        | 32 +++++++++++++----------
>  lisp/emacs-lisp/package.el           | 39 ++++++++++++++++++----------
>  lisp/use-package/use-package-core.el |  2 +-
>  4 files changed, 53 insertions(+), 28 deletions(-)
>
> diff --git a/doc/emacs/package.texi b/doc/emacs/package.texi
> index fd8a79aa922..34a867e9505 100644
> --- a/doc/emacs/package.texi
> +++ b/doc/emacs/package.texi
> @@ -715,6 +715,14 @@ Fetching Package Sources
>  use for loading the Lisp sources, which defaults to the root directory
>  of the repository.
>  
> +@item :lisp-subdirs
> +A string or a list of strings providing the names of subdirectories
> +containing additional Lisp sources.  These names are interpreted
> +relative to the directory specified by @code{:lisp-dir} (or the root
> +directory of the repository if @code{:lisp-dir} is unspecified).  The
> +specified subdirectories are added to the load path and scanned for
> +library dependencies and autoloads.

I have responded to this issue in bug#79874 (specifically
<87qzsvmvax.fsf <at> posteo.net>), I am not sure if you saw the message,
because IIRC I sent it to you before you sent this message?

>  @item :main-file
>  A string providing the main file of the project, from which to gather
>  package metadata.  If not given, the default is the package name with
> diff --git a/lisp/emacs-lisp/package-vc.el b/lisp/emacs-lisp/package-vc.el
> index 7cb4bfde51a..f3d96904da6 100644
> --- a/lisp/emacs-lisp/package-vc.el
> +++ b/lisp/emacs-lisp/package-vc.el
> @@ -530,6 +530,11 @@ package-vc--unpack-1
>           ;; Directory where package's Lisp code resides.  It may be
>           ;; equal to `checkout-dir' or be a subdirectory of it.
>           (lisp-dir (package-vc--checkout-dir pkg-desc 'lisp-dir))
> +         (lisp-subdirs (ensure-list (plist-get pkg-spec :lisp-subdirs)))
> +         (all-dirs (cons lisp-dir
> +                         (mapcar
> +                          (lambda (dir) (expand-file-name dir lisp-dir))
> +                          lisp-subdirs)))
>           missing)
>  
>      ;; In case the package was installed directly from source, the
> @@ -547,17 +552,19 @@ package-vc--unpack-1
>                  "\\|")
>               regexp-unmatchable))
>            (deps '()))
> -      (dolist (file (directory-files lisp-dir t "\\.el\\'" t))
> -        (unless (string-match-p ignored-files file)
> -          (with-temp-buffer
> -            (insert-file-contents file)
> -            (when-let* ((require-lines (lm-header-multiline "package-requires")))
> -              (thread-last
> -                (mapconcat #'identity require-lines " ")
> -                package-read-from-string
> -                lm--prepare-package-dependencies
> -                (nconc deps)
> -                (setq deps))))))
> +      (dolist (dir all-dirs)
> +        (dolist (file (directory-files dir t "\\.el\\'" t))

We can reduce the number of lines changed here by replacing the
(directory-files ...) call with something like

--8<---------------cut here---------------start------------->8---
(mapcan
 (lambda (dir)
   (directory-files dir t "\\.el\\'" t))
 all-dirs)
--8<---------------cut here---------------end--------------->8---

> +          (unless (string-match-p ignored-files file)
> +            (with-temp-buffer
> +              (insert-file-contents file)
> +              (when-let* ((require-lines (lm-header-multiline "package-requires")))
> +                (thread-last
> +                  (mapconcat #'identity require-lines " ")
> +                  package-read-from-string
> +                  lm--prepare-package-dependencies
> +                  (nconc deps)
> +                  (setq deps)))))))
> +      (setq deps (assq-delete-all (package-desc-name pkg-desc) deps))
>        (dolist (dep deps)
>          (cl-callf version-to-list (cadr dep)))
>        (setf (package-desc-reqs pkg-desc) deps)
> @@ -568,8 +575,7 @@ package-vc--unpack-1
>  
>      ;; Generate autoloads
>      (let* ((name (package-desc-name pkg-desc))
> -           (auto-name (format "%s-autoloads.el" name)))
> -      (package-generate-autoloads name lisp-dir)
> +           (auto-name (package-generate-autoloads name lisp-dir lisp-subdirs)))
>        ;; There are two cases when we wish to "indirect" the loading of
>        ;; autoload files:
>        ;;
> diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
> index 993216c6881..6f4f493eae7 100644
> --- a/lisp/emacs-lisp/package.el
> +++ b/lisp/emacs-lisp/package.el
> @@ -1127,29 +1127,40 @@ package-autoload-ensure-default-file
>  (defvar autoload-timestamps)
>  (defvar version-control)
>  
> -(defun package-generate-autoloads (name pkg-dir)
> -  "Generate autoloads in PKG-DIR for package named NAME."
> +(defun package-generate-autoloads (name pkg-dir &optional pkg-subdirs)
> +  "Generate autoloads in PKG-DIR for package named NAME.
> +If LISP-SUBDIRS is provided, it should be a list of subdirectory names
> +relative to PKG-DIR."
>    (let* ((auto-name (format "%s-autoloads.el" name))
>           ;;(ignore-name (concat name "-pkg.el"))
>           (output-file (expand-file-name auto-name pkg-dir))
> +         (all-dirs (cons pkg-dir
> +                         (mapcar
> +                          (lambda (dir) (expand-file-name dir pkg-dir))
> +                          pkg-subdirs)))
>           ;; We don't need 'em, and this makes the output reproducible.
>           (autoload-timestamps nil)
>           (backup-inhibited t)
>           (version-control 'never))
>      (loaddefs-generate
> -     pkg-dir output-file nil
> +     all-dirs output-file nil
>       (prin1-to-string
> -      '(add-to-list
> -        'load-path
> -        ;; Add the directory that will contain the autoload file to
> -        ;; the load path.  We don't hard-code `pkg-dir', to avoid
> -        ;; issues if the package directory is moved around.
> -        ;; `loaddefs-generate' has code to do this for us, but it's
> -        ;; not currently exposed.  (Bug#63625)
> -        (or (and load-file-name
> -                 (directory-file-name
> -                  (file-name-directory load-file-name)))
> -             (car load-path)))))
> +      `(let ((dir (or (and load-file-name
> +                           (directory-file-name
> +                            (file-name-directory load-file-name)))
> +                      (car load-path))))
> +         ;; Add the directory that will contain the autoload file to
> +         ;; the load path.  We don't hard-code `pkg-dir', to avoid
> +         ;; issues if the package directory is moved around.
> +         ;; `loaddefs-generate' has code to do this for us, but it's
> +         ;; not currently exposed.  (Bug#63625)
> +         (add-to-list 'load-path dir)
> +         ,@(mapcar
> +            (lambda (subdir)
> +              `(add-to-list
> +                'load-path
> +                (expand-file-name ,subdir dir)))
> +            pkg-subdirs))))

This is related to a point I may or may not have raised at some point:
Why couldn't we add a function (or probably a macro like
`package-get-version') to package.el that a package developer can
invoke/autoload that would take care of just this?  This would then
automatically cover both tarball and VC packages, without having to
adjust package specifications?

For backwards compatibility, we can then add the definition to Compat as
it shouldn't depend on any new functionality in the core.

>      (let ((buf (find-buffer-visiting output-file)))
>        (when buf (kill-buffer buf)))
>      auto-name))
> diff --git a/lisp/use-package/use-package-core.el b/lisp/use-package/use-package-core.el
> index 0f049a6f86b..a6c92d663cc 100644
> --- a/lisp/use-package/use-package-core.el
> +++ b/lisp/use-package/use-package-core.el
> @@ -1735,7 +1735,7 @@ use-package-handler/:vc
>      body))
>  
>  (defconst use-package-vc-valid-keywords
> -  '( :url :branch :lisp-dir :main-file :vc-backend :rev
> +  '( :url :branch :lisp-dir :lisp-subdirs :main-file :vc-backend :rev
>       :shell-command :make :ignored-files)
>    "Valid keywords for the `:vc' keyword.
>  See Info node `(emacs)Fetching Package Sources'.")

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.