GNU bug report logs - #52388
[PATCH] build-system: emacs: Add generation of -pkg.el files.

Previous Next

Package: guix-patches;

Reported by: Andrew Tropin <andrew <at> trop.in>

Date: Thu, 9 Dec 2021 12:53:02 UTC

Severity: normal

Tags: patch

Done: Liliana Marie Prikler <liliana.prikler <at> gmail.com>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 52388 in the body.
You can then email your comments to 52388 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to guix-patches <at> gnu.org:
bug#52388; Package guix-patches. (Thu, 09 Dec 2021 12:53:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Andrew Tropin <andrew <at> trop.in>:
New bug report received and forwarded. Copy sent to guix-patches <at> gnu.org. (Thu, 09 Dec 2021 12:53:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andrew Tropin <andrew <at> trop.in>
To: guix-patches <at> gnu.org, Liliana Marie Prikler <liliana.prikler <at> gmail.com>
Cc: Ivan Sokolov <ivan-p-sokolov <at> ya.ru>
Subject: [PATCH] build-system: emacs: Add generation of -pkg.el files.
Date: Thu, 9 Dec 2021 12:01:46 +0300
[Message part 1 (text/plain, inline)]
* guix/build/emacs-build-system.scm (%default-exclude): Add generation of
-pkg.el files for packages, which do not provide them.
---
Implemented phase, which generates -pkg.el from comments in library file.  The
solution for finding main el file of the package is a little hacky, because
package name isn't available build time.

I took a part of the elisp implementation from melpa source code.
https://github.com/melpa/melpa/blob/master/package-build/package-build.el#L553

 guix/build/emacs-build-system.scm | 71 ++++++++++++++++++++++++++++++-
 1 file changed, 70 insertions(+), 1 deletion(-)

diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm
index f13162d6c4..ee934f4bde 100644
--- a/guix/build/emacs-build-system.scm
+++ b/guix/build/emacs-build-system.scm
@@ -154,6 +154,74 @@ (define (substitute-program-names)
           (substitute-program-names))))
     #t))
 
+(define* (find-root-library-file name)
+  (let lp ((subnames (string-split
+                      (package-name-version->elpa-name-version name) #\-))
+           (possible-name ""))
+    (cond
+     ((file-exists? (string-append possible-name ".el")) possible-name)
+     ((null? subnames) #f)
+     (#t (lp (cdr subnames) (string-append
+                             possible-name
+                             (if (string-null? possible-name) "" "-")
+                             (car subnames)))))))
+
+(define (add-pkg-expr name)
+  `(progn
+    (require 'lisp-mnt)
+    (require 'package)
+
+    (defun build-package-desc-from-library (name)
+      (package-desc-from-define
+       name
+       ;; Workaround for malformed version string (for example "24 (beta)" in
+       ;; paredit.el), try to parse version obtained by lm-version, before
+       ;; trying to create package-desc.  Otherwis the whole process of
+       ;; generation -pkg.el will fail.
+       (condition-case
+        nil
+        (version-to-list (lm-version))
+        (lm-version)
+        (error "0.0.0"))
+       (or (save-excursion
+            (goto-char (point-min))
+            (and (re-search-forward
+                  "^;;; [^ ]*\\.el ---[ \t]*\\(.*?\\)[ \t]*\\(-\\*-.*-\\*-[ \t]*\\)?$"
+                  nil t)
+                 (match-string-no-properties 1)))
+           "No description available.")
+       (when-let ((require-lines (lm-header-multiline "package-requires")))
+                 (package--prepare-dependencies
+                  (package-read-from-string
+                   (mapconcat 'identity require-lines " "))))
+       :kind       'single
+       :url        (lm-homepage)
+       :keywords   (lm-keywords-list)
+       :maintainer (lm-maintainer)
+       :authors    (lm-authors)))
+
+    (defun generate-package-description-file (name)
+      (package-generate-description-file
+       (build-package-desc-from-library name)
+       (concat name "-pkg.el")))
+
+    (condition-case err
+     (generate-package-description-file ,name)
+     (message (concat ,name "-pkg.el file generated."))
+     (error
+      (message "There are some errors during generation of -pkg.el file:")
+      (message "%s" (error-message-string err))))))
+
+(define* (add-pkg-file #:key name outputs #:allow-other-keys)
+  (define source (getcwd))
+  (chdir (elpa-directory (assoc-ref outputs "out")))
+  (let ((el-lib (find-root-library-file name)))
+    (when (and el-lib (not (file-exists? (string-append el-lib "-pkg.el"))))
+      (emacs-batch-edit-file (string-append el-lib ".el")
+        (add-pkg-expr el-lib))))
+  (chdir source)
+  #t)
+
 (define* (check #:key tests? (test-command '("make" "check"))
                 (parallel-tests? #t) #:allow-other-keys)
   "Run the tests by invoking TEST-COMMAND.
@@ -294,7 +362,8 @@ (define %standard-phases
       enable-autoloads-compilation)
     (add-after 'enable-autoloads-compilation 'patch-el-files patch-el-files)
     ;; The .el files are byte compiled directly in the store.
-    (add-after 'patch-el-files 'build build)
+    (add-after 'patch-el-files 'add-pkg-file add-pkg-file)
+    (add-after 'add-pkg-file 'build build)
     (add-after 'build 'validate-compiled-autoloads validate-compiled-autoloads)
     (add-after 'validate-compiled-autoloads 'move-doc move-doc)))
 
-- 
2.34.0

[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches <at> gnu.org:
bug#52388; Package guix-patches. (Fri, 14 Jan 2022 20:58:01 GMT) Full text and rfc822 format available.

Message #8 received at 52388 <at> debbugs.gnu.org (full text, mbox):

From: Liliana Marie Prikler <liliana.prikler <at> gmail.com>
To: Andrew Tropin <andrew <at> trop.in>, 52388 <at> debbugs.gnu.org
Cc: Ivan Sokolov <ivan-p-sokolov <at> ya.ru>
Subject: Re: [PATCH] build-system: emacs: Add generation of -pkg.el files.
Date: Fri, 14 Jan 2022 21:56:56 +0100
[Message part 1 (text/plain, inline)]
Hi Andrew,

Am Donnerstag, dem 09.12.2021 um 12:01 +0300 schrieb Andrew Tropin:
> 
> * guix/build/emacs-build-system.scm (%default-exclude): Add
> generation of
> -pkg.el files for packages, which do not provide them.
> ---
> Implemented phase, which generates -pkg.el from comments in library
> file.  The solution for finding main el file of the package is a
> little hacky, because package name isn't available build time.
> 
> I took a part of the elisp implementation from melpa source code.
> https://github.com/melpa/melpa/blob/master/package-build/package-build.el#L553
As promised, I took a deeper look at your patch.  As already noted,
there were some clean-up actions I had to perform, such as keeping to
our line limit (it was not easy, I tell you) among other things.  Also,
confusingly, your condition-case code did not handle errors and wrong
handling of the version field blew up everything for me.  Did you test
this code?

In any case, attached is my revised patch.  I so far only checked it
with emacs-olivetti -- a package whose description is missing in
current Guix Emacs.  I'll give everyone some time to test things before
pushing this however; I don't want to break a bunch of Emacs packages
scattered around various files.

Cheers
[0001-build-system-emacs-Ensure-that-package-descriptions-.patch (text/x-patch, attachment)]

Information forwarded to guix-patches <at> gnu.org:
bug#52388; Package guix-patches. (Fri, 21 Jan 2022 12:12:01 GMT) Full text and rfc822 format available.

Message #11 received at 52388 <at> debbugs.gnu.org (full text, mbox):

From: Andrew Tropin <andrew <at> trop.in>
To: Liliana Marie Prikler <liliana.prikler <at> gmail.com>, 52388 <at> debbugs.gnu.org
Cc: Ivan Sokolov <ivan-p-sokolov <at> ya.ru>
Subject: Re: [PATCH] build-system: emacs: Add generation of -pkg.el files.
Date: Fri, 21 Jan 2022 15:11:36 +0300
[Message part 1 (text/plain, inline)]
On 2022-01-14 21:56, Liliana Marie Prikler wrote:

> Hi Andrew,
>
> Am Donnerstag, dem 09.12.2021 um 12:01 +0300 schrieb Andrew Tropin:
>> 
>> * guix/build/emacs-build-system.scm (%default-exclude): Add
>> generation of
>> -pkg.el files for packages, which do not provide them.
>> ---
>> Implemented phase, which generates -pkg.el from comments in library
>> file.  The solution for finding main el file of the package is a
>> little hacky, because package name isn't available build time.
>> 
>> I took a part of the elisp implementation from melpa source code.
>> https://github.com/melpa/melpa/blob/master/package-build/package-build.el#L553
> As promised, I took a deeper look at your patch.  As already noted,
> there were some clean-up actions I had to perform, such as keeping to
> our line limit (it was not easy, I tell you) among other things.  Also,
> confusingly, your condition-case code did not handle errors and wrong
> handling of the version field blew up everything for me.  Did you test
> this code?

I thought it was the version I used for past a few weeks, however I
could send a little outdated version, not the latest one by accident.
Tested it with guix build emacs-paredit.

See your fix which wraps both forms in one let, thank you, yep, it's
very likely I sent an intermediate version of the patch accidentially.

>
> In any case, attached is my revised patch.  I so far only checked it
> with emacs-olivetti -- a package whose description is missing in
> current Guix Emacs.  I'll give everyone some time to test things before
> pushing this however; I don't want to break a bunch of Emacs packages
> scattered around various files.
>
> Cheers
> From 9fa6a09a131cfe436ca053c960ed9625263bc650 Mon Sep 17 00:00:00 2001
> From: Andrew Tropin <andrew <at> trop.in>
> Date: Thu, 9 Dec 2021 12:01:46 +0300
> Subject: [PATCH] build-system: emacs: Ensure that package descriptions are
>  generated.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> This patch addresses the second part of <https://bugs.gnu.org/48331>.
> While existing -pkg.el files were previously installed, no such files
> were generated for packages lacking them, resulting in packages not
> being listed as installed and not being available towards
> “describe-package”.
>
> * guix/build/emacs-build-system.scm (find-root-library-file)
> (ensure-package-description): New variables.
> (%standard-phases): Add ‘ensure-package-description’.
> ---
>  guix/build/emacs-build-system.scm | 77 ++++++++++++++++++++++++++++++-
>  1 file changed, 76 insertions(+), 1 deletion(-)
>
> diff --git a/guix/build/emacs-build-system.scm b/guix/build/emacs-build-system.scm
> index ab77e57f33..58d2a9b9f4 100644
> --- a/guix/build/emacs-build-system.scm
> +++ b/guix/build/emacs-build-system.scm
> @@ -140,6 +140,79 @@ (define (substitute-program-names)
>            (substitute-program-names))))
>      #t))
>  
> +(define (find-root-library-file name)
> +  (let loop ((parts (string-split
> +                     (package-name-version->elpa-name-version name) #\-))
> +             (candidate ""))
> +    (cond
> +     ;; at least one version part is given, so we don't terminate "early"
> +     ((null? parts) #f)
> +     ((string-null? candidate) (loop (cdr parts) (car parts)))
> +     ((file-exists? (string-append candidate ".el")) candidate)
> +     (else
> +      (loop (cdr parts) (string-append candidate "-" (car parts)))))))
> +
> +(define* (ensure-package-description #:key outputs #:allow-other-keys)
> +  (define (write-pkg-file name)
> +    (define summary-regexp
> +      "^;;; [^ ]*\\.el ---[ \t]*\\(.*?\\)[ \t]*\\(-\\*-.*-\\*-[ \t]*\\)?$")
> +    (define %write-pkg-file-form
> +      `(progn
> +        (require 'lisp-mnt)
> +        (require 'package)
> +
> +        (defun build-package-desc-from-library (name)
> +          (package-desc-from-define
> +           name
> +           ;; Workaround for malformed version string (for example "24 (beta)"
> +           ;; in paredit.el), try to parse version obtained by lm-version,
> +           ;; before trying to create package-desc.  Otherwis the whole process

s/Otherwis/Otherwise,

> +           ;; of generation -pkg.el will fail.
> +           (condition-case
> +            nil
> +            (let ((version (lm-version)))
> +              ;; raises an error if version is invalid
> +              (and (version-to-list version) version))
> +            (error "0.0.0"))
> +           (or (save-excursion
> +                (goto-char (point-min))
> +                (and (re-search-forward ,summary-regexp nil t)
> +                     (match-string-no-properties 1)))
> +               package--default-summary)
> +           (let ((require-lines (lm-header-multiline "package-requires")))
> +             (and require-lines
> +                  (package--prepare-dependencies
> +                   (package-read-from-string
> +                    (mapconcat 'identity require-lines " ")))))
> +           :kind       'single
> +           :url        (lm-homepage)
> +           :keywords   (lm-keywords-list)
> +           :maintainer (lm-maintainer)
> +           :authors    (lm-authors)))
> +
> +        (defun generate-package-description-file (name)
> +          (package-generate-description-file
> +           (build-package-desc-from-library name)
> +           (concat name "-pkg.el")))
> +
> +        (condition-case
> +         err
> +         (let ((name (file-name-base (buffer-file-name))))
> +           (generate-package-description-file name)
> +           (message (concat name "-pkg.el file generated.")))
> +         (error
> +          (message "There are some errors during generation of -pkg.el file:")
> +          (message "%s" (error-message-string err))))))
> +
> +    (unless (file-exists? (string-append name "-pkg.el"))
> +      (emacs-batch-edit-file (string-append name ".el")
> +        %write-pkg-file-form)))
> +
> +  (let* ((out (assoc-ref outputs "out"))
> +         (elpa-name-ver (store-directory->elpa-name-version out)))
> +    (with-directory-excursion (elpa-directory out)
> +      (and=> (find-root-library-file elpa-name-ver) write-pkg-file))))
> +
>  (define* (check #:key tests? (test-command '("make" "check"))
>                  (parallel-tests? #t) #:allow-other-keys)
>    "Run the tests by invoking TEST-COMMAND.
> @@ -279,8 +352,10 @@ (define %standard-phases
>      (add-after 'make-autoloads 'enable-autoloads-compilation
>        enable-autoloads-compilation)
>      (add-after 'enable-autoloads-compilation 'patch-el-files patch-el-files)
> +    (add-after 'patch-el-files 'ensure-package-description
> +      ensure-package-description)
>      ;; The .el files are byte compiled directly in the store.
> -    (add-after 'patch-el-files 'build build)
> +    (add-after 'ensure-package-description 'build build)
>      (add-after 'build 'validate-compiled-autoloads validate-compiled-autoloads)
>      (add-after 'validate-compiled-autoloads 'move-doc move-doc)))

Looks cleaner than original implementation, thank you!)

Applied on my local checkout of guix channel, list-packages now shows
117 external packages =)  Will keep it for a while and let you know if
any problems will reveal themselves.

-- 
Best regards,
Andrew Tropin
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches <at> gnu.org:
bug#52388; Package guix-patches. (Fri, 21 Jan 2022 16:58:01 GMT) Full text and rfc822 format available.

Message #14 received at 52388 <at> debbugs.gnu.org (full text, mbox):

From: Liliana Marie Prikler <liliana.prikler <at> gmail.com>
To: Andrew Tropin <andrew <at> trop.in>, 52388 <at> debbugs.gnu.org
Cc: Ivan Sokolov <ivan-p-sokolov <at> ya.ru>
Subject: Re: [PATCH] build-system: emacs: Add generation of -pkg.el files.
Date: Fri, 21 Jan 2022 17:57:35 +0100
Hi Andrew,

Am Freitag, dem 21.01.2022 um 15:11 +0300 schrieb Andrew Tropin:
> > As promised, I took a deeper look at your patch.  As already noted,
> > there were some clean-up actions I had to perform, such as keeping
> > to our line limit (it was not easy, I tell you) among other
> > things.  Also, confusingly, your condition-case code did not handle
> > errors and wrong handling of the version field blew up everything
> > for me.  Did you test this code?
> 
> I thought it was the version I used for past a few weeks, however I
> could send a little outdated version, not the latest one by accident.
> Tested it with guix build emacs-paredit.
> 
> See your fix which wraps both forms in one let, thank you, yep, it's
> very likely I sent an intermediate version of the patch
> accidentially.
Git index not up to date, truly a classic :)

> > +           ;; before trying to create package-desc.  Otherwis the
> > whole process
> 
> s/Otherwis/Otherwise,
Thanks.
> 

> Applied on my local checkout of guix channel, list-packages now shows
> 117 external packages =)  Will keep it for a while and let you know
> if any problems will reveal themselves.
Fair enough, I'll reset my "14 days without review" clock then :)




Information forwarded to guix-patches <at> gnu.org:
bug#52388; Package guix-patches. (Fri, 28 Jan 2022 15:30:01 GMT) Full text and rfc822 format available.

Message #17 received at 52388 <at> debbugs.gnu.org (full text, mbox):

From: Andrew Tropin <andrew <at> trop.in>
To: Liliana Marie Prikler <liliana.prikler <at> gmail.com>, 52388 <at> debbugs.gnu.org
Cc: Ivan Sokolov <ivan-p-sokolov <at> ya.ru>
Subject: Re: [PATCH] build-system: emacs: Add generation of -pkg.el files.
Date: Fri, 28 Jan 2022 18:29:06 +0300
[Message part 1 (text/plain, inline)]
On 2022-01-21 17:57, Liliana Marie Prikler wrote:

> Hi Andrew,
>
> Am Freitag, dem 21.01.2022 um 15:11 +0300 schrieb Andrew Tropin:
>> > As promised, I took a deeper look at your patch.  As already noted,
>> > there were some clean-up actions I had to perform, such as keeping
>> > to our line limit (it was not easy, I tell you) among other
>> > things.  Also, confusingly, your condition-case code did not handle
>> > errors and wrong handling of the version field blew up everything
>> > for me.  Did you test this code?
>> 
>> I thought it was the version I used for past a few weeks, however I
>> could send a little outdated version, not the latest one by accident.
>> Tested it with guix build emacs-paredit.
>> 
>> See your fix which wraps both forms in one let, thank you, yep, it's
>> very likely I sent an intermediate version of the patch
>> accidentially.
> Git index not up to date, truly a classic :)
>
>> > +           ;; before trying to create package-desc.  Otherwis the
>> > whole process
>> 
>> s/Otherwis/Otherwise,
> Thanks.
>> 
>
>> Applied on my local checkout of guix channel, list-packages now shows
>> 117 external packages =)  Will keep it for a while and let you know
>> if any problems will reveal themselves.
> Fair enough, I'll reset my "14 days without review" clock then :)

Have been using your version of the patch for a week now, added a few
more emacs packages to my config, nothing stopped to build or launch, so
it seems to work good.

-- 
Best regards,
Andrew Tropin
[signature.asc (application/pgp-signature, inline)]

Reply sent to Liliana Marie Prikler <liliana.prikler <at> gmail.com>:
You have taken responsibility. (Sat, 29 Jan 2022 07:54:02 GMT) Full text and rfc822 format available.

Notification sent to Andrew Tropin <andrew <at> trop.in>:
bug acknowledged by developer. (Sat, 29 Jan 2022 07:54:02 GMT) Full text and rfc822 format available.

Message #22 received at 52388-done <at> debbugs.gnu.org (full text, mbox):

From: Liliana Marie Prikler <liliana.prikler <at> gmail.com>
To: Andrew Tropin <andrew <at> trop.in>, 52388-done <at> debbugs.gnu.org
Cc: Ivan Sokolov <ivan-p-sokolov <at> ya.ru>
Subject: Re: [PATCH] build-system: emacs: Add generation of -pkg.el files.
Date: Sat, 29 Jan 2022 08:53:40 +0100
Am Freitag, dem 28.01.2022 um 18:29 +0300 schrieb Andrew Tropin:
> Have been using your version of the patch for a week now, added a few
> more emacs packages to my config, nothing stopped to build or launch,
> so it seems to work good.
"Seems to work good" sounds good enough to me.  Let's mark this bug as
done :)




Information forwarded to guix-patches <at> gnu.org:
bug#52388; Package guix-patches. (Mon, 31 Jan 2022 17:19:02 GMT) Full text and rfc822 format available.

Message #25 received at 52388-done <at> debbugs.gnu.org (full text, mbox):

From: Andrew Tropin <andrew <at> trop.in>
To: Liliana Marie Prikler <liliana.prikler <at> gmail.com>,
 52388-done <at> debbugs.gnu.org
Cc: Ivan Sokolov <ivan-p-sokolov <at> ya.ru>
Subject: Re: [PATCH] build-system: emacs: Add generation of -pkg.el files.
Date: Mon, 31 Jan 2022 20:18:38 +0300
[Message part 1 (text/plain, inline)]
On 2022-01-29 08:53, Liliana Marie Prikler wrote:

> Am Freitag, dem 28.01.2022 um 18:29 +0300 schrieb Andrew Tropin:
>> Have been using your version of the patch for a week now, added a few
>> more emacs packages to my config, nothing stopped to build or launch,
>> so it seems to work good.
> "Seems to work good" sounds good enough to me.  Let's mark this bug as
> done :)

Yay!  Thank you for helping with it! :)

-- 
Best regards,
Andrew Tropin
[signature.asc (application/pgp-signature, inline)]

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Tue, 01 Mar 2022 12:24:10 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 29 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.