GNU bug report logs - #36948
[PATCH 0/2] Fix the CPAN importer

Previous Next

Package: guix-patches;

Reported by: Christopher Baines <mail <at> cbaines.net>

Date: Tue, 6 Aug 2019 19:08:02 UTC

Severity: normal

Tags: patch

Done: Christopher Baines <mail <at> cbaines.net>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36948 in the body.
You can then email your comments to 36948 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to guix-patches <at> gnu.org:
bug#36948; Package guix-patches. (Tue, 06 Aug 2019 19:08:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Christopher Baines <mail <at> cbaines.net>:
New bug report received and forwarded. Copy sent to guix-patches <at> gnu.org. (Tue, 06 Aug 2019 19:08:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-patches <at> gnu.org
Subject: [PATCH 0/2] Fix the CPAN importer
Date: Tue, 06 Aug 2019 20:07:27 +0100
[Message part 1 (text/plain, inline)]
Christopher Baines (2):
  import: utils: Add hash-ref*.
  import: cpan: Adapt for the change to guile-json version 3.

 guix/import/cpan.scm  | 30 ++++++++++++++++--------------
 guix/import/utils.scm | 10 ++++++++++
 tests/cpan.scm        | 13 ++++++++-----
 3 files changed, 34 insertions(+), 19 deletions(-)
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches <at> gnu.org:
bug#36948; Package guix-patches. (Tue, 06 Aug 2019 19:18:01 GMT) Full text and rfc822 format available.

Message #8 received at 36948 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 36948 <at> debbugs.gnu.org
Subject: [PATCH 1/2] import: utils: Add hash-ref*.
Date: Tue,  6 Aug 2019 20:17:27 +0100
With the change to guile-json version 3, JSON objects are represented as hash
tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,
so add an equivalent function for hash tables.

* guix/import/utils.scm (hash-ref*): New procedure.
---
 guix/import/utils.scm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 2a3b7341fb..ed6c3ce6af 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -47,6 +47,7 @@
 
             flatten
             assoc-ref*
+            hash-ref*
 
             url-fetch
             guix-hash-url
@@ -116,6 +117,15 @@ recursively apply the procedure to the sub-list."
       (assoc-ref alist key)
       (apply assoc-ref* (assoc-ref alist key) rest)))
 
+(define (hash-ref* hash-table key . rest)
+  "Return the value for KEY from HASH-TABLE.  For each additional key specified,
+recursively apply the procedure to the sub-hash-table."
+  (if (hash-table? hash-table)
+      (if (null? rest)
+          (hash-ref hash-table key)
+          (apply hash-ref* (hash-ref hash-table key) rest))
+      #f))                              ; For consistency with assoc-ref*
+
 (define (url-fetch url file-name)
   "Save the contents of URL to FILE-NAME.  Return #f on failure."
   (parameterize ((current-output-port (current-error-port)))
-- 
2.22.0





Information forwarded to guix-patches <at> gnu.org:
bug#36948; Package guix-patches. (Tue, 06 Aug 2019 19:18:02 GMT) Full text and rfc822 format available.

Message #11 received at 36948 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 36948 <at> debbugs.gnu.org
Subject: [PATCH 2/2] import: cpan: Adapt for the change to guile-json version
 3.
Date: Tue,  6 Aug 2019 20:17:28 +0100
In guile-json version 3, JSON objects are represented as hash tables, rather
than alists.

* guix/import/cpan.scm (string->license): Change the match expression to match
on lists, rather than vectors.
(module->dist-name, cpan-source-url, cpan-version): Change assoc-ref to
hash-ref.
(cpan-module->sexp): Change assoc-ref to hash-ref, and assoc-ref* to
hash-ref*.
* tests/cpan.scm ("source-url-http", "source-url-https"): Convert the alist to
a hash table.
---
 guix/import/cpan.scm | 30 ++++++++++++++++--------------
 tests/cpan.scm       | 13 ++++++++-----
 2 files changed, 24 insertions(+), 19 deletions(-)

diff --git a/guix/import/cpan.scm b/guix/import/cpan.scm
index ec86f11743..0be37e715e 100644
--- a/guix/import/cpan.scm
+++ b/guix/import/cpan.scm
@@ -34,7 +34,7 @@
   #:use-module (guix ui)
   #:use-module ((guix download) #:select (download-to-store url-fetch))
   #:use-module ((guix import utils) #:select (factorize-uri
-                                              flatten assoc-ref*))
+                                              flatten hash-ref*))
   #:use-module (guix import json)
   #:use-module (guix packages)
   #:use-module (guix upstream)
@@ -76,8 +76,8 @@
    ;; ssleay
    ;; sun
    ("zlib" 'zlib)
-   (#(x) (string->license x))
-   (#(lst ...) `(list ,@(map string->license lst)))
+   ((x) (string->license x))
+   ((lst ...) `(list ,@(map string->license lst)))
    (_ #f)))
 
 (define (module->name module)
@@ -88,11 +88,11 @@
   "Return the base distribution module for a given module.  E.g. the 'ok'
 module is distributed with 'Test::Simple', so (module->dist-name \"ok\") would
 return \"Test-Simple\""
-  (assoc-ref (json-fetch (string-append
-                          "https://fastapi.metacpan.org/v1/module/"
-                          module
-                          "?fields=distribution"))
-             "distribution"))
+  (hash-ref (json-fetch (string-append
+                         "https://fastapi.metacpan.org/v1/module/"
+                         module
+                         "?fields=distribution"))
+            "distribution"))
 
 (define (package->upstream-name package)
   "Return the CPAN name of PACKAGE."
@@ -122,12 +122,12 @@ or #f on failure.  MODULE should be e.g. \"Test::Script\""
 (define (cpan-source-url meta)
   "Return the download URL for a module's source tarball."
   (regexp-substitute/global #f "http[s]?://cpan.metacpan.org"
-                            (assoc-ref meta "download_url")
+                            (hash-ref meta "download_url")
                             'pre "mirror://cpan" 'post))
 
 (define (cpan-version meta)
   "Return the version number from META."
-  (match (assoc-ref meta "version")
+  (match (hash-ref meta "version")
     ((? number? version)
      ;; version is sometimes not quoted in the module json, so it gets
      ;; imported into Guile as a number, so convert it to a string.
@@ -183,7 +183,7 @@ depend on (gnu packages perl)."
   "Return the `package' s-expression for a CPAN module from the metadata in
 META."
   (define name
-    (assoc-ref meta "distribution"))
+    (hash-ref meta "distribution"))
 
   (define (guix-name name)
     (if (string-prefix? "perl-" name)
@@ -198,7 +198,9 @@ META."
     (match (flatten
             (map (lambda (ph)
                    (filter-map (lambda (t)
-                                 (assoc-ref* meta "metadata" "prereqs" ph t))
+                                 (and=> (hash-ref* meta "metadata" "prereqs" ph t)
+                                        (lambda (h)
+                                          (hash-map->list cons h))))
                                '("requires" "recommends" "suggests")))
                  phases))
       (#f
@@ -251,9 +253,9 @@ META."
        ,@(maybe-inputs 'propagated-inputs
                        (convert-inputs '("runtime")))
        (home-page ,(cpan-home name))
-       (synopsis ,(assoc-ref meta "abstract"))
+       (synopsis ,(hash-ref meta "abstract"))
        (description fill-in-yourself!)
-       (license ,(string->license (assoc-ref meta "license"))))))
+       (license ,(string->license (hash-ref meta "license"))))))
 
 (define (cpan->guix-package module-name)
   "Fetch the metadata for PACKAGE-NAME from metacpan.org, and return the
diff --git a/tests/cpan.scm b/tests/cpan.scm
index 189dd027e6..cdd6c0e76a 100644
--- a/tests/cpan.scm
+++ b/tests/cpan.scm
@@ -24,7 +24,8 @@
   #:use-module (guix tests)
   #:use-module (guix grafts)
   #:use-module (srfi srfi-64)
-  #:use-module (ice-9 match))
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 hash-table))
 
 ;; Globally disable grafts because they can trigger early builds.
 (%graft? #f)
@@ -109,14 +110,16 @@
 
 (test-equal "source-url-http"
   ((@@ (guix import cpan) cpan-source-url)
-   `(("download_url" .
-      "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))
+   (alist->hash-table
+    `(("download_url" .
+       "http://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz"))))
   "mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")
 
 (test-equal "source-url-https"
   ((@@ (guix import cpan) cpan-source-url)
-   `(("download_url" .
-      "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")))
+   (alist->hash-table
+    `(("download_url" .
+       "https://cpan.metacpan.org/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz"))))
   "mirror://cpan/authors/id/T/TE/TEST/Foo-Bar-0.1.tar.gz")
 
 (test-end "cpan")
-- 
2.22.0





Reply sent to Christopher Baines <mail <at> cbaines.net>:
You have taken responsibility. (Wed, 21 Aug 2019 17:31:02 GMT) Full text and rfc822 format available.

Notification sent to Christopher Baines <mail <at> cbaines.net>:
bug acknowledged by developer. (Wed, 21 Aug 2019 17:31:02 GMT) Full text and rfc822 format available.

Message #16 received at 36948-done <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 36948-done <at> debbugs.gnu.org
Subject: Re: [bug#36948] [PATCH 0/2] Fix the CPAN importer
Date: Wed, 21 Aug 2019 18:30:42 +0100
[Message part 1 (text/plain, inline)]
Christopher Baines <mail <at> cbaines.net> writes:

> Christopher Baines (2):
>   import: utils: Add hash-ref*.
>   import: cpan: Adapt for the change to guile-json version 3.
>
>  guix/import/cpan.scm  | 30 ++++++++++++++++--------------
>  guix/import/utils.scm | 10 ++++++++++
>  tests/cpan.scm        | 13 ++++++++-----
>  3 files changed, 34 insertions(+), 19 deletions(-)

I've pushed these patches as 01ce7af25add55514f737af48ea6c127bedfde67
now.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to guix-patches <at> gnu.org:
bug#36948; Package guix-patches. (Thu, 22 Aug 2019 12:05:02 GMT) Full text and rfc822 format available.

Message #19 received at 36948 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 36948 <at> debbugs.gnu.org
Subject: Re: [bug#36948] [PATCH 1/2] import: utils: Add hash-ref*.
Date: Thu, 22 Aug 2019 14:04:28 +0200
Hello Chris,

Christopher Baines <mail <at> cbaines.net> skribis:

> With the change to guile-json version 3, JSON objects are represented as hash
> tables, rather than alists. The cpan importer uses assoc-ref* on a hash table,
> so add an equivalent function for hash tables.
>
> * guix/import/utils.scm (hash-ref*): New procedure.

[...]

> In guile-json version 3, JSON objects are represented as hash tables, rather
> than alists.

I seems to me that this is adapting for Guile-JSON v1, not v3: in v3,
JSON arrays map to Scheme vectors, and JSON dictionaries map to alists;
JSON dictionaries used to map to hash tables in v1.

Indeed, ‘tests/cpan.scm’ now fails for me:

--8<---------------cut here---------------start------------->8---
actual-error:
+ (wrong-type-arg
+   "scm_hash_fn_get_handle"
+   "Wrong type argument in position ~A (expecting ~A): ~S"
+   (1
+    "hash-table"
+    (("version" . "0.1")
+     ("author" . "Guix")
+     ("download_url"
+      .
+      "http://example.com/Foo-Bar-0.1.tar.gz")
[…]
--8<---------------cut here---------------end--------------->8---

Could it be that you were testing this in an environment containing v1
and not v3?

Sorry for not noticing earlier!

Thanks,
Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 20 Sep 2019 11:24:08 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 191 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.