GNU bug report logs - #41690
[PATCH 0/1] Add svn-multi, hg, url/{tarbomb, zipbomb} to {sources, packages}.json

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 41690 in the body.
You can then email your comments to 41690 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: guix-patches <at> gnu.org
Cc: ludo <at> gnu.org, zimoun <zimon.toutoune <at> gmail.com>
Subject: [PATCH 0/1] Add svn-multi, hg, url/{tarbomb, zipbomb} to {sources,
 packages}.json
Date: Wed,  3 Jun 2020 16:58:10 +0200

Dear,

This patch applies against commit 4efa5ce of guix-artwork.  It adds missing
sources; in preparation to SWH fetcher.  Let me know if it is preferable to
split it in 3 parts: svn-multi, hg and url bombs.


The source of packages is summarized by,

--8<---------------cut here---------------start------------->8---
guix build -f /tmp/origin/lister.scm \
    | grep '#<procedure' | cut -d' ' -f2 | sort | uniq -c | sort
      1 bzr-fetch
      3 cvs-fetch
      9 url-fetch/tarbomb
     24 url-fetch/zipbomb
     28 hg-fetch
     30 computed-origin-method
     67 no-origin
    115 svn-fetch
    135 svn-multi-fetch
   3574 git-fetch
   9690 url-fetch
--8<---------------cut here---------------end--------------->8---



These sources are still unarchivable by SWH:

      1 bzr-fetch
      3 cvs-fetch
     30 computed-origin-method
     67 no-origin


Well, the 4 "historical" packages are

--8<---------------cut here---------------start------------->8---
guix build -f /tmp/origin/lister.scm \
    | grep ';;' | grep -E '(cvs|bzr)'
;; "bzr-fetch" 	# "libmemcached"
;; "cvs-fetch" 	# "emacs-w3m"
;; "cvs-fetch" 	# "gnu-standards"
;; "cvs-fetch" 	# "tidy"
--8<---------------cut here---------------end--------------->8---

and "guix refresh -l" says

--8<---------------cut here---------------start------------->8---
libmemcached
Building the following 2 packages would ensure 2 dependent packages are rebuilt: python-pylibmc <at> 1.6.1 python2-pylibmc <at> 1.6.1

emacs-w3m
No dependents other than itself: emacs-w3m <at> 2018-11-11

gnu-standards
No dependents other than itself: gnu-standards <at> 2018-02-18

tidy
Building the following 5 packages would ensure 9 dependent packages are rebuilt: hoedown <at> 3.0.7 hugin <at> 2019.2.0 emacs-telega <at> 0.6.0-0.ae09592 arcanist <at> 0.0.0-1.45a8d22 pumpa <at> 0.9.3
--8<---------------cut here---------------end--------------->8---


What do we do for these 4 packages?  Especially libmemcached and tidy?
Last, what about the 30 'computed-origin-method' packages?


All the best,
simon


--8<---------------cut here---------------start------------->8---
(define-module (lister)
  #:use-module (guix packages)
  #:use-module (guix download)
  #:use-module (guix git-download)
  #:use-module (guix svn-download)
  #:use-module (guix hg-download)
  #:use-module (guix cvs-download)
  #:use-module (guix bzr-download)
  #:use-module (gnu packages))

(fold-packages (lambda (package result)
                 (let ((method
                        (if (origin? (package-source package))
                            (let* ((method (origin-method
                                            (package-source package)))
                                   (pkg (package-name package))
                                   (print (lambda (p m)
                                            (format #t ";; ~s \t# ~s\n" m p))))
                              (cond
                               ((eq? method bzr-fetch)
                                (print pkg "bzr-fetch"))
                               ((eq? method cvs-fetch)
                                (print pkg "cvs-fetch"))
                               ((eq? method svn-multi-fetch)
                                (print pkg "svn-multi-fetch"))
                               ((eq? method hg-fetch)
                                (print pkg "hg-fetch")))
                              method)
                            (begin
                              (format #t ";; no-origin \t# ~s\n" (package-name
                                                                  package))
                              "#<procedure no-origin >"))))
                   (format #t "~s\n" method)))
               #f)


(format #t "\n\n")
(specification->package "hello")
--8<---------------cut here---------------end--------------->8---



zimoun (1):
  website: Add fetch methods to JSON sources and packages list.

 website/apps/packages/builder.scm | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

-- 
2.26.2

Message #8 received at 41690 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: 41690 <at> debbugs.gnu.org
Cc: zimoun <zimon.toutoune <at> gmail.com>
Subject: [PATCH 1/1] website: Add fetch methods to JSON sources and packages
 list.
Date: Wed,  3 Jun 2020 17:00:10 +0200

* website/apps/packages/builder.scm (origin->json): Add 'url-fetch/tarbomb',
  'url-fetc/zipbomb', 'svn-multi-fetch' and 'hg-fetch' methods.
---
 website/apps/packages/builder.scm | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/website/apps/packages/builder.scm b/website/apps/packages/builder.scm
index 85494a6..d2bccd7 100644
--- a/website/apps/packages/builder.scm
+++ b/website/apps/packages/builder.scm
@@ -43,6 +43,7 @@
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (guix svn-download)
+  #:use-module (guix hg-download)
   #:use-module (guix utils)                       ;location
   #:use-module ((guix build download) #:select (maybe-expand-mirrors))
   #:use-module (json)
@@ -102,11 +103,17 @@
          (append-map (cut maybe-expand-mirrors <> %mirrors)
                      (map string->uri urls))))
 
-  `((type . ,(cond ((eq? url-fetch method) 'url)
+  `((type . ,(cond ((or (eq? url-fetch method)
+                        (eq? url-fetch/tarbomb method)
+                        (eq? url-fetch/zipbomb method)) 'url)
                    ((eq? git-fetch method) 'git)
-                   ((eq? svn-fetch method) 'svn)
+                   ((or (eq? svn-fetch method)
+                        (eq? svn-multi-fetch method)) 'svn)
+                   ((eq? hg-fetch method) 'hg)
                    (else                   #nil)))
-    ,@(cond ((eq? url-fetch method)
+    ,@(cond ((or (eq? url-fetch method)
+                 (eq? url-fetch/tarbomb method)
+                 (eq? url-fetch/zipbomb method))
              `(("url" . ,(list->vector
                           (resolve
                            (match uri
@@ -116,13 +123,22 @@
              `(("git_url" . ,(git-reference-url uri))))
             ((eq? svn-fetch method)
              `(("svn_url" . ,(svn-reference-url uri))))
+            ((eq? svn-multi-fetch method)
+             `(("svn_url" . ,(svn-multi-reference-url uri))))
+            ((eq? hg-fetch method)
+             `(("hg_url" . ,(hg-reference-url uri))))
             (else '()))
     ,@(if (eq? method git-fetch)
           `(("git_ref" . ,(git-reference-commit uri)))
           '())
     ,@(if (eq? method svn-fetch)
-          `(("svn_revision" . ,(svn-reference-revision
-                                uri)))
+          `(("svn_revision" . ,(svn-reference-revision uri)))
+          '())
+    ,@(if (eq? method svn-multi-fetch)
+          `(("svn_revision" . ,(svn-multi-reference-revision uri)))
+          '())
+    ,@(if (eq? method hg-fetch)
+          `(("hg_changeset" . ,(hg-reference-changeset uri)))
           '())))
 
 (define (packages-json-builder)
-- 
2.26.2

Message #13 received at 41690-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: 41690-done <at> debbugs.gnu.org
Subject: Re: [bug#41690] [PATCH 0/1] Add svn-multi, hg, url/{tarbomb,
 zipbomb} to {sources, packages}.json
Date: Sat, 06 Jun 2020 12:29:45 +0200

Hi,

zimoun <zimon.toutoune <at> gmail.com> skribis:

> The source of packages is summarized by,
>
> guix build -f /tmp/origin/lister.scm \
>     | grep '#<procedure' | cut -d' ' -f2 | sort | uniq -c | sort
>       1 bzr-fetch
>       3 cvs-fetch
>       9 url-fetch/tarbomb
>      24 url-fetch/zipbomb
>      28 hg-fetch
>      30 computed-origin-method
>      67 no-origin
>     115 svn-fetch
>     135 svn-multi-fetch
>    3574 git-fetch
>    9690 url-fetch

Interesting.

> * website/apps/packages/builder.scm (origin->json): Add 'url-fetch/tarbomb',
>   'url-fetc/zipbomb', 'svn-multi-fetch' and 'hg-fetch' methods.

Applied, thank you!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.