GNU bug report logs - #24450
pypi importer outputs strange character series in optional dependency case.

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Reported by: ng0 <ng0@HIDDEN>; Keywords: patch; merged with #24557, #33047, #33569, #34266; dated Fri, 16 Sep 2016 20:02:01 UTC; Maintainer for guix is bug-guix@HIDDEN.
Added tag(s) patch. Request was from Maxim Cournoyer <maxim.cournoyer@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 24450 <at> debbugs.gnu.org:


Received: (at 24450) by debbugs.gnu.org; 31 Mar 2019 14:40:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 31 10:40:46 2019
Received: from localhost ([127.0.0.1]:38385 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hAbde-00082y-B7
	for submit <at> debbugs.gnu.org; Sun, 31 Mar 2019 10:40:46 -0400
Received: from mail-it1-f177.google.com ([209.85.166.177]:56201)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1hAbdb-00082k-Km
 for 24450 <at> debbugs.gnu.org; Sun, 31 Mar 2019 10:40:44 -0400
Received: by mail-it1-f177.google.com with SMTP id z126so10707984itd.5
 for <24450 <at> debbugs.gnu.org>; Sun, 31 Mar 2019 07:40:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=0G0xcICAc43GZzoldViEH4c2GNH7/Cn3abH0qMFli20=;
 b=i4LGtZyC6lEFTkP6Tn9INjCNVsCYxZ4m+6Fo0cjJ4NyhVB82WDnnICufzj0W9EIPxk
 IrzTK8kNMDn5zt24IslpQG8kLtzzMZl8tUZuSUcWR+6Cac5QS/xgmBNt0kPYMidj5oRe
 LekJeVhLj/BCraqrcpY1eQWntizHCxItQxZMC8HNv0AOMbgBp01DYCDLl4D82zDRGlI8
 9+PCjv4EN6MwHWJJXjQHX1N8QXodkix92g9Bak2gtMejz4hvZYc6puCEl1MF7qlgqm42
 2bDnzRzQ/fxwWQG3Sh7uQlgrRUMRIuKZ7fO8UJovfGGS86aw/IEzhuQ7EozPHNDMeQ8z
 T1Rg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=0G0xcICAc43GZzoldViEH4c2GNH7/Cn3abH0qMFli20=;
 b=PR2MNaOTn3FxLirvQhoGcdp07EdNC+Yc2knmNBLHAtlLH/tdmquppUDgRlGDuMed4b
 IqaGtupJ1LsFqdhzKpzU0kWTGaYY/ciKd/L8Et/JZ52j+RG/J5gYavHnxeUmcma25ECV
 iqCmrG0qTgfcaNvlPvWnAqD/VvdB0ISAzFkEzZo5Km5TxJBy/GwFEY2q2HXQ76jeyPz3
 gZ+A82iWPXPHz96VA37DeYaaHnQpcXE+LQOH17JVP5LBjo3fnT90iBE8nKgC1Hhijy9L
 of3/ondd72m8n5IS13H7bls4I00oN2aZyVq1C/lBQbzr7q53DClOx9+oqyATggTtMJij
 1yNg==
X-Gm-Message-State: APjAAAXaZr9tZoOgJumfTl4h3XiTNYxVV3TuDnyaYSuQDiwBrNn8ZwWi
 wAekn+MNIc9HlTW2ta2wlSd8vQh6
X-Google-Smtp-Source: APXvYqyygj12BQYizOr2VYfk+yOF2hPhw62wEGkmz8uAVbJWfe6Rc/PL2G+zJJs87i7SUfHEAFCTUQ==
X-Received: by 2002:a24:ad2:: with SMTP id 201mr11237869itw.26.1554043237593; 
 Sun, 31 Mar 2019 07:40:37 -0700 (PDT)
Received: from kwak ([2607:f2c0:94b4:fa00::235])
 by smtp.gmail.com with ESMTPSA id v10sm5039307itv.8.2019.03.31.07.40.35
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Sun, 31 Mar 2019 07:40:36 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: ng0 <ng0@HIDDEN>
Subject: [PATCH] bug#24450: pypi importer outputs strange character series in
 optional dependency case.
References: <87h99fipj1.fsf@HIDDEN> <87tvfm1eos.fsf@HIDDEN>
 <877ech5cvd.fsf_-_@HIDDEN>
Date: Sun, 31 Mar 2019 10:40:34 -0400
In-Reply-To: <877ech5cvd.fsf_-_@HIDDEN>
 (T460s laptop's message of "Fri, 29 Mar 2019 22:12:38 -0400")
Message-ID: <87ftr32jkt.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="==-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 24450
Cc: 24450 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--==-=-=
Content-Type: multipart/mixed; boundary="=-=-="

--=-=-=
Content-Type: text/plain

Hello!

I've yet another commit to add on the top of the current stack of
patches already sent. This one makes it so that the source archive is
searched for the '[...].egg-info/requires.txt' file rather than check
the more common, fixed location.

This makes the importer more useful, as some packages such as
robotframework-sshlibrary use a "src" directory in their hierarchy which
would cause the previous scheme to not find requires.txt.


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0008-import-pypi-Scan-source-archive-to-find-requires.txt.patch
Content-Transfer-Encoding: quoted-printable

From=20cfde6e09f8f8c692fe252d76ed27e8c50a9e5377 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Sat, 30 Mar 2019 23:13:26 -0400
Subject: [PATCH] import: pypi: Scan source archive to find requires.txt fil=
e.

* guix/import/pypi.scm (use-modules): Use invoke from (guix build utils).
(guess-requirements)[archive-root-directory]: Remove procedure.
[guess-requirements-from-wheel]: Re-ident.
[guess-requirements-from-source]: Search for the requires.txt file in the
archive instead of using a static, expected location.
* tests/pypi.scm ("pypi->guix-package, no wheel"): Mock the requires.txt at=
 a
non-standard location to test the new feature.
("pypi->guix-package, no usable requirement file."): Adapt.
=2D--
 guix/import/pypi.scm | 65 ++++++++++++++++++--------------------------
 tests/pypi.scm       | 17 +++++++-----
 2 files changed, 37 insertions(+), 45 deletions(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 099768f0c8..a2ce14b192 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -36,7 +36,8 @@
   #:use-module ((guix build utils)
                 #:select ((package-name->name+version
                            . hyphen-package-name->name+version)
=2D                          find-files))
+                          find-files
+                          invoke))
   #:use-module (guix import utils)
   #:use-module ((guix download) #:prefix download:)
   #:use-module (guix import json)
@@ -267,19 +268,6 @@ omitted since these can be difficult or expensive to s=
atisfy."
 of the required packages specified in the requirements.txt file.  ARCHIVE =
will
 be extracted in a temporary directory."
=20
=2D  (define (archive-root-directory url)
=2D    ;; Given the URL of the package's archive, return the name of the di=
rectory
=2D    ;; that will be created upon decompressing it. If the filetype is not
=2D    ;; supported, return #f.
=2D    (if (compressed-file? url)
=2D        (let ((root-directory (file-sans-extension (basename url))))
=2D          (if (string=3D? "tar" (file-extension root-directory))
=2D              (file-sans-extension root-directory)
=2D              root-directory))
=2D        (begin
=2D          (warning (G_ "Unsupported archive format (~a): \
=2Dcannot determine package dependencies") (file-extension url))
=2D          #f)))
=20
   (define (read-wheel-metadata wheel-archive)
     ;; Given WHEEL-ARCHIVE, a ZIP Python wheel archive, return the package=
's
@@ -305,33 +293,34 @@ cannot determine package dependencies") (file-extensi=
on url))
     (call-with-temporary-output-file
      (lambda (temp port)
        (if wheel-url
=2D         (and (url-fetch wheel-url temp)
=2D              (read-wheel-metadata temp))
=2D         #f))))
+           (and (url-fetch wheel-url temp)
+                (read-wheel-metadata temp))
+           #f))))
=20
   (define (guess-requirements-from-source)
     ;; Return the package's requirements by guessing them from the source.
=2D    (let ((dirname (archive-root-directory source-url))
=2D          (extension (file-extension source-url)))
=2D      (if (string? dirname)
=2D          (call-with-temporary-directory
=2D           (lambda (dir)
=2D             (let* ((pypi-name (string-take dirname (string-rindex dirna=
me #\-)))
=2D                    (requires.txt (string-append dirname "/" pypi-name
=2D                                                 ".egg-info" "/requires.=
txt"))
=2D                    (exit-code
=2D                     (parameterize ((current-error-port (%make-void-port=
 "rw+"))
=2D                                    (current-output-port (%make-void-por=
t "rw+")))
=2D                       (if (string=3D? "zip" extension)
=2D                           (system* "unzip" archive "-d" dir requires.tx=
t)
=2D                           (system* "tar" "xf" archive "-C" dir requires=
.txt)))))
=2D               (if (zero? exit-code)
=2D                   (parse-requires.txt (string-append dir "/" requires.t=
xt))
=2D                   (begin
=2D                     (warning
=2D                      (G_ "Failed to extract file: ~a from source.~%")
=2D                      requires.txt)
=2D                     (list '() '()))))))
+    (if (compressed-file? source-url)
+        (call-with-temporary-directory
+         (lambda (dir)
+           (parameterize ((current-error-port (%make-void-port "rw+"))
+                          (current-output-port (%make-void-port "rw+")))
+             (if (string=3D? "zip" (file-extension source-url))
+                 (invoke "unzip" archive "-d" dir)
+                 (invoke "tar" "xf" archive "-C" dir)))
+           (let ((requires.txt-files
+                  (find-files dir (lambda (abs-file-name _)
+		                    (string-match "\\.egg-info/requires.txt$"
+                                                  abs-file-name)))))
+             (if (> (length requires.txt-files) 0)
+                 (begin
+                   (parse-requires.txt (first requires.txt-files)))
+                 (begin (warning (G_ "Cannot guess requirements from sourc=
e archive:\
+ no requires.txt file found.~%"))
+                        (list '() '()))))))
+        (begin
+          (warning (G_ "Unsupported archive format; \
+cannot determine package dependencies from source archive: ~a~%")
+                   (basename source-url))
           (list '() '()))))
=20
   ;; First, try to compute the requirements using the wheel, else, fallbac=
k to
diff --git a/tests/pypi.scm b/tests/pypi.scm
index aa08e2cb54..ad188df16c 100644
=2D-- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -177,8 +177,9 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testin=
g'
              (match url
                ("https://example.com/foo-1.0.0.tar.gz"
                 (begin
=2D                  (mkdir-p "foo-1.0.0/foo.egg-info/")
=2D                  (with-output-to-file "foo-1.0.0/foo.egg-info/requires.=
txt"
+                  ;; Unusual requires.txt location should still be found.
+                  (mkdir-p "foo-1.0.0/src/bizarre.egg-info")
+                  (with-output-to-file "foo-1.0.0/src/bizarre.egg-info/req=
uires.txt"
                     (lambda ()
                       (display test-requires.txt)))
                   (parameterize ((current-output-port (%make-void-port "rw=
+")))
@@ -299,10 +300,13 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'test=
ing'
          (lambda (url file-name)
            (match url
              ("https://example.com/foo-1.0.0.tar.gz"
+              (mkdir-p "foo-1.0.0/foo.egg-info/")
+              (parameterize ((current-output-port (%make-void-port "rw+")))
+                (system* "tar" "czvf" file-name "foo-1.0.0/"))
+              (delete-file-recursively "foo-1.0.0")
               (set! test-source-hash
=2D                (call-with-input-file file-name port-sha256))
=2D              #t)
=2D             ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #t)
+                (call-with-input-file file-name port-sha256)))
+             ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #f)
              (_ (error "Unexpected URL: " url)))))
         (mock ((guix http-client) http-fetch
                (lambda (url . rest)
@@ -334,7 +338,6 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testin=
g'
                             test-source-hash)
                            hash))
                 (x
=2D                 (pk 'fail x #f)))
=2D              )))
+                 (pk 'fail x #f))))))
=20
 (test-end "pypi")
=2D-=20
2.20.1


--=-=-=
Content-Type: text/plain


Thanks,

Maxim

--=-=-=--

--==-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEJ9WGpPiQCFQyn/CfEmDkZILmNWIFAlyg0WIACgkQEmDkZILm
NWIEMA/9HsMlrdzTwtllz/jPaAM13pPbyvI1sx1uwQi6AYessiZYBr/vGiFEBBTB
FJJUQ9Sz0XQOTW3wOitgTRz1kr55c17cv80IvwWYhTZLGZ+O2F29hFpp38OucRS7
d4XeZSJwBSn9ccYgytebd0Xtkp340HHeLOHnNTnA600TLwTq/6YXP4hLi5hiU7p1
zIOHe3Nze9dk1B30tuivqwvIVkSMnnPQyCmPSE+1vhzczVQ+m+f5PxxmSbP6Kznx
KF7Rlm+Ltjup0JfiKeglNiqLYYOKRpYuWM+oMzWSJgl1zA/MzDRZCnbs30TD/XKd
dkixB0Hmazer2BBRJoNSgCKu/Mx8hZg6ML/NcgrbmJnUd4Sw4r/M2My/Iq1D6RLA
Kkj7gzoqqowy+PcsU+/l+XrLdvT0wd6VR2x+DwY+y3Fiv8DSQRWI0mXxl9V4M4St
gF27p1RhfyUhIQo801AtJ+Tin/N5cYik8orRnARHnWNXVy7Iqz1RGr4MIqGheS0l
8GMOMVeLskAqagLynhWY4fz0teGCnaGAjVyzda7R1tYBkWzsioLugzsDoIftpEA8
p0TBiVEu0/BAWlL/BYdWyOIbkPQ05xV+Yad+QvECS3uu4/MckAcEwLMQpPjqKP8O
J5tRiRMRkEP1Dxl7oBuKC1IOHDhGvUOuIqFKqmyGRcBInwTC5Is=
=H7lm
-----END PGP SIGNATURE-----
--==-=-=--




Information forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.

Message received at 24450 <at> debbugs.gnu.org:


Received: (at 24450) by debbugs.gnu.org; 30 Mar 2019 07:57:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 30 03:57:19 2019
Received: from localhost ([127.0.0.1]:36219 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hA8rf-00082j-Ev
	for submit <at> debbugs.gnu.org; Sat, 30 Mar 2019 03:57:19 -0400
Received: from mx1.riseup.net ([198.252.153.129]:39846)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <swedebugia@HIDDEN>)
 id 1hA8rc-00082X-VN; Sat, 30 Mar 2019 03:57:17 -0400
Received: from capuchin.riseup.net (capuchin-pn.riseup.net [10.0.1.176])
 (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
 (Client CN "*.riseup.net",
 Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK))
 by mx1.riseup.net (Postfix) with ESMTPS id 9D6DF1A04CC;
 Sat, 30 Mar 2019 00:57:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak;
 t=1553932635; bh=+c+whlUv6xmEB/A02sIIyLCNBnGlDBqd7p+pMl2SEMU=;
 h=Subject:To:Cc:References:From:Date:In-Reply-To:From;
 b=UjPMNuNooQDLMyOMo8czC4erYlKvDRSc+J6EuXGEtpEBPNUfDDKkyqXxrENv+Si4L
 /1dZ0vPnoo9JvBTKevJIIhC+5d2pSxjKl8mUjtp13CkhIFhN3jmbtzuyss9C1eI9k3
 AIsuusR2jh/0JG+4BC8boqJPjLtiKbc/6teP/K9w=
X-Riseup-User-ID: C37BD159BBAC4720BE3C07E4F3E98F87960E1CCC9A7021F4FF2DE41F17106A67
Received: from [127.0.0.1] (localhost [127.0.0.1])
 by capuchin.riseup.net (Postfix) with ESMTPSA id 47427120527;
 Sat, 30 Mar 2019 00:57:13 -0700 (PDT)
Subject: Re: bug#34266: pypi importer cannot handle [ and ] correctly
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
References: <b67ec137-b642-2e1c-7648-702ae7a2cf4f@HIDDEN>
 <87va022tz3.fsf@HIDDEN>
From: swedebugia <swedebugia@HIDDEN>
Message-ID: <991a432d-e5b8-88cd-ee47-bb60af1a8dbe@HIDDEN>
Date: Sat, 30 Mar 2019 08:59:12 +0100
MIME-Version: 1.0
In-Reply-To: <87va022tz3.fsf@HIDDEN>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="D6uQrkeYOdfpFinkcF0h2Gd5DPiR87m3v"
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 24450
Cc: 34266 <at> debbugs.gnu.org, 24450 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--D6uQrkeYOdfpFinkcF0h2Gd5DPiR87m3v
Content-Type: multipart/mixed; boundary="jl6FJnS6n97xldJs8GtZBPzjbQChLA20K";
 protected-headers="v1"
From: swedebugia <swedebugia@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Cc: 34266 <at> debbugs.gnu.org, 24450 <at> debbugs.gnu.org
Message-ID: <991a432d-e5b8-88cd-ee47-bb60af1a8dbe@HIDDEN>
Subject: Re: bug#34266: pypi importer cannot handle [ and ] correctly
References: <b67ec137-b642-2e1c-7648-702ae7a2cf4f@HIDDEN>
 <87va022tz3.fsf@HIDDEN>
In-Reply-To: <87va022tz3.fsf@HIDDEN>

--jl6FJnS6n97xldJs8GtZBPzjbQChLA20K
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable

On 2019-03-29 05:19, Maxim Cournoyer wrote:
> swedebugia <swedebugia@HIDDEN> writes:
>=20
>> $ ./pre-inst-env guix import pypi beaker
>>
>> following redirection to `https://pypi.org/pypi/Beaker/json'...
>>
>> Starting download of /tmp/guix-file.p15GJZ
>> From
>> https://files.pythonhosted.org/packages/c2/21/b052b2fbfee3def06670923d=
5d34b0d353d4c278013e4a714c3fb663f150/Beaker-1.10.0.tar.gz...
>>  ...0.0.tar.gz  40KiB                 521KiB/s 00:00
>> [##################] 100.0%
>> (package
>>   (name "python-beaker")
>>   (version "1.10.0")
>>   (source
>>     (origin
>>       (method url-fetch)
>>       (uri (pypi-uri "beaker" version))
>>       (sha256
>>         (base32
>>           "0l047yl3n9b3w7ba0wrqdb5fpww5y8pjy20kah2mlpr230lqjwk0"))))
>>   (build-system python-build-system)
>>   (propagated-inputs
>>     `(("python-[crypto]" ,#{python-\x5b;crypto\x5d;}#)
>>       ("python-[cryptography]"
>>        ,#{python-\x5b;cryptography\x5d;}#)
>>       ("python-[pycrypto]"
>>        ,#{python-\x5b;pycrypto\x5d;}#)
>>       ("python-[pycryptodome]"
>>        ,#{python-\x5b;pycryptodome\x5d;}#)
>>       ("python-[testsuite]"
>>        ,#{python-\x5b;testsuite\x5d;}#)
>>       ("python-coverage" ,python-coverage)
>>       ("python-cryptography" ,python-cryptography)
>>       ("python-cryptography" ,python-cryptography)
>>       ("python-funcsigs" ,python-funcsigs)
>>       ("python-memcached" ,python-memcached)
>>       ("python-mock" ,python-mock)
>>       ("python-nose" ,python-nose)
>>       ("python-pycrypto" ,python-pycrypto)
>>       ("python-pycryptodome" ,python-pycryptodome)
>>       ("python-pycryptodome" ,python-pycryptodome)
>>       ("python-pycryptopp" ,python-pycryptopp)
>>       ("python-pylibmc" ,python-pylibmc)
>>       ("python-pymongo" ,python-pymongo)
>>       ("python-redis" ,python-redis)
>>       ("python-sqlalchemy" ,python-sqlalchemy)
>>       ("python-webtest" ,python-webtest)))
>>   (home-page "https://beaker.readthedocs.io/")
>>   (synopsis
>>     "A Session and Caching library with WSGI Middleware")
>>   (description
>>     "A Session and Caching library with WSGI Middleware")
>>   (license license:bsd-3))
>=20
> Testing with my soon-to-be sent for review changes:
>=20
> --8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix import pypi beaker
> following redirection to `https://pypi.org/pypi/Beaker/json'...
>=20
> Starting download of /tmp/guix-file.0MWu4B
> From https://files.pythonhosted.org/packages/76/87/ecc1a222f0caaa7ba7b8=
928737e89b2e91b8c22450c12b8a51ee625a4d87/Beaker-1.10.1.tar.gz...
>  =E2=80=A60.1.tar.gz  40KiB                   487KiB/s 00:00 [#########=
#########] 100.0%
> (package
>   (name "python-beaker")
>   (version "1.10.1")
>   (source
>     (origin
>       (method url-fetch)
>       (uri (pypi-uri "beaker" version))
>       (sha256
>         (base32
>           "16zdjfl8v73yl1capph0n371vd26c7zpzb48n505ip32ffgmvc4f"))))
>   (build-system python-build-system)
>   (native-inputs
>     `(("python-coverage" ,python-coverage)
>       ("python-cryptography" ,python-cryptography)
>       ("python-memcached" ,python-memcached)
>       ("python-mock" ,python-mock)
>       ("python-nose" ,python-nose)
>       ("python-pycryptodome" ,python-pycryptodome)
>       ("python-pylibmc" ,python-pylibmc)
>       ("python-pymongo" ,python-pymongo)
>       ("python-redis" ,python-redis)
>       ("python-sqlalchemy" ,python-sqlalchemy)
>       ("python-webtest" ,python-webtest)))
>   (home-page "https://beaker.readthedocs.io/")
>   (synopsis
>     "A Session and Caching library with WSGI Middleware")
>   (description
>     "A Session and Caching library with WSGI Middleware")
>   (license license:bsd-3))
> --8<---------------cut here---------------end--------------->8---
>=20
> Looking better?

Perfect!

--=20
Cheers Swedebugia


--jl6FJnS6n97xldJs8GtZBPzjbQChLA20K--

--D6uQrkeYOdfpFinkcF0h2Gd5DPiR87m3v
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----

iHUEARYIAB0WIQR6IWNlOiLn9hD6a1XPzUNbKAts0gUCXJ8h0QAKCRDPzUNbKAts
0tuQAP9R2vpUsnHIQ6Pk0La+OsLBsQrgs9d0cC4P2OQqbJx27QD+MrOAJdhoebrd
7oBT8HcsUREpWzV60AgRQIXVFwVZWAk=
=NPhy
-----END PGP SIGNATURE-----

--D6uQrkeYOdfpFinkcF0h2Gd5DPiR87m3v--




Information forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.
Forcibly Merged 24450 24557 33047 33569 34266. Request was from T460s laptop <maxim.cournoyer@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 24450 <at> debbugs.gnu.org:


Received: (at 24450) by debbugs.gnu.org; 30 Mar 2019 02:12:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 29 22:12:51 2019
Received: from localhost ([127.0.0.1]:36098 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hA3UI-0001p6-2Q
	for submit <at> debbugs.gnu.org; Fri, 29 Mar 2019 22:12:50 -0400
Received: from mail-it1-f170.google.com ([209.85.166.170]:38884)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1hA3UF-0001ot-8U
 for 24450 <at> debbugs.gnu.org; Fri, 29 Mar 2019 22:12:48 -0400
Received: by mail-it1-f170.google.com with SMTP id f22so6909223ita.3
 for <24450 <at> debbugs.gnu.org>; Fri, 29 Mar 2019 19:12:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=EgbkHK8tXP2pXrN2Xadripcz517aU8UocDjP1CF46e0=;
 b=TAxh8ud45GfYrUIRyIkuBIlk7QXlEP7AGicuzjnimK0d50uuiKBhHdVmvNTS8YiIc1
 UIgyBRIrjAtpNlsAO9aP1ggOM5Ymai8OVXbYfnfkV1ljTi43DrXMVKt1GeRElBgrlFBM
 MqARB04X7IyzDo+NcPglHdXDEw2yEp2tdSK3B/QtW0eba3OrwNpjgiyFjY4lV9bldZyY
 kwPnsT23NB+anT3A1AnIP674mUFp8yNxSJt2a2zedlIBZS/i6lk4l+jgHeSYdPM7it1+
 wG78hLAWl4sos1NzyefZFJKkxiodjqK6/qSV0Z4ecaQFUKbiq2i2ZOPjJ7nQlk1o4JsM
 s0ig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=EgbkHK8tXP2pXrN2Xadripcz517aU8UocDjP1CF46e0=;
 b=D6r2k2QhSacML7DGrG65A62nx7Zcjd15DgqiSXafviHrL32aXVy67MpxozFaqlK8nE
 RQ+YDnET4zGIUnWUVt9XNxM/T69ONxmy+rCcu+7MCjuwvHY1cJdqJflKYWsRfIZvW+FA
 0aU0qGAs3OnmheHswLc2xsae3ZsqFRwci5TBcERqlhOji1i2pvO4+B1lBJaqRYuHlg8N
 ecRHMDjNwzPflLKT/Pn2Bx2iJ9hTVYqMC7GF2rFtmZhjmndfJzjICRTAXZpBLBDmUBZL
 MtuJ3E3WljE4gIK0sl3Xcxl6wbpENOWmaNXiu35O8VEQqer4aFOxhVVtYbBCRlBu+nok
 BiOg==
X-Gm-Message-State: APjAAAWmHtbjGhjRcMuTpkRwT0efyvIlJ+dsJpB9ZFUoGX/TwzH8JYJG
 YGbiKzcFQnGNZfI79psMy+/kFMe7Ewc=
X-Google-Smtp-Source: APXvYqyOPSgw/BDGn4hNSHnAfSraKno3Ok9bdqvqQiKH01HmFtWNSLUDn+KGzQtWaL7eXJc9rZOkDw==
X-Received: by 2002:a24:c106:: with SMTP id e6mr7056521itg.21.1553911961563;
 Fri, 29 Mar 2019 19:12:41 -0700 (PDT)
Received: from kwak ([2607:f2c0:94b4:fa00::235])
 by smtp.gmail.com with ESMTPSA id e10sm1383385iok.85.2019.03.29.19.12.39
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Fri, 29 Mar 2019 19:12:40 -0700 (PDT)
From: T460s laptop <maxim.cournoyer@HIDDEN>
X-Google-Original-From: T460s laptop
 <mcournoyer@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: [PATCHv2] bug#24450: pypi importer outputs strange character series
 in optional dependency case.
References: <87h99fipj1.fsf@HIDDEN> <87tvfm1eos.fsf@HIDDEN>
Date: Fri, 29 Mar 2019 22:12:38 -0400
In-Reply-To: <87tvfm1eos.fsf@HIDDEN> (Maxim Cournoyer's message of "Fri, 29
 Mar 2019 00:34:43 -0400")
Message-ID: <877ech5cvd.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="==-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 24450
Cc: ng0 <ng0@HIDDEN>, 24450 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--==-=-=
Content-Type: multipart/mixed; boundary="=-=-="

--=-=-=
Content-Type: text/plain


The previous 0007 patch had broken the recursive importer.  This
reworked version fixes this.  The rest of the patches stack sent in the
previous message is still good as is.


--=-=-=
Content-Type: text/x-patch; charset=utf-8
Content-Disposition: attachment;
 filename=0007-import-pypi-Include-optional-test-inputs-as-native-i.patch
Content-Transfer-Encoding: quoted-printable

From=2037e499d5d5d5f690aa0a065c730e13f6a31dd30d Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Thu, 28 Mar 2019 23:12:26 -0400
Subject: [PATCH] import: pypi: Include optional test inputs as native-input=
s.

* guix/import/pypi.scm (maybe-inputs): Add INPUT-TYPE argument, and use it.
(test-section?): New predicate.
(parse-requires.txt): Collect the optional test inputs, and return them as =
the
second element of the returned list.
(parse-wheel-metadata): Likewise.
(guess-requirements): Adapt, and hide unzip output.
(make-pypi-sexp): Likewise, and include the test inputs requirements as nat=
ive
inputs in the returned package expression.

* tests/pypi.scm (test-requires.txt): Include a test section in the
test-requires.txt data.
(test-requires.txt-beaker): New variable.
("parse-requires.txt"): Adapt.
("parse-requires.txt - Beaker"): New test.
("parse-wheel-metadata, with extras"): Adapt.
("parse-wheel-metadata, with extras - Jedi"): Adapt.
("pypi->guix-package, no wheel"): Re-indent, and add the expected
native-inputs.
("pypi->guix-package, wheels"): Likewise.
("pypi->guix-package, no usable requirement file."): New test.
=2D--
 guix/import/pypi.scm | 195 ++++++++++++++++++++++++++++---------------
 tests/pypi.scm       | 123 ++++++++++++++++++++-------
 2 files changed, 222 insertions(+), 96 deletions(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index c520213b6a..099768f0c8 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -4,6 +4,7 @@
 ;;; Copyright =C2=A9 2015, 2016, 2017 Ludovic Court=C3=A8s <ludo@HIDDEN>
 ;;; Copyright =C2=A9 2017 Mathieu Othacehe <m.othacehe@HIDDEN>
 ;;; Copyright =C2=A9 2018 Ricardo Wurmus <rekado@HIDDEN>
+;;; Copyright =C2=A9 2019 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -26,6 +27,7 @@
   #:use-module (ice-9 receive)
   #:use-module ((ice-9 rdelim) #:select (read-line))
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
@@ -106,14 +108,15 @@ package on PyPI."
     ((name version _ ...)
      (string-append name "-" version ".dist-info"))))
=20
=2D(define (maybe-inputs package-inputs)
+(define (maybe-inputs package-inputs input-type)
   "Given a list of PACKAGE-INPUTS, tries to generate the 'inputs' field of=
 a
=2Dpackage definition."
+package definition.  INPUT-TYPE, a symbol, is used to populate the name of
+the input field."
   (match package-inputs
     (()
      '())
     ((package-inputs ...)
=2D     `((propagated-inputs (,'quasiquote ,package-inputs))))))
+     `((,input-type (,'quasiquote ,package-inputs))))))
=20
 (define %requirement-name-regexp
   ;; Regexp to match the requirement name in a requirement specification.
@@ -147,11 +150,21 @@ package definition."
    (or (regexp-exec %requirement-name-regexp spec)
        (error (G_ "Could not extract requirement name in spec:") spec))))
=20
+(define (test-section? name)
+  "Return #t if the section name contains 'test' or 'dev'."
+  (any (cut string-contains-ci name <>)
+       '("test" "dev")))
+
 (define (parse-requires.txt requires.txt)
=2D  "Given REQUIRES.TXT, a Setuptools requires.txt file, return a list of
=2Drequirement names."
=2D  ;; This is a very incomplete parser, which job is to select the non-op=
tional
=2D  ;; dependencies and strip them out of any version information.
+  "Given REQUIRES.TXT, a Setuptools requires.txt file, return a pair of re=
quirements.
+
+The first element of the pair contains the required dependencies while the
+second the optional test dependencies.  Note that currently, optional,
+non-test dependencies are omitted since these can be difficult or expensiv=
e to
+satisfy."
+
+  ;; This is a very incomplete parser, which job is to read in the require=
ment
+  ;; specification lines, and strip them out of any version information.
   ;; Alternatively, we could implement a PEG parser with the (ice-9 peg)
   ;; library and the requirements grammar defined by PEP-0508
   ;; (https://www.python.org/dev/peps/pep-0508/).
@@ -168,57 +181,89 @@ requirement names."
=20
   (call-with-input-file requires.txt
     (lambda (port)
=2D      (let loop ((result '()))
+      (let loop ((required-deps '())
+                 (test-deps '())
+                 (inside-test-section? #f)
+                 (optional? #f))
         (let ((line (read-line port)))
=2D          ;; Stop when a section is encountered, as sections contains op=
tional
=2D          ;; (extra) requirements.  Non-optional requirements must appear
=2D          ;; before any section is defined.
=2D          (if (or (eof-object? line) (section-header? line))
+          (if (eof-object? line)
               ;; Duplicates can occur, since the same requirement can be
               ;; listed multiple times with different conditional markers,=
 e.g.
               ;; pytest >=3D 3 ; python_version >=3D "3.3"
               ;; pytest < 3 ; python_version < "3.3"
=2D              (reverse (delete-duplicates result))
+              (map (compose reverse delete-duplicates)
+                   (list required-deps test-deps))
               (cond
                ((or (string-null? line) (comment? line))
=2D                (loop result))
=2D               (else
+                (loop required-deps test-deps inside-test-section? optiona=
l?))
+               ((section-header? line)
+                ;; Encountering a section means that all the requirements
+                ;; listed below are optional. Since we want to pick only t=
he
+                ;; test dependencies from the optional dependencies, we mu=
st
+                ;; track those separately.
+                (loop required-deps test-deps (test-section? line) #t))
+               (inside-test-section?
+                (loop required-deps
+                      (cons (specification->requirement-name line)
+                            test-deps)
+                      inside-test-section? optional?))
+               ((not optional?)
                 (loop (cons (specification->requirement-name line)
=2D                            result))))))))))
+                            required-deps)
+                      test-deps inside-test-section? optional?))
+               (optional?
+                ;; Skip optional items.
+                (loop required-deps test-deps inside-test-section? optiona=
l?))
+               (else
+                (warning (G_ "parse-requires.txt reached an unexpected \
+condition on line ~a~%") line)))))))))
=20
 (define (parse-wheel-metadata metadata)
=2D  "Given METADATA, a Wheel metadata file, return a list of requirement n=
ames."
+  "Given METADATA, a Wheel metadata file, return a pair of requirements.
+
+The first element of the pair contains the required dependencies while the=
 second the optional
+test dependencies.  Note that currently, optional, non-test dependencies a=
re
+omitted since these can be difficult or expensive to satisfy."
   ;; METADATA is a RFC-2822-like, header based file.
=20
   (define (requires-dist-header? line)
     ;; Return #t if the given LINE is a Requires-Dist header.
=2D    (regexp-match? (string-match "^Requires-Dist: " line)))
+    (string-match "^Requires-Dist: " line))
=20
   (define (requires-dist-value line)
     (string-drop line (string-length "Requires-Dist: ")))
=20
   (define (extra? line)
     ;; Return #t if the given LINE is an "extra" requirement.
=2D    (regexp-match? (string-match "extra =3D=3D " line)))
+    (string-match "extra =3D=3D '(.*)'" line))
+
+  (define (test-requirement? line)
+    (let ((extra-label (match:substring (extra? line) 1)))
+      (and extra-label (test-section? extra-label))))
=20
   (call-with-input-file metadata
     (lambda (port)
=2D      (let loop ((requirements '()))
+      (let loop ((required-deps '())
+                 (test-deps '()))
         (let ((line (read-line port)))
=2D          ;; Stop at the first 'Provides-Extra' section: the non-optional
=2D          ;; requirements appear before the optional ones.
           (if (eof-object? line)
=2D              (reverse (delete-duplicates requirements))
+              (map (compose reverse delete-duplicates)
+                   (list required-deps test-deps))
               (cond
                ((and (requires-dist-header? line) (not (extra? line)))
                 (loop (cons (specification->requirement-name
                              (requires-dist-value line))
=2D                            requirements)))
+                            required-deps)
+                      test-deps))
+               ((and (requires-dist-header? line) (test-requirement? line))
+                (loop required-deps
+                      (cons (specification->requirement-name (requires-dis=
t-value line))
+                            test-deps)))
                (else
=2D                (loop requirements)))))))))
+                (loop required-deps test-deps))))))))) ;skip line
=20
 (define (guess-requirements source-url wheel-url archive)
=2D  "Given SOURCE-URL, WHEEL-URL and a ARCHIVE of the package, return a li=
st
+  "Given SOURCE-URL, WHEEL-URL and an ARCHIVE of the package, return a list
 of the required packages specified in the requirements.txt file.  ARCHIVE =
will
 be extracted in a temporary directory."
=20
@@ -244,7 +289,10 @@ cannot determine package dependencies") (file-extensio=
n url))
            (metadata (string-append dirname "/METADATA")))
       (call-with-temporary-directory
        (lambda (dir)
=2D         (if (zero? (system* "unzip" "-q" wheel-archive "-d" dir metadat=
a))
+         (if (zero?
+              (parameterize ((current-error-port (%make-void-port "rw+"))
+                             (current-output-port (%make-void-port "rw+")))
+                (system* "unzip" wheel-archive "-d" dir metadata)))
              (parse-wheel-metadata (string-append dir "/" metadata))
              (begin
                (warning
@@ -283,32 +331,41 @@ cannot determine package dependencies") (file-extensi=
on url))
                      (warning
                       (G_ "Failed to extract file: ~a from source.~%")
                       requires.txt)
=2D                     '())))))
=2D          '())))
+                     (list '() '()))))))
+          (list '() '()))))
=20
   ;; First, try to compute the requirements using the wheel, else, fallbac=
k to
   ;; reading the "requires.txt" from the egg-info directory from the source
=2D  ;; tarball.
+  ;; archive.
   (or (guess-requirements-from-wheel)
       (guess-requirements-from-source)))
=20
 (define (compute-inputs source-url wheel-url archive)
=2D  "Given the SOURCE-URL of an already downloaded ARCHIVE, return a list =
of
=2Dname/variable pairs describing the required inputs of this package.  Also
+  "Given the SOURCE-URL and WHEEL-URL of an already downloaded ARCHIVE, re=
turn
+a pair of lists, each consisting of a list of name/variable pairs, for the
+propagated inputs and the native inputs, respectively.  Also
 return the unaltered list of upstream dependency names."
=2D  (let ((dependencies
=2D         (remove (cut string=3D? "argparse" <>)
=2D                 (guess-requirements source-url wheel-url archive))))
=2D    (values (sort
=2D             (map (lambda (input)
=2D                    (let ((guix-name (python->package-name input)))
=2D                      (list guix-name (list 'unquote (string->symbol gui=
x-name)))))
=2D                  dependencies)
=2D             (lambda args
=2D               (match args
=2D                 (((a _ ...) (b _ ...))
=2D                  (string-ci<? a b)))))
=2D            dependencies)))
+
+  (define (strip-argparse deps)
+    (remove (cut string=3D? "argparse" <>) deps))
+
+  (define (requirement->package-name/sort deps)
+    (sort
+     (map (lambda (input)
+            (let ((guix-name (python->package-name input)))
+              (list guix-name (list 'unquote (string->symbol guix-name)))))
+          deps)
+     (lambda args
+       (match args
+         (((a _ ...) (b _ ...))
+          (string-ci<? a b))))))
+
+  (define process-requirements
+    (compose requirement->package-name/sort strip-argparse))
+
+  (let ((dependencies (guess-requirements source-url wheel-url archive)))
+    (values (map process-requirements dependencies)
+            (concatenate dependencies))))
=20
 (define (make-pypi-sexp name version source-url wheel-url home-page synops=
is
                         description license)
@@ -317,29 +374,33 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION=
, and LICENSE."
   (call-with-temporary-output-file
    (lambda (temp port)
      (and (url-fetch source-url temp)
=2D          (receive (input-package-names upstream-dependency-names)
+          (receive (guix-dependencies upstream-dependencies)
               (compute-inputs source-url wheel-url temp)
=2D            (values
=2D             `(package
=2D                (name ,(python->package-name name))
=2D                (version ,version)
=2D                (source (origin
=2D                          (method url-fetch)
=2D
=2D                          ;; Sometimes 'pypi-uri' doesn't quite work due=
 to mixed
=2D                          ;; cases in NAME, for instance, as is the case=
 with
=2D                          ;; "uwsgi".  In that case, fall back to a full=
 URL.
=2D                          (uri (pypi-uri ,(string-downcase name) version=
))
=2D                          (sha256
=2D                           (base32
=2D                            ,(guix-hash-url temp)))))
=2D                (build-system python-build-system)
=2D                ,@(maybe-inputs input-package-names)
=2D                (home-page ,home-page)
=2D                (synopsis ,synopsis)
=2D                (description ,description)
=2D                (license ,(license->symbol license)))
=2D             upstream-dependency-names))))))
+            (match guix-dependencies
+              ((required-inputs test-inputs)
+               (values
+                `(package
+                   (name ,(python->package-name name))
+                   (version ,version)
+                   (source (origin
+                             (method url-fetch)
+                             ;; Sometimes 'pypi-uri' doesn't quite work du=
e to mixed
+                             ;; cases in NAME, for instance, as is the cas=
e with
+                             ;; "uwsgi".  In that case, fall back to a ful=
l URL.
+                             (uri (pypi-uri ,(string-downcase name) versio=
n))
+                             (sha256
+                              (base32
+                               ,(guix-hash-url temp)))))
+                   (build-system python-build-system)
+                   ,@(maybe-inputs required-inputs 'propagated-inputs)
+                   ,@(maybe-inputs test-inputs 'native-inputs)
+                   (home-page ,home-page)
+                   (synopsis ,synopsis)
+                   (description ,description)
+                   (license ,(license->symbol license)))
+                ;; Flatten the nested lists and return the upstream
+                ;; dependencies.
+                upstream-dependencies))))))))
=20
 (define pypi->guix-package
   (memoize
diff --git a/tests/pypi.scm b/tests/pypi.scm
index ca8cb5f6de..aa08e2cb54 100644
=2D-- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright =C2=A9 2014 David Thompson <davet@HIDDEN>
 ;;; Copyright =C2=A9 2016 Ricardo Wurmus <rekado@HIDDEN>
+;;; Copyright =C2=A9 2019 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -65,11 +66,6 @@
 sha1=3Dda9234ee9982d4bbb3c72346a6de940a148ea686"))
=20
 (define test-requires.txt "\
=2Dbar
=2Dbaz > 13.37
=2D")
=2D
=2D(define test-requires-with-sections "\
 # A comment
 foo ~=3D 3
 bar !=3D 2
@@ -78,12 +74,25 @@ bar !=3D 2
 pytest (>=3D2.5.0)
 ")
=20
+;; Beaker contains only optional dependencies.
+(define test-requires.txt-beaker "\
+[crypto]
+pycryptopp>=3D0.5.12
+
+[cryptography]
+cryptography
+
+[testsuite]
+Mock
+coverage
+")
+
 (define test-metadata "\
 Classifier: Programming Language :: Python :: 3.7
 Requires-Dist: baz ~=3D 3
 Requires-Dist: bar !=3D 2
 Provides-Extra: test
=2Dpytest (>=3D2.5.0)
+Requires-Dist: pytest (>=3D2.5.0) ; extra =3D=3D 'test'
 ")
=20
 (define test-metadata-with-extras "
@@ -137,25 +146,31 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'test=
ing'
   '("Fizzy" "PickyThing" "SomethingWithMarker" "requests" "pip")
   (map specification->requirement-name test-specifications))
=20
=2D(test-equal "parse-requires.txt, with sections"
=2D  '("foo" "bar")
+(test-equal "parse-requires.txt"
+  (list '("foo" "bar") '("pytest"))
   (mock ((ice-9 ports) call-with-input-file
          call-with-input-string)
=2D        (parse-requires.txt test-requires-with-sections)))
+        (parse-requires.txt test-requires.txt)))
+
+(test-equal "parse-requires.txt - Beaker"
+  (list '() '("Mock" "coverage"))
+  (mock ((ice-9 ports) call-with-input-file
+         call-with-input-string)
+        (parse-requires.txt test-requires.txt-beaker)))
=20
 (test-equal "parse-wheel-metadata, with extras"
=2D  '("wrapt" "bar")
+  (list '("wrapt" "bar") '("tox" "bumpversion"))
   (mock ((ice-9 ports) call-with-input-file
          call-with-input-string)
         (parse-wheel-metadata test-metadata-with-extras)))
=20
 (test-equal "parse-wheel-metadata, with extras - Jedi"
=2D  '("parso")
+  (list '("parso") '("pytest"))
   (mock ((ice-9 ports) call-with-input-file
          call-with-input-string)
         (parse-wheel-metadata test-metadata-with-extras-jedi)))
=20
=2D(test-assert "pypi->guix-package"
+(test-assert "pypi->guix-package, no wheel"
   ;; Replace network resources with sample data.
     (mock ((guix import utils) url-fetch
            (lambda (url file-name)
@@ -195,7 +210,10 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testi=
ng'
                      ('propagated-inputs
                       ('quasiquote
                        (("python-bar" ('unquote 'python-bar))
=2D                        ("python-baz" ('unquote 'python-baz)))))
+                        ("python-foo" ('unquote 'python-foo)))))
+                     ('native-inputs
+                      ('quasiquote
+                       (("python-pytest" ('unquote 'python-pytest)))))
                      ('home-page "http://example.com")
                      ('synopsis "summary")
                      ('description "summary")
@@ -216,25 +234,25 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'test=
ing'
               (begin
                 (mkdir-p "foo-1.0.0/foo.egg-info/")
                 (with-output-to-file "foo-1.0.0/foo.egg-info/requires.txt"
=2D                   (lambda ()
=2D                     (display "wrong data to make sure we're testing whe=
els ")))
+                  (lambda ()
+                    (display "wrong data to make sure we're testing wheels=
 ")))
                 (parameterize ((current-output-port (%make-void-port "rw+"=
)))
                   (system* "tar" "czvf" file-name "foo-1.0.0/"))
=2D                 (delete-file-recursively "foo-1.0.0")
=2D                 (set! test-source-hash
=2D                       (call-with-input-file file-name port-sha256))))
+                (delete-file-recursively "foo-1.0.0")
+                (set! test-source-hash
+                  (call-with-input-file file-name port-sha256))))
              ("https://example.com/foo-1.0.0-py2.py3-none-any.whl"
=2D               (begin
=2D                 (mkdir "foo-1.0.0.dist-info")
=2D                 (with-output-to-file "foo-1.0.0.dist-info/METADATA"
=2D                   (lambda ()
=2D                     (display test-metadata)))
=2D                 (let ((zip-file (string-append file-name ".zip")))
=2D                   ;; zip always adds a "zip" extension to the file it c=
reates,
=2D                   ;; so we need to rename it.
=2D                   (system* "zip" zip-file "foo-1.0.0.dist-info/METADATA=
")
=2D                   (rename-file zip-file file-name))
=2D                 (delete-file-recursively "foo-1.0.0.dist-info")))
+              (begin
+                (mkdir "foo-1.0.0.dist-info")
+                (with-output-to-file "foo-1.0.0.dist-info/METADATA"
+                  (lambda ()
+                    (display test-metadata)))
+                (let ((zip-file (string-append file-name ".zip")))
+                  ;; zip always adds a "zip" extension to the file it crea=
tes,
+                  ;; so we need to rename it.
+                  (system* "zip" "-q" zip-file "foo-1.0.0.dist-info/METADA=
TA")
+                  (rename-file zip-file file-name))
+                (delete-file-recursively "foo-1.0.0.dist-info")))
              (_ (error "Unexpected URL: " url)))))
         (mock ((guix http-client) http-fetch
                (lambda (url . rest)
@@ -262,6 +280,9 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testin=
g'
                     ('quasiquote
                      (("python-bar" ('unquote 'python-bar))
                       ("python-baz" ('unquote 'python-baz)))))
+                   ('native-inputs
+                    ('quasiquote
+                     (("python-pytest" ('unquote 'python-pytest)))))
                    ('home-page "http://example.com")
                    ('synopsis "summary")
                    ('description "summary")
@@ -272,4 +293,48 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testi=
ng'
                 (x
                  (pk 'fail x #f))))))
=20
+(test-assert "pypi->guix-package, no usable requirement file."
+  ;; Replace network resources with sample data.
+  (mock ((guix import utils) url-fetch
+         (lambda (url file-name)
+           (match url
+             ("https://example.com/foo-1.0.0.tar.gz"
+              (set! test-source-hash
+                (call-with-input-file file-name port-sha256))
+              #t)
+             ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #t)
+             (_ (error "Unexpected URL: " url)))))
+        (mock ((guix http-client) http-fetch
+               (lambda (url . rest)
+                 (match url
+                   ("https://pypi.org/pypi/foo/json"
+                    (values (open-input-string test-json)
+                            (string-length test-json)))
+                   ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #=
f)
+                   (_ (error "Unexpected URL: " url)))))
+              ;; Not clearing the memoization cache here would mean return=
ing the value
+              ;; computed in the previous test.
+              (invalidate-memoization! pypi->guix-package)
+              (match (pypi->guix-package "foo")
+                (('package
+                   ('name "python-foo")
+                   ('version "1.0.0")
+                   ('source ('origin
+                              ('method 'url-fetch)
+                              ('uri ('pypi-uri "foo" 'version))
+                              ('sha256
+                               ('base32
+                                (? string? hash)))))
+                   ('build-system 'python-build-system)
+                   ('home-page "http://example.com")
+                   ('synopsis "summary")
+                   ('description "summary")
+                   ('license 'license:lgpl2.0))
+                 (string=3D? (bytevector->nix-base32-string
+                            test-source-hash)
+                           hash))
+                (x
+                 (pk 'fail x #f)))
+              )))
+
 (test-end "pypi")
=2D-=20
2.20.1


--=-=-=--

--==-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEJ9WGpPiQCFQyn/CfEmDkZILmNWIFAlye0JYACgkQEmDkZILm
NWJGEg/9HElFyj+z1M2Z6ffboYKW3WJ0D7q+vV0fvkacd3X/8XkC6DPEjI8veFYf
3qiygovcK6tgI5sAVzRWZ4WJ7oW284YoUURGjfax1ilvJl3kGkzqNy45nD3CQo3A
JK/rfW432Pu2UgC/Ewu+4V1NZqU5gCoHnz/rVNt0gRwWANBCLQG9fia/0u7zq8WK
E0UIlKJZeOi6/1xr+QYcSYtMV9Nr3mb7OOC3TleHrz+lrT8YbmrsqdxXfZuJw3iA
FY4jMX6SGAlfhKc7/+2F2z+iwkev7tTg1AAVhLWyqeR+zqYwBOCj5TOEAFMBcO8e
FMUr6GONxdJfHwSapnzzEq6HcTqhx+MutMCiynap3hBVxONDA2uXmGTM/pe0zrZm
Dqdr4XgFgaSJcaAtc3Ar/v6jwRy3muNKHBl6opQjd7zbxIju9hgzsiXvTbC/RUgg
9Oc7Yd2PT2kCBvJ1GgCk996EHcY1ACOeHHI0Yg0XHO6JCx1Dqi9i0vImGqcoOg/o
OmA0cxRntaCgiCSyGaSyBzcyHjsfieKwZlRQk5DaeMfguzGHGFS9b6OHuJ7Ofdb7
QESgYGuD5YlQhAsNc64A0GTm6+JDh9v84MwUqyR0EcU3toPO3r2zAWeGoi8k3FDZ
IaPuNC3qZ9XY2Pp7kHsHcTCRs+lXcoIDxZElkOgNMWCvW5wKpr4=
=2WjY
-----END PGP SIGNATURE-----
--==-=-=--




Information forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.

Message received at 24450 <at> debbugs.gnu.org:


Received: (at 24450) by debbugs.gnu.org; 29 Mar 2019 04:35:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 29 00:35:01 2019
Received: from localhost ([127.0.0.1]:34964 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1h9jEF-0006JX-Ow
	for submit <at> debbugs.gnu.org; Fri, 29 Mar 2019 00:35:01 -0400
Received: from mail-io1-f53.google.com ([209.85.166.53]:37937)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1h9jEC-0006JD-1A
 for 24450 <at> debbugs.gnu.org; Fri, 29 Mar 2019 00:34:54 -0400
Received: by mail-io1-f53.google.com with SMTP id v4so720376ioj.5
 for <24450 <at> debbugs.gnu.org>; Thu, 28 Mar 2019 21:34:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=6mzXAgxhQEFz3WJtxhSlcuJcz7t5auGz15CWS+yDmas=;
 b=sjO4EEMnfZzZaacfxMLXfCUq96B5sSGlNDF7Pj1gu+FWcFD/mvoh92A5+j+KB//8Qw
 cPs+lZapvEQP6MPZJ2TnsJVCNhr8vwBhkjXuo864u9qu6mVk8/wiYgpTzZfCMEpU7fgD
 N/Xj4khvTs73BPmS0/ccf/EO9QJ1x5ajb9DIxk1Va+PBRx5DiaQL2vReYOTUV9hOowlN
 qj1sKKV+dE4vUxDAzYmi+9PKH2v4d9+l/kuOJdKyWdEEt6V6dQwVGQ7wBu9om2NrNPv5
 hGSmRvPqICpHVRkUPOnhLnbC8bbzNH+zBUrcUGYHTjeXxWQzyZSpZrk2UG8J0YVUJKx4
 9+kg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=6mzXAgxhQEFz3WJtxhSlcuJcz7t5auGz15CWS+yDmas=;
 b=AWf5ShgZXVHYMjmvFYnv0Nbbb8+udWTySLmdi1JT5X/5Wtlc9cIlh3QfVl2V2rYt+I
 JjRqvgsWqGxP4cVjKw/rxf2Yc+7aXmRbidjx/60tE8XDrSUGCV34fKmEZ0E1cx21UtjX
 hQxyNsnOg2Ibo5YReNE5imF7rPQlJJHSNkGkLv48MAFMUMvtAdLO3cKt3GvaawxqT0dh
 l/USxkliXgGWmshu4lfILMWFl38JU1KT4E6PSgTfvhFFzvwGE0bt+VEW96Y6Xm/Lk2Fe
 Gcu8UKTHmn/jbbK/WuiklhTdJG1F7+tHMoDejkkX/MTynmy4PqxdJeoGGei8cs/9OnFd
 EQjg==
X-Gm-Message-State: APjAAAW6HVgp4l2MqMqtnbaDGAJZGNTFrObX5YAHqhetkiGqNjyLrYcM
 7zCorCch5RdsthHriZxWRZI/xFSfhhE=
X-Google-Smtp-Source: APXvYqxYBmqHGPCE5cB8PfRj8zcUadcObTJHrelEUvXiycq+I/tDpJdLYTQ9JxUHpeDr8YOtbAymhw==
X-Received: by 2002:a6b:4a09:: with SMTP id w9mr31624350iob.79.1553834086245; 
 Thu, 28 Mar 2019 21:34:46 -0700 (PDT)
Received: from kwak ([2607:f2c0:94b4:fa00::235])
 by smtp.gmail.com with ESMTPSA id y16sm450522ion.64.2019.03.28.21.34.44
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Thu, 28 Mar 2019 21:34:45 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: ng0 <ng0@HIDDEN>
Subject: [PATCH] bug#24450: pypi importer outputs strange character series in
 optional dependency case.
References: <87h99fipj1.fsf@HIDDEN>
Date: Fri, 29 Mar 2019 00:34:43 -0400
In-Reply-To: <87h99fipj1.fsf@HIDDEN> (ng0's message of "Fri, 16
 Sep 2016 20:00:02 +0000")
Message-ID: <87tvfm1eos.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="==-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 24450
Cc: 24450 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--==-=-=
Content-Type: multipart/mixed; boundary="=-=-="

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hello,

ng0 <ng0@HIDDEN> writes:

> I think this should not happen with pypi import:
>
>   (inputs
>     `(("python-certifi=3D=3D2016.2.28"
>        ,python-certifi=3D=3D2016.2.28)
>       ("python-dateutil=3D=3D2.5.3"
>        ,python-dateutil=3D=3D2.5.3)
>       ("python-flask-babel=3D=3D0.11.1"
>        ,python-flask-babel=3D=3D0.11.1)
>       ("python-flask=3D=3D0.11.1" ,python-flask=3D=3D0.11.1)
>       ("python-lxml=3D=3D3.6.0" ,python-lxml=3D=3D3.6.0)
>       ("python-ndg-httpsclient=3D=3D0.4.1"
>        ,python-ndg-httpsclient=3D=3D0.4.1)
>       ("python-pyasn1-modules=3D=3D0.0.8"
>        ,python-pyasn1-modules=3D=3D0.0.8)
>       ("python-pyasn1=3D=3D0.1.9" ,python-pyasn1=3D=3D0.1.9)
>       ("python-pygments=3D=3D2.1.3"
>        ,python-pygments=3D=3D2.1.3)
>       ("python-pyopenssl=3D=3D0.15.1"
>        ,python-pyopenssl=3D=3D0.15.1)
>       ("python-pyyaml=3D=3D3.11" ,python-pyyaml=3D=3D3.11)
>       ("python-requests[socks]=3D=3D2.10.0"
>        ,#{python-requests\x5b;socks\x5d;=3D=3D2.10.0}#)
>       ("python-setuptools" ,python-setuptools)))
>
>
> I can understand the version numbers, I can also understand the optional
> socks building/module of the python-requests, but why does it read like
> Gobbledygook?  Can't we improve the output here?
>
> For version numbers, this is not a format which happened recently which
> is exclusive for python build system right? This is just bad formated
> because of the pypi query.
> I will first try and not pin the application to these version numbers,
> maybe itjustworks=E2=84=A2.
>
>
> To reproduce: "guix import pypi searx"

The following patches fix this, and more!


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0001-import-pypi-Do-not-consider-requirements.txt-files.patch
Content-Transfer-Encoding: quoted-printable

From=2054e44b7397f17910d95dbdb233d23e5c97c095aa Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Thu, 28 Mar 2019 00:26:00 -0400
Subject: [PATCH 1/7] import: pypi: Do not consider requirements.txt files.

* guix/import/pypi.scm (guess-requirements): Update comment.
[guess-requirements-from-source]: Do not attempt to parse the file
requirements.txt.  Streamline logic.
=2D--
 guix/import/pypi.scm | 35 +++++++++++++----------------------
 tests/pypi.scm       | 23 +++++++++++------------
 2 files changed, 24 insertions(+), 34 deletions(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 3a20fc4b9b..8269aa61d7 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -206,35 +206,26 @@ cannot determine package dependencies"))
           (call-with-temporary-directory
            (lambda (dir)
              (let* ((pypi-name (string-take dirname (string-rindex dirname=
 #\-)))
=2D                    (req-files (list (string-append dirname "/requiremen=
ts.txt")
=2D                                     (string-append dirname "/" pypi-nam=
e ".egg-info"
=2D                                                    "/requires.txt")))
=2D                    (exit-codes (map (lambda (file-name)
=2D                                       (parameterize ((current-error-por=
t (%make-void-port "rw+"))
=2D                                                      (current-output-po=
rt (%make-void-port "rw+")))
=2D                                         (system* "tar" "xf" tarball "-C=
" dir file-name)))
=2D                                     req-files)))
=2D               ;; Only one of these files needs to exist.
=2D               (if (any zero? exit-codes)
=2D                   (match (find-files dir)
=2D                     ((file . _)
=2D                      (read-requirements file))
=2D                     (()
=2D                      (warning (G_ "No requirements file found.\n"))))
+                    (requires.txt (string-append dirname "/" pypi-name
+                                                 ".egg-info" "/requires.tx=
t"))
+                    (exit-code (parameterize ((current-error-port (%make-v=
oid-port "rw+"))
+                                              (current-output-port (%make-=
void-port "rw+")))
+                                 (system* "tar" "xf" tarball "-C" dir requ=
ires.txt))))
+               (if (zero? exit-code)
+                   (read-requirements (string-append dir "/" requires.txt))
                    (begin
=2D                     (warning (G_ "Failed to extract requirements files\=
n"))
+                     (warning
+                      (G_ "Failed to extract file: ~a from source.~%")
+                      requires.txt)
                      '())))))
           '())))
=20
=2D  ;; First, try to compute the requirements using the wheel, since that =
is the
=2D  ;; most reliable option. If a wheel is not provided for this package, =
try
=2D  ;; getting them by reading either the "requirements.txt" file or the
=2D  ;; "requires.txt" from the egg-info directory from the source tarball.=
 Note
=2D  ;; that "requirements.txt" is not mandatory, so this is likely to fail.
+  ;; First, try to compute the requirements using the wheel, else, fallbac=
k to
+  ;; reading the "requires.txt" from the egg-info directory from the source
+  ;; tarball.
   (or (guess-requirements-from-wheel)
       (guess-requirements-from-source)))
=20
=2D
 (define (compute-inputs source-url wheel-url tarball)
   "Given the SOURCE-URL of an already downloaded TARBALL, return a list of
 name/variable pairs describing the required inputs of this package.  Also
diff --git a/tests/pypi.scm b/tests/pypi.scm
index 6daa44a6e7..335be42644 100644
=2D-- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -23,7 +23,7 @@
   #:use-module (gcrypt hash)
   #:use-module (guix tests)
   #:use-module (guix build-system python)
=2D  #:use-module ((guix build utils) #:select (delete-file-recursively whi=
ch))
+  #:use-module ((guix build utils) #:select (delete-file-recursively which=
 mkdir-p))
   #:use-module (srfi srfi-64)
   #:use-module (ice-9 match))
=20
@@ -55,11 +55,10 @@
 (define test-source-hash
   "")
=20
=2D(define test-requirements
=2D"# A comment
=2D # A comment after a space
+(define test-requires.txt "\
 bar
=2Dbaz > 13.37")
+baz > 13.37
+")
=20
 (define test-metadata
   "{
@@ -107,10 +106,10 @@ baz > 13.37")
              (match url
                ("https://example.com/foo-1.0.0.tar.gz"
                 (begin
=2D                  (mkdir "foo-1.0.0")
=2D                  (with-output-to-file "foo-1.0.0/requirements.txt"
+                  (mkdir-p "foo-1.0.0/foo.egg-info/")
+                  (with-output-to-file "foo-1.0.0/foo.egg-info/requires.tx=
t"
                     (lambda ()
=2D                      (display test-requirements)))
+                      (display test-requires.txt)))
                   (system* "tar" "czvf" file-name "foo-1.0.0/")
                   (delete-file-recursively "foo-1.0.0")
                   (set! test-source-hash
@@ -157,11 +156,11 @@ baz > 13.37")
          (lambda (url file-name)
            (match url
              ("https://example.com/foo-1.0.0.tar.gz"
=2D               (begin
=2D                 (mkdir "foo-1.0.0")
=2D                 (with-output-to-file "foo-1.0.0/requirements.txt"
+              (begin
+                (mkdir-p "foo-1.0.0/foo.egg-info/")
+                (with-output-to-file "foo-1.0.0/foo.egg-info/requires.txt"
                    (lambda ()
=2D                     (display test-requirements)))
+                     (display test-requires.txt)))
                  (system* "tar" "czvf" file-name "foo-1.0.0/")
                  (delete-file-recursively "foo-1.0.0")
                  (set! test-source-hash
=2D-=20
2.20.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0002-import-pypi-Do-not-parse-optional-requirements-from-.patch
Content-Transfer-Encoding: quoted-printable

From=205f79b0502f62bd1dacc8ea143c1dbd9ef7cfc29d Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Thu, 28 Mar 2019 00:26:00 -0400
Subject: [PATCH 2/7] import: pypi: Do not parse optional requirements from
 source.

* guix/import/pypi.scm: Export PARSE-REQUIRES.TXT.
(guess-requirements): Move the READ-REQUIREMENTS procedure to the top level,
and rename it to PARSE-REQUIRES.TXT.  Move the CLEAN-REQUIREMENT and COMMEN=
T?
functions inside the READ-REQUIREMENTS procedure.
(parse-requires.txt): Add a SECTION-HEADER? predicate, and use it to prevent
parsing optional requirements.

* tests/pypi.scm (test-requires-with-sections): New variable.
("parse-requires.txt, with sections"): New test.
("pypi->guix-package"): Mute tar output to stdout.
=2D--
 guix/import/pypi.scm | 76 +++++++++++++++++++++++++++-----------------
 tests/pypi.scm       | 21 ++++++++++--
 2 files changed, 65 insertions(+), 32 deletions(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 8269aa61d7..91e987e9f1 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -47,7 +47,8 @@
   #:use-module (guix upstream)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix build-system python)
=2D  #:export (guix-package->pypi-name
+  #:export (parse-requires.txt
+            guix-package->pypi-name
             pypi-recursive-import
             pypi->guix-package
             %pypi-updater))
@@ -117,6 +118,49 @@ package definition."
     ((package-inputs ...)
      `((propagated-inputs (,'quasiquote ,package-inputs))))))
=20
+(define (clean-requirement s)
+  ;; Given a requirement LINE, as can be found in a setuptools requires.txt
+  ;; file, remove everything other than the actual name of the required
+  ;; package, and return it.
+  (string-take s (or (string-index s (lambda (chr)
+                                       (member chr '(#\space #\> #\=3D #\<=
))))
+                     (string-length s))))
+
+(define (parse-requires.txt requires.txt)
+  "Given REQUIRES.TXT, a Setuptools requires.txt file, return a list of
+requirement names."
+  ;; This is a very incomplete parser, which job is to select the non-opti=
onal
+  ;; dependencies and strip them out of any version information.
+  ;; Alternatively, we could implement a PEG parser with the (ice-9 peg)
+  ;; library and the requirements grammar defined by PEP-0508
+  ;; (https://www.python.org/dev/peps/pep-0508/).
+
+  (define (comment? line)
+    ;; Return #t if the given LINE is a comment, #f otherwise.
+    (eq? (string-ref (string-trim line) 0) #\#))
+
+  (define (section-header? line)
+    ;; Return #t if the given LINE is a section header, #f otherwise.
+    (let ((trimmed-line (string-trim line)))
+      (and (not (string-null? trimmed-line))
+           (eq? (string-ref trimmed-line 0) #\[))))
+
+  (call-with-input-file requires.txt
+    (lambda (port)
+      (let loop ((result '()))
+        (let ((line (read-line port)))
+          ;; Stop when a section is encountered, as sections contains opti=
onal
+          ;; (extra) requirements.  Non-optional requirements must appear
+          ;; before any section is defined.
+          (if (or (eof-object? line) (section-header? line))
+              (reverse result)
+              (cond
+               ((or (string-null? line) (comment? line))
+                (loop result))
+               (else
+                (loop (cons (clean-requirement line)
+                            result))))))))))
+
 (define (guess-requirements source-url wheel-url tarball)
   "Given SOURCE-URL, WHEEL-URL and a TARBALL of the package, return a list
 of the required packages specified in the requirements.txt file.  TARBALL =
will
@@ -139,34 +183,6 @@ be extracted in a temporary directory."
 cannot determine package dependencies"))
           #f)))))
=20
=2D  (define (clean-requirement s)
=2D    ;; Given a requirement LINE, as can be found in a Python requirement=
s.txt
=2D    ;; file, remove everything other than the actual name of the required
=2D    ;; package, and return it.
=2D    (string-take s
=2D      (or (string-index s (lambda (chr) (member chr '(#\space #\> #\=3D =
#\<))))
=2D          (string-length s))))
=2D
=2D  (define (comment? line)
=2D    ;; Return #t if the given LINE is a comment, #f otherwise.
=2D    (eq? (string-ref (string-trim line) 0) #\#))
=2D
=2D  (define (read-requirements requirements-file)
=2D    ;; Given REQUIREMENTS-FILE, a Python requirements.txt file, return a=
 list
=2D    ;; of name/variable pairs describing the requirements.
=2D    (call-with-input-file requirements-file
=2D      (lambda (port)
=2D        (let loop ((result '()))
=2D          (let ((line (read-line port)))
=2D            (if (eof-object? line)
=2D                result
=2D                (cond
=2D                 ((or (string-null? line) (comment? line))
=2D                  (loop result))
=2D                 (else
=2D                  (loop (cons (clean-requirement line)
=2D                              result))))))))))
=2D
   (define (read-wheel-metadata wheel-archive)
     ;; Given WHEEL-ARCHIVE, a ZIP Python wheel archive, return the package=
's
     ;; requirements.
@@ -212,7 +228,7 @@ cannot determine package dependencies"))
                                               (current-output-port (%make-=
void-port "rw+")))
                                  (system* "tar" "xf" tarball "-C" dir requ=
ires.txt))))
                (if (zero? exit-code)
=2D                   (read-requirements (string-append dir "/" requires.tx=
t))
+                   (parse-requires.txt (string-append dir "/" requires.txt=
))
                    (begin
                      (warning
                       (G_ "Failed to extract file: ~a from source.~%")
diff --git a/tests/pypi.scm b/tests/pypi.scm
index 335be42644..e4b7142311 100644
=2D-- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -60,6 +60,15 @@ bar
 baz > 13.37
 ")
=20
+(define test-requires-with-sections "\
+# A comment
+foo ~=3D 3
+bar !=3D 2
+
+[test]
+pytest (>=3D2.5.0)
+")
+
 (define test-metadata
   "{
   \"run_requires\": [
@@ -99,6 +108,12 @@ baz > 13.37
                     (uri (list "https://bitheap.org/cram/cram-0.7.tar.gz"
                                (pypi-uri "cram" "0.7"))))))))
=20
+(test-equal "parse-requires.txt, with sections"
+  '("foo" "bar")
+  (mock ((ice-9 ports) call-with-input-file
+         call-with-input-string)
+        (parse-requires.txt test-requires-with-sections)))
+
 (test-assert "pypi->guix-package"
   ;; Replace network resources with sample data.
     (mock ((guix import utils) url-fetch
@@ -110,7 +125,8 @@ baz > 13.37
                   (with-output-to-file "foo-1.0.0/foo.egg-info/requires.tx=
t"
                     (lambda ()
                       (display test-requires.txt)))
=2D                  (system* "tar" "czvf" file-name "foo-1.0.0/")
+                  (parameterize ((current-output-port (%make-void-port "rw=
+")))
+                    (system* "tar" "czvf" file-name "foo-1.0.0/"))
                   (delete-file-recursively "foo-1.0.0")
                   (set! test-source-hash
                     (call-with-input-file file-name port-sha256))))
@@ -161,7 +177,8 @@ baz > 13.37
                 (with-output-to-file "foo-1.0.0/foo.egg-info/requires.txt"
                    (lambda ()
                      (display test-requires.txt)))
=2D                 (system* "tar" "czvf" file-name "foo-1.0.0/")
+                (parameterize ((current-output-port (%make-void-port "rw+"=
)))
+                  (system* "tar" "czvf" file-name "foo-1.0.0/"))
                  (delete-file-recursively "foo-1.0.0")
                  (set! test-source-hash
                        (call-with-input-file file-name port-sha256))))
=2D-=20
2.20.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0003-import-pypi-Improve-parsing-of-requirement-specifica.patch
Content-Transfer-Encoding: quoted-printable

From=200c62b541a3e8925b5ca31fe55dbe7536cf95151f Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Thu, 28 Mar 2019 00:26:01 -0400
Subject: [PATCH 3/7] import: pypi: Improve parsing of requirement
 specifications.

The previous solution was fragile and could leave unwanted characters in a
requirement name, such as '[' or ']'.

Partially fixes issue #33047 (see:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D33047).

* guix/import/pypi.scm (use-modules): Export SPECIFICATION->REQUIREMENT-NAME
(%requirement-name-regexp): New variable.
(clean-requirement): Rename to...
(specification->requirement-name): this, which now uses
%requirement-name-regexp to select the requirement name from the requirement
specification.
(parse-requires.txt): Adapt.
=2D--
 guix/import/pypi.scm | 43 ++++++++++++++++++++++++++++++++++---------
 tests/pypi.scm       | 12 ++++++++++++
 2 files changed, 46 insertions(+), 9 deletions(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 91e987e9f1..efb5939c78 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -48,6 +48,7 @@
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix build-system python)
   #:export (parse-requires.txt
+            specification->requirement-name
             guix-package->pypi-name
             pypi-recursive-import
             pypi->guix-package
@@ -118,13 +119,37 @@ package definition."
     ((package-inputs ...)
      `((propagated-inputs (,'quasiquote ,package-inputs))))))
=20
=2D(define (clean-requirement s)
=2D  ;; Given a requirement LINE, as can be found in a setuptools requires.=
txt
=2D  ;; file, remove everything other than the actual name of the required
=2D  ;; package, and return it.
=2D  (string-take s (or (string-index s (lambda (chr)
=2D                                       (member chr '(#\space #\> #\=3D #=
\<))))
=2D                     (string-length s))))
+(define %requirement-name-regexp
+  ;; Regexp to match the requirement name in a requirement specification.
+
+  ;; Some grammar, taken from PEP-0508 (see:
+  ;; https://www.python.org/dev/peps/pep-0508/).
+
+  ;; The unified rule can be expressed as:
+  ;; specification =3D wsp* ( url_req | name_req ) wsp*
+
+  ;; where url_req is:
+  ;; url_req =3D name wsp* extras? wsp* urlspec wsp+ quoted_marker?
+
+  ;; and where name_req is:
+  ;; name_req =3D name wsp* extras? wsp* versionspec? wsp* quoted_marker?
+
+  ;; Thus, we need only matching NAME, which is expressed as:
+  ;; identifer_end =3D letterOrDigit | (('-' | '_' | '.' )* letterOrDigit)
+  ;; identifier    =3D letterOrDigit identifier_end*
+  ;; name          =3D identifier
+  (let* ((letter-or-digit "[A-Za-z0-9]")
+         (identifier-end (string-append "(" letter-or-digit "|"
+                                        "[-_.]*" letter-or-digit ")"))
+         (identifier (string-append "^" letter-or-digit identifier-end "*"=
))
+         (name identifier))
+    (make-regexp name)))
+
+(define (specification->requirement-name spec)
+  "Given a specification SPEC, return the requirement name."
+  (match:substring
+   (or (regexp-exec %requirement-name-regexp spec)
+       (error (G_ "Could not extract requirement name in spec:") spec))))
=20
 (define (parse-requires.txt requires.txt)
   "Given REQUIRES.TXT, a Setuptools requires.txt file, return a list of
@@ -158,7 +183,7 @@ requirement names."
                ((or (string-null? line) (comment? line))
                 (loop result))
                (else
=2D                (loop (cons (clean-requirement line)
+                (loop (cons (specification->requirement-name line)
                             result))))))))))
=20
 (define (guess-requirements source-url wheel-url tarball)
@@ -200,7 +225,7 @@ cannot determine package dependencies"))
                                             (hash-ref (list-ref run_requir=
es 0)
                                                        "requires")
                                             '())))
=2D                     (map clean-requirement requirements)))))
+                     (map specification->requirement-name requirements)))))
              (lambda ()
                (delete-file json-file)
                (rmdir dirname))))))
diff --git a/tests/pypi.scm b/tests/pypi.scm
index e4b7142311..82d6bba8dd 100644
=2D-- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -55,6 +55,14 @@
 (define test-source-hash
   "")
=20
+(define test-specifications
+  '("Fizzy [foo, bar]"
+    "PickyThing<1.6,>1.9,!=3D1.9.6,<2.0a0,=3D=3D2.4c1"
+    "SomethingWithMarker[foo]>1.0;python_version<\"2.7\""
+    "requests [security,tests] >=3D 2.8.1, =3D=3D 2.8.* ; python_version <=
 \"2.7\""
+    "pip @ https://github.com/pypa/pip/archive/1.3.1.zip#\
+sha1=3Dda9234ee9982d4bbb3c72346a6de940a148ea686"))
+
 (define test-requires.txt "\
 bar
 baz > 13.37
@@ -108,6 +116,10 @@ pytest (>=3D2.5.0)
                     (uri (list "https://bitheap.org/cram/cram-0.7.tar.gz"
                                (pypi-uri "cram" "0.7"))))))))
=20
+(test-equal "specification->requirement-name"
+  '("Fizzy" "PickyThing" "SomethingWithMarker" "requests" "pip")
+  (map specification->requirement-name test-specifications))
+
 (test-equal "parse-requires.txt, with sections"
   '("foo" "bar")
   (mock ((ice-9 ports) call-with-input-file
=2D-=20
2.20.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0004-import-pypi-Deduplicate-requirements.patch
Content-Transfer-Encoding: quoted-printable

From=2076e4a3150f8126e0b952c6129b6e1371afba80c0 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Thu, 28 Mar 2019 00:26:01 -0400
Subject: [PATCH 4/7] import: pypi: Deduplicate requirements.

* guix/import/pypi.scm (parse-requires.txt): Remove potential duplicates.
=2D--
 guix/import/pypi.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index efb5939c78..a90be67bb0 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -178,7 +178,11 @@ requirement names."
           ;; (extra) requirements.  Non-optional requirements must appear
           ;; before any section is defined.
           (if (or (eof-object? line) (section-header? line))
=2D              (reverse result)
+              ;; Duplicates can occur, since the same requirement can be
+              ;; listed multiple times with different conditional markers,=
 e.g.
+              ;; pytest >=3D 3 ; python_version >=3D "3.3"
+              ;; pytest < 3 ; python_version < "3.3"
+              (reverse (delete-duplicates result))
               (cond
                ((or (string-null? line) (comment? line))
                 (loop result))
=2D-=20
2.20.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0005-import-pypi-Support-more-types-of-archives.patch
Content-Transfer-Encoding: quoted-printable

From=2073e27235cac1275ba7671fd2364325cf5788cb3c Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Thu, 28 Mar 2019 00:26:02 -0400
Subject: [PATCH 5/7] import: pypi: Support more types of archives.

This change enables the PyPI importer to look for requirements in a source
archive of a different type than "tar.gz" or "tar.bz2".

* guix/import/pypi.scm: (guess-requirements)[tarball-directory]: Rename to.=
..
[archive-root-directory]: this. Use COMPRESSED-FILED? to determine if an
archive is supported or not.
[guess-requirements-from-source]: Adapt to use the new method, and use unzip
to extract ZIP archives.
(guess-requirements): Rename the TARBALL argument to ARCHIVE, to denote the
archive format is no longer bound specifically to the Tar format.
=2D--
 guix/import/pypi.scm | 47 ++++++++++++++++++++++----------------------
 1 file changed, 24 insertions(+), 23 deletions(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index a90be67bb0..8e93653717 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -190,27 +190,24 @@ requirement names."
                 (loop (cons (specification->requirement-name line)
                             result))))))))))
=20
=2D(define (guess-requirements source-url wheel-url tarball)
=2D  "Given SOURCE-URL, WHEEL-URL and a TARBALL of the package, return a li=
st
=2Dof the required packages specified in the requirements.txt file.  TARBAL=
L will
+(define (guess-requirements source-url wheel-url archive)
+  "Given SOURCE-URL, WHEEL-URL and a ARCHIVE of the package, return a list
+of the required packages specified in the requirements.txt file.  ARCHIVE =
will
 be extracted in a temporary directory."
=20
=2D  (define (tarball-directory url)
=2D    ;; Given the URL of the package's tarball, return the name of the di=
rectory
+  (define (archive-root-directory url)
+    ;; Given the URL of the package's archive, return the name of the dire=
ctory
     ;; that will be created upon decompressing it. If the filetype is not
     ;; supported, return #f.
=2D    ;; TODO: Support more archive formats.
=2D    (let ((basename (substring url (+ 1 (string-rindex url #\/)))))
=2D      (cond
=2D       ((string-suffix? ".tar.gz" basename)
=2D        (string-drop-right basename 7))
=2D       ((string-suffix? ".tar.bz2" basename)
=2D        (string-drop-right basename 8))
=2D       (else
+    (if (compressed-file? url)
+        (let ((root-directory (file-sans-extension (basename url))))
+          (if (string=3D? "tar" (file-extension root-directory))
+              (file-sans-extension root-directory)
+              root-directory))
         (begin
=2D          (warning (G_ "Unsupported archive format: \
=2Dcannot determine package dependencies"))
=2D          #f)))))
+          (warning (G_ "Unsupported archive format (~a): \
+cannot determine package dependencies") (file-extension url))
+          #f)))
=20
   (define (read-wheel-metadata wheel-archive)
     ;; Given WHEEL-ARCHIVE, a ZIP Python wheel archive, return the package=
's
@@ -246,16 +243,20 @@ cannot determine package dependencies"))
=20
   (define (guess-requirements-from-source)
     ;; Return the package's requirements by guessing them from the source.
=2D    (let ((dirname (tarball-directory source-url)))
+    (let ((dirname (archive-root-directory source-url))
+          (extension (file-extension source-url)))
       (if (string? dirname)
           (call-with-temporary-directory
            (lambda (dir)
              (let* ((pypi-name (string-take dirname (string-rindex dirname=
 #\-)))
                     (requires.txt (string-append dirname "/" pypi-name
                                                  ".egg-info" "/requires.tx=
t"))
=2D                    (exit-code (parameterize ((current-error-port (%make=
-void-port "rw+"))
=2D                                              (current-output-port (%mak=
e-void-port "rw+")))
=2D                                 (system* "tar" "xf" tarball "-C" dir re=
quires.txt))))
+                    (exit-code
+                     (parameterize ((current-error-port (%make-void-port "=
rw+"))
+                                    (current-output-port (%make-void-port =
"rw+")))
+                       (if (string=3D? "zip" extension)
+                           (system* "unzip" archive "-d" dir requires.txt)
+                           (system* "tar" "xf" archive "-C" dir requires.t=
xt)))))
                (if (zero? exit-code)
                    (parse-requires.txt (string-append dir "/" requires.txt=
))
                    (begin
@@ -271,13 +272,13 @@ cannot determine package dependencies"))
   (or (guess-requirements-from-wheel)
       (guess-requirements-from-source)))
=20
=2D(define (compute-inputs source-url wheel-url tarball)
=2D  "Given the SOURCE-URL of an already downloaded TARBALL, return a list =
of
+(define (compute-inputs source-url wheel-url archive)
+  "Given the SOURCE-URL of an already downloaded ARCHIVE, return a list of
 name/variable pairs describing the required inputs of this package.  Also
 return the unaltered list of upstream dependency names."
   (let ((dependencies
          (remove (cut string=3D? "argparse" <>)
=2D                 (guess-requirements source-url wheel-url tarball))))
+                 (guess-requirements source-url wheel-url archive))))
     (values (sort
              (map (lambda (input)
                     (let ((guix-name (python->package-name input)))
=2D-=20
2.20.1


--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
 filename=0006-import-pypi-Parse-wheel-METADATA-instead-of-metadata.patch
Content-Transfer-Encoding: quoted-printable

From=20fb0547ef225103c0f8355a7eccc41e0d028f6563 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Thu, 28 Mar 2019 00:26:03 -0400
Subject: [PATCH 6/7] import: pypi: Parse wheel METADATA instead of
 metadata.json.

With newer Wheel releases, there is no more metadata.json file; the METADATA
file should be used instead (see: https://github.com/pypa/wheel/issues/195).

This change updates our PyPI importer so that it uses the later.

* guix/import/pypi.scm (define-module): Remove unnecessary modules and expo=
rt
  the PARSE-WHEEL-METADATA method.
(parse-wheel-metadata): Add method.
(guess-requirements): Use it.
* tests/pypi.scm (test-metadata): Test it.
=2D--
 guix/import/pypi.scm | 66 +++++++++++++++++++++++++++++---------------
 tests/pypi.scm       | 60 ++++++++++++++++++++++++++++++----------
 2 files changed, 89 insertions(+), 37 deletions(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 8e93653717..c520213b6a 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -21,9 +21,7 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
=20
 (define-module (guix import pypi)
=2D  #:use-module (ice-9 binary-ports)
   #:use-module (ice-9 match)
=2D  #:use-module (ice-9 pretty-print)
   #:use-module (ice-9 regex)
   #:use-module (ice-9 receive)
   #:use-module ((ice-9 rdelim) #:select (read-line))
@@ -31,9 +29,6 @@
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
=2D  #:use-module (rnrs bytevectors)
=2D  #:use-module (json)
=2D  #:use-module (web uri)
   #:use-module (guix ui)
   #:use-module (guix utils)
   #:use-module ((guix build utils)
@@ -48,6 +43,7 @@
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix build-system python)
   #:export (parse-requires.txt
+            parse-wheel-metadata
             specification->requirement-name
             guix-package->pypi-name
             pypi-recursive-import
@@ -190,6 +186,37 @@ requirement names."
                 (loop (cons (specification->requirement-name line)
                             result))))))))))
=20
+(define (parse-wheel-metadata metadata)
+  "Given METADATA, a Wheel metadata file, return a list of requirement nam=
es."
+  ;; METADATA is a RFC-2822-like, header based file.
+
+  (define (requires-dist-header? line)
+    ;; Return #t if the given LINE is a Requires-Dist header.
+    (regexp-match? (string-match "^Requires-Dist: " line)))
+
+  (define (requires-dist-value line)
+    (string-drop line (string-length "Requires-Dist: ")))
+
+  (define (extra? line)
+    ;; Return #t if the given LINE is an "extra" requirement.
+    (regexp-match? (string-match "extra =3D=3D " line)))
+
+  (call-with-input-file metadata
+    (lambda (port)
+      (let loop ((requirements '()))
+        (let ((line (read-line port)))
+          ;; Stop at the first 'Provides-Extra' section: the non-optional
+          ;; requirements appear before the optional ones.
+          (if (eof-object? line)
+              (reverse (delete-duplicates requirements))
+              (cond
+               ((and (requires-dist-header? line) (not (extra? line)))
+                (loop (cons (specification->requirement-name
+                             (requires-dist-value line))
+                            requirements)))
+               (else
+                (loop requirements)))))))))
+
 (define (guess-requirements source-url wheel-url archive)
   "Given SOURCE-URL, WHEEL-URL and a ARCHIVE of the package, return a list
 of the required packages specified in the requirements.txt file.  ARCHIVE =
will
@@ -211,25 +238,18 @@ cannot determine package dependencies") (file-extensi=
on url))
=20
   (define (read-wheel-metadata wheel-archive)
     ;; Given WHEEL-ARCHIVE, a ZIP Python wheel archive, return the package=
's
=2D    ;; requirements.
+    ;; requirements, or #f if the metadata file contained therein couldn't=
 be
+    ;; extracted.
     (let* ((dirname (wheel-url->extracted-directory wheel-url))
=2D           (json-file (string-append dirname "/metadata.json")))
=2D      (and (zero? (system* "unzip" "-q" wheel-archive json-file))
=2D           (dynamic-wind
=2D             (const #t)
=2D             (lambda ()
=2D               (call-with-input-file json-file
=2D                 (lambda (port)
=2D                   (let* ((metadata (json->scm port))
=2D                          (run_requires (hash-ref metadata "run_requires=
"))
=2D                          (requirements (if run_requires
=2D                                            (hash-ref (list-ref run_requ=
ires 0)
=2D                                                       "requires")
=2D                                            '())))
=2D                     (map specification->requirement-name requirements))=
)))
=2D             (lambda ()
=2D               (delete-file json-file)
=2D               (rmdir dirname))))))
+           (metadata (string-append dirname "/METADATA")))
+      (call-with-temporary-directory
+       (lambda (dir)
+         (if (zero? (system* "unzip" "-q" wheel-archive "-d" dir metadata))
+             (parse-wheel-metadata (string-append dir "/" metadata))
+             (begin
+               (warning
+                (G_ "Failed to extract file: ~a from wheel.~%") metadata)
+               #f))))))
=20
   (define (guess-requirements-from-wheel)
     ;; Return the package's requirements using the wheel, or #f if an error
diff --git a/tests/pypi.scm b/tests/pypi.scm
index 82d6bba8dd..ca8cb5f6de 100644
=2D-- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -21,6 +21,7 @@
   #:use-module (guix import pypi)
   #:use-module (guix base32)
   #:use-module (gcrypt hash)
+  #:use-module (guix memoization)
   #:use-module (guix tests)
   #:use-module (guix build-system python)
   #:use-module ((guix build utils) #:select (delete-file-recursively which=
 mkdir-p))
@@ -77,17 +78,33 @@ bar !=3D 2
 pytest (>=3D2.5.0)
 ")
=20
=2D(define test-metadata
=2D  "{
=2D  \"run_requires\": [
=2D    {
=2D      \"requires\": [
=2D        \"bar\",
=2D        \"baz (>13.37)\"
=2D      ]
=2D    }
=2D  ]
=2D}")
+(define test-metadata "\
+Classifier: Programming Language :: Python :: 3.7
+Requires-Dist: baz ~=3D 3
+Requires-Dist: bar !=3D 2
+Provides-Extra: test
+pytest (>=3D2.5.0)
+")
+
+(define test-metadata-with-extras "
+Classifier: Programming Language :: Python :: 3.7
+Requires-Python: >=3D2.7, !=3D3.0.*, !=3D3.1.*, !=3D3.2.*, !=3D3.3.*
+Requires-Dist: wrapt (<2,>=3D1)
+Requires-Dist: bar
+
+Provides-Extra: dev
+Requires-Dist: tox ; extra =3D=3D 'dev'
+Requires-Dist: bumpversion (<1) ; extra =3D=3D 'dev'
+")
+
+;;; Provides-Extra can appear before Requires-Dist.
+(define test-metadata-with-extras-jedi "\
+Requires-Python: >=3D2.7, !=3D3.0.*, !=3D3.1.*, !=3D3.2.*, !=3D3.3.*
+Provides-Extra: testing
+Requires-Dist: parso (>=3D0.3.0)
+Provides-Extra: testing
+Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testing'
+")
=20
 (test-begin "pypi")
=20
@@ -126,6 +143,18 @@ pytest (>=3D2.5.0)
          call-with-input-string)
         (parse-requires.txt test-requires-with-sections)))
=20
+(test-equal "parse-wheel-metadata, with extras"
+  '("wrapt" "bar")
+  (mock ((ice-9 ports) call-with-input-file
+         call-with-input-string)
+        (parse-wheel-metadata test-metadata-with-extras)))
+
+(test-equal "parse-wheel-metadata, with extras - Jedi"
+  '("parso")
+  (mock ((ice-9 ports) call-with-input-file
+         call-with-input-string)
+        (parse-wheel-metadata test-metadata-with-extras-jedi)))
+
 (test-assert "pypi->guix-package"
   ;; Replace network resources with sample data.
     (mock ((guix import utils) url-fetch
@@ -188,7 +217,7 @@ pytest (>=3D2.5.0)
                 (mkdir-p "foo-1.0.0/foo.egg-info/")
                 (with-output-to-file "foo-1.0.0/foo.egg-info/requires.txt"
                    (lambda ()
=2D                     (display test-requires.txt)))
+                     (display "wrong data to make sure we're testing wheel=
s ")))
                 (parameterize ((current-output-port (%make-void-port "rw+"=
)))
                   (system* "tar" "czvf" file-name "foo-1.0.0/"))
                  (delete-file-recursively "foo-1.0.0")
@@ -197,13 +226,13 @@ pytest (>=3D2.5.0)
              ("https://example.com/foo-1.0.0-py2.py3-none-any.whl"
                (begin
                  (mkdir "foo-1.0.0.dist-info")
=2D                 (with-output-to-file "foo-1.0.0.dist-info/metadata.json"
+                 (with-output-to-file "foo-1.0.0.dist-info/METADATA"
                    (lambda ()
                      (display test-metadata)))
                  (let ((zip-file (string-append file-name ".zip")))
                    ;; zip always adds a "zip" extension to the file it cre=
ates,
                    ;; so we need to rename it.
=2D                   (system* "zip" zip-file "foo-1.0.0.dist-info/metadata=
.json")
+                   (system* "zip" zip-file "foo-1.0.0.dist-info/METADATA")
                    (rename-file zip-file file-name))
                  (delete-file-recursively "foo-1.0.0.dist-info")))
              (_ (error "Unexpected URL: " url)))))
@@ -215,6 +244,9 @@ pytest (>=3D2.5.0)
                             (string-length test-json)))
                    ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #=
f)
                    (_ (error "Unexpected URL: " url)))))
+              ;; Not clearing the memoization cache here would mean return=
ing the value
+              ;; computed in the previous test.
+              (invalidate-memoization! pypi->guix-package)
               (match (pypi->guix-package "foo")
                 (('package
                    ('name "python-foo")
=2D-=20
2.20.1


--=-=-=
Content-Type: text/x-patch; charset=utf-8
Content-Disposition: attachment;
 filename=0007-import-pypi-Include-optional-test-inputs-as-native-i.patch
Content-Transfer-Encoding: quoted-printable

From=20ea0f24eb7b19c57ebb24ec48ba776b240bccfc99 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Date: Thu, 28 Mar 2019 23:12:26 -0400
Subject: [PATCH 7/7] import: pypi: Include optional test inputs as
 native-inputs.

* guix/import/pypi.scm (maybe-inputs): Add INPUT-TYPE argument, and use it.
(test-section?): New predicate.
(parse-requires.txt): Collect the optional test inputs, and return them as =
the
second element of the returned list.
(parse-wheel-metadata): Likewise.
(guess-requirements): Adapt, and hide unzip output.
(make-pypi-sexp): Likewise, and include the test inputs requirements as nat=
ive
inputs in the returned package expression.

* tests/pypi.scm (test-requires.txt): Include a test section in the
test-requires.txt data.
(test-requires.txt-beaker): New variable.
("parse-requires.txt"): Adapt.
("parse-requires.txt - Beaker"): New test.
("parse-wheel-metadata, with extras"): Adapt.
("parse-wheel-metadata, with extras - Jedi"): Adapt.
("pypi->guix-package, no wheel"): Re-indent, and add the expected
native-inputs.
("pypi->guix-package, wheels"): Likewise.
("pypi->guix-package, no usable requirement file."): New test.
=2D--
 guix/import/pypi.scm | 158 ++++++++++++++++++++++++++++---------------
 tests/pypi.scm       | 123 +++++++++++++++++++++++++--------
 2 files changed, 199 insertions(+), 82 deletions(-)

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index c520213b6a..f84ad88e44 100644
=2D-- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -4,6 +4,7 @@
 ;;; Copyright =C2=A9 2015, 2016, 2017 Ludovic Court=C3=A8s <ludo@HIDDEN>
 ;;; Copyright =C2=A9 2017 Mathieu Othacehe <m.othacehe@HIDDEN>
 ;;; Copyright =C2=A9 2018 Ricardo Wurmus <rekado@HIDDEN>
+;;; Copyright =C2=A9 2019 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -26,6 +27,7 @@
   #:use-module (ice-9 receive)
   #:use-module ((ice-9 rdelim) #:select (read-line))
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
@@ -106,14 +108,15 @@ package on PyPI."
     ((name version _ ...)
      (string-append name "-" version ".dist-info"))))
=20
=2D(define (maybe-inputs package-inputs)
+(define (maybe-inputs package-inputs input-type)
   "Given a list of PACKAGE-INPUTS, tries to generate the 'inputs' field of=
 a
=2Dpackage definition."
+package definition.  INPUT-TYPE, a symbol, is used to populate the name of
+the input field."
   (match package-inputs
     (()
      '())
     ((package-inputs ...)
=2D     `((propagated-inputs (,'quasiquote ,package-inputs))))))
+     `((,input-type (,'quasiquote ,package-inputs))))))
=20
 (define %requirement-name-regexp
   ;; Regexp to match the requirement name in a requirement specification.
@@ -147,11 +150,21 @@ package definition."
    (or (regexp-exec %requirement-name-regexp spec)
        (error (G_ "Could not extract requirement name in spec:") spec))))
=20
+(define (test-section? name)
+  "Return #t if the section name contains 'test' or 'dev'."
+  (any (cut string-contains-ci name <>)
+       '("test" "dev")))
+
 (define (parse-requires.txt requires.txt)
=2D  "Given REQUIRES.TXT, a Setuptools requires.txt file, return a list of
=2Drequirement names."
=2D  ;; This is a very incomplete parser, which job is to select the non-op=
tional
=2D  ;; dependencies and strip them out of any version information.
+  "Given REQUIRES.TXT, a Setuptools requires.txt file, return a pair of re=
quirements.
+
+The first element of the pair contains the required dependencies while the
+second the optional test dependencies.  Note that currently, optional,
+non-test dependencies are omitted since these can be difficult or expensiv=
e to
+satisfy."
+
+  ;; This is a very incomplete parser, which job is to read in the require=
ment
+  ;; specification lines, and strip them out of any version information.
   ;; Alternatively, we could implement a PEG parser with the (ice-9 peg)
   ;; library and the requirements grammar defined by PEP-0508
   ;; (https://www.python.org/dev/peps/pep-0508/).
@@ -168,57 +181,89 @@ requirement names."
=20
   (call-with-input-file requires.txt
     (lambda (port)
=2D      (let loop ((result '()))
+      (let loop ((required-deps '())
+                 (test-deps '())
+                 (inside-test-section? #f)
+                 (optional? #f))
         (let ((line (read-line port)))
=2D          ;; Stop when a section is encountered, as sections contains op=
tional
=2D          ;; (extra) requirements.  Non-optional requirements must appear
=2D          ;; before any section is defined.
=2D          (if (or (eof-object? line) (section-header? line))
+          (if (eof-object? line)
               ;; Duplicates can occur, since the same requirement can be
               ;; listed multiple times with different conditional markers,=
 e.g.
               ;; pytest >=3D 3 ; python_version >=3D "3.3"
               ;; pytest < 3 ; python_version < "3.3"
=2D              (reverse (delete-duplicates result))
+              (map (compose reverse delete-duplicates)
+                   (list required-deps test-deps))
               (cond
                ((or (string-null? line) (comment? line))
=2D                (loop result))
=2D               (else
+                (loop required-deps test-deps inside-test-section? optiona=
l?))
+               ((section-header? line)
+                ;; Encountering a section means that all the requirements
+                ;; listed below are optional. Since we want to pick only t=
he
+                ;; test dependencies from the optional dependencies, we mu=
st
+                ;; track those separately.
+                (loop required-deps test-deps (test-section? line) #t))
+               (inside-test-section?
+                (loop required-deps
+                      (cons (specification->requirement-name line)
+                            test-deps)
+                      inside-test-section? optional?))
+               ((not optional?)
                 (loop (cons (specification->requirement-name line)
=2D                            result))))))))))
+                            required-deps)
+                      test-deps inside-test-section? optional?))
+               (optional?
+                ;; Skip optional items.
+                (loop required-deps test-deps inside-test-section? optiona=
l?))
+               (else
+                (warning (G_ "parse-requires.txt reached an unexpected \
+condition on line ~a~%") line)))))))))
=20
 (define (parse-wheel-metadata metadata)
=2D  "Given METADATA, a Wheel metadata file, return a list of requirement n=
ames."
+  "Given METADATA, a Wheel metadata file, return a pair of requirements.
+
+The first element of the pair contains the required dependencies while the=
 second the optional
+test dependencies.  Note that currently, optional, non-test dependencies a=
re
+omitted since these can be difficult or expensive to satisfy."
   ;; METADATA is a RFC-2822-like, header based file.
=20
   (define (requires-dist-header? line)
     ;; Return #t if the given LINE is a Requires-Dist header.
=2D    (regexp-match? (string-match "^Requires-Dist: " line)))
+    (string-match "^Requires-Dist: " line))
=20
   (define (requires-dist-value line)
     (string-drop line (string-length "Requires-Dist: ")))
=20
   (define (extra? line)
     ;; Return #t if the given LINE is an "extra" requirement.
=2D    (regexp-match? (string-match "extra =3D=3D " line)))
+    (string-match "extra =3D=3D '(.*)'" line))
+
+  (define (test-requirement? line)
+    (let ((extra-label (match:substring (extra? line) 1)))
+      (and extra-label (test-section? extra-label))))
=20
   (call-with-input-file metadata
     (lambda (port)
=2D      (let loop ((requirements '()))
+      (let loop ((required-deps '())
+                 (test-deps '()))
         (let ((line (read-line port)))
=2D          ;; Stop at the first 'Provides-Extra' section: the non-optional
=2D          ;; requirements appear before the optional ones.
           (if (eof-object? line)
=2D              (reverse (delete-duplicates requirements))
+              (map (compose reverse delete-duplicates)
+                   (list required-deps test-deps))
               (cond
                ((and (requires-dist-header? line) (not (extra? line)))
                 (loop (cons (specification->requirement-name
                              (requires-dist-value line))
=2D                            requirements)))
+                            required-deps)
+                      test-deps))
+               ((and (requires-dist-header? line) (test-requirement? line))
+                (loop required-deps
+                      (cons (specification->requirement-name (requires-dis=
t-value line))
+                            test-deps)))
                (else
=2D                (loop requirements)))))))))
+                (loop required-deps test-deps))))))))) ;skip line
=20
 (define (guess-requirements source-url wheel-url archive)
=2D  "Given SOURCE-URL, WHEEL-URL and a ARCHIVE of the package, return a li=
st
+  "Given SOURCE-URL, WHEEL-URL and an ARCHIVE of the package, return a list
 of the required packages specified in the requirements.txt file.  ARCHIVE =
will
 be extracted in a temporary directory."
=20
@@ -244,7 +289,10 @@ cannot determine package dependencies") (file-extensio=
n url))
            (metadata (string-append dirname "/METADATA")))
       (call-with-temporary-directory
        (lambda (dir)
=2D         (if (zero? (system* "unzip" "-q" wheel-archive "-d" dir metadat=
a))
+         (if (zero?
+              (parameterize ((current-error-port (%make-void-port "rw+"))
+                             (current-output-port (%make-void-port "rw+")))
+                (system* "unzip" wheel-archive "-d" dir metadata)))
              (parse-wheel-metadata (string-append dir "/" metadata))
              (begin
                (warning
@@ -283,32 +331,38 @@ cannot determine package dependencies") (file-extensi=
on url))
                      (warning
                       (G_ "Failed to extract file: ~a from source.~%")
                       requires.txt)
=2D                     '())))))
=2D          '())))
+                     (list '() '()))))))
+          (list '() '()))))
=20
   ;; First, try to compute the requirements using the wheel, else, fallbac=
k to
   ;; reading the "requires.txt" from the egg-info directory from the source
=2D  ;; tarball.
+  ;; archive.
   (or (guess-requirements-from-wheel)
       (guess-requirements-from-source)))
=20
 (define (compute-inputs source-url wheel-url archive)
=2D  "Given the SOURCE-URL of an already downloaded ARCHIVE, return a list =
of
=2Dname/variable pairs describing the required inputs of this package.  Also
=2Dreturn the unaltered list of upstream dependency names."
=2D  (let ((dependencies
=2D         (remove (cut string=3D? "argparse" <>)
=2D                 (guess-requirements source-url wheel-url archive))))
=2D    (values (sort
=2D             (map (lambda (input)
=2D                    (let ((guix-name (python->package-name input)))
=2D                      (list guix-name (list 'unquote (string->symbol gui=
x-name)))))
=2D                  dependencies)
=2D             (lambda args
=2D               (match args
=2D                 (((a _ ...) (b _ ...))
=2D                  (string-ci<? a b)))))
=2D            dependencies)))
+  "Given the SOURCE-URL and WHEEL-URL of an already downloaded ARCHIVE, re=
turn
+a pair of lists, each consisting of a list of name/variable pairs, for the
+propagated inputs and the native inputs, respectively."
+
+  (define (strip-argparse deps)
+    (remove (cut string=3D? "argparse" <>) deps))
+
+  (define (requirement->package-name/sort deps)
+    (sort
+     (map (lambda (input)
+            (let ((guix-name (python->package-name input)))
+              (list guix-name (list 'unquote (string->symbol guix-name)))))
+          deps)
+     (lambda args
+       (match args
+         (((a _ ...) (b _ ...))
+          (string-ci<? a b))))))
+
+  (define process-requirements
+    (compose requirement->package-name/sort strip-argparse))
+
+  (map process-requirements (guess-requirements source-url wheel-url archi=
ve)))
=20
 (define (make-pypi-sexp name version source-url wheel-url home-page synops=
is
                         description license)
@@ -317,15 +371,13 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION=
, and LICENSE."
   (call-with-temporary-output-file
    (lambda (temp port)
      (and (url-fetch source-url temp)
=2D          (receive (input-package-names upstream-dependency-names)
=2D              (compute-inputs source-url wheel-url temp)
=2D            (values
+          (match (compute-inputs source-url wheel-url temp)
+            ((required-inputs test-inputs)
              `(package
                 (name ,(python->package-name name))
                 (version ,version)
                 (source (origin
                           (method url-fetch)
=2D
                           ;; Sometimes 'pypi-uri' doesn't quite work due t=
o mixed
                           ;; cases in NAME, for instance, as is the case w=
ith
                           ;; "uwsgi".  In that case, fall back to a full U=
RL.
@@ -334,12 +386,12 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION=
, and LICENSE."
                            (base32
                             ,(guix-hash-url temp)))))
                 (build-system python-build-system)
=2D                ,@(maybe-inputs input-package-names)
+                ,@(maybe-inputs required-inputs 'propagated-inputs)
+                ,@(maybe-inputs test-inputs 'native-inputs)
                 (home-page ,home-page)
                 (synopsis ,synopsis)
                 (description ,description)
=2D                (license ,(license->symbol license)))
=2D             upstream-dependency-names))))))
+                (license ,(license->symbol license)))))))))
=20
 (define pypi->guix-package
   (memoize
diff --git a/tests/pypi.scm b/tests/pypi.scm
index ca8cb5f6de..aa08e2cb54 100644
=2D-- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright =C2=A9 2014 David Thompson <davet@HIDDEN>
 ;;; Copyright =C2=A9 2016 Ricardo Wurmus <rekado@HIDDEN>
+;;; Copyright =C2=A9 2019 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -65,11 +66,6 @@
 sha1=3Dda9234ee9982d4bbb3c72346a6de940a148ea686"))
=20
 (define test-requires.txt "\
=2Dbar
=2Dbaz > 13.37
=2D")
=2D
=2D(define test-requires-with-sections "\
 # A comment
 foo ~=3D 3
 bar !=3D 2
@@ -78,12 +74,25 @@ bar !=3D 2
 pytest (>=3D2.5.0)
 ")
=20
+;; Beaker contains only optional dependencies.
+(define test-requires.txt-beaker "\
+[crypto]
+pycryptopp>=3D0.5.12
+
+[cryptography]
+cryptography
+
+[testsuite]
+Mock
+coverage
+")
+
 (define test-metadata "\
 Classifier: Programming Language :: Python :: 3.7
 Requires-Dist: baz ~=3D 3
 Requires-Dist: bar !=3D 2
 Provides-Extra: test
=2Dpytest (>=3D2.5.0)
+Requires-Dist: pytest (>=3D2.5.0) ; extra =3D=3D 'test'
 ")
=20
 (define test-metadata-with-extras "
@@ -137,25 +146,31 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'test=
ing'
   '("Fizzy" "PickyThing" "SomethingWithMarker" "requests" "pip")
   (map specification->requirement-name test-specifications))
=20
=2D(test-equal "parse-requires.txt, with sections"
=2D  '("foo" "bar")
+(test-equal "parse-requires.txt"
+  (list '("foo" "bar") '("pytest"))
   (mock ((ice-9 ports) call-with-input-file
          call-with-input-string)
=2D        (parse-requires.txt test-requires-with-sections)))
+        (parse-requires.txt test-requires.txt)))
+
+(test-equal "parse-requires.txt - Beaker"
+  (list '() '("Mock" "coverage"))
+  (mock ((ice-9 ports) call-with-input-file
+         call-with-input-string)
+        (parse-requires.txt test-requires.txt-beaker)))
=20
 (test-equal "parse-wheel-metadata, with extras"
=2D  '("wrapt" "bar")
+  (list '("wrapt" "bar") '("tox" "bumpversion"))
   (mock ((ice-9 ports) call-with-input-file
          call-with-input-string)
         (parse-wheel-metadata test-metadata-with-extras)))
=20
 (test-equal "parse-wheel-metadata, with extras - Jedi"
=2D  '("parso")
+  (list '("parso") '("pytest"))
   (mock ((ice-9 ports) call-with-input-file
          call-with-input-string)
         (parse-wheel-metadata test-metadata-with-extras-jedi)))
=20
=2D(test-assert "pypi->guix-package"
+(test-assert "pypi->guix-package, no wheel"
   ;; Replace network resources with sample data.
     (mock ((guix import utils) url-fetch
            (lambda (url file-name)
@@ -195,7 +210,10 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testi=
ng'
                      ('propagated-inputs
                       ('quasiquote
                        (("python-bar" ('unquote 'python-bar))
=2D                        ("python-baz" ('unquote 'python-baz)))))
+                        ("python-foo" ('unquote 'python-foo)))))
+                     ('native-inputs
+                      ('quasiquote
+                       (("python-pytest" ('unquote 'python-pytest)))))
                      ('home-page "http://example.com")
                      ('synopsis "summary")
                      ('description "summary")
@@ -216,25 +234,25 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'test=
ing'
               (begin
                 (mkdir-p "foo-1.0.0/foo.egg-info/")
                 (with-output-to-file "foo-1.0.0/foo.egg-info/requires.txt"
=2D                   (lambda ()
=2D                     (display "wrong data to make sure we're testing whe=
els ")))
+                  (lambda ()
+                    (display "wrong data to make sure we're testing wheels=
 ")))
                 (parameterize ((current-output-port (%make-void-port "rw+"=
)))
                   (system* "tar" "czvf" file-name "foo-1.0.0/"))
=2D                 (delete-file-recursively "foo-1.0.0")
=2D                 (set! test-source-hash
=2D                       (call-with-input-file file-name port-sha256))))
+                (delete-file-recursively "foo-1.0.0")
+                (set! test-source-hash
+                  (call-with-input-file file-name port-sha256))))
              ("https://example.com/foo-1.0.0-py2.py3-none-any.whl"
=2D               (begin
=2D                 (mkdir "foo-1.0.0.dist-info")
=2D                 (with-output-to-file "foo-1.0.0.dist-info/METADATA"
=2D                   (lambda ()
=2D                     (display test-metadata)))
=2D                 (let ((zip-file (string-append file-name ".zip")))
=2D                   ;; zip always adds a "zip" extension to the file it c=
reates,
=2D                   ;; so we need to rename it.
=2D                   (system* "zip" zip-file "foo-1.0.0.dist-info/METADATA=
")
=2D                   (rename-file zip-file file-name))
=2D                 (delete-file-recursively "foo-1.0.0.dist-info")))
+              (begin
+                (mkdir "foo-1.0.0.dist-info")
+                (with-output-to-file "foo-1.0.0.dist-info/METADATA"
+                  (lambda ()
+                    (display test-metadata)))
+                (let ((zip-file (string-append file-name ".zip")))
+                  ;; zip always adds a "zip" extension to the file it crea=
tes,
+                  ;; so we need to rename it.
+                  (system* "zip" "-q" zip-file "foo-1.0.0.dist-info/METADA=
TA")
+                  (rename-file zip-file file-name))
+                (delete-file-recursively "foo-1.0.0.dist-info")))
              (_ (error "Unexpected URL: " url)))))
         (mock ((guix http-client) http-fetch
                (lambda (url . rest)
@@ -262,6 +280,9 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testin=
g'
                     ('quasiquote
                      (("python-bar" ('unquote 'python-bar))
                       ("python-baz" ('unquote 'python-baz)))))
+                   ('native-inputs
+                    ('quasiquote
+                     (("python-pytest" ('unquote 'python-pytest)))))
                    ('home-page "http://example.com")
                    ('synopsis "summary")
                    ('description "summary")
@@ -272,4 +293,48 @@ Requires-Dist: pytest (>=3D3.1.0); extra =3D=3D 'testi=
ng'
                 (x
                  (pk 'fail x #f))))))
=20
+(test-assert "pypi->guix-package, no usable requirement file."
+  ;; Replace network resources with sample data.
+  (mock ((guix import utils) url-fetch
+         (lambda (url file-name)
+           (match url
+             ("https://example.com/foo-1.0.0.tar.gz"
+              (set! test-source-hash
+                (call-with-input-file file-name port-sha256))
+              #t)
+             ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #t)
+             (_ (error "Unexpected URL: " url)))))
+        (mock ((guix http-client) http-fetch
+               (lambda (url . rest)
+                 (match url
+                   ("https://pypi.org/pypi/foo/json"
+                    (values (open-input-string test-json)
+                            (string-length test-json)))
+                   ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #=
f)
+                   (_ (error "Unexpected URL: " url)))))
+              ;; Not clearing the memoization cache here would mean return=
ing the value
+              ;; computed in the previous test.
+              (invalidate-memoization! pypi->guix-package)
+              (match (pypi->guix-package "foo")
+                (('package
+                   ('name "python-foo")
+                   ('version "1.0.0")
+                   ('source ('origin
+                              ('method 'url-fetch)
+                              ('uri ('pypi-uri "foo" 'version))
+                              ('sha256
+                               ('base32
+                                (? string? hash)))))
+                   ('build-system 'python-build-system)
+                   ('home-page "http://example.com")
+                   ('synopsis "summary")
+                   ('description "summary")
+                   ('license 'license:lgpl2.0))
+                 (string=3D? (bytevector->nix-base32-string
+                            test-source-hash)
+                           hash))
+                (x
+                 (pk 'fail x #f)))
+              )))
+
 (test-end "pypi")
=2D-=20
2.20.1


--=-=-=
Content-Type: text/plain


Thanks,

Maxim

--=-=-=--

--==-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEJ9WGpPiQCFQyn/CfEmDkZILmNWIFAlydoGMACgkQEmDkZILm
NWJMOA/+Nk3rtALS/r5Su8aNXIZfC5z0p5Zk3WBc/FSOkZ0e+56qdiiLLQ6bEBrT
xEmr23LqIiHfwU2H3H+mpZCKWDmqBb0edLW0u6cKRRO0eU5T55pEA71CyQj8c+mu
UHNyFGm6y/hmaMKPVsyP8p3rhNl8TIPJe2vFXnN4LWaCJaj3E0X6Ge2fsnNTgzXk
zpe5eU1vAFNfYOaf7uc658cfIPFzTEWgBfHASsfuKTLE34ZNB9HOxwskTNd1CvlN
USjs3kSFRGQl2q0tXQzrxt9eNwQzcXWMk61bCYtDrZ4d87dhENKd2EAivMgGLGq/
0poA4ijhb/zfmK7RNtHItivipzaZ+xAOpIEhcfwlmz2WxIZfV4eKkpLEB1q6QGAQ
0OKUmzAPHdYvNcHkhhAZIZl195JZ11HlrpTX5gUMqjL11d4ReuhlqNlAnynt8+tJ
8WgjVemJ7dhn8oZVLcXELBdFH01DZnbvF/8kheAyO6/dhTzKbxua+mPbNTpMn0w4
Ipe94BiF5DlaDffvWfmTebR9j+x7j8WSQeezc0GiNQuV8wOXKdVGsm+B66MdCpOH
ioQ8B/62Tu/9ul4L+hkaZCBrVM/JI8W+SAeAn+CMQ0Zye20POpwn1xXK5SCn1FRU
wXoFrEGE1L7yG+B406fRiABr711AItzTO9MvZQQmEquZBCN0qqI=
=2rRk
-----END PGP SIGNATURE-----
--==-=-=--




Information forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.

Message received at 24450 <at> debbugs.gnu.org:


Received: (at 24450) by debbugs.gnu.org; 29 Mar 2019 04:24:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 29 00:24:35 2019
Received: from localhost ([127.0.0.1]:34960 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1h9j4E-0005zH-K0
	for submit <at> debbugs.gnu.org; Fri, 29 Mar 2019 00:24:34 -0400
Received: from mail-io1-f50.google.com ([209.85.166.50]:40671)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1h9j4C-0005z2-4O
 for 24450 <at> debbugs.gnu.org; Fri, 29 Mar 2019 00:24:33 -0400
Received: by mail-io1-f50.google.com with SMTP id d201so692341iof.7
 for <24450 <at> debbugs.gnu.org>; Thu, 28 Mar 2019 21:24:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version:content-transfer-encoding;
 bh=z5dv62qPuuOkTnSSRikJYv2S4fhzUYtvKYib860yK9I=;
 b=HHi+ve/cJWBCv+TsssBw3lCtzDph0H/VF8X2KNlZWpmHUO5Y0e4CSEbCV7RzgWxOLT
 7OpRliS4l26ZGETMPNOIwcPyr4tXK6LKkyfZBpGoDEvOV7iz+w6KOPP2Rzd4Pv44WAnN
 aqECFwTUAM6g+MUXL36ezBb0WrY0/pT100RIj1nUfUaSd/kCt33w2VrGPT57rtD4AhlW
 SdptxmtVdnUESmpxGKhBw/YHR6gyY7i2xYMUHoMZXToQTZFiOsO2Flbon4mHrHZlatuF
 Xfv1Y83BlL2n9xAlKjo6xbrjG3gxTQlL67sTU2GI9tpVzLHRFEE3UkMKs6UWO9fnMO+E
 cQtg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version:content-transfer-encoding;
 bh=z5dv62qPuuOkTnSSRikJYv2S4fhzUYtvKYib860yK9I=;
 b=dpFX7YC38cZKu9GuBZN+ohflmP8jX7pWLUkOoQoNdcIdVbn8rZ8bEJCr1il/v0JKcP
 f0tnbN0gkZLmMUsN9qG7QOWc+5Tdieucw5WTi4/al3kB82A6uXM7NLYdwPHV+O6NM021
 RsJIdcuEIj2n+HC2Cl+vG8s4orHklZTVgBz+wnLjdKqhEjMSqf157P5XhPpFBPJFrgWM
 o0PHzEpeoMH45BKb6wVlVLsceYf3U260ZNy3KKqJTwl0TC4VPcBDzUSDS0bZrW8zg56k
 zY2HixYXq5T622e+fBlSTUVFmFaRp2GxgmiI5qDHOsPTyYefj80qViQiEl5K+xtJegtZ
 q6hA==
X-Gm-Message-State: APjAAAU9yRS7ytp6IRfTQmi5UOEAqGJr6QCqo3aGs5W/zvydJrNHbmBk
 RKGXDOmoiQFJ5qGPAKitTn9L1MjsoK8=
X-Google-Smtp-Source: APXvYqxfoaUVGgYoDv+0BlMedX4k+TjozDDJdIpAqyBl6/RwJmQHlY80QQEYKkp2juOXiTAfwY7JwA==
X-Received: by 2002:a5e:c019:: with SMTP id u25mr32624713iol.104.1553833466372; 
 Thu, 28 Mar 2019 21:24:26 -0700 (PDT)
Received: from kwak ([2607:f2c0:94b4:fa00::235])
 by smtp.gmail.com with ESMTPSA id q1sm530329itb.22.2019.03.28.21.24.25
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Thu, 28 Mar 2019 21:24:25 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: ng0 <ng0@HIDDEN>
Subject: Re: bug#24450: pypi importer outputs strange character series in
 optional dependency case.
References: <87h99fipj1.fsf@HIDDEN>
Date: Fri, 29 Mar 2019 00:24:24 -0400
In-Reply-To: <87h99fipj1.fsf@HIDDEN> (ng0's message of "Fri, 16
 Sep 2016 20:00:02 +0000")
Message-ID: <87imw22tqf.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 24450
Cc: 24450 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

ng0 <ng0@HIDDEN> writes:

> I think this should not happen with pypi import:
>
>   (inputs
>     `(("python-certifi=3D=3D2016.2.28"
>        ,python-certifi=3D=3D2016.2.28)
>       ("python-dateutil=3D=3D2.5.3"
>        ,python-dateutil=3D=3D2.5.3)
>       ("python-flask-babel=3D=3D0.11.1"
>        ,python-flask-babel=3D=3D0.11.1)
>       ("python-flask=3D=3D0.11.1" ,python-flask=3D=3D0.11.1)
>       ("python-lxml=3D=3D3.6.0" ,python-lxml=3D=3D3.6.0)
>       ("python-ndg-httpsclient=3D=3D0.4.1"
>        ,python-ndg-httpsclient=3D=3D0.4.1)
>       ("python-pyasn1-modules=3D=3D0.0.8"
>        ,python-pyasn1-modules=3D=3D0.0.8)
>       ("python-pyasn1=3D=3D0.1.9" ,python-pyasn1=3D=3D0.1.9)
>       ("python-pygments=3D=3D2.1.3"
>        ,python-pygments=3D=3D2.1.3)
>       ("python-pyopenssl=3D=3D0.15.1"
>        ,python-pyopenssl=3D=3D0.15.1)
>       ("python-pyyaml=3D=3D3.11" ,python-pyyaml=3D=3D3.11)
>       ("python-requests[socks]=3D=3D2.10.0"
>        ,#{python-requests\x5b;socks\x5d;=3D=3D2.10.0}#)
>       ("python-setuptools" ,python-setuptools)))
>
>
> I can understand the version numbers, I can also understand the optional
> socks building/module of the python-requests, but why does it read like
> Gobbledygook?  Can't we improve the output here?
>
> For version numbers, this is not a format which happened recently which
> is exclusive for python build system right? This is just bad formated
> because of the pypi query.
> I will first try and not pin the application to these version numbers,
> maybe itjustworks=E2=84=A2.
>
>
> To reproduce: "guix import pypi searx"

This would now give (change to be sent for review soon):

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix import pypi searx

Starting download of /tmp/guix-file.1wD8K4
From https://files.pythonhosted.org/packages/75/3f/5941ad2d500ff7cf6f8da102=
2c78013dcd2207941d533586a8e7bfe699d3/searx-0.15.0.tar.gz...
 =E2=80=A65.0.tar.gz  1.6MiB                  729KiB/s 00:02 [#############=
#####] 100.0%
(package
  (name "python-searx")
  (version "0.15.0")
  (source
    (origin
      (method url-fetch)
      (uri (pypi-uri "searx" version))
      (sha256
        (base32
          "1gmww73q7wydkvlyz73wnr3sybpjn40wha7avnz9ak9m365zcjxf"))))
  (build-system python-build-system)
  (propagated-inputs
    `(("python-certifi" ,python-certifi)
      ("python-dateutil" ,python-dateutil)
      ("python-flask" ,python-flask)
      ("python-flask-babel" ,python-flask-babel)
      ("python-idna" ,python-idna)
      ("python-lxml" ,python-lxml)
      ("python-pygments" ,python-pygments)
      ("python-pyopenssl" ,python-pyopenssl)
      ("python-pyyaml" ,python-pyyaml)
      ("python-requests" ,python-requests)))
  (native-inputs
    `(("python-babel" ,python-babel)
      ("python-cov-core" ,python-cov-core)
      ("python-mock" ,python-mock)
      ("python-nose2" ,python-nose2)
      ("python-pep8" ,python-pep8)
      ("python-plone.testing" ,python-plone.testing)
      ("python-selenium" ,python-selenium)
      ("python-splinter" ,python-splinter)
      ("python-transifex-client"
       ,python-transifex-client)
      ("python-unittest2" ,python-unittest2)
      ("python-zope.testrunner"
       ,python-zope.testrunner)))
  (home-page "https://github.com/asciimoo/searx")
  (synopsis
    "A privacy-respecting, hackable metasearch engine")
  (description
    "A privacy-respecting, hackable metasearch engine")
  (license #f))
--8<---------------cut here---------------end--------------->8---




Information forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.

Message received at 24450 <at> debbugs.gnu.org:


Received: (at 24450) by debbugs.gnu.org; 29 Mar 2019 04:23:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 29 00:23:14 2019
Received: from localhost ([127.0.0.1]:34953 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1h9j2w-0005wa-7h
	for submit <at> debbugs.gnu.org; Fri, 29 Mar 2019 00:23:14 -0400
Received: from mail-it1-f170.google.com ([209.85.166.170]:54156)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>)
 id 1h9j2t-0005wC-OM; Fri, 29 Mar 2019 00:23:12 -0400
Received: by mail-it1-f170.google.com with SMTP id y204so1798109itf.3;
 Thu, 28 Mar 2019 21:23:11 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version:content-transfer-encoding;
 bh=HQ7rZmIcfvilLf6um1+ebAf+xDRVm3uc8x0K915DRFs=;
 b=dBcUGhrRXYCND9EBrDhIOyxvooS7fp09ZBoqAS6vt52BYmCB8V+0YVI0awK3aZ6mnU
 3NMdWgfO7jj0c24UI/BK+gi7wckk5IicwEsGh80nROyOEC4lg2zYW9VBwE1HpoNVICCk
 lRcNVNOLl+7V7jayLflZJV1IJag0bvA+H3CuHJ9uYli4SDaAXS8jP7Fdkqj6Zt0EyF8f
 vJHzhPx3hnQRNL0ojsekAKrnEML1PUbjygScSIF27NbwB2ge8d/QqQGuyguNHeekjGLv
 r8rQ88UeCe2exaky780nlFKaElijGfP0PwVomKjiFX1CDweC4n4Amidi1g/IiFczKIp2
 VDpA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version:content-transfer-encoding;
 bh=HQ7rZmIcfvilLf6um1+ebAf+xDRVm3uc8x0K915DRFs=;
 b=ImR4Oa6q+Gq/OTpp/AhMtHcKwRPWk5ZLTjGU/Ng2X/oCEnR36qqply4GEidFz8D1gK
 dj5vDdJtGozxSOstigHIhjamRVMf9T3zL79oWnqC/RK5OaxXlmZcFAN44hmevX3+0pux
 M6G6UJPjmeUq3yvwjG2lg1HwWgOu4O+0VT6z7L1g9aHA0bfUgFwQL7PC0gw8Y9z0AlHL
 IWkwJXOQhJKsoj4d+j9WqHaw2uBPp+Bxu+qyIVJ/Nlidlb7/KAVfF/GFIz4sbOyW4aFs
 7vOUYQLYWcYdqSHU1NFmG1ExNCAloEQSboGTt+KcN2ymp6qD8BFmFcbs53n2I4TzLQ38
 7fEQ==
X-Gm-Message-State: APjAAAXhKxKofr6C4buCSX+6on/Rdjntz3bO77sh9FWXsi3hjQwWX38R
 2H4GDA73ur7x4cDwKTFBUuDaVr00jnU=
X-Google-Smtp-Source: APXvYqyx0e/wpvQf8iB163o1WLhS6ScRuaYaNkb/yAUHAu9GZWUESdu8bN0l2N3e9xjcxlTrOycBag==
X-Received: by 2002:a24:a85:: with SMTP id 127mr2649853itw.40.1553833386007;
 Thu, 28 Mar 2019 21:23:06 -0700 (PDT)
Received: from kwak ([2607:f2c0:94b4:fa00::235])
 by smtp.gmail.com with ESMTPSA id d133sm607815ita.5.2019.03.28.21.23.04
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Thu, 28 Mar 2019 21:23:05 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Julien Lepiller <julien@HIDDEN>
Subject: Re: bug#33047: pypi importer uses incorrect package names
References: <e5965fe7462e7825cfe61169762a3562@HIDDEN>
Date: Fri, 29 Mar 2019 00:23:03 -0400
In-Reply-To: <e5965fe7462e7825cfe61169762a3562@HIDDEN> (Julien Lepiller's
 message of "Mon, 15 Oct 2018 15:43:02 +0200")
Message-ID: <87mule2tso.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 24450
Cc: 33047 <at> debbugs.gnu.org, 24450 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Julien Lepiller <julien@HIDDEN> writes:

> Hi,
>
> I found that sometimes the pypi importer had trouble importing
> packages correctly. For instance, running "guix import pypi txaio"
> gave me this list of dependencies:
>
> (propagated-inputs
>  `(("python-[all]" ,#{python-\x5b;all\x5d;}#)
>    ("python-[asyncio]"
>     ,#{python-\x5b;asyncio\x5d;}#)
>    ...))
>
> guix import pypi magic-wormhole had this:
>
> (propagated-inputs
>   ("python-autobahn[twisted]"
>    ,#{python-autobahn\x5b;twisted\x5d;}#)
>   ...))
>
> Of course, they break the recursive importer, which makes it difficult
> to import these packages correctly.

Testing local branch:

./pre-inst-env guix import pypi txaio

Starting download of /tmp/guix-file.jTNBQz
From https://files.pythonhosted.org/packages/c1/99/81de004578e9afe017bb1d4c=
8968088a33621c05449fe330bdd7016d5377/txaio-18.8.1.tar.gz...
 =E2=80=A68.1.tar.gz  50KiB                   894KiB/s 00:00 [#############=
#####] 100.0%

Starting download of /tmp/guix-file.ZB3Q2n
From https://files.pythonhosted.org/packages/e9/6d/e1a6f7835cde86728e5bb1f5=
77be9b2d7d273fdb33c286e70b087d418ded/txaio-18.8.1-py2.py3-none-any.whl...
 =E2=80=A6.py3-none-any.whl  27KiB            746KiB/s 00:00 [#############=
#####] 100.0%
(package
  (name "python-txaio")
  (version "18.8.1")
  (source
    (origin
      (method url-fetch)
      (uri (pypi-uri "txaio" version))
      (sha256
        (base32
          "1zmpdph6zddgrnkkcykh6qk5s46l7s5mzfqrh82m4b5iffn61qv7"))))
  (build-system python-build-system)
  (propagated-inputs `(("python-six" ,python-six)))
  (native-inputs
    `(("python-mock" ,python-mock)
      ("python-pep8" ,python-pep8)
      ("python-pyenchant" ,python-pyenchant)
      ("python-pytest" ,python-pytest)
      ("python-pytest-cov" ,python-pytest-cov)
      ("python-sphinx" ,python-sphinx)
      ("python-sphinx-rtd-theme"
       ,python-sphinx-rtd-theme)
      ("python-sphinxcontrib-spelling"
       ,python-sphinxcontrib-spelling)
      ("python-tox" ,python-tox)
      ("python-twine" ,python-twine)
      ("python-wheel" ,python-wheel)))
  (home-page "https://github.com/crossbario/txaio")
  (synopsis
    "Compatibility API between asyncio/Twisted/Trollius")
  (description
    "Compatibility API between asyncio/Twisted/Trollius")
  (license #f))

and

./pre-inst-env guix import pypi magic-wormhole

Starting download of /tmp/guix-file.80RRqk
From https://files.pythonhosted.org/packages/77/15/9438290bab8146efc0213f7c=
3d9645d9bc5a2e885e4049477e7432e40336/magic-wormhole-0.11.2.tar.gz...
 =E2=80=A6-0.11.2.tar.gz  193KiB              911KiB/s 00:00 [#############=
#####] 100.0%

Starting download of /tmp/guix-file.mRGAx3
From https://files.pythonhosted.org/packages/82/98/3e8d12fdb90457e8f3e1f5b8=
77ee27f5db58dbaf4a4fbe95f7287a568401/magic_wormhole-0.11.2-py2.py3-none-any=
.whl...
 =E2=80=A6-py2.py3-none-any.whl  128KiB      1009KiB/s 00:00 [#############=
#####] 100.0%
(package
  (name "python-magic-wormhole")
  (version "0.11.2")
  (source
    (origin
      (method url-fetch)
      (uri (pypi-uri "magic-wormhole" version))
      (sha256
        (base32
          "01fr4bi6kc6fz9n3c4qq892inrc3nf6p2djy65yvm7xkvdxncydf"))))
  (build-system python-build-system)
  (propagated-inputs
    `(("python-attrs" ,python-attrs)
      ("python-autobahn" ,python-autobahn)
      ("python-automat" ,python-automat)
      ("python-click" ,python-click)
      ("python-hkdf" ,python-hkdf)
      ("python-humanize" ,python-humanize)
      ("python-pynacl" ,python-pynacl)
      ("python-pywin32" ,python-pywin32)
      ("python-six" ,python-six)
      ("python-spake2" ,python-spake2)
      ("python-tqdm" ,python-tqdm)
      ("python-twisted" ,python-twisted)
      ("python-txtorcon" ,python-txtorcon)))
  (native-inputs
    `(("python-magic-wormhole-mailbox-server"
       ,python-magic-wormhole-mailbox-server)
      ("python-magic-wormhole-transit-relay"
       ,python-magic-wormhole-transit-relay)
      ("python-mock" ,python-mock)
      ("python-pyflakes" ,python-pyflakes)
      ("python-tox" ,python-tox)))
  (home-page
    "https://github.com/warner/magic-wormhole")
  (synopsis
    "Securely transfer data between computers")
  (description
    "Securely transfer data between computers")
  (license license:expat))
=20=20




Information forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.

Message received at 24450 <at> debbugs.gnu.org:


Received: (at 24450) by debbugs.gnu.org; 29 Mar 2019 04:21:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 29 00:21:03 2019
Received: from localhost ([127.0.0.1]:34946 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1h9j0p-0005rV-Bq
	for submit <at> debbugs.gnu.org; Fri, 29 Mar 2019 00:21:03 -0400
Received: from mail-io1-f42.google.com ([209.85.166.42]:43104)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>)
 id 1h9j0n-0005qP-1Z; Fri, 29 Mar 2019 00:21:01 -0400
Received: by mail-io1-f42.google.com with SMTP id x3so672898iol.10;
 Thu, 28 Mar 2019 21:21:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version:content-transfer-encoding;
 bh=/45T5Zrchgzc5rzndzC+fgq5GVCdzqYhDsakWW1Wc28=;
 b=MrdGrKUMLsoHOZdPGmuzbEPmNG7V31QvFjGOSreEkmevg8DJzb02VGmr4LGRjHWza5
 DfJfm7pUtv2x3g1BnZ4Q8uM4cINRZwJ+QCuEQxAIGdLb5MZLzPN636cuh78p+2pk8qH6
 A87Eqp4FOgVCk+A8gdXmHhoCfnl/b3zlmXcg3CIHEGgVEj+bKd5WIsMKwADuVek0w5DK
 X15m0GUOXDWXxO/XY0f9bmnfw5qEsndcRnQMCYScvcWzKAqitiK2/88bij2wyox1JQC/
 9rCk/r5bkC67zVOoRR6javwki6Ib5/N4JUug3VwrMWAiniEqqGbp/NOcGhnRcvtqdaQE
 xFSw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version:content-transfer-encoding;
 bh=/45T5Zrchgzc5rzndzC+fgq5GVCdzqYhDsakWW1Wc28=;
 b=cAWFNliUC7uLY1BuswlQYMdsaTaBmb7Gb4+L+SjjbBXGERVMwl5O3ekNnDdm4N1inN
 M5cqpCNZPmHFqaaj9dQ8H+YOCfOeOoEV5M7RLpm4ckn3zdRRwEk32Jy/D3OK1EkkGu98
 LEmRauScd63RpbCAdIR8V3bRjPN1i2n0U2moCJj42LjdoC2zb1ZDYEuhV7FSzqRVE5bY
 YPujT08prSRcnwSqUMl9yu8cVSfPsl77N0OXfIU2qULF9Ns+w4JdHJJ3HJUtFai9r1jz
 SEZcq9tHU8vBL71HSSOqS/xHTgN+i2FBNf8Tjz3LglK9xxb1AvNqBjcdmXmRwRlNPDB7
 FbKA==
X-Gm-Message-State: APjAAAVH5aeMkOL4+ONWBVrGIeJ21c2fEv+sFGXQCqSX0EfgK274pwdG
 QXRsRZ0AjT+Ps4/hnWczNsESKHlqH4s=
X-Google-Smtp-Source: APXvYqxAhhNaho89QTiivGtM6wMd9ALgdcJjS5y8YSJ1t4gV4OtB3/ewJB/bFeKTDeAFLNzocwHxsg==
X-Received: by 2002:a6b:7804:: with SMTP id j4mr12887382iom.171.1553833254992; 
 Thu, 28 Mar 2019 21:20:54 -0700 (PDT)
Received: from kwak ([2607:f2c0:94b4:fa00::235])
 by smtp.gmail.com with ESMTPSA id t68sm531752ita.4.2019.03.28.21.20.53
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Thu, 28 Mar 2019 21:20:54 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: swedebugia <swedebugia@HIDDEN>
Subject: Re: bug#33569: Missing sanitizing of '[]' in pypi-importer
References: <a9fc3247-e04c-5d63-8e90-8ecc761e6bed@HIDDEN>
Date: Fri, 29 Mar 2019 00:20:53 -0400
In-Reply-To: <a9fc3247-e04c-5d63-8e90-8ecc761e6bed@HIDDEN> (swedebugia's
 message of "Sun, 2 Dec 2018 01:32:02 +0100")
Message-ID: <87r2aq2twa.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 24450
Cc: 33569 <at> debbugs.gnu.org, 24450 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

swedebugia <swedebugia@HIDDEN> writes:

> E.g.
> sdb@komputilo ~/guix-tree$ ~/guix-tree/pre-inst-env guix import pypi
> snakemake
> ...
>   (propagated-inputs
>     `(("python-[reports]"
>        ,#{python-\x5b;reports\x5d;}#)
>       ("python-appdirs" ,python-appdirs)
> ...

This one now gives (local branch):

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix import pypi snakemake

Starting download of /tmp/guix-file.4XvWMX
From https://files.pythonhosted.org/packages/4a/aa/aab1515d220be06fbdccf3c8=
9335d9585b08ac6be74b8e3c9e8c3c32798e/snakemake-5.4.4.tar.gz...
 =E2=80=A6.4.4.tar.gz  169KiB                 723KiB/s 00:00 [#############=
#####] 100.0%
(package
  (name "python-snakemake")
  (version "5.4.4")
  (source
    (origin
      (method url-fetch)
      (uri (pypi-uri "snakemake" version))
      (sha256
        (base32
          "0prpr5qajqwr8sh4gzggpj8l4np2rcm9nfdzvcp30d5yw7h26wqm"))))
  (build-system python-build-system)
  (propagated-inputs
    `(("python-appdirs" ,python-appdirs)
      ("python-configargparse" ,python-configargparse)
      ("python-datrie" ,python-datrie)
      ("python-docutils" ,python-docutils)
      ("python-gitpython" ,python-gitpython)
      ("python-jsonschema" ,python-jsonschema)
      ("python-pyyaml" ,python-pyyaml)
      ("python-ratelimiter" ,python-ratelimiter)
      ("python-requests" ,python-requests)
      ("python-wrapt" ,python-wrapt)))
  (home-page "http://snakemake.bitbucket.io")
  (synopsis
    "Snakemake is a workflow management system that aims to reduce the comp=
lexity of creating workflows by providing a fast and comfortable execution =
environment, together with a clean and modern specification language in pyt=
hon style. Snakemake workflows are essentially Python scripts extended by d=
eclarative code to define rules. Rules describe how to create output files =
from input files.")
  (description
    "Snakemake is a workflow management system that aims to reduce the comp=
lexity of creating workflows by providing a fast and comfortable execution =
environment, together with a clean and modern specification language in pyt=
hon style. Snakemake workflows are essentially Python scripts extended by d=
eclarative code to define rules. Rules describe how to create output files =
from input files.")
  (license license:expat))
--8<---------------cut here---------------end--------------->8---




Information forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.

Message received at 24450 <at> debbugs.gnu.org:


Received: (at 24450) by debbugs.gnu.org; 29 Mar 2019 04:19:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Mar 29 00:19:22 2019
Received: from localhost ([127.0.0.1]:34939 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1h9izC-0005n9-98
	for submit <at> debbugs.gnu.org; Fri, 29 Mar 2019 00:19:22 -0400
Received: from mail-it1-f174.google.com ([209.85.166.174]:40504)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>)
 id 1h9izA-0005mq-E2; Fri, 29 Mar 2019 00:19:21 -0400
Received: by mail-it1-f174.google.com with SMTP id y63so1812056itb.5;
 Thu, 28 Mar 2019 21:19:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version:content-transfer-encoding;
 bh=H8z7PvZNCOiPhlqPYauL8U8reG8tvLXY+vlMB5BaQ9o=;
 b=uONJ/uZ++grtBT8Dhoj7GirM4ogB5yWePEz+hbhHkYmdyi3GXYTHAF9HCzhCc94TDk
 pgxzNj4gvsj21zesmZg2sTYjEYbzVbyXalxmy+IBwERiYm1RmW6EKPkxwLIB4TlHzKMQ
 mzN/w5x6UM+4KaVQJwF/3PH6Z3G00ULjV4psp73bMRkUZaIFsom85bojzzW6IefU1Crs
 TVpA6qIDg4bC6DzkTfjHhf7of/d9iQ78PMpUtw5VysAKlkUEsyI2Sm1dL2qgOebgDrz3
 RkEWZwgQMXqkPRRz+9WmF0/jdaiiEZAsLeV9XikKoGtN2enxeXRWMvC3aPFLrsJrT48I
 zIbw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version:content-transfer-encoding;
 bh=H8z7PvZNCOiPhlqPYauL8U8reG8tvLXY+vlMB5BaQ9o=;
 b=beeKPkHzMKUO+SSRenmcNTrXoz7nVK44tfWeA9eDI0L09cLGzbbEoL78+ubSxTT0GJ
 XMry6rElOgl/8t7HD0n7941wCKYN6vTpMzA2fVkgTkr81TDdZBH1vPuY6kBs3Z4I8wiw
 XmdsxLK+1COSUK5JKfh2Zdm1XdbpvWEDx0C2330V/eFMVgN7CIknmUo7UaTOZm2cpLx7
 avUAGfLiMEYiQwxYdRSuZ4pLMzijQNeJjB5SyqwSZUyX4Xn1TjT5fX1tgw/LIiHnYFOR
 y51eSstUvmayTrr/QGoa6rtV5QkauodIefXM5bTprrobD7FtmPYqrWzDLPyJe1/1ngFd
 Jb7A==
X-Gm-Message-State: APjAAAU/VTx/8dROHKKRVCvFPZG1qNjt4tqP+8EtwQBNyr3OITxThTEJ
 FOooXujGaig0DOJc+Jx8oL2gC9U4kQE=
X-Google-Smtp-Source: APXvYqyRT0loXREb4DoefVGxnVLypjgEU6JcCW3XbuFTviqLhdcZLMF3AZcyWADMRdtsL7F32LFDWA==
X-Received: by 2002:a02:8c4d:: with SMTP id j13mr35138401jal.48.1553833154577; 
 Thu, 28 Mar 2019 21:19:14 -0700 (PDT)
Received: from kwak ([2607:f2c0:94b4:fa00::235])
 by smtp.gmail.com with ESMTPSA id r184sm9696440ita.3.2019.03.28.21.19.13
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Thu, 28 Mar 2019 21:19:13 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: swedebugia <swedebugia@HIDDEN>
Subject: Re: bug#34266: pypi importer cannot handle [ and ] correctly
References: <b67ec137-b642-2e1c-7648-702ae7a2cf4f@HIDDEN>
Date: Fri, 29 Mar 2019 00:19:12 -0400
In-Reply-To: <b67ec137-b642-2e1c-7648-702ae7a2cf4f@HIDDEN> (swedebugia's
 message of "Thu, 31 Jan 2019 19:25:12 +0100")
Message-ID: <87va022tz3.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 24450
Cc: 34266 <at> debbugs.gnu.org, 24450 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

swedebugia <swedebugia@HIDDEN> writes:

> $ ./pre-inst-env guix import pypi beaker
>
> following redirection to `https://pypi.org/pypi/Beaker/json'...
>
> Starting download of /tmp/guix-file.p15GJZ
> From
> https://files.pythonhosted.org/packages/c2/21/b052b2fbfee3def06670923d5d3=
4b0d353d4c278013e4a714c3fb663f150/Beaker-1.10.0.tar.gz...
>  ...0.0.tar.gz  40KiB                 521KiB/s 00:00
> [##################] 100.0%
> (package
>   (name "python-beaker")
>   (version "1.10.0")
>   (source
>     (origin
>       (method url-fetch)
>       (uri (pypi-uri "beaker" version))
>       (sha256
>         (base32
>           "0l047yl3n9b3w7ba0wrqdb5fpww5y8pjy20kah2mlpr230lqjwk0"))))
>   (build-system python-build-system)
>   (propagated-inputs
>     `(("python-[crypto]" ,#{python-\x5b;crypto\x5d;}#)
>       ("python-[cryptography]"
>        ,#{python-\x5b;cryptography\x5d;}#)
>       ("python-[pycrypto]"
>        ,#{python-\x5b;pycrypto\x5d;}#)
>       ("python-[pycryptodome]"
>        ,#{python-\x5b;pycryptodome\x5d;}#)
>       ("python-[testsuite]"
>        ,#{python-\x5b;testsuite\x5d;}#)
>       ("python-coverage" ,python-coverage)
>       ("python-cryptography" ,python-cryptography)
>       ("python-cryptography" ,python-cryptography)
>       ("python-funcsigs" ,python-funcsigs)
>       ("python-memcached" ,python-memcached)
>       ("python-mock" ,python-mock)
>       ("python-nose" ,python-nose)
>       ("python-pycrypto" ,python-pycrypto)
>       ("python-pycryptodome" ,python-pycryptodome)
>       ("python-pycryptodome" ,python-pycryptodome)
>       ("python-pycryptopp" ,python-pycryptopp)
>       ("python-pylibmc" ,python-pylibmc)
>       ("python-pymongo" ,python-pymongo)
>       ("python-redis" ,python-redis)
>       ("python-sqlalchemy" ,python-sqlalchemy)
>       ("python-webtest" ,python-webtest)))
>   (home-page "https://beaker.readthedocs.io/")
>   (synopsis
>     "A Session and Caching library with WSGI Middleware")
>   (description
>     "A Session and Caching library with WSGI Middleware")
>   (license license:bsd-3))

Testing with my soon-to-be sent for review changes:

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix import pypi beaker
following redirection to `https://pypi.org/pypi/Beaker/json'...

Starting download of /tmp/guix-file.0MWu4B
From https://files.pythonhosted.org/packages/76/87/ecc1a222f0caaa7ba7b89287=
37e89b2e91b8c22450c12b8a51ee625a4d87/Beaker-1.10.1.tar.gz...
 =E2=80=A60.1.tar.gz  40KiB                   487KiB/s 00:00 [#############=
#####] 100.0%
(package
  (name "python-beaker")
  (version "1.10.1")
  (source
    (origin
      (method url-fetch)
      (uri (pypi-uri "beaker" version))
      (sha256
        (base32
          "16zdjfl8v73yl1capph0n371vd26c7zpzb48n505ip32ffgmvc4f"))))
  (build-system python-build-system)
  (native-inputs
    `(("python-coverage" ,python-coverage)
      ("python-cryptography" ,python-cryptography)
      ("python-memcached" ,python-memcached)
      ("python-mock" ,python-mock)
      ("python-nose" ,python-nose)
      ("python-pycryptodome" ,python-pycryptodome)
      ("python-pylibmc" ,python-pylibmc)
      ("python-pymongo" ,python-pymongo)
      ("python-redis" ,python-redis)
      ("python-sqlalchemy" ,python-sqlalchemy)
      ("python-webtest" ,python-webtest)))
  (home-page "https://beaker.readthedocs.io/")
  (synopsis
    "A Session and Caching library with WSGI Middleware")
  (description
    "A Session and Caching library with WSGI Middleware")
  (license license:bsd-3))
--8<---------------cut here---------------end--------------->8---

Looking better?

Maxim




Information forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.
Forcibly Merged 24450 33047 33569 34266. Request was from Maxim Cournoyer <maxim.cournoyer@HIDDEN> to control <at> debbugs.gnu.org. Full text available.
Merged 24450 33047 33569 34266. Request was from T460s laptop <maxim.cournoyer@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 16 Sep 2016 20:01:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 16 16:01:03 2016
Received: from localhost ([127.0.0.1]:33566 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1bkzJm-0007I5-TD
	for submit <at> debbugs.gnu.org; Fri, 16 Sep 2016 16:01:03 -0400
Received: from eggs.gnu.org ([208.118.235.92]:52955)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ng0@HIDDEN>) id 1bkzJl-0007HU-NF
 for submit <at> debbugs.gnu.org; Fri, 16 Sep 2016 16:01:02 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ng0@HIDDEN>) id 1bkzJf-0004Pm-PU
 for submit <at> debbugs.gnu.org; Fri, 16 Sep 2016 16:00:56 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:37803)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <ng0@HIDDEN>) id 1bkzJf-0004PT-Hv
 for submit <at> debbugs.gnu.org; Fri, 16 Sep 2016 16:00:55 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:43906)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <ng0@HIDDEN>) id 1bkzJd-0002Nn-E4
 for bug-guix@HIDDEN; Fri, 16 Sep 2016 16:00:54 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ng0@HIDDEN>) id 1bkzJY-0004Ng-6J
 for bug-guix@HIDDEN; Fri, 16 Sep 2016 16:00:53 -0400
Received: from aibo.runbox.com ([91.220.196.211]:50329)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <ng0@HIDDEN>) id 1bkzJX-0004NM-VS
 for bug-guix@HIDDEN; Fri, 16 Sep 2016 16:00:48 -0400
Received: from [10.9.9.210] (helo=mailfront10.runbox.com)
 by bars.runbox.com with esmtp (Exim 4.71)
 (envelope-from <ng0@HIDDEN>) id 1bkzJV-0001uH-2M
 for bug-guix@HIDDEN; Fri, 16 Sep 2016 22:00:45 +0200
Received: from xd9bb9d06.dyn.telefonica.de ([217.187.157.6] helo=localhost)
 by mailfront10.runbox.com with esmtpsa (uid:892961 )
 (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) id 1bkzIp-0002xQ-KP
 for bug-guix@HIDDEN; Fri, 16 Sep 2016 22:00:03 +0200
From: ng0 <ng0@HIDDEN>
To: bug-guix@HIDDEN
Subject: pypi importer outputs strange character series in optional dependency
 case.
Date: Fri, 16 Sep 2016 20:00:02 +0000
Message-ID: <87h99fipj1.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

I think this should not happen with pypi import:

  (inputs
    `(("python-certifi==2016.2.28"
       ,python-certifi==2016.2.28)
      ("python-dateutil==2.5.3"
       ,python-dateutil==2.5.3)
      ("python-flask-babel==0.11.1"
       ,python-flask-babel==0.11.1)
      ("python-flask==0.11.1" ,python-flask==0.11.1)
      ("python-lxml==3.6.0" ,python-lxml==3.6.0)
      ("python-ndg-httpsclient==0.4.1"
       ,python-ndg-httpsclient==0.4.1)
      ("python-pyasn1-modules==0.0.8"
       ,python-pyasn1-modules==0.0.8)
      ("python-pyasn1==0.1.9" ,python-pyasn1==0.1.9)
      ("python-pygments==2.1.3"
       ,python-pygments==2.1.3)
      ("python-pyopenssl==0.15.1"
       ,python-pyopenssl==0.15.1)
      ("python-pyyaml==3.11" ,python-pyyaml==3.11)
      ("python-requests[socks]==2.10.0"
       ,#{python-requests\x5b;socks\x5d;==2.10.0}#)
      ("python-setuptools" ,python-setuptools)))


I can understand the version numbers, I can also understand the optional
socks building/module of the python-requests, but why does it read like
Gobbledygook?  Can't we improve the output here?

For version numbers, this is not a format which happened recently which
is exclusive for python build system right? This is just bad formated
because of the pypi query.
I will first try and not pin the application to these version numbers,
maybe itjustworks™.


To reproduce: "guix import pypi searx"
-- 
              ng0




Acknowledgement sent to ng0 <ng0@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#24450; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 31 Mar 2019 14:45:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.