GNU bug report logs - #36194
[PATCH 00/10] Add csvkit.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36194 in the body.
You can then email your comments to 36194 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: guix-patches <at> gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 00/10] Add csvkit.
Date: Thu, 13 Jun 2019 16:01:26 +0100

Hello Guix!

Here are patches to add csvkit [0] and its dependencies to guix. It provides
various useful commands to deal with CSV data.

A lot of the packages come from the same https://github.com/wireservice
account and their documentation is built the same way so I've added them in a
new 'wireservice.scm' file, so they could inherit from the same package
definition.

What do you think?

Thanks,
Pierre

[0]: https://csvkit.readthedocs.io/en/latest

Pierre Langlois (10):
  gnu: Add python-slugify.
  gnu: Add python-pytimeparse.
  gnu: Add python-crate.
  gnu: Add python-dbfread.
  gnu: Add python-leather.
  gnu: Add python-agate.
  gnu: Add python-agate-sql.
  gnu: Add python-agate-dbf.
  gnu: Add python-agate-excel.
  gnu: Add csvkit.

 gnu/local.mk                                  |   3 +
 gnu/packages/databases.scm                    |  25 ++
 gnu/packages/patches/csvkit-fix-tests.patch   |  45 +++
 .../python-slugify-depend-on-unidecode.patch  |  22 ++
 gnu/packages/python-web.scm                   |  31 ++
 gnu/packages/python-xyz.scm                   |  22 ++
 gnu/packages/time.scm                         |  21 ++
 gnu/packages/wireservice.scm                  | 266 ++++++++++++++++++
 8 files changed, 435 insertions(+)
 create mode 100644 gnu/packages/patches/csvkit-fix-tests.patch
 create mode 100644 gnu/packages/patches/python-slugify-depend-on-unidecode.patch
 create mode 100644 gnu/packages/wireservice.scm

--
2.22.0

Message #8 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 03/10] gnu: Add python-crate.
Date: Thu, 13 Jun 2019 16:06:21 +0100

* gnu/packages/databases.scm (python-crate): New variable.
---
 gnu/packages/databases.scm | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 09a262d139..54cec24d8d 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -34,6 +34,7 @@
 ;;; Copyright © 2018 Joshua Sierles, Nextjournal <joshua <at> nextjournal.com>
 ;;; Copyright © 2018 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
 ;;; Copyright © 2019 Jack Hill <jackhill <at> jackhill.us>
+;;; Copyright © 2019 Pierre Langlois <pierre.langlois <at> gmx.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -90,6 +91,7 @@
   #:use-module (gnu packages popt)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-crypto)
+  #:use-module (gnu packages python-web)
   #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages rdf)
   #:use-module (gnu packages readline)
@@ -3082,3 +3084,26 @@ NumPy, and other traditional Python scientific computing packages.")

 (define-public python2-pyarrow
   (package-with-python2 python-pyarrow))
+
+(define-public python-crate
+  (package
+    (name "python-crate")
+    (version "0.23.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "crate" version))
+              (sha256
+               (base32
+                "0s3s7yg4m2zflg9q96aibwb5hizsn10ql63fsj6h5z624qkavnlp"))))
+    (build-system python-build-system)
+    (propagated-inputs
+     `(("python-urllib3" ,python-urllib3)))
+    (home-page "https://github.com/crate/crate-python")
+    (synopsis "CrateDB Python client")
+    (description
+      "A Python client library for CrateDB.
+@itemize
+@item Implements the Python DB API 2.0 specification
+@item Includes support for SQLAlchemy (>= 1.0.0)
+@end itemize")
+    (license license:asl2.0)))
--
2.22.0

Message #11 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 01/10] gnu: Add python-slugify.
Date: Thu, 13 Jun 2019 16:06:19 +0100

* gnu/packages/python-web.scm (python-slugify): New variable.
* gnu/packages/patches/python-slugify-depend-on-unidecode.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
---
 gnu/local.mk                                  |  1 +
 .../python-slugify-depend-on-unidecode.patch  | 22 +++++++++++++
 gnu/packages/python-web.scm                   | 31 +++++++++++++++++++
 3 files changed, 54 insertions(+)
 create mode 100644 gnu/packages/patches/python-slugify-depend-on-unidecode.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index c7b557ca61..f8c33dd8c2 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1214,6 +1214,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/python2-pygobject-2-gi-info-type-error-domain.patch \
   %D%/packages/patches/python-pygpgme-fix-pinentry-tests.patch	\
   %D%/packages/patches/python-robotframework-honor-source-date-epoch.patch \
+  %D%/packages/patches/python-slugify-depend-on-unidecode.patch	\
   %D%/packages/patches/python2-subprocess32-disable-input-test.patch	\
   %D%/packages/patches/python-unittest2-python3-compat.patch	\
   %D%/packages/patches/python-unittest2-remove-argparse.patch	\
diff --git a/gnu/packages/patches/python-slugify-depend-on-unidecode.patch b/gnu/packages/patches/python-slugify-depend-on-unidecode.patch
new file mode 100644
index 0000000000..93a3aaebe9
--- /dev/null
+++ b/gnu/packages/patches/python-slugify-depend-on-unidecode.patch
@@ -0,0 +1,22 @@
+diff --git a/setup.py b/setup.py
+index 4800173..6bdd77f 100755
+--- a/setup.py
++++ b/setup.py
+@@ -14,8 +14,7 @@ url = 'https://github.com/un33k/python-slugify'
+ author = 'Val Neekman'
+ author_email = 'info <at> neekware.com'
+ license = 'MIT'
+-install_requires = ['text-unidecode==1.2']
+-extras_require = {'unidecode': ['Unidecode==1.0.23']}
++install_requires = ['Unidecode==1.0.23']
+
+ classifiers = [
+     'Development Status :: 5 - Production/Stable',
+@@ -67,7 +66,6 @@ setup(
+     author_email=author_email,
+     packages=find_packages(exclude=EXCLUDE_FROM_PACKAGES),
+     install_requires=install_requires,
+-    extras_require=extras_require,
+     classifiers=classifiers,
+     entry_points={'console_scripts': ['slugify=slugify.slugify:main']},
+ )
diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 042484c6a1..a5cc2cc326 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -29,6 +29,7 @@
 ;;; Copyright © 2018 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
 ;;; Copyright © 2019 Vagrant Cascadian <vagrant <at> debian.org>
 ;;; Copyright © 2019 Brendan Tildesley <mail <at> brendan.scot>
+;;; Copyright © 2019 Pierre Langlois <pierre.langlois <at> gmx.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -3156,3 +3157,33 @@ Python.")
     (propagated-inputs
      `(("python-gevent" ,python2-gevent)
        ("python-tornado" ,python2-tornado)))))
+
+(define-public python-slugify
+  (package
+    (name "python-slugify")
+    (version "3.0.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "python-slugify" version))
+       (sha256
+        (base32
+         "0n6pfmsq899c54plpvzi46l7zrpa3zfpm8im6h32czjw6kxky5jp"))
+       (patches
+        (search-patches "python-slugify-depend-on-unidecode.patch"))))
+    (native-inputs
+     `(("python-wheel" ,python-wheel)))
+    (propagated-inputs
+     `(("python-unidecode" ,python-unidecode)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'check
+           (lambda _
+             (invoke "python" "test.py"))))))
+    (build-system python-build-system)
+    (home-page "https://github.com/un33k/python-slugify")
+    (synopsis "Python Slugify application that handles Unicode")
+    (description "This package provides a @command{slufigy} command and
+library to create slugs from unicode strings while keeping it DRY.")
+    (license license:expat)))
--
2.22.0

Message #14 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 09/10] gnu: Add python-agate-excel.
Date: Thu, 13 Jun 2019 16:06:27 +0100

* gnu/packages/wireservice.scm (python-agate-excel): New variable.
---
 gnu/packages/wireservice.scm | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/gnu/packages/wireservice.scm b/gnu/packages/wireservice.scm
index 4a8e7681ed..78c2a4264d 100644
--- a/gnu/packages/wireservice.scm
+++ b/gnu/packages/wireservice.scm
@@ -168,3 +168,26 @@ support to all @code{agate.Table} instances.")))
     (synopsis "Add read support for dbf files to agate")
     (description "@code{agatedbf} uses a monkey patching pattern to add read
 for dbf files support to all @code{agate.Table} instances.")))
+
+(define-public python-agate-excel
+  (package
+    (inherit base-package)
+    (name "python-agate-excel")
+    (version "0.2.3")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/wireservice/agate-excel.git")
+                    (commit version)))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "1k5lv21k19s7kgbj5srd1xgrkqvxqqs49qwj33zncs9l7851afy7"))))
+    (propagated-inputs
+     `(("python-agate" ,python-agate)
+       ("python-openpyxl" ,python-openpyxl)
+       ("python-xlrd" ,python-xlrd)))
+    (home-page "https://agate-excel.rtfd.org")
+    (synopsis "Add read support for Excel files (xls and xlsx) to agate")
+    (description "@code{agateexcel} uses a monkey patching pattern to add read
+for xls and xlsx files support to all @code{agate.Table} instances.")))
--
2.22.0

Message #17 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 02/10] gnu: Add python-pytimeparse.
Date: Thu, 13 Jun 2019 16:06:20 +0100

* gnu/packages/time.scm (python-pytimeparse): New variable.
---
 gnu/packages/time.scm | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/gnu/packages/time.scm b/gnu/packages/time.scm
index 747e4cf080..f944c74652 100644
--- a/gnu/packages/time.scm
+++ b/gnu/packages/time.scm
@@ -15,6 +15,7 @@
 ;;; Copyright © 2017 Julien Lepiller <julien <at> lepiller.eu>
 ;;; Copyright © 2018 Alex Vong <alexvong1995 <at> gmail.com>
 ;;; Copyright © 2019 Kyle Meyer <kyle <at> kyleam.com>
+;;; Copyright © 2019 Pierre Langlois <pierre.langlois <at> gmx.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -65,6 +66,26 @@ program uses.  The display output of the program can be customized or saved
 to a file.")
     (license gpl3+)))

+(define-public python-pytimeparse
+  (package
+    (name "python-pytimeparse")
+    (version "1.1.8")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "pytimeparse" version))
+       (sha256
+        (base32
+         "02kaambsgpjx3zi42j6l11rwms2p35b9hsk4f3kdf979gd3kcqg8"))))
+    (native-inputs
+     `(("python-nose" ,python-nose)))
+    (build-system python-build-system)
+    (home-page "https://github.com/wroberts/pytimeparse")
+    (synopsis "Time expression parser")
+    (description "Small Python module to parse various kinds of time
+expressions.")
+    (license expat)))
+
 (define-public python-pytzdata
   (package
     (name "python-pytzdata")
--
2.22.0

Message #20 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 06/10] gnu: Add python-agate.
Date: Thu, 13 Jun 2019 16:06:24 +0100

* gnu/packages/wireservice.scm (python-agate): New variable.
---
 gnu/packages/wireservice.scm | 35 ++++++++++++++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/wireservice.scm b/gnu/packages/wireservice.scm
index 290b027e7b..db17c55cd8 100644
--- a/gnu/packages/wireservice.scm
+++ b/gnu/packages/wireservice.scm
@@ -25,7 +25,8 @@
   #:use-module (gnu packages check)
   #:use-module (gnu packages python-web)
   #:use-module (gnu packages python-xyz)
-  #:use-module (gnu packages sphinx))
+  #:use-module (gnu packages sphinx)
+  #:use-module (gnu packages time))

 ;; Base package definition for packages from https://github.com/wireservice.
 ;; This is done so we can share how to run tests and build documentation.
@@ -89,3 +90,35 @@
     (synopsis "Python charting for 80% of humans")
     (description "Leather is a Python charting library for those who need
 charts now and don't care if they're perfect.")))
+
+(define-public python-agate
+  (package
+    (inherit base-package)
+    (name "python-agate")
+    (version "1.6.1")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/wireservice/agate.git")
+                    (commit version)))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "077zj8xad8hsa3nqywvf7ircirmx3krxdipl8wr3dynv3l3khcpl"))))
+    (native-inputs
+     `(("python-cssselect" ,python-cssselect)
+       ("python-lxml" ,python-lxml)
+       ,@(package-native-inputs base-package)))
+    (propagated-inputs
+     `(("python-babel" ,python-babel)
+       ("python-isodate" ,python-isodate)
+       ("python-leather" ,python-leather)
+       ("python-parsedatetime" ,python-parsedatetime)
+       ("python-pytimeparse" ,python-pytimeparse)
+       ("python-six" ,python-six)
+       ("python-slugify" ,python-slugify)))
+    (home-page "https://agate.rtfd.org")
+    (synopsis "Data analysis library")
+    (description "Agate is a Python data analysis library.  It is an
+alternative to numpy and pandas that solves real-world problems with readable
+code.  Agate was previously known as journalism.")))
--
2.22.0

Message #23 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 07/10] gnu: Add python-agate-sql.
Date: Thu, 13 Jun 2019 16:06:25 +0100

* gnu/packages/wireservice.scm (python-agate-sql): New variable.
---
 gnu/packages/wireservice.scm | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/gnu/packages/wireservice.scm b/gnu/packages/wireservice.scm
index db17c55cd8..575b63792b 100644
--- a/gnu/packages/wireservice.scm
+++ b/gnu/packages/wireservice.scm
@@ -23,6 +23,7 @@
   #:use-module (guix packages)
   #:use-module (gnu packages)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages databases)
   #:use-module (gnu packages python-web)
   #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages sphinx)
@@ -122,3 +123,26 @@ charts now and don't care if they're perfect.")))
     (description "Agate is a Python data analysis library.  It is an
 alternative to numpy and pandas that solves real-world problems with readable
 code.  Agate was previously known as journalism.")))
+
+(define-public python-agate-sql
+  (package
+    (inherit base-package)
+    (name "python-agate-sql")
+    (version "0.5.4")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/wireservice/agate-sql.git")
+                    (commit version)))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "16q0b211n5b1qmhzkfl2jr56lda0rvyh5j1wzw26h2n4pm4wxlx2"))))
+    (propagated-inputs
+     `(("python-agate" ,python-agate)
+       ("python-crate" ,python-crate)
+       ("python-sqlalchemy" ,python-sqlalchemy)))
+    (home-page "https://agate-sql.rtfd.org")
+    (synopsis "SQL read/write support to agate")
+    (description "@code{agatesql} uses a monkey patching pattern to add SQL
+support to all @code{agate.Table} instances.")))
--
2.22.0

Message #26 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 08/10] gnu: Add python-agate-dbf.
Date: Thu, 13 Jun 2019 16:06:26 +0100

* gnu/packages/wireservice.scm (python-agate-dbf): New variable.
---
 gnu/packages/wireservice.scm | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/gnu/packages/wireservice.scm b/gnu/packages/wireservice.scm
index 575b63792b..4a8e7681ed 100644
--- a/gnu/packages/wireservice.scm
+++ b/gnu/packages/wireservice.scm
@@ -146,3 +146,25 @@ code.  Agate was previously known as journalism.")))
     (synopsis "SQL read/write support to agate")
     (description "@code{agatesql} uses a monkey patching pattern to add SQL
 support to all @code{agate.Table} instances.")))
+
+(define-public python-agate-dbf
+  (package
+    (inherit base-package)
+    (name "python-agate-dbf")
+    (version "0.2.1")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/wireservice/agate-dbf.git")
+                    (commit version)))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "1y49fi6pmm7gzhajvqmfpcca2sqnwj24fqnsvzwk7r1hg2iaa2gi"))))
+    (propagated-inputs
+     `(("python-agate" ,python-agate)
+       ("python-dbfread" ,python-dbfread)))
+    (home-page "https://agate-dbf.rtfd.org")
+    (synopsis "Add read support for dbf files to agate")
+    (description "@code{agatedbf} uses a monkey patching pattern to add read
+for dbf files support to all @code{agate.Table} instances.")))
--
2.22.0

Message #29 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 10/10] gnu: Add csvkit.
Date: Thu, 13 Jun 2019 16:07:21 +0100

* gnu/packages/wireservice.scm (csvkit): New variable.
* gnu/packages/patches/csvkit-fix-tests.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
---
 gnu/local.mk                                |  1 +
 gnu/packages/patches/csvkit-fix-tests.patch | 45 +++++++++++++
 gnu/packages/wireservice.scm                | 73 +++++++++++++++++++++
 3 files changed, 119 insertions(+)
 create mode 100644 gnu/packages/patches/csvkit-fix-tests.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index e9e5009276..b49592b5d5 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -734,6 +734,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/cpufrequtils-fix-aclocal.patch		\
   %D%/packages/patches/crawl-upgrade-saves.patch		\
   %D%/packages/patches/crda-optional-gcrypt.patch		\
+  %D%/packages/patches/csvkit-fix-tests.patch			\
   %D%/packages/patches/clucene-contribs-lib.patch               \
   %D%/packages/patches/cube-nocheck.patch			\
   %D%/packages/patches/cursynth-wave-rand.patch			\
diff --git a/gnu/packages/patches/csvkit-fix-tests.patch b/gnu/packages/patches/csvkit-fix-tests.patch
new file mode 100644
index 0000000000..e62b601fe4
--- /dev/null
+++ b/gnu/packages/patches/csvkit-fix-tests.patch
@@ -0,0 +1,45 @@
+diff --git a/tests/test_utilities/test_csvsql.py b/tests/test_utilities/test_csvsql.py
+index e6ec4af..4f47980 100644
+--- a/tests/test_utilities/test_csvsql.py
++++ b/tests/test_utilities/test_csvsql.py
+@@ -197,7 +197,7 @@ class TestCSVSQL(CSVKitTestCase, EmptyFileTests):
+         utility.run()
+         output = output_file.getvalue()
+         output_file.close()
+-        self.assertEqual(output, 'a,b,c\n1,2,3\n0,5,6\n')
++        self.assertEqual(output, 'a,b,c\n1,2.0,3.0\n0,5.0,6.0\n')
+
+     def test_no_prefix_unique_constraint(self):
+         self.get_output(['--db', 'sqlite:///' + self.db_file, '--insert', 'examples/dummy.csv', '--unique-constraint', 'a'])
+diff --git a/tests/test_utilities/test_sql2csv.py b/tests/test_utilities/test_sql2csv.py
+index a0c3d3e..babcfd6 100644
+--- a/tests/test_utilities/test_sql2csv.py
++++ b/tests/test_utilities/test_sql2csv.py
+@@ -121,23 +121,23 @@ class TestSQL2CSV(CSVKitTestCase, EmptyFileTests):
+         input_file.close()
+
+     def test_unicode(self):
+-        expected = self.csvsql('examples/test_utf8.csv')
++        self.csvsql('examples/test_utf8.csv')
+         csv = self.get_output(['--db', 'sqlite:///' + self.db_file, '--query', 'select * from foo'])
+-        self.assertEqual(csv.strip(), expected)
++        self.assertEqual(csv.strip(), 'foo,bar,baz\n1.0,2.0,3\n4.0,5.0,ʤ')
+
+     def test_no_header_row(self):
+         self.csvsql('examples/dummy.csv')
+         csv = self.get_output(['--db', 'sqlite:///' + self.db_file, '--no-header-row', '--query', 'select * from foo'])
+
+         self.assertTrue('a,b,c' not in csv)
+-        self.assertTrue('1,2,3' in csv)
++        self.assertTrue('1,2.0,3.0' in csv)
+
+     def test_linenumbers(self):
+         self.csvsql('examples/dummy.csv')
+         csv = self.get_output(['--db', 'sqlite:///' + self.db_file, '--linenumbers', '--query', 'select * from foo'])
+
+         self.assertTrue('line_number,a,b,c' in csv)
+-        self.assertTrue('1,1,2,3' in csv)
++        self.assertTrue('1,1,2.0,3.0' in csv)
+
+     def test_wildcard_on_sqlite(self):
+         self.csvsql('examples/iris.csv')
diff --git a/gnu/packages/wireservice.scm b/gnu/packages/wireservice.scm
index 78c2a4264d..ab86f21f2f 100644
--- a/gnu/packages/wireservice.scm
+++ b/gnu/packages/wireservice.scm
@@ -19,6 +19,7 @@
 (define-module (gnu packages wireservice)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix build-system python)
+  #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (guix packages)
   #:use-module (gnu packages)
@@ -191,3 +192,75 @@ for dbf files support to all @code{agate.Table} instances.")))
     (synopsis "Add read support for Excel files (xls and xlsx) to agate")
     (description "@code{agateexcel} uses a monkey patching pattern to add read
 for xls and xlsx files support to all @code{agate.Table} instances.")))
+
+(define-public csvkit
+  (package
+    (name "csvkit")
+    (version "1.0.4")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "csvkit" version))
+              (sha256
+               (base32
+                "1830lb95rh1iyi3drlwxzb6y3pqkii0qiyzd40c1kvhvaf1s6lqk"))
+              (patches (search-patches "csvkit-fix-tests.patch"))))
+    (build-system python-build-system)
+    (native-inputs
+     `(("python-psycopg2" ,python-psycopg2) ;; Used to test PostgreSQL support.
+       ("python-sphinx" ,python-sphinx)
+       ("python-sphinx-rtd-theme" ,python-sphinx-rtd-theme)))
+    (inputs
+     `(("python-agate-dbf" ,python-agate-dbf)
+       ("python-agate-excel" ,python-agate-excel)
+       ("python-agate-sql" ,python-agate-sql)
+       ("python-six" ,python-six)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-after 'install 'install-docs
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out  (assoc-ref outputs "out"))
+                    (man1 (string-append out "/share/man/man1")))
+               (with-directory-excursion "docs"
+                 (invoke "make" "man")
+                 (copy-recursively "_build/man" man1))
+               #t))))))
+    (home-page "https://csvkit.rtfd.org")
+    (synopsis "Command-line tools for working with CSV")
+    (description "csvkit is a suite of command-line tools for converting to
+and working with CSV.  It provides the following commands:
+@itemize
+@item Input:
+  @itemize
+  @item @command{in2csv}: Convert various formats to CSV.
+  @item @command{sql2csv}: Execute SQL commands on a database and return the
+data as CSV.
+  @end itemize
+@item Processing:
+  @itemize
+  @item @command{csvclean}: Remove common syntax errors.
+  @item @command{csvcut}: Filter and truncate CSV files.
+  @item @command{csvgrep}: Filter tabular data to only those rows where
+certain columns contain a given value or match a regular expression.
+  @item @command{csvjoin}: Merges two or more CSV tables together using a
+method analogous to SQL JOIN operation.
+  @item @command{csvsort}: Sort CSV files.
+  @item @command{csvstack}: Stack up the rows from multiple CSV files,
+optionally adding a grouping value to each row.
+  @end itemize
+@item Output and analysis:
+  @itemize
+  @item @command{csvformat}: Convert a CSV file to a custom output format.
+  @item @command{csvjson}: Converts a CSV file into JSON or GeoJSON.
+  @item @command{csvlook}: Renders a CSV to the command line in a
+Markdown-compatible, fixed-width format.
+  @item @command{csvpy}: Loads a CSV file into a @code{agate.csv.Reader}
+object and then drops into a Python shell so the user can inspect the data
+however they see fit.
+  @item @command{csvsql}: Generate SQL statements for a CSV file or execute
+those statements directly on a database.
+  @item @command{csvstat}: Prints descriptive statistics for all columns in a
+CSV file.
+  @end itemize
+@end itemize")
+    (license license:expat)))
--
2.22.0

Message #32 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 05/10] gnu: Add python-leather.
Date: Thu, 13 Jun 2019 16:06:23 +0100

* gnu/packages/wireservice.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
 gnu/local.mk                 |  1 +
 gnu/packages/wireservice.scm | 91 ++++++++++++++++++++++++++++++++++++
 2 files changed, 92 insertions(+)
 create mode 100644 gnu/packages/wireservice.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index f8c33dd8c2..e9e5009276 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -485,6 +485,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/packages/wget.scm				\
   %D%/packages/wicd.scm				\
   %D%/packages/wine.scm				\
+  %D%/packages/wireservice.scm				\
   %D%/packages/wm.scm				\
   %D%/packages/wordnet.scm			\
   %D%/packages/wv.scm				\
diff --git a/gnu/packages/wireservice.scm b/gnu/packages/wireservice.scm
new file mode 100644
index 0000000000..290b027e7b
--- /dev/null
+++ b/gnu/packages/wireservice.scm
@@ -0,0 +1,91 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2019 Pierre Langlois <pierre.langlois <at> gmx.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages wireservice)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix build-system python)
+  #:use-module (guix git-download)
+  #:use-module (guix packages)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages check)
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages python-xyz)
+  #:use-module (gnu packages sphinx))
+
+;; Base package definition for packages from https://github.com/wireservice.
+;; This is done so we can share how to run tests and build documentation.
+(define base-package
+  (package
+    (name #f)
+    (version #f)
+    (source #f)
+    (home-page #f)
+    (synopsis #f)
+    (description #f)
+    (build-system python-build-system)
+    (native-inputs
+     `(("python-nose" ,python-nose)
+       ("python-sphinx" ,python-sphinx)
+       ("python-sphinx-rtd-theme" ,python-sphinx-rtd-theme)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'check
+           (lambda _
+             (invoke "nosetests" "tests")))
+         (add-after 'install 'install-docs
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (doc (string-append out "/share/doc/"
+                                        ,(package-name this-package)
+                                        "-"
+                                        ,(package-version this-package))))
+               (with-directory-excursion "docs"
+                 (for-each
+                  (lambda (target)
+                    (invoke "make" target)
+                    (copy-recursively (string-append "_build/" target)
+                                      (string-append doc "/" target)))
+                  '("html" "dirhtml" "singlehtml" "text")))
+               #t))))))
+    (license license:expat)))
+
+(define-public python-leather
+  (package
+    (inherit base-package)
+    (name "python-leather")
+    (version "0.3.3")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/wireservice/leather.git")
+                    (commit version)))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "1ck3dplni99sv4s117cbm07ydwwjsrxkhdy19rnk0iglia1d4s5i"))))
+    (native-inputs
+     `(("python-cssselect" ,python-cssselect)
+       ("python-lxml" ,python-lxml)
+       ,@(package-native-inputs base-package)))
+    (propagated-inputs
+     `(("python-six" ,python-six)))
+    (home-page "https://leather.rtfd.org")
+    (synopsis "Python charting for 80% of humans")
+    (description "Leather is a Python charting library for those who need
+charts now and don't care if they're perfect.")))
--
2.22.0

Message #35 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: [PATCH 04/10] gnu: Add python-dbfread.
Date: Thu, 13 Jun 2019 16:06:22 +0100

* gnu/packages/python-xyz.scm (python-dbfread): New variable.
---
 gnu/packages/python-xyz.scm | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index d33571637d..f38f767bcc 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -60,6 +60,7 @@
 ;;; Copyright © 2019 Brett Gilio <brettg <at> posteo.net>
 ;;; Copyright © 2019 Sam <smbaines8 <at> gmail.com>
 ;;; Copyright © 2019 Jack Hill <jackhill <at> jackhill.us>
+;;; Copyright © 2019 Pierre Langlois <pierre.langlois <at> gmx.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -15396,3 +15397,24 @@ by Igor Pavlov.")

 (define-public python2-pylzma
   (package-with-python2 python-pylzma))
+
+(define-public python-dbfread
+  (package
+    (name "python-dbfread")
+    (version "2.0.7")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "dbfread" version))
+              (sha256
+               (base32
+                "0gdpwdzf1fngsi6jrdyj4qdf6cr7gnnr3zp80dpkzbgz0spskj07"))))
+    (build-system python-build-system)
+    (native-inputs
+     `(("python-pytest" ,python-pytest)))
+    (home-page "https://dbfread.readthedocs.io")
+    (synopsis "Read DBF Files with Python")
+    (description
+     "This library reads DBF files and returns the data as native Python data
+types for further processing.  It is primarily intended for batch jobs and
+one-off scripts.")
+    (license license:expat)))
--
2.22.0

Message #38 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: Re: [PATCH 01/10 v2] gnu: Add python-slugify.
Date: Fri, 21 Jun 2019 11:42:23 +0100

[Message part 1 (text/plain, inline)]

Pierre Langlois writes:

> * gnu/packages/python-web.scm (python-slugify): New variable.
> * gnu/packages/patches/python-slugify-depend-on-unidecode.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Add it.
> ---
>  gnu/local.mk                                  |  1 +
>  .../python-slugify-depend-on-unidecode.patch  | 22 +++++++++++++
>  gnu/packages/python-web.scm                   | 31 +++++++++++++++++++
>  3 files changed, 54 insertions(+)
>  create mode 100644 gnu/packages/patches/python-slugify-depend-on-unidecode.patch
>
> diff --git a/gnu/local.mk b/gnu/local.mk
> index c7b557ca61..f8c33dd8c2 100644
> --- a/gnu/local.mk
> +++ b/gnu/local.mk
> @@ -1214,6 +1214,7 @@ dist_patch_DATA =						\
>    %D%/packages/patches/python2-pygobject-2-gi-info-type-error-domain.patch \
>    %D%/packages/patches/python-pygpgme-fix-pinentry-tests.patch	\
>    %D%/packages/patches/python-robotframework-honor-source-date-epoch.patch \
> +  %D%/packages/patches/python-slugify-depend-on-unidecode.patch	\
>    %D%/packages/patches/python2-subprocess32-disable-input-test.patch	\
>    %D%/packages/patches/python-unittest2-python3-compat.patch	\
>    %D%/packages/patches/python-unittest2-remove-argparse.patch	\
> diff --git a/gnu/packages/patches/python-slugify-depend-on-unidecode.patch b/gnu/packages/patches/python-slugify-depend-on-unidecode.patch
> new file mode 100644
> index 0000000000..93a3aaebe9
> --- /dev/null
> +++ b/gnu/packages/patches/python-slugify-depend-on-unidecode.patch
> @@ -0,0 +1,22 @@
> +diff --git a/setup.py b/setup.py
> +index 4800173..6bdd77f 100755
> +--- a/setup.py
> ++++ b/setup.py
> +@@ -14,8 +14,7 @@ url = 'https://github.com/un33k/python-slugify'
> + author = 'Val Neekman'
> + author_email = 'info <at> neekware.com'
> + license = 'MIT'
> +-install_requires = ['text-unidecode==1.2']
> +-extras_require = {'unidecode': ['Unidecode==1.0.23']}
> ++install_requires = ['Unidecode==1.0.23']

Ah, the python-unidecode package was just updated to version 1.1.0 which
then breaks csvkit.

Here's a fixed patch that ignores the version number, the tests pass so
I think we're good still.

Thanks!
Pierre

[0001-gnu-Add-python-slugify.patch (text/x-patch, attachment)]

Message #41 received at 36194 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 36194 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: Re: [PATCH 00/10] Add csvkit.
Date: Sat, 06 Jul 2019 21:01:50 +0100

Hello,

Pierre Langlois writes:

> Hello Guix!
>
> Here are patches to add csvkit [0] and its dependencies to guix. It provides
> various useful commands to deal with CSV data.
>
> A lot of the packages come from the same https://github.com/wireservice
> account and their documentation is built the same way so I've added them in a
> new 'wireservice.scm' file, so they could inherit from the same package
> definition.
>
> What do you think?

Friendly ping :-)

Thanks,
Pierre

>
> Thanks,
> Pierre
>
> [0]: https://csvkit.readthedocs.io/en/latest
>
> Pierre Langlois (10):
>   gnu: Add python-slugify.
>   gnu: Add python-pytimeparse.
>   gnu: Add python-crate.
>   gnu: Add python-dbfread.
>   gnu: Add python-leather.
>   gnu: Add python-agate.
>   gnu: Add python-agate-sql.
>   gnu: Add python-agate-dbf.
>   gnu: Add python-agate-excel.
>   gnu: Add csvkit.
>
>  gnu/local.mk                                  |   3 +
>  gnu/packages/databases.scm                    |  25 ++
>  gnu/packages/patches/csvkit-fix-tests.patch   |  45 +++
>  .../python-slugify-depend-on-unidecode.patch  |  22 ++
>  gnu/packages/python-web.scm                   |  31 ++
>  gnu/packages/python-xyz.scm                   |  22 ++
>  gnu/packages/time.scm                         |  21 ++
>  gnu/packages/wireservice.scm                  | 266 ++++++++++++++++++
>  8 files changed, 435 insertions(+)
>  create mode 100644 gnu/packages/patches/csvkit-fix-tests.patch
>  create mode 100644 gnu/packages/patches/python-slugify-depend-on-unidecode.patch
>  create mode 100644 gnu/packages/wireservice.scm

Message #46 received at 36194-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Pierre Langlois <pierre.langlois <at> gmx.com>
Cc: 36194-done <at> debbugs.gnu.org
Subject: Re: [bug#36194] [PATCH 05/10] gnu: Add python-leather.
Date: Sun, 07 Jul 2019 15:47:11 +0200

Hi Pierre!

I applied the whole series after tweaking two descriptions.

Two minor comments:

Pierre Langlois <pierre.langlois <at> gmx.com> skribis:

> +;; Base package definition for packages from https://github.com/wireservice.
> +;; This is done so we can share how to run tests and build documentation.
> +(define base-package
> +  (package
> +    (name #f)
> +    (version #f)
> +    (source #f)
> +    (home-page #f)
> +    (synopsis #f)
> +    (description #f)
> +    (build-system python-build-system)

As a matter of style, I would prefer to never have fields with a value
of an invalid type, as is the case above; if one forgets to override
these fields, we end up with an incorrect package.

What about using ‘python-leather’ as the base package and defining a
macro like:

  (define-syntax-rule (wireservice-package fields ...)
    (package
      (build-system python-build-system)
      (arguments (package-arguments python-leather))
      (native-inputs (package-native-inputs python-leather))
      fields ...))
        
?

> diff --git a/gnu/packages/patches/csvkit-fix-tests.patch b/gnu/packages/patches/csvkit-fix-tests.patch
> new file mode 100644
> index 0000000000..e62b601fe4
> --- /dev/null
> +++ b/gnu/packages/patches/csvkit-fix-tests.patch
> @@ -0,0 +1,45 @@
> +diff --git a/tests/test_utilities/test_csvsql.py b/tests/test_utilities/test_csvsql.py
> +index e6ec4af..4f47980 100644
> +--- a/tests/test_utilities/test_csvsql.py
> ++++ b/tests/test_utilities/test_csvsql.py
> +@@ -197,7 +197,7 @@ class TestCSVSQL(CSVKitTestCase, EmptyFileTests):
> +         utility.run()

For your future self and for your fellow hackers ;-), it’d be great if
you could add just a couple of lines at the top of each patch stating
(1) where they come from, and (2) what they do.

Thanks for the patch series, and apologies for the delay!

Ludo’.

Message #49 received at 36194-done <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>, 36194-done <at> debbugs.gnu.org
Subject: Re: [bug#36194] [PATCH 05/10] gnu: Add python-leather.
Date: Sun, 07 Jul 2019 15:35:00 +0100

Ludovic Courtès writes:

> Hi Pierre!
>
> I applied the whole series after tweaking two descriptions.
>
> Two minor comments:
>
> Pierre Langlois <pierre.langlois <at> gmx.com> skribis:
>
>> +;; Base package definition for packages from https://github.com/wireservice.
>> +;; This is done so we can share how to run tests and build documentation.
>> +(define base-package
>> +  (package
>> +    (name #f)
>> +    (version #f)
>> +    (source #f)
>> +    (home-page #f)
>> +    (synopsis #f)
>> +    (description #f)
>> +    (build-system python-build-system)
>
> As a matter of style, I would prefer to never have fields with a value
> of an invalid type, as is the case above; if one forgets to override
> these fields, we end up with an incorrect package.
>
> What about using ‘python-leather’ as the base package and defining a
> macro like:
>
>   (define-syntax-rule (wireservice-package fields ...)
>     (package
>       (build-system python-build-system)
>       (arguments (package-arguments python-leather))
>       (native-inputs (package-native-inputs python-leather))
>       fields ...))
>
> ?

Ah yeah I agree it's a much a better idea, I'll look into it as a
follow-up!

>
>> diff --git a/gnu/packages/patches/csvkit-fix-tests.patch b/gnu/packages/patches/csvkit-fix-tests.patch
>> new file mode 100644
>> index 0000000000..e62b601fe4
>> --- /dev/null
>> +++ b/gnu/packages/patches/csvkit-fix-tests.patch
>> @@ -0,0 +1,45 @@
>> +diff --git a/tests/test_utilities/test_csvsql.py b/tests/test_utilities/test_csvsql.py
>> +index e6ec4af..4f47980 100644
>> +--- a/tests/test_utilities/test_csvsql.py
>> ++++ b/tests/test_utilities/test_csvsql.py
>> +@@ -197,7 +197,7 @@ class TestCSVSQL(CSVKitTestCase, EmptyFileTests):
>> +         utility.run()
>
> For your future self and for your fellow hackers ;-), it’d be great if
> you could add just a couple of lines at the top of each patch stating
> (1) where they come from, and (2) what they do.

Noted!

>
> Thanks for the patch series, and apologies for the delay!

Thanks for applying this and making adjustments!

Pierre

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.