GNU bug report logs - #63906
[PATCH] gnu: Add python-tree-sitter.

Previous Next

Package: guix-patches;

Reported by: Nicolas Graves <ngraves <at> ngraves.fr>

Date: Mon, 5 Jun 2023 11:56:01 UTC

Severity: normal

Tags: patch

Done: Ludovic Courtès <ludo <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 63906 in the body.
You can then email your comments to 63906 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to guix-patches <at> gnu.org:
bug#63906; Package guix-patches. (Mon, 05 Jun 2023 11:56:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Nicolas Graves <ngraves <at> ngraves.fr>:
New bug report received and forwarded. Copy sent to guix-patches <at> gnu.org. (Mon, 05 Jun 2023 11:56:01 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: guix-patches <at> gnu.org
Cc: ngraves <at> ngraves.fr
Subject: [PATCH] gnu: Add python-tree-sitter.
Date: Mon,  5 Jun 2023 13:54:40 +0200
* gnu/packages/tree-sitter.scm (python-tree-sitter): New variable.
---
 gnu/packages/tree-sitter.scm | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 853507be2d..2887fa823b 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2022 muradm <mail <at> muradm.net>
 ;;; Copyright © 2022 Aleksandr Vityazev <avityazev <at> posteo.org>
 ;;; Copyright © 2023 Andrew Tropin <andrew <at> trop.in>
+;;; Copyright © 2023 Nicolas Graves <ngraves <at> ngraves.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -29,12 +30,32 @@ (define-module (gnu packages tree-sitter)
   #:use-module (gnu packages node)
   #:use-module (guix build-system cargo)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system pyproject)
+  #:use-module ((guix build-system python) #:select (pypi-uri))
   #:use-module (guix build-system tree-sitter)
   #:use-module (guix gexp)
   #:use-module (guix git-download)
+  #:use-module (guix download)
   #:use-module (guix packages)
   #:use-module (guix utils))
 
+(define-public python-tree-sitter
+  (package
+    (name "python-tree-sitter")
+    (version "0.20.1")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "tree_sitter" version))
+              (sha256
+               (base32
+                "03f6xqpnjh4g28d9g219w9n4l07bap91ws2xzfy4jrjxahn0hgz9"))))
+    (build-system pyproject-build-system)
+    (home-page "https://github.com/tree-sitter/py-tree-sitter")
+    (synopsis "Python bindings to the Tree-sitter parsing library")
+    (description "This package provides Python bindings to the
+Tree-sitter parsing library.")
+    (license license:expat)))
+
 (define-public tree-sitter
   (package
     (name "tree-sitter")

base-commit: eed55a6544d5bda2245ec853e5fa4b28e1865bea
prerequisite-patch-id: a057b35ab55298bad50caab186b3e692a25230e1
prerequisite-patch-id: fb9054f780e6f97b92f00fdbe56058d1188ccf0a
prerequisite-patch-id: ca2f2591980b80c5cf27846e59e323bdc5a06b00
prerequisite-patch-id: ae5ad13b181ebb3c31d529af50622e3b78641442
prerequisite-patch-id: 34ed6acb0a1e5f79b5f6d18a6d4ef70cd97bf7ad
prerequisite-patch-id: 10d52b209b6e9c771050eef67ce566e79ab55c49
prerequisite-patch-id: e78e2a6daf59564caf5d2affe04ea7dde07f76c6
prerequisite-patch-id: 6aad4df7b83bfd5c2da38d9c2f80fba749f607b5
prerequisite-patch-id: da6a2d63ebb0ba1abb0b7c569d353724d900f95f
prerequisite-patch-id: 6279cff75e76e262f6ec82518db1fdf4c1810303
prerequisite-patch-id: 44453fcf2f2c38212a47d45d43ddcfa98167fabe
prerequisite-patch-id: 641eae2fa3842045ebe6072ad78214002f818221
prerequisite-patch-id: c19de9ee8c57210cbffc79945e69a858639f39bf
prerequisite-patch-id: 9833a747398a641803e203f8293382f55ad24ed1
prerequisite-patch-id: 94d5340918e3626726b6d32d93bf47425751898f
prerequisite-patch-id: e18164416e2c070b0b71f770c90d4c04af2635c1
prerequisite-patch-id: 31e98ea035053a965e87ad0164030cf909922d9e
prerequisite-patch-id: a1cf1f5c4a0ff2804fac986a69ffbc0328300afe
prerequisite-patch-id: 2a54e276f79fb57113a0be11e1ea2c07fdc2727d
prerequisite-patch-id: a463de1ba17ecb39588dfbd46c3bc5f9e0fb1b1c
prerequisite-patch-id: 3188de66dfc4bcb71f90601822428701528f4a98
prerequisite-patch-id: 6c93f771a1eca0747fd92a770fe750e2f15d8e52
prerequisite-patch-id: 12b76e9c2751da73ed64c9489b15f74ff17568cf
prerequisite-patch-id: eb618ab7b10483d917c308a38792af98baa517e2
prerequisite-patch-id: a471a4b7839bfb0ee9a3fd53ed962d729d38bd94
prerequisite-patch-id: 5e58202cc87a257c78033dafa62ffae4383e3718
prerequisite-patch-id: da6f4e25ed77113effc398694478bb392c4306ac
prerequisite-patch-id: febce01ba95d8c8f22e2088ab605ffd6959036ba
prerequisite-patch-id: caf73ad8377b81dd6f8e328bcaae935680df30ce
prerequisite-patch-id: 779a2f8d2ffa699e3c0029fde8dea22f5cce15b0
prerequisite-patch-id: a2dfb2fba1e1a3c8e270823022b6f462d27f17c8
prerequisite-patch-id: 843773f53ca319821185f9f9bc43ad905f081ee7
prerequisite-patch-id: cd7f69695aa47b7e1b1160841fe842a3acd160e7
-- 
2.40.1





Information forwarded to guix-patches <at> gnu.org:
bug#63906; Package guix-patches. (Mon, 05 Jun 2023 13:52:01 GMT) Full text and rfc822 format available.

Message #8 received at 63906 <at> debbugs.gnu.org (full text, mbox):

From: Jean Pierre De Jesus DIAZ <me <at> jeandudey.tech>
To: "63906 <at> debbugs.gnu.org" <63906 <at> debbugs.gnu.org>
Cc: "ngraves <at> ngraves.fr" <ngraves <at> ngraves.fr>
Subject: Re: [PATCH] gnu: Add python-tree-sitter.
Date: Mon, 05 Jun 2023 13:51:04 +0000
Hello Nicholas,

I took a look at the package and it bundles the tree-sitter code and
compiles it from source code.  As tree-sitter is already packaged in
GNU Guix the setup.py script could be patched to instead build the
bindings and link with the system tree-sitter.

So the `origin' should contain a `snippet' to remove the bundled
code and on the build phases `setup.py' (and any other files) should
be patched to use the system tree-sitter.

Cheers,

—
Jean-Pierre De Jesus DIAZ




Information forwarded to guix-patches <at> gnu.org:
bug#63906; Package guix-patches. (Tue, 06 Jun 2023 08:36:01 GMT) Full text and rfc822 format available.

Message #11 received at 63906 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 63906 <at> debbugs.gnu.org
Cc: ngraves <at> ngraves.fr
Subject: [PATCH v2] gnu: Add python-tree-sitter.
Date: Tue,  6 Jun 2023 10:35:08 +0200
* gnu/packages/tree-sitter.scm (python-tree-sitter): New variable.
---
 gnu/packages/tree-sitter.scm | 58 ++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 853507be2d..20bd4466cc 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2022 muradm <mail <at> muradm.net>
 ;;; Copyright © 2022 Aleksandr Vityazev <avityazev <at> posteo.org>
 ;;; Copyright © 2023 Andrew Tropin <andrew <at> trop.in>
+;;; Copyright © 2023 Nicolas Graves <ngraves <at> ngraves.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -22,6 +23,7 @@
 
 (define-module (gnu packages tree-sitter)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (gnu packages)
   #:use-module (gnu packages crates-graphics)
   #:use-module (gnu packages crates-io)
   #:use-module (gnu packages graphviz)
@@ -29,13 +31,69 @@ (define-module (gnu packages tree-sitter)
   #:use-module (gnu packages node)
   #:use-module (guix build-system cargo)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system pyproject)
   #:use-module (guix build-system tree-sitter)
   #:use-module (guix gexp)
   #:use-module (guix git-download)
   #:use-module (guix packages)
   #:use-module (guix utils))
 
+(define-public python-tree-sitter
+  (package
+    (name "python-tree-sitter")
+    (version "0.20.1")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/tree-sitter/py-tree-sitter")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "1rc8zqiry4n52xlf7pwx4s56ka9vwjzwgn7blwbkiscqdwvsai92"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'set-tree-sitter-lib-path
+            (lambda _
+              (let ((tree-sitter #$(this-package-input "tree-sitter")))
+                (substitute* "setup.py"
+                  (((string-append
+                     "( *)\\[\"tree_sitter\\/core\\/lib\\/src\\/lib\\.c\", "
+                     "\"tree_sitter\\/binding\\.c\"\\],") all tabs)
+                   (string-append
+                    tabs "[\"tree_sitter/binding.c\"],\n"
+                    tabs "library_dirs=[\"" tree-sitter "/lib\"],\n"
+                    tabs "libraries=[\"tree-sitter\"],"))
+                  (("include_dirs=.*")
+                   (string-append
+                    "include_dirs=[\"" tree-sitter "/include\"],\n"))))))
+          (add-before 'check 'set-test-lib-paths
+            (lambda _
+              (let ((py #$(this-package-native-input "tree-sitter-python"))
+                    (js #$(this-package-native-input "tree-sitter-javascript")))
+                (substitute* "tests/test_tree_sitter.py"
+                  (("Language\\.build_library")
+                   "_ =")
+                  (("LIB_PATH(, \"python\")" all name)
+                   (string-append
+                    "\"" py "/lib/tree-sitter/libtree-sitter-python.so\"" name))
+                  (("LIB_PATH(, \"javascript\")" all name)
+                   (string-append
+                    "\"" js "/lib/tree-sitter/libtree-sitter-javascript.so\""
+                    name)))))))))
+    (inputs (list tree-sitter))
+    (native-inputs
+     (list tree-sitter-python tree-sitter-javascript))
+    (home-page "https://github.com/tree-sitter/py-tree-sitter")
+    (synopsis "Python bindings to the Tree-sitter parsing library")
+    (description "This package provides Python bindings to the
+Tree-sitter parsing library.")
+    (license license:expat)))
+
 (define-public tree-sitter
   (package
     (name "tree-sitter")

-- 
2.40.1





Reply sent to Ludovic Courtès <ludo <at> gnu.org>:
You have taken responsibility. (Sat, 01 Jul 2023 21:38:02 GMT) Full text and rfc822 format available.

Notification sent to Nicolas Graves <ngraves <at> ngraves.fr>:
bug acknowledged by developer. (Sat, 01 Jul 2023 21:38:02 GMT) Full text and rfc822 format available.

Message #16 received at 63906-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Nicolas Graves <ngraves <at> ngraves.fr>
Cc: 63906-done <at> debbugs.gnu.org,
 Jean-Pierre De Jesus DIAZ <jean <at> foundationdevices.com>
Subject: Re: bug#63906: [PATCH] gnu: Add python-tree-sitter.
Date: Sat, 01 Jul 2023 23:37:40 +0200
Hi,

Nicolas Graves <ngraves <at> ngraves.fr> skribis:

> * gnu/packages/tree-sitter.scm (python-tree-sitter): New variable.

Applied!  Thanks, Jean-Pierre, for reviewing.

I wondered whether ‘tree_sitter/binding.c’ is a generated file.  If you
search carefully though, there are comments (few of them!) that appear
to be written by human beings.  So I guess it’s fine.

Thank you!

Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sun, 30 Jul 2023 11:24:06 GMT) Full text and rfc822 format available.

This bug report was last modified 241 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.