GNU bug report logs - #75732
[PATCH] gnu: python-libcst: Update to 1.6.0.

Previous Next

To reply to this bug, email your comments to 75732 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: guix-patches <at> gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH] gnu: python-libcst: Update to 1.6.0.
Date: Tue, 21 Jan 2025 18:17:10 +0200

* gnu/packages/python-xyz.scm (python-libcst): Update to 1.6.0.
[source]: Add snippet to satisfy cargo-build-system.
[build-system]: Switch to cargo-build-system.
[arguments]: Import cargo-build-system, pyproject-build-system modules.
Add cargo-inputs, cargo-development-inputs.  Replace 'build, 'install
with the pyproject-build-system phases.  Add a phase after 'install for
the extra pyproject phases.  Add 2 phases to help build the project.
[native-inputs]: Add python-minimal-wrapper, python-hypothesmith,
python-setuptools-rust, python-setuptools-scm.
[inputs]: Add maturin.
[propagated-inputs]: Remove python-typing-extensions,
python-typing-inspect.
(python-libcst-minimal)[arguments]: Inherit phases from python-libcst
while still skipping the tests.
[native-inputs]: Add python-minimal-wrapper, python-setutools-rust,
python-setuptools-scm.

Change-Id: I7b5ba5404f6e726976af25c0f7db7db2d00bedc5
---
 gnu/packages/python-xyz.scm | 111 ++++++++++++++++++++++++++++--------
 1 file changed, 86 insertions(+), 25 deletions(-)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 41735a78ead..8a634a77faf 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -197,6 +197,7 @@ (define-module (gnu packages python-xyz)
   #:use-module (gnu packages cmake)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages cpp)
+  #:use-module (gnu packages crates-check)
   #:use-module (gnu packages crates-io)
   #:use-module (gnu packages crates-windows)
   #:use-module (gnu packages crypto)
@@ -22981,40 +22982,95 @@ (define-public python-lark-parser
 (define-public python-libcst
   (package
     (name "python-libcst")
-    (version "0.3.18") ; starting from 0.4.0 project depends on Rust
+    (version "1.6.0")
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "libcst" version))
               (sha256
                (base32
-                "1ll0yyxbz8zyqcy9kfcqi3l5fah2zqisvpjbzjnz7s7dmb84q59h"))))
-    (build-system pyproject-build-system)
+                "1nzhvg52whpmfv8fkrnv7rlrmgc5m43hpyp8ra9kffx47yzcs3p8"))
+              (snippet
+               #~(begin (use-modules (guix build utils))
+                        ;; This is mostly to keep the cargo-build-system happy
+                        (with-output-to-file "Cargo.toml"
+                          (lambda ()
+                            (format #t "\
+[workspace]
+
+members = [
+    \"native\",]")))))))
+    (build-system cargo-build-system)
     (arguments
      (list
-      #:test-flags
-      #~(list
-         ;; Reported upstream: <https://github.com/Instagram/LibCST/issues/346>.
-         "--ignore=libcst/tests/test_fuzz.py"
-         ;; Reported upstream: <https://github.com/Instagram/LibCST/issues/347>.
-         "--ignore=libcst/tests/test_pyre_integration.py"
-         "--ignore=libcst/codemod/tests/test_codemod_cli.py"
-         "--ignore=libcst/metadata/tests/test_full_repo_manager.py"
-         "--ignore=libcst/metadata/tests/test_type_inference_provider.py"
-         "-k" (string-join
-               ;; AssertionError: False is not true : libcst.matchers.__init__
-               ;; needs new codegen!
-               (list "not test_codegen_clean_matcher_classes"
-                     "test_codegen_clean_return_types"
-                     "test_codegen_clean_visitor_functions")
-               " and not "))))
+      #:install-source? #false
+      #:cargo-test-flags
+      ;; According to .github/workflows/ci.yml
+      '(list "--manifest-path=native/Cargo.toml"
+             "--release"
+             "--no-default-features")
+      #:cargo-inputs
+      (list rust-chic-1
+            rust-memchr-2
+            rust-paste-1
+            rust-peg-0.8
+            rust-pyo3-0.22
+            rust-quote-1
+            rust-regex-1
+            rust-syn-2
+            rust-thiserror-1)
+      #:cargo-development-inputs
+      (list rust-criterion-0.5
+            rust-difference-2
+            rust-itertools-0.13
+            rust-rayon-1
+            rust-trybuild-1)
+      #:imported-modules `(,@%pyproject-build-system-modules
+                           ,@%cargo-build-system-modules)
+      #:modules `((guix build cargo-build-system)
+                  ((guix build pyproject-build-system) #:prefix py:)
+                  (guix build utils))
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'prepare-source
+            (lambda _
+              (delete-file "native/Cargo.lock")))
+          (add-after 'configure 'dont-vendor-self
+            (lambda* (#:key vendor-dir #:allow-other-keys)
+              ;; Don't keep the whole tarball in the vendor directory
+              (delete-file-recursively
+                (string-append vendor-dir "/libcst-" #$version ".tar.zst"))))
+          (replace 'build
+            (assoc-ref py:%standard-phases 'build))
+          (add-after 'install 'wrap
+            (lambda _
+              ;; Collection of python- and pyproject-build-system phases
+              ;; between 'install and 'check.
+              (assoc-ref py:%standard-phases 'add-install-to-pythonpath)
+              (assoc-ref py:%standard-phases 'add-install-to-path)
+              (assoc-ref py:%standard-phases 'wrap)
+              (assoc-ref py:%standard-phases 'create-entrypoints)
+              (assoc-ref py:%standard-phases 'compile-bytecode)))
+          ;; We are not the only ones who cannot load libcst.native
+          ;; during the 'check phase
+          ;; https://github.com/Instagram/LibCST/issues/1176
+          ;(add-after 'wrap 'python-check
+          ;  (lambda args
+          ;    (apply (assoc-ref py:%standard-phases 'check)
+          ;           #:test-flags '()
+          ;           args)))
+          (replace 'install
+            (assoc-ref py:%standard-phases 'install)))))
     (native-inputs
-     (list python-pytest
+     (list python-minimal-wrapper
+           python-hypothesmith
+           python-pytest
            python-setuptools
+           python-setuptools-rust
+           python-setuptools-scm
            python-wheel))
+    (inputs (list maturin))
     (propagated-inputs
-     (list python-typing-extensions
-           python-typing-inspect
-           python-pyyaml))
+     (list python-pyyaml))
     (home-page "https://github.com/Instagram/LibCST")
     (synopsis "Concrete Syntax Tree (CST) parser and serializer library for Python")
     (description
@@ -23037,9 +23093,14 @@ (define-public python-libcst-minimal
    (package
      (inherit python-libcst)
      (name "python-libcst-minimal")
-     (arguments '(#:tests? #f))
+     (arguments
+      (substitute-keyword-arguments (package-arguments python-libcst)
+        ((#:tests? _ #t) #f)))
     (native-inputs
-     (list python-setuptools
+     (list python-minimal-wrapper
+           python-setuptools
+           python-setuptools-rust
+           python-setuptools-scm
            python-wheel)))))
 
 (define-public python-typeapi

base-commit: cf34ba66cf614358dfac925212d6a9074e318d80
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #8 received at 75732 <at> debbugs.gnu.org (full text, mbox):

From: Sharlatan Hellseher <sharlatanus <at> gmail.com>
To: 75732 <at> debbugs.gnu.org
Cc: rekado <at> elephly.net, efraim <at> flashner.co.il
Subject: [PATCH] gnu: python-libcst: Update to 1.6.0.
Date: Fri, 24 Jan 2025 12:43:47 +0000

[Message part 1 (text/plain, inline)]

Hi Efraim,

Thanks for picking it up, more and more lower level Python packages
started requiring Rust, which would we a common strategy in such cases?


--8<---------------cut here---------------start------------->8---
Building the following 1 package would ensure 4 dependent packages are
rebuilt: orange <at> 3.32.0
--8<---------------cut here---------------end--------------->8---
There is no many direct depednencies on this package only on minimal
varian. Used in python-isort, python-hypothesmith, and Orange is broken
for a long time.

--8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix refresh --list-dependent python-isort python-hypothesmith
Building the following 72 packages would ensure 126 dependent packages are rebuilt:
--8<---------------cut here---------------end--------------->8---
I'd say it's save to push it on Master.

CC Ricardo if he works on it as well as part of python-team path to
Python 3.11.

Patch LGFM.

--
Oleg

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 75732 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Sharlatan Hellseher <sharlatanus <at> gmail.com>
Cc: efraim <at> flashner.co.il, 75732 <at> debbugs.gnu.org
Subject: Re: [PATCH] gnu: python-libcst: Update to 1.6.0.
Date: Fri, 24 Jan 2025 15:40:22 +0100

Sharlatan Hellseher <sharlatanus <at> gmail.com> writes:

> I'd say it's save to push it on Master.

Yes, should be fine.

> CC Ricardo if he works on it as well as part of python-team path to
> Python 3.11.

I haven't yet done much on python-team other than break thousands of
packages :)

-- 
Ricardo

Message #14 received at 75732 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: Sharlatan Hellseher <sharlatanus <at> gmail.com>, 75732 <at> debbugs.gnu.org
Subject: Re: [PATCH] gnu: python-libcst: Update to 1.6.0.
Date: Sun, 26 Jan 2025 09:45:24 +0200

[Message part 1 (text/plain, inline)]

On Fri, Jan 24, 2025 at 03:40:22PM +0100, Ricardo Wurmus wrote:
> Sharlatan Hellseher <sharlatanus <at> gmail.com> writes:
> 
> > I'd say it's save to push it on Master.
> 
> Yes, should be fine.
> 
> > CC Ricardo if he works on it as well as part of python-team path to
> > Python 3.11.
> 
> I haven't yet done much on python-team other than break thousands of
> packages :)

That still counts :)

If all the packages use maturin then I think we could make a
maturin-build-system, but perhaps a combo python-with-rust build system
would work better.  I'd love to run the sanity-check phase and have
easier access to all the flags of the different build systems, as well
as the option of toggling the rust tests separately.

We might also want to either wrap maturin with python-minimal-wrapper or
propagate it since I'm pretty sure we always need both.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.