GNU bug report logs - #73109
[PATCH 0/2] Add python-transformers

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 73109 in the body.
You can then email your comments to 73109 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: guix-patches <at> gnu.org
Cc: ngraves <at> ngraves.fr
Subject: [PATCH 0/2] Add python-transformers
Date: Sat,  7 Sep 2024 18:54:51 +0200

This patch adds the popular python-transformers package.

Nicolas Graves (2):
  gnu: python-huggingface-hub: Update to 0.23.2.
  gnu: Add python-transformers.

 gnu/packages/machine-learning.scm | 60 +++++++++++++++++++++++++++++++
 gnu/packages/python-web.scm       | 10 ++++--
 2 files changed, 67 insertions(+), 3 deletions(-)

-- 
2.45.2

Message #8 received at 73109 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 73109 <at> debbugs.gnu.org
Cc: ngraves <at> ngraves.fr
Subject: [PATCH 2/2] gnu: Add python-transformers.
Date: Sat,  7 Sep 2024 18:56:55 +0200

* gnu/packages/machine-learning.scm (python-transformers): New variable.

Change-Id: Ifd7fa3a0f4611d3298ab76ceb44b3aea1397b824
---
 gnu/packages/machine-learning.scm | 60 +++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 412499d424..42842d7d61 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5846,6 +5846,66 @@ (define-public python-tokenizers
 tokenizers, @code{rust-tokenizers}.")
     (license license:asl2.0)))
 
+(define-public python-transformers
+  (package
+    (name "python-transformers")
+    (version "4.44.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "transformers" version))
+       (sha256
+        (base32 "09h84wqqk2bgi4vr9d1m3dsliard99l53n96wic405gfjb61gain"))))
+    (build-system pyproject-build-system)
+    ;; The imported package contains ~60 more inputs, but they don't seem
+    ;; necessary to build a minimal version of the package.
+    (propagated-inputs
+     (list python-filelock
+           python-huggingface-hub
+           python-numpy
+           python-pytorch
+           python-pyyaml
+           python-regex
+           python-requests
+           python-safetensors
+           python-tokenizers
+           python-tqdm
+           tensorflow))
+    (home-page "https://github.com/huggingface/transformers")
+    (synopsis "Machine Learning for PyTorch and TensorFlow")
+    (description
+     "This package provides easy download of thousands of pretrained models to
+perform tasks on different modalities such as text, vision, and audio.
+
+These models can be applied on:
+@itemize
+@item Text, for tasks like text classification, information extraction,
+question answering, summarization, translation, and text generation, in over
+100 languages.
+@item Images, for tasks like image classification, object detection, and
+segmentation.
+@item Audio, for tasks like speech recognition and audio classification.
+@end itemize
+
+Transformer models can also perform tasks on several modalities combined, such
+as table question answering, optical character recognition, information
+extraction from scanned documents, video classification, and visual question
+answering.
+
+This package provides APIs to quickly download and use those pretrained models
+on a given text, fine-tune them on your own datasets and then share them with
+the community on our model hub.  At the same time, each python module defining
+an architecture is fully standalone and can be modified to enable quick
+research experiments.
+
+Transformers is backed by the three most popular deep learning libraries —
+Jax, PyTorch and TensorFlow — with a seamless integration between them.  It's
+straightforward to train your models with one before loading them for
+inference with the other.
+
+Note: This version doesn't support integration with JAX.")
+    (license license:asl2.0)))
+
 (define-public python-hmmlearn
   (package
     (name "python-hmmlearn")
-- 
2.45.2

Message #11 received at 73109 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 73109 <at> debbugs.gnu.org
Cc: ngraves <at> ngraves.fr
Subject: [PATCH 1/2] gnu: python-huggingface-hub: Update to 0.23.2.
Date: Sat,  7 Sep 2024 18:56:54 +0200

* gnu/packages/python-web.scm (python-huggingface-hub): Update to 0.23.2.

Change-Id: I06989e883f54d9f0fc8db002d0a580008f6c6b67
---
 gnu/packages/python-web.scm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index bb1b912c0c..2a7dc8f8b3 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -143,7 +143,7 @@ (define-module (gnu packages python-web)
 (define-public python-huggingface-hub
   (package
     (name "python-huggingface-hub")
-    (version "0.20.3")
+    (version "0.23.2")
     (source
      (origin
        (method git-fetch)
@@ -152,7 +152,7 @@ (define-public python-huggingface-hub
              (commit (string-append "v" version))))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "00indl9labvqvm4m0y5jbzl68cgj8i60a6qy498gpnjj2pqk4l6v"))))
+        (base32 "0hygxqcixkc1d9sr47j2km6z0p17aj4k1dzm4cvpddrvhrgqayq5"))))
     (build-system pyproject-build-system)
     (arguments
      (list
@@ -185,7 +185,11 @@ (define-public python-huggingface-hub
              "--ignore=tests/test_utils_git_credentials.py"
              "--ignore=tests/test_utils_http.py"
              "--ignore=tests/test_utils_pagination.py"
-             "--ignore=tests/test_webhooks_server.py")
+             "--ignore=tests/test_webhooks_server.py"
+             "-k" (string-append
+                   "not test_push_to_hub"
+                   " and not test_from_pretrained_model_id_only"
+                   " and not test_from_pretrained_model_id_and_revision"))
       #:phases
       '(modify-phases %standard-phases
          (add-before 'check 'pre-check
-- 
2.45.2

Message #20 received at 73109 <at> debbugs.gnu.org (full text, mbox):

From: jgart <jgart <at> dismail.de>
To: 73109 <at> debbugs.gnu.org
Cc: Nicolas Graves <ngraves <at> ngraves.fr>
Subject: Re: [PATCH 0/2] Add python-transformers
Date: Tue, 10 Sep 2024 00:07:45 -0500

Hi,

I applied the huggingface-hub update but the rest no longer applies.

Can you rebase on latest and send a v2 with just python-transformers?

Thanks!

-- 
all the best,
jgart

Message #23 received at 73109 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: jgart <jgart <at> dismail.de>, 73109 <at> debbugs.gnu.org, herman <at> rimm.ee
Subject: Re: [bug#73109] [PATCH 0/2] Add python-transformers
Date: Tue, 10 Sep 2024 10:25:53 +0200

On 2024-09-10 00:07, jgart via Guix-patches via wrote:

> Hi,

Hi jgart, thanks!

>
> I applied the huggingface-hub update but the rest no longer applies.
>
> Can you rebase on latest and send a v2 with just python-transformers?
>

The python-tokenizers [73106] patches are also necessary before merging
python-transformers.

Herman Rimm has already reviewed the python-safetensors patches, these
are the same kind.

I'll resend a patch series for python-transformers once these blocking
patch series will be merged ;)

> Thanks!

-- 
Best regards,
Nicolas Graves

Message #28 received at 73109-done <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: 73109-done <at> debbugs.gnu.org
Subject: [PATCH 0/2] Add python-transformers
Date: Mon, 07 Apr 2025 17:15:00 +0200

Pushed to master branch with commit 
67901abeec87f600efbc3ef9bec3eca01ddfbb5e.  Thank you!

-- 
Ricardo

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.