GNU bug report logs - #63897
[PATCH] gnu: tensorflow_lite: Update to 2.12.0

Previous Next

Package: guix-patches;

Reported by: Andy Tai <atai <at> atai.org>

Date: Sun, 4 Jun 2023 23:29:02 UTC

Severity: normal

Tags: patch

Done: Ricardo Wurmus <rekado <at> elephly.net>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 63897 in the body.
You can then email your comments to 63897 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Sun, 04 Jun 2023 23:29:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Andy Tai <atai <at> atai.org>:
New bug report received and forwarded. Copy sent to guix-patches <at> gnu.org. (Sun, 04 Jun 2023 23:29:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <atai <at> atai.org>
To: guix-patches <at> gnu.org
Cc: Andy Tai <atai <at> atai.org>
Subject: [PATCH] gnu: tensorflow_lite: Update to 2.12.0
Date: Sun,  4 Jun 2023 16:28:18 -0700
* gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable

* gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.0
  [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to
  find packages already in Guix as dependencies for absl-cpp, eigen,
  flatbuffer, neon2ssl, cpuinfo, ruy.
  [arguments] (phases): Remove operations setting up absl, eigen, ruy,
  and neon2ssl as native inouts as now we use Guix's packages of these
  as dependencies
  [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl,
  pthreadpool, ruy, vulkan and xnnpack as explicit inputs
  [arguments] (native-inputs): Remove local setup of neon2ssl and ruy
  as we now use Guix's packages of these as explicit dependencies

* gnu/packages/maths.scm: Remove extra statement

* gnu/packages/serialization.scm (flatbuffers-next-shared): New variable,
  flatbuffers-next built with -fPIC as needed by tensorflow-lite
---
 gnu/packages/algebra.scm          |  32 --------
 gnu/packages/machine-learning.scm | 118 ++++++++++++++++--------------
 gnu/packages/maths.scm            |   1 -
 gnu/packages/serialization.scm    |  12 +++
 4 files changed, 75 insertions(+), 88 deletions(-)

diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm
index 3babf90b3c..81df0c97ac 100644
--- a/gnu/packages/algebra.scm
+++ b/gnu/packages/algebra.scm
@@ -1155,38 +1155,6 @@ (define-public eigen-for-tensorflow
                  (substitute* "test/CMakeLists.txt"
                    (("ei_add_test\\(stddeque") "#")))))))))))
 
-(define-public eigen-for-tensorflow-lite
-  ;; This commit was taken from
-  ;; tensorflow/lite/tools/cmake/modules/eigen.cmake
-  (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9")
-        (revision "1"))
-    (package (inherit eigen)
-      (name "eigen-for-tensorflow-lite")
-      (version (git-version "3.3.7" revision commit))
-      (source (origin
-                (method git-fetch)
-                (uri (git-reference
-                      (url "https://gitlab.com/libeigen/eigen")
-                      (commit commit)))
-                (sha256
-                 (base32
-                  "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf"))
-                (file-name (git-file-name name version))
-                (modules '((guix build utils)))
-                (snippet
-                 ;; Ther are test failures in the "unsupported" directory, but
-                 ;; maintainers say it's unsupported anyway, so just skip
-                 ;; them.
-                 '(begin
-                    (substitute* "unsupported/CMakeLists.txt"
-                      (("add_subdirectory\\(test.*")
-                       "# Do not build the tests for unsupported features.\n"))))))
-      (arguments
-       (substitute-keyword-arguments (package-arguments eigen)
-         ((#:phases phases)
-          `(modify-phases ,phases
-             (delete 'disable-some-tests))))))))
-
 (define-public xtensor
   (package
     (name "xtensor")
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index c9bc5e8b68..15fe4be995 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -54,6 +54,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages)
   #:use-module (gnu packages adns)
   #:use-module (gnu packages algebra)
+  #:use-module (gnu packages assembly)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
@@ -88,6 +89,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages ninja)
   #:use-module (gnu packages ocaml)
   #:use-module (gnu packages onc-rpc)
+  #:use-module (gnu packages opencl)
   #:use-module (gnu packages parallel)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
@@ -107,6 +109,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages swig)
   #:use-module (gnu packages time)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages vulkan)
   #:use-module (gnu packages video)
   #:use-module (gnu packages web)
   #:use-module (gnu packages xml)
@@ -2642,7 +2645,7 @@ (define-public tensorflow
 (define-public tensorflow-lite
   (package
     (name "tensorflow-lite")
-    (version "2.5.0")
+    (version "2.12.0")
     (source
      (origin
        (method git-fetch)
@@ -2652,35 +2655,53 @@ (define-public tensorflow-lite
        (file-name (git-file-name name version))
        (sha256
         (base32
-         "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l"))))
+         "051knmq6ysg4936d9baaqsx6j3rfsp4ffi2pwrxx4llan682rvpi"))))
     (build-system cmake-build-system)
     (arguments
      `(#:tests? #false                  ; no "check" target
        #:build-type "Release"
        #:configure-flags
        (list
-        "-DTFLITE_ENABLE_GPU=OFF"
-        "-DTFLITE_ENABLE_RUY=OFF"
+       	;"--trace"
+       	;; Use Guix's own versions of library dependencies.
+       	"-DTFLITE_ENABLE_INSTALL=ON"   ;; so cmake can be used to find this from other packages
+       	"-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON"
+        "-DTFLITE_ENABLE_GPU=ON"
+        "-DTFLITE_ENABLE_RUY=ON"
+
+        ;; TO DO: turn on Farmhash
+        ;"-DSYSTEM_FARMHASH=ON"
+
+
+        (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp")
+        	"/lib/cmake/absl")
+        (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen")
+        	"/share/eigen3/cmake")
+        (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers")
+        	"/lib/cmake/flatbuffers")
+        (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse")
+        	"/lib/cmake/NEON_2_SSE")
+        (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo")
+        	"/share/cpuinfo")
+        (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy")
+        	"/lib/cmake/ruy")
 
         ;; TODO: The build system attempts to build xnnpack from source.  We
         ;; would like to use our xnnpack package here, but this requires more
         ;; work.
         "-DTFLITE_ENABLE_XNNPACK=OFF"
 
-        ;; Pretend we've already fetched abseil.  We won't actually build it
-        ;; but use the existing package.
-        "-Dabseil-cpp_POPULATED=TRUE"
-
-        ;; Don't fetch the sources.  We have already built flatbuffers.
-        "-Dflatbuffers_POPULATED=TRUE"
+        ;;; Don't fetch the sources.  We have these already
+        "-Degl_headers_POPULATED=TRUE"
+        "-Dfp16_headers_POPULATED=TRUE"
+        "-Dopencl_headers_POPULATED=TRUE"
+        "-Dopengl_headers_POPULATED=TRUE"
+        "-Dvulkan_headers_POPULATED=TRUE"
 
         "-DFFT2D_SOURCE_DIR=/tmp/fft2d"
-        "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse"
-        "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin"
         "-DFARMHASH_SOURCE_DIR=/tmp/farmhash"
         "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp"
-        (string-append "-DRUY_SOURCE_DIR="
-                       (assoc-ref %build-inputs "ruy-src")))
+        )
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'chdir
@@ -2689,20 +2710,19 @@ (define-public tensorflow-lite
            (lambda* (#:key inputs #:allow-other-keys)
              ;; Use external cmake finders instead of these stubs that won't
              ;; find anything but the bundled sources.
-             (delete-file "tools/cmake/modules/Findabsl.cmake")
-             (delete-file "tools/cmake/modules/Findeigen.cmake")
 
+             ;; TO DO: properly use Guix's pthreaqdpool
+             ;; roght now not using pthreadpool because
+             ;; we are not enabling xnnpack
+             (substitute* "CMakeLists.txt"
+              (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)")
+               "if(false)"))
              (substitute* "CMakeLists.txt"
-               (("find_package\\(eigen REQUIRED")
-                "find_package(eigen REQUIRED NAMES Eigen3"))
-             (substitute* "tools/cmake/modules/Findflatbuffers.cmake"
-               (("get_target_property.*")
-                (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n"
-                        (assoc-ref inputs "flatbuffers"))))
+              (("if\\(NOT TARGET pthreadpool\\)")
+               "if(false)"))
 
              ;; Don't fetch source code; we already have everything we need.
              (substitute* '("tools/cmake/modules/fft2d.cmake"
-                            "tools/cmake/modules/ruy.cmake"
                             "tools/cmake/modules/farmhash.cmake"
                             "tools/cmake/modules/neon2sse.cmake"
                             "tools/cmake/modules/gemmlowp.cmake")
@@ -2718,10 +2738,9 @@ (define-public tensorflow-lite
                (invoke "tar" "--strip-components=1"
                        "-xf" (assoc-ref inputs "fft2d-src")))
 
-             (copy-recursively (assoc-ref inputs "neon2sse-src")
-                               "/tmp/neon2sse/")
              (copy-recursively (assoc-ref inputs "gemmlowp-src")
-                               "/tmp/gemmlowp/")))
+                               "/tmp/gemmlowp/")
+             ))
          (add-after 'copy-sources 'prepare-shared-library-build
            (lambda _ (chdir "c")))
          (replace 'install
@@ -2738,9 +2757,21 @@ (define-public tensorflow-lite
                   (find-files "." "\\.h$")))))))))
     (inputs
      `(("abseil-cpp" ,abseil-cpp-20200923.3)
-       ("eigen" ,eigen-for-tensorflow-lite)
-       ("flatbuffers" ,flatbuffers)
-       ("python" ,python)))
+       ("cpuinfo" ,cpuinfo)
+       ("eigen" ,eigen)
+       ("fp16" ,fp16)
+       ("flatbuffers" ,flatbuffers-next-shared)
+       ;("gemmlowp" ,gemmlowp)  ;; TO DO
+       ("mesa-headers"  ,mesa-headers)
+       ("neon2sse" ,neon2sse)
+       ("opencl-clhpp" ,opencl-clhpp)
+       ("opencl-headers" ,opencl-headers)
+       ("opencl-icd-loader" ,opencl-icd-loader)
+       ("pthreadpool" ,pthreadpool)
+       ("python" ,python)
+       ("ruy" ,ruy)
+       ;("xnnpack" ,xnnpack)     ;; TO DO: properly use Guix's copy of xnnpack
+       ("vulkan-headers" , vulkan-headers)))
     (native-inputs
      `(("pkg-config" ,pkg-config)
        ("gemmlowp-src"
@@ -2756,17 +2787,6 @@ (define-public tensorflow-lite
              (sha256
               (base32
                "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl")))))
-       ("neon2sse-src"
-        ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/intel/ARM_NEON_2_x86_SSE")
-                   (commit commit)))
-             (file-name (git-file-name "neon2sse" (string-take commit 8)))
-             (sha256
-              (base32
-               "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i")))))
        ("farmhash-src"
         ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45"))
            (origin
@@ -2782,24 +2802,12 @@ (define-public tensorflow-lite
         ,(origin
            (method url-fetch)
            (uri (string-append "https://storage.googleapis.com/"
-                               "mirror.tensorflow.org/"
-                               "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz"))
+           		   "mirror.tensorflow.org/github.com/petewarden/"
+           		   "OouraFFT/archive/v1.0.tar.gz"))
            (file-name "fft2d.tar.gz")
            (sha256
             (base32
-             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))
-       ("ruy-src"
-        ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/google/ruy")
-                   (commit commit)
-                   (recursive? #true)))
-             (file-name (git-file-name "ruy" (string-take commit 8)))
-             (sha256
-              (base32
-               "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis")))))))
+             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))))
     (home-page "https://tensorflow.org")
     (synopsis "Machine learning framework")
     (description
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index c6fabf1973..25a7b61156 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -85,7 +85,6 @@ (define-module (gnu packages maths)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
-  #:use-module (guix gexp)
   #:use-module (guix utils)
   #:use-module ((guix build utils) #:select (alist-replace))
   #:use-module (guix build-system ant)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 39954d6764..268a5d3d9b 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -748,6 +748,18 @@ (define-public flatbuffers-next
                (base32
                 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
 
+(define-public flatbuffers-next-shared
+  (package
+    (inherit flatbuffers-next)
+    (name "flatbuffers-shared")
+    (version "23.1.21")
+    (arguments
+     (substitute-keyword-arguments (package-arguments flatbuffers-next)
+       ((#:configure-flags  flags)
+         ;; make C++ fiels compiled with -fPIC, needed for shared lib".
+              #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
+
+
 (define-public python-feather-format
   (package
     (name "python-feather-format")

base-commit: eed55a6544d5bda2245ec853e5fa4b28e1865bea
prerequisite-patch-id: 5e9e341f5896c5eee0f64a7a031cc57491e76eb9
prerequisite-patch-id: b17d63ac48fd66475dda11cd37f74bb63aca5cfb
-- 
2.40.1





Added blocking bug(s) 63889 and 63888 Request was from Andy Tai <atai <at> atai.org> to control <at> debbugs.gnu.org. (Sun, 04 Jun 2023 23:43:02 GMT) Full text and rfc822 format available.

Information forwarded to guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Sun, 04 Jun 2023 23:52:01 GMT) Full text and rfc822 format available.

Message #10 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <atai <at> atai.org>
To: 63897 <at> debbugs.gnu.org
Cc: Andy Tai <atai <at> atai.org>
Subject: [PATCH v2] gnu: tensorflow_lite: Update to 2.12.0
Date: Sun,  4 Jun 2023 16:50:56 -0700
* gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable

* gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.0
  [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to
  find packages already in Guix as dependencies for absl-cpp, eigen,
  flatbuffer, neon2ssl, cpuinfo, ruy.
  [arguments] (phases): Remove operations setting up absl, eigen, ruy,
  and neon2ssl as native inouts as now we use Guix's packages of these
  as dependencies
  [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl,
  pthreadpool, ruy, vulkan and xnnpack as explicit inputs
  [arguments] (native-inputs): Remove local setup of neon2ssl and ruy
  as we now use Guix's packages of these as explicit dependencies

* gnu/packages/maths.scm: Remove extra statement

* gnu/packages/serialization.scm (flatbuffers-next-shared): New variable,
  flatbuffers-next built with -fPIC as needed by tensorflow-lite
---
 gnu/packages/algebra.scm          |  32 --------
 gnu/packages/machine-learning.scm | 121 ++++++++++++++++--------------
 gnu/packages/maths.scm            |   1 -
 gnu/packages/serialization.scm    |  12 +++
 4 files changed, 78 insertions(+), 88 deletions(-)

diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm
index 3babf90b3c..81df0c97ac 100644
--- a/gnu/packages/algebra.scm
+++ b/gnu/packages/algebra.scm
@@ -1155,38 +1155,6 @@ (define-public eigen-for-tensorflow
                  (substitute* "test/CMakeLists.txt"
                    (("ei_add_test\\(stddeque") "#")))))))))))
 
-(define-public eigen-for-tensorflow-lite
-  ;; This commit was taken from
-  ;; tensorflow/lite/tools/cmake/modules/eigen.cmake
-  (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9")
-        (revision "1"))
-    (package (inherit eigen)
-      (name "eigen-for-tensorflow-lite")
-      (version (git-version "3.3.7" revision commit))
-      (source (origin
-                (method git-fetch)
-                (uri (git-reference
-                      (url "https://gitlab.com/libeigen/eigen")
-                      (commit commit)))
-                (sha256
-                 (base32
-                  "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf"))
-                (file-name (git-file-name name version))
-                (modules '((guix build utils)))
-                (snippet
-                 ;; Ther are test failures in the "unsupported" directory, but
-                 ;; maintainers say it's unsupported anyway, so just skip
-                 ;; them.
-                 '(begin
-                    (substitute* "unsupported/CMakeLists.txt"
-                      (("add_subdirectory\\(test.*")
-                       "# Do not build the tests for unsupported features.\n"))))))
-      (arguments
-       (substitute-keyword-arguments (package-arguments eigen)
-         ((#:phases phases)
-          `(modify-phases ,phases
-             (delete 'disable-some-tests))))))))
-
 (define-public xtensor
   (package
     (name "xtensor")
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index c9bc5e8b68..3740206b86 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -54,6 +54,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages)
   #:use-module (gnu packages adns)
   #:use-module (gnu packages algebra)
+  #:use-module (gnu packages assembly)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
@@ -88,6 +89,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages ninja)
   #:use-module (gnu packages ocaml)
   #:use-module (gnu packages onc-rpc)
+  #:use-module (gnu packages opencl)
   #:use-module (gnu packages parallel)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
@@ -107,6 +109,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages swig)
   #:use-module (gnu packages time)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages vulkan)
   #:use-module (gnu packages video)
   #:use-module (gnu packages web)
   #:use-module (gnu packages xml)
@@ -2642,7 +2645,7 @@ (define-public tensorflow
 (define-public tensorflow-lite
   (package
     (name "tensorflow-lite")
-    (version "2.5.0")
+    (version "2.12.0")
     (source
      (origin
        (method git-fetch)
@@ -2652,35 +2655,56 @@ (define-public tensorflow-lite
        (file-name (git-file-name name version))
        (sha256
         (base32
-         "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l"))))
+         "051knmq6ysg4936d9baaqsx6j3rfsp4ffi2pwrxx4llan682rvpi"))))
     (build-system cmake-build-system)
     (arguments
      `(#:tests? #false                  ; no "check" target
        #:build-type "Release"
        #:configure-flags
        (list
-        "-DTFLITE_ENABLE_GPU=OFF"
-        "-DTFLITE_ENABLE_RUY=OFF"
+       	;"--trace"
+       	"-DTFLITE_ENABLE_INSTALL=ON"
+       	;; so cmake can be used to find this from other packages
+
+       	"-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON"
+       	;; Use Guix's own packages as dependencies.
+
+        "-DTFLITE_ENABLE_GPU=ON"
+        "-DTFLITE_ENABLE_RUY=ON"
+
+        ;; TO DO: turn on Farmhash
+        ;"-DSYSTEM_FARMHASH=ON"
+
+
+        (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp")
+        	"/lib/cmake/absl")
+        (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen")
+        	"/share/eigen3/cmake")
+        (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers")
+        	"/lib/cmake/flatbuffers")
+        (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse")
+        	"/lib/cmake/NEON_2_SSE")
+        (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo")
+        	"/share/cpuinfo")
+        (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy")
+        	"/lib/cmake/ruy")
 
         ;; TODO: The build system attempts to build xnnpack from source.  We
         ;; would like to use our xnnpack package here, but this requires more
         ;; work.
         "-DTFLITE_ENABLE_XNNPACK=OFF"
 
-        ;; Pretend we've already fetched abseil.  We won't actually build it
-        ;; but use the existing package.
-        "-Dabseil-cpp_POPULATED=TRUE"
-
-        ;; Don't fetch the sources.  We have already built flatbuffers.
-        "-Dflatbuffers_POPULATED=TRUE"
+        ;;; Don't fetch the sources.  We have these already
+        "-Degl_headers_POPULATED=TRUE"
+        "-Dfp16_headers_POPULATED=TRUE"
+        "-Dopencl_headers_POPULATED=TRUE"
+        "-Dopengl_headers_POPULATED=TRUE"
+        "-Dvulkan_headers_POPULATED=TRUE"
 
         "-DFFT2D_SOURCE_DIR=/tmp/fft2d"
-        "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse"
-        "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin"
         "-DFARMHASH_SOURCE_DIR=/tmp/farmhash"
         "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp"
-        (string-append "-DRUY_SOURCE_DIR="
-                       (assoc-ref %build-inputs "ruy-src")))
+        )
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'chdir
@@ -2689,20 +2713,19 @@ (define-public tensorflow-lite
            (lambda* (#:key inputs #:allow-other-keys)
              ;; Use external cmake finders instead of these stubs that won't
              ;; find anything but the bundled sources.
-             (delete-file "tools/cmake/modules/Findabsl.cmake")
-             (delete-file "tools/cmake/modules/Findeigen.cmake")
 
+             ;; TO DO: properly use Guix's pthreaqdpool
+             ;; roght now not using pthreadpool because
+             ;; we are not enabling xnnpack
+             (substitute* "CMakeLists.txt"
+              (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)")
+               "if(false)"))
              (substitute* "CMakeLists.txt"
-               (("find_package\\(eigen REQUIRED")
-                "find_package(eigen REQUIRED NAMES Eigen3"))
-             (substitute* "tools/cmake/modules/Findflatbuffers.cmake"
-               (("get_target_property.*")
-                (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n"
-                        (assoc-ref inputs "flatbuffers"))))
+              (("if\\(NOT TARGET pthreadpool\\)")
+               "if(false)"))
 
              ;; Don't fetch source code; we already have everything we need.
              (substitute* '("tools/cmake/modules/fft2d.cmake"
-                            "tools/cmake/modules/ruy.cmake"
                             "tools/cmake/modules/farmhash.cmake"
                             "tools/cmake/modules/neon2sse.cmake"
                             "tools/cmake/modules/gemmlowp.cmake")
@@ -2718,10 +2741,9 @@ (define-public tensorflow-lite
                (invoke "tar" "--strip-components=1"
                        "-xf" (assoc-ref inputs "fft2d-src")))
 
-             (copy-recursively (assoc-ref inputs "neon2sse-src")
-                               "/tmp/neon2sse/")
              (copy-recursively (assoc-ref inputs "gemmlowp-src")
-                               "/tmp/gemmlowp/")))
+                               "/tmp/gemmlowp/")
+             ))
          (add-after 'copy-sources 'prepare-shared-library-build
            (lambda _ (chdir "c")))
          (replace 'install
@@ -2738,9 +2760,21 @@ (define-public tensorflow-lite
                   (find-files "." "\\.h$")))))))))
     (inputs
      `(("abseil-cpp" ,abseil-cpp-20200923.3)
-       ("eigen" ,eigen-for-tensorflow-lite)
-       ("flatbuffers" ,flatbuffers)
-       ("python" ,python)))
+       ("cpuinfo" ,cpuinfo)
+       ("eigen" ,eigen)
+       ("fp16" ,fp16)
+       ("flatbuffers" ,flatbuffers-next-shared)
+       ;("gemmlowp" ,gemmlowp)  ;; TO DO
+       ("mesa-headers"  ,mesa-headers)
+       ("neon2sse" ,neon2sse)
+       ("opencl-clhpp" ,opencl-clhpp)
+       ("opencl-headers" ,opencl-headers)
+       ("opencl-icd-loader" ,opencl-icd-loader)
+       ("pthreadpool" ,pthreadpool)
+       ("python" ,python)
+       ("ruy" ,ruy)
+       ;("xnnpack" ,xnnpack)     ;; TO DO: properly use Guix's copy of xnnpack
+       ("vulkan-headers" , vulkan-headers)))
     (native-inputs
      `(("pkg-config" ,pkg-config)
        ("gemmlowp-src"
@@ -2756,17 +2790,6 @@ (define-public tensorflow-lite
              (sha256
               (base32
                "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl")))))
-       ("neon2sse-src"
-        ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/intel/ARM_NEON_2_x86_SSE")
-                   (commit commit)))
-             (file-name (git-file-name "neon2sse" (string-take commit 8)))
-             (sha256
-              (base32
-               "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i")))))
        ("farmhash-src"
         ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45"))
            (origin
@@ -2782,24 +2805,12 @@ (define-public tensorflow-lite
         ,(origin
            (method url-fetch)
            (uri (string-append "https://storage.googleapis.com/"
-                               "mirror.tensorflow.org/"
-                               "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz"))
+           		   "mirror.tensorflow.org/github.com/petewarden/"
+           		   "OouraFFT/archive/v1.0.tar.gz"))
            (file-name "fft2d.tar.gz")
            (sha256
             (base32
-             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))
-       ("ruy-src"
-        ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/google/ruy")
-                   (commit commit)
-                   (recursive? #true)))
-             (file-name (git-file-name "ruy" (string-take commit 8)))
-             (sha256
-              (base32
-               "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis")))))))
+             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))))
     (home-page "https://tensorflow.org")
     (synopsis "Machine learning framework")
     (description
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index c6fabf1973..25a7b61156 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -85,7 +85,6 @@ (define-module (gnu packages maths)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
-  #:use-module (guix gexp)
   #:use-module (guix utils)
   #:use-module ((guix build utils) #:select (alist-replace))
   #:use-module (guix build-system ant)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 39954d6764..268a5d3d9b 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -748,6 +748,18 @@ (define-public flatbuffers-next
                (base32
                 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
 
+(define-public flatbuffers-next-shared
+  (package
+    (inherit flatbuffers-next)
+    (name "flatbuffers-shared")
+    (version "23.1.21")
+    (arguments
+     (substitute-keyword-arguments (package-arguments flatbuffers-next)
+       ((#:configure-flags  flags)
+         ;; make C++ fiels compiled with -fPIC, needed for shared lib".
+              #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
+
+
 (define-public python-feather-format
   (package
     (name "python-feather-format")

base-commit: eed55a6544d5bda2245ec853e5fa4b28e1865bea
prerequisite-patch-id: 5e9e341f5896c5eee0f64a7a031cc57491e76eb9
prerequisite-patch-id: b17d63ac48fd66475dda11cd37f74bb63aca5cfb
-- 
2.40.1





Information forwarded to guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Tue, 06 Jun 2023 08:42:02 GMT) Full text and rfc822 format available.

Message #13 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <atai <at> atai.org>
To: 63897 <at> debbugs.gnu.org
Cc: Andy Tai <atai <at> atai.org>
Subject: [PATCH v3] gnu: tensorflow-lite: Update to 2.12.0
Date: Tue,  6 Jun 2023 01:41:32 -0700
* gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable

* gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.0
  [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to
  find packages already in Guix as dependencies for absl-cpp, eigen,
  flatbuffer, neon2ssl, cpuinfo, ruy.
  [arguments] (phases): Remove operations setting up absl-cpp, eigen, ruy,
  and neon2ssl as native inouts as now we use Guix's packages of these
  as dependencies. Do default build phase.
  Add steps to install C shared library and benchmark_model tool
  [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl,
  pthreadpool, ruy, vulkan and xnnpack as explicit inputs
  [arguments] (native-inputs): Remove local setup of neon2ssl and ruy
  as we now use Guix's packages of these as explicit dependencies

* gnu/packages/maths.scm: Remove extra statement

* gnu/packages/serialization.scm (flatbuffers-next-shared): New variable,
  flatbuffers-next built with -fPIC as needed by tensorflow-lite
---
 gnu/packages/algebra.scm          |  32 ------
 gnu/packages/machine-learning.scm | 160 +++++++++++++++++-------------
 gnu/packages/maths.scm            |   1 -
 gnu/packages/serialization.scm    |  12 +++
 4 files changed, 101 insertions(+), 104 deletions(-)

diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm
index 3babf90b3c..81df0c97ac 100644
--- a/gnu/packages/algebra.scm
+++ b/gnu/packages/algebra.scm
@@ -1155,38 +1155,6 @@ (define-public eigen-for-tensorflow
                  (substitute* "test/CMakeLists.txt"
                    (("ei_add_test\\(stddeque") "#")))))))))))
 
-(define-public eigen-for-tensorflow-lite
-  ;; This commit was taken from
-  ;; tensorflow/lite/tools/cmake/modules/eigen.cmake
-  (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9")
-        (revision "1"))
-    (package (inherit eigen)
-      (name "eigen-for-tensorflow-lite")
-      (version (git-version "3.3.7" revision commit))
-      (source (origin
-                (method git-fetch)
-                (uri (git-reference
-                      (url "https://gitlab.com/libeigen/eigen")
-                      (commit commit)))
-                (sha256
-                 (base32
-                  "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf"))
-                (file-name (git-file-name name version))
-                (modules '((guix build utils)))
-                (snippet
-                 ;; Ther are test failures in the "unsupported" directory, but
-                 ;; maintainers say it's unsupported anyway, so just skip
-                 ;; them.
-                 '(begin
-                    (substitute* "unsupported/CMakeLists.txt"
-                      (("add_subdirectory\\(test.*")
-                       "# Do not build the tests for unsupported features.\n"))))))
-      (arguments
-       (substitute-keyword-arguments (package-arguments eigen)
-         ((#:phases phases)
-          `(modify-phases ,phases
-             (delete 'disable-some-tests))))))))
-
 (define-public xtensor
   (package
     (name "xtensor")
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index c9bc5e8b68..1f9ffc8ede 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -54,6 +54,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages)
   #:use-module (gnu packages adns)
   #:use-module (gnu packages algebra)
+  #:use-module (gnu packages assembly)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
@@ -88,6 +89,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages ninja)
   #:use-module (gnu packages ocaml)
   #:use-module (gnu packages onc-rpc)
+  #:use-module (gnu packages opencl)
   #:use-module (gnu packages parallel)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
@@ -107,6 +109,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages swig)
   #:use-module (gnu packages time)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages vulkan)
   #:use-module (gnu packages video)
   #:use-module (gnu packages web)
   #:use-module (gnu packages xml)
@@ -2642,7 +2645,7 @@ (define-public tensorflow
 (define-public tensorflow-lite
   (package
     (name "tensorflow-lite")
-    (version "2.5.0")
+    (version "2.12.0")
     (source
      (origin
        (method git-fetch)
@@ -2652,59 +2655,80 @@ (define-public tensorflow-lite
        (file-name (git-file-name name version))
        (sha256
         (base32
-         "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l"))))
+         "051knmq6ysg4936d9baaqsx6j3rfsp4ffi2pwrxx4llan682rvpi"))))
     (build-system cmake-build-system)
     (arguments
      `(#:tests? #false                  ; no "check" target
        #:build-type "Release"
+
+       #:modules ((ice-9 match)
+                  (guix build utils)
+                  (guix build cmake-build-system))
+
        #:configure-flags
        (list
-        "-DTFLITE_ENABLE_GPU=OFF"
-        "-DTFLITE_ENABLE_RUY=OFF"
+       	;; so cmake can be used to find this from other packages
+       	"-DTFLITE_ENABLE_INSTALL=ON"
+
+       	;; Use Guix's own packages as dependencies.
+       	"-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON"
+
+        "-DTFLITE_ENABLE_GPU=ON"
+        "-DTFLITE_ENABLE_RUY=ON"
+
+        ;; TO DO: turn on Farmhash
+        ;"-DSYSTEM_FARMHASH=ON"
+
+
+        (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp")
+        	"/lib/cmake/absl")
+        (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen")
+        	"/share/eigen3/cmake")
+        (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers")
+        	"/lib/cmake/flatbuffers")
+        (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse")
+        	"/lib/cmake/NEON_2_SSE")
+        (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo")
+        	"/share/cpuinfo")
+        (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy")
+        	"/lib/cmake/ruy")
 
         ;; TODO: The build system attempts to build xnnpack from source.  We
         ;; would like to use our xnnpack package here, but this requires more
         ;; work.
         "-DTFLITE_ENABLE_XNNPACK=OFF"
 
-        ;; Pretend we've already fetched abseil.  We won't actually build it
-        ;; but use the existing package.
-        "-Dabseil-cpp_POPULATED=TRUE"
-
-        ;; Don't fetch the sources.  We have already built flatbuffers.
-        "-Dflatbuffers_POPULATED=TRUE"
+        ;;; Don't fetch the sources.  We have these already
+        "-Degl_headers_POPULATED=TRUE"
+        "-Dfp16_headers_POPULATED=TRUE"
+        "-Dopencl_headers_POPULATED=TRUE"
+        "-Dopengl_headers_POPULATED=TRUE"
+        "-Dvulkan_headers_POPULATED=TRUE"
 
         "-DFFT2D_SOURCE_DIR=/tmp/fft2d"
-        "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse"
-        "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin"
         "-DFARMHASH_SOURCE_DIR=/tmp/farmhash"
         "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp"
-        (string-append "-DRUY_SOURCE_DIR="
-                       (assoc-ref %build-inputs "ruy-src")))
+        )
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'chdir
            (lambda _ (chdir "tensorflow/lite")))
          (add-after 'chdir 'copy-sources
            (lambda* (#:key inputs #:allow-other-keys)
-             ;; Use external cmake finders instead of these stubs that won't
-             ;; find anything but the bundled sources.
-             (delete-file "tools/cmake/modules/Findabsl.cmake")
-             (delete-file "tools/cmake/modules/Findeigen.cmake")
 
+             ;; TO DO: properly use Guix's pthreaqdpool
+             ;; roght now not using pthreadpool because
+             ;; we are not enabling xnnpack
+             (substitute* "CMakeLists.txt"
+              (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)")
+               "if(false)"))
              (substitute* "CMakeLists.txt"
-               (("find_package\\(eigen REQUIRED")
-                "find_package(eigen REQUIRED NAMES Eigen3"))
-             (substitute* "tools/cmake/modules/Findflatbuffers.cmake"
-               (("get_target_property.*")
-                (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n"
-                        (assoc-ref inputs "flatbuffers"))))
+              (("if\\(NOT TARGET pthreadpool\\)")
+               "if(false)"))
 
              ;; Don't fetch source code; we already have everything we need.
              (substitute* '("tools/cmake/modules/fft2d.cmake"
-                            "tools/cmake/modules/ruy.cmake"
                             "tools/cmake/modules/farmhash.cmake"
-                            "tools/cmake/modules/neon2sse.cmake"
                             "tools/cmake/modules/gemmlowp.cmake")
                (("OverridableFetchContent_Populate.*") ""))
 
@@ -2718,29 +2742,46 @@ (define-public tensorflow-lite
                (invoke "tar" "--strip-components=1"
                        "-xf" (assoc-ref inputs "fft2d-src")))
 
-             (copy-recursively (assoc-ref inputs "neon2sse-src")
-                               "/tmp/neon2sse/")
              (copy-recursively (assoc-ref inputs "gemmlowp-src")
                                "/tmp/gemmlowp/")))
-         (add-after 'copy-sources 'prepare-shared-library-build
-           (lambda _ (chdir "c")))
-         (replace 'install
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((out (assoc-ref outputs "out"))
-                    (lib (string-append out "/lib"))
-                    (headers (string-append out "/include/tensorflow/lite")))
-               (install-file "../build/libtensorflowlite_c.so" lib)
-               (with-directory-excursion ".."
-                 (for-each
-                  (lambda (file)
-                    (let ((target-dir (string-append headers "/" (dirname file))))
-                      (install-file file target-dir)))
-                  (find-files "." "\\.h$")))))))))
+
+         (add-after 'build 'build-shared-library
+             (lambda* (#:key configure-flags #:allow-other-keys)
+             	 (mkdir-p "c")
+                 (with-directory-excursion "c"
+                   (apply invoke "cmake" (append configure-flags (list "../../lite/c")))
+                   (invoke "cmake" "--build" "." "-j" (number->string
+                                              (parallel-job-count))))))
+         (add-after 'build-shared-library 'build-benchmark-model
+             (lambda _
+                 (invoke "cmake" "--build" "." "--target" "benchmark_model"
+                 	 "-j" (number->string (parallel-job-count)))))
+
+         (add-after 'install 'install-extra
+         	 (lambda* (#:key outputs #:allow-other-keys)
+         	 	 (let* ((out (assoc-ref outputs "out"))
+                   (lib (string-append out "/lib"))
+                   (bin (string-append out "/bin")))
+                 (install-file "../build/c/libtensorflowlite_c.so" lib)
+                 (install-file "../build/tools/benchmark/benchmark_model" bin))))
+      )))
     (inputs
      `(("abseil-cpp" ,abseil-cpp-20200923.3)
-       ("eigen" ,eigen-for-tensorflow-lite)
-       ("flatbuffers" ,flatbuffers)
-       ("python" ,python)))
+       ("cpuinfo" ,cpuinfo)
+       ("eigen" ,eigen)
+       ("fp16" ,fp16)
+       ("flatbuffers" ,flatbuffers-next-shared)
+       ;("gemmlowp" ,gemmlowp)  ;; TO DO
+       ("mesa-headers"  ,mesa-headers)
+       ("neon2sse" ,neon2sse)
+       ("opencl-clhpp" ,opencl-clhpp)
+       ("opencl-headers" ,opencl-headers)
+       ("opencl-icd-loader" ,opencl-icd-loader)
+       ("pthreadpool" ,pthreadpool)
+       ("python" ,python)
+       ("ruy" ,ruy)
+       ;("xnnpack" ,xnnpack)     ;; TO DO: properly use Guix's copy of xnnpack
+       ("vulkan-headers" , vulkan-headers)))
     (native-inputs
      `(("pkg-config" ,pkg-config)
        ("gemmlowp-src"
@@ -2756,17 +2797,6 @@ (define-public tensorflow-lite
              (sha256
               (base32
                "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl")))))
-       ("neon2sse-src"
-        ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/intel/ARM_NEON_2_x86_SSE")
-                   (commit commit)))
-             (file-name (git-file-name "neon2sse" (string-take commit 8)))
-             (sha256
-              (base32
-               "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i")))))
        ("farmhash-src"
         ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45"))
            (origin
@@ -2782,24 +2812,12 @@ (define-public tensorflow-lite
         ,(origin
            (method url-fetch)
            (uri (string-append "https://storage.googleapis.com/"
-                               "mirror.tensorflow.org/"
-                               "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz"))
+           		   "mirror.tensorflow.org/github.com/petewarden/"
+           		   "OouraFFT/archive/v1.0.tar.gz"))
            (file-name "fft2d.tar.gz")
            (sha256
             (base32
-             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))
-       ("ruy-src"
-        ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/google/ruy")
-                   (commit commit)
-                   (recursive? #true)))
-             (file-name (git-file-name "ruy" (string-take commit 8)))
-             (sha256
-              (base32
-               "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis")))))))
+             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))))
     (home-page "https://tensorflow.org")
     (synopsis "Machine learning framework")
     (description
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 7904d9e8e3..37480a8397 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -85,7 +85,6 @@ (define-module (gnu packages maths)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
-  #:use-module (guix gexp)
   #:use-module (guix utils)
   #:use-module ((guix build utils) #:select (alist-replace))
   #:use-module (guix build-system ant)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 39954d6764..268a5d3d9b 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -748,6 +748,18 @@ (define-public flatbuffers-next
                (base32
                 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
 
+(define-public flatbuffers-next-shared
+  (package
+    (inherit flatbuffers-next)
+    (name "flatbuffers-shared")
+    (version "23.1.21")
+    (arguments
+     (substitute-keyword-arguments (package-arguments flatbuffers-next)
+       ((#:configure-flags  flags)
+         ;; make C++ fiels compiled with -fPIC, needed for shared lib".
+              #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
+
+
 (define-public python-feather-format
   (package
     (name "python-feather-format")

base-commit: 8937898af9433c0b7a8d95a78c95e21576ae22e8
prerequisite-patch-id: 6a623f7c3bdc733b110fd6f59f31032adb7d987d
prerequisite-patch-id: 5e9e341f5896c5eee0f64a7a031cc57491e76eb9
-- 
2.40.1





Added blocking bug(s) 56202 Request was from Andy Tai <atai <at> atai.org> to control <at> debbugs.gnu.org. (Fri, 16 Jun 2023 05:37:02 GMT) Full text and rfc822 format available.

Information forwarded to guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Fri, 16 Jun 2023 06:16:02 GMT) Full text and rfc822 format available.

Message #18 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <atai <at> atai.org>
To: 63897 <at> debbugs.gnu.org
Cc: Andy Tai <atai <at> atai.org>
Subject: [PATCH v4] gnu: tensorflow-lite: Update to 2.12.0
Date: Thu, 15 Jun 2023 23:15:09 -0700
* gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable

* gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.0
  [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to
  find packages already in Guix as dependencies for absl-cpp, eigen,
  flatbuffer, neon2ssl, cpuinfo, ruy.
  [arguments] (phases): Remove operations setting up absl-cpp, eigen, ruy,
  and neon2ssl as native inouts as now we use Guix's packages of these
  as dependencies. Do default build phase.
  Add steps to install C shared library and benchmark_model tool
  [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl,
  pthreadpool, ruy, vulkan and xnnpack as explicit inputs
  [arguments] (native-inputs): Remove local setup of neon2ssl and ruy
  as we now use Guix's packages of these as explicit dependencies

* gnu/packages/maths.scm: Remove extra statement

* gnu/packages/serialization.scm (flatbuffers-next-shared): New variable,
  flatbuffers-next built with -fPIC as needed by tensorflow-lite
---
 gnu/packages/algebra.scm          |  32 ------
 gnu/packages/machine-learning.scm | 177 ++++++++++++++++++------------
 gnu/packages/maths.scm            |   1 -
 gnu/packages/serialization.scm    |  12 ++
 4 files changed, 117 insertions(+), 105 deletions(-)

diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm
index 3babf90b3c..81df0c97ac 100644
--- a/gnu/packages/algebra.scm
+++ b/gnu/packages/algebra.scm
@@ -1155,38 +1155,6 @@ (define-public eigen-for-tensorflow
                  (substitute* "test/CMakeLists.txt"
                    (("ei_add_test\\(stddeque") "#")))))))))))
 
-(define-public eigen-for-tensorflow-lite
-  ;; This commit was taken from
-  ;; tensorflow/lite/tools/cmake/modules/eigen.cmake
-  (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9")
-        (revision "1"))
-    (package (inherit eigen)
-      (name "eigen-for-tensorflow-lite")
-      (version (git-version "3.3.7" revision commit))
-      (source (origin
-                (method git-fetch)
-                (uri (git-reference
-                      (url "https://gitlab.com/libeigen/eigen")
-                      (commit commit)))
-                (sha256
-                 (base32
-                  "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf"))
-                (file-name (git-file-name name version))
-                (modules '((guix build utils)))
-                (snippet
-                 ;; Ther are test failures in the "unsupported" directory, but
-                 ;; maintainers say it's unsupported anyway, so just skip
-                 ;; them.
-                 '(begin
-                    (substitute* "unsupported/CMakeLists.txt"
-                      (("add_subdirectory\\(test.*")
-                       "# Do not build the tests for unsupported features.\n"))))))
-      (arguments
-       (substitute-keyword-arguments (package-arguments eigen)
-         ((#:phases phases)
-          `(modify-phases ,phases
-             (delete 'disable-some-tests))))))))
-
 (define-public xtensor
   (package
     (name "xtensor")
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 927b531cf2..4b915e509e 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -54,12 +54,14 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages)
   #:use-module (gnu packages adns)
   #:use-module (gnu packages algebra)
+  #:use-module (gnu packages assembly)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages boost)
   #:use-module (gnu packages bdw-gc)
+  #:use-module (gnu packages c)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages cmake)
@@ -88,6 +90,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages ninja)
   #:use-module (gnu packages ocaml)
   #:use-module (gnu packages onc-rpc)
+  #:use-module (gnu packages opencl)
   #:use-module (gnu packages parallel)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
@@ -99,6 +102,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages python-science)
   #:use-module (gnu packages python-web)
   #:use-module (gnu packages python-xyz)
+  #:use-module (gnu packages regex)
   #:use-module (gnu packages rpc)
   #:use-module (gnu packages serialization)
   #:use-module (gnu packages sphinx)
@@ -107,6 +111,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages swig)
   #:use-module (gnu packages time)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages vulkan)
   #:use-module (gnu packages video)
   #:use-module (gnu packages web)
   #:use-module (gnu packages xml)
@@ -2833,7 +2838,7 @@ (define-public tensorflow
 (define-public tensorflow-lite
   (package
     (name "tensorflow-lite")
-    (version "2.5.0")
+    (version "2.12.0")
     (source
      (origin
        (method git-fetch)
@@ -2843,59 +2848,85 @@ (define-public tensorflow-lite
        (file-name (git-file-name name version))
        (sha256
         (base32
-         "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l"))))
+         "051knmq6ysg4936d9baaqsx6j3rfsp4ffi2pwrxx4llan682rvpi"))))
     (build-system cmake-build-system)
     (arguments
-     `(#:tests? #false                  ; no "check" target
+     `(#:tests? #false                  ;tests not building now
        #:build-type "Release"
+
+       #:modules ((ice-9 match)
+                  (guix build utils)
+                  (guix build cmake-build-system))
+
        #:configure-flags
        (list
-        "-DTFLITE_ENABLE_GPU=OFF"
-        "-DTFLITE_ENABLE_RUY=OFF"
+       	; "-DTFLITE_KERNEL_TEST=ON"  ; TO DO: build tests
+       	;; so cmake can be used to find this from other packages
+       	"-DTFLITE_ENABLE_INSTALL=ON"
+
+       	;; Use Guix's own packages as dependencies.
+       	"-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON"
+
+        "-DTFLITE_ENABLE_GPU=ON"
+        "-DTFLITE_ENABLE_RUY=ON"
+
+        ;; TO DO: turn on Farmhash
+        ;"-DSYSTEM_FARMHASH=ON"
+
+
+        (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp")
+        	"/lib/cmake/absl")
+        (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen")
+        	"/share/eigen3/cmake")
+        (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers")
+        	"/lib/cmake/flatbuffers")
+        (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse")
+        	"/lib/cmake/NEON_2_SSE")
+        (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo")
+        	"/share/cpuinfo")
+        (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy")
+        	"/lib/cmake/ruy")
 
         ;; TODO: The build system attempts to build xnnpack from source.  We
         ;; would like to use our xnnpack package here, but this requires more
         ;; work.
         "-DTFLITE_ENABLE_XNNPACK=OFF"
 
-        ;; Pretend we've already fetched abseil.  We won't actually build it
-        ;; but use the existing package.
-        "-Dabseil-cpp_POPULATED=TRUE"
-
-        ;; Don't fetch the sources.  We have already built flatbuffers.
-        "-Dflatbuffers_POPULATED=TRUE"
+        ;;; Don't fetch the sources.  We have these already
+        "-Degl_headers_POPULATED=TRUE"
+        "-Dfp16_headers_POPULATED=TRUE"
+        "-Dopencl_headers_POPULATED=TRUE"
+        "-Dopengl_headers_POPULATED=TRUE"
+        "-Dvulkan_headers_POPULATED=TRUE"
+        "-Dgoogletest_POPULATED=TRUE"
+        "-Dgoogle_benchmark_POPULATED=TRUE"
+        "-Dnsync_POPULATED=TRUE"
+        "-Dre2_POPULATED=TRUE"
 
         "-DFFT2D_SOURCE_DIR=/tmp/fft2d"
-        "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse"
-        "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin"
         "-DFARMHASH_SOURCE_DIR=/tmp/farmhash"
         "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp"
-        (string-append "-DRUY_SOURCE_DIR="
-                       (assoc-ref %build-inputs "ruy-src")))
+        )
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'chdir
            (lambda _ (chdir "tensorflow/lite")))
          (add-after 'chdir 'copy-sources
            (lambda* (#:key inputs #:allow-other-keys)
-             ;; Use external cmake finders instead of these stubs that won't
-             ;; find anything but the bundled sources.
-             (delete-file "tools/cmake/modules/Findabsl.cmake")
-             (delete-file "tools/cmake/modules/Findeigen.cmake")
 
+             ;; TO DO: properly use Guix's pthreaqdpool
+             ;; roght now not using pthreadpool because
+             ;; we are not enabling xnnpack
              (substitute* "CMakeLists.txt"
-               (("find_package\\(eigen REQUIRED")
-                "find_package(eigen REQUIRED NAMES Eigen3"))
-             (substitute* "tools/cmake/modules/Findflatbuffers.cmake"
-               (("get_target_property.*")
-                (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n"
-                        (assoc-ref inputs "flatbuffers"))))
+              (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)")
+               "if(false)"))
+             (substitute* "CMakeLists.txt"
+              (("if\\(NOT TARGET pthreadpool\\)")
+               "if(false)"))
 
              ;; Don't fetch source code; we already have everything we need.
              (substitute* '("tools/cmake/modules/fft2d.cmake"
-                            "tools/cmake/modules/ruy.cmake"
                             "tools/cmake/modules/farmhash.cmake"
-                            "tools/cmake/modules/neon2sse.cmake"
                             "tools/cmake/modules/gemmlowp.cmake")
                (("OverridableFetchContent_Populate.*") ""))
 
@@ -2909,31 +2940,56 @@ (define-public tensorflow-lite
                (invoke "tar" "--strip-components=1"
                        "-xf" (assoc-ref inputs "fft2d-src")))
 
-             (copy-recursively (assoc-ref inputs "neon2sse-src")
-                               "/tmp/neon2sse/")
              (copy-recursively (assoc-ref inputs "gemmlowp-src")
                                "/tmp/gemmlowp/")))
-         (add-after 'copy-sources 'prepare-shared-library-build
-           (lambda _ (chdir "c")))
-         (replace 'install
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((out (assoc-ref outputs "out"))
-                    (lib (string-append out "/lib"))
-                    (headers (string-append out "/include/tensorflow/lite")))
-               (install-file "../build/libtensorflowlite_c.so" lib)
-               (with-directory-excursion ".."
-                 (for-each
-                  (lambda (file)
-                    (let ((target-dir (string-append headers "/" (dirname file))))
-                      (install-file file target-dir)))
-                  (find-files "." "\\.h$")))))))))
+
+         (add-after 'build 'build-shared-library
+             (lambda* (#:key configure-flags #:allow-other-keys)
+             	 (mkdir-p "c")
+                 (with-directory-excursion "c"
+                   (apply invoke "cmake" (append configure-flags (list "../../lite/c")))
+                   (invoke "cmake" "--build" "." "-j" (number->string
+                                              (parallel-job-count))))))
+         (add-after 'build-shared-library 'build-benchmark-model
+             (lambda _
+                 (invoke "cmake" "--build" "." "--target" "benchmark_model"
+                 	 "-j" (number->string (parallel-job-count)))))
+
+         (add-after 'install 'install-extra
+         	 (lambda* (#:key outputs #:allow-other-keys)
+         	 	 (let* ((out (assoc-ref outputs "out"))
+                   (lib (string-append out "/lib"))
+                   (bin (string-append out "/bin")))
+                 (install-file "../build/c/libtensorflowlite_c.so" lib)
+                 (install-file "../build/tools/benchmark/benchmark_model" bin))))
+         ; TO DO: test not building now
+      ;   (replace 'check
+      ;      (lambda* (#:key tests? #:allow-other-keys)
+      ;        (when tests?
+      ;          (invoke "ctest" "-L" "plain"))))
+      )))
     (inputs
      `(("abseil-cpp" ,abseil-cpp-20200923.3)
-       ("eigen" ,eigen-for-tensorflow-lite)
-       ("flatbuffers" ,flatbuffers)
-       ("python" ,python)))
+       ("cpuinfo" ,cpuinfo)
+       ("eigen" ,eigen)
+       ("fp16" ,fp16)
+       ("flatbuffers" ,flatbuffers-next-shared)
+       ;("gemmlowp" ,gemmlowp)  ;; TO DO
+       ("mesa-headers"  ,mesa-headers)
+       ("neon2sse" ,neon2sse)
+       ("nsync" ,nsync)
+       ("opencl-clhpp" ,opencl-clhpp)
+       ("opencl-headers" ,opencl-headers)
+       ("opencl-icd-loader" ,opencl-icd-loader)
+       ("pthreadpool" ,pthreadpool)
+       ("python" ,python)
+       ("ruy" ,ruy)
+       ("re2" ,re2)
+       ;("xnnpack" ,xnnpack)     ;; TO DO: properly use Guix's copy of xnnpack
+       ("vulkan-headers" , vulkan-headers)))
     (native-inputs
      `(("pkg-config" ,pkg-config)
+       ("googletest" ,googletest)
        ("gemmlowp-src"
         ;; The commit hash is taken from
         ;; "tensorflow/lite/tools/cmake/modules/gemmlowp.cmake".
@@ -2947,17 +3003,6 @@ (define-public tensorflow-lite
              (sha256
               (base32
                "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl")))))
-       ("neon2sse-src"
-        ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/intel/ARM_NEON_2_x86_SSE")
-                   (commit commit)))
-             (file-name (git-file-name "neon2sse" (string-take commit 8)))
-             (sha256
-              (base32
-               "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i")))))
        ("farmhash-src"
         ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45"))
            (origin
@@ -2973,24 +3018,12 @@ (define-public tensorflow-lite
         ,(origin
            (method url-fetch)
            (uri (string-append "https://storage.googleapis.com/"
-                               "mirror.tensorflow.org/"
-                               "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz"))
+           		   "mirror.tensorflow.org/github.com/petewarden/"
+           		   "OouraFFT/archive/v1.0.tar.gz"))
            (file-name "fft2d.tar.gz")
            (sha256
             (base32
-             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))
-       ("ruy-src"
-        ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/google/ruy")
-                   (commit commit)
-                   (recursive? #true)))
-             (file-name (git-file-name "ruy" (string-take commit 8)))
-             (sha256
-              (base32
-               "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis")))))))
+             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))))
     (home-page "https://tensorflow.org")
     (synopsis "Machine learning framework")
     (description
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index fd0e8eae3a..c7996e12a9 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -85,7 +85,6 @@ (define-module (gnu packages maths)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
-  #:use-module (guix gexp)
   #:use-module (guix utils)
   #:use-module ((guix build utils) #:select (alist-replace))
   #:use-module (guix build-system ant)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 39954d6764..268a5d3d9b 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -748,6 +748,18 @@ (define-public flatbuffers-next
                (base32
                 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
 
+(define-public flatbuffers-next-shared
+  (package
+    (inherit flatbuffers-next)
+    (name "flatbuffers-shared")
+    (version "23.1.21")
+    (arguments
+     (substitute-keyword-arguments (package-arguments flatbuffers-next)
+       ((#:configure-flags  flags)
+         ;; make C++ fiels compiled with -fPIC, needed for shared lib".
+              #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
+
+
 (define-public python-feather-format
   (package
     (name "python-feather-format")

base-commit: 31336e9f5d68512a9c1c6826bce9f17c892a2125
prerequisite-patch-id: 5e9e341f5896c5eee0f64a7a031cc57491e76eb9
prerequisite-patch-id: 6a623f7c3bdc733b110fd6f59f31032adb7d987d
prerequisite-patch-id: f672f4219dc90516d7e83bc2801e4a8dd51a9839
-- 
2.40.1





Information forwarded to andreas <at> enge.fr, efraim <at> flashner.co.il, bavier <at> posteo.net, guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Fri, 28 Jul 2023 08:45:01 GMT) Full text and rfc822 format available.

Message #21 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <atai <at> atai.org>
To: 63897 <at> debbugs.gnu.org
Cc: Andy Tai <atai <at> atai.org>
Subject: [PATCH v5] gnu: tensorflow-lite: Update to 2.12.1
Date: Fri, 28 Jul 2023 01:44:14 -0700
* gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable

* gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.1
  [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to
  find packages already in Guix as dependencies for absl-cpp, eigen,
  flatbuffer, neon2ssl, cpuinfo, ruy.
  [arguments] (phases): Remove operations setting up absl-cpp, eigen, ruy,
  and neon2ssl as native inouts as now we use Guix's packages of these
  as dependencies. Do default build phase.
  Add steps to install C shared library and benchmark_model tool
  [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl,
  pthreadpool, ruy, vulkan and xnnpack as explicit inputs
  [arguments] (native-inputs): Remove local setup of neon2ssl and ruy
  as we now use Guix's packages of these as explicit dependencies

* gnu/packages/maths.scm: Remove extra statement

* gnu/packages/serialization.scm (flatbuffers-next-shared): New variable,
  flatbuffers-next built with -fPIC as needed by tensorflow-lite
---
 gnu/packages/algebra.scm          |  32 ------
 gnu/packages/machine-learning.scm | 177 ++++++++++++++++++------------
 gnu/packages/maths.scm            |   1 -
 gnu/packages/serialization.scm    |  12 ++
 4 files changed, 117 insertions(+), 105 deletions(-)

diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm
index a717750c7b..b1ab755144 100644
--- a/gnu/packages/algebra.scm
+++ b/gnu/packages/algebra.scm
@@ -1154,38 +1154,6 @@ (define-public eigen-for-tensorflow
                  (substitute* "test/CMakeLists.txt"
                    (("ei_add_test\\(stddeque") "#")))))))))))
 
-(define-public eigen-for-tensorflow-lite
-  ;; This commit was taken from
-  ;; tensorflow/lite/tools/cmake/modules/eigen.cmake
-  (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9")
-        (revision "1"))
-    (package (inherit eigen)
-      (name "eigen-for-tensorflow-lite")
-      (version (git-version "3.3.7" revision commit))
-      (source (origin
-                (method git-fetch)
-                (uri (git-reference
-                      (url "https://gitlab.com/libeigen/eigen")
-                      (commit commit)))
-                (sha256
-                 (base32
-                  "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf"))
-                (file-name (git-file-name name version))
-                (modules '((guix build utils)))
-                (snippet
-                 ;; Ther are test failures in the "unsupported" directory, but
-                 ;; maintainers say it's unsupported anyway, so just skip
-                 ;; them.
-                 '(begin
-                    (substitute* "unsupported/CMakeLists.txt"
-                      (("add_subdirectory\\(test.*")
-                       "# Do not build the tests for unsupported features.\n"))))))
-      (arguments
-       (substitute-keyword-arguments (package-arguments eigen)
-         ((#:phases phases)
-          `(modify-phases ,phases
-             (delete 'disable-some-tests))))))))
-
 (define-public xtensor
   (package
     (name "xtensor")
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index e8225c76f3..6ae9c52625 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -55,12 +55,14 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages)
   #:use-module (gnu packages adns)
   #:use-module (gnu packages algebra)
+  #:use-module (gnu packages assembly)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages boost)
   #:use-module (gnu packages bdw-gc)
+  #:use-module (gnu packages c)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages cmake)
@@ -89,6 +91,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages ninja)
   #:use-module (gnu packages ocaml)
   #:use-module (gnu packages onc-rpc)
+  #:use-module (gnu packages opencl)
   #:use-module (gnu packages parallel)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
@@ -100,6 +103,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages python-science)
   #:use-module (gnu packages python-web)
   #:use-module (gnu packages python-xyz)
+  #:use-module (gnu packages regex)
   #:use-module (gnu packages rpc)
   #:use-module (gnu packages serialization)
   #:use-module (gnu packages sphinx)
@@ -108,6 +112,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages swig)
   #:use-module (gnu packages time)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages vulkan)
   #:use-module (gnu packages video)
   #:use-module (gnu packages web)
   #:use-module (gnu packages xml)
@@ -2908,7 +2913,7 @@ (define-public tensorflow
 (define-public tensorflow-lite
   (package
     (name "tensorflow-lite")
-    (version "2.5.0")
+    (version "2.12.1")
     (source
      (origin
        (method git-fetch)
@@ -2918,59 +2923,85 @@ (define-public tensorflow-lite
        (file-name (git-file-name name version))
        (sha256
         (base32
-         "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l"))))
+         "0jkgljdagdqllnxygl35r5bh3f9qmbczymfj357gm9krh59g2kmd"))))
     (build-system cmake-build-system)
     (arguments
-     `(#:tests? #false                  ; no "check" target
+     `(#:tests? #false                  ;tests not building now
        #:build-type "Release"
+
+       #:modules ((ice-9 match)
+                  (guix build utils)
+                  (guix build cmake-build-system))
+
        #:configure-flags
        (list
-        "-DTFLITE_ENABLE_GPU=OFF"
-        "-DTFLITE_ENABLE_RUY=OFF"
+       	; "-DTFLITE_KERNEL_TEST=ON"  ; TO DO: build tests
+       	;; so cmake can be used to find this from other packages
+       	"-DTFLITE_ENABLE_INSTALL=ON"
+
+       	;; Use Guix's own packages as dependencies.
+       	"-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON"
+
+        "-DTFLITE_ENABLE_GPU=ON"
+        "-DTFLITE_ENABLE_RUY=ON"
+
+        ;; TO DO: turn on Farmhash
+        ;"-DSYSTEM_FARMHASH=ON"
+
+
+        (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp")
+        	"/lib/cmake/absl")
+        (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen")
+        	"/share/eigen3/cmake")
+        (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers")
+        	"/lib/cmake/flatbuffers")
+        (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse")
+        	"/lib/cmake/NEON_2_SSE")
+        (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo")
+        	"/share/cpuinfo")
+        (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy")
+        	"/lib/cmake/ruy")
 
         ;; TODO: The build system attempts to build xnnpack from source.  We
         ;; would like to use our xnnpack package here, but this requires more
         ;; work.
         "-DTFLITE_ENABLE_XNNPACK=OFF"
 
-        ;; Pretend we've already fetched abseil.  We won't actually build it
-        ;; but use the existing package.
-        "-Dabseil-cpp_POPULATED=TRUE"
-
-        ;; Don't fetch the sources.  We have already built flatbuffers.
-        "-Dflatbuffers_POPULATED=TRUE"
+        ;;; Don't fetch the sources.  We have these already
+        "-Degl_headers_POPULATED=TRUE"
+        "-Dfp16_headers_POPULATED=TRUE"
+        "-Dopencl_headers_POPULATED=TRUE"
+        "-Dopengl_headers_POPULATED=TRUE"
+        "-Dvulkan_headers_POPULATED=TRUE"
+        "-Dgoogletest_POPULATED=TRUE"
+        "-Dgoogle_benchmark_POPULATED=TRUE"
+        "-Dnsync_POPULATED=TRUE"
+        "-Dre2_POPULATED=TRUE"
 
         "-DFFT2D_SOURCE_DIR=/tmp/fft2d"
-        "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse"
-        "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin"
         "-DFARMHASH_SOURCE_DIR=/tmp/farmhash"
         "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp"
-        (string-append "-DRUY_SOURCE_DIR="
-                       (assoc-ref %build-inputs "ruy-src")))
+        )
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'chdir
            (lambda _ (chdir "tensorflow/lite")))
          (add-after 'chdir 'copy-sources
            (lambda* (#:key inputs #:allow-other-keys)
-             ;; Use external cmake finders instead of these stubs that won't
-             ;; find anything but the bundled sources.
-             (delete-file "tools/cmake/modules/Findabsl.cmake")
-             (delete-file "tools/cmake/modules/Findeigen.cmake")
 
+             ;; TO DO: properly use Guix's pthreaqdpool
+             ;; roght now not using pthreadpool because
+             ;; we are not enabling xnnpack
              (substitute* "CMakeLists.txt"
-               (("find_package\\(eigen REQUIRED")
-                "find_package(eigen REQUIRED NAMES Eigen3"))
-             (substitute* "tools/cmake/modules/Findflatbuffers.cmake"
-               (("get_target_property.*")
-                (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n"
-                        (assoc-ref inputs "flatbuffers"))))
+              (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)")
+               "if(false)"))
+             (substitute* "CMakeLists.txt"
+              (("if\\(NOT TARGET pthreadpool\\)")
+               "if(false)"))
 
              ;; Don't fetch source code; we already have everything we need.
              (substitute* '("tools/cmake/modules/fft2d.cmake"
-                            "tools/cmake/modules/ruy.cmake"
                             "tools/cmake/modules/farmhash.cmake"
-                            "tools/cmake/modules/neon2sse.cmake"
                             "tools/cmake/modules/gemmlowp.cmake")
                (("OverridableFetchContent_Populate.*") ""))
 
@@ -2984,31 +3015,56 @@ (define-public tensorflow-lite
                (invoke "tar" "--strip-components=1"
                        "-xf" (assoc-ref inputs "fft2d-src")))
 
-             (copy-recursively (assoc-ref inputs "neon2sse-src")
-                               "/tmp/neon2sse/")
              (copy-recursively (assoc-ref inputs "gemmlowp-src")
                                "/tmp/gemmlowp/")))
-         (add-after 'copy-sources 'prepare-shared-library-build
-           (lambda _ (chdir "c")))
-         (replace 'install
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((out (assoc-ref outputs "out"))
-                    (lib (string-append out "/lib"))
-                    (headers (string-append out "/include/tensorflow/lite")))
-               (install-file "../build/libtensorflowlite_c.so" lib)
-               (with-directory-excursion ".."
-                 (for-each
-                  (lambda (file)
-                    (let ((target-dir (string-append headers "/" (dirname file))))
-                      (install-file file target-dir)))
-                  (find-files "." "\\.h$")))))))))
+
+         (add-after 'build 'build-shared-library
+             (lambda* (#:key configure-flags #:allow-other-keys)
+             	 (mkdir-p "c")
+                 (with-directory-excursion "c"
+                   (apply invoke "cmake" (append configure-flags (list "../../lite/c")))
+                   (invoke "cmake" "--build" "." "-j" (number->string
+                                              (parallel-job-count))))))
+         (add-after 'build-shared-library 'build-benchmark-model
+             (lambda _
+                 (invoke "cmake" "--build" "." "--target" "benchmark_model"
+                 	 "-j" (number->string (parallel-job-count)))))
+
+         (add-after 'install 'install-extra
+         	 (lambda* (#:key outputs #:allow-other-keys)
+         	 	 (let* ((out (assoc-ref outputs "out"))
+                   (lib (string-append out "/lib"))
+                   (bin (string-append out "/bin")))
+                 (install-file "../build/c/libtensorflowlite_c.so" lib)
+                 (install-file "../build/tools/benchmark/benchmark_model" bin))))
+         ; TO DO: test not building now
+      ;   (replace 'check
+      ;      (lambda* (#:key tests? #:allow-other-keys)
+      ;        (when tests?
+      ;          (invoke "ctest" "-L" "plain"))))
+      )))
     (inputs
      `(("abseil-cpp" ,abseil-cpp-20200923.3)
-       ("eigen" ,eigen-for-tensorflow-lite)
-       ("flatbuffers" ,flatbuffers)
-       ("python" ,python)))
+       ("cpuinfo" ,cpuinfo)
+       ("eigen" ,eigen)
+       ("fp16" ,fp16)
+       ("flatbuffers" ,flatbuffers-next-shared)
+       ;("gemmlowp" ,gemmlowp)  ;; TO DO
+       ("mesa-headers"  ,mesa-headers)
+       ("neon2sse" ,neon2sse)
+       ("nsync" ,nsync)
+       ("opencl-clhpp" ,opencl-clhpp)
+       ("opencl-headers" ,opencl-headers)
+       ("opencl-icd-loader" ,opencl-icd-loader)
+       ("pthreadpool" ,pthreadpool)
+       ("python" ,python)
+       ("ruy" ,ruy)
+       ("re2" ,re2)
+       ;("xnnpack" ,xnnpack)     ;; TO DO: properly use Guix's copy of xnnpack
+       ("vulkan-headers" , vulkan-headers)))
     (native-inputs
      `(("pkg-config" ,pkg-config)
+       ("googletest" ,googletest)
        ("gemmlowp-src"
         ;; The commit hash is taken from
         ;; "tensorflow/lite/tools/cmake/modules/gemmlowp.cmake".
@@ -3022,17 +3078,6 @@ (define-public tensorflow-lite
              (sha256
               (base32
                "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl")))))
-       ("neon2sse-src"
-        ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/intel/ARM_NEON_2_x86_SSE")
-                   (commit commit)))
-             (file-name (git-file-name "neon2sse" (string-take commit 8)))
-             (sha256
-              (base32
-               "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i")))))
        ("farmhash-src"
         ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45"))
            (origin
@@ -3048,24 +3093,12 @@ (define-public tensorflow-lite
         ,(origin
            (method url-fetch)
            (uri (string-append "https://storage.googleapis.com/"
-                               "mirror.tensorflow.org/"
-                               "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz"))
+           		   "mirror.tensorflow.org/github.com/petewarden/"
+           		   "OouraFFT/archive/v1.0.tar.gz"))
            (file-name "fft2d.tar.gz")
            (sha256
             (base32
-             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))
-       ("ruy-src"
-        ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/google/ruy")
-                   (commit commit)
-                   (recursive? #true)))
-             (file-name (git-file-name "ruy" (string-take commit 8)))
-             (sha256
-              (base32
-               "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis")))))))
+             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))))
     (home-page "https://tensorflow.org")
     (synopsis "Machine learning framework")
     (description
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index b2cb0a7308..4c92b22bac 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -86,7 +86,6 @@ (define-module (gnu packages maths)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
-  #:use-module (guix gexp)
   #:use-module (guix utils)
   #:use-module ((guix build utils) #:select (alist-replace))
   #:use-module (guix build-system ant)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 1f8c128eba..0a45920894 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -807,6 +807,18 @@ (define-public flatbuffers-next
                (base32
                 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
 
+(define-public flatbuffers-next-shared
+  (package
+    (inherit flatbuffers-next)
+    (name "flatbuffers-shared")
+    (version "23.1.21")
+    (arguments
+     (substitute-keyword-arguments (package-arguments flatbuffers-next)
+       ((#:configure-flags  flags)
+         ;; make C++ fiels compiled with -fPIC, needed for shared lib".
+              #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
+
+
 (define-public python-feather-format
   (package
     (name "python-feather-format")

base-commit: 35c8d631d2bd63c64e863b5509f29fdcb7f856e6
prerequisite-patch-id: 36485dd2de3da6fe04218b9a9fb621d749609628
prerequisite-patch-id: 6a623f7c3bdc733b110fd6f59f31032adb7d987d
prerequisite-patch-id: 4b8245be9f9ab7b2cf9713a404b7d1632d945278
-- 
2.41.0





Information forwarded to guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Mon, 07 Aug 2023 15:29:01 GMT) Full text and rfc822 format available.

Message #24 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Andy Tai <atai <at> atai.org>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 63897 <at> debbugs.gnu.org,
 Efraim Flashner <efraim <at> flashner.co.il>, Eric Bavier <bavier <at> posteo.net>
Subject: Re: [bug#63897] [PATCH v5] gnu: tensorflow-lite: Update to 2.12.1
Date: Mon, 7 Aug 2023 17:27:57 +0200
Hello,

it looks like I was in cc because of your touching maths.scm and/or
algebra.scm. I am definitely fine with removing eigen-for-tensorflow
if it is not needed any more!

Removal of the duplicate line in maths.scm is unrelated and should be done
in a separate commit, which I have just pushed; thanks for noticing it!

However I do not feel competent about tensorflow itself, so I am not going
to review your patch. I am adding Ricardo in cc, who has been working on the
package in the past.

Andreas





Information forwarded to andreas <at> enge.fr, efraim <at> flashner.co.il, bavier <at> posteo.net, guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Sun, 03 Sep 2023 04:56:01 GMT) Full text and rfc822 format available.

Message #27 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <atai <at> atai.org>
To: 63897 <at> debbugs.gnu.org
Cc: Andy Tai <atai <at> atai.org>
Subject: [PATCH v6] gnu: tensorflow-lite: Update to 2.12.1
Date: Sat,  2 Sep 2023 21:55:18 -0700
* gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable.

* gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.1
[arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to
find packages already in Guix as dependencies for absl-cpp, eigen,
flatbuffer, neon2ssl, cpuinfo, ruy.
[arguments] (phases): Remove operations setting up absl-cpp, eigen, ruy,
and neon2ssl as native inouts as now we use Guix's packages of these
as dependencies. Do default build phase.
Add steps to install C shared library and benchmark_model tool.
[arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl,
pthreadpool, ruy, vulkan and xnnpack as explicit inputs.
[arguments] (native-inputs): Remove local setup of neon2ssl and ruy
as we now use Guix's packages of these as explicit dependencies.

* gnu/packages/serialization.scm (flatbuffers-next-shared): New variable,
flatbuffers-next built with -fPIC as needed by tensorflow-lite.
---
 gnu/packages/algebra.scm          |  32 ------
 gnu/packages/machine-learning.scm | 177 ++++++++++++++++++------------
 gnu/packages/serialization.scm    |  12 ++
 3 files changed, 117 insertions(+), 104 deletions(-)

diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm
index a717750c7b..b1ab755144 100644
--- a/gnu/packages/algebra.scm
+++ b/gnu/packages/algebra.scm
@@ -1154,38 +1154,6 @@ (define-public eigen-for-tensorflow
                  (substitute* "test/CMakeLists.txt"
                    (("ei_add_test\\(stddeque") "#")))))))))))
 
-(define-public eigen-for-tensorflow-lite
-  ;; This commit was taken from
-  ;; tensorflow/lite/tools/cmake/modules/eigen.cmake
-  (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9")
-        (revision "1"))
-    (package (inherit eigen)
-      (name "eigen-for-tensorflow-lite")
-      (version (git-version "3.3.7" revision commit))
-      (source (origin
-                (method git-fetch)
-                (uri (git-reference
-                      (url "https://gitlab.com/libeigen/eigen")
-                      (commit commit)))
-                (sha256
-                 (base32
-                  "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf"))
-                (file-name (git-file-name name version))
-                (modules '((guix build utils)))
-                (snippet
-                 ;; Ther are test failures in the "unsupported" directory, but
-                 ;; maintainers say it's unsupported anyway, so just skip
-                 ;; them.
-                 '(begin
-                    (substitute* "unsupported/CMakeLists.txt"
-                      (("add_subdirectory\\(test.*")
-                       "# Do not build the tests for unsupported features.\n"))))))
-      (arguments
-       (substitute-keyword-arguments (package-arguments eigen)
-         ((#:phases phases)
-          `(modify-phases ,phases
-             (delete 'disable-some-tests))))))))
-
 (define-public xtensor
   (package
     (name "xtensor")
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 712abe9aff..f4d4b45084 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -55,12 +55,14 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages)
   #:use-module (gnu packages adns)
   #:use-module (gnu packages algebra)
+  #:use-module (gnu packages assembly)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages boost)
   #:use-module (gnu packages bdw-gc)
+  #:use-module (gnu packages c)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages cmake)
@@ -89,6 +91,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages ninja)
   #:use-module (gnu packages ocaml)
   #:use-module (gnu packages onc-rpc)
+  #:use-module (gnu packages opencl)
   #:use-module (gnu packages parallel)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
@@ -100,6 +103,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages python-science)
   #:use-module (gnu packages python-web)
   #:use-module (gnu packages python-xyz)
+  #:use-module (gnu packages regex)
   #:use-module (gnu packages rpc)
   #:use-module (gnu packages serialization)
   #:use-module (gnu packages sphinx)
@@ -108,6 +112,7 @@ (define-module (gnu packages machine-learning)
   #:use-module (gnu packages swig)
   #:use-module (gnu packages time)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages vulkan)
   #:use-module (gnu packages video)
   #:use-module (gnu packages web)
   #:use-module (gnu packages xml)
@@ -2936,7 +2941,7 @@ (define-public tensorflow
 (define-public tensorflow-lite
   (package
     (name "tensorflow-lite")
-    (version "2.5.0")
+    (version "2.12.1")
     (source
      (origin
        (method git-fetch)
@@ -2946,59 +2951,85 @@ (define-public tensorflow-lite
        (file-name (git-file-name name version))
        (sha256
         (base32
-         "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l"))))
+         "0jkgljdagdqllnxygl35r5bh3f9qmbczymfj357gm9krh59g2kmd"))))
     (build-system cmake-build-system)
     (arguments
-     `(#:tests? #false                  ; no "check" target
+     `(#:tests? #false                  ;tests not building now
        #:build-type "Release"
+
+       #:modules ((ice-9 match)
+                  (guix build utils)
+                  (guix build cmake-build-system))
+
        #:configure-flags
        (list
-        "-DTFLITE_ENABLE_GPU=OFF"
-        "-DTFLITE_ENABLE_RUY=OFF"
+       	; "-DTFLITE_KERNEL_TEST=ON"  ; TO DO: build tests
+       	;; so cmake can be used to find this from other packages
+       	"-DTFLITE_ENABLE_INSTALL=ON"
+
+       	;; Use Guix's own packages as dependencies.
+       	"-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON"
+
+        "-DTFLITE_ENABLE_GPU=ON"
+        "-DTFLITE_ENABLE_RUY=ON"
+
+        ;; TO DO: turn on Farmhash
+        ;"-DSYSTEM_FARMHASH=ON"
+
+
+        (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp")
+        	"/lib/cmake/absl")
+        (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen")
+        	"/share/eigen3/cmake")
+        (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers")
+        	"/lib/cmake/flatbuffers")
+        (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse")
+        	"/lib/cmake/NEON_2_SSE")
+        (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo")
+        	"/share/cpuinfo")
+        (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy")
+        	"/lib/cmake/ruy")
 
         ;; TODO: The build system attempts to build xnnpack from source.  We
         ;; would like to use our xnnpack package here, but this requires more
         ;; work.
         "-DTFLITE_ENABLE_XNNPACK=OFF"
 
-        ;; Pretend we've already fetched abseil.  We won't actually build it
-        ;; but use the existing package.
-        "-Dabseil-cpp_POPULATED=TRUE"
-
-        ;; Don't fetch the sources.  We have already built flatbuffers.
-        "-Dflatbuffers_POPULATED=TRUE"
+        ;;; Don't fetch the sources.  We have these already
+        "-Degl_headers_POPULATED=TRUE"
+        "-Dfp16_headers_POPULATED=TRUE"
+        "-Dopencl_headers_POPULATED=TRUE"
+        "-Dopengl_headers_POPULATED=TRUE"
+        "-Dvulkan_headers_POPULATED=TRUE"
+        "-Dgoogletest_POPULATED=TRUE"
+        "-Dgoogle_benchmark_POPULATED=TRUE"
+        "-Dnsync_POPULATED=TRUE"
+        "-Dre2_POPULATED=TRUE"
 
         "-DFFT2D_SOURCE_DIR=/tmp/fft2d"
-        "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse"
-        "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin"
         "-DFARMHASH_SOURCE_DIR=/tmp/farmhash"
         "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp"
-        (string-append "-DRUY_SOURCE_DIR="
-                       (assoc-ref %build-inputs "ruy-src")))
+        )
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'chdir
            (lambda _ (chdir "tensorflow/lite")))
          (add-after 'chdir 'copy-sources
            (lambda* (#:key inputs #:allow-other-keys)
-             ;; Use external cmake finders instead of these stubs that won't
-             ;; find anything but the bundled sources.
-             (delete-file "tools/cmake/modules/Findabsl.cmake")
-             (delete-file "tools/cmake/modules/Findeigen.cmake")
 
+             ;; TO DO: properly use Guix's pthreaqdpool
+             ;; roght now not using pthreadpool because
+             ;; we are not enabling xnnpack
              (substitute* "CMakeLists.txt"
-               (("find_package\\(eigen REQUIRED")
-                "find_package(eigen REQUIRED NAMES Eigen3"))
-             (substitute* "tools/cmake/modules/Findflatbuffers.cmake"
-               (("get_target_property.*")
-                (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n"
-                        (assoc-ref inputs "flatbuffers"))))
+              (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)")
+               "if(false)"))
+             (substitute* "CMakeLists.txt"
+              (("if\\(NOT TARGET pthreadpool\\)")
+               "if(false)"))
 
              ;; Don't fetch source code; we already have everything we need.
              (substitute* '("tools/cmake/modules/fft2d.cmake"
-                            "tools/cmake/modules/ruy.cmake"
                             "tools/cmake/modules/farmhash.cmake"
-                            "tools/cmake/modules/neon2sse.cmake"
                             "tools/cmake/modules/gemmlowp.cmake")
                (("OverridableFetchContent_Populate.*") ""))
 
@@ -3012,31 +3043,56 @@ (define-public tensorflow-lite
                (invoke "tar" "--strip-components=1"
                        "-xf" (assoc-ref inputs "fft2d-src")))
 
-             (copy-recursively (assoc-ref inputs "neon2sse-src")
-                               "/tmp/neon2sse/")
              (copy-recursively (assoc-ref inputs "gemmlowp-src")
                                "/tmp/gemmlowp/")))
-         (add-after 'copy-sources 'prepare-shared-library-build
-           (lambda _ (chdir "c")))
-         (replace 'install
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((out (assoc-ref outputs "out"))
-                    (lib (string-append out "/lib"))
-                    (headers (string-append out "/include/tensorflow/lite")))
-               (install-file "../build/libtensorflowlite_c.so" lib)
-               (with-directory-excursion ".."
-                 (for-each
-                  (lambda (file)
-                    (let ((target-dir (string-append headers "/" (dirname file))))
-                      (install-file file target-dir)))
-                  (find-files "." "\\.h$")))))))))
+
+         (add-after 'build 'build-shared-library
+             (lambda* (#:key configure-flags #:allow-other-keys)
+             	 (mkdir-p "c")
+                 (with-directory-excursion "c"
+                   (apply invoke "cmake" (append configure-flags (list "../../lite/c")))
+                   (invoke "cmake" "--build" "." "-j" (number->string
+                                              (parallel-job-count))))))
+         (add-after 'build-shared-library 'build-benchmark-model
+             (lambda _
+                 (invoke "cmake" "--build" "." "--target" "benchmark_model"
+                 	 "-j" (number->string (parallel-job-count)))))
+
+         (add-after 'install 'install-extra
+         	 (lambda* (#:key outputs #:allow-other-keys)
+         	 	 (let* ((out (assoc-ref outputs "out"))
+                   (lib (string-append out "/lib"))
+                   (bin (string-append out "/bin")))
+                 (install-file "../build/c/libtensorflowlite_c.so" lib)
+                 (install-file "../build/tools/benchmark/benchmark_model" bin))))
+         ; TO DO: test not building now
+      ;   (replace 'check
+      ;      (lambda* (#:key tests? #:allow-other-keys)
+      ;        (when tests?
+      ;          (invoke "ctest" "-L" "plain"))))
+      )))
     (inputs
      `(("abseil-cpp" ,abseil-cpp-20200923.3)
-       ("eigen" ,eigen-for-tensorflow-lite)
-       ("flatbuffers" ,flatbuffers)
-       ("python" ,python)))
+       ("cpuinfo" ,cpuinfo)
+       ("eigen" ,eigen)
+       ("fp16" ,fp16)
+       ("flatbuffers" ,flatbuffers-next-shared)
+       ;("gemmlowp" ,gemmlowp)  ;; TO DO
+       ("mesa-headers"  ,mesa-headers)
+       ("neon2sse" ,neon2sse)
+       ("nsync" ,nsync)
+       ("opencl-clhpp" ,opencl-clhpp)
+       ("opencl-headers" ,opencl-headers)
+       ("opencl-icd-loader" ,opencl-icd-loader)
+       ("pthreadpool" ,pthreadpool)
+       ("python" ,python)
+       ("ruy" ,ruy)
+       ("re2" ,re2)
+       ;("xnnpack" ,xnnpack)     ;; TO DO: properly use Guix's copy of xnnpack
+       ("vulkan-headers" , vulkan-headers)))
     (native-inputs
      `(("pkg-config" ,pkg-config)
+       ("googletest" ,googletest)
        ("gemmlowp-src"
         ;; The commit hash is taken from
         ;; "tensorflow/lite/tools/cmake/modules/gemmlowp.cmake".
@@ -3050,17 +3106,6 @@ (define-public tensorflow-lite
              (sha256
               (base32
                "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl")))))
-       ("neon2sse-src"
-        ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/intel/ARM_NEON_2_x86_SSE")
-                   (commit commit)))
-             (file-name (git-file-name "neon2sse" (string-take commit 8)))
-             (sha256
-              (base32
-               "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i")))))
        ("farmhash-src"
         ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45"))
            (origin
@@ -3076,24 +3121,12 @@ (define-public tensorflow-lite
         ,(origin
            (method url-fetch)
            (uri (string-append "https://storage.googleapis.com/"
-                               "mirror.tensorflow.org/"
-                               "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz"))
+           		   "mirror.tensorflow.org/github.com/petewarden/"
+           		   "OouraFFT/archive/v1.0.tar.gz"))
            (file-name "fft2d.tar.gz")
            (sha256
             (base32
-             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))
-       ("ruy-src"
-        ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90"))
-           (origin
-             (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/google/ruy")
-                   (commit commit)
-                   (recursive? #true)))
-             (file-name (git-file-name "ruy" (string-take commit 8)))
-             (sha256
-              (base32
-               "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis")))))))
+             "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd"))))))
     (home-page "https://tensorflow.org")
     (synopsis "Machine learning framework")
     (description
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 9c114aaf39..0090d650ac 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -811,6 +811,18 @@ (define-public flatbuffers-next
                (base32
                 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
 
+(define-public flatbuffers-next-shared
+  (package
+    (inherit flatbuffers-next)
+    (name "flatbuffers-shared")
+    (version "23.1.21")
+    (arguments
+     (substitute-keyword-arguments (package-arguments flatbuffers-next)
+       ((#:configure-flags  flags)
+         ;; make C++ fiels compiled with -fPIC, needed for shared lib".
+              #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
+
+
 (define-public python-feather-format
   (package
     (name "python-feather-format")

base-commit: 4818ad5ba204e094fd8995c90c16ee1940e76210
prerequisite-patch-id: 4b8245be9f9ab7b2cf9713a404b7d1632d945278
prerequisite-patch-id: 6a623f7c3bdc733b110fd6f59f31032adb7d987d
-- 
2.41.0





Information forwarded to guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Tue, 05 Sep 2023 07:59:02 GMT) Full text and rfc822 format available.

Message #30 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Andy Tai <atai <at> atai.org>
Cc: 63897 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>
Subject: Re: [bug#63897] [PATCH v6] gnu: tensorflow-lite: Update to 2.12.1
Date: Tue, 05 Sep 2023 09:50:31 +0200
Hi,

The update looks good to me.  My only concerns are cosmetic:

- the indentation throughout is rather wonky.  Would be good to reformat
  this with Emacs C-M-q or “guix style”.

- “TO DO” —> “TODO” (so we can find it easily)

- please use “;;” for line comments; a single semicolon is only used for
  margin comments

- it would be good not to use “assoc-ref %build-inputs” and use G-exps
  instead.

I could make all these changes before pushing it.

Actually, there is one question I have that is not of cosmetic nature:
What’s up with the commented gemmlowp in the inputs?  We have
gemmlowp-src in the native-inputs.

If you could answer this question I can take care of the rest and apply
it.  Thank you for your patience!

-- 
Ricardo




Information forwarded to guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Tue, 05 Sep 2023 12:47:02 GMT) Full text and rfc822 format available.

Message #33 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <atai <at> atai.org>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 63897 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>
Subject: Re: [bug#63897] [PATCH v6] gnu: tensorflow-lite: Update to 2.12.1
Date: Tue, 5 Sep 2023 05:45:52 -0700
Hi, I did created a package definition in Guix for gemmlowp and that
was merged.  So I was trying to refer to that.  This is not in this
patch but feel free to get that (and everything else) right and to
push.

On Tue, Sep 5, 2023 at 12:58 AM Ricardo Wurmus <rekado <at> elephly.net> wrote:
>
> Actually, there is one question I have that is not of cosmetic nature:
> What’s up with the commented gemmlowp in the inputs?  We have
> gemmlowp-src in the native-inputs.
>
> If you could answer this question I can take care of the rest and apply
> it.  Thank you for your patience!
>
> --
> Ricardo




Information forwarded to guix-patches <at> gnu.org:
bug#63897; Package guix-patches. (Tue, 05 Sep 2023 14:56:01 GMT) Full text and rfc822 format available.

Message #36 received at 63897 <at> debbugs.gnu.org (full text, mbox):

From: Andy Tai <atai <at> atai.org>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 63897 <at> debbugs.gnu.org, Andreas Enge <andreas <at> enge.fr>
Subject: Re: [bug#63897] [PATCH v6] gnu: tensorflow-lite: Update to 2.12.1
Date: Tue, 5 Sep 2023 07:55:06 -0700
[Message part 1 (text/plain, inline)]
Also note this patch requires two other patches as prerequisites. These are
marked with block relationships in debugs but here are them for reference

https://issues.guix.gnu.org/63889#4
https://issues.guix.gnu.org/56202#5

On Tue, Sep 5, 2023 at 5:45 AM Andy Tai <atai <at> atai.org> wrote:

> Hi, I did created a package definition in Guix for gemmlowp and that
> was merged.  So I was trying to refer to that.  This is not in this
> patch but feel free to get that (and everything else) right and to
> push.
>
> On Tue, Sep 5, 2023 at 12:58 AM Ricardo Wurmus <rekado <at> elephly.net> wrote:
> >
> > Actually, there is one question I have that is not of cosmetic nature:
> > What’s up with the commented gemmlowp in the inputs?  We have
> > gemmlowp-src in the native-inputs.
> >
> > If you could answer this question I can take care of the rest and apply
> > it.  Thank you for your patience!
> >
> > --
> > Ricardo
>


-- 
Andy Tai, atai <at> atai.org, Skype: licheng.tai, Line: andy_tai, WeChat:
andytai1010
Year 2023 民國112年
自動的精神力是信仰與覺悟
自動的行為力是勞動與技能
[Message part 2 (text/html, inline)]

Reply sent to Ricardo Wurmus <rekado <at> elephly.net>:
You have taken responsibility. (Fri, 08 Sep 2023 08:56:02 GMT) Full text and rfc822 format available.

Notification sent to Andy Tai <atai <at> atai.org>:
bug acknowledged by developer. (Fri, 08 Sep 2023 08:56:02 GMT) Full text and rfc822 format available.

Message #41 received at 63897-done <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: 63897-done <at> debbugs.gnu.org
Subject: [PATCH] gnu: tensorflow_lite: Update to 2.12.0
Date: Fri, 08 Sep 2023 10:54:40 +0200
Thanks for the patches and your patience.

I’ve made the changes I mentioned earlier and pushed the update together
with the other two patches this depends on.

-- 
Ricardo




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 06 Oct 2023 11:24:20 GMT) Full text and rfc822 format available.

This bug report was last modified 174 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.