GNU bug report logs - #57814
[PATCH 0/2] SciPy 1.9.1

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 57814 in the body.
You can then email your comments to 57814 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <marius <at> gnu.org>
To: guix-patches <at> gnu.org
Subject: [PATCH 0/2] SciPy 1.9.1
Date: Wed, 14 Sep 2022 23:36:30 +0200

This PR updates SciPy and uses its new Meson-backed build system.

Notable changes:

* Build time vastly improved.
* It now uses 'pythran'.
* Using scipy.test() instead of runtests.py which will get deprecated.
* Documentation is disabled (see comment).
* RUNPATH contains a bogus entry, will be fixed in next meson-python.

Marius Bakke (2):
  gnu: Add meson-python.
  gnu: python-scipy: Update to 1.9.1.

 gnu/packages/build-tools.scm    |  79 +++++++++++++++++++++-
 gnu/packages/python-science.scm | 116 +++++++++++++++++---------------
 2 files changed, 141 insertions(+), 54 deletions(-)

-- 
2.37.3

Message #8 received at 57814 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <marius <at> gnu.org>
To: 57814 <at> debbugs.gnu.org
Subject: [PATCH 1/2] gnu: Add meson-python.
Date: Wed, 14 Sep 2022 23:42:09 +0200

* gnu/packages/build-tools.scm (meson-python): New variable.
---
 gnu/packages/build-tools.scm | 79 +++++++++++++++++++++++++++++++++++-
 1 file changed, 78 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/build-tools.scm b/gnu/packages/build-tools.scm
index b5b9d501f9..84a62d0fd6 100644
--- a/gnu/packages/build-tools.scm
+++ b/gnu/packages/build-tools.scm
@@ -45,9 +45,11 @@ (define-module (gnu packages build-tools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages cpp)
+  #:use-module (gnu packages elf)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages logging)
   #:use-module (gnu packages lua)
+  #:use-module (gnu packages ninja)
   #:use-module (gnu packages package-management)
   #:use-module (gnu packages pcre)
   #:use-module (gnu packages pkg-config)
@@ -62,7 +64,7 @@ (define-module (gnu packages build-tools)
   #:use-module (gnu packages rpc)
   #:use-module (gnu packages sqlite)
   #:use-module (gnu packages tls)
-  #:use-module (gnu packages ninja)
+  #:use-module (gnu packages version-control)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system python))
 
@@ -342,6 +344,81 @@ (define-public meson-0.59
 ;; TODO: Bump this in the next rebuild cycle.
 (define-public meson meson-0.60)
 
+(define-public meson-python
+  (package
+    (name "meson-python")
+    (version "0.8.1")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "meson_python" version))
+              (sha256
+               (base32
+                "0k2yn0iws1n184sdznzmfw4xgbqgq5cn02hpc7m0xdaxryj1ybs4"))))
+    (build-system python-build-system)
+    (arguments
+     (list #:phases
+           #~(modify-phases %standard-phases
+               (add-after 'unpack 'avoid-ninja-dependency
+                 (lambda _
+                   ;; Avoid dependency on the "ninja" PyPI distribution,
+                   ;; which is a meta-package that simply downloads and
+                   ;; installs ninja from the web ...
+                   (substitute* "pyproject.toml"
+                     (("'ninja',")
+                      ""))))
+               (replace 'build
+                 (lambda _
+                   ;; ZIP does not support timestamps before 1980.
+                   (setenv "SOURCE_DATE_EPOCH" "315532800")
+                   (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
+               (replace 'install
+                 (lambda _
+                   (let ((whl (car (find-files "dist" "\\.whl$"))))
+                     (invoke "pip" "--no-cache-dir" "--no-input"
+                             "install" "--no-deps" "--prefix" #$output whl))))
+               (replace 'check
+                 (lambda* (#:key tests? #:allow-other-keys)
+                   (when tests?
+                     (invoke "pytest" "-vv" "tests" "-k"
+                             (string-append
+                              "not "
+                              ;; These tests require a git checkout.
+                              (string-join '("test_contents_unstaged"
+                                             "test_no_pep621"
+                                             "test_pep621"
+                                             "test_dynamic_version"
+                                             "test_contents"
+                                             "test_contents_subdirs")
+                                           " and not ")))))))))
+    (propagated-inputs
+     (list meson-0.63                   ;>=0.62 required
+           ninja
+           ;; XXX: python-meson forcefully sets the RUNPATH of binaries
+           ;; for vendoring purposes, and uses PatchELF for that(!).  This
+           ;; functionality is not useful in Guix, but removing this
+           ;; dependency is tricky.  There is discussion upstream about making
+           ;; it optional, but for now we'll just carry it:
+           ;; https://github.com/FFY00/meson-python/issues/125
+           patchelf
+           python-colorama
+           python-pyproject-metadata
+           python-tomli
+           python-wheel))
+    (native-inputs
+     (list python-pypa-build
+           python-wheel
+
+           ;; For tests.
+           pkg-config
+           python-gitpython
+           python-pytest
+           python-pytest-mock))
+    (home-page "https://github.com/FFY00/mesonpy")
+    (synopsis "Meson-based build backend for Python")
+    (description
+     "meson-python is a PEP 517 build backend for Meson projects.")
+    (license license:expat)))
+
 (define-public premake4
   (package
     (name "premake")
-- 
2.37.3

Message #11 received at 57814 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <marius <at> gnu.org>
To: 57814 <at> debbugs.gnu.org
Subject: [PATCH 2/2] gnu: python-scipy: Update to 1.9.1.
Date: Wed, 14 Sep 2022 23:42:10 +0200

* gnu/packages/python-science.scm (python-scipy): Update to 1.9.1.
[source]: Switch to GIT-FETCH.
[outputs]: Remove.
[arguments]: Don't disable Pythran.  Don't configure OpenBLAS which is now
discovered by pkg-config.  Do a "manual" PEP 517 style build.  Make
install-doc phase conditional on the presence of Sphinx.  Simplify tests.
[propagated-inputs]: Add PYTHON-PYTHRAN.
[native-inputs]: Remove PERL, WHICH, PYTHON-NUMPYDOC,
PYTHON-PYDATA-SPHINX-THEME, PYTHON-SPHINX, and PYTHON-SPHINX-PANELS.  Add
GCC, PKG-CONFIG, PYTHON-PYPA-BUILD and PYTHON-MESON.
---
 gnu/packages/python-science.scm | 112 +++++++++++++++++---------------
 1 file changed, 59 insertions(+), 53 deletions(-)

diff --git a/gnu/packages/python-science.scm b/gnu/packages/python-science.scm
index 9a75bf3089..d1698e5f67 100644
--- a/gnu/packages/python-science.scm
+++ b/gnu/packages/python-science.scm
@@ -42,6 +42,7 @@ (define-module (gnu packages python-science)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bioinformatics)
   #:use-module (gnu packages boost)
+  #:use-module (gnu packages build-tools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages cpp)
   #:use-module (gnu packages databases)
@@ -75,81 +76,86 @@ (define-module (gnu packages python-science)
 (define-public python-scipy
   (package
     (name "python-scipy")
-    (version "1.8.0")
+    (version "1.9.1")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "scipy" version))
        (sha256
-        (base32 "1gghkwn93niyasm36333xbqrnn3yiadq9d97wnc9mg14nzbg5m1i"))))
-    (outputs '("out" "doc"))
+        (base32 "1jcb94xal7w7ax80kaivqqics36v8smi4a3xngyxbrh0i538rli6"))))
     (build-system python-build-system)
     (arguments
      (list
-      #:modules '((guix build utils)
-                  (guix build python-build-system)
-                  (ice-9 format))
       #:phases
       #~(modify-phases %standard-phases
-          (add-after 'unpack 'disable-pythran
-            (lambda _
-              (setenv "SCIPY_USE_PYTHRAN" "0")))
-          (add-before 'build 'change-home-dir
+          (add-after 'unpack 'loosen-requirements
             (lambda _
-              ;; Change from /homeless-shelter to /tmp for write permission.
-              (setenv "HOME" "/tmp")))
-          (add-before 'build 'configure-openblas
+              (substitute* "pyproject.toml"
+                (("numpy==") "numpy>=")
+                (("meson==") "meson>="))))
+          (replace 'build
             (lambda _
-              (call-with-output-file "site.cfg"
-                (lambda (port)
-                  (format port
-                          "\
-[blas]
-libraries = openblas
-library_dirs = ~a/lib
-include_dirs = ~:*~a/include
-
-[atlas]
-library_dirs = ~:*~a/lib
-atlas_libs = openblas~%"  #$(this-package-input "openblas"))))))
-          (add-before 'build 'parallelize-build
+              ;; ZIP does not support timestamps before 1980.
+              (setenv "SOURCE_DATE_EPOCH" "315532800")
+              (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
+          (replace 'install
             (lambda _
-              (setenv "NPY_NUM_BUILD_JOBS"
-                      (number->string (parallel-job-count)))))
-          (add-before 'check 'install-doc
-            (lambda* (#:key outputs #:allow-other-keys)
-              (let* ((data (string-append (assoc-ref outputs "doc") "/share"))
-                     (doc (string-append data "/doc/" #$name "-" #$version))
-                     (html (string-append doc "/html")))
-                (with-directory-excursion "doc"
-                  ;; Build doc.
-                  (invoke "make" "html"
-                          ;; Building the documentation takes a very long time.
-                          ;; Parallelize it.
-                          (string-append "SPHINXOPTS=-j"
-                                         (number->string (parallel-job-count))))
-                  ;; Install doc.
-                  (mkdir-p html)
-                  (copy-recursively "build/html" html)))))
+              (let ((whl (car (find-files "dist" "\\.whl$"))))
+                (invoke "pip" "--no-cache-dir" "--no-input"
+                        "install" "--no-deps" "--prefix" #$output whl))))
           (replace 'check
             (lambda* (#:key tests? #:allow-other-keys)
               (when tests?
-                (invoke "./runtests.py" "-vv" "--no-build" "--mode=fast"
-                        "-j" (number->string (parallel-job-count)))))))))
-    (propagated-inputs (list python-numpy python-matplotlib python-pyparsing))
+                ;; Step out of the source directory to avoid interference.
+                (with-directory-excursion "/tmp"
+                  (invoke "python" "-c"
+                          (string-append
+                           "import scipy; scipy.test('fast', parallel="
+                           (number->string (parallel-job-count))
+                           ", verbose=2)"))))))
+          (add-after 'check 'install-doc
+            (lambda* (#:key outputs #:allow-other-keys)
+              ;; FIXME: Documentation cannot be built because it requires
+              ;; a newer version of pydata-sphinx-theme, which currently
+              ;; cannot build without internet access:
+              ;; <https://github.com/pydata/pydata-sphinx-theme/issues/628>.
+              ;; Keep the phase for easy testing.
+              (let ((sphinx-build (false-if-exception
+                                   (search-input-file input "bin/sphinx-build"))))
+                (if sphinx-build
+                    (let* ((doc (assoc-ref outputs "doc"))
+                           (data (string-append doc "/share"))
+                           (docdir (string-append
+                                    data "/doc/"
+                                    #$(package-name this-package) "-"
+                                    #$(package-version this-package)))
+                           (html (string-append docdir "/html")))
+                      (with-directory-excursion "doc"
+                        ;; Build doc.
+                        (invoke "make" "html"
+                                ;; Building the documentation takes a very long time.
+                                ;; Parallelize it.
+                                (string-append "SPHINXOPTS=-j"
+                                               (number->string (parallel-job-count))))
+                        ;; Install doc.
+                        (mkdir-p html)
+                        (copy-recursively "build/html" html)))
+                    (format #t "sphinx-build not found, skipping~%"))))))))
+    (propagated-inputs
+     (list python-numpy python-matplotlib python-pyparsing python-pythran))
     (inputs (list openblas pybind11))
     (native-inputs
      (list gfortran
-           perl
+           ;; XXX: Adding gfortran shadows GCC headers, causing a compilation
+           ;; failure.  Somehow also providing GCC works around it ...
+           gcc
+           meson-python
+           pkg-config
            python-cython
-           python-numpydoc
-           python-pydata-sphinx-theme
+           python-pypa-build
            python-pytest
            python-pytest-xdist
-           python-sphinx
-           python-sphinx-panels
-           python-threadpoolctl
-           which))
+           python-threadpoolctl))
     (home-page "https://scipy.org/")
     (synopsis "The Scipy library provides efficient numerical routines")
     (description "The SciPy library is one of the core packages that make up
-- 
2.37.3

Message #16 received at 57814-done <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <marius <at> gnu.org>
To: 57814-done <at> debbugs.gnu.org
Subject: Re: [bug#57814] [PATCH 0/2] SciPy 1.9.1
Date: Fri, 16 Sep 2022 19:15:49 +0200

[Message part 1 (text/plain, inline)]

Marius Bakke <marius <at> gnu.org> skriver:

> This PR updates SciPy and uses its new Meson-backed build system.
>
> Notable changes:
>
> * Build time vastly improved.
> * It now uses 'pythran'.
> * Using scipy.test() instead of runtests.py which will get deprecated.
> * Documentation is disabled (see comment).
> * RUNPATH contains a bogus entry, will be fixed in next meson-python.
>
> Marius Bakke (2):
>   gnu: Add meson-python.
>   gnu: python-scipy: Update to 1.9.1.

Pushed in 6ce5a6c2e3..888dfc8eb6.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.