GNU bug report logs - #64448
[PATCH 0/2] Fix hdf5-1.14 wrappers and remove generated source files from all hdf5 versions.

Previous Next

To reply to this bug, email your comments to 64448 AT debbugs.gnu.org.
There is no need to reopen the bug first.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: David Elsing <david.elsing <at> posteo.net>
To: guix-patches <at> gnu.org
Cc: David Elsing <david.elsing <at> posteo.net>
Subject: [PATCH 0/2] Fix hdf5-1.14 wrappers and remove generated source files
 from all hdf5 versions.
Date: Mon,  3 Jul 2023 21:58:45 +0000

When using hdf5 with CMake, I noticed that it does not work for the 1.14
version, because the h5cc and h5c++ wrappers contain the flag
"-I/tmp/guix-build-hdf5-1.14.0.drv-0/hdf5-1.14.0/src/H5FDsubfiling". It
is required in the 'configure phase, but can be removed afterwards.

Additionally, I noticed that the hdf5 source tarballs contain many
autogenerated files, so I also removed them. Somehow, the 1.8.23 release
is missing two m4 scripts (which are included in later releases, but
different from this one), so they are copied from the 1.8.21 tarball
(the latest earlier release which includes them).
The genparser script is missing as well, which is copied from the 1.10.9
release.

David Elsing (2):
  gnu: hdf5-1.8: Remove generated files from source
  gnu: hdf5-1.14: Remove spurious include directories.

 gnu/packages/maths.scm                      | 234 +++++++++++++++-----
 gnu/packages/patches/hdf5-config-date.patch |  14 +-
 2 files changed, 188 insertions(+), 60 deletions(-)


base-commit: 76e39909e886bb51cf9b2fa03a1523545faca880
-- 
2.40.1

Message #8 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: David Elsing <david.elsing <at> posteo.net>
To: 64448 <at> debbugs.gnu.org
Cc: David Elsing <david.elsing <at> posteo.net>
Subject: [PATCH 1/2] gnu: hdf5-1.8: Remove generated files from source
Date: Mon,  3 Jul 2023 22:00:46 +0000

* gnu/packages/maths.scm (hdf5-snippet): New variable.
(hdf5-1.8.21-source): New variable.
(hdf5-1.8)[source](modules, snippet): New fields.
[native-inputs]: Add autoconf, automake, bison, flex, libtool and which.
[arguments] Use Gexps.
<#:phases>: Add 'copy-scripts, 'make-gen-deterministic, 'generate-flexbison
and 'generate-headers phases.
(hdf5-1.10)[source](modules, snippet): New fields.
[arguments]<#:phases>: Use 'substitute-keyword-arguments' and delete
'copy-scripts phase.
(hdf5-1.12): Inherit from hdf5-1.10.
[source](modules, snippet): New fields.
(hdf5-1.14): Inherit from hdf5-1.10.
[source](modules, snippet): New fields.
* gnu/packages/patches/hdf5-config-date.patch: Apply to configure.ac instead
of configure.
---
 gnu/packages/maths.scm                      | 221 +++++++++++++++-----
 gnu/packages/patches/hdf5-config-date.patch |  14 +-
 2 files changed, 176 insertions(+), 59 deletions(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 653e76027a..b2029db615 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -60,6 +60,7 @@
 ;;; Copyright © 2022 Akira Kyle <akira <at> akirakyle.com>
 ;;; Copyright © 2022 Roman Scherer <roman.scherer <at> burningswell.com>
 ;;; Copyright © 2023 Jake Leporte <jakeleporte <at> outlook.com>
+;;; Copyright © 2023 David Elsing <david.elsing <at> posteo.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1376,12 +1377,48 @@ (define-public hdf4-alt
     (synopsis
      "HDF4 without netCDF API, can be combined with the regular netCDF library")))
 
-(define-public hdf5-1.8
-  (package
-    (name "hdf5")
-    (version "1.8.23")
-    (source
-     (origin
+(define hdf5-snippet
+  #~(begin
+      (for-each delete-file (find-files "." "Makefile\\.in$"))
+      (when (file-exists? "autom4te.cache")
+        (delete-file-recursively "autom4te.cache"))
+      (for-each (lambda (file)
+                  (when (file-exists? file)
+                    (delete-file file)))
+                (list
+                 "configure"
+                 "bin/ltmain.sh"
+                 "bin/compile"
+                 "bin/config.guess"
+                 "bin/config.sub"
+                 "bin/install-sh"
+                 "bin/missing"
+                 "bin/test-driver"
+                 "bin/depcomp"
+                 "src/H5config.h.in"
+                 "src/H5Epubgen.h"
+                 "src/H5Einit.h"
+                 "src/H5Eterm.h"
+                 "src/H5Edefin.h"
+                 "src/H5version.h"
+                 "src/H5overflow.h"
+                 "aclocal.m4"
+                 "hl/tools/gif2h5/testfiles/ex_image2.h5"
+                 "hl/tools/gif2h5/testfiles/5giftst.h5"
+                 "hl/src/H5LTparse.h"
+                 "hl/src/H5LTparse.c"
+                 "hl/src/H5LTanalyze.c"
+                 ;; In later releases
+                 "m4/libtool.m4"
+                 "m4/ltoptions.m4"
+                 "m4/ltsugar.m4"
+                 "m4/ltversion.m4"
+                 "m4/lt~obsolete.m4"))))
+
+;; In the 1.8.22 and 1.8.23 release, some m4 scripts are missing
+(define hdf5-1.8.21-source
+  (let ((version "1.8.21"))
+    (origin
       (method url-fetch)
       (uri (list (string-append "https://support.hdfgroup.org/ftp/HDF5/releases/"
                                 "hdf5-" (version-major+minor version)
@@ -1394,18 +1431,53 @@ (define-public hdf5-1.8
                                    (string-append major minor)))
                                 "/src/hdf5-" version ".tar.bz2")))
       (sha256
-       (base32 "0km65mr6dgk4ia2dqr1b9dzw9qg15j5z35ymbys9cnny51z1zb39"))
-      (patches (search-patches "hdf5-config-date.patch"))))
+       (base32 "03glk4w4wyb1jyb443g53y3y1ncnf6mj2cqwm6avfr2awkgb3cg5"))
+      (modules '((guix build utils)))
+      (snippet hdf5-snippet)
+      (patches (search-patches "hdf5-config-date.patch")))))
+
+(define-public hdf5-1.8
+  (package
+    (name "hdf5")
+    (version "1.8.23")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (list (string-append "https://support.hdfgroup.org/ftp/HDF5/releases/"
+                                 "hdf5-" (version-major+minor version)
+                                 "/hdf5-" version "/src/hdf5-"
+                                 version ".tar.bz2")
+                  (string-append "https://support.hdfgroup.org/ftp/HDF5/"
+                                 "current"
+                                 (match (string-split version #\.)
+                                   ((major minor _ ...)
+                                    (string-append major minor)))
+                                 "/src/hdf5-" version ".tar.bz2")))
+       (sha256
+        (base32 "0km65mr6dgk4ia2dqr1b9dzw9qg15j5z35ymbys9cnny51z1zb39"))
+       (modules '((guix build utils)))
+       (snippet hdf5-snippet)
+       (patches (search-patches "hdf5-config-date.patch"))))
     (build-system gnu-build-system)
     (inputs
      (list zlib))
     (native-inputs
-     (list gfortran perl))                 ;part of the test machinery needs Perl
+     (list autoconf
+           automake
+           bison
+           flex
+           gfortran
+           libtool
+           ;; Needed to generate some headers and for tests
+           perl
+           which))
     (outputs '("out"       ; core library
                "fortran")) ; fortran interface
     (arguments
-     `(;; Some of the users, notably Flann, need the C++ interface.
-       #:configure-flags '("--enable-cxx"
+     (list
+      ;; Some of the users, notably Flann, need the C++ interface.
+      #:configure-flags #~(list
+                           "--enable-cxx"
                            "--enable-fortran"
                            "--enable-fortran2003"
 
@@ -1418,44 +1490,71 @@ (define-public hdf5-1.8
                            "--enable-threadsafe"
                            "--with-pthread"
                            "--enable-unsupported")
-       ;; Use -fPIC to allow the R bindings to link with the static libraries
-       #:make-flags (list "CFLAGS=-fPIC"
-                          "CXXFLAGS=-fPIC")
-       #:phases
-       (modify-phases %standard-phases
-         (add-before 'configure 'patch-configure
-           (lambda* (#:key outputs #:allow-other-keys)
-             (substitute* "configure"
-               (("/bin/mv") "mv"))
-             (substitute* "fortran/src/Makefile.in"
-               (("libhdf5_fortran_la_LDFLAGS =")
-                (string-append "libhdf5_fortran_la_LDFLAGS = -Wl,-rpath="
-                               (assoc-ref outputs "fortran") "/lib")))
-             (substitute* "hl/fortran/src/Makefile.in"
-               (("libhdf5hl_fortran_la_LDFLAGS =")
-                (string-append "libhdf5hl_fortran_la_LDFLAGS = -Wl,-rpath="
-                               (assoc-ref outputs "fortran") "/lib")))))
-         (add-after 'configure 'patch-settings
-           (lambda _
-             ;; libhdf5.settings contains the full path of the
-             ;; compilers used, and its contents are included in
-             ;; libhdf5.so.  We truncate the hashes to avoid
-             ;; unnecessary store references to those compilers:
-             (substitute* "src/libhdf5.settings"
-              (("(/gnu/store/)([a-zA-Z0-9]*)" all prefix hash)
-               (string-append prefix (string-take hash 10) "..."))
-              ;; Don't record the build-time kernel version to make the
-              ;; settings file reproducible.
-              (("Uname information:.*")
-               "Uname information: Linux\n"))))
-         (add-after 'install 'patch-references
-           (lambda* (#:key inputs outputs #:allow-other-keys)
-             (let ((bin (string-append (assoc-ref outputs "out") "/bin"))
-                   (zlib (assoc-ref inputs "zlib")))
-               (substitute* (find-files bin "h5p?cc")
-                 (("-lz" lib)
-                  (string-append "-L" zlib "/lib " lib))))))
-         (add-after 'install 'split
+      ;; Use -fPIC to allow the R bindings to link with the static libraries
+      #:make-flags #~(list "CFLAGS=-fPIC"
+                           "CXXFLAGS=-fPIC")
+      #:phases
+      #~(modify-phases %standard-phases
+          ;; These scripts are missing in this release
+          (add-after 'unpack 'copy-scripts
+            (lambda _
+              (invoke "tar" "-xf" #$hdf5-1.8.21-source
+                      "-C" "m4" "--strip-components=2"
+                      "hdf5-1.8.21/m4/aclocal_fc.m4"
+                      "hdf5-1.8.21/m4/aclocal_cxx.m4")
+              (invoke "tar" "-xf" #$(package-source hdf5-1.10)
+                      "-C" "bin" "--strip-components=2"
+                      "hdf5-1.10.9/bin/genparser")))
+          (add-after 'copy-scripts 'make-gen-deterministic
+            (lambda _
+              (substitute* "bin/make_err"
+                (("keys %major" all)
+                 (string-append "sort " all))
+                (("while.*each \\(%section\\).*")
+                 (string-append
+                  "foreach $sect_name (sort keys %section) {\n"
+                  "        $sect_desc = $section{$sect_name};\n")))))
+          (add-after 'copy-scripts 'generate-flexbison
+            (lambda _
+              (invoke "bash" "bin/genparser" "hl/src")))
+          (add-before 'configure 'patch-configure
+            (lambda* (#:key outputs #:allow-other-keys)
+              (substitute* "configure"
+                (("/bin/mv") "mv"))
+              (substitute* "fortran/src/Makefile.in"
+                (("libhdf5_fortran_la_LDFLAGS =")
+                 (string-append "libhdf5_fortran_la_LDFLAGS = -Wl,-rpath="
+                                (assoc-ref outputs "fortran") "/lib")))
+              (substitute* "hl/fortran/src/Makefile.in"
+                (("libhdf5hl_fortran_la_LDFLAGS =")
+                 (string-append "libhdf5hl_fortran_la_LDFLAGS = -Wl,-rpath="
+                                (assoc-ref outputs "fortran") "/lib")))))
+          (add-after 'configure 'patch-settings
+            (lambda _
+              ;; libhdf5.settings contains the full path of the
+              ;; compilers used, and its contents are included in
+              ;; libhdf5.so.  We truncate the hashes to avoid
+              ;; unnecessary store references to those compilers:
+              (substitute* "src/libhdf5.settings"
+                (("(/gnu/store/)([a-zA-Z0-9]*)" all prefix hash)
+                 (string-append prefix (string-take hash 10) "..."))
+                ;; Don't record the build-time kernel version to make the
+                ;; settings file reproducible.
+                (("Uname information:.*")
+                 "Uname information: Linux\n"))))
+          (add-after 'configure 'generate-headers
+            (lambda _
+              (invoke "perl" "bin/make_err" "src/H5err.txt")
+              (invoke "perl" "bin/make_vers" "src/H5vers.txt")
+              (invoke "perl" "bin/make_overflow" "src/H5overflow.txt")))
+          (add-after 'install 'patch-references
+            (lambda* (#:key inputs outputs #:allow-other-keys)
+              (let ((bin (string-append (assoc-ref outputs "out") "/bin"))
+                    (zlib (assoc-ref inputs "zlib")))
+                (substitute* (find-files bin "h5p?cc")
+                  (("-lz" lib)
+                   (string-append "-L" zlib "/lib " lib))))))
+          (add-after 'install 'split
             (lambda* (#:key inputs outputs #:allow-other-keys)
               ;; Move all fortran-related files
               (let* ((out (assoc-ref outputs "out"))
@@ -1515,11 +1614,25 @@ (define-public hdf5-1.10
                                  "/src/hdf5-" version ".tar.bz2")))
        (sha256
         (base32 "14gih7kmjx4h3lc7pg4fwcl28hf1qqkf2x7rljpxqvzkjrqbxi00"))
-       (patches (search-patches "hdf5-config-date.patch"))))))
+       (modules '((guix build utils)))
+       (snippet hdf5-snippet)
+       (patches (search-patches "hdf5-config-date.patch"))))
+    (arguments
+     (substitute-keyword-arguments (package-arguments hdf5-1.8)
+       ((#:phases phases)
+        #~(modify-phases #$phases
+            (delete 'copy-scripts)
+            (add-after 'unpack 'patch-trace-shebang
+              (lambda _
+                (for-each
+                 patch-shebang
+                 (find-files "bin"
+                             (lambda (file stat)
+                               (executable-file? file))))))))))))
 
 (define-public hdf5-1.12
   (package
-    (inherit hdf5-1.8)
+    (inherit hdf5-1.10)
     (version "1.12.2")
     (source
      (origin
@@ -1535,11 +1648,13 @@ (define-public hdf5-1.12
                                  "/src/hdf5-" version ".tar.bz2")))
        (sha256
         (base32 "1zlawdzb0gsvcxif14fwr5ap2gk4b6j02wirr2hcx8hkcbivp20s"))
+       (modules '((guix build utils)))
+       (snippet hdf5-snippet)
        (patches (search-patches "hdf5-config-date.patch"))))))
 
 (define-public hdf5-1.14
   (package
-    (inherit hdf5-1.8)
+    (inherit hdf5-1.10)
     (version "1.14.0")
     (source
      (origin
@@ -1555,6 +1670,8 @@ (define-public hdf5-1.14
                                  "/src/hdf5-" version ".tar.bz2")))
        (sha256
         (base32 "181bdh8hp7v9xqwcby3lknr92lxlicc2hqscba3f5nhf8lrr9rz4"))
+       (modules '((guix build utils)))
+       (snippet hdf5-snippet)
        (patches (search-patches "hdf5-config-date.patch"))))))
 
 (define-public hdf5
diff --git a/gnu/packages/patches/hdf5-config-date.patch b/gnu/packages/patches/hdf5-config-date.patch
index c105435dc2..d9710f91e9 100644
--- a/gnu/packages/patches/hdf5-config-date.patch
+++ b/gnu/packages/patches/hdf5-config-date.patch
@@ -1,14 +1,14 @@
 Honor SOURCE_DATE_EPOCH when exporting configuration date.
 Autoconf-level patch submitted upstream on Wed Apr 13 17:03:23 UTC 2016
 
---- a/configure
-+++ b/configure
-@@ -27737,7 +28573,14 @@
- 
+--- a/configure.ac
++++ b/configure.ac
+@@ -3566,7 +3566,14 @@
+ AC_SUBST([H5_VERSION])
  
  ## Configuration date
-- CONFIG_DATE="`date`"
-+ CONFIG_DATE="`date -u`"
+-AC_SUBST([CONFIG_DATE]) CONFIG_DATE="`date`"
++AC_SUBST([CONFIG_DATE]) CONFIG_DATE="`date -u`"
 +if test -n "$SOURCE_DATE_EPOCH"; then
 +  CONFIG_DATE=`date -u -d "@$SOURCE_DATE_EPOCH" 2>/dev/null \
 +               || date -u -r "$SOURCE_DATE_EPOCH" 2>/dev/null`
@@ -18,4 +18,4 @@ Autoconf-level patch submitted upstream on Wed Apr 13 17:03:23 UTC 2016
 +fi
  
  ## User doing the configuration
-  CONFIG_USER="`whoami`@`hostname`"
+ AC_SUBST([CONFIG_USER]) CONFIG_USER="`whoami`@`hostname`"
\ No newline at end of file
-- 
2.40.1

Message #11 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: David Elsing <david.elsing <at> posteo.net>
To: 64448 <at> debbugs.gnu.org
Cc: David Elsing <david.elsing <at> posteo.net>
Subject: [PATCH 2/2] gnu: hdf5-1.14: Remove spurious include directories.
Date: Mon,  3 Jul 2023 22:00:47 +0000

* gnu/packages/maths.scm (hdf5-1.14)[arguments]: Use
'substitute-keyword-arguments' to add 'remove-subfiling-cppflags phase.
---
 gnu/packages/maths.scm | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index b2029db615..5ba9e07058 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -1672,7 +1672,18 @@ (define-public hdf5-1.14
         (base32 "181bdh8hp7v9xqwcby3lknr92lxlicc2hqscba3f5nhf8lrr9rz4"))
        (modules '((guix build utils)))
        (snippet hdf5-snippet)
-       (patches (search-patches "hdf5-config-date.patch"))))))
+       (patches (search-patches "hdf5-config-date.patch"))))
+    (arguments
+     (substitute-keyword-arguments (package-arguments hdf5-1.10)
+       ((#:phases phases)
+        #~(modify-phases #$phases
+            ;; Remove include flag to directory in source tree
+            (add-after 'configure 'remove-subfiling-cppflags
+              (lambda _
+                (substitute* '("src/libhdf5.settings"
+                               "bin/h5cc"
+                               "c++/src/h5c++")
+                  (("-I.*/src/H5FDsubfiling") ""))))))))))
 
 (define-public hdf5
   ;; Default version of HDF5.
-- 
2.40.1

Message #14 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: David Elsing <david.elsing <at> posteo.net>
To: 64448 <at> debbugs.gnu.org
Subject: Re: [PATCH 0/2] Fix hdf5-1.14 wrappers and remove generated source
 files from all hdf5 versions.
Date: Tue, 04 Jul 2023 21:14:01 +0000

I saw that the Git repository is also available, with the releases as
tags: https://github.com/HDFGroup/hdf5. Would it be preferred to use
them instead and make the snippet shorter? The missing scripts are also
included for the 1.8.23 release.

Message #17 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: David Elsing <david.elsing <at> posteo.net>
Cc: bavier <at> posteo.net, 64448 <at> debbugs.gnu.org, Gerd Heber <gerd.heber <at> gmail.com>,
 efraim <at> flashner.co.il
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Mon, 24 Jul 2023 15:49:49 +0200

Hello,

adding Gerd Heber in cc, who seems to be the last person having worked on
these packages.

I am not competent at all about them, so commenting more as an outsider.
Using "guix package -A hdf":
hdf4                	4.2.14          	out        	gnu/packages/maths.scm:1289:2
hdf4-alt            	4.2.14          	out        	gnu/packages/maths.scm:1372:2
hdf5                	1.8.23          	out,fortran	gnu/packages/maths.scm:1382:2
hdf5                	1.14.0          	out,fortran	gnu/packages/maths.scm:1543:2
hdf5                	1.12.2          	out,fortran	gnu/packages/maths.scm:1523:2
hdf5                	1.10.9          	out,fortran	gnu/packages/maths.scm:1503:2
hdf5-blosc          	1.0.0           	out        	gnu/packages/maths.scm:1807:2
hdf5-parallel-openmpi	1.10.9          	out,fortran	gnu/packages/maths.scm:1775:2

and a lot of packages in other languages, for instance:
cl-hdf5-cffi        	1.8.18-1.5b5c88f	out        	gnu/packages/lisp-xyz.scm:12238:4
ecl-hdf5-cffi       	1.8.18-1.5b5c88f	out        	gnu/packages/lisp-xyz.scm:12238:4
(apparently based on 1.8.18? but it has hdf5-1.10 as input).

So there are lots of versions, and in the middle of them,
(define-public hdf5
  ;; Default version of HDF5.
  hdf5-1.10)

And
hdf5-parallel-openmpi	1.10.9          	out,fortran	gnu/packages/maths.scm:1775:2
which inherits like this:
  (package/inherit hdf5-1.10                      ;use the latest
but takes inputs like this:
    (inputs
     `(("mpi" ,openmpi)
       ,@(package-inputs hdf5)))
which, I suppose, works by chance since currently hdf5-1.10 and hdf5 are
the same (but not "the latest").

Are all of these needed? If only the latest version 1.14.0 could be kept,
this would make the patch for 1.8.23 obsolete, for instance.

Andreas

Message #20 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: David Elsing <david.elsing <at> posteo.net>
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Mon, 24 Jul 2023 16:00:23 +0200

Am Mon, Jul 24, 2023 at 03:49:49PM +0200 schrieb Andreas Enge:
> Are all of these needed?

Partial answer at
   https://portal.hdfgroup.org/display/support/HDF5+1.8.23 :
"Future of HDF5-1.8
Please be aware that this will be the last release for HDF5 1.8.
We encourage users to move to HDF5 1.10 or HDF5 1.14. Please refer to
the release schedule for future releases of these versions."

The schedule at
   https://github.com/HDFGroup/hdf5#release-schedule
shows that 1.10 and 1.12 should see their last release this autumn,
so probably we should really move to 1.14.

$ guix refresh -l hdf5 <at> 1.8
Building the following 9 packages would ensure 16 dependent packages are rebuilt: pigx-sars-cov-2 <at> 0.0.9 pigx <at> 0.0.3 nip2 <at> 8.7.1 python-pyvips <at> 2.2.1 gnss-sdr <at> 0.0.17 hdf-eos5 <at> 1.15 scilab <at> 5.5.0 hdf-java <at> 3.3.2 h5check <at> 2.0.1

$ guix refresh -l hdf5 <at> 1.10
Building the following 210 packages would ensure 411 dependent packages are rebuilt:
...

$ guix refresh -l hdf5 <at> 1.12
No dependents other than itself: hdf5 <at> 1.12.2
So this could probably be dropped immediately.

$ guix refresh -l hdf5 <at> 1.14
Building the following 2 packages would ensure 2 dependent packages are rebuilt: ecl-hdf5-cffi <at> 1.8.18-1.5b5c88f cl-hdf5-cffi <at> 1.8.18-1.5b5c88f

I suppose that the many dependencies on 1.10 come from the definition of
the variable hd5 as hdf5-10; it is quite possible we could simply upgrade
this to hdf5-14. Someone™ should give it a try...

Andreas

Message #23 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Gerd Heber <gerd.heber <at> gmail.com>
To: Andreas Enge <andreas <at> enge.fr>
Cc: David Elsing <david.elsing <at> posteo.net>, bavier <at> posteo.net,
 64448 <at> debbugs.gnu.org, efraim <at> flashner.co.il
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Sun, 20 Aug 2023 19:36:50 -0500

[Message part 1 (text/plain, inline)]

Only HDF5 1.14.x (currently, 1.14.2) should be kept. However, hdf5 and
hdf5-parallel should be kept separate. Aside from the MPI dependence,
the parallel build is NOT a superset of the sequential version. There could
be even two versions of hdf5: a non-threadsafe build (hdf5) and a
threadsafe build (hdf5-ts? hdf5-mt?).

For HDF4, 4.2.16-2 is the latest HDF4 release. We are maintaining that
indefinitely (for NASA).

Best, G.

On Mon, Jul 24, 2023 at 8:49 AM Andreas Enge <andreas <at> enge.fr> wrote:

> Hello,
>
> adding Gerd Heber in cc, who seems to be the last person having worked on
> these packages.
>
> I am not competent at all about them, so commenting more as an outsider.
> Using "guix package -A hdf":
> hdf4                    4.2.14                  out
>  gnu/packages/maths.scm:1289:2
> hdf4-alt                4.2.14                  out
>  gnu/packages/maths.scm:1372:2
> hdf5                    1.8.23                  out,fortran
>  gnu/packages/maths.scm:1382:2
> hdf5                    1.14.0                  out,fortran
>  gnu/packages/maths.scm:1543:2
> hdf5                    1.12.2                  out,fortran
>  gnu/packages/maths.scm:1523:2
> hdf5                    1.10.9                  out,fortran
>  gnu/packages/maths.scm:1503:2
> hdf5-blosc              1.0.0                   out
>  gnu/packages/maths.scm:1807:2
> hdf5-parallel-openmpi   1.10.9                  out,fortran
>  gnu/packages/maths.scm:1775:2
>
> and a lot of packages in other languages, for instance:
> cl-hdf5-cffi            1.8.18-1.5b5c88f        out
>  gnu/packages/lisp-xyz.scm:12238:4
> ecl-hdf5-cffi           1.8.18-1.5b5c88f        out
>  gnu/packages/lisp-xyz.scm:12238:4
> (apparently based on 1.8.18? but it has hdf5-1.10 as input).
>
> So there are lots of versions, and in the middle of them,
> (define-public hdf5
>   ;; Default version of HDF5.
>   hdf5-1.10)
>
> And
> hdf5-parallel-openmpi   1.10.9                  out,fortran
>  gnu/packages/maths.scm:1775:2
> which inherits like this:
>   (package/inherit hdf5-1.10                      ;use the latest
> but takes inputs like this:
>     (inputs
>      `(("mpi" ,openmpi)
>        ,@(package-inputs hdf5)))
> which, I suppose, works by chance since currently hdf5-1.10 and hdf5 are
> the same (but not "the latest").
>
> Are all of these needed? If only the latest version 1.14.0 could be kept,
> this would make the patch for 1.8.23 obsolete, for instance.
>
> Andreas
>
>

[Message part 2 (text/html, inline)]

Message #26 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Gerd Heber <gerd.heber <at> gmail.com>
Cc: David Elsing <david.elsing <at> posteo.net>, bavier <at> posteo.net,
 64448 <at> debbugs.gnu.org, efraim <at> flashner.co.il
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Mon, 21 Aug 2023 22:24:36 +0200

Am Sun, Aug 20, 2023 at 07:36:50PM -0500 schrieb Gerd Heber:
> Only HDF5 1.14.x (currently, 1.14.2) should be kept. However, hdf5 and
> hdf5-parallel should be kept separate. Aside from the MPI dependence,
> the parallel build is NOT a superset of the sequential version. There could be
> even two versions of hdf5: a non-threadsafe build (hdf5) and a threadsafe build
> (hdf5-ts? hdf5-mt?).
> 
> For HDF4, 4.2.16-2 is the latest HDF4 release. We are maintaining that
> indefinitely (for NASA).

Thanks for the info, which helps us set a target!

I am trying to update hdf4, see
   https://issues.guix.gnu.org/65443

Andreas

Message #29 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Gerd Heber <gerd.heber <at> gmail.com>
Cc: David Elsing <david.elsing <at> posteo.net>, bavier <at> posteo.net,
 64448 <at> debbugs.gnu.org, efraim <at> flashner.co.il
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Mon, 21 Aug 2023 22:38:16 +0200

And I have removed hdf5 <at> 1.12, which had no dependent packages.

Andreas

Message #32 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: David Elsing <david.elsing <at> posteo.net>
To: Andreas Enge <andreas <at> enge.fr>
Cc: 64448 <at> debbugs.gnu.org
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Mon, 25 Sep 2023 22:06:10 +0000

Hello,

> $ guix refresh -l hdf5 <at> 1.8
> Building the following 9 packages would ensure 16 dependent packages are rebuilt: pigx-sars-cov-2 <at> 0.0.9 pigx <at> 0.0.3 nip2 <at> 8.7.1 python-pyvips <at> 2.2.1 gnss-sdr <at> 0.0.17 hdf-eos5 <at> 1.15 scilab <at> 5.5.0 hdf-java <at> 3.3.2 h5check <at> 2.0.1

Currently, 5 packages depend directly on hdf5 <at> 1.8 (from guix graph -t reverse-package):
-scilab, for which the latest version can be built with hdf5 <at> 1.14:
 https://issues.guix.gnu.org/66201
-hdf5-eos, for which the latest version can also be built with
 hdf5 <at> 1.14: https://issues.guix.gnu.org/66204
-java-cisd-jhdf5, where newer versions seem to be built only with
 Gradle: https://sissource.ethz.ch/sispub/jhdf5. It has fastqc and pigx(-*)
 as dependencies.
-hdf-java, where the latest version is from 2017. Maybe it can be built
 with a newer HDF5 version? No other packages depend on it.
-h5check, where the latest version is from 2014. When I tried to compile
 it with hdf5 <at> 1.14, some tests failed because the file format is not
 accepted anymore. No other packages depend on it.

Cheers,
David

Message #35 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: David Elsing <david.elsing <at> posteo.net>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 64448 <at> debbugs.gnu.org
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Wed, 26 Mar 2025 17:05:53 +0100

Hello,

I have submitted two new issues with the goal of getting closer to
removing hdf5 <at> 1.8 and hdf5 <at> 1.10, see
   https://issues.guix.gnu.org/77285
   https://issues.guix.gnu.org/77287

Ricardo, since you seem to be interested in this package, your input
there would be very welcome. And here also; after the proposed removal
of h5check (which, I just see, does not even build any more), the only
remaining direct dependency of hdf5 <at> 1.8 is
   java-cisd-jhdf5
and "guix refresh -l hdf5 <at> 1.8" lists
   pigx-sars-cov-2 <at> 0.0.9 trinityrnaseq <at> 2.13.2 pigx <at> 0.0.3
as leaf packages.

Andreas

Message #38 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Andreas Enge <andreas <at> enge.fr>
Cc: David Elsing <david.elsing <at> posteo.net>, 64448 <at> debbugs.gnu.org
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Wed, 26 Mar 2025 18:29:39 +0100

Hi Andreas,

> I have submitted two new issues with the goal of getting closer 
> to
> removing hdf5 <at> 1.8 and hdf5 <at> 1.10, see
>    https://issues.guix.gnu.org/77285
>    https://issues.guix.gnu.org/77287
>
> Ricardo, since you seem to be interested in this package, your 
> input
> there would be very welcome. And here also; after the proposed 
> removal
> of h5check (which, I just see, does not even build any more), 
> the only
> remaining direct dependency of hdf5 <at> 1.8 is
>    java-cisd-jhdf5
> and "guix refresh -l hdf5 <at> 1.8" lists
>    pigx-sars-cov-2 <at> 0.0.9 trinityrnaseq <at> 2.13.2 pigx <at> 0.0.3
> as leaf packages.

On the python-team branch the only user of java-cisd-jhdf5 is 
fastqc, which should be updated to a more recent version.

-- 
Ricardo

Message #41 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: David Elsing <david.elsing <at> posteo.net>, 64448 <at> debbugs.gnu.org
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Fri, 28 Mar 2025 21:38:57 +0100

Hello Ricardo,

Am Wed, Mar 26, 2025 at 06:29:39PM +0100 schrieb Ricardo Wurmus:
> On the python-team branch the only user of java-cisd-jhdf5 is fastqc, which
> should be updated to a more recent version.

looking at its git repository, fastqc still requires java-cisd-jhdf5
(there is an eight year old jar file), so it does not look as if
updating fastqc will help us get rid of java-cisd-jhdf5 and then
hdf5 <at> 1.8.

Andreas

Message #44 received at 64448 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: David Elsing <david.elsing <at> posteo.net>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 64448 <at> debbugs.gnu.org
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Mon, 31 Mar 2025 12:24:07 +0200

Hello David,

with a lot of work on hdf5 unrelated to your original issue, this bug
report has become a bit unwieldy (sorry for that, but I think we made
good progress around this package complex!).

So I wonder whether you would be okay closing this issue and opening
new ones with updated patches?

Useful remaining work would be updating hdf5 <at> 1.14 to its latest version;
working towards getting rid of hdf5 <at> 1.8 and hdf5 <at> 1.10;
and your original issue about generated files, which maybe could be
combined with an update to hdf5 <at> 1.14 (I think there is no point in
spending much work on the outdated versions that we would prefer to
remove).
For testing updates, it might make sense to wait until the non-building
dependencies have been removed in about a month.

What do you think?

Andreas

Message #49 received at 64448-done <at> debbugs.gnu.org (full text, mbox):

From: David Elsing <david.elsing <at> posteo.net>
To: Andreas Enge <andreas <at> enge.fr>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 64448-done <at> debbugs.gnu.org
Subject: Re: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove
 generated source files from all hdf5 versions.
Date: Sun, 06 Apr 2025 23:33:55 +0000

Hi Andreas,

Andreas Enge <andreas <at> enge.fr> writes:

> So I wonder whether you would be okay closing this issue and opening
> new ones with updated patches?

Sure, I'm closing it now.

> Useful remaining work would be updating hdf5 <at> 1.14 to its latest version;
> working towards getting rid of hdf5 <at> 1.8 and hdf5 <at> 1.10;
> and your original issue about generated files, which maybe could be
> combined with an update to hdf5 <at> 1.14 (I think there is no point in
> spending much work on the outdated versions that we would prefer to
> remove).

Yes, that's a good suggestion, I made a new patch series which only
changes hdf5 <at> 1.14 and hdf-java: https://issues.guix.gnu.org/77590

Thanks,
David

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.