GNU bug report logs - #45734
[PATCH] gnu: update zfs to 0.8.6.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 45734 in the body.
You can then email your comments to 45734 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: raid5atemyhomework <raid5atemyhomework <at> protonmail.com>
To: "guix-patches <at> gnu.org" <guix-patches <at> gnu.org>
Subject: [PATCH] gnu: update zfs to 0.8.6.
Date: Sat, 09 Jan 2021 05:50:50 +0000

From 4c0d59071fa888f6ad0f3b67e3962b5ee84d50da Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework <at> protonmail.com>
Date: Sat, 9 Jan 2021 11:53:58 +0800
Subject: [PATCH] gnu: update zfs to 0.8.6.

* gnu/packages/file-systems.scm (zfs): Update version to 0.8.6.
---
 gnu/packages/file-systems.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm
index f74d519b33..9b40a6603a 100644
--- a/gnu/packages/file-systems.scm
+++ b/gnu/packages/file-systems.scm
@@ -847,7 +847,7 @@ APFS.")
 (define-public zfs
   (package
     (name "zfs")
-    (version "0.8.5")
+    (version "0.8.6")
     (outputs '("out" "module" "src"))
     (source
       (origin
@@ -856,7 +856,7 @@ APFS.")
                               "/download/zfs-" version
                               "/zfs-" version ".tar.gz"))
           (sha256
-           (base32 "0gfdnynmsxbhi97q73smrgmcw1k8zmlr1hgljfn38sk0kimivd6v"))))
+           (base32 "0xx6ssdpx86nwpnsq1zccqia6qilm3rf3vhpa77fpsvzq0yr3nw8"))))
     (build-system linux-module-build-system)
     (arguments
      `(;; The ZFS kernel module should not be downloaded since the license
--
2.30.0

Message #8 received at 45734 <at> debbugs.gnu.org (full text, mbox):

From: raid5atemyhomework <raid5atemyhomework <at> protonmail.com>
To: "45734 <at> debbugs.gnu.org" <45734 <at> debbugs.gnu.org>
Subject: [PATCH v2] gnu: update zfs.
Date: Sun, 10 Jan 2021 09:58:03 +0000

This patch instead updates to 2.0.1.

The upstream changes that most affect the transition from 0.8.x to 2.0.x are:

* FreeBSD and Linux have merged codebases, hence the reason why some files now have an extra `/os/linux` directory.
  * Files installed in `/etc/default/` now use simpler installation that supposedly works on both Linux and FreeBSD, but seems not to work in Guix specifically, for reasons I haven't explored. I just disabled its installation instead.
* The Linux build of the userland tools will now attempt to load the ZFS module, but this uses `/sbin/modprobe` which does not exist in Guix; I substitute this as well.

This new version of the patch, however, requires https://issues.guix.gnu.org/45723

I've checked that this works in a VM. I added patches from https://issues.guix.gnu.org/45692 https://issues.guix.gnu.org/45722 https://issues.guix.gnu.org/45723 in order to get a VM image with a ZFS 2.0.1.

The OpenZFS project will end-of-life the ZFS 0.8.x series soon. The ZFS 0.8.x series was inherited from the ZFS on Linux project and that particular sequence of releases is Linux-specific. Since OpenZFS has merged the Linux and FreeBSD forks, they decided to use the later version numbering 2.0.x and will now maintain only a single Linux-or-FreeBSD codebase, so Guix also needs to follow suit if Guix intends to have ZFS support in the foreseeable future.


From bad3a65ae19a5a3c74c700163138f6f17f3017de Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework <at> protonmail.com>
Date: Sat, 9 Jan 2021 11:53:58 +0800
Subject: [PATCH] gnu: update zfs.

* gnu/packages/file-systems.scm (zfs): Update version to 2.0.1.
---
 gnu/packages/file-systems.scm | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm
index f74d519b33..49afe61e95 100644
--- a/gnu/packages/file-systems.scm
+++ b/gnu/packages/file-systems.scm
@@ -847,7 +847,7 @@ APFS.")
 (define-public zfs
   (package
     (name "zfs")
-    (version "0.8.5")
+    (version "2.0.1")
     (outputs '("out" "module" "src"))
     (source
       (origin
@@ -856,7 +856,7 @@ APFS.")
                               "/download/zfs-" version
                               "/zfs-" version ".tar.gz"))
           (sha256
-           (base32 "0gfdnynmsxbhi97q73smrgmcw1k8zmlr1hgljfn38sk0kimivd6v"))))
+           (base32 "0y3992l4nzr67q18lz1kizw0za1shvqbpmsjz9shv4frh5ihllbi"))))
     (build-system linux-module-build-system)
     (arguments
      `(;; The ZFS kernel module should not be downloaded since the license
@@ -887,18 +887,31 @@ APFS.")
                    (util-linux (assoc-ref inputs "util-linux"))
                    (nfs-utils  (assoc-ref inputs "nfs-utils"))
                    (kmod       (assoc-ref inputs "kmod-runtime")))
+               (substitute* "etc/Makefile.in"
+                 ;; This just contains an example configuration file for
+                 ;; configuring ZFS on traditional init systems, skip it
+                 ;; since we cannot use it anyway; the install target becomes
+                 ;; misdirected.
+                 (("= default ") "= "))
+               (substitute* "lib/libzfs/os/linux/libzfs_util_os.c"
+                 ;; Use path to /gnu/store/*-kmod in actual path that is exec'ed.
+                 (("\"/sbin/modprobe\"")
+                  (string-append "\"" kmod "/bin/modprobe" "\""))
+                 ;; Just use 'modprobe' in message to user, since Guix
+                 ;; does not have a traditional /sbin/
+                 (("'/sbin/modprobe ") "'modprobe "))
                (substitute* "contrib/Makefile.in"
                  ;; This is not configurable nor is its hard-coded /usr prefix.
                  ((" initramfs") ""))
-               (substitute* "module/zfs/zfs_ctldir.c"
+               (substitute* "module/os/linux/zfs/zfs_ctldir.c"
                  (("/usr/bin/env\", \"umount")
                   (string-append util-linux "/bin/umount\", \"-n"))
                  (("/usr/bin/env\", \"mount")
                   (string-append util-linux "/bin/mount\", \"-n")))
-               (substitute* "lib/libzfs/libzfs_mount.c"
+               (substitute* "lib/libzfs/os/linux/libzfs_mount_os.c"
                  (("/bin/mount") (string-append util-linux "/bin/mount"))
                  (("/bin/umount") (string-append util-linux "/bin/umount")))
-               (substitute* "lib/libshare/nfs.c"
+               (substitute* "lib/libshare/os/linux/nfs.c"
                  (("/usr/sbin/exportfs")
                   (string-append nfs-utils "/sbin/exportfs")))
                (substitute* "config/zfs-build.m4"
--
2.30.0

Message #11 received at 45734 <at> debbugs.gnu.org (full text, mbox):

From: raid5atemyhomework <raid5atemyhomework <at> protonmail.com>
To: "45734 <at> debbugs.gnu.org" <45734 <at> debbugs.gnu.org>
Subject: Re: [PATCH v2] gnu: update zfs.
Date: Mon, 11 Jan 2021 10:23:16 +0000

For the patch to 2.0.1, I did the following testing:

* Included patches https://issues.guix.gnu.org/45692 https://issues.guix.gnu.org/45722 https://issues.guix.gnu.org/45723
  * Created a new VM image that includes ZFS using `(service zfs-service-type ...)`.  Linux Libre 5.4 though.
    * Expanded this image +10G and created a new partition and created a ZFS pool there, with a ZFS dataset, and wrote some text files and also downloaded the ZFS source release into the ZFS filesystem.  Then rebooted the VM and checked that the ZFS filesystem was still automounted, the contents look like they are as expected.
    * Created three extra disk images and booted the same image with the extra disks.  Added two of them as a mirror SLOG and the third as a L2ARC.  Then rebooted and checked that the pool still mounts fine.
    * Started the VM again with the extra disk images rearranged.  Checked ZFS pool status, the L2ARC and SLOG devices were correctly rearranged as well.  Did a few more rearrangements and checked that ZFS assigned the ZFS device to the correct use.
    * Started the VM again with one of the mirror SLOG devices missing.  Checked ZFS pool status, confirmed that the SLOG mirror was degraded but the pool is still up.

So all of it seems to be working fine so far.  I'm mostly satisfied with this. I'll probably need to add more code to make it work closer to how ZFS on other systems works (the current patches scan all devices rather than use `/etc/zfs/zpool.cache`, because I don't really know what `/etc/zfs/zpool.cache`).

With all those patches, ZFS on Guix supports:

* Automatic importing and mounting of ZFS filesystems (does not use `/etc/zfs/zpool.cache`; this theoretically speeds up the case where the computer has dozens or hundreds of disks, and protects in a setting where someone could potentially gain physical access to the computer and override sensitive locations by plugging in a USB that gets auto-imported (and auto-mounted) at boot by ZFS).
* `/home` on ZFS.
* L2ARC and SLOG.
* ZVOLs, accessible over `/dev/zvol/*` hierarchy.
* Can have pools on LUKS containers by adding them as dependencies of the `zfs-service-type` (untested).
* `file-system` declarations mounted on ZVOLs (untested).

Some other stuff is not supported yet:

* `zpool.cache` file, which replaces `fstab` but is not user-editable, for faster importing of ZFS pools.
* ZFS Event Daemon.  Traditionally this is configured by having the sysad manage a `/etc/zfs/zed.d/` directory; some bits of ZFS automation are provided by the ZFS release and the sysad is supposed to either symlink to those,  or copy it and modify, or remove, or replace with their own script.
* ZFS sharing over the network.  Probably need to go look at how NFS and Samba are started on Guix then figure this part out; NFS and Samba need to get started first, but I'm not sure how ZFS talks to those to get its filesystems shared.
* `/` on ZFS. Probably we need to have some kind of `initrd-kernel-module-service-type`, `initrd-kernel-module-loader-service-type`, and have kernel module parameter configuration passed in either by the kernel command line, or by the early `initrd` module loader (which isn't modprobe, by the way).
* Mounting in "legacy" mode where datasets are declared via `(file-system ...)` declarations.  Actually https://issues.guix.gnu.org/45643#3 has a patch for this as well.

Message #14 received at 45734 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: raid5atemyhomework <raid5atemyhomework <at> protonmail.com>
Cc: "45734-done <at> debbugs.gnu.org" <45734 <at> debbugs.gnu.org>
Subject: Re: [bug#45734] [PATCH v2] gnu: update zfs.
Date: Wed, 20 Jan 2021 14:44:01 +0200

[Message part 1 (text/plain, inline)]

Thanks for mentioning all the testing you did. Patch looks good.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 45734-done <at> debbugs.gnu.org (full text, mbox):

From: raid5atemyhomework <raid5atemyhomework <at> protonmail.com>
To: "45734-done <at> debbugs.gnu.org" <45734-done <at> debbugs.gnu.org>
Subject: [PATCH v2] gnu: update zfs.
Date: Mon, 25 Jan 2021 00:20:13 +0000

Merged in, closing.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.