GNU bug report logs - #42478
[PATCH] services: Add zram-device-service.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 42478 in the body.
You can then email your comments to 42478 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: guix-patches <at> gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH] services: Add zram-device-service.
Date: Wed, 22 Jul 2020 21:10:25 +0300

* gnu/services/linux.scm (<zram-device-configuration>): New record.
(zram-device-service-type): New variable.
* doc/guix.texi (Linux Services): Document it.
* tests/services/linux.scm (zram-device-configuration->udev-string): New
test.
---
 doc/guix.texi            | 45 +++++++++++++++++++++++++
 gnu/services/linux.scm   | 73 +++++++++++++++++++++++++++++++++++++++-
 tests/services/linux.scm | 29 ++++++++++++++++
 3 files changed, 146 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 8696a9b554..f656c31fab 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -27127,6 +27127,51 @@ parameters, can be done as follow:
 @end lisp
 @end deffn
 
+@cindex zram
+@cindex compressed swap
+@cindex Compressed RAM-based block devices
+@subsubheading Zram Device Service
+
+The Zram device service provides a compressed swap device in system
+memory.  The Linux Kernel documentation has more information about
+@uref{https://www.kernel.org/doc/html/latest/admin-guide/blockdev/zram.html,zram}
+devices.
+
+@deffn {Scheme Variable} zram-device-service-type
+This service creates the zram block device, formats it as swap and
+enables it as a swap device.  The service's value is a
+@code{zram-device-configuration} record.
+
+@deftp {Data Type} zram-device-configuration
+This is the data type representing the configuration for the zram-device
+service.
+
+@table @asis
+@item @code{disksize} (default @var{"0"})
+This is the amount of space you wish to provide for the zram device.  It
+accepts a string and can be a number of bytes or use a suffix, eg.:
+@var{2G}.
+@item @code{comp_algorithm} (default @var{"lzo"})
+This is the compression algorithm you wish to use.  It is difficult to
+list all the possible compression options, but common ones supported by
+Guix's Linux Libre Kernel include @var{lzo}, @var{lz4} and @var{zstd}.
+@item @code{mem_limit} (default @var{"0"})
+This is the maximum amount of memory which the zram device can use.
+Setting it to '0' disables the limit.  While it is generally expected
+that compression will be 2:1, it is possible that uncompressable data
+can be written to swap and this is a method to limit how much memory can
+be used.  It accepts a string and can be a number of bytes or use a
+suffix, eg.: @var{2G}.
+@item @code{priority} (default @var{"-1"})
+This is the priority of the swap device created from the zram device.
+@code{swapon} accepts values between -1 and 32767, with higher values
+indicating higher priority.  Higher priority swap will generally be used
+first.
+@end table
+
+@end deftp
+@end deffn
+
 @node Hurd Services
 @subsection Hurd Services
 
diff --git a/gnu/services/linux.scm b/gnu/services/linux.scm
index 12934c2084..7073a06cd7 100644
--- a/gnu/services/linux.scm
+++ b/gnu/services/linux.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
 ;;; Copyright © 2020 Brice Waegeneire <brice <at> waegenei.re>
+;;; Copyright © 2020 Efraim Flashner <efraim <at> flashner.co.il>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -22,6 +23,7 @@
   #:use-module (guix records)
   #:use-module (guix modules)
   #:use-module (gnu services)
+  #:use-module (gnu services base)
   #:use-module (gnu services shepherd)
   #:use-module (gnu packages linux)
   #:use-module (srfi srfi-1)
@@ -42,7 +44,15 @@
             earlyoom-configuration-send-notification-command
             earlyoom-service-type
 
-            kernel-module-loader-service-type))
+            kernel-module-loader-service-type
+
+            zram-device-configuration
+            zram-device-configuration?
+            zram-device-configuration-disksize
+            zram-device-configuration-comp_algorithm
+            zram-device-configuration-mem_limit
+            zram-device-configuration-priority
+            zram-device-service-type))
 
 
 ;;;
@@ -177,3 +187,64 @@ representation."
    (compose concatenate)
    (extend append)
    (default-value '())))
+
+
+;;;
+;;; Kernel module loader.
+;;;
+
+(define-record-type* <zram-device-configuration>
+  zram-device-configuration make-zram-device-configuration
+  zram-device-configuration?
+  (disksize         zram-device-configration-disksize
+                    (default "0"))      ; string
+  (comp_algorithm   zram-device-configuration-comp_algorithm
+                    (default "lzo"))    ; string
+  (mem_limit        zram-device-configuration-mem_limit
+                    (default "0"))      ; string
+  (priority         zram-device-configuration-priority
+                    (default "-1")))    ; string
+
+(define (zram-device-configuration->udev-string config)
+  "Translate a <zram-device-configuration> into a string which can be
+placed in a udev rules file."
+  (match config
+    (($ <zram-device-configuration> disksize comp_algorithm mem_limit priority)
+     (string-append
+       "KERNEL==\"zram0\", "
+       "ATTR{comp_algorithm}=\"" comp_algorithm "\" "
+       (if (not (equal? "0" disksize))
+         (string-append "ATTR{disksize}=\"" disksize "\" ")
+         "")
+       (if (not (equal? "0" mem_limit))
+         (string-append "ATTR{mem_limit}=\"" mem_limit "\" ")
+         "")
+       "RUN+=\"/run/current-system/profile/sbin/mkswap /dev/zram0\" "
+       "RUN+=\"/run/current-system/profile/sbin/swapon "
+       (if (not (equal? "-1" priority))
+         (string-append "--priority " priority " ")
+         "")
+       "/dev/zram0\"\n"))))
+
+(define %zram-device-config
+  `("modprobe.d/zram.conf"
+    ,(plain-file "zram.conf"
+                 "options zram num_devices=1")))
+
+(define (zram-device-udev-rule config)
+  (file->udev-rule "99-zram.rules"
+                   (plain-file "99-zram.rules"
+                               (zram-device-configuration->udev-string config))))
+
+(define zram-device-service-type
+  (service-type
+    (name 'zram)
+    (default-value (zram-device-configuration))
+    (extensions
+      (list (service-extension kernel-module-loader-service-type
+                               (const (list "zram")))
+            (service-extension etc-service-type
+                               (const (list %zram-device-config)))
+            (service-extension udev-service-type
+                               (compose list zram-device-udev-rule))))
+    (description "Creates a zram swap device.")))
diff --git a/tests/services/linux.scm b/tests/services/linux.scm
index 8ad119c49f..828aa86bd6 100644
--- a/tests/services/linux.scm
+++ b/tests/services/linux.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
+;;; Copyright © 2020 Efraim Flashner <efraim <at> flashner.co.il>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -54,4 +55,32 @@
         "-N" "python \"/some/path/notify-all-users.py\"")
   (earlyoom-configuration->command-line-args %earlyoom-configuration-sample))
 
+
+;;;
+;;; Zram swap device.
+;;;
+
+(define zram-device-configuration->udev-string
+  (@@ (gnu services linux) zram-device-configuration->udev-string))
+
+(define %zram-swap-device-test-1
+  (zram-device-configuration
+    (disksize "2G")
+    (comp_algorithm "zstd")
+    (mem_limit "1G")
+    (priority "42")))
+
+(test-equal "zram-device-configuration->udev-string"
+  "KERNEL==\"zram0\", ATTR{comp_algorithm}=\"zstd\" ATTR{disksize}=\"2G\" ATTR{mem_limit}=\"1G\" RUN+=\"/run/current-system/profile/sbin/mkswap /dev/zram0\" RUN+=\"/run/current-system/profile/sbin/swapon --priority 42 /dev/zram0\"\n"
+  (zram-device-configuration->udev-string %zram-swap-device-test-1))
+
+(define %zram-swap-device-test-2
+  (zram-device-configuration
+    (disksize "1G")
+    (comp_algorithm "lz4")))
+
+(test-equal "zram-device-configuration->udev-string"
+  "KERNEL==\"zram0\", ATTR{comp_algorithm}=\"lz4\" ATTR{disksize}=\"1G\" RUN+=\"/run/current-system/profile/sbin/mkswap /dev/zram0\" RUN+=\"/run/current-system/profile/sbin/swapon /dev/zram0\"\n"
+  (zram-device-configuration->udev-string %zram-swap-device-test-2))
+
 (test-end "linux-services")
-- 
2.27.0

Message #8 received at 42478 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <marius <at> gnu.org>
To: Efraim Flashner <efraim <at> flashner.co.il>, 42478 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: [bug#42478] [PATCH] services: Add zram-device-service.
Date: Sat, 25 Jul 2020 19:00:54 +0200

[Message part 1 (text/plain, inline)]

Efraim Flashner <efraim <at> flashner.co.il> writes:

> * gnu/services/linux.scm (<zram-device-configuration>): New record.
> (zram-device-service-type): New variable.
> * doc/guix.texi (Linux Services): Document it.
> * tests/services/linux.scm (zram-device-configuration->udev-string): New
> test.

[...]

> diff --git a/doc/guix.texi b/doc/guix.texi
> index 8696a9b554..f656c31fab 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -27127,6 +27127,51 @@ parameters, can be done as follow:
>  @end lisp
>  @end deffn
>  
> +@cindex zram
> +@cindex compressed swap
> +@cindex Compressed RAM-based block devices
> +@subsubheading Zram Device Service
> +
> +The Zram device service provides a compressed swap device in system
> +memory.  The Linux Kernel documentation has more information about
> +@uref{https://www.kernel.org/doc/html/latest/admin-guide/blockdev/zram.html,zram}
> +devices.
> +
> +@deffn {Scheme Variable} zram-device-service-type
> +This service creates the zram block device, formats it as swap and
> +enables it as a swap device.  The service's value is a
> +@code{zram-device-configuration} record.
> +
> +@deftp {Data Type} zram-device-configuration
> +This is the data type representing the configuration for the zram-device
> +service.
> +
> +@table @asis
> +@item @code{disksize} (default @var{"0"})
> +This is the amount of space you wish to provide for the zram device.  It
> +accepts a string and can be a number of bytes or use a suffix, eg.:
> +@var{2G}.

Perhaps this could accept both an integer and a string?  What does a
size 0 device do, would it make sense to not have a default here?

Also, would it make sense to name it "size" instad of "disksize"?

> +@item @code{comp_algorithm} (default @var{"lzo"})
> +This is the compression algorithm you wish to use.  It is difficult to
> +list all the possible compression options, but common ones supported by
> +Guix's Linux Libre Kernel include @var{lzo}, @var{lz4} and @var{zstd}.

"comp_algorithm" is not very idiomatic :-)

Either "compression-algorithm" or just "compression" IMO.  I'd also
prefer a symbol instead of a string (or both!), but no strong opinion.

> +@item @code{mem_limit} (default @var{"0"})

"mem_limit" should instead be "memory-limit" or just "limit".

Accepting an integer here too would be nice!  :-)

> +This is the maximum amount of memory which the zram device can use.
> +Setting it to '0' disables the limit.  While it is generally expected
> +that compression will be 2:1, it is possible that uncompressable data
> +can be written to swap and this is a method to limit how much memory can
> +be used.  It accepts a string and can be a number of bytes or use a
> +suffix, eg.: @var{2G}.
> +@item @code{priority} (default @var{"-1"})

Just an integer I suppose?
  
> +
> +;;;
> +;;; Zram swap device.
> +;;;
> +
> +(define zram-device-configuration->udev-string
> +  (@@ (gnu services linux) zram-device-configuration->udev-string))

Would it make sense to export this, or is it strictly for internal use?

Great that you were able to add unit tests for the functionality.  I
think in this case we could also have a system test that checks that a
zram device was created?  But it can come later.

Other than the cosmetic/idiomatic issues looks great to me!

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 42478 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Marius Bakke <marius <at> gnu.org>
Cc: 42478 <at> debbugs.gnu.org
Subject: Re: [bug#42478] [PATCH] services: Add zram-device-service.
Date: Mon, 27 Jul 2020 15:01:29 +0300

[Message part 1 (text/plain, inline)]

On Sat, Jul 25, 2020 at 07:00:54PM +0200, Marius Bakke wrote:
> Efraim Flashner <efraim <at> flashner.co.il> writes:
> 
> > * gnu/services/linux.scm (<zram-device-configuration>): New record.
> > (zram-device-service-type): New variable.
> > * doc/guix.texi (Linux Services): Document it.
> > * tests/services/linux.scm (zram-device-configuration->udev-string): New
> > test.
> 
> [...]
> 
> > diff --git a/doc/guix.texi b/doc/guix.texi
> > index 8696a9b554..f656c31fab 100644
> > --- a/doc/guix.texi
> > +++ b/doc/guix.texi
> > @@ -27127,6 +27127,51 @@ parameters, can be done as follow:
> >  @end lisp
> >  @end deffn
> >  
> > +@cindex zram
> > +@cindex compressed swap
> > +@cindex Compressed RAM-based block devices
> > +@subsubheading Zram Device Service
> > +
> > +The Zram device service provides a compressed swap device in system
> > +memory.  The Linux Kernel documentation has more information about
> > +@uref{https://www.kernel.org/doc/html/latest/admin-guide/blockdev/zram.html,zram}
> > +devices.
> > +
> > +@deffn {Scheme Variable} zram-device-service-type
> > +This service creates the zram block device, formats it as swap and
> > +enables it as a swap device.  The service's value is a
> > +@code{zram-device-configuration} record.
> > +
> > +@deftp {Data Type} zram-device-configuration
> > +This is the data type representing the configuration for the zram-device
> > +service.
> > +
> > +@table @asis
> > +@item @code{disksize} (default @var{"0"})
> > +This is the amount of space you wish to provide for the zram device.  It
> > +accepts a string and can be a number of bytes or use a suffix, eg.:
> > +@var{2G}.
> 
> Perhaps this could accept both an integer and a string?  What does a
> size 0 device do, would it make sense to not have a default here?

An integer or a string would certainly be more convenient for users, it
just needs a bit more logic to add in number->string as needed.

A size 0 device is pretty useless. It still creates the zram device but
with a size of 0. I put in 0 as the default because that's the default
if you don't choose anything when creating it. I suppose it would be
better to make the default 1G or 1024**3

> 
> Also, would it make sense to name it "size" instad of "disksize"?

That's the internal name but I'm not opposed to changing it to 'size'.

> 
> > +@item @code{comp_algorithm} (default @var{"lzo"})
> > +This is the compression algorithm you wish to use.  It is difficult to
> > +list all the possible compression options, but common ones supported by
> > +Guix's Linux Libre Kernel include @var{lzo}, @var{lz4} and @var{zstd}.
> 
> "comp_algorithm" is not very idiomatic :-)
> 
> Either "compression-algorithm" or just "compression" IMO.  I'd also
> prefer a symbol instead of a string (or both!), but no strong opinion.

compression-algorithm does seem better. I'll change it to a symbol.

> 
> > +@item @code{mem_limit} (default @var{"0"})
> 
> "mem_limit" should instead be "memory-limit" or just "limit".

I like memory-limit, limit would make me wonder what the difference is
between this an size.

> 
> Accepting an integer here too would be nice!  :-)

Noted :)

> 
> > +This is the maximum amount of memory which the zram device can use.
> > +Setting it to '0' disables the limit.  While it is generally expected
> > +that compression will be 2:1, it is possible that uncompressable data
> > +can be written to swap and this is a method to limit how much memory can
> > +be used.  It accepts a string and can be a number of bytes or use a
> > +suffix, eg.: @var{2G}.
> > +@item @code{priority} (default @var{"-1"})
> 
> Just an integer I suppose?

It would make more sense than a string.

>   
> > +
> > +;;;
> > +;;; Zram swap device.
> > +;;;
> > +
> > +(define zram-device-configuration->udev-string
> > +  (@@ (gnu services linux) zram-device-configuration->udev-string))
> 
> Would it make sense to export this, or is it strictly for internal use?

I'm not opposed to exporting it but I'm not really sure what else you'd
do with it. I think it would be nice to not hardcode zram0 so there can
be more than one or to make it so it can be mounted as /tmp but there's
nothing currently exposed as a variable to change how it works.

> 
> Great that you were able to add unit tests for the functionality.  I
> think in this case we could also have a system test that checks that a
> zram device was created?  But it can come later.

I think it'd be good to add a system test to make sure there's actually
42 MB of swap or something. I'll have to think about how to do that.

> 
> Other than the cosmetic/idiomatic issues looks great to me!

Thanks

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 42478 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Marius Bakke <marius <at> gnu.org>
Cc: 42478 <at> debbugs.gnu.org
Subject: Re: [bug#42478] [PATCH v2] services: Add zram-device-service.
Date: Mon, 27 Jul 2020 15:03:47 +0300

[Message part 1 (text/plain, inline)]

version 2 of the patch


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[v2-0001-services-Add-zram-device-service.patch (text/plain, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 42478-done <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: 42478-done <at> debbugs.gnu.org
Subject: Re: [bug#42478] [PATCH v2] services: Add zram-device-service.
Date: Sun, 2 Aug 2020 15:56:54 +0300

[Message part 1 (text/plain, inline)]

Patch pushed!


-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.