GNU bug report logs - #45703
kernel-module-configuration-service for configuring kernel parameters

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix-patches; Reported by: Danny Milosavljevic <dannym@HIDDEN>; Keywords: patch; merged with #45643, #45692; dated Wed, 6 Jan 2021 19:42:02 UTC; Maintainer for guix-patches is guix-patches@HIDDEN.
Merged 45643 45692 45703. Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> debbugs.gnu.org. Full text available.
Merged 45692 45703. Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 7 Jan 2021 00:04:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 06 19:04:16 2021
Received: from localhost ([127.0.0.1]:45916 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kxIml-0007pL-KF
	for submit <at> debbugs.gnu.org; Wed, 06 Jan 2021 19:04:15 -0500
Received: from lists.gnu.org ([209.51.188.17]:32792)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raid5atemyhomework@HIDDEN>)
 id 1kxImk-0007pE-BS
 for submit <at> debbugs.gnu.org; Wed, 06 Jan 2021 19:04:14 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:54282)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <raid5atemyhomework@HIDDEN>)
 id 1kxImj-0007NT-1J
 for guix-patches@HIDDEN; Wed, 06 Jan 2021 19:04:14 -0500
Received: from mail-40135.protonmail.ch ([185.70.40.135]:47327)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <raid5atemyhomework@HIDDEN>)
 id 1kxIme-0004eO-1G
 for guix-patches@HIDDEN; Wed, 06 Jan 2021 19:04:12 -0500
Date: Thu, 07 Jan 2021 00:04:02 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;
 s=protonmail; t=1609977844;
 bh=d3Ry/xjDaP+dE6gE2ujwfTaJHs3PWHZ1uPkKd4mERVY=;
 h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From;
 b=movietufHyBuzlxm4+HNuyzY07Jp8L1HSwhvFyHXR6tDIX58we/ica2El8hg0Oe8o
 yh6EiwnKBdBCUk0WSewdsHp7vN/Tgv8HvEqRxYG0dHmyUP33cyxQKAyXbT4H9ktTV7
 7OJkgTlWw0KYS+gNcCRO5M2eTKxJWwZdpfrgY0tE=
To: Danny Milosavljevic <dannym@HIDDEN>
From: raid5atemyhomework <raid5atemyhomework@HIDDEN>
Subject: Re: kernel-module-configuration-service for configuring kernel
 parameters
Message-ID: <pilwAVCb0NEG83BVTSEqXkLCNEvBBrnb-QyKV_9PoF4xuCjSXKuF-F3_L5lSOke5BRjagVrUoSzJCST1fkigF162f9j76K12jBraNMAj7E4=@protonmail.com>
In-Reply-To: <20210106204134.38e83db4@HIDDEN>
References: <f9YNSdmHSIAVHWwDRwnh8QVeNjUG4M6YMJMntulZ3s8cYPVCBHrVIirwkuGxpYIduWVNAyi13dFVlqUKs7_QXPDBvbDe_JNNEC_SoNBTprk=@protonmail.com>
 <6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com>
 <20210106204134.38e83db4@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED,
 DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no
 autolearn=disabled version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 mailout.protonmail.ch
Received-SPF: pass client-ip=185.70.40.135;
 envelope-from=raid5atemyhomework@HIDDEN; helo=mail-40135.protonmail.ch
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
Cc: "ludo@HIDDEN" <ludo@HIDDEN>,
 raid5atemyhomework via Guix-patches via <guix-patches@HIDDEN>,
 "45692 <at> debbugs.gnu.org" <45692 <at> debbugs.gnu.org>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: raid5atemyhomework <raid5atemyhomework@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Hello Danny,


> See also https://issues.guix.info/issue/42193 for an earlier attempt (whi=
ch
> is already very far--but it has a bug somewhere). There's also already a
> kernel profile thing like you wrote in that patchset.
> (Note that I would prefer there not to be a "LOAD?" in there because it
> confuses loading the module (which is usually NOT started by user space
> but by the kernel on its own) and confguring the module (which has to be
> done by user space because it's specifying policy, not mechanism))

Looks like that patchset was merged in, so basically I can just depend on t=
hat? So the first patch in this patchset would be dropped?

> But I guess the ZFS Linux kernel module can't be built-in into the kernel
> anyway.
>
> But that's a special case--in general, it's very much possible to make mo=
dules
> built-in.

ZFS *can* be built-in to the kernel, Ubuntu does it. You can't distribute i=
t like that (Ubuntu distributes it like that but presumably they have enoug=
h lawyers to muddy the waters so that they can get away with it), but as th=
e documentation in this patch notes: the user has every right to do whateve=
r they want on the machine they own, including build a Linux kernel that ha=
s ZFS built-in and run it, they just can't make that version available to s=
omebody else.

So to go whole-hog, we would have a service that replaces the kernel packag=
e and inserts kernel module sources in-tree somehow, then compiles Linux-li=
bre on the user's machine. That would probably be a lot more painful to ins=
tall ZFS with (the user has to recompile the whole kernel at each update of=
 either kernel or ZFS, whereas with a kernel module the user has to recompi=
le just the kernel module), so maybe kernel module is still better overall.


> > -                       ;; You'd think we could've used kernel-module-l=
oader-service-type,
> >
> >
>
> Definitely.
>
> > -                       ;; but the kernel-module-loader shepherd servic=
e is dependent on
> >
> >
> > -                       ;; file-systems,
> >
> >
>
> Yes--but why is that dependent on 'file-systems ? Is it because it needs =
/proc ?
> Or is it an oversight ? I would prefer to get rid of this dependency and =
then
> use kernel-module-loader-service-type.

Dunno --- one VM I tested, I removed the `zfs-scan-automount` shepherd serv=
ice from the `file-systems` target, and the VM still wouldn't boot, claimin=
g a stack overflow (the same error which I got when I was still trying to u=
se kernel-module-loader-service-type here).  Or maybe I just got confused w=
ith which VM was which, testing VMs wasn't a stress-free vacation.  I just =
want ZFS, because MD RAID5 ate my homework, this is getting tiresome...

One thing I notice about `kernel-module-loader-service-type` is that it's n=
ot instantiated in essential services, or indeed anywhere in Guix.  A few s=
ervices *do* extend it.  But my **very rough** understanding is that if you=
're going to extend a service, it had better be instantiated *once* in the =
list of services.

In particular I note that the documentation for `kernel-module-loader-servi=
ce-type` shows an example where it uses `service` to program the `kernel-mo=
dule-loader-service-type`, not `simple-service`.  This suggests to me that =
`kernel-module-loader-service-type` is broken because it's not in the list =
of essential services but is extensible.  Maybe.  It's designed as an exten=
sible service, but isn't instantiated at default.  Maybe that's what really=
 bit me and not the shepherd circular dependency loop? *shrug*


>
> Also, this manual loading of kernel modules is not supposed to be the way=
 to
> do things in Linux. That a kernel module was compiled as a module is
> an implementation detail--so Linux should (and usually does) automaticall=
y
> load kernel modules the first time a device for them is accessed (after a=
ll,
> how would user space know whether something is compiled as a module or
> built-in--that would be too much to ask).

So how do I get ZFS loaded? Note that the devices it targets are block devi=
ces and it needs to scan for block devices that are formatted for ZFS.  Do =
other filesystems have some autoload rule?

Thanks
raid5atemyhomework




Information forwarded to guix-patches@HIDDEN:
bug#45703; Package guix-patches. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 6 Jan 2021 19:41:52 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 06 14:41:51 2021
Received: from localhost ([127.0.0.1]:45622 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kxEgp-0001Iv-FD
	for submit <at> debbugs.gnu.org; Wed, 06 Jan 2021 14:41:51 -0500
Received: from lists.gnu.org ([209.51.188.17]:42166)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dannym@HIDDEN>) id 1kxEgo-0001Io-13
 for submit <at> debbugs.gnu.org; Wed, 06 Jan 2021 14:41:50 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:39946)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <dannym@HIDDEN>)
 id 1kxEgn-0005de-Pm
 for guix-patches@HIDDEN; Wed, 06 Jan 2021 14:41:49 -0500
Received: from dd26836.kasserver.com ([85.13.145.193]:35380)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <dannym@HIDDEN>)
 id 1kxEgi-0006pb-Pe; Wed, 06 Jan 2021 14:41:46 -0500
Received: from localhost (80-110-127-104.cgn.dynamic.surfer.at
 [80.110.127.104])
 by dd26836.kasserver.com (Postfix) with ESMTPSA id B8C183365398;
 Wed,  6 Jan 2021 20:41:39 +0100 (CET)
Date: Wed, 6 Jan 2021 20:41:34 +0100
From: Danny Milosavljevic <dannym@HIDDEN>
Subject: kernel-module-configuration-service for configuring kernel parameters
Message-ID: <20210106204134.38e83db4@HIDDEN>
In-Reply-To: <6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com>
References: <f9YNSdmHSIAVHWwDRwnh8QVeNjUG4M6YMJMntulZ3s8cYPVCBHrVIirwkuGxpYIduWVNAyi13dFVlqUKs7_QXPDBvbDe_JNNEC_SoNBTprk=@protonmail.com>
 <6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com>
X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/xXlbAOfUGZoWGj8WvcLyYPd";
 protocol="application/pgp-signature"; micalg=pgp-sha512
Received-SPF: none client-ip=85.13.145.193;
 envelope-from=dannym@HIDDEN; helo=dd26836.kasserver.com
X-Spam_score_int: -15
X-Spam_score: -1.6
X-Spam_bar: -
X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, MISSING_HEADERS=1.021,
 RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001,
 SPF_NONE=0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.1 (-)
X-Debbugs-Envelope-To: submit
Cc: raid5atemyhomework <raid5atemyhomework@HIDDEN>, ludo@HIDDEN,
 raid5atemyhomework via Guix-patches via <guix-patches@HIDDEN>,
 "45692 <at> debbugs.gnu.org" <45692 <at> debbugs.gnu.org>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.1 (--)

--Sig_/xXlbAOfUGZoWGj8WvcLyYPd
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Hi raid5atemyhomework,
Hi everyone,

@raid5atemyhomework, thanks for all the patches!

On Wed, 06 Jan 2021 15:57:19 +0000
raid5atemyhomework via Guix-patches via <guix-patches@HIDDEN> wrote:

> +@item @code{options} (default: @code{'()})
> +A list of string options to pass as options to the ZFS module.

> +These will be put in a @file{/etc/modprobe.d/zfs.conf} file,
> +for example setting this to @code{'("zfs_admin_snapshot=3D1"
> +"zfs_trim_extent_bytes_min=3D0")} will create the following file:

Sure, but it would be better to create a way to configure those module para=
meters
in Guix in a declarative way, first.

Your new kernel-loadable-module-service-type would be a good template
to write a kernel-module-configuration-service that can be extended by
other services.  The latter should allow users to parametrize the kernel
in general via the operating-system form, and also allow services to extend
such configuration (with merging, and conflict detection).

It would be used:

(1) To declaratively specify the contents of something like /etc/modprobe.d=
 .
It shouldn't even be called "/etc/modprobe.d"--it should also be in the sto=
re
instead.  This directory is only useful for non-Linux-builtins.

(2) If the "module" is built-in then the kernel command line must get the
options instead.  (in fact, it works as a kernel command line option also
if it's a loadable module--so not sure we need /etc/modprobe.d at all--at
least at first.  But there's probably a maximal length for the kernel comma=
nd
line that we could exceed if we did that long term)

I know it's annoying that Guix doesn't have this facility already, but the
time to introduce an interface for /etc/modprobe.d and the kernel command
line for builtin modules is before other services introduce their own
ad hoc way to create /etc/modprobe.d--like this tries to do here.

See also https://issues.guix.info/issue/42193 for an earlier attempt (which
is already very far--but it has a bug somewhere).  There's also already a
kernel profile thing like you wrote in that patchset.
(Note that I would prefer there not to be a "LOAD?" in there because it
confuses loading the module (which is usually NOT started by user space
but by the kernel on its own) and confguring the module (which has to be
done by user space because it's specifying policy, not mechanism))

Also, because the kernel usually loads loadable modules on its own (potenti=
ally
really early), /etc/modprobe.d has to be preset and known to the modprobe
executable VERY EARLY (via environment variable MODPROBE_OPTIONS--see=20
gnu/services.scm %modprobe-wrapper).

It is totally possible that some modules in the initrd need options, too
(see load-linux-modules-from-directory for where this would need to go).
load-linux-module/fd already accepts options and flags--but both are not gi=
ven
on the call.  For this, part of future kernel-module-configuration entries
(the ones needed for modules in the initrd) should be copied into the initr=
d,
too.

Then there's the handoff between initrd and main system.  It would be bad
if the kernel tried and succeeded to load a module that is not in the initrd
just before the modprobe.d directory is set up (because it would be loaded
without passing the options the user configured)--so that needs to be avoid=
ed.

@ludo: Could you help here?

> +@example
> +options zfs zfs_admin_snapshot=3D1 zfs_trim_extent_bytes_min=3D0

Note:

This can be usefully put in a modprobe.d-like directory if zfs is a module,=
 but
not if it's built into the kernel.  But it can be put into the kernel comma=
nd
line in both cases.

But I guess the ZFS Linux kernel module can't be built-in into the kernel
anyway.

But that's a special case--in general, it's very much possible to make modu=
les
built-in.

> +                     ;; You'd think we could've used kernel-module-loade=
r-service-type,

Definitely.

> +                     ;; but the kernel-module-loader shepherd service is=
 dependent on
> +                     ;; file-systems,=20

Yes--but why is that dependent on 'file-systems ? Is it because it needs /p=
roc ?
Or is it an oversight ?  I would prefer to get rid of this dependency and t=
hen
use kernel-module-loader-service-type.

Also, this manual loading of kernel modules is not supposed to be the way to
do things in Linux.  That a kernel module was compiled as a module is
an *implementation detail*--so Linux should (and usually does) automatically
load kernel modules the first time a device for them is accessed (after all,
how would user space know whether something is compiled as a module or
built-in--that would be too much to ask).

Linux is not a microkernel, so the kind of modularily modprobe.d suggests
exists does not in fact exist in kernel space--even though Linux does a good
job faking it:  modprobe.d contains:

* "alias": a feature to configure aliases, with wildcards (only one level of
aliases allowed!)
* "options" per module (also works for aliases with wildcards!  That will be
"fun" to map to Guix)
* "install" in order to run some custom executable instead of loading the
module.
* "remove" in order to run some custom executable instead of unloading the
module.
* "blacklist" to ignore specific internal aliases of a module (that does
not do what one would intuitively think!).

If the file name of the regular file under /etc/modprobe.d is not used for
anything, then we can just have one file /gnu/store/*modprobe.d/guix.conf
in total in there.

Then there are sysctl kernel parameters--but those Guix already exposes via
sysctl-service-type.  But those should also be made able to be extended
by other services, and merge conflicts should be handled.  For example, use=
rs
often set net.ipv4.ip_forward=3D1 (for example via sysctl).

Thank you for all your effort to make ZFS work nicely in Guix.

--Sig_/xXlbAOfUGZoWGj8WvcLyYPd
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/2Em4ACgkQ5xo1VCww
uqX7VQf/bHxCWBZt0FXRb5E0zvb0AZVwMPajkYvOBjfXJC+XcMbgjP3FKNW7VLGK
FakRsyWnVX6gtFPxUDImjnCIGcY5LGYuaHxnDR4CTqzDBb06LRAMeJ1YTIyr+j+S
ZO17TG4hZa+xWLqM6MUpfEwuc1I/b5o9oN39XJJLJAWJOdXMi2ZGwOKY8dpYhaDw
zDPoOb3xHj9oDyXyWjOfb/8VK8bF8+L9Fzy3Bjt2WiG58j5jXpkYz9ZQe3zA1hbH
zwFmmKqkFbL2gFhTxGWuQrp6x29T60lzBwUcjG58waCQee23TVFR9K/v7fvjrBal
kHMS+9/Dc8FBIdzlHAbxj8/74qYkUQ==
=nYdc
-----END PGP SIGNATURE-----

--Sig_/xXlbAOfUGZoWGj8WvcLyYPd--




Acknowledgement sent to Danny Milosavljevic <dannym@HIDDEN>:
New bug report received and forwarded. Copy sent to guix-patches@HIDDEN. Full text available.
Report forwarded to guix-patches@HIDDEN:
bug#45703; Package guix-patches. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Wed, 10 Feb 2021 16:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.