Received: (at 48266) by debbugs.gnu.org; 11 May 2021 21:08:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 11 17:08:47 2021 Received: from localhost ([127.0.0.1]:37646 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1lgZcU-00049F-Qh for submit <at> debbugs.gnu.org; Tue, 11 May 2021 17:08:47 -0400 Received: from eggs.gnu.org ([209.51.188.92]:36464) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1lgZcS-000492-BJ for 48266 <at> debbugs.gnu.org; Tue, 11 May 2021 17:08:45 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:38198) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1lgZcM-0004pZ-Ka; Tue, 11 May 2021 17:08:38 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=33400 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1lgZcM-0001rQ-75; Tue, 11 May 2021 17:08:38 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Vagrant Cascadian <vagrant@HIDDEN> Subject: Re: bug#48266: support dynamic loading of modules from initrd References: <87pmy3shrq.fsf@yucca> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 22 =?utf-8?Q?Flor=C3=A9al?= an 229 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 11 May 2021 23:08:35 +0200 In-Reply-To: <87pmy3shrq.fsf@yucca> (Vagrant Cascadian's message of "Thu, 06 May 2021 13:56:41 -0700") Message-ID: <87sg2tugfg.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 48266 Cc: 48266 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi! Vagrant Cascadian <vagrant@HIDDEN> skribis: > Initially, I tried adding just the obviously mmc related modules, but > this gave me guile prompt from the initramfs as it failed to find the > rootfs. Notably, even with the above list, I still need to explore > additional modules to load in order to get the display and keyboard to > work from the initramfs, in case I wanted to use this with encrypted > rootfs... > > The above list of modules could almost certainly be trimmed, but even > getting a bootable system for pinebook pro took about 20 tries, and the > process of defining the modules is further complicated by several > factors... > > * The filesystem names of the modules (e.g. dw_mmc-pltfm) do not > necessarily match the runtime name from lsmod (e.g. dw_mmc_pltfm). > This becomes a good deal of trial and error to figure out which > modules to add. > > * One needs to manually resolve the soft and hard dependencies of the > modules, and ensure they are loaded, and include them in the list. > > * If upstream changes the module name (which does happen from time to > time), you have to update the system config.scm to the new module > names. > > * If some functionality changes from a module to a built-in, or > vice-versa, the system config.scm needs manual updating. > > * Switching system between two different arm boards potentially requires > entirely different lists of modules. Note that =E2=80=98guix system init=E2=80=99, =E2=80=98reconfigure=E2=80=99= , and =E2=80=98deploy=E2=80=99 error out if drivers for a storage device are missing (see =E2=80=98check-device-initrd-modules=E2=80=99). Now, that doesn=E2=80=99t help if you=E2=80=99re using =E2=80=98guix system= image=E2=80=99, which perhaps is what you were doing? I wonder how we could take advantage of that code in such a scenario. > Rather than handling modules one at a time, I would propose to at least > add an option that can add whole directory trees of modules to the > initrd (e.g. kernel/drivers/usb/)... and then use modprobe (or udev?) > to handle the dependencies. Maybe opt-in at first, but longer-term, > explore making it default? I remember Danny and I worked on something along these lines in the past but it didn=E2=80=99t completely materialize (some of the machinery is alre= ady here, though). That said, we still wouldn=E2=80=99t want to include too mu= ch in the initrd, would we? Thanks, Ludo=E2=80=99.
bug-guix@HIDDEN:bug#48266; Package guix.
Full text available.
Received: (at submit) by debbugs.gnu.org; 6 May 2021 20:56:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu May 06 16:56:59 2021
Received: from localhost ([127.0.0.1]:39617 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1lel3L-00082c-EQ
for submit <at> debbugs.gnu.org; Thu, 06 May 2021 16:56:59 -0400
Received: from lists.gnu.org ([209.51.188.17]:43812)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <vagrant@HIDDEN>) id 1lel3I-00082U-BZ
for submit <at> debbugs.gnu.org; Thu, 06 May 2021 16:56:58 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:36164)
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <vagrant@HIDDEN>)
id 1lel3I-000385-1F
for bug-guix@HIDDEN; Thu, 06 May 2021 16:56:56 -0400
Received: from cascadia.aikidev.net ([2600:3c01:e000:267:0:a171:de7:c]:60458)
by eggs.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <vagrant@HIDDEN>) id 1lel3C-0005j1-BK
for bug-guix@HIDDEN; Thu, 06 May 2021 16:56:55 -0400
Received: from localhost (97-120-1-76.ptld.qwest.net [97.120.1.76])
(Authenticated sender: vagrant@HIDDEN)
by cascadia.aikidev.net (Postfix) with ESMTPSA id AC6E91A904
for <bug-guix@HIDDEN>; Thu, 6 May 2021 13:56:45 -0700 (PDT)
From: Vagrant Cascadian <vagrant@HIDDEN>
To: bug-guix@HIDDEN
Subject: support dynamic loading of modules from initrd
Date: Thu, 06 May 2021 13:56:41 -0700
Message-ID: <87pmy3shrq.fsf@yucca>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha512; protocol="application/pgp-signature"
Received-SPF: none client-ip=2600:3c01:e000:267:0:a171:de7:c;
envelope-from=vagrant@HIDDEN; helo=cascadia.aikidev.net
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001,
SPF_NONE=0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable
There should be an option to support dynamic loading of modules from the
initrd.
I recently pushed some changes to use the "linux-libre" kernel with
pinebook pro:
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3Dd330d63a29f35ebcb=
ebce19b13d49c69d38a5815
But in order to even boot, I need to add a lot of modules to
initrd-modules:
(initrd-modules=20
(append
(list
"rtc-rk808"
"dw_mmc"
"dw_mmc-pltfm"
"dw_mmc-rockchip"
"joydev"
"bluetooth"
"jitterentropy_rng"
"hid_generic"
"videodev"
"ghash_ce"
"gf128mul"
"ansi_cprng"
"mc"
"sha2_ce"
"usbhid"
"panfrost"
"ecdh_generic"
;; "fusb302"
"ofpart"
;; "tcpm"
"hid"
"ecc"
"evdev"
"leds_gpio"
"io_domain"
"dw_wdt"
;; "rockchip-thermal"
"cw2015_battery"
"pwm-rockchip"
;; "gpio_charger"
"cpufreq_dt"
"configfs"
"xhci-plat-hcd"
"xhci-hcd"
"rk808-regulator"
"clk-rk808"
"udc-core"
"ulpi"
"fan53555"
"rk808"
"pwm-regulator"
"fixed"
"gpio_keys"
"cec"
;; "phy-rockchip-typec"
"phy-rockchip-emmc"
"phy-rockchip-inno-usb2"
;; "analogix-dp"
"sdhci-of-arasan"
"sdhci-pltfm"
"cqhci"
"drm_kms_helper"
"ohci-platform"
"ohci-hcd"
"ehci-platform"
"panel-simple"
"ehci-hcd"
"sdhci"
"drm"
"i2c-rk3x"
"usbcore"
"pl330"
"pwm_bl"
"dwc3"
)
%base-initrd-modules))
Initially, I tried adding just the obviously mmc related modules, but
this gave me guile prompt from the initramfs as it failed to find the
rootfs. Notably, even with the above list, I still need to explore
additional modules to load in order to get the display and keyboard to
work from the initramfs, in case I wanted to use this with encrypted
rootfs...
The above list of modules could almost certainly be trimmed, but even
getting a bootable system for pinebook pro took about 20 tries, and the
process of defining the modules is further complicated by several
factors...
* The filesystem names of the modules (e.g. dw_mmc-pltfm) do not
necessarily match the runtime name from lsmod (e.g. dw_mmc_pltfm).
This becomes a good deal of trial and error to figure out which
modules to add.
* One needs to manually resolve the soft and hard dependencies of the
modules, and ensure they are loaded, and include them in the list.
* If upstream changes the module name (which does happen from time to
time), you have to update the system config.scm to the new module
names.
* If some functionality changes from a module to a built-in, or
vice-versa, the system config.scm needs manual updating.
* Switching system between two different arm boards potentially requires
entirely different lists of modules.
Rather than handling modules one at a time, I would propose to at least
add an option that can add whole directory trees of modules to the
initrd (e.g. kernel/drivers/usb/)... and then use modprobe (or udev?)
to handle the dependencies. Maybe opt-in at first, but longer-term,
explore making it default?
In Debian, adding modules to the intird is done in initramfs-tools:
https://salsa.debian.org/kernel-team/initramfs-tools/-/blob/f350f122a244c=
60f91a3e3e5f8b58f9cb75308b6/hook-functions#L599
As for which modules to load at runtime, initramfs-tools uses udev;
maybe that could be integrated into the guix initramfs as an option?
Obviously, adding more modules than a strict bare minimum to the initrd
will increase boot times a bit, and adding further dependencies
(modprobe, udev) to the initrd will add to that even more, but the
current hard-coded list of modules to load, that you can extend with
another hard-coded list, is a bit painful when trying to get a new
system working...
Maybe the Guix way to do this is to write some guile code that can
generate the list of modules to include in the initrd at build time? But
that still doesn't resolve the dynamic loading of modules at runtime,
and it would be impractical to load *all* the modules at runtime... and
maybe impractical to re-implement modprobe and/or udev in guile?
Well, thanks for considering!
live well,
vagrant
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYJRYCgAKCRDcUY/If5cW
qoGHAQDyNF3KxmMy43ZiU4oIK7WWhj/Qb8sAqkjqGPGSPi/IhwEAjPdH5SHFAA3p
uSnpUt8FgERfhAHFHii1VmuUzDbG3AM=
=FSHa
-----END PGP SIGNATURE-----
--=-=-=--
Vagrant Cascadian <vagrant@HIDDEN>:bug-guix@HIDDEN.
Full text available.bug-guix@HIDDEN:bug#48266; Package guix.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.