GNU bug report logs - #70897
Guix system hangs on boot with LUKS root partition

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 70897 in the body.
You can then email your comments to 70897 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Kaelyn <kaelyn.alexi <at> protonmail.com>
To: "bug-guix <at> gnu.org" <bug-guix <at> gnu.org>
Subject: Guix system hangs on boot with LUKS root partition
Date: Sun, 12 May 2024 15:26:23 +0000

Hi Ludo' and others,

I recently updated my systems after finally finding https://issues.guix.gnu.org/70051 and seeing the issue I was having with booting with a non-root LUKS partition configured had been fixed. After updating to a commit past these two:

  49f82fca41 mapped-devices: luks: Specify modules needed at the top-level.
  6062339156 mapped-devices: <mapped-device-type> can specify modules to import.

I am now seeing a different error, which I am pretty sure is related to the module import changes in 49f82fca41. The error I get is about an unknown symbol "system*/tty" when the initramfs tries to prompt for a password to unlock the LUKS partition containing the root filesystem. I don't know how the module plumbing of Shepherd and the generated initramfs work, but I suspect the fix for Shepherd opening LUKS partition broke the import of system*/tty in the initramfs (for example, at the early REPL that booting my latest system generation ends up at, system*/tty is undefined initially, but after evaluating "(use-modules (gnu build file-systems))" system*/tty resolves to a procedure as exected--so the module is at least present in the initramfs). I have encountered this error with two different systems, and I believe the reproduction is simply trying to open a LUKS device without a keyfile so that a password prompt is necessary.

Cheers,
Kaelyn

Message #8 received at 70897 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Kaelyn <kaelyn.alexi <at> protonmail.com>
Cc: 70897 <at> debbugs.gnu.org
Subject: Re: bug#70897: Guix system hangs on boot with LUKS root partition
Date: Mon, 13 May 2024 12:14:34 +0200

Hi Kaelyn,

Kaelyn <kaelyn.alexi <at> protonmail.com> skribis:

> I recently updated my systems after finally finding https://issues.guix.gnu.org/70051 and seeing the issue I was having with booting with a non-root LUKS partition configured had been fixed. After updating to a commit past these two:
>
>   49f82fca41 mapped-devices: luks: Specify modules needed at the top-level.
>   6062339156 mapped-devices: <mapped-device-type> can specify modules to import.
>
> I am now seeing a different error, which I am pretty sure is related
> to the module import changes in 49f82fca41. The error I get is about
> an unknown symbol "system*/tty" when the initramfs tries to prompt for
> a password to unlock the LUKS partition containing the root
> filesystem.

To be clear, you have both a LUKS-encrypted root and a non-root
LUKS-encrypted partition?

(FWIW I tested (1) with a LUKS-encrypted root, and (2) with a cleartext
root and LUKS-encrypted /home.  The bug you mention affected #2.)

Could you share your OS config or a relevant subset thereof?

> I don't know how the module plumbing of Shepherd and the generated
> initramfs work, but I suspect the fix for Shepherd opening LUKS
> partition broke the import of system*/tty in the initramfs (for
> example, at the early REPL that booting my latest system generation
> ends up at, system*/tty is undefined initially, but after evaluating
> "(use-modules (gnu build file-systems))" system*/tty resolves to a
> procedure as exected--so the module is at least present in the
> initramfs). I have encountered this error with two different systems,
> and I believe the reproduction is simply trying to open a LUKS device
> without a keyfile so that a password prompt is necessary.

Hmm.  Thanks for investigating!

Ludo’.

Message #11 received at 70897 <at> debbugs.gnu.org (full text, mbox):

From: Kaelyn <kaelyn.alexi <at> protonmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 70897 <at> debbugs.gnu.org
Subject: Re: bug#70897: Guix system hangs on boot with LUKS root partition
Date: Mon, 13 May 2024 19:44:41 +0000

Hi Ludo',

On Monday, May 13th, 2024 at 3:14 AM, Ludovic Courtès <ludo <at> gnu.org> wrote:

> 
> 
> Hi Kaelyn,
> 
> Kaelyn kaelyn.alexi <at> protonmail.com skribis:
> 
> > I recently updated my systems after finally finding https://issues.guix.gnu.org/70051 and seeing the issue I was having with booting with a non-root LUKS partition configured had been fixed. After updating to a commit past these two:
> > 
> > 49f82fca41 mapped-devices: luks: Specify modules needed at the top-level.
> > 6062339156 mapped-devices: <mapped-device-type> can specify modules to import.
> > 
> > I am now seeing a different error, which I am pretty sure is related
> > to the module import changes in 49f82fca41. The error I get is about
> > an unknown symbol "system*/tty" when the initramfs tries to prompt for
> > a password to unlock the LUKS partition containing the root
> > filesystem.
> 
> 
> To be clear, you have both a LUKS-encrypted root and a non-root
> LUKS-encrypted partition?
> 
> (FWIW I tested (1) with a LUKS-encrypted root, and (2) with a cleartext
> root and LUKS-encrypted /home. The bug you mention affected #2.)

More accurately, I have one system that has a mirrored btrfs root with two LUKS-encrypted partitions (and a few quirks in the setup that make rebooting a bit tedious, such as grub slowly unlocking two drives, and a ZFS pool that has to be unlocked manually after boot), and one with a single LUKS-encrypted btrfs partition. I hit (2) on the first system about a month ago when I updated both, with the second system booting fine. I hit (1) on the second system when updating much more recently after seeing (2) was fixed, and hadn't tried rebooting the first system with the new generation.

> Could you share your OS config or a relevant subset thereof?

My full OS config is decidedly non-trivial, with parts (e.g. common services and user accounts) shared between host configurations. The mapped-devices and file-systems fragments for the two systems are below.

For the first system, with the mirrored btrfs root:

  (mapped-devices
    (list (mapped-device
            (source
              (uuid "7bcca55e-8a41-44a8-beab-2047eed0af41"))
            (target "cryptroot1")
            (type luks-device-mapping))
          (mapped-device
            (source
              (uuid "9472b8ae-c90c-4712-b90d-ca07602514d7"))
            (target "cryptroot2")
            (type luks-device-mapping))
          ))
  (file-systems
    (let ((rootfs (file-system
             (mount-point "/")
             (device "/dev/mapper/cryptroot1")
             (type "btrfs")
             (check? #f)
             (options "compress=zstd,subvol=@guix")
             (dependencies mapped-devices))))
    (cons* rootfs
           (file-system
             (mount-point "/boot/efi")
             (device (file-system-label "EFI"))
             (type "vfat")
             (mount-may-fail? #t)
             (dependencies mapped-devices))
           (file-system
             (mount-point "/gnu")
             (device "/dev/mapper/cryptroot1")
             (type "btrfs")
             (check? #f)
             (options "compress=zstd,subvol=@gnu_store")
             (dependencies (cons rootfs mapped-devices)))
           %base-file-systems))))


The second system, with the single-drive encrypted btrfs root:

  (mapped-devices
    (list (mapped-device
            (source
              (uuid "e6aaafc5-49cb-477b-a665-daf065611195"))
            (target "cryptroot1")
            (type luks-device-mapping))
          ))
  (file-systems
    (let ((rootfs (file-system
             (mount-point "/")
             (device "/dev/mapper/cryptroot1")
             (type "btrfs")
             (check? #f)
             (options "compress=zstd,subvol=@guix")
             (dependencies mapped-devices))))
    (cons* rootfs
           (file-system
             (mount-point "/boot/efi")
             (device (file-system-label "EFI"))
             (type "vfat")
             (mount-may-fail? #t)
             (dependencies mapped-devices))
           (file-system
             (mount-point "/gnu")
             (device "/dev/mapper/cryptroot1")
             (type "btrfs")
             (check? #f)
             (options "compress=zstd,subvol=@gnu_store")
             (dependencies (cons rootfs mapped-devices)))
           %common-file-systems))))


(Note the %common-file-systems is simply %base-file-systems plus a couple of NFS mounts from the first system, which are shared with several computers.)

For both computers, I make use of the 6.1 or 6.6 LTS kernels since I also use ZFS. When I hit (1), I eventually figured out where the hang during boot was happening by removing "quiet" from the kernel command line, which also caused shepherd to be more verbose (something I hadn't realized). When I hit (2), the boot process was still in the initrd due to failing to unlock and mount the root filesystem.

If there is any further information I can provide, please let me know.

Cheers,
Kaelyn

> 
> > I don't know how the module plumbing of Shepherd and the generated
> > initramfs work, but I suspect the fix for Shepherd opening LUKS
> > partition broke the import of system*/tty in the initramfs (for
> > example, at the early REPL that booting my latest system generation
> > ends up at, system*/tty is undefined initially, but after evaluating
> > "(use-modules (gnu build file-systems))" system*/tty resolves to a
> > procedure as exected--so the module is at least present in the
> > initramfs). I have encountered this error with two different systems,
> > and I believe the reproduction is simply trying to open a LUKS device
> > without a keyfile so that a password prompt is necessary.
> 
> 
> Hmm. Thanks for investigating!
> 
> Ludo’.

Message #14 received at 70897 <at> debbugs.gnu.org (full text, mbox):

From: Kaelyn <kaelyn.alexi <at> protonmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 70897 <at> debbugs.gnu.org
Subject: Re: bug#70897: Guix system hangs on boot with LUKS root partition
Date: Sat, 18 May 2024 17:04:37 +0000

Hi Ludo',

I wanted to let you know that I have figured out what the failure is, and it turned out to be a customization buried within some shared operating-system configuration that I had forgotten about (and wasn't finding because it was due to a missing import, and simply grepping for "system*/tty" didn't match the file that was missing the import). Thank you for your time and help, and sorry about the red herring of a bug report!

Cheers,
Kaelyn

Message #19 received at 70897-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Kaelyn <kaelyn.alexi <at> protonmail.com>
Cc: 70897-done <at> debbugs.gnu.org
Subject: Re: bug#70897: Guix system hangs on boot with LUKS root partition
Date: Sun, 19 May 2024 16:05:23 +0200

Hi,

Kaelyn <kaelyn.alexi <at> protonmail.com> skribis:

> I wanted to let you know that I have figured out what the failure is,
> and it turned out to be a customization buried within some shared
> operating-system configuration that I had forgotten about (and wasn't
> finding because it was due to a missing import, and simply grepping
> for "system*/tty" didn't match the file that was missing the
> import). Thank you for your time and help, and sorry about the red
> herring of a bug report!

Great, glad you found the problem!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.