GNU bug report logs - #50566
[core-updates-frozen] Grub fails to decrypt LUKS partition (cryptomount not found)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 50566 in the body.
You can then email your comments to 50566 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: bug-guix <at> gnu.org
Subject: [core-updates-frozen] Grub fails to decrypt LUKS partition
 (cryptomount not found)
Date: Mon, 13 Sep 2021 13:54:36 +0200

Hi folks,

today I finally got my system reconfiguring on core-updates-frozen with
almost all packages I use (minus telegram-desktop, josm and lilypond).
But when I rebooted my PC it got stuck in the grub rescue console :(

```
Welcome to GRUB!

Unkown command 'crpytomount'.
error: disk 'cryptouuid/ad06...' not found.
Entering rescue mode...
[...]
grub rescue> crpytomount -u ad06...
Unknown command 'cryptomount'.
[...]
grub rescue> set
cmdpath=(hd0)/EFI/Guix
prefix=(cryptouuid)/ad06...)/boot/grub
root=cryptouuid/ad06...
```

commit: d1732a007fff1549fd49bae595ef9541bea2a332 (pulled from)
arch: x86_66
config:
```
  (bootloader
    (bootloader-configuration
      (bootloader grub-efi-bootloader)
      (target "/boot/efi")
      (keyboard-layout keyboard-layout)))
  (mapped-devices
    (list (mapped-device
            (source (uuid "ad069897-6b43-4607-b5e9-3f00f9af74aa"))
            (target "cryptroot")
            (type luks-device-mapping))))
  (file-systems
    (append (list
             (file-system
              (mount-point "/boot/efi")
              (device (uuid "2954-F644" 'fat32))
              (type "vfat"))
             (file-system
              (device "/dev/mapper/cryptroot")
              (mount-point "/")
              (type "ext4")
              (dependencies mapped-devices))
             (file-system
              (mount-point "/media/jonathan/arbeit")
              (device (uuid "094e6082-fae5-4ba9-ad94-1a4b6d04a094"))
              (type "ext4")))
            %base-file-systems))
```

I have reconfigured the same system on master two or three days ago. The
reboot back then was successful.

I'll try to collect more information later today...

~Jonathan

Message #10 received at 50566 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: 50566 <at> debbugs.gnu.org
Subject: [core-updates-frozen] Grub fails to decrypt LUKS partition
 (cryptomount not found)
Date: Wed, 15 Sep 2021 00:57:54 +0200

I went ahead and did a trick Tobias (nckx) told me in IRC. I booted a
Guix installer from an USB stick and hit 'c' (command line) in its GRUB
menu. Then I gave in the following commands:

```
insmod luks
cryptomount (hd1,gpt2)
Enter passphrase ...:
set root=(crypto0)
configfile /boot/grub/grub.cfg
```

I can then choose between the Guix system generations: 1x
core-updates-frozen and 3x master. The latter did worked before my
reconfigure on c-u-f.

But non of them really worked. I have to enter the LUKS password a
second time and then shepherd fails to start all the services. The cpu
fan blows a bit loud:
```
failed to start service 'file-systems'
failed to start service 'file-system-/boot/efi'
failed to start service 'user-processes'
[...]
```

An Alt+SysReq+E (SIGTERM all apart INIT) gives:
```
shepherd[1]: Service udev has been disabled.
shepherd[1]: Respawning libvirtd.
shepherd[1]: Service libvirtd has been started.
```

And some audit log message. Can audit break the boot?

Has anyone an idea how to find out why those services start to fail? How
to obtain logs?

~Jonathan

Message #13 received at 50566 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: 50566 <at> debbugs.gnu.org
Subject: [core-updates-frozen] Grub fails to decrypt LUKS partition
 (cryptomount not found)
Date: Fri, 17 Sep 2021 17:08:16 +0200

Hi,

I found today an easier reproducer. I built an installation image from
current core-updates-frozen (716470cda447e8cc3e73b09a8dfdcf1032aa37f2).
Booting from an USB stick leads to the same error that it comes only to
"grub rescue>" saying "Unknown filesystem".

I double checked and booted from an USB stick with an image from current
master -> just fine.

It would be interesting if others try to boot from an c-u-f image,
especially on UEFI.

I'll try to bisect the issue as I have no an "easy" reproducer :)

Message #16 received at 50566 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
Cc: 50566 <at> debbugs.gnu.org
Subject: Re: bug#50566: [core-updates-frozen] Grub fails to decrypt LUKS
 partition (cryptomount not found)
Date: Sat, 18 Sep 2021 17:48:03 +0200

[Message part 1 (text/plain, inline)]

Hello Jonathan,

> I'll try to bisect the issue as I have no an "easy" reproducer :)

I experienced the exact same issue. To be able to boot again, I booted
from a Guix System USB drive, hitted 'c' in Grub, then typed the
following commands:

--8<---------------cut here---------------start------------->8---
set root=(hd0,gpt2)
configfile /boot/grub/grub.cfg
--8<---------------cut here---------------end--------------->8---

Now regarding the cause of the issue, it appears that running the
following command from a Guix "master" Grub fixes the issue:

--8<---------------cut here---------------start------------->8---
sudo /gnu/store/w8v5d1i6xfqlpj78w89jg1x7f8dchh4k-grub-efi-2.06/sbin/grub-install --boot-directory /boot --efi-directory=/boot/efi
--8<---------------cut here---------------end--------------->8---

So this means that we probably have an issue related to the Grub package
on core-updates-frozen itself. Both master and core-updates-frozen are
using Grub 2.06 release.

The issue might then be related to the GCC 10 switch?

Here are the log files of Grub installation of master and
core-updates-frozen branches. I edited the /gnu/store/xxx-grub-efi-2.06
strings to remove the noise.

It looks like the core-updates-frozen Grub has a new .note.grub.property
section but I cannot find any other noticeable differences.

To be continued,

Mathieu

[master.log (application/octet-stream, attachment)]

[core.log (application/octet-stream, attachment)]

Message #19 received at 50566 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
Cc: 50566 <at> debbugs.gnu.org
Subject: Re: bug#50566: [core-updates-frozen] Grub fails to decrypt LUKS
 partition (cryptomount not found)
Date: Sun, 19 Sep 2021 15:02:15 +0000

Turns out stripping Grub modules was causing this issue. I'm not sure
why we do not experience this issue on master.

Anyway, 71aa29911cf3f4e6db5f9bff9237308b5f93283d fixes it for me.

I also discovered another issue related to image creation that I'll
report separately.

Thanks,

Mathieu

Message #22 received at 50566 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50566 <at> debbugs.gnu.org
Subject: Re: bug#50566: [core-updates-frozen] Grub fails to decrypt LUKS
 partition (cryptomount not found)
Date: Sun, 19 Sep 2021 23:58:35 +0200

Am 19.09.21 um 17:02 schrieb Mathieu Othacehe:
>
> Turns out stripping Grub modules was causing this issue. I'm not sure
> why we do not experience this issue on master.
>
> Anyway, 71aa29911cf3f4e6db5f9bff9237308b5f93283d fixes it for me.

Good catch and thanks for the fix, which works for me.

> I also discovered another issue related to image creation that I'll
> report separately.

During boot I'll run into https://issues.guix.gnu.org/50566 like you.

So I think we can close this bug and work on the next one :)

Message #25 received at 50566 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50566 <at> debbugs.gnu.org, Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
Subject: Re: bug#50566: [core-updates-frozen] Grub fails to decrypt LUKS
 partition (cryptomount not found)
Date: Mon, 20 Sep 2021 00:47:44 +0200

Hi!

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

> Turns out stripping Grub modules was causing this issue. I'm not sure
> why we do not experience this issue on master.

Woow, good catch.

> Anyway, 71aa29911cf3f4e6db5f9bff9237308b5f93283d fixes it for me.

‘master’ uses ‘--strip-debug’ whereas ‘core-updates-frozen’ uses
‘--strip-unneeded’ (from commit
e0f31baacc6ad30096a332b69433c85f5830bb2c).

Instead of #:strip-binaries? #f, we could try:

  #:strip-flags '("--strip-debug" "--enable-deterministic-archives")

Thanks,
Ludo’.

Message #28 received at 50566 <at> debbugs.gnu.org (full text, mbox):

From: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
To: Ludovic Courtès <ludo <at> gnu.org>,
 Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50566 <at> debbugs.gnu.org
Subject: Re: bug#50566: [core-updates-frozen] Grub fails to decrypt LUKS
 partition (cryptomount not found)
Date: Mon, 20 Sep 2021 10:04:34 +0200

Am 20.09.21 um 00:47 schrieb Ludovic Courtès:
> ‘master’ uses ‘--strip-debug’ whereas ‘core-updates-frozen’ uses
> ‘--strip-unneeded’ (from commit
> e0f31baacc6ad30096a332b69433c85f5830bb2c).
>
> Instead of #:strip-binaries? #f, we could try:
>
>    #:strip-flags '("--strip-debug" "--enable-deterministic-archives")

That one works as well. I tested it :)

Message #31 received at 50566 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Jonathan Brielmaier <jonathan.brielmaier <at> web.de>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 50566 <at> debbugs.gnu.org
Subject: Re: bug#50566: [core-updates-frozen] Grub fails to decrypt LUKS
 partition (cryptomount not found)
Date: Mon, 20 Sep 2021 09:19:09 +0000

Hey,

>>    #:strip-flags '("--strip-debug" "--enable-deterministic-archives")

Pushed as 650e85d85514c5fae06adf97ae615643a41bbbd8 :).

> That one works as well. I tested it :)

Thanks for the suggestion Ludo and for the testing Jonathan.

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.