GNU bug report logs - #61011
‘guix system vm’ spawns QEMU and hangs

Previous Next

Package: guix;

Reported by: Ludovic Courtès <ludo <at> gnu.org>

Date: Sun, 22 Jan 2023 21:37:02 UTC

Severity: important

Done: Ludovic Courtès <ludo <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 61011 in the body.
You can then email your comments to 61011 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to efraim <at> flashner.co.il, marius <at> gnu.org, bug-guix <at> gnu.org:
bug#61011; Package guix. (Sun, 22 Jan 2023 21:37:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Ludovic Courtès <ludo <at> gnu.org>:
New bug report received and forwarded. Copy sent to efraim <at> flashner.co.il, marius <at> gnu.org, bug-guix <at> gnu.org. (Sun, 22 Jan 2023 21:37:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: bug-guix <at> gnu.org
Subject: ‘guix system vm’ spawns
 QEMU and hangs
Date: Sun, 22 Jan 2023 22:36:21 +0100
Hello,

On my Guix System machine, the ‘qemu-system-x86_64’ spawned by ‘guix
system vm’ hangs after printing “Booting from ROM...”; it has to be
terminated with SIGKILL, SIGINT is not enough.

Specifically:

  $(guix time-machine --commit=66188398c446bdf9ce044fa539536e9b54c28c60 \
    -- system vm gnu/system/examples/bare-bones.tmpl) -m 1024  # Good.

… whereas:

  $(guix time-machine --commit=9923100a42ffa80f604c1c13a5e999e6a4c15146 \
    -- system vm gnu/system/examples/bare-bones.tmpl) -m 1024  # Bad!

I thought the culprit might be this commit:

  commit 9923100a42ffa80f604c1c13a5e999e6a4c15146
  Date:   Fri Dec 23 09:42:27 2022 +0200

      gnu: sgabios: Fix build on cross-build architectures.

      * gnu/packages/firmware.scm (sgabios)[arguments]: When cross-building
      add a make-flag to use the correct objcopy.

… but even after reverting it on today’s master, QEMU occasionally hangs
as before, though not always.

‘qemu-minimal’ as used for “make check-system” seems to work fine.

There have been a number of packages unbundled, so I wonder if another
one of these might be causing problems.

What do you think?

Ludo’.




Severity set to 'important' from 'normal' Request was from Ludovic Courtès <ludo <at> gnu.org> to control <at> debbugs.gnu.org. (Sun, 22 Jan 2023 21:58:01 GMT) Full text and rfc822 format available.

Information forwarded to bug-guix <at> gnu.org:
bug#61011; Package guix. (Mon, 23 Jan 2023 04:20:02 GMT) Full text and rfc822 format available.

Message #10 received at 61011 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 61011 <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il>,
 Marius Bakke <marius <at> gnu.org>
Subject: Re: bug#61011: ‘guix system vm’
 spawns QEMU and hangs
Date: Sun, 22 Jan 2023 23:19:47 -0500
On Sun, Jan 22, 2023 at 10:36:21PM +0100, Ludovic Courtès wrote:
>   $(guix time-machine --commit=9923100a42ffa80f604c1c13a5e999e6a4c15146 \
>     -- system vm gnu/system/examples/bare-bones.tmpl) -m 1024  # Bad!

I can reproduce.

It seems to work fine without '-m 1024', in which case it only has 512
MB RAM.




Information forwarded to bug-guix <at> gnu.org:
bug#61011; Package guix. (Mon, 23 Jan 2023 21:00:02 GMT) Full text and rfc822 format available.

Message #13 received at 61011 <at> debbugs.gnu.org (full text, mbox):

From: Josselin Poiret <dev <at> jpoiret.xyz>
To: Leo Famulari <leo <at> famulari.name>, Ludovic Courtès
 <ludo <at> gnu.org>
Cc: 61011 <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il>,
 Marius Bakke <marius <at> gnu.org>
Subject: Re: bug#61011: ‘guix system vm’ spawns QEMU and hangs
Date: Mon, 23 Jan 2023 21:59:35 +0100
Hi,
Leo Famulari <leo <at> famulari.name> writes:

> On Sun, Jan 22, 2023 at 10:36:21PM +0100, Ludovic Courtès wrote:
>>   $(guix time-machine --commit=9923100a42ffa80f604c1c13a5e999e6a4c15146 \
>>     -- system vm gnu/system/examples/bare-bones.tmpl) -m 1024  # Bad!
>
> I can reproduce.
>
> It seems to work fine without '-m 1024', in which case it only has 512
> MB RAM.

This is probably due to the following kernel bug [1], which could be
related to the khugepaged hangs I'm getting on my system since 6.1.

[1]
https://lore.kernel.org/kvm/b8017e09-f336-3035-8344-c549086c2340 <at> kernel.org/ 


Best,
-- 
Josselin Poiret




Information forwarded to bug-guix <at> gnu.org:
bug#61011; Package guix. (Mon, 23 Jan 2023 22:22:01 GMT) Full text and rfc822 format available.

Message #16 received at 61011 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Josselin Poiret <dev <at> jpoiret.xyz>
Cc: 61011 <at> debbugs.gnu.org, Marius Bakke <marius <at> gnu.org>,
 Efraim Flashner <efraim <at> flashner.co.il>, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#61011: ‘guix system vm’ spawns QEMU and hangs
Date: Mon, 23 Jan 2023 23:21:39 +0100
Hello,

Josselin Poiret <dev <at> jpoiret.xyz> skribis:

> Leo Famulari <leo <at> famulari.name> writes:
>
>> On Sun, Jan 22, 2023 at 10:36:21PM +0100, Ludovic Courtès wrote:
>>>   $(guix time-machine --commit=9923100a42ffa80f604c1c13a5e999e6a4c15146 \
>>>     -- system vm gnu/system/examples/bare-bones.tmpl) -m 1024  # Bad!
>>
>> I can reproduce.
>>
>> It seems to work fine without '-m 1024', in which case it only has 512
>> MB RAM.
>
> This is probably due to the following kernel bug [1], which could be
> related to the khugepaged hangs I'm getting on my system since 6.1.
>
> [1]
> https://lore.kernel.org/kvm/b8017e09-f336-3035-8344-c549086c2340 <at> kernel.org/ 

Ouch.  I’m running 6.1 since January 16th, which is about the time I
first experienced the issue.

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#61011; Package guix. (Tue, 24 Jan 2023 10:25:02 GMT) Full text and rfc822 format available.

Message #19 received at 61011 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 61011 <at> debbugs.gnu.org, Marius Bakke <marius <at> gnu.org>
Subject: Re: bug#61011: ‘guix system vm’
 spawns QEMU and hangs
Date: Tue, 24 Jan 2023 12:24:33 +0200
[Message part 1 (text/plain, inline)]
On Sun, Jan 22, 2023 at 10:36:21PM +0100, Ludovic Courtès wrote:
> Hello,
> 
> On my Guix System machine, the ‘qemu-system-x86_64’ spawned by ‘guix
> system vm’ hangs after printing “Booting from ROM...”; it has to be
> terminated with SIGKILL, SIGINT is not enough.
> 
> Specifically:
> 
>   $(guix time-machine --commit=66188398c446bdf9ce044fa539536e9b54c28c60 \
>     -- system vm gnu/system/examples/bare-bones.tmpl) -m 1024  # Good.
> 
> … whereas:
> 
>   $(guix time-machine --commit=9923100a42ffa80f604c1c13a5e999e6a4c15146 \
>     -- system vm gnu/system/examples/bare-bones.tmpl) -m 1024  # Bad!
> 
> I thought the culprit might be this commit:
> 
>   commit 9923100a42ffa80f604c1c13a5e999e6a4c15146
>   Date:   Fri Dec 23 09:42:27 2022 +0200
> 
>       gnu: sgabios: Fix build on cross-build architectures.
> 
>       * gnu/packages/firmware.scm (sgabios)[arguments]: When cross-building
>       add a make-flag to use the correct objcopy.
> 
> … but even after reverting it on today’s master, QEMU occasionally hangs
> as before, though not always.
> 
> ‘qemu-minimal’ as used for “make check-system” seems to work fine.
> 
> There have been a number of packages unbundled, so I wonder if another
> one of these might be causing problems.
> 
> What do you think?

I remember feeling overwhelmed by the build failures after the
unbundling (but I didn't reach out! I should've said something.) and
worked to try and quickly fix the builds.

I looked at reverting it locally, but with or without that patch I got
the same derivation for sgabios when built on x86_64. I tried firing up
diffoscope and I found no differences between the sgabios built on
x86_64, aarch64 or armhf (wow!).

I've run diffoscope against the sga I tried firing up diffoscope and I
found no differences between the sgabios built on x86_64, aarch64 or
armhf (wow!).

I've run diffoscope against the sgabios.bin that we build and the one
that comes in the qemu release tarball and I've included the output in
the email.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[sgabios.bin.diff.gz (application/x-gunzip, attachment)]
[signature.asc (application/pgp-signature, inline)]

bug closed, send any further explanations to 61011 <at> debbugs.gnu.org and Ludovic Courtès <ludo <at> gnu.org> Request was from Ludovic Courtès <ludo <at> gnu.org> to control <at> debbugs.gnu.org. (Thu, 15 Jun 2023 20:55:02 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 14 Jul 2023 11:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 259 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.