GNU bug report logs - #35660
guix weather runaway memory consumption

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 35660 in the body.
You can then email your comments to 35660 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: <bug-guix <at> gnu.org>
Subject: guix weather runaway memory consumption
Date: Thu, 9 May 2019 22:40:44 +0200

[Message part 1 (text/plain, inline)]

With guix master a0dc97a517cbc4c82640e30cacb2a564d128bbe9 guix weather consumes
8 GB of memory on a machine with 8 GB of memory and 1 GB of swap, then is killed
by Linux OOM killer.

Reproducible every time.

Reduced problem to GUIX_PACKAGE_PATH being set.

If I unset it, it works.

Attached contents of $GUIX_PACKAGE_PATH directory.

[wip2.tar.gz (application/gzip, attachment)]

[Message part 3 (application/pgp-signature, inline)]

Message #8 received at 35660 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 35660 <at> debbugs.gnu.org
Subject: Re: bug#35660: guix weather runaway memory consumption
Date: Mon, 13 May 2019 22:46:22 +0200

Hi Danny,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> With guix master a0dc97a517cbc4c82640e30cacb2a564d128bbe9 guix weather consumes
> 8 GB of memory on a machine with 8 GB of memory and 1 GB of swap, then is killed
> by Linux OOM killer.
>
> Reproducible every time.
>
> Reduced problem to GUIX_PACKAGE_PATH being set.
>
> If I unset it, it works.

I don’t think this is the place to debug personal package collections.
:-)

I’d love to give a hand but I feel we’re already overwhelmed with bugs
related to things happening on ‘master’.

WDYT?

Thanks,
Ludo’.

Message #11 received at 35660 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 35660 <at> debbugs.gnu.org
Subject: Re: bug#35660: guix weather runaway memory consumption
Date: Mon, 13 May 2019 23:25:58 +0200

[Message part 1 (text/plain, inline)]

Hi Ludo,

On Mon, 13 May 2019 22:46:22 +0200
Ludovic Courtès <ludo <at> gnu.org> wrote:

> I don’t think this is the place to debug personal package collections.
> :-)

I didn't even remember the personal package collection at first.
This here is a bug in guix that happens to manifest with that personal package
collection.

The failure mode here is very very bad.  Guix will consume all available
memory and then start on the swap, at which point the computer will
become unresponsive to any input and the user can't save any open
documents and has to kill the power to the computer.

After the user did that and restarted the computer, it will happen all
over again on the next "guix weather" (100% of the time).

> I’d love to give a hand but I feel we’re already overwhelmed with bugs
> related to things happening on ‘master’.
> 
> WDYT?

Oh, it's not time-critical at all for me.

I reported this bug for documentation--I already know how to work
around it.

Though there is a bug somewhere--it's not going to magically vanish.

But a user is going to encounter the same problem eventually, then not
reduce it to his GUIX_PACKAGE_PATH and give up.  

Furthermore, if he wanted to find out where exactly the problem is, he'd have
to delete files in $GUIX_PACKAGE_PATH one by one and invoke guix weather
in-between, each time crashing his entire computer after like 10 min per try.
That's not a good user experience.

[Message part 2 (application/pgp-signature, inline)]

Message #14 received at 35660 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 35660 <at> debbugs.gnu.org
Subject: Re: bug#35660: guix weather runaway memory consumption
Date: Tue, 14 May 2019 22:52:31 +0200

Hi Danny,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> The failure mode here is very very bad.  Guix will consume all available
> memory and then start on the swap, at which point the computer will
> become unresponsive to any input and the user can't save any open
> documents and has to kill the power to the computer.

I agree that the failure mode is terrible.  It’s very likely a cycle in
the package graph, given the symptoms you describe.  So ‘guix build
OFFENDING-PACKAGE’ would probably give you the same result.

Chris Baines proposed a patch a while back to detect and report cycles,
but we never got around to polishing and integrating it.  That would
probably help a lot in these cases.

We can open a new bug for that (if there’s not already one), but I think
it should be framed in terms of cycle detection in the package graph and
error reporting.

Thanks,
Ludo’.

Message #17 received at 35660 <at> debbugs.gnu.org (full text, mbox):

From: swedebugia <swedebugia <at> riseup.net>
To: Ludovic Courtès <ludo <at> gnu.org>,
 Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 35660 <at> debbugs.gnu.org
Subject: Re: bug#35660: guix weather runaway memory consumption
Date: Wed, 15 May 2019 13:53:54 +0200

On 2019-05-14 22:52, Ludovic Courtès wrote:
> Hi Danny,
> 
> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
> 
>> The failure mode here is very very bad.  Guix will consume all available
>> memory and then start on the swap, at which point the computer will
>> become unresponsive to any input and the user can't save any open
>> documents and has to kill the power to the computer.
> 
> I agree that the failure mode is terrible.  It’s very likely a cycle in
> the package graph, given the symptoms you describe.  So ‘guix build
> OFFENDING-PACKAGE’ would probably give you the same result.
> 
> Chris Baines proposed a patch a while back to detect and report cycles,
> but we never got around to polishing and integrating it.  That would
> probably help a lot in these cases.
> 
> We can open a new bug for that (if there’s not already one), but I think
> it should be framed in terms of cycle detection in the package graph and
> error reporting.

+1

When working on the NPM importer earlier this was very very common 
happening with "guix build NPM-PACKAGE" because of cycles.

My strategy was to manually kill the build after a minute or two if no 
output was produced.

-- 
Cheers Swedebugia

Message #20 received at 35660 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: swedebugia <swedebugia <at> riseup.net>
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>,
 Ludovic Courtès <ludo <at> gnu.org>, 35660 <at> debbugs.gnu.org
Subject: Re: bug#35660: guix weather runaway memory consumption
Date: Tue, 17 Aug 2021 21:03:48 -0400

retitle 35660 Implement cycle detection with proper reporting
thanks

swedebugia <swedebugia <at> riseup.net> writes:

> On 2019-05-14 22:52, Ludovic Courtès wrote:
>> Hi Danny,
>> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
>> 
>>> The failure mode here is very very bad.  Guix will consume all available
>>> memory and then start on the swap, at which point the computer will
>>> become unresponsive to any input and the user can't save any open
>>> documents and has to kill the power to the computer.
>> I agree that the failure mode is terrible.  It’s very likely a cycle
>> in
>> the package graph, given the symptoms you describe.  So ‘guix build
>> OFFENDING-PACKAGE’ would probably give you the same result.
>> Chris Baines proposed a patch a while back to detect and report
>> cycles,
>> but we never got around to polishing and integrating it.  That would
>> probably help a lot in these cases.
>> We can open a new bug for that (if there’s not already one), but I
>> think
>> it should be framed in terms of cycle detection in the package graph and
>> error reporting.
>
> +1

This would indeed be very useful.  I'm retitling this bug so that it is
more actionable.  Could someone link to the patch Chris Baines had
written going in this direction?

Thanks,

Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.