GNU bug report logs - #51787
GC takes more than 9 hours on berlin

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Severity: important; Reported by: Mathieu Othacehe <othacehe@HIDDEN>; dated Fri, 12 Nov 2021 11:50:02 UTC; Maintainer for guix is bug-guix@HIDDEN.
Severity set to 'important' from 'normal' Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 51787 <at> debbugs.gnu.org:


Received: (at 51787) by debbugs.gnu.org; 22 Nov 2021 09:17:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Nov 22 04:17:02 2021
Received: from localhost ([127.0.0.1]:46889 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mp5RY-0007JC-KH
	for submit <at> debbugs.gnu.org; Mon, 22 Nov 2021 04:17:02 -0500
Received: from mail-io1-f43.google.com ([209.85.166.43]:44659)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1mp5RV-0007Ih-5n
 for 51787 <at> debbugs.gnu.org; Mon, 22 Nov 2021 04:16:55 -0500
Received: by mail-io1-f43.google.com with SMTP id f9so22208395ioo.11
 for <51787 <at> debbugs.gnu.org>; Mon, 22 Nov 2021 01:16:53 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=NYCN6cozve7Z2bKbdcaLN05FFlPxkoWiVk0fgsNrHVg=;
 b=SgyXz5DYKuGoHuIeV86voH3w0b2IEs1Rmq8JMYhBJyTJZ00WAwBgmIuPVPvsfXsFs+
 k7K3RsJiFS+a64jXWDok6xnuJnvjf7nRZSMR/STZQUdOdeTuMhiQdqH2sStqzhC+Ryp1
 HhEcFarn609kJVjqKy4kImDDRnTJ59wMbC3KM0mP6YGpWmUgXrfMfT6u3JcavCf0k0TC
 /sxL2tkqzp7HD6XGHNS2Pq3tk9bNG+KcollRxTwHeBtYf0PeB+lMxe133KvxNFcdtZ8t
 t0tcXgYLo4xDxGkSzul4jeQpPvkD7+8Wgo5Mgvsd3lRGR/VFgYjUoY6/shFP/ls3gpo8
 VY1Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=NYCN6cozve7Z2bKbdcaLN05FFlPxkoWiVk0fgsNrHVg=;
 b=VqPWvxQ7zWWZ+Fm1xyQ4pdyS0z/IZBB1rNgVq8w77AL/TZ+ynl0mXtA0eIHlSwgSUU
 zJvYVo0sJ561EsP44B8vDd2pnRDBr+a6sTzbCMkqKilgY0lzLRk0WMH5/e2WNm47zVD3
 o3lN2zJ4K9mh5j1VXuGNTyx7tGz1mwFYnUcwC0xuBwU2GsfElx/gPnPsu7WriGBtPu7E
 XBpmhHToXJahvz6Wo7eo2aXPn8DbcZzlQZDwBZywnSMqJx3RyXe3IAL+hj+1sFctPLvs
 Yd+wMLVAbGnsr2ubRnZK64SFXTBG/ak33I2jKxmXjrZ/SJToAC3h6cRtGl2pY37iAVyI
 3xJQ==
X-Gm-Message-State: AOAM531LeSLemOs9eIqyrdRHajjt7yUtpOFl/tb7yxxU9ssIv3fLAP95
 JwA14vgsOvAQ2Nv7isUaExs2CQUPHcJZrNECmCM=
X-Google-Smtp-Source: ABdhPJxXDPeoD8GKRtBplwk/UsVv2f608EnYiQud3/JtB60pknPiTY9ZfRYd4oZj29q6Uu5kQUQM7Wy0Wk34VSHCkfk=
X-Received: by 2002:a05:6638:2bb:: with SMTP id
 d27mr48187375jaq.66.1637572607699; 
 Mon, 22 Nov 2021 01:16:47 -0800 (PST)
MIME-Version: 1.0
References: <87o86pegr3.fsf@HIDDEN>
In-Reply-To: <87o86pegr3.fsf@HIDDEN>
From: zimoun <zimon.toutoune@HIDDEN>
Date: Mon, 22 Nov 2021 10:16:36 +0100
Message-ID: <CAJ3okZ3Y4WRm74jWEW9ax4QhRgimUKVxHiBr8d-CDN6UFh++fQ@HIDDEN>
Subject: Re: bug#51787: GC takes more than 9 hours on berlin
To: Mathieu Othacehe <othacehe@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 51787
Cc: 51787 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi,

On Fri, 12 Nov 2021 at 12:51, Mathieu Othacehe <othacehe@HIDDEN> wrote:

> On berlin, the daily GC command is still running whereas it was started
> 9 hours ago.
>
> --8<---------------cut here---------------start------------->8---
> guix processes
> [...]
> SessionPID: 37231
> ClientPID: 37195
> ClientCommand: /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/libexec/guix/guile \ /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/bin/guix gc -F10995116277760
> --8<---------------cut here---------------end--------------->8---
>
> and
>
> --8<---------------cut here---------------start------------->8---
> mathieu@berlin ~$ ps auxww|grep 37195
> root      37195  0.0  0.0 183260 33440 ?        Sl   03:59   0:00 /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/libexec/guix/guile \ /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/bin/guix gc -F10995116277760
> --8<---------------cut here---------------end--------------->8---
>
> That's really problematic as it is blocking some other berlin services
> such as Cuirass, which has 4564 packages in its fetch queue:
>
> --8<---------------cut here---------------start------------->8---
> mathieu@berlin ~$ less /var/log/cuirass-remote-server.log
> [...]
> 2021-11-12T12:47:01 period update: 0 resumable, 0 failed builds.
> 2021-11-12T12:47:01 period update: 4564 items in the fetch queue.
> --8<---------------cut here---------------end--------------->8---

How is it possible to investigate?

Cheers,
simon




Information forwarded to bug-guix@HIDDEN:
bug#51787; Package guix. Full text available.

Message received at 51787 <at> debbugs.gnu.org:


Received: (at 51787) by debbugs.gnu.org; 12 Nov 2021 19:17:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Nov 12 14:17:34 2021
Received: from localhost ([127.0.0.1]:45986 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlc3K-00039J-DZ
	for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 14:17:34 -0500
Received: from eggs.gnu.org ([209.51.188.92]:56042)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <othacehe@HIDDEN>) id 1mlc3I-000392-9X
 for 51787 <at> debbugs.gnu.org; Fri, 12 Nov 2021 14:17:33 -0500
Received: from [2001:470:142:3::e] (port=59220 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1mlc3D-0005sP-3N
 for 51787 <at> debbugs.gnu.org; Fri, 12 Nov 2021 14:17:27 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To:
 From; bh=d1CaP/54GIHT1n78g1ZV/QjEqV0vXxA9aC8GmXtO+b4=; b=YLf/HHNk3KIrNR3FbDyq
 WTvZ1esOm4qrt2kknNViSaZkzYbhhKWXKUOqZoDQlRd8cLJv16qa9Rz0wC1O2rkmsYwxb1f6OktvC
 3h/Gzz+bi/Y65lqsf1qFJbCfZm2z91X289H/CF804D8DvqT1a45gxy2wfCKipiwzK6pjsUvuVPiVC
 WVBMzDd26FtY4rlRaTbE4ds2WoUdhrh9wfVxuV/CpPz/mY8JEQ4HyLaywncquh27xjILduYZ+f2B7
 p6C192M0sTareD+azFnGlnhMOGnpRMYYjzuAitTC/7xCTbvhnuF2zRS1EbdePWkbOJNULlvuUsbbp
 zHd8rRwzBMl3gA==;
Received: from [2a01:e0a:19b:d9a0:b4a5:c7aa:fee0:cc15] (port=60382 helo=meije)
 by fencepost.gnu.org with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1)
 (envelope-from <othacehe@HIDDEN>) id 1mlc3C-0007DQ-Re
 for 51787 <at> debbugs.gnu.org; Fri, 12 Nov 2021 14:17:27 -0500
From: Mathieu Othacehe <othacehe@HIDDEN>
To: 51787 <at> debbugs.gnu.org
Subject: Re: bug#51787: GC takes more than 9 hours on berlin
References: <87o86pegr3.fsf@HIDDEN>
Date: Fri, 12 Nov 2021 19:17:24 +0000
In-Reply-To: <87o86pegr3.fsf@HIDDEN> (Mathieu Othacehe's message of "Fri, 12
 Nov 2021 11:49:04 +0000")
Message-ID: <87ee7lji9n.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 51787
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)


> mathieu@berlin ~$ ps auxww|grep 37195
> root      37195  0.0  0.0 183260 33440 ?        Sl   03:59   0:00 /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/libexec/guix/guile \ /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/bin/guix gc -F10995116277760

I just killed this process, it was running for more than 16 hours.

Mathieu




Information forwarded to bug-guix@HIDDEN:
bug#51787; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 12 Nov 2021 11:49:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Nov 12 06:49:14 2021
Received: from localhost ([127.0.0.1]:43602 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mlV3S-0003Zo-5F
	for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 06:49:14 -0500
Received: from lists.gnu.org ([209.51.188.17]:49518)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <othacehe@HIDDEN>) id 1mlV3Q-0003Zg-GF
 for submit <at> debbugs.gnu.org; Fri, 12 Nov 2021 06:49:13 -0500
Received: from eggs.gnu.org ([209.51.188.92]:45604)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1mlV3Q-0000Bq-8v
 for bug-guix@HIDDEN; Fri, 12 Nov 2021 06:49:12 -0500
Received: from [2001:470:142:3::e] (port=53196 helo=fencepost.gnu.org)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <othacehe@HIDDEN>) id 1mlV3K-0003Du-VQ
 for bug-guix@HIDDEN; Fri, 12 Nov 2021 06:49:11 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to:
 references; bh=L6+QnlrYn47J+INqvi/3e31WWkGtQ/lDSyWSm+AW93k=; b=iReggNJs7bcFKh
 BBq5onFnOS7UGiCwIZ3dXGmUZ/UBu71mjK0u0A1iqhenHjSPFTalHWaOXRFWjRGzeg+hm60XAefrG
 socrDU2E1HwYxGaaJxyvCqLC6rqbVbORB0m4Ex8F5Lxx/9Q2062cMHP1hNHz49nBE4mN6fkm32N9L
 ZhRELQnln6cv2atOdfHB5+GtwYPbNuscVN58B7lIep2zfL0SEnAjmzR6PLszEq6mO4DjSCHYOI3rP
 9gCgplWoVxFh/8/S9Y6AgwlyvHf392gx6RYWymzO4oFAbjjj9495Wv8PhT/FfuvFRqWA/Xo9WTBsb
 PTXlsn7Nq5oODAaH8KOQ==;
Received: from [2a01:e0a:19b:d9a0:b4a5:c7aa:fee0:cc15] (port=60362 helo=meije)
 by fencepost.gnu.org with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1)
 (envelope-from <othacehe@HIDDEN>) id 1mlV3K-0001zW-J2
 for bug-guix@HIDDEN; Fri, 12 Nov 2021 06:49:06 -0500
From: Mathieu Othacehe <othacehe@HIDDEN>
To: bug-guix@HIDDEN
Subject: GC takes more than 9 hours on berlin
Date: Fri, 12 Nov 2021 11:49:04 +0000
Message-ID: <87o86pegr3.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)


Hello,

On berlin, the daily GC command is still running whereas it was started
9 hours ago.

--8<---------------cut here---------------start------------->8---
guix processes
[...]
SessionPID: 37231
ClientPID: 37195
ClientCommand: /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/libexec/guix/guile \ /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/bin/guix gc -F10995116277760
--8<---------------cut here---------------end--------------->8---

and

--8<---------------cut here---------------start------------->8---
mathieu@berlin ~$ ps auxww|grep 37195
root      37195  0.0  0.0 183260 33440 ?        Sl   03:59   0:00 /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/libexec/guix/guile \ /gnu/store/49vfv8l1j96bbd73ssbqanpnjz83arss-guix-1.3.0-11.014f1b6/bin/guix gc -F10995116277760
--8<---------------cut here---------------end--------------->8---

That's really problematic as it is blocking some other berlin services
such as Cuirass, which has 4564 packages in its fetch queue:

--8<---------------cut here---------------start------------->8---
mathieu@berlin ~$ less /var/log/cuirass-remote-server.log
[...]
2021-11-12T12:47:01 period update: 0 resumable, 0 failed builds.
2021-11-12T12:47:01 period update: 4564 items in the fetch queue.
--8<---------------cut here---------------end--------------->8---

Thanks,

Mathieu




Acknowledgement sent to Mathieu Othacehe <othacehe@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#51787; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Tue, 23 Nov 2021 17:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.