GNU bug report logs - #65419
[Shepherd] Non-reponding service control fiber

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Severity: important; Reported by: Ludovic Courtès <ludovic.courtes@HIDDEN>; merged with #65178; dated Mon, 21 Aug 2023 09:39:02 UTC; Maintainer for guix is bug-guix@HIDDEN.
Severity set to 'important' from 'normal' Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> debbugs.gnu.org. Full text available.
Merged 65178 65419. Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 65419 <at> debbugs.gnu.org:


Received: (at 65419) by debbugs.gnu.org; 24 Aug 2023 08:09:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Aug 24 04:09:17 2023
Received: from localhost ([127.0.0.1]:36075 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qZ5P3-0002hR-0T
	for submit <at> debbugs.gnu.org; Thu, 24 Aug 2023 04:09:17 -0400
Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]:43645)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludovic.courtes@HIDDEN>) id 1qZ5P0-0002gx-2L
 for 65419 <at> debbugs.gnu.org; Thu, 24 Aug 2023 04:09:16 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inria.fr; s=dc;
 h=from:to:cc:subject:references:date:in-reply-to:
 message-id:mime-version:content-transfer-encoding;
 bh=Or23m5I+7NdpzBi/tbhjDeMyI4hdOP/asj/K4IlPfs8=;
 b=ZTFHu12snE7HFNub4NVx3STOpHJKTiTdINmfwq5ELcH70sDAn4Nz9sGq
 Lh9GvD5Kp4EqS7T3LBqau0Dou9mGUneauOD3m+zwmLvP7CNjfh0O7DccF
 IpMKSa1CVD0unOo8TP93otLVJIyUbnptqTwJrvjqgqaldSF3XWrgHgIJb M=;
Authentication-Results: mail2-relais-roc.national.inria.fr;
 dkim=none (message not signed) header.i=none;
 spf=SoftFail smtp.mailfrom=ludovic.courtes@HIDDEN;
 dmarc=fail (p=none dis=none) d=inria.fr
X-IronPort-AV: E=Sophos;i="6.01,195,1684792800"; d="scan'208";a="122277804"
Received: from unknown (HELO ribbon) ([193.50.110.195])
 by mail2-relais-roc.national.inria.fr with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Aug 2023 10:09:03 +0200
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludovic.courtes@HIDDEN>
To: Giovanni Biscuolo <g@HIDDEN>
Subject: Re: bug#65419: [Shepherd] Non-reponding service control fiber
References: <87il98burf.fsf@HIDDEN> <87pm3ejii8.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Septidi 7 Fructidor an 231 de la =?utf-8?Q?R=C3=A9vo?=
 =?utf-8?Q?lution=2C?= jour du Sucrion
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Thu, 24 Aug 2023 10:09:02 +0200
In-Reply-To: <87pm3ejii8.fsf@HIDDEN> (Giovanni Biscuolo's message of "Wed, 
 23 Aug 2023 10:00:15 +0200")
Message-ID: <87jztk276p.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 65419
Cc: 65419 <at> debbugs.gnu.org, Christopher Baines <mail@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi,

Giovanni Biscuolo <g@HIDDEN> skribis:

> I've had a look at /var/log/messages but nothing seems wrong except
> messages like this one:
>
>
> Aug 21 14:48:42 localhost shepherd[1]: 6 connections still in use after s=
shd-13752 termination.=20
> Aug 21 14:48:42 localhost shepherd[1]: Service sshd-13752 (PID 29977) exi=
ted with 255.=20
> Aug 21 14:48:42 localhost shepherd[1]: Service sshd-13752 has been disabl=
ed.=20
> Aug 21 14:48:42 localhost shepherd[1]: Transient service sshd-13752 termi=
nated, now unregistered.=20

Yeah, I think it happened earlier but unfortunately the previously logs
got deleted (rottlog is not behaving as expected).

> Is it useful configuring the monitoring service [1] on milano-guix-1 to
> have useful data in the logs in case we get a similar issue?

It wouldn=E2=80=99t help in this case, but it=E2=80=99s still interesting t=
o have it
around.

  sudo herd eval root '(begin (use-modules (shepherd service monitoring)) (=
register-services (list (monitoring-service))))'
  sudo herd start monitoring

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#65419; Package guix. Full text available.

Message received at 65419 <at> debbugs.gnu.org:


Received: (at 65419) by debbugs.gnu.org; 23 Aug 2023 08:00:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Aug 23 04:00:39 2023
Received: from localhost ([127.0.0.1]:60974 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qYin9-0000NI-0s
	for submit <at> debbugs.gnu.org; Wed, 23 Aug 2023 04:00:39 -0400
Received: from ns13.heimat.it ([46.4.214.66]:60424)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <g@HIDDEN>) id 1qYin6-0000N3-6o
 for 65419 <at> debbugs.gnu.org; Wed, 23 Aug 2023 04:00:37 -0400
Received: from localhost (ip6-localhost [127.0.0.1])
 by ns13.heimat.it (Postfix) with ESMTP id 9543430087D;
 Wed, 23 Aug 2023 08:00:26 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at ns13.heimat.it
Received: from ns13.heimat.it ([127.0.0.1])
 by localhost (ns13.heimat.it [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id 3x5vXba3ze8P; Wed, 23 Aug 2023 08:00:24 +0000 (UTC)
Received: from bourrache.mug.xelera.it (unknown [93.56.171.217])
 (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by ns13.heimat.it (Postfix) with ESMTPSA id CE53730085C;
 Wed, 23 Aug 2023 08:00:24 +0000 (UTC)
Received: from roquette.mug.biscuolo.net (roquette [10.38.2.14])
 by bourrache.mug.xelera.it (Postfix) with SMTP id 61C632911B85;
 Wed, 23 Aug 2023 10:00:24 +0200 (CEST)
Received: (nullmailer pid 9310 invoked by uid 1000);
 Wed, 23 Aug 2023 08:00:23 -0000
From: Giovanni Biscuolo <g@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludovic.courtes@HIDDEN>,
 65419 <at> debbugs.gnu.org
Subject: Re: bug#65419: [Shepherd] Non-reponding service control fiber
In-Reply-To: <87il98burf.fsf@HIDDEN>
Organization: Xelera.eu
References: <87il98burf.fsf@HIDDEN>
Date: Wed, 23 Aug 2023 10:00:15 +0200
Message-ID: <87pm3ejii8.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 65419
Cc: Christopher Baines <mail@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hello,

Ludovic Court=C3=A8s <ludovic.courtes@HIDDEN> writes:

[...]

> The conclusion seems to be that the control fiber of the =E2=80=98root=E2=
=80=99 service
> is not responding: it is blocked on a get/put? did it exit?
>
> Unfortunately we don=E2=80=99t have data from the logs that would give cl=
ues as
> to what went wrong.

I've had a look at /var/log/messages but nothing seems wrong except
messages like this one:

=2D-8<---------------cut here---------------start------------->8---

Aug 21 14:48:42 localhost shepherd[1]: 6 connections still in use after ssh=
d-13752 termination.=20
Aug 21 14:48:42 localhost shepherd[1]: Service sshd-13752 (PID 29977) exite=
d with 255.=20
Aug 21 14:48:42 localhost shepherd[1]: Service sshd-13752 has been disabled=
.=20
Aug 21 14:48:42 localhost shepherd[1]: Transient service sshd-13752 termina=
ted, now unregistered.=20

=2D-8<---------------cut here---------------end--------------->8---

Is it useful configuring the monitoring service [1] on milano-guix-1 to
have useful data in the logs in case we get a similar issue?

Thanks, Gio'


[1] https://www.gnu.org/software/shepherd/manual/shepherd.html#Monitoring-S=
ervice

=2D-=20
Giovanni Biscuolo

Xelera IT Infrastructures

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJABAEBCgAqFiEERcxjuFJYydVfNLI5030Op87MORIFAmTlvI8MHGdAeGVsZXJh
LmV1AAoJENN9DqfOzDkSkpoP/RYvViPSKbaIclDijHJkUfl7a9FwtR9Wujj2uhaA
HfOSJpP0GOaCdIh2CCM5BzyWQXg4Vg/OWBtyCFs9XCmG7QZe7oevZU8nvZV+/w7/
uT0aHM0OOeJ/oc4DIumXSpozMqE46S5ew4A2Z/3ZQZ/0LxwG0sRKySnwW7aUwlrX
aHF/2op+33oDxnrIuxPBxZWPiyoXGqmxssfDydElmptqeS2lbbQ24igrT8b9Z57D
wkdfbIM148C/rGTNgmdaO7Xtu7gNeTVhZl70jVOBHmDZYMN4AiXFhG4xBq8ix+fZ
6ZvblosqeHDRmbPVrYzTm2ztLrgejoxBoyeoAIvuy1E4/KxnrqRIM9crp4zr491f
ttkupIh0RQgsfXc+adyKyLF0DeVDrJ+Vh1oH0RAjwhms+lX/KhGuED/ds8d3gExE
Ne8ADcXQXQeRmzLTaPRNSp5do5LioWs65n4UDguyck5Wdj6s8hPdBwO10+IudkSv
w+7k/pZjoC+7mPPsn7AzSdP6sZ1+lylzT0TIknDgtDKUxchRnAkhPoJLpc2tLzRK
P0eKmSJ42wfSBXyNOHRjSLjmTgmXfAhC4+1Pqn8XtQulTJ2TaFRKd4+S6jBbXlZp
lJaRMEm1ZJ3eXXQYp1Ca9N//S54JSEDNbnY7yPtSxfi0OYV7PEXuYT5B9ZtYiegd
zXBe
=gm+N
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to bug-guix@HIDDEN:
bug#65419; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 21 Aug 2023 09:38:40 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Aug 21 05:38:40 2023
Received: from localhost ([127.0.0.1]:55609 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qY1Mt-0007SQ-MU
	for submit <at> debbugs.gnu.org; Mon, 21 Aug 2023 05:38:39 -0400
Received: from lists.gnu.org ([2001:470:142::17]:52336)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludovic.courtes@HIDDEN>) id 1qY1Mr-0007SE-JK
 for submit <at> debbugs.gnu.org; Mon, 21 Aug 2023 05:38:38 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludovic.courtes@HIDDEN>)
 id 1qY1Mi-0008PF-RM
 for bug-guix@HIDDEN; Mon, 21 Aug 2023 05:38:29 -0400
Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludovic.courtes@HIDDEN>)
 id 1qY1Mg-0006Rn-Ac
 for bug-guix@HIDDEN; Mon, 21 Aug 2023 05:38:28 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inria.fr; s=dc;
 h=from:to:subject:date:message-id:mime-version:
 content-transfer-encoding;
 bh=090wlqVYbCgmCcWOP1LNAaOEDmTFSIL82T1iWi3bFco=;
 b=ALcVStg6R8DM4FNJGuiC423xnvm8yEDp6ImDp36mnBC+4+Kk6PKF8yqS
 oN8SUkZ+GqywVZLT+A74i8WW7uZe5WbWOePutfsU4cKfsLbOEk7suZa3I
 wNfzKiNHcbj7jZ1Ai2VeeJZ4/tEgmnc10I0xNtJjQQFgrxq0HwFStymBB c=;
Authentication-Results: mail3-relais-sop.national.inria.fr;
 dkim=none (message not signed) header.i=none;
 spf=SoftFail smtp.mailfrom=ludovic.courtes@HIDDEN;
 dmarc=fail (p=none dis=none) d=inria.fr
X-IronPort-AV: E=Sophos;i="6.01,189,1684792800"; d="scan'208";a="63847005"
Received: from unknown (HELO ribbon) ([193.50.110.195])
 by mail3-relais-sop.national.inria.fr with
 ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Aug 2023 11:38:13 +0200
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludovic.courtes@HIDDEN>
To: bug-guix@HIDDEN
Subject: [Shepherd] Non-reponding service control fiber
X-Debbugs-Cc: Christopher Baines <mail@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Quartidi 4 Fructidor an 231 de la =?utf-8?Q?R=C3=A9v?=
 =?utf-8?Q?olution=2C?= jour de l'Escourgeon
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 21 Aug 2023 11:38:12 +0200
Message-ID: <87il98burf.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=192.134.164.104;
 envelope-from=ludovic.courtes@HIDDEN;
 helo=mail3-relais-sop.national.inria.fr
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

Hello,

On milano-guix-1 (a build machine behind bayfront, running shepherd
0.10.2), =E2=80=98herd status=E2=80=99 and =E2=80=98herd status guix-build-=
coordinator-agent=E2=80=99
would hang (there=E2=80=99s no =E2=80=98guix-build-coordinator=E2=80=99 pro=
cess running).

=E2=80=98herd stop childhurd2=E2=80=99 hangs and has no effect.

Conversely, =E2=80=98herd status nscd=E2=80=99 and similar for most other s=
ervices works
fine.  When terminating a service=E2=80=99s process, the service gets respa=
wned
just fine.

The conclusion seems to be that the control fiber of the =E2=80=98root=E2=
=80=99 service
is not responding: it is blocked on a get/put? did it exit?

Unfortunately we don=E2=80=99t have data from the logs that would give clue=
s as
to what went wrong.

Ludo=E2=80=99.




Acknowledgement sent to Ludovic Courtès <ludovic.courtes@HIDDEN>:
New bug report received and forwarded. Copy sent to mail@HIDDEN, bug-guix@HIDDEN. Full text available.
Report forwarded to mail@HIDDEN, bug-guix@HIDDEN:
bug#65419; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 3 Sep 2023 20:15:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.