GNU bug report logs - #68800
Guix waits for termination of a kernel thread

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Severity: important; Reported by: Tomas Volf <~@wolfsden.cz>; dated Mon, 29 Jan 2024 17:35:02 UTC; Maintainer for guix is bug-guix@HIDDEN.
Severity set to 'important' from 'normal' Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 68800 <at> debbugs.gnu.org:


Received: (at 68800) by debbugs.gnu.org; 29 Jan 2024 17:44:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 29 12:44:14 2024
Received: from localhost ([127.0.0.1]:33376 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1rUVg5-0005eK-Tg
	for submit <at> debbugs.gnu.org; Mon, 29 Jan 2024 12:44:14 -0500
Received: from wolfsden.cz ([37.205.8.62]:58620)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <~@wolfsden.cz>) id 1rUVg3-0005eA-EZ
 for 68800 <at> debbugs.gnu.org; Mon, 29 Jan 2024 12:44:12 -0500
Received: by wolfsden.cz (Postfix, from userid 104)
 id 6E256282AC5; Mon, 29 Jan 2024 17:44:01 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail;
 t=1706550241; bh=Au24tLn8CRtJ7i3p4JuKQryZLhixJkobSlwHKOGT8I0=;
 h=Date:From:To:Subject:References:In-Reply-To;
 b=vxJrFED16o2kacaI0/Rkhhu8fViqSNV6Y0/Ni6nTGUWntum7KObEgMWB9BDMxN9FU
 nzwS/Ogy9GJNqxWIKX3IGulwKkdlYR2QNTPHS3YZgfJ71etGe+d7NU6hYKeN5cC7KV
 4jlyy19Soc7nynyPUuMSCwQ1OzwNpRUybT25oDCdIhkJpZraLIVVZSpW9yuMB2VMum
 OLNWz+djWDIOriuI6sAdjFz4Ajdy4MT2Kxcj0Qaovdg0e0Nq+9gJY6eX4TtrsWlKs1
 mnb878Yeq/mOHJw4kOxuysMXl+sFSUp0Hiqog7L6jg535jBDnIK0Hk4CDSHXIvioRO
 uPG3tRpQTUYIaRE6ru4566XMDShirKYzAnh+D1Owb+P3fNs5c42iCYEO9a+2xhULtk
 gRctIe13jIKVACayBURIc0ihlL71VJxbMu5r/zKC6lFdA1tTQu1aEcKIOAg523jWZz
 kSJt+XkpAN3kAhqHyYIMxU9pZXbe8MPcMAdQlptyYpR2hUFuXej/LGEaFLGLWj1rNf
 VN5tfN6DRAcu4toxvj1P2gzsqzcB1uEUw3d1kGCdyF2ZJxyJ2GunHIKpy/SOp+WlhT
 vqd3R+a9Eo4Ww2A2pSs0DrNKDs1/m8r52Yx/1cJVYEXIYrivPJlcJgEbZ9SeXBrSGw
 E0R9j7p3t1mZyC84N3D1geLI=
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on wolfsden
X-Spam-Level: 
X-Spam-Status: No, score=-1.2 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED,
 DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,T_SCC_BODY_TEXT_LINE,
 URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6
Received: from localhost (unknown [193.32.127.154])
 by wolfsden.cz (Postfix) with ESMTPSA id 09BE528257C
 for <68800 <at> debbugs.gnu.org>; Mon, 29 Jan 2024 17:44:00 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail;
 t=1706550241; bh=Au24tLn8CRtJ7i3p4JuKQryZLhixJkobSlwHKOGT8I0=;
 h=Date:From:To:Subject:References:In-Reply-To;
 b=vxJrFED16o2kacaI0/Rkhhu8fViqSNV6Y0/Ni6nTGUWntum7KObEgMWB9BDMxN9FU
 nzwS/Ogy9GJNqxWIKX3IGulwKkdlYR2QNTPHS3YZgfJ71etGe+d7NU6hYKeN5cC7KV
 4jlyy19Soc7nynyPUuMSCwQ1OzwNpRUybT25oDCdIhkJpZraLIVVZSpW9yuMB2VMum
 OLNWz+djWDIOriuI6sAdjFz4Ajdy4MT2Kxcj0Qaovdg0e0Nq+9gJY6eX4TtrsWlKs1
 mnb878Yeq/mOHJw4kOxuysMXl+sFSUp0Hiqog7L6jg535jBDnIK0Hk4CDSHXIvioRO
 uPG3tRpQTUYIaRE6ru4566XMDShirKYzAnh+D1Owb+P3fNs5c42iCYEO9a+2xhULtk
 gRctIe13jIKVACayBURIc0ihlL71VJxbMu5r/zKC6lFdA1tTQu1aEcKIOAg523jWZz
 kSJt+XkpAN3kAhqHyYIMxU9pZXbe8MPcMAdQlptyYpR2hUFuXej/LGEaFLGLWj1rNf
 VN5tfN6DRAcu4toxvj1P2gzsqzcB1uEUw3d1kGCdyF2ZJxyJ2GunHIKpy/SOp+WlhT
 vqd3R+a9Eo4Ww2A2pSs0DrNKDs1/m8r52Yx/1cJVYEXIYrivPJlcJgEbZ9SeXBrSGw
 E0R9j7p3t1mZyC84N3D1geLI=
Date: Mon, 29 Jan 2024 18:44:00 +0100
From: Tomas Volf <~@wolfsden.cz>
To: 68800 <at> debbugs.gnu.org
Subject: Re: Guix waits for termination of a kernel thread
Message-ID: <Zbfj4EjZHIqIcUyJ@ws>
References: <ZbfhqeMUdS63NB93@ws>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature"; boundary="SJ55xua6WttYwtt4"
Content-Disposition: inline
In-Reply-To: <ZbfhqeMUdS63NB93@ws>
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 68800
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


--SJ55xua6WttYwtt4
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

> Date: Mon, 29 Jan 2024 17:31:33 +0100
> From: Ludovic Court=E8s <ludo@HIDDEN>
> To: guix-devel@HIDDEN
> Subject: Re: GNU Shepherd 0.10.3 released
>
> Hi,
>
> Tomas Volf <~@wolfsden.cz> skribis:
>
> > Ah, that code indeed returns #f for the pid in question:
> >
> >     scheme@(guix-user)> ((@@ (guix build syscalls) kernel?) 688)
> >     $1 =3D #f
> >
> > The stat file:
> >
> >     $ cat /proc/688/stat
> >     688 (mt76-tx phy0) S 2 0 0 0 -1 2129984 0 0 0 0 0 0 0 0 -2 0 1 0 96=
4 0 0 18446744073709551615 0 0 0 0 0 0 0 2147483647 0 0 0 0 17 5 1 1 0 0 0 =
0 0 0 0 0 0 0 0
> >
> > So the start_code is not zero (I would guess it is -1).  I have no idea=
 what
> > that means though.
>
> What about this method (from shepherd)?
>
> --8<---------------cut here---------------start------------->8---
> (define (linux-process-flags pid)
>   "Return the process flags of @var{pid} (or'd @code{PF_} constants), ass=
uming
> the Linux /proc file system is mounted; raise a @code{system-error} excep=
tion
> otherwise."
>   (call-with-input-file (string-append "/proc/" (number->string pid)
>                                        "/stat")
>     (lambda (port)
>       (define line
>         (get-string-all port))
>
>       ;; Parse like systemd's 'is_kernel_thread' function.
>       (let ((offset (string-index line #\))))     ;offset past 'tcomm' fi=
eld
>         (match (and offset
>                     (string-tokenize (string-drop line (+ offset 1))))
>           ((state ppid pgrp sid tty-nr tty-pgrp flags . _)
>            (or (string->number flags) 0))
>           (_
>            0))))))
>
> ;; Per-process flag defined in <linux/sched.h>.
> (define PF_KTHREAD #x00200000)                    ;I am a kernel thread
>
> (define (linux-kernel-thread? pid)
>   "Return true if @var{pid} is a Linux kernel thread."
>   (=3D PF_KTHREAD (logand (linux-process-flags pid) PF_KTHREAD)))
> --8<---------------cut here---------------end--------------->8---
>
> If it works better, we can use that in syscalls.scm as well.

Yes, that does seem to work:

    scheme@(guile-user)> ,use (ice-9 match)
    scheme@(guile-user)> ,use (ice-9 textual-ports)
    scheme@(guile-user)> (define (linux-process-flags pid)
      "Return the process flags of @var{pid} (or'd @code{PF_} constants), a=
ssuming
    the Linux /proc file system is mounted; raise a @code{system-error} exc=
eption
    otherwise."
      (call-with-input-file (string-append "/proc/" (number->string pid)
                                           "/stat")
        (lambda (port)
          (define line
            (get-string-all port))

          ;; Parse like systemd's 'is_kernel_thread' function.
          (let ((offset (string-index line #\))))     ;offset past 'tcomm' =
field
            (match (and offset
                        (string-tokenize (string-drop line (+ offset 1))))
              ((state ppid pgrp sid tty-nr tty-pgrp flags . _)
               (or (string->number flags) 0))
              (_
               0))))))
    scheme@(guile-user)> ;; Per-process flag defined in <linux/sched.h>.
    (define PF_KTHREAD #x00200000)                    ;I am a kernel thread
    scheme@(guile-user)> (define (linux-kernel-thread? pid)
      "Return true if @var{pid} is a Linux kernel thread."
      (=3D PF_KTHREAD (logand (linux-process-flags pid) PF_KTHREAD)))
    scheme@(guile-user)> (linux-kernel-thread? 685)
    $5 =3D #t

Tomas

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

--SJ55xua6WttYwtt4
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEt4NJs4wUfTYpiGikL7/ufbZ/wakFAmW34+AACgkQL7/ufbZ/
walllQ/+MMP8zMtfyQq/pP6AgKUs+m5cs4OsVhHEhoZjoVPPsewzN8UhMtsnycm0
WelkCJhCmI0JiM/mlNUEaJdgZUWI7jX4V5QbJM/k3DmH05E37UXtqMSShFjkyIJs
YdBgNwanxz31A9XhaPutizoWAN3M6V2Tj4wDQlsjI39fkQ0d2N8iuq2Lo7JOs81c
7S/xZqH5gcE1u2wfyw23equFOD88KGpvvdr/xBwM7pBZTP52FY7vYgE9/mLAUyQz
B5NrSERCVrbOHZHAorh50JFWKRDHHawK9oekac6+v2jPPTnBUtOg0LPZ+ms34QIe
Ln8wxNoUrPZmt47hXj6xw6Q/5sKHW9HQsBoo+4P/gpkGMNJbOAnms1L4vSv1NCq1
UtlvQOQiY3cW5rzU3iyoAuoctVmoPFn2yODAjNgCEs7pp8IxpdJ+pA+p4stVykyf
jDN5FV8G0ZFUc7um+gHIx8/vBTinoV/yE6JNF8l0jdWhJhnNO+BaHjGtJRDPZesd
ZPGywofauv07y+4sGOoKZ2N4FqvVTn2o0CXpGom74ZMYjpWwFam6hhs2fiygavmI
xjTS3M8Gv804vnQyPHmLb+Rbkf5ENRnTyoHCZgirK/PNk1n/Hlnls7VpIJRdfcxX
QdDMGCgfClZoZTEd2HVl0TyIF4Gj8igTwmvogKMlsdJM+atXHfM=
=cLVC
-----END PGP SIGNATURE-----

--SJ55xua6WttYwtt4--




Information forwarded to bug-guix@HIDDEN:
bug#68800; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 29 Jan 2024 17:35:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 29 12:35:00 2024
Received: from localhost ([127.0.0.1]:33366 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1rUVX9-0005HR-4J
	for submit <at> debbugs.gnu.org; Mon, 29 Jan 2024 12:35:00 -0500
Received: from lists.gnu.org ([2001:470:142::17]:59778)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <~@wolfsden.cz>) id 1rUVX6-0005Gz-Jp
 for submit <at> debbugs.gnu.org; Mon, 29 Jan 2024 12:34:57 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <~@wolfsden.cz>) id 1rUVWs-0008GZ-Ax
 for bug-guix@HIDDEN; Mon, 29 Jan 2024 12:34:42 -0500
Received: from wolfsden.cz ([37.205.8.62])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <~@wolfsden.cz>) id 1rUVWp-00066O-3e
 for bug-guix@HIDDEN; Mon, 29 Jan 2024 12:34:41 -0500
Received: by wolfsden.cz (Postfix, from userid 104)
 id DD4362823CF; Mon, 29 Jan 2024 17:34:35 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail;
 t=1706549675; bh=jFOpjirVrdVbrV70pmiOSYfzNl+r9HEweDV2BNLPDOk=;
 h=Date:From:To:Subject;
 b=na81pW5xk3x4Lfz51J3dgB8gfvglkSyG0aAZcr+N8sr1iWi6TS2p6PpGNCHCEhhLs
 /G++OlnbFp1FaPX9axlUCagbHFf8NBzrkRBcspoaH+F0zEDbKjNtDu/Swe9LBeI7B+
 tJLcHqofsRuqZ6jKuIp21cPTwZ1lYGDfO9e+YsmV5Rw/DO/JKI9xU/Igcc93JEDWvZ
 wFUGU0wq2/blZ9Tns4jJN+CEktC/0lKnE8s4+I/TWQgsDw7YXmyVdupqgmf0XARmx3
 svmP3n2Mli9tF9h+1kZvIQIAdySG6C90uPfJc9kU7b6MQ5PMbvSn7XbT8CsG5IaAvw
 x3NUhUTk6iEruSDR8bqGgomPPCSQ42m+H3v3hlkx1FnRl87hZlBkyO1O2boMePVpYJ
 l8q/9KeZaGQ8ch1xxl5XFOaxH+JMcQcS1o4ccTyof+5iMcj3uBeW8iH39O35IZYf1X
 cl+esxYBL0svWR46iNJziUm10hiLzr86jwOnMRZIIwXzTNiqwTU2v95NEub69W9bET
 ELPWUvXMLQLjPp3A8RUY2MiGSwQjeGmxx2IYklGHlFSlnx9+Q0ttwolE+f7NP/Ezj9
 i87WIAM6GieJTZ0SJZA0BqSYQYu+7woxz4jvmlhm9DYQLFv1RR3LohziruOod5fFt/
 6/e1f8e+AUYQ3eGgBIJ1PbVY=
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on wolfsden
X-Spam-Level: 
X-Spam-Status: No, score=-1.2 required=5.0 tests=ALL_TRUSTED,DKIM_SIGNED,
 DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,T_SCC_BODY_TEXT_LINE,
 URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6
Received: from localhost (unknown [193.32.127.154])
 by wolfsden.cz (Postfix) with ESMTPSA id 2E2CA28389D
 for <bug-guix@HIDDEN>; Mon, 29 Jan 2024 17:34:35 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail;
 t=1706549675; bh=jFOpjirVrdVbrV70pmiOSYfzNl+r9HEweDV2BNLPDOk=;
 h=Date:From:To:Subject;
 b=na81pW5xk3x4Lfz51J3dgB8gfvglkSyG0aAZcr+N8sr1iWi6TS2p6PpGNCHCEhhLs
 /G++OlnbFp1FaPX9axlUCagbHFf8NBzrkRBcspoaH+F0zEDbKjNtDu/Swe9LBeI7B+
 tJLcHqofsRuqZ6jKuIp21cPTwZ1lYGDfO9e+YsmV5Rw/DO/JKI9xU/Igcc93JEDWvZ
 wFUGU0wq2/blZ9Tns4jJN+CEktC/0lKnE8s4+I/TWQgsDw7YXmyVdupqgmf0XARmx3
 svmP3n2Mli9tF9h+1kZvIQIAdySG6C90uPfJc9kU7b6MQ5PMbvSn7XbT8CsG5IaAvw
 x3NUhUTk6iEruSDR8bqGgomPPCSQ42m+H3v3hlkx1FnRl87hZlBkyO1O2boMePVpYJ
 l8q/9KeZaGQ8ch1xxl5XFOaxH+JMcQcS1o4ccTyof+5iMcj3uBeW8iH39O35IZYf1X
 cl+esxYBL0svWR46iNJziUm10hiLzr86jwOnMRZIIwXzTNiqwTU2v95NEub69W9bET
 ELPWUvXMLQLjPp3A8RUY2MiGSwQjeGmxx2IYklGHlFSlnx9+Q0ttwolE+f7NP/Ezj9
 i87WIAM6GieJTZ0SJZA0BqSYQYu+7woxz4jvmlhm9DYQLFv1RR3LohziruOod5fFt/
 6/e1f8e+AUYQ3eGgBIJ1PbVY=
Date: Mon, 29 Jan 2024 18:34:33 +0100
From: Tomas Volf <~@wolfsden.cz>
To: bug-guix@HIDDEN
Subject: Guix waits for termination of a kernel thread
Message-ID: <ZbfhqeMUdS63NB93@ws>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature"; boundary="e/gKb4hM0SJIFpOF"
Content-Disposition: inline
Received-SPF: pass client-ip=37.205.8.62; envelope-from=~@wolfsden.cz;
 helo=wolfsden.cz
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)


--e/gKb4hM0SJIFpOF
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hello,

when Guix machine is shutting down, it keeps waiting for PID associated with
[mt76-tx phy0] to terminate.  Since it is a kernel thread, it does not happ=
en.

Previous discussion on this bug was done via email, and is copied here:



Date: Sun, 7 Jan 2024 15:59:51 +0100
=46rom: Tomas Volf <~@wolfsden.cz>
To: guix-devel@HIDDEN
Subject: Re: GNU Shepherd 0.10.3 released

On 2024-01-07 15:08:59 +0100, Ludovic Court=C3=A8s wrote:
> We are pleased to announce the GNU Shepherd version 0.10.3, a bug-fix
> release of the new 0.10.x series, representing 51 commits over 6 months.

Congratulations on the release :)

>
>   ** Do not accidentally wait for Linux kernel thread completion
>      (<https://issues.guix.gnu.org/67132>)
>
>   In cases a PID file contained a bogus PID or one that=E2=80=99s only va=
lid in a
>   separate PID namespace, shepherd could end up waiting for the terminati=
on of
>   what=E2=80=99s actually a Linux kernel thread, such as PID 2 (=E2=80=9C=
kthreadd=E2=80=9D).  This
>   situation is now recognized and avoided.

This is great, I will not have to remember to run `modprobe -r mt7921e' bef=
ore
each shutdown anymore.  I hope.  Looking forward to getting it in the Guix =
:)

Have a nice 2024,
Tomas Volf



Date: Wed, 10 Jan 2024 00:34:48 +0100
=46rom: Ludovic Court=C3=A8s <ludo@HIDDEN>
To: guix-devel@HIDDEN
Subject: Re: GNU Shepherd 0.10.3 released

Tomas Volf <~@wolfsden.cz> skribis:

> On 2024-01-07 15:08:59 +0100, Ludovic Court=C3=A8s wrote:

[...]

>>   ** Do not accidentally wait for Linux kernel thread completion
>>      (<https://issues.guix.gnu.org/67132>)
>>
>>   In cases a PID file contained a bogus PID or one that=E2=80=99s only v=
alid in a
>>   separate PID namespace, shepherd could end up waiting for the terminat=
ion of
>>   what=E2=80=99s actually a Linux kernel thread, such as PID 2 (=E2=80=
=9Ckthreadd=E2=80=9D).  This
>>   situation is now recognized and avoided.
>
> This is great, I will not have to remember to run `modprobe -r mt7921e' b=
efore
> each shutdown anymore.  I hope.  Looking forward to getting it in the Gui=
x :)

D=E2=80=99oh, why did you have to do that?  How did Shepherd end up with =
=E2=80=9Cwrong=E2=80=9D PID?

I hope this release fixes it!

Ludo=E2=80=99.



Date: Wed, 10 Jan 2024 17:38:17 +0100
=46rom: Tomas Volf <~@wolfsden.cz>
To: Ludovic Court=C3=A8s <ludo@HIDDEN>
Cc: guix-devel@HIDDEN
Subject: Re: GNU Shepherd 0.10.3 released

On 2024-01-10 00:34:48 +0100, Ludovic Court=C3=A8s wrote:
> Tomas Volf <~@wolfsden.cz> skribis:
>
> > On 2024-01-07 15:08:59 +0100, Ludovic Court=C3=A8s wrote:
>
> [...]
>
> >>   ** Do not accidentally wait for Linux kernel thread completion
> >>      (<https://issues.guix.gnu.org/67132>)
> >>
> >>   In cases a PID file contained a bogus PID or one that=E2=80=99s only=
 valid in a
> >>   separate PID namespace, shepherd could end up waiting for the termin=
ation of
> >>   what=E2=80=99s actually a Linux kernel thread, such as PID 2 (=E2=80=
=9Ckthreadd=E2=80=9D).  This
> >>   situation is now recognized and avoided.
> >
> > This is great, I will not have to remember to run `modprobe -r mt7921e'=
 before
> > each shutdown anymore.  I hope.  Looking forward to getting it in the G=
uix :)
>
> D=E2=80=99oh, why did you have to do that?

Otherwise the shepherd would be stuck on shutdown waiting for process named

    [mt76-tx phy0]

to terminate with messages along the lines of:

    shepherd[1]: waiting for process termination (processes left: (1 678))

It is a kernel thread as far as I can tell (based on
https://stackoverflow.com/a/12231039):

    $ cd /proc/678
    $ cat cmdline
    $ readlink exe; echo $?
    1

Removing the module mt7921e stops the thread, so shepherd does not wait for=
 it.

> How did Shepherd end up with =E2=80=9Cwrong=E2=80=9D PID?

That I do not know.  It is visible in `ps' output, so I assume shepherd pic=
ked
it up on its own somehow.

>
> I hope this release fixes it!

As far as I can tell, the 0.10.3 was already added into guix:

    $ ps 1 | cat
      PID TTY      STAT   TIME COMMAND
        1 ?        Sl     0:01 /gnu/store/bhynhk0c6ssq3fqqc59fvhxjzwywsjbb-=
guile-3.0.9/bin/guile --no-auto-compile /gnu/store/06mz0yjkghi7r6d7lmhvv7gr=
yipljhdd-shepherd-0.10.3/bin/shepherd
+--config /gnu/store/klkqq2y65k141rlipq4ls0w2rlhds12h-shepherd.conf

So I have to say it sadly did not resolve this issue.  I am unsure why thou=
gh.
I am not familiar with Shepherd's code base, but quick look at the git log
suggested that procedure (@@ (shepherd service) pseudo-process?) is the rel=
evant
one.  When I try it from a REPL, it returns #t.

    $ guix shell guile shepherd guile-fibers -- guile
    GNU Guile 3.0.9
    Copyright (C) 1995-2023 Free Software Foundation, Inc.

    Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
    This program is free software, and you are welcome to redistribute it
    under certain conditions; type `,show c' for details.

    Enter `,help' for help.
    scheme@(guile-user)> ,use (shepherd service)
    scheme@(guile-user)> ((@@ (shepherd service) pseudo-process?) 688)
    $1 =3D #t

So it *should* work?  However the issue is caused by non-free WiFi driver o=
n a
corrupted kernel, so I am not sure if it is even problem that needs to be
solved...  I would (obviously) like to see it resolved, but I probably cann=
ot
even bug report it, since it requires non-free hardware and software to
reproduce.

Tomas

PS: It is interesting that `guix shell guile shepherd' is not enough, the
    guile-fibers have to be explicitly specified as well.  Is that expected?



Date: Wed, 10 Jan 2024 17:50:19 +0100
=46rom: Tomas Volf <~@wolfsden.cz>
To: Ludovic Court=C3=A8s <ludo@HIDDEN>, guix-devel@HIDDEN
Subject: Re: GNU Shepherd 0.10.3 released

PS:

On 2024-01-10 17:38:17 +0100, Tomas Volf wrote:
>     scheme@(guile-user)> ((@@ (shepherd service) pseudo-process?) 688)

The pid is different than above, because this was after a reboot.

Tomas



Date: Thu, 11 Jan 2024 13:41:39 +0100
=46rom: Ludovic Court=C3=A8s <ludo@HIDDEN>
To: guix-devel@HIDDEN
Subject: Re: GNU Shepherd 0.10.3 released

Hello,

Tomas Volf <~@wolfsden.cz> skribis:

> Otherwise the shepherd would be stuck on shutdown waiting for process nam=
ed
>
>     [mt76-tx phy0]
>
> to terminate with messages along the lines of:
>
>     shepherd[1]: waiting for process termination (processes left: (1 678))
>
> It is a kernel thread as far as I can tell (based on
> https://stackoverflow.com/a/12231039):
>
>     $ cd /proc/678
>     $ cat cmdline
>     $ readlink exe; echo $?
>     1
>
> Removing the module mt7921e stops the thread, so shepherd does not wait f=
or it.

Ooooh.

Then I=E2=80=99m afraid this bug isn=E2=80=99t fixed yet because that code =
(=E2=80=9Cwaiting for
process termination=E2=80=9D) is currently in Guix, not in Shepherd.

However, =E2=80=98processes=E2=80=99, which is what is used here and which =
is defined in
(guix build syscalls), already checks for kernel threads, though it
does it differently than what I implemented in shepherd:

  (define (kernel? pid)
    "Return #t if PID designates a \"kernel thread\" rather than a normal
  user-land process."
    (let ((stat (call-with-input-file (format #f "/proc/~a/stat" pid)
                  (compose string-tokenize read-string))))
      ;; See proc.txt in Linux's documentation for the list of fields.
      (match stat
        ((pid tcomm state ppid pgrp sid tty_nr tty_pgrp flags min_flt
              cmin_flt maj_flt cmaj_flt utime stime cutime cstime
              priority nice num_thread it_real_value start_time
              vsize rss rsslim
              (=3D string->number start_code) (=3D string->number end_code)=
 _ ...)
         ;; Got this obscure trick from sysvinit's 'killall5' program.
         (and (zero? start_code) (zero? end_code))))))

It would be great if you could check whether this approach works for
you.

(I had completely forgotten about this code.  Funny thing is this one
was inspired by sysvinit, whereas the one in Shepherd was inspired by
systemd.  A sign of times!)

Ludo=E2=80=99.



Date: Thu, 11 Jan 2024 14:12:51 +0100
=46rom: Tomas Volf <~@wolfsden.cz>
To: Ludovic Court=C3=A8s <ludo@HIDDEN>
Cc: guix-devel@HIDDEN
Subject: Re: GNU Shepherd 0.10.3 released

On 2024-01-11 13:41:39 +0100, Ludovic Court=C3=A8s wrote:
> Tomas Volf <~@wolfsden.cz> skribis:
>
> > Otherwise the shepherd would be stuck on shutdown waiting for process n=
amed
> >
> >     [mt76-tx phy0]
> >
> > to terminate with messages along the lines of:
> >
> >     shepherd[1]: waiting for process termination (processes left: (1 67=
8))
> >
> > It is a kernel thread as far as I can tell (based on
> > https://stackoverflow.com/a/12231039):
> >
> >     $ cd /proc/678
> >     $ cat cmdline
> >     $ readlink exe; echo $?
> >     1
> >
> > Removing the module mt7921e stops the thread, so shepherd does not wait=
 for it.
>
> Ooooh.
>
> Then I=E2=80=99m afraid this bug isn=E2=80=99t fixed yet because that cod=
e (=E2=80=9Cwaiting for
> process termination=E2=80=9D) is currently in Guix, not in Shepherd.
>
> However, =E2=80=98processes=E2=80=99, which is what is used here and whic=
h is defined in
> (guix build syscalls), already checks for kernel threads, though it
> does it differently than what I implemented in shepherd:
>
>   (define (kernel? pid)
>     "Return #t if PID designates a \"kernel thread\" rather than a normal
>   user-land process."
>     (let ((stat (call-with-input-file (format #f "/proc/~a/stat" pid)
>                   (compose string-tokenize read-string))))
>       ;; See proc.txt in Linux's documentation for the list of fields.
>       (match stat
>         ((pid tcomm state ppid pgrp sid tty_nr tty_pgrp flags min_flt
>               cmin_flt maj_flt cmaj_flt utime stime cutime cstime
>               priority nice num_thread it_real_value start_time
>               vsize rss rsslim
>               (=3D string->number start_code) (=3D string->number end_cod=
e) _ ...)
>          ;; Got this obscure trick from sysvinit's 'killall5' program.
>          (and (zero? start_code) (zero? end_code))))))
>
> It would be great if you could check whether this approach works for
> you.

Ah, that code indeed returns #f for the pid in question:

    scheme@(guix-user)> ((@@ (guix build syscalls) kernel?) 688)
    $1 =3D #f

The stat file:

    $ cat /proc/688/stat
    688 (mt76-tx phy0) S 2 0 0 0 -1 2129984 0 0 0 0 0 0 0 0 -2 0 1 0 964 0 =
0 18446744073709551615 0 0 0 0 0 0 0 2147483647 0 0 0 0 17 5 1 1 0 0 0 0 0 =
0 0 0 0 0 0

So the start_code is not zero (I would guess it is -1).  I have no idea what
that means though.

Tomas



Date: Mon, 29 Jan 2024 17:31:33 +0100
=46rom: Ludovic Court=C3=A8s <ludo@HIDDEN>
To: guix-devel@HIDDEN
Subject: Re: GNU Shepherd 0.10.3 released

Hi,

Tomas Volf <~@wolfsden.cz> skribis:

> Ah, that code indeed returns #f for the pid in question:
>
>     scheme@(guix-user)> ((@@ (guix build syscalls) kernel?) 688)
>     $1 =3D #f
>
> The stat file:
>
>     $ cat /proc/688/stat
>     688 (mt76-tx phy0) S 2 0 0 0 -1 2129984 0 0 0 0 0 0 0 0 -2 0 1 0 964 =
0 0 18446744073709551615 0 0 0 0 0 0 0 2147483647 0 0 0 0 17 5 1 1 0 0 0 0 =
0 0 0 0 0 0 0
>
> So the start_code is not zero (I would guess it is -1).  I have no idea w=
hat
> that means though.

What about this method (from shepherd)?

--8<---------------cut here---------------start------------->8---
(define (linux-process-flags pid)
  "Return the process flags of @var{pid} (or'd @code{PF_} constants), assum=
ing
the Linux /proc file system is mounted; raise a @code{system-error} excepti=
on
otherwise."
  (call-with-input-file (string-append "/proc/" (number->string pid)
                                       "/stat")
    (lambda (port)
      (define line
        (get-string-all port))

      ;; Parse like systemd's 'is_kernel_thread' function.
      (let ((offset (string-index line #\))))     ;offset past 'tcomm' field
        (match (and offset
                    (string-tokenize (string-drop line (+ offset 1))))
          ((state ppid pgrp sid tty-nr tty-pgrp flags . _)
           (or (string->number flags) 0))
          (_
           0))))))

;; Per-process flag defined in <linux/sched.h>.
(define PF_KTHREAD #x00200000)                    ;I am a kernel thread

(define (linux-kernel-thread? pid)
  "Return true if @var{pid} is a Linux kernel thread."
  (=3D PF_KTHREAD (logand (linux-process-flags pid) PF_KTHREAD)))
--8<---------------cut here---------------end--------------->8---

If it works better, we can use that in syscalls.scm as well.

Ludo=E2=80=99.

PS: Having an entry in bug-guix would ensure we don=E2=80=99t lose track of=
 this.


--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

--e/gKb4hM0SJIFpOF
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEt4NJs4wUfTYpiGikL7/ufbZ/wakFAmW34akACgkQL7/ufbZ/
walrvxAAtplW7nd59Y3C5bPa2M8hb/ykYPFDE7gOKIvrpb7rtd3+CV72oEKceM5g
YRvmhITrW98eZA22xm43qZYHKSo8JzpmTiU+Pq/ern4nnUyzLphzO6SzaP/qzfsL
KmPCqwnVrzlfrw6katYuu36fgpLSIejmEUdrOWPbq7tdO+cIMAL5+YMN8nhJVlVV
hUBLQogP+K0NzOKtImhQliZ5sK6Ggr5MbK/OdjZwwEThV0mCnzHli2fK+tnP8tD8
PAptDlEdf6OGmSYLInI/rAFaH/w9ZtPz4UxF1wx8+eRPlaklGIg37Q0dNDrIdaA/
QMAk0A8Y+NljTMuCdEoNSZX19Kh9FHPTbuB4jc8unEGh3fwEz/C5twvLEBIGZIFC
v0HEw5VgvbK6oNzvLAvpNEyVe6TwIuycGFYjJD0wMI7M43NNSsclsQQGYd8IPKt/
OFpIkYrtfzKffDFztXAWYEm8HgcAS0en8ovXhqRkpVolioXjMhDCMuwK3A+FWGNl
/yXkkN4Nxx/80Bk1gUnJG2R3GL4ud5n16IWB455EDdigMK5Lpl37moN5If9/R0dY
4jk4TI3AKP/tmGdUo4EENEuCJMW44qrE6dKqwxkqQn/ZOY7TiRoQaKat+JbYAeU8
tH/mKZ/J492AU3fbyxBE3SaWt+Y8l44StIPjuvj8yMvKd70O/MI=
=RIKL
-----END PGP SIGNATURE-----

--e/gKb4hM0SJIFpOF--




Acknowledgement sent to Tomas Volf <~@wolfsden.cz>:
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#68800; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Tue, 20 Feb 2024 09:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.