GNU bug report logs - #33018
26.1.50; thread starvation with async processes and accept-process-output

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: "Basil L. Contovounesios" <contovob@HIDDEN>; dated Thu, 11 Oct 2018 14:59:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 33018 <at> debbugs.gnu.org:


Received: (at 33018) by debbugs.gnu.org; 12 Oct 2018 12:49:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 12 08:49:37 2018
Received: from localhost ([127.0.0.1]:46002 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gAwsr-00088B-4x
	for submit <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:49:37 -0400
Received: from mout.gmx.net ([212.227.17.22]:54245)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael.albinus@HIDDEN>) id 1gAwsp-00087y-0v
 for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:49:35 -0400
Received: from detlef.gmx.de ([212.91.243.207]) by mail.gmx.com (mrgmx103
 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MHXd2-1gEFCd1E3Y-003Lkm; Fri, 12
 Oct 2018 14:49:27 +0200
Received: from detlef.gmx.de ([212.91.243.207]) by mail.gmx.com (mrgmx103
 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MHXd2-1gEFCd1E3Y-003Lkm; Fri, 12
 Oct 2018 14:49:27 +0200
From: Michael Albinus <michael.albinus@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
 thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
 <83d0sfwd14.fsf@HIDDEN>
Date: Fri, 12 Oct 2018 14:49:25 +0200
In-Reply-To: <83d0sfwd14.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 12 Oct
 2018 15:43:51 +0300")
Message-ID: <871s8vcotm.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K1:6qIHB5ce0VxEcW9PcdryqKVLX7GhWrplb5w9py7gf0kfNPIiv5G
 Cxg/Pekdzdj4/BaZ2tWurb8dYZ8ilTJ9jUUOeVjY8zOPKcgzlt8oUfrFzd4Hb3Uddm0pU0X
 3XpffhSZr40TTAz2MlUnwNeaL7uMECyNIiSPik5pAWEVJII3dbdmQ23J+fUg59zw72c5JHh
 Y5iAmTY2smerFtzKuoK3w==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V01:K0:Hssog6mhvpM=:c2o9KJbHyl9FOo5dyo99XI
 GzuF3YjK+jZa7VtFgW25/MRl8F8LhsdJaiubJQhFBY8FsejaXtItCETazs6dxe2ff6iXbxii+
 lIYBRNnQVHrp061e3D+xzsgJxIzmbyGSqpsMAsHA6FEzGGLZKy/8a8Zh3QgjQYbSMfwfi93CX
 JcerVbLyAzb3bMR5215t4OdVJjgN4gnGGejsLQqRD0XA8sA+qBhTZbfRv4Vq8wzwdOIjRQQNi
 ApVwr2Qo/4HVAhbQHKPe9TkN6fIOWERIOk000SSUCqiWPFdITwZHqcwWqBJg9A5101zJMPHII
 SdrW4Z76MYJTMnlumPKFGQSBvDN9Tkc5H8Y2N0Wj9aYH+CGMoD6QJn0MPkIaf0LGQF8/cztFl
 k1UlM1c0ejfX0Ry44DMyhVS+E+UKPFN+9Ol7ImPQ+26eD3j4oJ26GuwEc2luWe8eM6NWvYzoG
 xKXMVwu+k4/uqZJI8/86wdUk1dGtG1LzYh94LBO4l/wLZtXgqr4hXUgPGPW62L7L6BK1ekEMt
 D33mXy2XOiNqpSRmnxm82WbE7bHTH5Xl/KuqidthuZO4EU56jkKCZbptkgaJcwApFSMiJHjE3
 NpABQ6QyLwFhTqRy/aPRBXUXNgrrwn0i2XoQhvrQslSeEsxD73yrLeyH+gmMpaPTPJDD5Eubc
 Qx4Ke6qBvFlYYqMr4yCE6A6EXFayBXyo4QY5om5kcGugtM1AE4N+7wuv6A2yXjvRFUzcuCp6b
 Lb8SwWtUAaVANIbeVfZJSnUsHVUicStETyfae5+K3PfX+75UEzUKJcr3SuXt5sjRmPZJ7YvoK
 h4ICgi+it+tV83VUcNXj57zuIEiZJguqi3SrcutzwW92BjLH7o=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 33018
Cc: contovob@HIDDEN, 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Eli Zaretskii <eliz@HIDDEN> writes:

>> If you want a process to communicate in a given thread, you must call
>> `set-process-thread'. See the elisp manual.
>
> But the default is that the process is locked to the thread that
> created it, so it sounds like this should have just worked (if that is
> the problem).

I'm not sure. In the branch feature/tramp-thread-safe there were also
mysterious blockings in accept-process-output, until I've applied
set-process-thread explicitly.

At least it is worth a try.

Best regards, Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#33018; Package emacs. Full text available.

Message received at 33018 <at> debbugs.gnu.org:


Received: (at 33018) by debbugs.gnu.org; 12 Oct 2018 12:43:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 12 08:43:59 2018
Received: from localhost ([127.0.0.1]:45994 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gAwnP-0007zQ-7D
	for submit <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:43:59 -0400
Received: from eggs.gnu.org ([208.118.235.92]:41585)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1gAwnN-0007zB-Bd
 for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:43:57 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1gAwnG-0005xL-7r
 for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:43:52 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled
 version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34923)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1gAwnE-0005vR-VC; Fri, 12 Oct 2018 08:43:49 -0400
Received: from [176.228.60.248] (port=1531 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1gAwnE-0000kG-9p; Fri, 12 Oct 2018 08:43:48 -0400
Date: Fri, 12 Oct 2018 15:43:51 +0300
Message-Id: <83d0sfwd14.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Michael Albinus <michael.albinus@HIDDEN>
In-reply-to: <875zy7cqzd.fsf@HIDDEN> (message from Michael Albinus on Fri, 12
 Oct 2018 14:02:46 +0200)
Subject: Re: bug#33018: 26.1.50;
 thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 33018
Cc: contovob@HIDDEN, 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -6.0 (------)

> From: Michael Albinus <michael.albinus@HIDDEN>
> Date: Fri, 12 Oct 2018 14:02:46 +0200
> Cc: 33018 <at> debbugs.gnu.org
> 
> If you want a process to communicate in a given thread, you must call
> `set-process-thread'. See the elisp manual.

But the default is that the process is locked to the thread that
created it, so it sounds like this should have just worked (if that is
the problem).




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#33018; Package emacs. Full text available.

Message received at 33018 <at> debbugs.gnu.org:


Received: (at 33018) by debbugs.gnu.org; 12 Oct 2018 12:02:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 12 08:02:56 2018
Received: from localhost ([127.0.0.1]:45977 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gAw9g-00071Z-MK
	for submit <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:02:56 -0400
Received: from mout.gmx.net ([212.227.17.22]:41241)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael.albinus@HIDDEN>) id 1gAw9f-00071O-EI
 for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:02:56 -0400
Received: from detlef.gmx.de ([212.91.243.207]) by mail.gmx.com (mrgmx102
 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MhNk6-1fyNLp2nq6-00Mf7y; Fri, 12
 Oct 2018 14:02:47 +0200
Received: from detlef.gmx.de ([212.91.243.207]) by mail.gmx.com (mrgmx102
 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MhNk6-1fyNLp2nq6-00Mf7y; Fri, 12
 Oct 2018 14:02:47 +0200
From: Michael Albinus <michael.albinus@HIDDEN>
To: "Basil L. Contovounesios" <contovob@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
 thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN>
Date: Fri, 12 Oct 2018 14:02:46 +0200
In-Reply-To: <87r2gwbmma.fsf@HIDDEN> (Basil L. Contovounesios's message of
 "Thu, 11 Oct 2018 15:57:50 +0100")
Message-ID: <875zy7cqzd.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K1:AjiYQbQ/rtPN6GaEJ8yeQKPPv53heoLxEIChbV5yzXGxytdWst2
 /QbigNT+vhaI/Hz7DhN7jvzx/9wkgaZVMdvONy7gusZS+ETG1ARsNuJg54CVqnzxsIjVrFy
 MO4RK8AyUqZJ5deU5Xp+5NhrTyDK9X/abSUcTJAbihPS9GUmVyGEJBZpoAVK8duWrtCp6qm
 T/Jp73cBIfZFwww4Or6zA==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V01:K0:kwdGLWAAmrw=:3O2n2J0vZ6w+YRSmha4QIQ
 3r9uRUr6WVmPdzr/hV061SPvapirOrONO1KG6mca7pR0qjbtgn7eKpyqoqg2nksfDaIVdA+P1
 //ifkI0uHGrMMp3VOSDZSDYtMKv2hOMDQF0oZv2cSPdBOltNfvQwx6jhxw9rwtOiI+r8iBLd1
 yE0Q2sHLw7NslH0fvIj81HCo9tqZ+HVj6LbbQmUaZggiPnnpZom4whQTfv0SLXCgdU4+bIxCD
 w1oKnyXwaHy2ezAdqXsNy84DJ1wKdpBPqIObh8UUv1zHb7yrIYYNnxmfu+L9pP23Yk09vTyMW
 m9BgzkT+1l4BXqzhp6QbQ9U7WKssndo5C0JFvwIP/QNfF+0p+FD3ZywB4iox5nT7nQjGxyaV8
 2WBj3vy4s6DfvIxUU5IykdqMJgkt4AlTWeCU9COGsJc4i9Uh6aUjJ++6LndOL8Xna1R4GUOcZ
 ubBjQ0EmbVELSq4Oa6++X0pge/8f4dIHTfUldpMGUVGqAhEgzxvUJ+yzEtfXf9euoRqbYRpDL
 vQqK3NYNIx+9etjgotPWM1Y8oRpYE/Lw46Xu6JJ1ImCq6LikQ2eX67QR2hAXgOqYGkVfW0QWI
 6PqCM93LTzQI99vejC1kKRBmUNuwKV51Roq7cfrRwpDhTCgdhb8l1/g26um7+wuK2Let/OdPN
 xMPz1Tb5Rr13HV8w3YwS3UGO/qDa8yIZVfmgc4vu84FZBHzhT730jX7r3XHQD3S/1OpAVSED1
 pKhpgGZgw8V7odx1C1K3JiQ16WEBgN0Lq5LC2eP/+vBwp2nbY4Yj4UeFf/mLNlCSmmqnlh7Cf
 MuKWvXRzNeFe+KbCWA0Aarorupo7GpGGTOsvfefgvXLiM1Bt94=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 33018
Cc: 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

"Basil L. Contovounesios" <contovob@HIDDEN> writes:

Hi Basil,

> I attach a sample program test.el whose central function, test-slave,
> invokes wget asynchronously before waiting for the process to exit.
>
> The issue I'm facing is that running test-slave twice in succession,
> each time in a new thread, causes accept-process-output to hang with no
> output (unless a timeout argument is given, in which case the function
> returns nil) the second time around.

If you want a process to communicate in a given thread, you must call
`set-process-thread'. See the elisp manual.

> Thanks,

Best regards, Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#33018; Package emacs. Full text available.

Message received at 33018 <at> debbugs.gnu.org:


Received: (at 33018) by debbugs.gnu.org; 12 Oct 2018 08:07:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 12 04:07:46 2018
Received: from localhost ([127.0.0.1]:45890 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gAsU5-0005o3-Pl
	for submit <at> debbugs.gnu.org; Fri, 12 Oct 2018 04:07:46 -0400
Received: from eggs.gnu.org ([208.118.235.92]:39547)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1gAsU4-0005nm-H3
 for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 04:07:44 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1gAsTt-00034N-75
 for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 04:07:37 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled
 version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:47618)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1gAsTn-00031Z-62; Fri, 12 Oct 2018 04:07:30 -0400
Received: from [176.228.60.248] (port=4335 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1gAsTf-000718-8x; Fri, 12 Oct 2018 04:07:22 -0400
Date: Fri, 12 Oct 2018 11:07:21 +0300
Message-Id: <83o9bzwpty.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: "Basil L. Contovounesios" <contovob@HIDDEN>
In-reply-to: <87r2gwbmma.fsf@HIDDEN> (contovob@HIDDEN)
Subject: Re: bug#33018: 26.1.50;
 thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 33018
Cc: 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -6.0 (------)

> From: "Basil L. Contovounesios" <contovob@HIDDEN>
> Date: Thu, 11 Oct 2018 15:57:50 +0100
> 
> I attach a sample program test.el whose central function, test-slave,
> invokes wget asynchronously before waiting for the process to exit.
> 
> The issue I'm facing is that running test-slave twice in succession,
> each time in a new thread, causes accept-process-output to hang with no
> output (unless a timeout argument is given, in which case the function
> returns nil) the second time around.

When the hang happens, is there any wget process still alive, or did
they all exit?  Please use OS tools to find that out, don't rely on
what Emacs thinks.

> I could very well be doing or assuming something incorrectly, but what
> baffles me is that the "hang" does not occur either when Emacs is run
> non-interactively, or when "https://en.wikipedia.org/wiki/Emacs" is
> replaced with "https://www.gnu.org/software/emacs/"

Could be different properties of the servers related to async
connections, like TLS handshake or even async getaddrinfo.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#33018; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 11 Oct 2018 14:58:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 11 10:58:16 2018
Received: from localhost ([127.0.0.1]:45384 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gAcPo-0002UC-4b
	for submit <at> debbugs.gnu.org; Thu, 11 Oct 2018 10:58:16 -0400
Received: from eggs.gnu.org ([208.118.235.92]:44674)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <contovob@HIDDEN>) id 1gAcPm-0002Tz-6l
 for submit <at> debbugs.gnu.org; Thu, 11 Oct 2018 10:58:14 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <contovob@HIDDEN>) id 1gAcPf-00036G-Fd
 for submit <at> debbugs.gnu.org; Thu, 11 Oct 2018 10:58:08 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:38962)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <contovob@HIDDEN>) id 1gAcPf-00036A-BT
 for submit <at> debbugs.gnu.org; Thu, 11 Oct 2018 10:58:07 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:54269)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <contovob@HIDDEN>) id 1gAcPe-0008MZ-8I
 for bug-gnu-emacs@HIDDEN; Thu, 11 Oct 2018 10:58:07 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <contovob@HIDDEN>) id 1gAcPa-00033L-6H
 for bug-gnu-emacs@HIDDEN; Thu, 11 Oct 2018 10:58:06 -0400
Received: from mail-ed1-x530.google.com ([2a00:1450:4864:20::530]:33923)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <contovob@HIDDEN>) id 1gAcPZ-000325-Oz
 for bug-gnu-emacs@HIDDEN; Thu, 11 Oct 2018 10:58:02 -0400
Received: by mail-ed1-x530.google.com with SMTP id w19-v6so8564032eds.1
 for <bug-gnu-emacs@HIDDEN>; Thu, 11 Oct 2018 07:58:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=tcd-ie.20150623.gappssmtp.com; s=20150623;
 h=from:to:subject:message-id:user-agent:date:mime-version;
 bh=1fAPAhyfsrUs5ys8GTfE2w//twZLn/LwhXUXVmkrhME=;
 b=s75hc5ndvYO5rLDmY1xU6kJ/nxe3s/9nHTZDwtNwElZgOMkooJYeT+sHHeld9faqBA
 15PZAHmUyfLOEKq2kFFx5d6ujHIYHxQVyozv7vwUcqPkbGMWGPlODhDFOyDjDk7yQEEj
 xTDVcuR87f/BajP5lQHNI2/fTLfW8bL9AL4ZDFve0tsZCSaQwUcw3w6PYTTweNBRwk5n
 QUOcN0ahSUNvwia2vVDLAjJeM+yMpFAVfO5AOy8ZObFcIC/fjlEAL1Q33VdvhwviQo14
 g+MrrfK7ur0nnzUapuf9aqVFjVro7V79abVHpaeFNlBx+4XSEu1gJY7NCdLR0tLAMoOT
 NvTA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:subject:message-id:user-agent:date
 :mime-version;
 bh=1fAPAhyfsrUs5ys8GTfE2w//twZLn/LwhXUXVmkrhME=;
 b=QkxoebHKjuOL+Y/nvnqpcTbFuSj2HivvsaWmkL/xbQf9IcbDz+7tt9ViVaw414jhGI
 pEyi3shja+PHFQg6cdsMOSx7l2Ao1MxPeZOU0yfBiOA+DFoYvdVV8iJRXWB5Z1Q7nQUi
 lhJ6aa16apnWzR6GvzlTAJZaaF85LtwrVhOEVgGfdwap4md8jOTg07s5EBa6qQ/bXGfT
 RwGIErFuB/4FwCRe1uBC4Xuh/KRk63n7k+4JZcpq+EB+zoQE7J3t1ahxyzA2siGfIH04
 pP3ek0KZFmouhB0T1pg6wNAVIyHPsAAQZwvYzDf+KxlmaElpf1HB05Q9VKSgfMSlZTc7
 612g==
X-Gm-Message-State: ABuFfogYs0GzgFpXYeT1OFSjbsrLAY88CoCdgEKULa60ctQ2QRbNRyqD
 +eFfsx+k2tRit8GEWHQWZP4f4lc1OWA=
X-Google-Smtp-Source: ACcGV62L3Mwrs7M430dKlpa4OJ9vrxxbpyen6pIvSZvlhQGiWAisz1HCUvcBNaXXxBhDCdZyJ8WrTA==
X-Received: by 2002:a17:906:5d10:: with SMTP id
 g16-v6mr2650055ejt.168.1539269878370; 
 Thu, 11 Oct 2018 07:57:58 -0700 (PDT)
Received: from localhost ([213.233.148.8])
 by smtp.gmail.com with ESMTPSA id q12-v6sm9823462edd.35.2018.10.11.07.57.57
 for <bug-gnu-emacs@HIDDEN>
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Thu, 11 Oct 2018 07:57:57 -0700 (PDT)
From: "Basil L. Contovounesios" <contovob@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 26.1.50;
 thread starvation with async processes and accept-process-output
Message-ID: <87r2gwbmma.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
Date: Thu, 11 Oct 2018 15:57:50 +0100
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -6.0 (------)

--=-=-=
Content-Type: application/emacs-lisp
Content-Disposition: attachment; filename=test.el
Content-Transfer-Encoding: quoted-printable

;;; test.el --- test -*- lexical-binding: t -*-

;;; Code:

(defun test-debug (&rest args)
  "Print `current-thread' and ARGS in timestamped `message'."
  (push (current-thread) args)
  (message "[%s] %s"
           (format-time-string "%T:%3N")
           (mapconcat #'prin1-to-string args " ")))

(defun test-sentinel (proc msg)
  "Kill buffer of PROC when it exits."
  (test-debug proc 'sentinel (process-status proc) (substring msg 0 -1))
  (and (not (eq (process-status proc) 'run))
       (buffer-live-p (process-buffer proc))
       (kill-buffer   (process-buffer proc))))

(defun test-slave ()
  "Create an asynchronous process and wait for it to exit."
  (let ((proc (make-process
               :name "test"
               :buffer (generate-new-buffer " *test*")
               :command '("wget" "-qO-" "https://en.wikipedia.org/wiki/Emac=
s")
               :connection-type 'pipe
               :sentinel #'test-sentinel)))
    (while (eq (process-status proc) 'run)
      (test-debug proc 'accept-output (accept-process-output proc 5)))
    (test-debug proc 'exit (process-status proc) (process-exit-status proc)=
)))

(defun test-master (async)
  "Successively run `test-slave'.
With non-nil ASYNC, use a new thread for each invocation."
  (dotimes (i 5)
    (if async
        (thread-join (make-thread #'test-slave (format "test-%d" i)))
      (test-slave))))

(defun test-job (async)
  "Successively run `test-slave'.
Like `test-master', but avoid blocking interactive sessions by
running `test-master' in its own thread."
  (if noninteractive
      (test-master async)
    (make-thread (apply-partially #'test-master async) "test-master")))

(defun test-threads ()
  "Run `test-master' with threads."
  (test-job t))

(defun test-no-threads ()
  "Run `test-master' without threads."
  (test-job nil))

;;; test.el ends here

--=-=-=
Content-Type: text/plain


I attach a sample program test.el whose central function, test-slave,
invokes wget asynchronously before waiting for the process to exit.

The issue I'm facing is that running test-slave twice in succession,
each time in a new thread, causes accept-process-output to hang with no
output (unless a timeout argument is given, in which case the function
returns nil) the second time around.

When this happens, the process sentinel is never called, which is why
I'm assuming accept-process-output is indeed "hanging" in some sense and
it's not just that the process has already exited and so has no further
output.

I could very well be doing or assuming something incorrectly, but what
baffles me is that the "hang" does not occur either when Emacs is run
non-interactively, or when "https://en.wikipedia.org/wiki/Emacs" is
replaced with "https://www.gnu.org/software/emacs/", or when test-slave
is run in the current thread (and not in make-thread).

Since I can reliably reproduce this on both an optimised build of master
and a non-optimised build of emacs-26, I hope to be able to provide
further insights using gdb as time allows.  Please let me know if there
are any specific details/output you would like me to provide.  As a
relatively inexperienced gdb user I welcome any tips and tricks for
debugging threads and processes.

Here are some ways test.el can be run to illustrate the issue:

# All five processes exit successfully.
emacs -batch -l test.el -f test-no-threads

# All five processes (and threads) exit successfully.
emacs -batch -l test.el -f test-threads

# All five processes exit successfully.
emacs -Q -l test.el -f test-no-threads

# First process and thread exit successfully,
# but accept-process-output starts timing out in second thread.
# Warning: may leave empty wget-log files lying around.
emacs -Q -l test.el -f test-threads

Details of the two Emacs versions I'm using follow:

In GNU Emacs 26.1.50 (build 2, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2018-10-11 built on thunk
Repository revision: a7ebc6bf633bd3849ccab032dad6b1fd31b1ef43
Windowing system distributor 'The X.Org Foundation', version 11.0.12001000
System Description:	Debian GNU/Linux testing (buster)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O0 -g3 -ggdb -gdwarf-4 -pipe'
 --config-cache --prefix=/home/blc/.local --program-suffix=26
 --enable-checking=yes,glyphs --enable-check-lisp-object-type
 --with-mailutils --with-x-toolkit=lucid --with-modules
 --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT
ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS LIBSYSTEMD
LCMS2

In GNU Emacs 27.0.50 (build 21, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2018-10-11 built on thunk
Repository revision: 5bd8cfc14d4b0c78c07e65a583f42a10c4cbc06d
Windowing system distributor 'The X.Org Foundation', version 11.0.12001000
System Description: Debian GNU/Linux buster/sid

Configured using:
 'configure --config-cache --prefix=/home/blc/.local --with-mailutils
 --with-x-toolkit=lucid --with-modules --with-file-notification=yes
 --with-x 'CC=ccache gcc' 'CFLAGS=-O2 -march=native -pipe''

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT
ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS LIBSYSTEMD
JSON LCMS2 GMP

Thanks,

-- 
Basil

--=-=-=--




Acknowledgement sent to "Basil L. Contovounesios" <contovob@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#33018; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Fri, 12 Oct 2018 13:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.