Received: (at 33018) by debbugs.gnu.org; 20 Oct 2018 08:35:10 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 20 04:35:10 2018 Received: from localhost ([127.0.0.1]:60249 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gDmiz-0002kc-L3 for submit <at> debbugs.gnu.org; Sat, 20 Oct 2018 04:35:09 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51581) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1gDmiy-0002kK-2Q for 33018 <at> debbugs.gnu.org; Sat, 20 Oct 2018 04:35:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gDmip-0003KF-KI for 33018 <at> debbugs.gnu.org; Sat, 20 Oct 2018 04:35:02 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=BAYES_05 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:36384) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gDmip-0003K6-Fe; Sat, 20 Oct 2018 04:34:59 -0400 Received: from [176.228.60.248] (port=2670 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <eliz@HIDDEN>) id 1gDmip-0002SL-2T; Sat, 20 Oct 2018 04:34:59 -0400 Date: Sat, 20 Oct 2018 11:34:40 +0300 Message-Id: <83efcluicf.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: "Basil L. Contovounesios" <contovob@HIDDEN> In-reply-to: <87zhvc2te8.fsf@HIDDEN> (contovob@HIDDEN) Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN> <878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN> <877eiipu8g.fsf@HIDDEN> <83k1mirl9n.fsf@HIDDEN> <874ldkjwzn.fsf@HIDDEN> <83zhvcqwxc.fsf@HIDDEN> <87o9bswirc.fsf@HIDDEN> <83y3awqvtl.fsf@HIDDEN> <83woqgqvl2.fsf@HIDDEN> <87zhvc2te8.fsf@HIDDEN> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33018 Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -6.0 (------) > From: "Basil L. Contovounesios" <contovob@HIDDEN> > Cc: <michael.albinus@HIDDEN>, <33018 <at> debbugs.gnu.org> > Date: Wed, 17 Oct 2018 21:46:39 +0100 > > > I think the relevant code should be instrumented to show which thread > > waits for what process(es). > > Each thread launches a single wget process which it then waits for > before dying, and current-thread is always eq to that thread around > calls to accept-process-output. Or are you talking about some other > type of thread? I was talking about low-level details: we set up the file-descriptor mask passed to pselect, to tell it which descriptors to wait on. The call to accept-process-output is supposed to arrange for the descriptor where the corresponding process will write to be one of those on which the corresponding pselect will wait. I was thinking that perhaps we become confused and don't ask pselect called by a thread to wait on the process which was launched by that thread. > Either way, I'll report back when I've had a deeper look into what Emacs > is doing, unless someone beats me to it. Thanks. > I'm sorry I've been talking more than doing, but university and house > hunting will dominate my free time for the next few weeks. No need to be sorry, we all have our lives.
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 17 Oct 2018 20:46:49 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 17 16:46:49 2018 Received: from localhost ([127.0.0.1]:55834 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gCsiP-0007CG-9s for submit <at> debbugs.gnu.org; Wed, 17 Oct 2018 16:46:49 -0400 Received: from mail-ed1-f41.google.com ([209.85.208.41]:39400) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <contovob@HIDDEN>) id 1gCsiN-0007C3-N5 for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 16:46:48 -0400 Received: by mail-ed1-f41.google.com with SMTP id d15-v6so26190752edq.6 for <33018 <at> debbugs.gnu.org>; Wed, 17 Oct 2018 13:46:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tcd-ie.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=0hILc1GM7YBstkA/IhmeqByE0F3D+HnFUecP9zJoSho=; b=xkRdFQc2bBrf83ODmqK9tIcNWPgtcDUFYytYzFZzzS0kd+nsC52wR8lTFzj35CNOkR lotcgWTlJ94779OEW7zEmjRw7k3p3tje28Hka1cNrPCACmIkGtuAUXIAzZVLGkChJZJI IbVGfrJ2j12njjA6KTRpsrL2bfYpW5J2MkjnGI4TW6RHdbo3UubEOH31VogewJd3DD3i aOWJwKPvEqMnB+YUOnvPf2FPxzQkNZLJq3kHRycNYAoANRLOU6IjjTCjFCTvHw1biHIy 3k7sPXFabhKs1ShAF9zRFd7NwgZ98J0qWW8uJM6SJGQ2g7KD0YMU6MU4+bN7AfT40wS9 Jlrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=0hILc1GM7YBstkA/IhmeqByE0F3D+HnFUecP9zJoSho=; b=KUmGP4c1SLWXCqyypEqNjT6ik7PYQQGqXRKJIqx8YmlBQvTVCf0+eybLBK+cvfdrB5 Qg1h/mK2zgCuBXdc6/iU8mFzZ03FFWzImWdLLKwB4/M/5x/US2ZnXxi2s1R7jvoZCqTZ GI+bschHU6W6SW0Gd+qRTGWZ1oe4YDYmA8FVJZ5JwsDwjFyIqujwc7Bu4P98G/KN5Mdv ox6NLv9fDTEkLXW6VrNfLqZPAohFUOFHLKRgdPMSZmnylVTZ7fjz1Mqw5lMtEp7HEFtB lBK3mG2JhGdMmxP7JvQAGPGPM/UG2NmcOiUBwq6qk4ZQDqfX5GzFRgXCOhoNJkNsZ398 zxcg== X-Gm-Message-State: ABuFfohG1JAkmLgYr0B5YA5GPgf1kUwHJDreeUIIB+/CIlh8TpvPsoRp ziPaPXJjlHmFS14L20qonPm85A== X-Google-Smtp-Source: ACcGV63vcd9L06Wv3CajHnLQ5BA7xBUBiO0t2T677CvFr8ZO3PJdDrk+jPq7kNXi7Q6QT4BpDVP1YQ== X-Received: by 2002:a17:906:88f:: with SMTP id n15-v6mr27897968eje.119.1539809201734; Wed, 17 Oct 2018 13:46:41 -0700 (PDT) Received: from localhost (51-171-243-2-dynamic.agg2.clk.blp-srl.eircom.net. [51.171.243.2]) by smtp.gmail.com with ESMTPSA id z2-v6sm8007539edz.31.2018.10.17.13.46.40 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 17 Oct 2018 13:46:41 -0700 (PDT) From: "Basil L. Contovounesios" <contovob@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN> <878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN> <877eiipu8g.fsf@HIDDEN> <83k1mirl9n.fsf@HIDDEN> <874ldkjwzn.fsf@HIDDEN> <83zhvcqwxc.fsf@HIDDEN> <87o9bswirc.fsf@HIDDEN> <83y3awqvtl.fsf@HIDDEN> <83woqgqvl2.fsf@HIDDEN> Date: Wed, 17 Oct 2018 21:46:39 +0100 In-Reply-To: <83woqgqvl2.fsf@HIDDEN> (Eli Zaretskii's message of "Wed, 17 Oct 2018 21:25:29 +0300") Message-ID: <87zhvc2te8.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33018 Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> Date: Wed, 17 Oct 2018 21:20:22 +0300 >> From: Eli Zaretskii <eliz@HIDDEN> >> Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org >> >> > From: "Basil L. Contovounesios" <contovob@HIDDEN> >> > Cc: <michael.albinus@HIDDEN>, <33018 <at> debbugs.gnu.org> >> > Date: Wed, 17 Oct 2018 19:05:59 +0100 >> > >> > > Looks like the buffer of the pipe through which Emacs reads the stuff >> > > is full, and wget waits for some space there? >> > >> > Would that imply that different threads/processes are (re)using the same >> > buffer/pipe? >> >> Could be, but it's more likely that Emacs simply doesn't read the >> output from wget. > > I think the relevant code should be instrumented to show which thread > waits for what process(es). Each thread launches a single wget process which it then waits for before dying, and current-thread is always eq to that thread around calls to accept-process-output. Or are you talking about some other type of thread? Either way, I'll report back when I've had a deeper look into what Emacs is doing, unless someone beats me to it. > Btw, are you sure this is not a bug in your program? No, but the fact that it reliably works when Emacs is run with -batch, and reliably hangs when run with -Q is at least somewhat intriguing. > Michael caused your program to work twice by simple changes, AFAIU, > no? Michael avoided the hang by rewriting the create-join-...-create-join sequence as create-create-...-join-join. But if the latter is done twice from within the same master thread, the hang still occurs. As I said: > There's something about going through a complete create-join cycle more > than once within a non-main-thread which is triggering this behaviour. (Actually, I haven't checked whether the hang occurs when two create-join cycles are completed within main-thread; I was just specifically describing my sample program.) In his second rewrite, Michael replaced wget with echo, which does not suffer from any hangs. As I said: > Indeed, there are many subprocess-within-a-thread examples which don't > suffer from a hang, e.g. by using a different URL. I would like to get > to the bottom of why network programs like wget/curl in particular > eventually hang, though. In other words, I don't (yet) see why my recipe shouldn't work, and I'm curious to eventually get to the bottom of this. I'm sorry I've been talking more than doing, but university and house hunting will dominate my free time for the next few weeks. Thanks, -- Basil
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at 33018) by debbugs.gnu.org; 17 Oct 2018 20:03:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 17 16:03:10 2018
Received: from localhost ([127.0.0.1]:55697 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gCs29-0005uW-U7
for submit <at> debbugs.gnu.org; Wed, 17 Oct 2018 16:03:10 -0400
Received: from mail-ed1-f52.google.com ([209.85.208.52]:42515)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <contovob@HIDDEN>) id 1gCs27-0005uH-Sp
for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 16:03:08 -0400
Received: by mail-ed1-f52.google.com with SMTP id b7-v6so26055508edd.9
for <33018 <at> debbugs.gnu.org>; Wed, 17 Oct 2018 13:03:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=tcd-ie.20150623.gappssmtp.com; s=20150623;
h=from:to:cc:subject:references:date:in-reply-to:message-id
:user-agent:mime-version;
bh=1ILYLc6+Z7/cv1fOEdQHV//aZQTMPKxjLj2H3fhVWgg=;
b=g+95zDl6fulcSZiFpNyT0w8OVBAPOzTW08K+uJ5iyuvqhRTsJ0kQuJ+kDiBoxD3T0E
VoddNgQSgx8Tiu6nvY5kfhvS+ohl/rYfDaASQjKjWKKMVxJ4cD2UG4iKq1Fgd216m3U9
qg8TPH7YgWG/AfeeY7oGt3l745ujYJFfIwgOlPSr40OzJApIm360aKChERwMrvGXFOYd
tmEiQg6bhE+xKOxGRSLpF7uHGkPI/WltptMKDGgYUJzvePhNUmTxzsubXKhLb2WGW3Pj
JmrUtOMmXzH42r8VVJp+n9kyrKqYBGGMRN2qzK6Q78PR+sSm/8MfD1LIHVZy7QI8muq2
a+kg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
:message-id:user-agent:mime-version;
bh=1ILYLc6+Z7/cv1fOEdQHV//aZQTMPKxjLj2H3fhVWgg=;
b=TPAeXSC3ynu6/mtnswa/dnuw5SweRtdXhz3t4wbnk8x6K0SfWyc6AOh4ilq5XEzm1x
20nYu7Yncvvw3tjgXmW58MJgcC+meLe1vbInWUD/Fxw6OEfxdbIePcW5uz5f/mvSL2y0
SiG1LtoTU2nWJmFF+ui0SUN+Z9AmnobYluAoKANamSD7qvEmtYFrSyAghUjRQvbjO7rL
NDqk2/PgTdCO7VMPyC4vg0d6IeSRTlqzuvqZ8MzfP7hD7EbusdHzXfzhIdhvppj5sltn
lRQO9e+cOuZrbLttgTQr4W2Cr7pWhLmMHm47/AT61FAdxS9oEL15MxvIZKhEt80LIgMn
1thw==
X-Gm-Message-State: ABuFfogxYmOYXnzppVEvmNWz5RGKlsnHXtt9Z+Deo4NpHJiti9bYhT20
Fo+pzdnV845CzMvsQNm0vBfG2w==
X-Google-Smtp-Source: ACcGV630M40bM241a9yaHHzyL6kGvsUz4Xu1PQLqL2Tg0c8yWswBEJKgqniZLX6RD9zlRIO/b7ooyg==
X-Received: by 2002:a50:a0e2:: with SMTP id 89-v6mr1508380edo.95.1539806582200;
Wed, 17 Oct 2018 13:03:02 -0700 (PDT)
Received: from localhost (51-171-243-2-dynamic.agg2.clk.blp-srl.eircom.net.
[51.171.243.2])
by smtp.gmail.com with ESMTPSA id q3-v6sm3844747ejj.50.2018.10.17.13.03.00
(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
Wed, 17 Oct 2018 13:03:01 -0700 (PDT)
From: "Basil L. Contovounesios" <contovob@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
<878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN>
<877eiipu8g.fsf@HIDDEN> <83k1mirl9n.fsf@HIDDEN>
<874ldkjwzn.fsf@HIDDEN> <83zhvcqwxc.fsf@HIDDEN>
<87o9bswirc.fsf@HIDDEN> <83y3awqvtl.fsf@HIDDEN>
Date: Wed, 17 Oct 2018 21:02:59 +0100
In-Reply-To: <83y3awqvtl.fsf@HIDDEN> (Eli Zaretskii's message of "Wed, 17 Oct
2018 21:20:22 +0300")
Message-ID: <87k1mg49zg.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 33018
Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: "Basil L. Contovounesios" <contovob@HIDDEN>
>> Cc: <michael.albinus@HIDDEN>, <33018 <at> debbugs.gnu.org>
>> Date: Wed, 17 Oct 2018 19:05:59 +0100
>>
>> pselect6(14, [6 7], [], NULL, {tv_sec=99975, tv_nsec=320947003}, {NULL, 8}
>>
>> and strace -p <pip of stuck wget> gives:
>>
>> write(1, ">]</span></span></h2>\n<p>Emacs i"..., 4096
>>
>> The former reminded me of bug#24201: https://debbugs.gnu.org/24201
>
> I could be wrong, but it doesn't look similar to me. In that bug, the
> CPU was pegged, whereas you said that CPU is idle.
Right, I was only reminded of that bug because of the common
accept-process-output/pselect hang in the context of a network-related
process. I'm not suggesting the underlying cause is the same.
--
Basil
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 17 Oct 2018 18:25:56 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 17 14:25:56 2018 Received: from localhost ([127.0.0.1]:55610 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gCqW4-0001DB-D0 for submit <at> debbugs.gnu.org; Wed, 17 Oct 2018 14:25:56 -0400 Received: from eggs.gnu.org ([208.118.235.92]:39944) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1gCqW2-0001Cu-Oz for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 14:25:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gCqVt-0001PI-5b for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 14:25:49 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:43140) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gCqVs-0001Og-Vs; Wed, 17 Oct 2018 14:25:45 -0400 Received: from [176.228.60.248] (port=1716 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <eliz@HIDDEN>) id 1gCqVs-0003w1-0w; Wed, 17 Oct 2018 14:25:44 -0400 Date: Wed, 17 Oct 2018 21:25:29 +0300 Message-Id: <83woqgqvl2.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: contovob@HIDDEN In-reply-to: <83y3awqvtl.fsf@HIDDEN> (message from Eli Zaretskii on Wed, 17 Oct 2018 21:20:22 +0300) Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN> <878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN> <877eiipu8g.fsf@HIDDEN> <83k1mirl9n.fsf@HIDDEN> <874ldkjwzn.fsf@HIDDEN> <83zhvcqwxc.fsf@HIDDEN> <87o9bswirc.fsf@HIDDEN> <83y3awqvtl.fsf@HIDDEN> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33018 Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -6.0 (------) > Date: Wed, 17 Oct 2018 21:20:22 +0300 > From: Eli Zaretskii <eliz@HIDDEN> > Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org > > > From: "Basil L. Contovounesios" <contovob@HIDDEN> > > Cc: <michael.albinus@HIDDEN>, <33018 <at> debbugs.gnu.org> > > Date: Wed, 17 Oct 2018 19:05:59 +0100 > > > > > Looks like the buffer of the pipe through which Emacs reads the stuff > > > is full, and wget waits for some space there? > > > > Would that imply that different threads/processes are (re)using the same > > buffer/pipe? > > Could be, but it's more likely that Emacs simply doesn't read the > output from wget. I think the relevant code should be instrumented to show which thread waits for what process(es). Btw, are you sure this is not a bug in your program? Michael caused your program to work twice by simple changes, AFAIU, no?
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at 33018) by debbugs.gnu.org; 17 Oct 2018 18:20:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 17 14:20:43 2018
Received: from localhost ([127.0.0.1]:55606 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gCqR0-00014M-MT
for submit <at> debbugs.gnu.org; Wed, 17 Oct 2018 14:20:42 -0400
Received: from eggs.gnu.org ([208.118.235.92]:38169)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1gCqQz-00014A-IP
for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 14:20:41 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <eliz@HIDDEN>) id 1gCqQr-00039H-C4
for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 14:20:36 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level:
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled
version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42987)
by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
id 1gCqQq-00037n-NI; Wed, 17 Oct 2018 14:20:33 -0400
Received: from [176.228.60.248] (port=1403 helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <eliz@HIDDEN>)
id 1gCqQq-0003TQ-5I; Wed, 17 Oct 2018 14:20:32 -0400
Date: Wed, 17 Oct 2018 21:20:22 +0300
Message-Id: <83y3awqvtl.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: "Basil L. Contovounesios" <contovob@HIDDEN>
In-reply-to: <87o9bswirc.fsf@HIDDEN> (contovob@HIDDEN)
Subject: Re: bug#33018: 26.1.50;
thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
<878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN>
<877eiipu8g.fsf@HIDDEN> <83k1mirl9n.fsf@HIDDEN>
<874ldkjwzn.fsf@HIDDEN> <83zhvcqwxc.fsf@HIDDEN> <87o9bswirc.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 33018
Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -6.0 (------)
> From: "Basil L. Contovounesios" <contovob@HIDDEN>
> Cc: <michael.albinus@HIDDEN>, <33018 <at> debbugs.gnu.org>
> Date: Wed, 17 Oct 2018 19:05:59 +0100
>
> > Looks like the buffer of the pipe through which Emacs reads the stuff
> > is full, and wget waits for some space there?
>
> Would that imply that different threads/processes are (re)using the same
> buffer/pipe?
Could be, but it's more likely that Emacs simply doesn't read the
output from wget.
> pselect6(14, [6 7], [], NULL, {tv_sec=99975, tv_nsec=320947003}, {NULL, 8}
>
> and strace -p <pip of stuck wget> gives:
>
> write(1, ">]</span></span></h2>\n<p>Emacs i"..., 4096
>
> The former reminded me of bug#24201: https://debbugs.gnu.org/24201
I could be wrong, but it doesn't look similar to me. In that bug, the
CPU was pegged, whereas you said that CPU is idle.
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at 33018) by debbugs.gnu.org; 17 Oct 2018 18:06:08 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 17 14:06:08 2018
Received: from localhost ([127.0.0.1]:55599 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gCqCu-0000eh-7n
for submit <at> debbugs.gnu.org; Wed, 17 Oct 2018 14:06:08 -0400
Received: from mail-ed1-f45.google.com ([209.85.208.45]:40057)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <contovob@HIDDEN>) id 1gCqCt-0000eI-5B
for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 14:06:07 -0400
Received: by mail-ed1-f45.google.com with SMTP id r1-v6so25767875edd.7
for <33018 <at> debbugs.gnu.org>; Wed, 17 Oct 2018 11:06:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=tcd-ie.20150623.gappssmtp.com; s=20150623;
h=from:to:cc:subject:references:date:in-reply-to:message-id
:user-agent:mime-version;
bh=cdjYCVYWXprgtT1Yt2FyTzUtqTPAFHOriGREOju9mfI=;
b=H/H/pPwYxZ+uOVUnb6K6C7UsyAQCIP0QmEMfWtg3AV4sUC/uLqPnGSlREXxg3xKXHn
x1HS740wbjYz1Cya5XVUHlPYk91KyOaclk2Ab0ZIVFLnVYDZYzZmmeBm7TpW0T9hDSz5
j/90yg4g7HhlCuw7IRTarz09ik8sugAWdFwNJfIUPbzbO/SP6fXEOEdJX+qH9CAyZJCn
hFTED0Vpqw21STBw2Cdpn3Z5TqiRwTs4esXbcg9KB8sgdrZVygk+VMR1+MOaCJr7++lG
kRHOpUVQuaG3kzQ7aILskBznL28FdouKoVq3U2qhG722ZLeZbe2kCrdhtspWNdE++Bl+
MXYg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
:message-id:user-agent:mime-version;
bh=cdjYCVYWXprgtT1Yt2FyTzUtqTPAFHOriGREOju9mfI=;
b=BUYi0oR9aa6fgN6Yz6TFFZ6p27PrhAQWfdUOAmto21bhpD9EiftteuloIjDRDmZ9Ub
aTWSe6393VuFBChU+ptS6ms8XXiziLW9IdVvFCH3px307+vIKY7ii3z77s5ewp/X6cAG
aZcRAfUFEvtOxAerl5Ow06u34i1YTGLZtoB7mOEk10U4l0BSaDb3h6zWZMznWbXXbnqD
4YJzB/UWGUf98p1HdZWuyU7YnZoafNgpsZZD3BZWXldITaEEW0POQc6J4CNNnCIPyztn
0jkhKhkbmiOx/xU6YSDxtLHPMv9L116KT4sGTLLhy34dE3C39EK2E61iQItawqrelVCD
jrkw==
X-Gm-Message-State: ABuFfohQwHKQe9S/OWbnhuMZhboWL16TyNCVMozeVNQZNZTBvPXVCsDn
uXIa0mShMTXA8hSmbJ+Tlm+l2w==
X-Google-Smtp-Source: ACcGV601p46nbghJc+SQtDHbtE8isKuuIm34H0TgfSS0GWrAOKEsr+ggNsvaLJjZ+yhrBk+nvVGaTw==
X-Received: by 2002:a50:b8a5:: with SMTP id
l34-v6mr1284036ede.273.1539799561495;
Wed, 17 Oct 2018 11:06:01 -0700 (PDT)
Received: from localhost ([213.233.155.132])
by smtp.gmail.com with ESMTPSA id e21-v6sm7969509edb.22.2018.10.17.11.06.00
(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
Wed, 17 Oct 2018 11:06:00 -0700 (PDT)
From: "Basil L. Contovounesios" <contovob@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
<878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN>
<877eiipu8g.fsf@HIDDEN> <83k1mirl9n.fsf@HIDDEN>
<874ldkjwzn.fsf@HIDDEN> <83zhvcqwxc.fsf@HIDDEN>
Date: Wed, 17 Oct 2018 19:05:59 +0100
In-Reply-To: <83zhvcqwxc.fsf@HIDDEN> (Eli Zaretskii's message of "Wed, 17 Oct
2018 20:56:31 +0300")
Message-ID: <87o9bswirc.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 33018
Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: "Basil L. Contovounesios" <contovob@HIDDEN>
>> Cc: <michael.albinus@HIDDEN>, <33018 <at> debbugs.gnu.org>
>> Date: Wed, 17 Oct 2018 18:37:00 +0100
>>
>> #0 0x00007ffff766a2a4 in __GI___libc_write (fd=1, buf=0x5555559c1b30, nbytes=4096)
>> at ../sysdeps/unix/sysv/linux/write.c:27
>> #1 0x00007ffff75fb56d in _IO_new_file_write (f=0x7ffff7739760 <_IO_2_1_stdout_>,
>> data=0x5555559c1b30, n=4096) at fileops.c:1203
>> #2 0x00007ffff75fa88f in new_do_write (fp=0x7ffff7739760 <_IO_2_1_stdout_>,
>> data=0x5555559c1b30 "pan></span></h2>\n<p>Emacs is primarily a <a
>> href=\"/wiki/Text_editor\" title=\"Text editor\">text editor</a> and is
>> designed for manipulating pieces of text, although it is capable of formatting
>> and print"..., to_do=to_do@entry=4096) at fileops.c:457
>> #3 0x00007ffff75fc6f9 in _IO_new_do_write (fp=<optimized out>, data=<optimized out>,
>> to_do=4096) at fileops.c:433
>> #4 0x00007ffff75fa6d8 in _IO_new_file_sync (fp=0x7ffff7739760 <_IO_2_1_stdout_>)
>> at fileops.c:813
>> #5 0x00007ffff75ef6ed in __GI__IO_fflush (fp=0x7ffff7739760 <_IO_2_1_stdout_>)
>> at iofflush.c:40
>> #6 0x00005555555918bb in write_data (out=0x7ffff7739760 <_IO_2_1_stdout_>, out2=0x0,
>> buf=0x5555559a9590 "pan></span></h2>\n<p>Emacs is primarily a <a
>> href=\"/wiki/Text_editor\" title=\"Text editor\">text editor</a> and is
>> designed for manipulating pieces of text, although it is capable of formatting
>> and print"..., bufsize=4096, skip=0x7fffffffd0e8,
>> written=0x7fffffffd0e0) at retr.c:207
>> #7 0x000055555559204f in fd_read_body (downloaded_filename=0x5555555e18a0 "-", fd=4,
>> out=0x7ffff7739760 <_IO_2_1_stdout_>, toread=198224, startpos=0, qtyread=0x7fffffffda20,
>> qtywritten=0x7fffffffd9d0, elapsed=0x7fffffffda28, flags=1, out2=0x0) at retr.c:498
>
> Looks like the buffer of the pipe through which Emacs reads the stuff
> is full, and wget waits for some space there?
Would that imply that different threads/processes are (re)using the same
buffer/pipe?
FWIW, strace -p <pip of stuck emacs> gives:
pselect6(14, [6 7], [], NULL, {tv_sec=99975, tv_nsec=320947003}, {NULL, 8}
and strace -p <pip of stuck wget> gives:
write(1, ">]</span></span></h2>\n<p>Emacs i"..., 4096
The former reminded me of bug#24201: https://debbugs.gnu.org/24201
--
Basil
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 17 Oct 2018 17:56:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 17 13:56:53 2018 Received: from localhost ([127.0.0.1]:55595 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gCq3x-0000Oh-8n for submit <at> debbugs.gnu.org; Wed, 17 Oct 2018 13:56:53 -0400 Received: from eggs.gnu.org ([208.118.235.92]:57664) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1gCq3v-0000OT-JM for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 13:56:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gCq3m-0003hz-28 for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 13:56:46 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:42408) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gCq3k-0003hB-56; Wed, 17 Oct 2018 13:56:40 -0400 Received: from [176.228.60.248] (port=3903 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <eliz@HIDDEN>) id 1gCq3j-0001oO-Lc; Wed, 17 Oct 2018 13:56:40 -0400 Date: Wed, 17 Oct 2018 20:56:31 +0300 Message-Id: <83zhvcqwxc.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: "Basil L. Contovounesios" <contovob@HIDDEN> In-reply-to: <874ldkjwzn.fsf@HIDDEN> (contovob@HIDDEN) Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN> <878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN> <877eiipu8g.fsf@HIDDEN> <83k1mirl9n.fsf@HIDDEN> <874ldkjwzn.fsf@HIDDEN> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33018 Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -6.0 (------) > From: "Basil L. Contovounesios" <contovob@HIDDEN> > Cc: <michael.albinus@HIDDEN>, <33018 <at> debbugs.gnu.org> > Date: Wed, 17 Oct 2018 18:37:00 +0100 > > #0 0x00007ffff766a2a4 in __GI___libc_write (fd=1, buf=0x5555559c1b30, nbytes=4096) > at ../sysdeps/unix/sysv/linux/write.c:27 > #1 0x00007ffff75fb56d in _IO_new_file_write (f=0x7ffff7739760 <_IO_2_1_stdout_>, > data=0x5555559c1b30, n=4096) at fileops.c:1203 > #2 0x00007ffff75fa88f in new_do_write (fp=0x7ffff7739760 <_IO_2_1_stdout_>, > data=0x5555559c1b30 "pan></span></h2>\n<p>Emacs is primarily a <a href=\"/wiki/Text_editor\" title=\"Text editor\">text editor</a> and is designed for manipulating pieces of text, although it is capable of formatting and print"..., to_do=to_do@entry=4096) at fileops.c:457 > #3 0x00007ffff75fc6f9 in _IO_new_do_write (fp=<optimized out>, data=<optimized out>, > to_do=4096) at fileops.c:433 > #4 0x00007ffff75fa6d8 in _IO_new_file_sync (fp=0x7ffff7739760 <_IO_2_1_stdout_>) > at fileops.c:813 > #5 0x00007ffff75ef6ed in __GI__IO_fflush (fp=0x7ffff7739760 <_IO_2_1_stdout_>) at iofflush.c:40 > #6 0x00005555555918bb in write_data (out=0x7ffff7739760 <_IO_2_1_stdout_>, out2=0x0, > buf=0x5555559a9590 "pan></span></h2>\n<p>Emacs is primarily a <a href=\"/wiki/Text_editor\" title=\"Text editor\">text editor</a> and is designed for manipulating pieces of text, although it is capable of formatting and print"..., bufsize=4096, skip=0x7fffffffd0e8, > written=0x7fffffffd0e0) at retr.c:207 > #7 0x000055555559204f in fd_read_body (downloaded_filename=0x5555555e18a0 "-", fd=4, > out=0x7ffff7739760 <_IO_2_1_stdout_>, toread=198224, startpos=0, qtyread=0x7fffffffda20, > qtywritten=0x7fffffffd9d0, elapsed=0x7fffffffda28, flags=1, out2=0x0) at retr.c:498 Looks like the buffer of the pipe through which Emacs reads the stuff is full, and wget waits for some space there?
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at 33018) by debbugs.gnu.org; 17 Oct 2018 17:37:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 17 13:37:16 2018
Received: from localhost ([127.0.0.1]:55567 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gCpkx-0008KF-SW
for submit <at> debbugs.gnu.org; Wed, 17 Oct 2018 13:37:16 -0400
Received: from mail-ed1-f67.google.com ([209.85.208.67]:37725)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <contovob@HIDDEN>) id 1gCpkv-0008Jz-Lx
for 33018 <at> debbugs.gnu.org; Wed, 17 Oct 2018 13:37:14 -0400
Received: by mail-ed1-f67.google.com with SMTP id c22-v6so25681938edc.4
for <33018 <at> debbugs.gnu.org>; Wed, 17 Oct 2018 10:37:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=tcd-ie.20150623.gappssmtp.com; s=20150623;
h=from:to:cc:subject:references:date:in-reply-to:message-id
:user-agent:mime-version;
bh=UN44GWjQG0etD1hhH28egGDclowuoEeWETCHPiFMuas=;
b=Q+W/p5NKRiyM07XoUaWlbHJSlc6xI2gsZGIbNV28ivXY+k0mZncPC9a+4hVmKhbgWy
AzhrNsGLZU3Vel8s4V6POBqWK4tc9I0Lb8dtqq44PNhkzVC+N0tDLClxuJL8JSxhvv3c
b9aLXbXVybUBp1JbTNJJzQAJQlO3u42nlyXtcLNFmJLSuTQeAqUpgQxODG6hiaAqI46Q
Cahb+h/I4eCyi+sro0qcKQR5+9H3Ao8u6teJgph/DyJ1+uaOeUCreaAv3ODemzipV6j6
xkZRbtuxMFG7qAwdEm0aO+lu+NjOBrSKH/K6zuS6AOuEMM1ns7e+Z1Ft578ub/kErom5
NaAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
:message-id:user-agent:mime-version;
bh=UN44GWjQG0etD1hhH28egGDclowuoEeWETCHPiFMuas=;
b=SY/fDy6HcRq5I6Spk78DSyff8XbqIUWGZPChyZ0Ptf2FXJAJmyXUxENCcozsgnV7lP
H4WKzCDqMYAADcLm/MJjuAuOkglHIaNAc/1qojC3gpuuMgbOn7dP3A/ViR6FoxRvX/cx
GUeRvT8D2OgVAh7bNkA4FxUYzgPNQWxP7aby3e50EnMsJSI+VlZcersigQLVVYX7STGi
Imp4wWbpo9ZA+sm00wr6A3nr4k5aI0v3/S0H/F2aybdN0n+N1052CRUoWKzTM0gNxoYS
4vpGbvwLrCEXwBxb3xGbkZfL9aJEewjFV0JyJ0QbfqMrp5L02klX7vVjiJKaJ+Vov23o
EimQ==
X-Gm-Message-State: ABuFfojxxqAjBzVqArEaE+Awa3zKnoUiJmllCBZ+D7v3bcVpL6dfcd/X
97Qss+hoSuCPzEWJOHC+5wdGTA==
X-Google-Smtp-Source: ACcGV62bMpHUc6gpXDjKQ/PSXW0K+9aGxAhvuCF9p9SM9LYnt+7kyI++LjdROimYDYwfEfesIAN3HA==
X-Received: by 2002:a17:906:3181:: with SMTP id
1-v6mr27152839ejy.80.1539797827591;
Wed, 17 Oct 2018 10:37:07 -0700 (PDT)
Received: from localhost ([213.233.148.7])
by smtp.gmail.com with ESMTPSA id g48-v6sm7146967edc.93.2018.10.17.10.37.05
(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
Wed, 17 Oct 2018 10:37:06 -0700 (PDT)
From: "Basil L. Contovounesios" <contovob@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
<878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN>
<877eiipu8g.fsf@HIDDEN> <83k1mirl9n.fsf@HIDDEN>
Date: Wed, 17 Oct 2018 18:37:00 +0100
In-Reply-To: <83k1mirl9n.fsf@HIDDEN> (Eli Zaretskii's message of "Tue, 16 Oct
2018 17:58:28 +0300")
Message-ID: <874ldkjwzn.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 33018
Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain
Content-Disposition: attachment; filename=gdb.txt
#0 0x00007ffff766a2a4 in __GI___libc_write (fd=1, buf=0x5555559c1b30, nbytes=4096)
at ../sysdeps/unix/sysv/linux/write.c:27
#1 0x00007ffff75fb56d in _IO_new_file_write (f=0x7ffff7739760 <_IO_2_1_stdout_>,
data=0x5555559c1b30, n=4096) at fileops.c:1203
#2 0x00007ffff75fa88f in new_do_write (fp=0x7ffff7739760 <_IO_2_1_stdout_>,
data=0x5555559c1b30 "pan></span></h2>\n<p>Emacs is primarily a <a href=\"/wiki/Text_editor\" title=\"Text editor\">text editor</a> and is designed for manipulating pieces of text, although it is capable of formatting and print"..., to_do=to_do@entry=4096) at fileops.c:457
#3 0x00007ffff75fc6f9 in _IO_new_do_write (fp=<optimized out>, data=<optimized out>,
to_do=4096) at fileops.c:433
#4 0x00007ffff75fa6d8 in _IO_new_file_sync (fp=0x7ffff7739760 <_IO_2_1_stdout_>)
at fileops.c:813
#5 0x00007ffff75ef6ed in __GI__IO_fflush (fp=0x7ffff7739760 <_IO_2_1_stdout_>) at iofflush.c:40
#6 0x00005555555918bb in write_data (out=0x7ffff7739760 <_IO_2_1_stdout_>, out2=0x0,
buf=0x5555559a9590 "pan></span></h2>\n<p>Emacs is primarily a <a href=\"/wiki/Text_editor\" title=\"Text editor\">text editor</a> and is designed for manipulating pieces of text, although it is capable of formatting and print"..., bufsize=4096, skip=0x7fffffffd0e8,
written=0x7fffffffd0e0) at retr.c:207
#7 0x000055555559204f in fd_read_body (downloaded_filename=0x5555555e18a0 "-", fd=4,
out=0x7ffff7739760 <_IO_2_1_stdout_>, toread=198224, startpos=0, qtyread=0x7fffffffda20,
qtywritten=0x7fffffffd9d0, elapsed=0x7fffffffda28, flags=1, out2=0x0) at retr.c:498
#8 0x000055555557d09d in read_response_body (hs=0x7fffffffd9d0, sock=4,
fp=0x7ffff7739760 <_IO_2_1_stdout_>, contlen=198224, contrange=0,
chunked_transfer_encoding=false, url=0x5555555ff920 "https://en.wikipedia.org/wiki/Emacs",
warc_timestamp_str=0x7fffffffd470 "\f\335\377\377\377\177",
warc_request_uuid=0x7fffffffd440 ",", warc_ip=0x0, type=0x5555559c8fc0 "text/html",
statcode=200,
head=0x5555559b35d0 "HTTP/1.1 200 OK\r\nDate: Wed, 17 Oct 2018 17:29:35 GMT\r\nContent-Type: text/html; charset=UTF-8\r\nContent-Length: 198224\r\nConnection: keep-alive\r\nServer: mw1325.eqiad.wmnet\r\nVary: Accept-Encoding,Cookie,A"...) at http.c:1717
#9 0x0000555555581b06 in gethttp (u=0x5555555ff8b0, original_url=0x5555555ff8b0,
hs=0x7fffffffd9d0, dt=0x7fffffffdd0c, proxy=0x0, iri=0x5555555ff880, count=1) at http.c:4129
#10 0x0000555555582239 in http_loop (u=0x5555555ff8b0, original_url=0x5555555ff8b0,
newloc=0x7fffffffdb68, local_file=0x7fffffffdb58, referer=0x0, dt=0x7fffffffdd0c,
proxy=0x0, iri=0x5555555ff880) at http.c:4363
#11 0x0000555555592ca4 in retrieve_url (orig_parsed=0x5555555ff8b0,
origurl=0x5555555ff490 "https://en.wikipedia.org/wiki/Emacs", file=0x7fffffffdd18,
newloc=0x7fffffffdd10, refurl=0x0, dt=0x7fffffffdd0c, recursive=false, iri=0x5555555ff880,
register_status=true) at retr.c:957
#12 0x000055555558b33e in main (argc=3, argv=0x7fffffffdef8) at main.c:2158
--=-=-=
Content-Type: text/plain
Eli Zaretskii <eliz@HIDDEN> writes:
> Can you attach a debugger to the wget process that's stuck, and see
> where it is stuck? You will probably need to rebuild wget with debug
> info, or install one from your package repository (if they offer
> such). This could give us hints for where to look further.
I did the following:
0. Build wget 1.19.5 from Debian's repositories:
apt-get build-dep wget
apt-get source wget
configure CC='ccache gcc' CFLAGS='-O0 -g3 -ggdb -gdwarf-4 -pipe'
--config-cache --enable-assert --with-gnu-ld
make
1. Substitute resulting wget file name in original test.el program
2. emacs26 -Q -l test.el -f test-threads
3. gdb -p <pid of stuck wget>
4. set logging on
5. bt
I attach the resulting gdb.txt log file.
Thanks,
--
Basil
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 16 Oct 2018 14:58:29 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 16 10:58:29 2018 Received: from localhost ([127.0.0.1]:53812 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gCQnk-0007IC-VQ for submit <at> debbugs.gnu.org; Tue, 16 Oct 2018 10:58:29 -0400 Received: from eggs.gnu.org ([208.118.235.92]:52931) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1gCQnj-0007I0-GR for 33018 <at> debbugs.gnu.org; Tue, 16 Oct 2018 10:58:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gCQnZ-0000Ms-Ky for 33018 <at> debbugs.gnu.org; Tue, 16 Oct 2018 10:58:20 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41716) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gCQnZ-0000MA-FI; Tue, 16 Oct 2018 10:58:17 -0400 Received: from [176.228.60.248] (port=2614 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <eliz@HIDDEN>) id 1gCQnY-0002ti-Al; Tue, 16 Oct 2018 10:58:17 -0400 Date: Tue, 16 Oct 2018 17:58:28 +0300 Message-Id: <83k1mirl9n.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: "Basil L. Contovounesios" <contovob@HIDDEN> In-reply-to: <877eiipu8g.fsf@HIDDEN> (contovob@HIDDEN) Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN> <878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN> <877eiipu8g.fsf@HIDDEN> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33018 Cc: michael.albinus@HIDDEN, 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -6.0 (------) > From: "Basil L. Contovounesios" <contovob@HIDDEN> > Date: Tue, 16 Oct 2018 02:15:27 +0100 > Cc: 33018 <at> debbugs.gnu.org > > Thanks, creating all threads before waiting for any of them to exit > indeed does not suffer from the same hang. Doing this twice (see > attached update), however, still hangs. > > There's something about going through a complete create-join cycle more > than once within a non-main-thread which is triggering this behaviour. Can you attach a debugger to the wget process that's stuck, and see where it is stuck? You will probably need to rebuild wget with debug info, or install one from your package repository (if they offer such). This could give us hints for where to look further. Thanks.
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 16 Oct 2018 14:55:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 16 10:55:42 2018 Received: from localhost ([127.0.0.1]:53808 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gCQl4-0007EK-Ft for submit <at> debbugs.gnu.org; Tue, 16 Oct 2018 10:55:42 -0400 Received: from mail-ed1-f45.google.com ([209.85.208.45]:33108) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <contovob@HIDDEN>) id 1gCQl3-0007E8-07 for 33018 <at> debbugs.gnu.org; Tue, 16 Oct 2018 10:55:41 -0400 Received: by mail-ed1-f45.google.com with SMTP id l14-v6so12653954edq.0 for <33018 <at> debbugs.gnu.org>; Tue, 16 Oct 2018 07:55:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tcd-ie.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=di/Z/+3iqHDOVjBzasbgFTcF610xoUOm9/YVhFlCk8o=; b=rWRKD5+gTQ8e4VNUb8YMYwZ1MIsqpV2HrfDvc2kfTFvxXzTPCEEXvaiBo2YndF4jrq zA6Op7KCOQ4xnnP94zQ4UmhZloWq4J89nQXqbMUqOiJzLnNs0pX4lNi7BOuNVvg/bkVU pDpUZN/I/FthjTRV4dLxRYxYEBuBMB+HgwqdYzU4TREUgck9xP4tqUfPoibA0l6YXTcO 3FCdZBvduwvHaafD74qWxLCHGzcoqtK9gXOlIm4QZHK1R+dAb0rxw0dVE8ICiZn4vgoM 2H01Hoz264qDzeup5rQ+0D4/aTcT0W8txWuzTlQZfLiCCvOeJXsm4ur7XhfDN1W1SEWM 2+LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=di/Z/+3iqHDOVjBzasbgFTcF610xoUOm9/YVhFlCk8o=; b=rjGuHemkkvJRlIDvxB0tpFomDM7VGVJLGpg85ounb8x+MErxSTGiUP67uhXnyGSwB0 bXmZ2m0558WSfrDCyqu24Dt6klEV1W1Aa5oItttm6hvBsPdNmXRkLG4R40NFDbO0zXAg iKp0NubD0L+Fc26XYUAtfC5VB2YgE1wn1a92FHlRrz1Ipr/hn63FvZ+uEEfAEIIU72IZ W/23JQCWyZqkgH4/1jLVBm5XWL6W4pBa20R2IMRhTSKcdf4nv8nl8VfMwWFax6eHNZnz 9RJxk1jBBVbetK82SiyDYXVtBbib9EXArrwyyeloSQQf9OIXpSusMrWLgczmZ4wW3TpJ eK3w== X-Gm-Message-State: ABuFfojdadCewkyZg40QbF5t14dtOmiuVq5a0DFBoJqp0wwIsDYN7AGJ t16ID1cFNl3+eBrxTMZf6NkQSQ== X-Google-Smtp-Source: ACcGV632D2UgZzienNHy7N8oTWChLPw4ZZIAB6RNlBvwB39k9CK8N74XomJSLb0pojg7L1rdHdWzVQ== X-Received: by 2002:a17:906:4e03:: with SMTP id z3-v6mr23263480eju.187.1539701734893; Tue, 16 Oct 2018 07:55:34 -0700 (PDT) Received: from localhost ([213.233.148.5]) by smtp.gmail.com with ESMTPSA id x15-v6sm6474371edm.26.2018.10.16.07.55.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 16 Oct 2018 07:55:34 -0700 (PDT) From: "Basil L. Contovounesios" <contovob@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN> <878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN> <877eiipu8g.fsf@HIDDEN> <87sh16ro91.fsf@HIDDEN> Date: Tue, 16 Oct 2018 15:55:28 +0100 In-Reply-To: <87sh16ro91.fsf@HIDDEN> (Michael Albinus's message of "Tue, 16 Oct 2018 15:54:02 +0200") Message-ID: <87r2gqvt3z.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33018 Cc: 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Michael Albinus <michael.albinus@HIDDEN> writes: > "Basil L. Contovounesios" <contovob@HIDDEN> writes: > >> There's something about going through a complete create-join cycle more >> than once within a non-main-thread which is triggering this behaviour. > > I'm not sure that it is related to threads. It has to be, because there is never an issue when I run the same asynchronous wget processes without threads, and with threads the hang reliably occurs 100% of the time. > It looks, like some of your processes do not exit properly, and then > thread-join is blocked. Indeed, but there is something about the interaction of Emacs threads and subprocesses which is causing unsuccessful process termination. Note that I am not ruling out pilot error; I simply haven't debugged this issue any deeper yet. The fact that no-one has yet pointed out any obvious blunders on my part gives me more confidence that there is indeed some ghost in the wire. > I've modified your example, again. It runs perfectly. Indeed, there are many subprocess-within-a-thread examples which don't suffer from a hang, e.g. by using a different URL. I would like to get to the bottom of why network programs like wget/curl in particular eventually hang, though. Thanks, -- Basil
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at 33018) by debbugs.gnu.org; 16 Oct 2018 13:54:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 16 09:54:12 2018
Received: from localhost ([127.0.0.1]:51948 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gCPnY-0005J2-Iy
for submit <at> debbugs.gnu.org; Tue, 16 Oct 2018 09:54:12 -0400
Received: from mout.gmx.net ([212.227.15.15]:51147)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <michael.albinus@HIDDEN>) id 1gCPnW-0005Io-RY
for 33018 <at> debbugs.gnu.org; Tue, 16 Oct 2018 09:54:11 -0400
Received: from detlef.gmx.de ([212.86.50.126]) by mail.gmx.com (mrgmx003
[212.227.17.190]) with ESMTPSA (Nemesis) id 0M243n-1fNsbg33mt-00u59Q; Tue, 16
Oct 2018 15:54:03 +0200
Received: from detlef.gmx.de ([212.86.50.126]) by mail.gmx.com (mrgmx003
[212.227.17.190]) with ESMTPSA (Nemesis) id 0M243n-1fNsbg33mt-00u59Q; Tue, 16
Oct 2018 15:54:03 +0200
From: Michael Albinus <michael.albinus@HIDDEN>
To: "Basil L. Contovounesios" <contovob@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
<878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN>
<877eiipu8g.fsf@HIDDEN>
Date: Tue, 16 Oct 2018 15:54:02 +0200
In-Reply-To: <877eiipu8g.fsf@HIDDEN> (Basil L. Contovounesios's message of
"Tue, 16 Oct 2018 02:15:27 +0100")
Message-ID: <87sh16ro91.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Provags-ID: V03:K1:KtQC+Mroc7+sdnkSV91A3f8zOsjmlDN2x+Fvp0HvPrvbIsJ/vhi
xa0Q+8Fx7mjPg9G9F1/K/yvv+5xD2UZECxN8NHDOxu0FeKycZnExUVjuKycLvyLTn5nLZEU
q2e0vpY7pfeN0DZ1t8wRYsyP+4dIgBqhoqsTpQvMaCwdyXVWmDYEbMnC4Vti03GmJPTz1Vg
qO7eoDpmIHZ56KXUm3qWw==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V01:K0:nUfzXSnlqr0=:a1S58+3vSGPDsgoSyByalH
7K3BGD2CpRoIbrFH7iXoNDBhdZb4NVnHmQyaMSUtDqR5Db6VLtt471Ll724I/ecmEiCL8urAM
qYYKVzqP44Ajbtsvo257R53TWAbb/k9uKLdqSz2S+73uOl+ZLqNga4KconFL2lynG2/iBA2qE
DJKDrHal94vkWt+ChFuDiwNMFuwoecbogH2eXGfKSh4A8Xak5NN8jiPUf6uCqJjiP3t1nNcIA
KaFAgTgaMxjNvSVga4zoyeuW3GZMKaCrObr9gjuRHOoOkvOBATJHeGrl72uURdh3F2yuvwX4z
pdITzneoc7vFwwnsD9IkwzQ/3p0my6d3JPmLaw541N5spTpeEXLGy8JJVJ1nRCVy7YXRyJXPc
IiFPrPC8XNEOjefzlJcFA0nTrFQF/Qq4YX35P1mIKNv4vABRZ/wAjoM7Yg6lUJhYaBRylrgvA
0H7xuzgdiIst8y3qnilpwRRXMFyPQaDQQzT8vTMnGnjQGeboTGpszZgAIzkUSK899RvREW725
tfehBm95A9unx4fZnEN6bBebP5jnhSGt6vZNZG26xt7llHGuLXJV6onNkpBsPljr5/8qdZxp+
7R79nkjZ3h/CHZaYv0ZpMgrQ+pqhrYxvopZwWHgDcWyEVwyHCrNhAWIeu01jjzhZgsyDfaQEe
WJSzr920MPwIrtSZhy4utBOV5bQGJbgDP7iq1Rlf6B4e/F9BuSrbJas1Zs0/qBqI7eGO85b9J
UTyfuvz5jjGGTrLwu7uBZljXuA88P8A/nxOBgXa4NZntYuabOlUHEo+jiqhm+7964/Hhnvxc2
oQfiGHckvwXwCmZiwvHB+KqgQJxrQ==
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 33018
Cc: 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
--=-=-=
Content-Type: text/plain
"Basil L. Contovounesios" <contovob@HIDDEN> writes:
> There's something about going through a complete create-join cycle more
> than once within a non-main-thread which is triggering this behaviour.
I'm not sure that it is related to threads. It looks, like some of your
processes do not exit properly, and then thread-join is blocked.
I've modified your example, again. It runs perfectly. And during its
work, you could call "M-x list-threads" and see how the threads are
created and die.
Best regards, Michael.
--=-=-=
Content-Type: text/plain
Content-Disposition: attachment
diff -u /home/albinus/tmp/test.el.\~1\~ /home/albinus/tmp/test.el
--- /home/albinus/tmp/test.el.~1~ 2018-10-16 14:46:47.000000000 +0200
+++ /home/albinus/tmp/test.el 2018-10-16 15:47:05.581276541 +0200
@@ -19,9 +19,9 @@
(defun test-slave ()
"Create an asynchronous process and wait for it to exit."
(let ((proc (make-process
- :name "test"
+ :name (thread-name (current-thread))
:buffer (generate-new-buffer " *test*")
- :command '("wget" "-qO-" "https://en.wikipedia.org/wiki/Emacs")
+ :command `("echo" ,(thread-name (current-thread)))
:connection-type 'pipe
:sentinel #'test-sentinel)))
(while (eq (process-status proc) 'run)
@@ -32,10 +32,10 @@
"Successively run `test-slave'.
With non-nil ASYNC, use a new thread for each invocation."
(if async
- (dotimes (_ 2)
+ (dotimes (i 100)
(let (threads)
- (dotimes (i 2)
- (push (make-thread #'test-slave (format "test-%d" i)) threads))
+ (dotimes (j 100)
+ (push (make-thread #'test-slave (format "test-%d-%d" i j)) threads))
(mapc #'thread-join threads)))
(dotimes (_ 2)
(test-slave))))
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at 33018) by debbugs.gnu.org; 16 Oct 2018 01:15:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 15 21:15:44 2018
Received: from localhost ([127.0.0.1]:51643 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gCDxX-0006BQ-US
for submit <at> debbugs.gnu.org; Mon, 15 Oct 2018 21:15:44 -0400
Received: from mail-ed1-f47.google.com ([209.85.208.47]:37602)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <contovob@HIDDEN>) id 1gCDxV-0006BB-4A
for 33018 <at> debbugs.gnu.org; Mon, 15 Oct 2018 21:15:42 -0400
Received: by mail-ed1-f47.google.com with SMTP id c22-v6so19653508edc.4
for <33018 <at> debbugs.gnu.org>; Mon, 15 Oct 2018 18:15:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=tcd-ie.20150623.gappssmtp.com; s=20150623;
h=from:to:cc:subject:references:date:in-reply-to:message-id
:user-agent:mime-version;
bh=Qu5NrC7ZxHOFneqBJimL1EW028BtpdtZs1JiIJxZQDU=;
b=OUPKfWOi09riNKml4/YhCEoTgmILvU9HjgtU2Q8ZPZ80sqmFre40i4zhCHmgs+uMwJ
6opHdInz7cNw5mLcXy0KiNVETQ/nXKR981JOQ6LuyLTXmp+di3IfOCN+0i/yu2T5Yk/z
oJFjPd5veeVHAqkI6oiScl4n7MppLln0VoFXOxY1pg+sBGfVAUuljojO4maZhb1y+Z4+
8rOWzAiBQp+stZznN40JcmJGw1lltIkW0vX7uFoRCIYSGir1/yBTj2Qts8RZkyWv7L62
4DY1o3PsiW4o2UYQeMCjnyN10woE1PzRcEIe/qJjLm9yIFLDmpdaEQ5oEU/zFRDEZMI/
8n/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
:message-id:user-agent:mime-version;
bh=Qu5NrC7ZxHOFneqBJimL1EW028BtpdtZs1JiIJxZQDU=;
b=sW2+xfIYsKFyWN16o9Y8NLFwWiXD0Tm91scxNVDis4kToRbd2mro7J8h4FCPtKwwKR
Nek/UPvCTqg5F/5RtsfVLLAD10zeb2kVJyOXbw8x1PCdYKLqKy9oxvpz/FYuge36LPZf
RljDnTUp/If9YXG7imEW8lBvaTa23AQpHmaaOumslJ5pFeTabTywT1iHOLfpnGOEDrp7
D3Il3ZJtz9RN4Z6KlR77q5vdEu3BsgzzSAhWDtFfcKyV9Vk+xhfEl4OzIBehnO7RK4em
X95ORABZOXPuUiGPc0aH0V8Hcb3UkYW1eG53lSfKPVk9hWpXnSCQKwEi/h03QVnbIX6a
Kqbw==
X-Gm-Message-State: ABuFfoi6Q5n1J5GwpcIkL4VUkokH3pssvoGDYFkSNCzb6p0LZvgXVhrN
8mZS+mf5DlPZpnOrE0iwl1Qx5g==
X-Google-Smtp-Source: ACcGV604jjEz3ELxeKcuiQ6Nsru2JXZfoHNcep5yFh/1uQBUQbZzTsViflFdutkeU407SslcHJ11Pg==
X-Received: by 2002:a50:d78b:: with SMTP id
w11-v6mr28672589edi.19.1539652535019;
Mon, 15 Oct 2018 18:15:35 -0700 (PDT)
Received: from localhost (51-171-243-2-dynamic.agg2.clk.blp-srl.eircom.net.
[51.171.243.2])
by smtp.gmail.com with ESMTPSA id y47-v6sm5801266edc.27.2018.10.15.18.15.33
(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
Mon, 15 Oct 2018 18:15:34 -0700 (PDT)
From: "Basil L. Contovounesios" <contovob@HIDDEN>
To: Michael Albinus <michael.albinus@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
<878t30h823.fsf@HIDDEN> <87sh177i36.fsf@HIDDEN>
Date: Tue, 16 Oct 2018 02:15:27 +0100
In-Reply-To: <87sh177i36.fsf@HIDDEN> (Michael Albinus's message of "Mon, 15
Oct 2018 10:02:53 +0200")
Message-ID: <877eiipu8g.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 33018
Cc: 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: application/emacs-lisp
Content-Disposition: attachment; filename=test.el
Content-Transfer-Encoding: quoted-printable
;;; test.el --- test -*- lexical-binding: t -*-
;;; Code:
(defun test-debug (&rest args)
"Print `current-thread' and ARGS in timestamped `message'."
(push (current-thread) args)
(message "[%s] %s"
(format-time-string "%T:%3N")
(mapconcat #'prin1-to-string args " ")))
(defun test-sentinel (proc msg)
"Kill buffer of PROC when it exits."
(test-debug proc 'sentinel (process-status proc) (substring msg 0 -1))
(and (not (eq (process-status proc) 'run))
(buffer-live-p (process-buffer proc))
(kill-buffer (process-buffer proc))))
(defun test-slave ()
"Create an asynchronous process and wait for it to exit."
(let ((proc (make-process
:name "test"
:buffer (generate-new-buffer " *test*")
:command '("wget" "-qO-" "https://en.wikipedia.org/wiki/Emac=
s")
:connection-type 'pipe
:sentinel #'test-sentinel)))
(while (eq (process-status proc) 'run)
(test-debug proc 'accept-output (accept-process-output proc 5)))
(test-debug proc 'exit (process-status proc) (process-exit-status proc)=
)))
(defun test-master (async)
"Successively run `test-slave'.
With non-nil ASYNC, use a new thread for each invocation."
(if async
(dotimes (_ 2)
(let (threads)
(dotimes (i 2)
(push (make-thread #'test-slave (format "test-%d" i)) threads))
(mapc #'thread-join threads)))
(dotimes (_ 2)
(test-slave))))
(defun test-job (async)
"Successively run `test-slave'.
Like `test-master', but avoid blocking interactive sessions by
running `test-master' in its own thread."
(if noninteractive
(test-master async)
(make-thread (apply-partially #'test-master async) "test-master")))
(defun test-threads ()
"Run `test-master' with threads."
(test-job t))
(defun test-no-threads ()
"Run `test-master' without threads."
(test-job nil))
;;; test.el ends here
--=-=-=
Content-Type: text/plain
Michael Albinus <michael.albinus@HIDDEN> writes:
> Well, I've played with your example. As Eli said, `set-process-thread'
> is not needed here.
>
> With your original example, I could reproduce the problem, However, if I
> call
>
> emacs -l test.el -f test-threads
>
> the problem does NOT happen. My .emacs is quite long, so I didn't bisect
> in order to find out what makes the difference.
>
> I have changed your example a little bit wrt `thread-join', see
> appended. This version runs w/o any problem even if emacs is called with
> -Q. Maybe this helps you to debug further.
Thanks, creating all threads before waiting for any of them to exit
indeed does not suffer from the same hang. Doing this twice (see
attached update), however, still hangs.
There's something about going through a complete create-join cycle more
than once within a non-main-thread which is triggering this behaviour.
--
Basil
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at 33018) by debbugs.gnu.org; 15 Oct 2018 08:03:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 15 04:03:03 2018
Received: from localhost ([127.0.0.1]:49754 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gBxqB-0004HT-HO
for submit <at> debbugs.gnu.org; Mon, 15 Oct 2018 04:03:03 -0400
Received: from mout.gmx.net ([212.227.15.19]:50787)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <michael.albinus@HIDDEN>) id 1gBxq9-0004Gz-WE
for 33018 <at> debbugs.gnu.org; Mon, 15 Oct 2018 04:03:02 -0400
Received: from detlef.gmx.de ([212.86.50.126]) by mail.gmx.com (mrgmx001
[212.227.17.190]) with ESMTPSA (Nemesis) id 0M2t0Q-1fKgzg3Ilq-00sih9; Mon, 15
Oct 2018 10:02:54 +0200
Received: from detlef.gmx.de ([212.86.50.126]) by mail.gmx.com (mrgmx001
[212.227.17.190]) with ESMTPSA (Nemesis) id 0M2t0Q-1fKgzg3Ilq-00sih9; Mon, 15
Oct 2018 10:02:54 +0200
From: Michael Albinus <michael.albinus@HIDDEN>
To: "Basil L. Contovounesios" <contovob@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
<878t30h823.fsf@HIDDEN>
Date: Mon, 15 Oct 2018 10:02:53 +0200
In-Reply-To: <878t30h823.fsf@HIDDEN> (Basil L. Contovounesios's message of
"Sun, 14 Oct 2018 16:17:08 +0100")
Message-ID: <87sh177i36.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Provags-ID: V03:K1:Qpnh7mnRu8gLYavULjdmsgd4ECgrnOZHm9C8C5b+M6d/JBuBEhk
XCUxQ0MCnsE22ccZT5x4Op3+tH+4boJbEvmTu+S7btIC1/K5xBXffAdIeoNLs6ZnuP4wg62
cx/aagAArvU+L3yGpxveIN62cjzKuAf1B/Ykzb4mwD+m7TY2ZaBUivI51+OheL0biqRYDbL
70B4j9905VtdUW0wRTDHg==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V01:K0:Y8pTnWUnAlc=:R+jz4/EE2fW6NMVXGPRDzP
7qk1hU2VQfKRZNB0vGWfKNW2Gf/cE00NkeSJBP71k4d9t7nECwj4itHB2Aj16aayEaJzkAo//
8yBJ8saY2bP7bc5eT/XO++LYqHo97MkBmUt+h8Ixu1QGIXj6jWdl/ZALedk7+M0+CNbEuz2+Z
NxLK8TANIEda8CQKCsbpv7dsasIkn/W5PuyAxIbP3nmAwg9NAFUxA8YHaanuwp0ip4W3zcN5I
6o6qbv4wlhOWPX7HCjH7dcariD2iMy1d29s7afC9ws2/zvKK3TFld8rBRrZLCiN4aKEC8JAKd
RUa14sUrrZtQCeS7RROKuMDKDKk/LXJ/TMhoVDXucKLHpjMD1+rS/4R7N42/qb0onnvK7Jmnn
pW8XTyB95xGG+GoMKBC1FGksEb6pEH/rDcj1xeLQ2o5d8bzxzrlcR8aX8x799DTjF5onu3QOy
5yqaPHdI2B4/2Q2nmn+g92K91cloSgR2tCYH4338kFEbquDJnOWAEEzw52xHFS5+RiNLy0Jzz
i+iXPAtKsH5P35jNYCuvGG06urkM3dBCx5171vU0kMNRFDriPiXGoknOK0fcKwmSJJ/5Sc1F4
poKvcwRrWa8hiVrO8SkfEn5TXWPEcbPooSrZIOMrGkRkkD9D9OwxfKlVcs0S8J5Q28pB7g3ci
OD3iSkELm0dEtXrIOGwRcWmzrNqxqMsPkQJ5JpEp6v9m0bEXOC1Z7Ou1ubmURp1tyj9PQVOgu
PgmUchmpl/3yqNvZBHuTWVqK+HkxMkoWv1K6zrJryj15ejbV4imcJcmvvDMjhkzlHZ84TksoB
ngeFFJzd5bjDxBaJbebpqJQywJcKw==
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 33018
Cc: 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
--=-=-=
Content-Type: text/plain
"Basil L. Contovounesios" <contovob@HIDDEN> writes:
Hi Basil,
>> If you want a process to communicate in a given thread, you must call
>> `set-process-thread'. See the elisp manual.
>
> Thanks, this is the first thing I tried when earlier experiments started
> to hang. I tried both the following redundant but explicit call:
Well, I've played with your example. As Eli said, `set-process-thread'
is not needed here.
With your original example, I could reproduce the problem, However, if I
call
emacs -l test.el -f test-threads
the problem does NOT happen. My .emacs is quite long, so I didn't bisect
in order to find out what makes the difference.
I have changed your example a little bit wrt `thread-join', see
appended. This version runs w/o any problem even if emacs is called with
-Q. Maybe this helps you to debug further.
Best regards, Michael.
--=-=-=
Content-Type: text/plain
Content-Disposition: attachment
;;; test.el --- test -*- lexical-binding: t -*-
;;; Code:
(defun test-debug (proc &rest args)
"Print `current-thread' and ARGS in timestamped `message'."
(message
"[%s] %s"
(format-time-string "%T:%3N")
(mapconcat
#'prin1-to-string `(,(current-thread) ,(process-thread proc) ,@args) " ")))
(defun test-sentinel (proc msg)
"Kill buffer of PROC when it exits."
(test-debug proc 'sentinel (process-status proc) (substring msg 0 -1))
(and (not (eq (process-status proc) 'run))
(buffer-live-p (process-buffer proc))
(kill-buffer (process-buffer proc))))
(defun test-slave ()
"Create an asynchronous process and wait for it to exit."
(let ((proc (make-process
:name "test"
:buffer (generate-new-buffer " *test*")
:command '("wget" "-qO-" "https://en.wikipedia.org/wiki/Emacs")
:connection-type 'pipe
:sentinel #'test-sentinel)))
(while (eq (process-status proc) 'run)
(test-debug proc 'accept-output (accept-process-output proc 5)))
(test-debug proc 'exit (process-status proc) (process-exit-status proc))))
(defun test-master (async)
"Successively run `test-slave'.
With non-nil ASYNC, use a new thread for each invocation."
(if async
(progn
(dotimes (i 5) (make-thread #'test-slave (format "test-%d" i)))
(while (cdr (all-threads) (thread-join (cadr (all-threads))))))
(dotimes (i 5)
(test-slave))))
(defun test-job (async)
"Successively run `test-slave'.
Like `test-master', but avoid blocking interactive sessions by
running `test-master' in its own thread."
(if noninteractive
(test-master async)
(make-thread (apply-partially #'test-master async) "test-master")))
(defun test-threads ()
"Run `test-master' with threads."
(test-job t))
(defun test-no-threads ()
"Run `test-master' without threads."
(test-job nil))
;;; test.el ends here
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 14 Oct 2018 15:36:29 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 14 11:36:29 2018 Received: from localhost ([127.0.0.1]:49457 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gBiRR-0000vR-DU for submit <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:36:29 -0400 Received: from mail-ed1-f53.google.com ([209.85.208.53]:43236) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <contovob@HIDDEN>) id 1gBiRP-0000vE-SE for 33018 <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:36:28 -0400 Received: by mail-ed1-f53.google.com with SMTP id y20-v6so15552500eds.10 for <33018 <at> debbugs.gnu.org>; Sun, 14 Oct 2018 08:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tcd-ie.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=k524uxsgCINAvw2xVFPDeEgrsdBmFBA1jtsOc1M8GPA=; b=YN0a++J78QoAE6BI/UwBFEsCGTUqwKXBCQTjcdfKGI5byAFFjq1IffC9O9xVi+mdai wUAkRKhrG7GCYf1NL6OnuruIRtuOlC/qpFGsZ04rz39hB7QOIYflTSrWPS1vJTBvNlso 1cWLtrqujCzFdm/GGscUsxOI83EwIG6tTzgeJqkjhfSg4aupk45tCUibOfV768+ZeNJ8 T4mYdN7mvpDS3x5QdhEAOw0yzpswmBD1b00tJqX/sTWEoj5dEdS0rZa9sMcQoKOZEKAN wRcQbMDQBuOXfC5n8j5j7i7o6bFxMIVWQREvIZwGvvRoYlfbJScOhRBYYJCsEEAm2BiB JHmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=k524uxsgCINAvw2xVFPDeEgrsdBmFBA1jtsOc1M8GPA=; b=etcG6AmRtXFVDub8+xj1itWcnfsxU83yigz9I17eagtIfNJgeBDkS8i9Y4wEuKeoQ7 yAd+hwDkCWUEMRsIxelCFePcyC1HwE5nbWkRc4xhPrq2jnmlxtpGQ88lnPIKsEV4h6Om uKH7qeViZziCs/7epcCguBWiE66Mv5WXZYPCTImg3SwOfDEY9MiHoKa+iPa6xN+IoI3D Lrp7uYYm49cf8fVrvz7umcOObSw9rB2BG83zAteZG84DNCzpkDv/PQIkg+hPvsS4un9c UKqDeRnPCVhzp4FQLd+sz/8La3Mc/M2YA2ru+xZo6GUIXYMfkUWtgKai+3PBcUZo0IMA BK7g== X-Gm-Message-State: ABuFfojuN8Q8ZkJH58YCxwNpeD7HujF3IHwCJ/9nJ06ejifl0u7Mha/G z9EOQXY10Iiy69ZKnjLF4gZrCQ== X-Google-Smtp-Source: ACcGV626s0nJdEbtltq/ur0iRZE4jvr2b8N0G7ejMlP37UeRfogYO/1aazRN73uk32E1wwQqHcvyVg== X-Received: by 2002:a50:8f64:: with SMTP id 91-v6mr19959285edy.264.1539531382071; Sun, 14 Oct 2018 08:36:22 -0700 (PDT) Received: from localhost ([213.233.155.142]) by smtp.gmail.com with ESMTPSA id l42-v6sm2996352edb.24.2018.10.14.08.36.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 14 Oct 2018 08:36:21 -0700 (PDT) From: "Basil L. Contovounesios" <contovob@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <83o9bzwpty.fsf@HIDDEN> <87h8hoh8t3.fsf@HIDDEN> <83murgtv5y.fsf@HIDDEN> Date: Sun, 14 Oct 2018 16:36:20 +0100 In-Reply-To: <83murgtv5y.fsf@HIDDEN> (Eli Zaretskii's message of "Sun, 14 Oct 2018 18:17:13 +0300") Message-ID: <87woqkfsln.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 33018 Cc: 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: "Basil L. Contovounesios" <contovob@HIDDEN> >> Cc: <33018 <at> debbugs.gnu.org> >> Date: Sun, 14 Oct 2018 16:00:56 +0100 >> >> > When the hang happens, is there any wget process still alive, or did >> > they all exit? Please use OS tools to find that out, don't rely on >> > what Emacs thinks. >> >> When the hang happens, the wget process launched by the waiting thread >> is still alive but asleep (idle), as reported by ps and top. > > Any idea why is that? No idea, sorry. I haven't yet found the time to look into this deeper. > And if this is the situation, doesn't it explain why > accept-process-output times out? Possibly, but the question is why do we enter either of these situations (process sleeping and accept-process-output timing out) exclusively when using Emacs threads, no? By the way, the problem isn't specific to wget; the same thing happens with curl. I also find it strange that the first thread doesn't suffer from the same problem as the second thread, especially given each thread is created only after the last thread exited. -- Basil
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at 33018) by debbugs.gnu.org; 14 Oct 2018 15:17:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 14 11:17:23 2018
Received: from localhost ([127.0.0.1]:49446 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gBi8x-0000UR-AT
for submit <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:17:23 -0400
Received: from mail-ed1-f46.google.com ([209.85.208.46]:45162)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <contovob@HIDDEN>) id 1gBi8t-0000Tw-JT
for 33018 <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:17:19 -0400
Received: by mail-ed1-f46.google.com with SMTP id v18-v6so15520847edq.12
for <33018 <at> debbugs.gnu.org>; Sun, 14 Oct 2018 08:17:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=tcd-ie.20150623.gappssmtp.com; s=20150623;
h=from:to:cc:subject:references:date:in-reply-to:message-id
:user-agent:mime-version;
bh=XppSNzBezTqqCrMm+3mt6E9JaOGxsiIOA2+sRXzxMa0=;
b=dd/9zqc34TgwCJ6zyl62zK/bIZA6hRWZc2YbJ9/Mq+ZnlA1Zn1wOyV/jfIiyD9Oikw
qFh50TIWKWdSh6qN0BmuBqfGxQemF3cIcv6DFou3UImYRcIYFONn6J1JWowRnDKI4QAs
21cpFwhVyoV1Tp4RBZhfBTbJqL+cG5rdcJk8ZTKGKwBeikOProidQ83aGMVT8DfzVzJN
VXdLn3n2Ftr/SPlJvZ2ZYl3pfQlqm5yKShZaM3JG3fdLLdvJknHkC9dI4LLaWFl7f5+4
/A2EZQy5PorEOeRBkF6xjkzXQ7EqyJ4WWiXKgFQG35ENMpHWA/TVRdgjf5OL0qLFi2Pl
Jm4w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
:message-id:user-agent:mime-version;
bh=XppSNzBezTqqCrMm+3mt6E9JaOGxsiIOA2+sRXzxMa0=;
b=jxejEkiBSuD/h+oQa/dRPDPJPnwJ+HprGdXdZ0bIuuLPAz+ZTtjxblqgPHav6EDIQQ
KpN9LrboiNsW45qOfzbhu9BJYpmgEiXbt7NL1HBHrs43tFOLlOjGDvP7pvn020g7TDAK
kIuIzJ4Ox3X+IMIZ9Eht/mFBHThMr9IybcaYkXQe/6GEtORLO2IcBxTsnAgWKRhsTRnI
uhV2TzMhlacSSlZY0/JRBJYL60M8v8mHRMDLQHPhf8N6kHS7QK0aWC5a+5T245NlN5y1
nvHHl4hRMFVzV4NiVd3LCOQ1Sj5JPoOaR0OFfNWIFLHFa4IZiKG+aNfRSLJKGcuVqbxR
n1Mw==
X-Gm-Message-State: ABuFfojMXHm98xbYaeGFsXQIImoDV0OaJdo6t7Hujln7N/O7Ho9c/65R
cwYVaFPUEq5LkLBe1QjFG7+d1Q==
X-Google-Smtp-Source: ACcGV61Cip2pDn0PgGil7FDoUelUrOTr6NWoZNTAdfV5C8O/aWGvQdM3GpQTGUdUceoRfrYLqYTrzg==
X-Received: by 2002:a50:fc97:: with SMTP id
f23-v6mr19752846edq.276.1539530233933;
Sun, 14 Oct 2018 08:17:13 -0700 (PDT)
Received: from localhost ([213.233.155.133])
by smtp.gmail.com with ESMTPSA id y47-v6sm3729289edc.27.2018.10.14.08.17.12
(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
Sun, 14 Oct 2018 08:17:13 -0700 (PDT)
From: "Basil L. Contovounesios" <contovob@HIDDEN>
To: Michael Albinus <michael.albinus@HIDDEN>
Subject: Re: bug#33018: 26.1.50;
thread starvation with async processes and accept-process-output
References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN>
Date: Sun, 14 Oct 2018 16:17:08 +0100
In-Reply-To: <875zy7cqzd.fsf@HIDDEN> (Michael Albinus's message of "Fri, 12
Oct 2018 14:02:46 +0200")
Message-ID: <878t30h823.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 33018
Cc: 33018 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain
Michael Albinus <michael.albinus@HIDDEN> writes:
> "Basil L. Contovounesios" <contovob@HIDDEN> writes:
>
>> I attach a sample program test.el whose central function, test-slave,
>> invokes wget asynchronously before waiting for the process to exit.
>>
>> The issue I'm facing is that running test-slave twice in succession,
>> each time in a new thread, causes accept-process-output to hang with no
>> output (unless a timeout argument is given, in which case the function
>> returns nil) the second time around.
>
> If you want a process to communicate in a given thread, you must call
> `set-process-thread'. See the elisp manual.
Thanks, this is the first thing I tried when earlier experiments started
to hang. I tried both the following redundant but explicit call:
--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline; filename=test.diff
diff -u --label /tmp/test.el --label \#\<buffer\ /tmp/test.el\> /tmp/test.el /tmp/buffer-content-hY0EDf
--- /tmp/test.el
+++ #<buffer /tmp/test.el>
@@ -24,6 +24,7 @@
:command '("wget" "-qO-" "https://en.wikipedia.org/wiki/Emacs")
:connection-type 'pipe
:sentinel #'test-sentinel)))
+ (set-process-thread proc (current-thread))
(while (eq (process-status proc) 'run)
(test-debug proc 'accept-output (accept-process-output proc 5)))
(test-debug proc 'exit (process-status proc) (process-exit-status proc))))
Diff finished. Sun Oct 14 16:04:48 2018
--=-=-=
Content-Type: text/plain
as well as replacing (current-thread) with nil, to unlock the process.
Adding test-debug calls before and after accept-process-output revealed
nothing out of the ordinary, and explicitly un/locking the process
didn't fix the hang. Should I be calling set-process-thread elsewhere?
--
Basil
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 14 Oct 2018 15:17:20 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 14 11:17:20 2018 Received: from localhost ([127.0.0.1]:49444 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gBi8u-0000UF-13 for submit <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:17:20 -0400 Received: from eggs.gnu.org ([208.118.235.92]:51265) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1gBi8s-0000Tu-M6 for 33018 <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:17:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gBi8h-0003eT-Ie for 33018 <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:17:11 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:49503) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gBi8f-0003d7-UC; Sun, 14 Oct 2018 11:17:06 -0400 Received: from [176.228.60.248] (port=4262 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <eliz@HIDDEN>) id 1gBi8f-0004mu-Cm; Sun, 14 Oct 2018 11:17:05 -0400 Date: Sun, 14 Oct 2018 18:17:13 +0300 Message-Id: <83murgtv5y.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: "Basil L. Contovounesios" <contovob@HIDDEN> In-reply-to: <87h8hoh8t3.fsf@HIDDEN> (contovob@HIDDEN) Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <83o9bzwpty.fsf@HIDDEN> <87h8hoh8t3.fsf@HIDDEN> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33018 Cc: 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -6.0 (------) > From: "Basil L. Contovounesios" <contovob@HIDDEN> > Cc: <33018 <at> debbugs.gnu.org> > Date: Sun, 14 Oct 2018 16:00:56 +0100 > > > When the hang happens, is there any wget process still alive, or did > > they all exit? Please use OS tools to find that out, don't rely on > > what Emacs thinks. > > When the hang happens, the wget process launched by the waiting thread > is still alive but asleep (idle), as reported by ps and top. Any idea why is that? And if this is the situation, doesn't it explain why accept-process-output times out?
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 14 Oct 2018 15:01:10 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 14 11:01:10 2018 Received: from localhost ([127.0.0.1]:49436 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gBhtG-000085-Id for submit <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:01:10 -0400 Received: from mail-ed1-f41.google.com ([209.85.208.41]:44756) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <contovob@HIDDEN>) id 1gBhtE-00007q-Ih for 33018 <at> debbugs.gnu.org; Sun, 14 Oct 2018 11:01:09 -0400 Received: by mail-ed1-f41.google.com with SMTP id z21-v6so15489168edb.11 for <33018 <at> debbugs.gnu.org>; Sun, 14 Oct 2018 08:01:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tcd-ie.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=e11quNtDda6k3lXSN32F/Ho4Ec37JjL0miEVGKBJXLU=; b=JUUrtzBpqsz//VOUrlIJ09jM5NaGEJeLCUdmT6xeB7lN/YSuKvvOtyPuYKrR1A2wzU fKnfRA9YMuhWwdetPexRu+zspkTI9b7pAq/0riUZmx01AtbU7SuD/VEp5uinLD1oNOL2 Es31V7ZnI6vsKCoEHhLIyf0pFkbHOqvM+aV2RdH/qotpCQIYTq4zniIfwFXkuZxa+mof cy0Bfgk3nuS1bIoQ+6OZZ4UauKg0kVp6hPf5h5FfAoXHs19RnPWwyU58+Uhv9sZDevmY K1g4lTHv5Q86K7gaivMvKIRb5njA3I+b8wRKrD71hNcsk2CGtNaH/+USP1+IzrshzLd4 zIcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=e11quNtDda6k3lXSN32F/Ho4Ec37JjL0miEVGKBJXLU=; b=MH5hKzUDD/l7Y02xf3FZhupUpImJyA8l9YUQJ45rm6TdTuhnUP8m6pNp2ATcQM23IN cIQJ78QV8CDMIIMgK59RgaXz+I8eU/Q4sdxhP7MTWJne/+w5Ug5GfTG1Ci1yPwh3ehnT srCtuioFSbFvZvXwLH1EOQrrgPm+IshyLMi32SXA5SfNyQqeCXyFEE3gPhM0RQqGB4Bv CIkzk8Vrs1VAT1k5oltP9/nBzfyuzGmHwuffF6IAx2M72tOtBLKmdCyJeotFAcfBqNQb NKCDjBNEX1V2627g08oyC/ZhErBatkozmxPdTTtihKKB3wLAVL/BJ6ZkyMBmc0kddjZB 7GBg== X-Gm-Message-State: ABuFfoggXfwCJ+ZpP7Y5mcwwxR+aH2M9btrYtR+EU89BDCrTFlvWTuNt IiAPurQcNL24Bo/qtnZSFdeOmlqjMNc= X-Google-Smtp-Source: ACcGV60f2JbHWhaR+rZ7xXvRUCiJnbXWI+AqW6hRC+7CEgJZtltwpwEoKBGut/yIYc+8CKJ/AXABbA== X-Received: by 2002:a17:906:2ce:: with SMTP id 14-v6mr15334996ejk.66.1539529262516; Sun, 14 Oct 2018 08:01:02 -0700 (PDT) Received: from localhost ([213.233.155.142]) by smtp.gmail.com with ESMTPSA id 7-v6sm1595949ejy.11.2018.10.14.08.01.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 14 Oct 2018 08:01:01 -0700 (PDT) From: "Basil L. Contovounesios" <contovob@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <83o9bzwpty.fsf@HIDDEN> Date: Sun, 14 Oct 2018 16:00:56 +0100 In-Reply-To: <83o9bzwpty.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 12 Oct 2018 11:07:21 +0300") Message-ID: <87h8hoh8t3.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 33018 Cc: 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: "Basil L. Contovounesios" <contovob@HIDDEN> >> Date: Thu, 11 Oct 2018 15:57:50 +0100 >> >> I attach a sample program test.el whose central function, test-slave, >> invokes wget asynchronously before waiting for the process to exit. >> >> The issue I'm facing is that running test-slave twice in succession, >> each time in a new thread, causes accept-process-output to hang with no >> output (unless a timeout argument is given, in which case the function >> returns nil) the second time around. > > When the hang happens, is there any wget process still alive, or did > they all exit? Please use OS tools to find that out, don't rely on > what Emacs thinks. When the hang happens, the wget process launched by the waiting thread is still alive but asleep (idle), as reported by ps and top. -- Basil
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 12 Oct 2018 12:49:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 12 08:49:37 2018 Received: from localhost ([127.0.0.1]:46002 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gAwsr-00088B-4x for submit <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:49:37 -0400 Received: from mout.gmx.net ([212.227.17.22]:54245) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1gAwsp-00087y-0v for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:49:35 -0400 Received: from detlef.gmx.de ([212.91.243.207]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MHXd2-1gEFCd1E3Y-003Lkm; Fri, 12 Oct 2018 14:49:27 +0200 Received: from detlef.gmx.de ([212.91.243.207]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MHXd2-1gEFCd1E3Y-003Lkm; Fri, 12 Oct 2018 14:49:27 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN> <83d0sfwd14.fsf@HIDDEN> Date: Fri, 12 Oct 2018 14:49:25 +0200 In-Reply-To: <83d0sfwd14.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 12 Oct 2018 15:43:51 +0300") Message-ID: <871s8vcotm.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:6qIHB5ce0VxEcW9PcdryqKVLX7GhWrplb5w9py7gf0kfNPIiv5G Cxg/Pekdzdj4/BaZ2tWurb8dYZ8ilTJ9jUUOeVjY8zOPKcgzlt8oUfrFzd4Hb3Uddm0pU0X 3XpffhSZr40TTAz2MlUnwNeaL7uMECyNIiSPik5pAWEVJII3dbdmQ23J+fUg59zw72c5JHh Y5iAmTY2smerFtzKuoK3w== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V01:K0:Hssog6mhvpM=:c2o9KJbHyl9FOo5dyo99XI GzuF3YjK+jZa7VtFgW25/MRl8F8LhsdJaiubJQhFBY8FsejaXtItCETazs6dxe2ff6iXbxii+ lIYBRNnQVHrp061e3D+xzsgJxIzmbyGSqpsMAsHA6FEzGGLZKy/8a8Zh3QgjQYbSMfwfi93CX JcerVbLyAzb3bMR5215t4OdVJjgN4gnGGejsLQqRD0XA8sA+qBhTZbfRv4Vq8wzwdOIjRQQNi ApVwr2Qo/4HVAhbQHKPe9TkN6fIOWERIOk000SSUCqiWPFdITwZHqcwWqBJg9A5101zJMPHII SdrW4Z76MYJTMnlumPKFGQSBvDN9Tkc5H8Y2N0Wj9aYH+CGMoD6QJn0MPkIaf0LGQF8/cztFl k1UlM1c0ejfX0Ry44DMyhVS+E+UKPFN+9Ol7ImPQ+26eD3j4oJ26GuwEc2luWe8eM6NWvYzoG xKXMVwu+k4/uqZJI8/86wdUk1dGtG1LzYh94LBO4l/wLZtXgqr4hXUgPGPW62L7L6BK1ekEMt D33mXy2XOiNqpSRmnxm82WbE7bHTH5Xl/KuqidthuZO4EU56jkKCZbptkgaJcwApFSMiJHjE3 NpABQ6QyLwFhTqRy/aPRBXUXNgrrwn0i2XoQhvrQslSeEsxD73yrLeyH+gmMpaPTPJDD5Eubc Qx4Ke6qBvFlYYqMr4yCE6A6EXFayBXyo4QY5om5kcGugtM1AE4N+7wuv6A2yXjvRFUzcuCp6b Lb8SwWtUAaVANIbeVfZJSnUsHVUicStETyfae5+K3PfX+75UEzUKJcr3SuXt5sjRmPZJ7YvoK h4ICgi+it+tV83VUcNXj57zuIEiZJguqi3SrcutzwW92BjLH7o= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 33018 Cc: contovob@HIDDEN, 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> If you want a process to communicate in a given thread, you must call >> `set-process-thread'. See the elisp manual. > > But the default is that the process is locked to the thread that > created it, so it sounds like this should have just worked (if that is > the problem). I'm not sure. In the branch feature/tramp-thread-safe there were also mysterious blockings in accept-process-output, until I've applied set-process-thread explicitly. At least it is worth a try. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 12 Oct 2018 12:43:59 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 12 08:43:59 2018 Received: from localhost ([127.0.0.1]:45994 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gAwnP-0007zQ-7D for submit <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:43:59 -0400 Received: from eggs.gnu.org ([208.118.235.92]:41585) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1gAwnN-0007zB-Bd for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:43:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gAwnG-0005xL-7r for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:43:52 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34923) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gAwnE-0005vR-VC; Fri, 12 Oct 2018 08:43:49 -0400 Received: from [176.228.60.248] (port=1531 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <eliz@HIDDEN>) id 1gAwnE-0000kG-9p; Fri, 12 Oct 2018 08:43:48 -0400 Date: Fri, 12 Oct 2018 15:43:51 +0300 Message-Id: <83d0sfwd14.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> In-reply-to: <875zy7cqzd.fsf@HIDDEN> (message from Michael Albinus on Fri, 12 Oct 2018 14:02:46 +0200) Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> <875zy7cqzd.fsf@HIDDEN> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33018 Cc: contovob@HIDDEN, 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -6.0 (------) > From: Michael Albinus <michael.albinus@HIDDEN> > Date: Fri, 12 Oct 2018 14:02:46 +0200 > Cc: 33018 <at> debbugs.gnu.org > > If you want a process to communicate in a given thread, you must call > `set-process-thread'. See the elisp manual. But the default is that the process is locked to the thread that created it, so it sounds like this should have just worked (if that is the problem).
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 12 Oct 2018 12:02:56 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 12 08:02:56 2018 Received: from localhost ([127.0.0.1]:45977 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gAw9g-00071Z-MK for submit <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:02:56 -0400 Received: from mout.gmx.net ([212.227.17.22]:41241) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1gAw9f-00071O-EI for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 08:02:56 -0400 Received: from detlef.gmx.de ([212.91.243.207]) by mail.gmx.com (mrgmx102 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MhNk6-1fyNLp2nq6-00Mf7y; Fri, 12 Oct 2018 14:02:47 +0200 Received: from detlef.gmx.de ([212.91.243.207]) by mail.gmx.com (mrgmx102 [212.227.17.168]) with ESMTPSA (Nemesis) id 0MhNk6-1fyNLp2nq6-00Mf7y; Fri, 12 Oct 2018 14:02:47 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: "Basil L. Contovounesios" <contovob@HIDDEN> Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> Date: Fri, 12 Oct 2018 14:02:46 +0200 In-Reply-To: <87r2gwbmma.fsf@HIDDEN> (Basil L. Contovounesios's message of "Thu, 11 Oct 2018 15:57:50 +0100") Message-ID: <875zy7cqzd.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:AjiYQbQ/rtPN6GaEJ8yeQKPPv53heoLxEIChbV5yzXGxytdWst2 /QbigNT+vhaI/Hz7DhN7jvzx/9wkgaZVMdvONy7gusZS+ETG1ARsNuJg54CVqnzxsIjVrFy MO4RK8AyUqZJ5deU5Xp+5NhrTyDK9X/abSUcTJAbihPS9GUmVyGEJBZpoAVK8duWrtCp6qm T/Jp73cBIfZFwww4Or6zA== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V01:K0:kwdGLWAAmrw=:3O2n2J0vZ6w+YRSmha4QIQ 3r9uRUr6WVmPdzr/hV061SPvapirOrONO1KG6mca7pR0qjbtgn7eKpyqoqg2nksfDaIVdA+P1 //ifkI0uHGrMMp3VOSDZSDYtMKv2hOMDQF0oZv2cSPdBOltNfvQwx6jhxw9rwtOiI+r8iBLd1 yE0Q2sHLw7NslH0fvIj81HCo9tqZ+HVj6LbbQmUaZggiPnnpZom4whQTfv0SLXCgdU4+bIxCD w1oKnyXwaHy2ezAdqXsNy84DJ1wKdpBPqIObh8UUv1zHb7yrIYYNnxmfu+L9pP23Yk09vTyMW m9BgzkT+1l4BXqzhp6QbQ9U7WKssndo5C0JFvwIP/QNfF+0p+FD3ZywB4iox5nT7nQjGxyaV8 2WBj3vy4s6DfvIxUU5IykdqMJgkt4AlTWeCU9COGsJc4i9Uh6aUjJ++6LndOL8Xna1R4GUOcZ ubBjQ0EmbVELSq4Oa6++X0pge/8f4dIHTfUldpMGUVGqAhEgzxvUJ+yzEtfXf9euoRqbYRpDL vQqK3NYNIx+9etjgotPWM1Y8oRpYE/Lw46Xu6JJ1ImCq6LikQ2eX67QR2hAXgOqYGkVfW0QWI 6PqCM93LTzQI99vejC1kKRBmUNuwKV51Roq7cfrRwpDhTCgdhb8l1/g26um7+wuK2Let/OdPN xMPz1Tb5Rr13HV8w3YwS3UGO/qDa8yIZVfmgc4vu84FZBHzhT730jX7r3XHQD3S/1OpAVSED1 pKhpgGZgw8V7odx1C1K3JiQ16WEBgN0Lq5LC2eP/+vBwp2nbY4Yj4UeFf/mLNlCSmmqnlh7Cf MuKWvXRzNeFe+KbCWA0Aarorupo7GpGGTOsvfefgvXLiM1Bt94= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 33018 Cc: 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) "Basil L. Contovounesios" <contovob@HIDDEN> writes: Hi Basil, > I attach a sample program test.el whose central function, test-slave, > invokes wget asynchronously before waiting for the process to exit. > > The issue I'm facing is that running test-slave twice in succession, > each time in a new thread, causes accept-process-output to hang with no > output (unless a timeout argument is given, in which case the function > returns nil) the second time around. If you want a process to communicate in a given thread, you must call `set-process-thread'. See the elisp manual. > Thanks, Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.Received: (at 33018) by debbugs.gnu.org; 12 Oct 2018 08:07:46 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 12 04:07:46 2018 Received: from localhost ([127.0.0.1]:45890 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gAsU5-0005o3-Pl for submit <at> debbugs.gnu.org; Fri, 12 Oct 2018 04:07:46 -0400 Received: from eggs.gnu.org ([208.118.235.92]:39547) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1gAsU4-0005nm-H3 for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 04:07:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gAsTt-00034N-75 for 33018 <at> debbugs.gnu.org; Fri, 12 Oct 2018 04:07:37 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:47618) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>) id 1gAsTn-00031Z-62; Fri, 12 Oct 2018 04:07:30 -0400 Received: from [176.228.60.248] (port=4335 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <eliz@HIDDEN>) id 1gAsTf-000718-8x; Fri, 12 Oct 2018 04:07:22 -0400 Date: Fri, 12 Oct 2018 11:07:21 +0300 Message-Id: <83o9bzwpty.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: "Basil L. Contovounesios" <contovob@HIDDEN> In-reply-to: <87r2gwbmma.fsf@HIDDEN> (contovob@HIDDEN) Subject: Re: bug#33018: 26.1.50; thread starvation with async processes and accept-process-output References: <87r2gwbmma.fsf@HIDDEN> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: 33018 Cc: 33018 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -6.0 (------) > From: "Basil L. Contovounesios" <contovob@HIDDEN> > Date: Thu, 11 Oct 2018 15:57:50 +0100 > > I attach a sample program test.el whose central function, test-slave, > invokes wget asynchronously before waiting for the process to exit. > > The issue I'm facing is that running test-slave twice in succession, > each time in a new thread, causes accept-process-output to hang with no > output (unless a timeout argument is given, in which case the function > returns nil) the second time around. When the hang happens, is there any wget process still alive, or did they all exit? Please use OS tools to find that out, don't rely on what Emacs thinks. > I could very well be doing or assuming something incorrectly, but what > baffles me is that the "hang" does not occur either when Emacs is run > non-interactively, or when "https://en.wikipedia.org/wiki/Emacs" is > replaced with "https://www.gnu.org/software/emacs/" Could be different properties of the servers related to async connections, like TLS handshake or even async getaddrinfo.
bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
Received: (at submit) by debbugs.gnu.org; 11 Oct 2018 14:58:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 11 10:58:16 2018
Received: from localhost ([127.0.0.1]:45384 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1gAcPo-0002UC-4b
for submit <at> debbugs.gnu.org; Thu, 11 Oct 2018 10:58:16 -0400
Received: from eggs.gnu.org ([208.118.235.92]:44674)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <contovob@HIDDEN>) id 1gAcPm-0002Tz-6l
for submit <at> debbugs.gnu.org; Thu, 11 Oct 2018 10:58:14 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <contovob@HIDDEN>) id 1gAcPf-00036G-Fd
for submit <at> debbugs.gnu.org; Thu, 11 Oct 2018 10:58:08 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level:
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:38962)
by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
(Exim 4.71) (envelope-from <contovob@HIDDEN>) id 1gAcPf-00036A-BT
for submit <at> debbugs.gnu.org; Thu, 11 Oct 2018 10:58:07 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:54269)
by lists.gnu.org with esmtp (Exim 4.71)
(envelope-from <contovob@HIDDEN>) id 1gAcPe-0008MZ-8I
for bug-gnu-emacs@HIDDEN; Thu, 11 Oct 2018 10:58:07 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <contovob@HIDDEN>) id 1gAcPa-00033L-6H
for bug-gnu-emacs@HIDDEN; Thu, 11 Oct 2018 10:58:06 -0400
Received: from mail-ed1-x530.google.com ([2a00:1450:4864:20::530]:33923)
by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
(Exim 4.71) (envelope-from <contovob@HIDDEN>) id 1gAcPZ-000325-Oz
for bug-gnu-emacs@HIDDEN; Thu, 11 Oct 2018 10:58:02 -0400
Received: by mail-ed1-x530.google.com with SMTP id w19-v6so8564032eds.1
for <bug-gnu-emacs@HIDDEN>; Thu, 11 Oct 2018 07:58:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=tcd-ie.20150623.gappssmtp.com; s=20150623;
h=from:to:subject:message-id:user-agent:date:mime-version;
bh=1fAPAhyfsrUs5ys8GTfE2w//twZLn/LwhXUXVmkrhME=;
b=s75hc5ndvYO5rLDmY1xU6kJ/nxe3s/9nHTZDwtNwElZgOMkooJYeT+sHHeld9faqBA
15PZAHmUyfLOEKq2kFFx5d6ujHIYHxQVyozv7vwUcqPkbGMWGPlODhDFOyDjDk7yQEEj
xTDVcuR87f/BajP5lQHNI2/fTLfW8bL9AL4ZDFve0tsZCSaQwUcw3w6PYTTweNBRwk5n
QUOcN0ahSUNvwia2vVDLAjJeM+yMpFAVfO5AOy8ZObFcIC/fjlEAL1Q33VdvhwviQo14
g+MrrfK7ur0nnzUapuf9aqVFjVro7V79abVHpaeFNlBx+4XSEu1gJY7NCdLR0tLAMoOT
NvTA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:subject:message-id:user-agent:date
:mime-version;
bh=1fAPAhyfsrUs5ys8GTfE2w//twZLn/LwhXUXVmkrhME=;
b=QkxoebHKjuOL+Y/nvnqpcTbFuSj2HivvsaWmkL/xbQf9IcbDz+7tt9ViVaw414jhGI
pEyi3shja+PHFQg6cdsMOSx7l2Ao1MxPeZOU0yfBiOA+DFoYvdVV8iJRXWB5Z1Q7nQUi
lhJ6aa16apnWzR6GvzlTAJZaaF85LtwrVhOEVgGfdwap4md8jOTg07s5EBa6qQ/bXGfT
RwGIErFuB/4FwCRe1uBC4Xuh/KRk63n7k+4JZcpq+EB+zoQE7J3t1ahxyzA2siGfIH04
pP3ek0KZFmouhB0T1pg6wNAVIyHPsAAQZwvYzDf+KxlmaElpf1HB05Q9VKSgfMSlZTc7
612g==
X-Gm-Message-State: ABuFfogYs0GzgFpXYeT1OFSjbsrLAY88CoCdgEKULa60ctQ2QRbNRyqD
+eFfsx+k2tRit8GEWHQWZP4f4lc1OWA=
X-Google-Smtp-Source: ACcGV62L3Mwrs7M430dKlpa4OJ9vrxxbpyen6pIvSZvlhQGiWAisz1HCUvcBNaXXxBhDCdZyJ8WrTA==
X-Received: by 2002:a17:906:5d10:: with SMTP id
g16-v6mr2650055ejt.168.1539269878370;
Thu, 11 Oct 2018 07:57:58 -0700 (PDT)
Received: from localhost ([213.233.148.8])
by smtp.gmail.com with ESMTPSA id q12-v6sm9823462edd.35.2018.10.11.07.57.57
for <bug-gnu-emacs@HIDDEN>
(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
Thu, 11 Oct 2018 07:57:57 -0700 (PDT)
From: "Basil L. Contovounesios" <contovob@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 26.1.50;
thread starvation with async processes and accept-process-output
Message-ID: <87r2gwbmma.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux)
Date: Thu, 11 Oct 2018 15:57:50 +0100
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
recognized.
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -6.0 (------)
--=-=-=
Content-Type: application/emacs-lisp
Content-Disposition: attachment; filename=test.el
Content-Transfer-Encoding: quoted-printable
;;; test.el --- test -*- lexical-binding: t -*-
;;; Code:
(defun test-debug (&rest args)
"Print `current-thread' and ARGS in timestamped `message'."
(push (current-thread) args)
(message "[%s] %s"
(format-time-string "%T:%3N")
(mapconcat #'prin1-to-string args " ")))
(defun test-sentinel (proc msg)
"Kill buffer of PROC when it exits."
(test-debug proc 'sentinel (process-status proc) (substring msg 0 -1))
(and (not (eq (process-status proc) 'run))
(buffer-live-p (process-buffer proc))
(kill-buffer (process-buffer proc))))
(defun test-slave ()
"Create an asynchronous process and wait for it to exit."
(let ((proc (make-process
:name "test"
:buffer (generate-new-buffer " *test*")
:command '("wget" "-qO-" "https://en.wikipedia.org/wiki/Emac=
s")
:connection-type 'pipe
:sentinel #'test-sentinel)))
(while (eq (process-status proc) 'run)
(test-debug proc 'accept-output (accept-process-output proc 5)))
(test-debug proc 'exit (process-status proc) (process-exit-status proc)=
)))
(defun test-master (async)
"Successively run `test-slave'.
With non-nil ASYNC, use a new thread for each invocation."
(dotimes (i 5)
(if async
(thread-join (make-thread #'test-slave (format "test-%d" i)))
(test-slave))))
(defun test-job (async)
"Successively run `test-slave'.
Like `test-master', but avoid blocking interactive sessions by
running `test-master' in its own thread."
(if noninteractive
(test-master async)
(make-thread (apply-partially #'test-master async) "test-master")))
(defun test-threads ()
"Run `test-master' with threads."
(test-job t))
(defun test-no-threads ()
"Run `test-master' without threads."
(test-job nil))
;;; test.el ends here
--=-=-=
Content-Type: text/plain
I attach a sample program test.el whose central function, test-slave,
invokes wget asynchronously before waiting for the process to exit.
The issue I'm facing is that running test-slave twice in succession,
each time in a new thread, causes accept-process-output to hang with no
output (unless a timeout argument is given, in which case the function
returns nil) the second time around.
When this happens, the process sentinel is never called, which is why
I'm assuming accept-process-output is indeed "hanging" in some sense and
it's not just that the process has already exited and so has no further
output.
I could very well be doing or assuming something incorrectly, but what
baffles me is that the "hang" does not occur either when Emacs is run
non-interactively, or when "https://en.wikipedia.org/wiki/Emacs" is
replaced with "https://www.gnu.org/software/emacs/", or when test-slave
is run in the current thread (and not in make-thread).
Since I can reliably reproduce this on both an optimised build of master
and a non-optimised build of emacs-26, I hope to be able to provide
further insights using gdb as time allows. Please let me know if there
are any specific details/output you would like me to provide. As a
relatively inexperienced gdb user I welcome any tips and tricks for
debugging threads and processes.
Here are some ways test.el can be run to illustrate the issue:
# All five processes exit successfully.
emacs -batch -l test.el -f test-no-threads
# All five processes (and threads) exit successfully.
emacs -batch -l test.el -f test-threads
# All five processes exit successfully.
emacs -Q -l test.el -f test-no-threads
# First process and thread exit successfully,
# but accept-process-output starts timing out in second thread.
# Warning: may leave empty wget-log files lying around.
emacs -Q -l test.el -f test-threads
Details of the two Emacs versions I'm using follow:
In GNU Emacs 26.1.50 (build 2, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
of 2018-10-11 built on thunk
Repository revision: a7ebc6bf633bd3849ccab032dad6b1fd31b1ef43
Windowing system distributor 'The X.Org Foundation', version 11.0.12001000
System Description: Debian GNU/Linux testing (buster)
Configured using:
'configure 'CC=ccache gcc' 'CFLAGS=-O0 -g3 -ggdb -gdwarf-4 -pipe'
--config-cache --prefix=/home/blc/.local --program-suffix=26
--enable-checking=yes,glyphs --enable-check-lisp-object-type
--with-mailutils --with-x-toolkit=lucid --with-modules
--with-file-notification=yes --with-x'
Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT
ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS LIBSYSTEMD
LCMS2
In GNU Emacs 27.0.50 (build 21, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
of 2018-10-11 built on thunk
Repository revision: 5bd8cfc14d4b0c78c07e65a583f42a10c4cbc06d
Windowing system distributor 'The X.Org Foundation', version 11.0.12001000
System Description: Debian GNU/Linux buster/sid
Configured using:
'configure --config-cache --prefix=/home/blc/.local --with-mailutils
--with-x-toolkit=lucid --with-modules --with-file-notification=yes
--with-x 'CC=ccache gcc' 'CFLAGS=-O2 -march=native -pipe''
Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT
ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS LIBSYSTEMD
JSON LCMS2 GMP
Thanks,
--
Basil
--=-=-=--
"Basil L. Contovounesios" <contovob@HIDDEN>:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#33018; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.