Received: (at 78773) by debbugs.gnu.org; 15 Jun 2025 17:08:11 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jun 15 13:08:10 2025 Received: from localhost ([127.0.0.1]:59740 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQqq2-0007y5-8q for submit <at> debbugs.gnu.org; Sun, 15 Jun 2025 13:08:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54622) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQqpz-0007xX-FD for 78773 <at> debbugs.gnu.org; Sun, 15 Jun 2025 13:08:08 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQqpt-00058Q-NC; Sun, 15 Jun 2025 13:08:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=wp9YQ6Qsk7IShDkjHXlxDK4zMxxBVGrTr2DmfDuuJ3Y=; b=Z93JXq/dXH2D 2chOsDl2qmy30GauS4MlmBT7t8IF8z2CdORIAM5QeDxDOBtDbTxAjnDNkcakig5OJqu+ErYewWxwg 37MA2IzQ0Z5yO9xUdok9d0ltPcmGyyLt97R9fCeTiFUjx0MN3YCesN0Eck64BbKj+zXQRYtJ7LXWG Mq0hG3SCOFKAhhFDWyjLj47GUr9EUZIdvp6wBWanB3S0ZMYg0CZxtyroC7lUH8JXJoFMBwArflwNT 3SId0kIro8i4/JQcpglStnkQbmsmR6WbC22pH9FDednI1kAb991HbyMfnq7yEMVc6BpHeq4b0Sd1t Z3J8fBV7rA/P6bjRM8qw8Q==; Date: Sun, 15 Jun 2025 20:07:57 +0300 Message-Id: <86msa9orv6.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <87frg1aqqv.fsf@HIDDEN> (message from Steven Allen on Sun, 15 Jun 2025 09:55:52 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <86ikkysgck.fsf@HIDDEN> <87ecvm9u63.fsf@HIDDEN> <86msaaqo3u.fsf@HIDDEN> <87o6uq84mn.fsf@HIDDEN> <86ecvlr3bu.fsf@HIDDEN> <87frg1aqqv.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN, ian@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN, ian@HIDDEN > Date: Sun, 15 Jun 2025 09:55:52 -0700 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> > That could indeed fly, but process-adaptive-read-buffering is non-nil > >> > by default. Does url set it to nil? > >> > >> It is nil by default as of bug#75574 (8a669b6be523e043423b81571a8c94cb49ccc8e5). > > > > So the problem happens only with the master branch, and doesn't exist > > in Emacs 30 and older? > > The problem being discussed here happens in the latest release as well. > I'm simply pointing out that your assertion that > process-adaptive-read-buffering is non-nil by default is incorrect on > the latest master so we're all on the same page. > > Additionally, I should note that adaptive read buffering only applies to > system processes and pipes, not network connections (it's not enabled in > `make-network-process') so it's not relevant to > `url-retrieve-synchronously'. I only bring it up is that I agree that > it's important to preserve its behavior in > `wait_reading_process_output'. > > >> Looking at this more, I'm becoming more and more convinced that this was > >> a bug inadvertently introduced by > >> 12a2691fb254db20607b2067a12b795a6d7c6649. That patch was trying to make > >> adaptive read buffering work again but it went too far and applied the > >> delay everywhere. I've attached what I think is the correct patch, but > >> I've also CCed Ian (author of #20978 and the associated patch) in hopes > >> of getting a bit more context here. > > > > This cannot be TRT, as you have already discovered. > > > > Can we please agree that as long as we think the problem happens due > > to some specific feature of the code, such as > > process-adaptive-read-buffering, the fix should be limited _only_ to > > that feature, and should not affect any other uses? Because that's > > the only way to make changes in this code without risking regressions. > > I don't think the problem happens because of > `process-adaptive-read-buffering'. I think the problem happens because, > in an attempt to make adaptive read buffering work again, an > unconditional 10ms delay was added to `accept-process-output'. I don't disagree, I'm just saying that if the problem happens only when process-adaptive-read-buffering is (or should be) nil, then whatever changes we discuss should not touch the cases where process-adaptive-read-buffering is non-nil. Did you have a chance to investigate how come these small waits add up to a full second? I think understanding that is very important for the decision what fixes should be considered. Thanks.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 15 Jun 2025 16:56:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jun 15 12:56:03 2025 Received: from localhost ([127.0.0.1]:59595 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQqeI-0006yq-Hy for submit <at> debbugs.gnu.org; Sun, 15 Jun 2025 12:56:03 -0400 Received: from fhigh-b3-smtp.messagingengine.com ([202.12.124.154]:44853) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uQqeG-0006xO-20 for 78773 <at> debbugs.gnu.org; Sun, 15 Jun 2025 12:56:00 -0400 Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfhigh.stl.internal (Postfix) with ESMTP id 7B6CC25401A1; Sun, 15 Jun 2025 12:55:54 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Sun, 15 Jun 2025 12:55:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1750006554; x= 1750092954; bh=7yqXLlRM9oBEbqwjkA1pPkKFycmE4+QDGkQ4vNvZOYs=; b=h sRTKsIGBJdB34RDL+nkK5Kl+tb83Oy5dx2QcI2HBXO/KUvaGOhs/DtQKjEKhUI8d jZWu3y+CNjyJhAEVobD1wTTmN1SHltFGHsfIb2kr7XWgaid3jttXxianGwLwngj8 XaRNpyjEO/GOavLzjZGWYMKRl/Vhrimm++mpqgo8k/Agw/GjInTmNT9yC16i3uq4 zK2cMCEUJw6uRuZPi+wvS7oDYiTOlTrj0eELjfkd7EtIh22UUu6lQk5Pp3WG5uO0 wSqiah5lezwmGtHkEURJPA9tCNIagNYL6knbuoiVfqAlvI41T5iBCvjRZ/JCDr2N 7mCSs2i7lmWPhteKtflWA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1750006554; x=1750092954; bh=7yqXLlRM9oBEbqwjkA1pPkKFycmE4+QDGkQ 4vNvZOYs=; b=SzHbjg+4Ual3auc3jrmWOM0XiDbqP/00glEihdWLNMGF7qn1eMv J2QNFVkrsSc6MvLq4TIhQrXS2hPk2XLjGonMDqC1wl4Vp0mQCi2HE5UUNXFuHPZ5 atdVtCXiI8q6y0G1lNSsQfT0KkaRKqDdlFWJZU1B6OBttuq8Kdn0aYFxCNs0MgQA +t7iGdFBABYSH9JO3NdIivxQhkVBv/B54WiZwmX4x0Hxad01ZnPR2se69eTy6Nuq PaITqPEIGkQSbuoCQL3O1ununju1LMrhi1ienb2OSyOcj0rhCAd6dt6TmdORonwt 3zsh5EhtrX4bnR4Jft0NgiU4YTbgj5h8gWw== X-ME-Sender: <xms:GftOaGaTLdTeVTvcEHO0Y-K9E3EsTT8QUsQF64p8m82RZUOXon-Gmw> <xme:GftOaJaC0-dd0Hye15EgvJdEER-vkvEcqlUuzbuqiXqZADhLhp9cEleOGYXeOO8jP Qb7mHru1Pq4WOlvKWY> X-ME-Received: <xmr:GftOaA-2CSpvCtpwxZxYJkS7wvBe-xH4j35erDRNFcURuqegoPya21Mo2YpKp0ODUOoea3bczKaYlQC4fl_RJqZiajYut6E> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddvgedvgecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg hnrdgtohhmqeenucggtffrrghtthgvrhhnpeejffefuddvieefgeevkeeggfelhefhffeg leetffetuedttefggeffheeufeektdenucffohhmrghinhepghhnuhdrohhrghenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsthgvvhgvnhes shhtvggsrghlihgvnhdrtghomhdpnhgspghrtghpthhtohephedpmhhouggvpehsmhhtph houhhtpdhrtghpthhtohepvghlihiisehgnhhurdhorhhgpdhrtghpthhtoheprhhplhhu ihhmsehgmhgrihhlrdgtohhmpdhrtghpthhtohepjeekjeejfeesuggvsggsuhhgshdrgh hnuhdrohhrghdprhgtphhtthhopehlrghrshhisehgnhhushdrohhrghdprhgtphhtthho pehirghnsehirghnkhgvlhhlihhnghdrohhrgh X-ME-Proxy: <xmx:GftOaIpqa7xW_gMcPe5MpVjbrmC1v8O-1RmKR2GsIR4A8fbn9IZX2g> <xmx:GftOaBod8NrRwdYq3uCzvEilK23gSHV2WsnGheO1NitLvezNjfQ5OQ> <xmx:GftOaGRcQNMnBPco8-AdtHrJmmYb-K1YfxSn6IXGo0yQPFBwoAeMNQ> <xmx:GftOaBrANhyFU5RC0v67HARVMKV7lbrhLwFCKv03l8ljXau_wSAgOQ> <xmx:GvtOaITN5s1FEv5DiUww8I5dQ0BhDYQSKsd6oeQWx23x9hHq48prnr6z> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 15 Jun 2025 12:55:53 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <86ecvlr3bu.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <86ikkysgck.fsf@HIDDEN> <87ecvm9u63.fsf@HIDDEN> <86msaaqo3u.fsf@HIDDEN> <87o6uq84mn.fsf@HIDDEN> <86ecvlr3bu.fsf@HIDDEN> Date: Sun, 15 Jun 2025 09:55:52 -0700 Message-ID: <87frg1aqqv.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN, ian@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Steven Allen <steven@HIDDEN> >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN, Ian Kelling >> <ian@HIDDEN> >> Date: Sat, 14 Jun 2025 13:12:00 -0700 >> >> Ian (CCed): I've run into an issue where `accept-process-output' is >> always waiting an extra 10ms (READ_OUTPUT_DELAY_INCREMENT) after >> receiving output as long as the PROCESS passed to it is nil. This >> appears to be related to the change you made in bug#20978 and I'm >> wondering if you can shed more light on when `accept-process-output' >> should wait for more input. > > Yes, Ian's input will be appreciated, especially if he can explain the > rationale for this part of the patch in enough detail for us to > understand what parts of it are essential and which aren't. > >> Eli Zaretskii <eliz@HIDDEN> writes: >> >> >> From: Steven Allen <steven@HIDDEN> >> >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN >> >> Date: Sat, 14 Jun 2025 09:15:00 -0700 >> >> >> >> Eli Zaretskii <eliz@HIDDEN> writes: >> >> >> >> > We wait the second time because there could be more input, so I don't >> >> > see how this could be wrong in general. I also don't quite see the >> >> > "gotcha!" in your analysis below: >> >> >> >> There is no available input. There might be more input in 10ms >> > >> > That's what I meant by "there could be more input". >> >> I'd rather make use of what we know: we don't know if we'll get more >> data in the near future but we *do* know that we've just received some >> data that the caller (of `accept-process-output`) may want to handle. So >> we might as well return to the caller and give them a chance to do >> something about it. > > This would be wrong. In most cases, especially those where > performance matters, sub-process output arrives in many chunks, and > there almost always is another chunk coming very soon. That's how I > understand the rationale for READ_OUTPUT_DELAY_INCREMENT stuff and its > use. Hence my second paragraph: >> The exception to this rule is adaptive read buffering where explicitly >> _don't_ want to yield too little data at a time if we expect we'll >> receive more in the near future. I agree we should wait a bit longer in >> that case (the attached patch should handle that). When adaptive read buffering is enabled, if we read less then 256 bytes from one or more processes, we wait a bit for more output to arrive from these processes (the process_output_skip flag is set to 1 when the adaptive read buffering algorithm kicks in): https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=ebdad09c5a0a822acb52ec58b3319d77d156f0ce#n5646 However, the current code unconditionally waits 10ms when we're not applying the adaptive read algorithm: https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=ebdad09c5a0a822acb52ec58b3319d77d156f0ce#n5679 >> > That could indeed fly, but process-adaptive-read-buffering is non-nil >> > by default. Does url set it to nil? >> >> It is nil by default as of bug#75574 (8a669b6be523e043423b81571a8c94cb49ccc8e5). > > So the problem happens only with the master branch, and doesn't exist > in Emacs 30 and older? The problem being discussed here happens in the latest release as well. I'm simply pointing out that your assertion that process-adaptive-read-buffering is non-nil by default is incorrect on the latest master so we're all on the same page. Additionally, I should note that adaptive read buffering only applies to system processes and pipes, not network connections (it's not enabled in `make-network-process') so it's not relevant to `url-retrieve-synchronously'. I only bring it up is that I agree that it's important to preserve its behavior in `wait_reading_process_output'. >> Looking at this more, I'm becoming more and more convinced that this was >> a bug inadvertently introduced by >> 12a2691fb254db20607b2067a12b795a6d7c6649. That patch was trying to make >> adaptive read buffering work again but it went too far and applied the >> delay everywhere. I've attached what I think is the correct patch, but >> I've also CCed Ian (author of #20978 and the associated patch) in hopes >> of getting a bit more context here. > > This cannot be TRT, as you have already discovered. > > Can we please agree that as long as we think the problem happens due > to some specific feature of the code, such as > process-adaptive-read-buffering, the fix should be limited _only_ to > that feature, and should not affect any other uses? Because that's > the only way to make changes in this code without risking regressions. I don't think the problem happens because of `process-adaptive-read-buffering'. I think the problem happens because, in an attempt to make adaptive read buffering work again, an unconditional 10ms delay was added to `accept-process-output'. (see my first response at the top of this message)
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 15 Jun 2025 05:17:49 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jun 15 01:17:48 2025 Received: from localhost ([127.0.0.1]:51978 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQfkX-0002uq-9z for submit <at> debbugs.gnu.org; Sun, 15 Jun 2025 01:17:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32952) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQfkQ-0002tY-NS for 78773 <at> debbugs.gnu.org; Sun, 15 Jun 2025 01:17:42 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQfkJ-0001SE-5a; Sun, 15 Jun 2025 01:17:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=m8v2SOILg1YSRfOte/cUZQVxkgtHlQrmLTdOgiinhjU=; b=k0RaZXzNjjyO h1jHUjbVnw3GDrxoZO0kOp4IiTP9AqrILIw9+L+HVcCX3mC9t15f+K4DRUHTQxW5JFGCJtu+i5aj8 DoyCwn/e8qnOs4B3C7+thhj2RX3AxyLbRJOjfHl2Rgyznhgve40/1jJz2oBQC25sea0wUdRdII+gI t9Y6X5z1Yg5mx6Nm6GMVsS5pKlLzTxxOQb61pRIv3/59u+fObrdXZi6UsiqBqzAejQY9PkfYtDf6a W+C2q3GGTCJAwlP3H7U308Ew1JOlYruaXRuG4qFvI17NMnNVtwNTpz2ywZlZzwZWrhKOASvXveXwH 9EeUkKQFbVHniv4gyZB91A==; Date: Sun, 15 Jun 2025 08:17:25 +0300 Message-Id: <86ecvlr3bu.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <87o6uq84mn.fsf@HIDDEN> (message from Steven Allen on Sat, 14 Jun 2025 13:12:00 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <86ikkysgck.fsf@HIDDEN> <87ecvm9u63.fsf@HIDDEN> <86msaaqo3u.fsf@HIDDEN> <87o6uq84mn.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN, ian@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN, Ian Kelling > <ian@HIDDEN> > Date: Sat, 14 Jun 2025 13:12:00 -0700 > > Ian (CCed): I've run into an issue where `accept-process-output' is > always waiting an extra 10ms (READ_OUTPUT_DELAY_INCREMENT) after > receiving output as long as the PROCESS passed to it is nil. This > appears to be related to the change you made in bug#20978 and I'm > wondering if you can shed more light on when `accept-process-output' > should wait for more input. Yes, Ian's input will be appreciated, especially if he can explain the rationale for this part of the patch in enough detail for us to understand what parts of it are essential and which aren't. > Eli Zaretskii <eliz@HIDDEN> writes: > > >> From: Steven Allen <steven@HIDDEN> > >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > >> Date: Sat, 14 Jun 2025 09:15:00 -0700 > >> > >> Eli Zaretskii <eliz@HIDDEN> writes: > >> > >> > We wait the second time because there could be more input, so I don't > >> > see how this could be wrong in general. I also don't quite see the > >> > "gotcha!" in your analysis below: > >> > >> There is no available input. There might be more input in 10ms > > > > That's what I meant by "there could be more input". > > I'd rather make use of what we know: we don't know if we'll get more > data in the near future but we *do* know that we've just received some > data that the caller (of `accept-process-output`) may want to handle. So > we might as well return to the caller and give them a chance to do > something about it. This would be wrong. In most cases, especially those where performance matters, sub-process output arrives in many chunks, and there almost always is another chunk coming very soon. That's how I understand the rationale for READ_OUTPUT_DELAY_INCREMENT stuff and its use. > > That could indeed fly, but process-adaptive-read-buffering is non-nil > > by default. Does url set it to nil? > > It is nil by default as of bug#75574 (8a669b6be523e043423b81571a8c94cb49ccc8e5). So the problem happens only with the master branch, and doesn't exist in Emacs 30 and older? > Looking at this more, I'm becoming more and more convinced that this was > a bug inadvertently introduced by > 12a2691fb254db20607b2067a12b795a6d7c6649. That patch was trying to make > adaptive read buffering work again but it went too far and applied the > delay everywhere. I've attached what I think is the correct patch, but > I've also CCed Ian (author of #20978 and the associated patch) in hopes > of getting a bit more context here. This cannot be TRT, as you have already discovered. Can we please agree that as long as we think the problem happens due to some specific feature of the code, such as process-adaptive-read-buffering, the fix should be limited _only_ to that feature, and should not affect any other uses? Because that's the only way to make changes in this code without risking regressions. Thanks.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 15 Jun 2025 00:38:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 20:38:48 2025
Received: from localhost ([127.0.0.1]:48162 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uQbOZ-0001mB-SC
for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 20:38:48 -0400
Received: from fhigh-b1-smtp.messagingengine.com ([202.12.124.152]:38097)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <steven@HIDDEN>)
id 1uQbOU-0001ka-Vy
for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 20:38:45 -0400
Received: from phl-compute-03.internal (phl-compute-03.phl.internal
[10.202.2.43])
by mailfhigh.stl.internal (Postfix) with ESMTP id 173ED2540118;
Sat, 14 Jun 2025 20:38:37 -0400 (EDT)
Received: from phl-mailfrontend-02 ([10.202.2.163])
by phl-compute-03.internal (MEProxy); Sat, 14 Jun 2025 20:38:37 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com;
h=cc:cc:content-type:content-type:date:date:from:from
:in-reply-to:in-reply-to:message-id:mime-version:references
:reply-to:subject:subject:to:to; s=fm3; t=1749947916; x=
1750034316; bh=g7s7p9ZgnHVbr57ZTvOA348/Kdr7f2eEeO2bIKKgvRk=; b=m
YO7PMOVa5NsHYhhlAZayc8Ct2VpXtwmrcuaID8vagwWSE5RLi1fnIXNpmWVpCe2L
5K3Tr6URpBZ9uDPPKd5+x2JIP4ZD25cfVsVqJeAFhvDgeSGJWRLL5RCH1BUzM46A
9+41N3dV2yqkHZ7GmTOgREJXyH2W/HYvHskaH3A/FZRz4UQgj5ufs/gBQcxG7/2d
Oa2RwfVUP6FT3I71RhqAVAS1jozhYxC8ZyWDt41ifYm3IOQf2Kj5cRZWCzhuaUG/
D0wDEUrzjHUvWMUFQ1eL4TLK9pP5Yanv/+LkRMzQXacJDgib8JdMQTR4wVP+wMUQ
UQ3NPuu6o6DbwSnX2guSA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-type:content-type:date:date
:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
:message-id:mime-version:references:reply-to:subject:subject:to
:to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
1749947916; x=1750034316; bh=g7s7p9ZgnHVbr57ZTvOA348/Kdr7f2eEeO2
bIKKgvRk=; b=jyOsJ9E8V210vHouCzS5xQcKm4Uz+9xYsZsfC12iCXsU8m3XkHX
W3OMl4quA4h6XXKAe8wKIT9traTW41QFqLxiAoOtOdqhYUCoK17iGoYVFRcQtZGE
n2OJumQkf72niol/AXsC95x+D77qB8h+pfCGP+H3zS1rewQFsfblw1ajpStscITJ
WHiFTrP1RfdWNkvNjrMa7+IAJSVzigrDJVK559kaeVHzjBbDyoPahH//nbWBvH1G
ZqWqv/MtJ6DRvJE4vphaL6hNKvOcvna5zvae0ZC9Kw0f8SgBzUBxzakCvfvPc5vj
mftM57UAM+fcw/iA+cYdBjVzwkL0HkQ1oKw==
X-ME-Sender: <xms:DBZOaJBchSQrokXdBVjvJKIydVRCxNzc-xWobuFwrgVpC7D7CvdoJQ>
<xme:DBZOaHjlp9qTa4pVCwmgOZhvvKgqsZmS77jKlnal3IGQFlEk1IWia1Rhv6BRCikR8
Inr_-cE9mDnCEuZuK4>
X-ME-Received: <xmr:DBZOaEncpoMoGnl3EjibGaZiP1muDXYcuIMBoCK5EJLQ8k6IYFZ1pcQvLrWOgA785klBAYVeqBSsQmQVTl5xZW4T5yquZx0>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddvvddvlecutefuodetggdotefrod
ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp
uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg
hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesmhdtreertddttden
ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg
hnrdgtohhmqeenucggtffrrghtthgvrhhnpeejudefvdeijeeukedttdegudegffevjeeh
heeiueelgfffhfelffehfeevhfdvgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrh
grmhepmhgrihhlfhhrohhmpehsthgvvhgvnhesshhtvggsrghlihgvnhdrtghomhdpnhgs
pghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepvghlihiise
hgnhhurdhorhhgpdhrtghpthhtoheprhhplhhuihhmsehgmhgrihhlrdgtohhmpdhrtghp
thhtohepjeekjeejfeesuggvsggsuhhgshdrghhnuhdrohhrghdprhgtphhtthhopehlrg
hrshhisehgnhhushdrohhrgh
X-ME-Proxy: <xmx:DBZOaDzNJJ6L5bdEbvci9po98SO5yo_7xjN3L42kjQg0Sb5i-NSjaQ>
<xmx:DBZOaOQOo91RPbAMH4S46Nrd4NohP7QfcHtJ3UrWr7cDwC-Nbj_hMg>
<xmx:DBZOaGbL5NApgbVLxhJeW-Sz8PXF4xffaEVi5EsIaLnw_CPRkbi1ug>
<xmx:DBZOaPTPzei7peZWurrl0A3bg4B5HvViLRoW0EqXhTcGIf4WmZfc4g>
<xmx:DBZOaNxwZrZ9tAQNJlzQZ63ywz3SkD8P4MJ1omWXiD6Xzv9-NWB7TaAj>
Feedback-ID: ie8a146a7:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat,
14 Jun 2025 20:38:36 -0400 (EDT)
From: Steven Allen <steven@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <86frg2qeio.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN>
<87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN>
<8734c3h6xp.fsf@HIDDEN> <86frg2sfp2.fsf@HIDDEN>
<87bjqq9tob.fsf@HIDDEN> <86ldpuqmpg.fsf@HIDDEN>
<871prm9o75.fsf@HIDDEN> <86ikkyqisb.fsf@HIDDEN>
<87v7oy867x.fsf@HIDDEN> <86frg2qeio.fsf@HIDDEN>
Date: Sat, 14 Jun 2025 17:38:34 -0700
Message-ID: <87sek1q1o5.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
--=-=-=
Content-Type: text/plain
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: Steven Allen <steven@HIDDEN>
>> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN
>> Date: Sat, 14 Jun 2025 12:37:38 -0700
>>
>> Eli Zaretskii <eliz@HIDDEN> writes:
>>
>> >> To be clear, calling `accept-process-output' with a nil process is the
>> >> exception, not the norm (8 out of 105 call sites).
>> >
>> > That might be, but in this particular case we changed the call to give
>> > it the nil argument for a reason, and it will be hard to convince me
>> > to go back on that decision.
>>
>> It was changed in 93e1248c2085dfb675d7ed916ec5621e3fe6e2c6 as a blind
>> attempt to fix a hard to reproduce bug. The commit message was:
>>
>> * lisp/url/url.el (url-retrieve-synchronously): Use
>> `accept-process-output' on a null process. That is the safer, more
>> conventional way of achieving non-blocking sleep-for (bug#49897).
>>
>> I understand if you want to leave well enough alone, but I also want to
>> find SOME way to fix this bug. Which leads us to...
>>
>> > So I suggest to explore other ways of solving this first.
>>
>> This is what I've been trying to do. There are only two ways to fix
>> this:
>>
>> 1. Change `accept-process-output' to not unconditionally wait an
>> additional 10ms after receiving output when PROCESS is nil "just in
>> case" there's some additional data to read.
>>
>> 2. Change `url-retrieve-synchronously' pass a non-nil PROCESS to
>> `accept-process-output' thereby avoiding the additional 10ms wait.
>>
>> I've been trying to go down the first path (I think that *is* the
>> correct path) but fixing it will necessarily require
>> `accept-process-output` to return immediately after receiving output
>> when PROCESS is nil, instead of waiting around in case some additional
>> output arrives.
>
> Taking a step back, you never explained how come waiting for 10 msec
> causes some operation to take a full second. Could you please
> describe what happens in more details, so that such a significant
> effect of such a small delay could be understood without knowing what
> emacs-syncthing is or does?
emacs-syncthing makes 36+ RPC calls to the local syncthing process. Each
request takes ~1-2ms on my machine so the 10ms extra delay dominates the
request time.
However, you do bring up a good point as this change (either the first
or second patch) improves performance more than I'd expect:
1. Before this patch, opening the syncthing buffer took 900ms (much
greater than the expected 36*10ms).
2. After this patch, it takes 50ms (expected given 1-2ms for 36 RPC
calls plus the time to draw the buffer).
So it would appear that there's something going on here beyond the 10ms
delay from READ_OUTPUT_DELAY_INCREMENT (more like a 25ms delay). I'll do
some more digging.
> One aspect of this which I'd like to understand is how general is this
> problem and what kind of use patterns will bump into it. Because if
> it turns out that just emacs-syncthing is affected, a simple solution
> would be for emacs-syncthing to have its own variant of
> url-retrieve-synchronously, in which case it can do anything it likes
> with its variant of that function. By contrast, if we want to modify
> the code in Emacs, we need at least to understand what use cases need
> that and consider their importance vs the other kinds, which might
> suffer degradation in performance due to such a change.
This applies to any case where one makes multiple sequential RPC
requests to a local server, waiting on that request via
`accept-process-output' with a nil PROCESS.
However, there appears to be an escape hatch that lets, e.g., jsonrpc
avoid this issue: you can throw an error out of
`accept-process-output'. I've attached a patch that does this and it
appears to work, but throwing from a sentinel seems sketchy.
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
filename=0001-Rewrite-url-retrieve-synchronously-to-throw-when-don.patch
From 2a7c99cef6644984bc7c240f925171ba14dc089c Mon Sep 17 00:00:00 2001
From: Steven Allen <steven@HIDDEN>
Date: Sat, 14 Jun 2025 15:27:22 -0700
Subject: [PATCH] Rewrite url-retrieve-synchronously to throw when done
When called with no PROCESS, `accept-process-output' waits ~10ms after
first receiving process output before returning, adding (at least) 10ms
of latency to any `url-retrieve-synchronously' calls.
This patch avoids this by throwing from the network process sentinel and
catching it in the outer `url-retrieve-synchronously', thereby returning
immediately when the response is ready.
---
lisp/url/url.el | 93 ++++++++++++++++++++++++-------------------------
1 file changed, 45 insertions(+), 48 deletions(-)
diff --git a/lisp/url/url.el b/lisp/url/url.el
index 090a952cf4c..995c97a1a2f 100644
--- a/lisp/url/url.el
+++ b/lisp/url/url.el
@@ -235,58 +235,55 @@ url-retrieve-synchronously
TIMEOUT is passed, it should be a number that says (in seconds)
how long to wait for a response before giving up."
(url-do-setup)
- (let* (;; Ensure we can stop during connection setup (bug#71295).
+ (let* ((tag (gensym "url-retrieve-synchronously-tag"))
+ ;; Ensure we can stop during connection setup (bug#71295).
(url-asynchronous (not (null timeout)))
- data-buffer
- (callback (lambda (&rest _args)
- (setq data-buffer (current-buffer))
- (url-debug 'retrieval
- "Synchronous fetching done (%S)"
- data-buffer)))
+ (throw-on-input nil)
(start-time (current-time))
- (proc-buffer (url-retrieve url callback nil silent
- inhibit-cookies)))
- (if (not proc-buffer)
+ (callback (lambda (&rest _args)
+ (let ((data-buffer (current-buffer)))
+ (url-debug 'retrieval
+ "Synchronous fetching done (%S)"
+ data-buffer)
+ (throw tag data-buffer))))
+ proc-buffer)
+ (catch tag
+ (setq proc-buffer (url-retrieve url callback nil silent
+ inhibit-cookies))
+ (unless proc-buffer
(url-debug 'retrieval "Synchronous fetching unnecessary %s" url)
+ (throw nil))
+
(unwind-protect
- (catch 'done
- (while (not data-buffer)
- (when (and timeout (time-less-p timeout
- (time-since start-time)))
- (url-debug 'retrieval "Timed out %s (after %ss)" url
- (float-time (time-since start-time)))
- (throw 'done 'timeout))
- (url-debug 'retrieval
- "Spinning in url-retrieve-synchronously: nil (%S)"
- proc-buffer)
- (when-let* ((redirect-buffer
- (buffer-local-value 'url-redirect-buffer
- proc-buffer)))
- (unless (eq redirect-buffer proc-buffer)
- (url-debug
- 'retrieval "Redirect in url-retrieve-synchronously: %S -> %S"
- proc-buffer redirect-buffer)
- (let (kill-buffer-query-functions)
- (kill-buffer proc-buffer))
- ;; Accommodate hack in commit 55d1d8b.
- (setq proc-buffer redirect-buffer)))
- (when-let* ((proc (get-buffer-process proc-buffer)))
- (when (memq (process-status proc)
- '(closed exit signal failed))
- ;; Process sentinel vagaries occasionally cause
- ;; url-retrieve to fail calling callback.
- (unless data-buffer
- (url-debug 'retrieval "Dead process %s" url)
- (throw 'done 'exception))))
- ;; Querying over consumer internet in the US takes 100
- ;; ms, so split the difference.
- (accept-process-output nil 0.05)))
- ;; Kill the process buffer on redirects.
- (when (and data-buffer
- (not (eq data-buffer proc-buffer)))
- (let (kill-buffer-query-functions)
- (kill-buffer proc-buffer)))))
- data-buffer))
+ (while t
+ (when (and timeout (time-less-p timeout (time-since start-time)))
+ (url-debug 'retrieval "Timed out %s (after %ss)" url
+ (float-time (time-since start-time)))
+ (throw tag 'timeout))
+ (url-debug 'retrieval
+ "Spinning in url-retrieve-synchronously: nil (%S)"
+ proc-buffer)
+ ;; Querying over consumer internet in the US takes 100
+ ;; ms, so split the difference.
+ (accept-process-output nil 0.05))
+ ;; Cleanup
+ (let ((buf proc-buffer))
+ (while buf
+ ;; If there are still running processes, we must have
+ ;; canceled the request (quit) and/or timed out. Kill the
+ ;; process.
+ (when-let* ((proc (get-buffer-process buf)))
+ ;; Catch & ignore throws from the sentinel (can happen if
+ ;; the process is still live).
+ (catch tag (delete-process proc)))
+ ;; Follow the chain of redirect buffers and clean them up
+ ;; one by one.
+ (setq buf (when-let* ((redirect-buffer
+ (buffer-local-value
+ 'url-redirect-buffer buf)))
+ (let (kill-buffer-query-functions)
+ (kill-buffer proc-buffer))
+ redirect-buffer))))))))
;; url-mm-callback called from url-mm, which requires mm-decode.
(declare-function mm-dissect-buffer "mm-decode"
--
2.49.0
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 20:51:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 16:51:47 2025 Received: from localhost ([127.0.0.1]:45224 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQXqs-0001Lj-SG for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 16:51:47 -0400 Received: from fhigh-b7-smtp.messagingengine.com ([202.12.124.158]:32901) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uQXqq-0001L0-SE for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 16:51:45 -0400 Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfhigh.stl.internal (Postfix) with ESMTP id 4015225400DB; Sat, 14 Jun 2025 16:51:39 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Sat, 14 Jun 2025 16:51:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1749934299; x= 1750020699; bh=HT8HVDSYPgCajXO7RwQdPZdWqvEntj2g5P7DmbWoGXs=; b=Y zZlukpPZt+n7HZiRxmxOVXhIdLPTDHVprQjoCnXQ6T49IHtMvVKKAzxZShluO2UP PhjPMaAz/uEQfVPFeRUwAccQk6/lCtEZrGvfyNwDrPsytmPfMpSzGD+HffTvmdh8 VHn2/GMHQBikjT6WPdo/WDmm/iL0c+gBL987o8Uc2E8MIxZ4qu4oQxyU/LsmMACN 9cDpZScroRAmF6rH0yUThPYVug6mmp+egiWHDybhI2NYWfjSLVZIzil47YG0BE06 s0oagXrBHrn9QhDGU8tvt1zrKwezpJHUhOlhqnu6OKFGqNLrT5W/yHEel/h0tDVX lcNkrtjhElF+yol+v0Cpw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1749934299; x=1750020699; bh=HT8HVDSYPgCajXO7RwQdPZdWqvEntj2g5P7 DmbWoGXs=; b=IRIuixT7tCcnXcmX6MOevCxzv6mrpRIEK6xiPgCyO0y13I/EOii PRKOUl1qwC/5Qx0amauUol3IX8uNsxtbnr3aYjRYuHdwtn+/lHMkXtH3EBisNgLe yAm+4gaBIclN5oJ/AAAE4ifMLF0en4qgQG+ZD+MkAMWHQHhuGyHCnXNK+VTc83Yp FTAXihmLrZCnHMxJN3Z+ST1qmhRQqvD+lPi1sdLrxs6Ekse/BOC5dEzVJR5J5AjF LzEdqw+IFXQt2OekNxO3IlBP4tmwBFIwH71l/p0I5oz9txwrJrmt6O59KRjynRhZ WFdUaiLwGIAoj/50+9EJ3Ort6/LriMvBUaA== X-ME-Sender: <xms:2uBNaDHkfbKC9ScqVtN0pjY1jqP-8VmZxcXwcAnMOqsbSkCf9OVxoQ> <xme:2uBNaAVLwD1gOxzy6sGSnEylUDzz7ONpudGPFQRFQpux1BVBXsT5lFsSQ2x6Qwe0- olykzNWVYyDsA2bU7I> X-ME-Received: <xmr:2uBNaFLHQoTxqYhUVC7tzgnb2eF-DcuXXPMryj41lIKn7fyGxAOR6uVX77cx8FW2ifp7tGuxK9UMwQ4p_lXv6Aibq_bccLa-BalJ> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddvudekgecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg hnrdgtohhmqeenucggtffrrghtthgvrhhnpedvkeehkeegleehheeggfduleektefhhffg ueffteekgedtvdefuddutddtjeejvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehsthgvvhgvnhesshhtvggsrghlihgvnhdrtghomhdpnhgs pghrtghpthhtohephedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepvghlihiise hgnhhurdhorhhgpdhrtghpthhtoheprhhplhhuihhmsehgmhgrihhlrdgtohhmpdhrtghp thhtohepjeekjeejfeesuggvsggsuhhgshdrghhnuhdrohhrghdprhgtphhtthhopehlrg hrshhisehgnhhushdrohhrghdprhgtphhtthhopehirghnsehirghnkhgvlhhlihhnghdr ohhrgh X-ME-Proxy: <xmx:2uBNaBH2_W_00N1re5LWJtgoCmwR1-0es0gBqmZNjGS1RMlULr7moQ> <xmx:2uBNaJXPJKyGnmvotKG7BY0LKS-ytWhslYx78NdUiLMY-1vTfDGFhA> <xmx:2uBNaMPcbnLT1HgF59WgeNEMCW6D6y1tSZHnN7ILNQ5EweMUAQ533A> <xmx:2uBNaI3C7GMDzq1k6lrjzDKs4GZV7thcSUW60lnUapDEh-AzJlGkcQ> <xmx:2-BNaDtl__NPEEvE2Du5E6JQare5xbWpQci50JoGrwoLm4MDQ88hDvZ_> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 14 Jun 2025 16:51:38 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <87o6uq84mn.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <86ikkysgck.fsf@HIDDEN> <87ecvm9u63.fsf@HIDDEN> <86msaaqo3u.fsf@HIDDEN> <87o6uq84mn.fsf@HIDDEN> Date: Sat, 14 Jun 2025 13:51:36 -0700 Message-ID: <87ikkyqc6f.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN, Ian Kelling <ian@HIDDEN> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Steven Allen <steven@HIDDEN> writes: > > Looking at this more, I'm becoming more and more convinced that this was > a bug inadvertently introduced by > 12a2691fb254db20607b2067a12b795a6d7c6649. That patch was trying to make > adaptive read buffering work again but it went too far and applied the > delay everywhere. I've attached what I think is the correct patch, but > I've also CCed Ian (author of #20978 and the associated patch) in hopes > of getting a bit more context here. > > From 8ab170b8ad4718e6d5d12bbc85e9bcda02fffc0d Mon Sep 17 00:00:00 2001 > From: Steven Allen <steven@HIDDEN> > Date: Sat, 14 Jun 2025 10:05:05 -0700 > Subject: [PATCH] Exit accept-process-output early when we get output from any > process > This last patch is definitely buggy (causes random hangs) and didn't make the performance any faster. I'm going to poke at this code a bit more.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 20:12:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 16:12:14 2025
Received: from localhost ([127.0.0.1]:44728 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uQXEZ-0004Cu-P3
for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 16:12:14 -0400
Received: from fhigh-b6-smtp.messagingengine.com ([202.12.124.157]:35485)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <steven@HIDDEN>)
id 1uQXEW-0004BI-I1
for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 16:12:09 -0400
Received: from phl-compute-06.internal (phl-compute-06.phl.internal
[10.202.2.46])
by mailfhigh.stl.internal (Postfix) with ESMTP id 85A12254008E;
Sat, 14 Jun 2025 16:12:02 -0400 (EDT)
Received: from phl-mailfrontend-01 ([10.202.2.162])
by phl-compute-06.internal (MEProxy); Sat, 14 Jun 2025 16:12:02 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com;
h=cc:cc:content-type:content-type:date:date:from:from
:in-reply-to:in-reply-to:message-id:mime-version:references
:reply-to:subject:subject:to:to; s=fm3; t=1749931922; x=
1750018322; bh=tGodLXKsXGkE49m9GDrmkwPnYb0ZccFTV+I5FifZ+AI=; b=k
BCw299J3xUE/CBWMWMpIl/Z6oNbjA7O6iCvGB+4+GjsoWlfClwhQoefs0fk9YREB
FiTT8y9X3ZYIh+3AQ63fA8aIKY5GrzAURqlu8R/8oiBmk0q1gM1aPoHoqcODh1s2
6GuMjdmtujda7685MFU9Ra7LWSwU4nT2Q3nm507pgnuyXr8T89gGQebg+8h/6SoP
I8J8HCZCxSarkm6V1KrKyXSSHNpz95mqtbnv8QQvs1eoqzIo3y5WtbnnpZ5PrqWd
Aw2DmqTz3Hh9FaupMSDcEKZUvSX/EnMBqPA7+Rbb8X2yQ+/JgN7+Q3tbTq5XofWC
ziHzlSpb9yY1mvELBuPxQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-type:content-type:date:date
:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
:message-id:mime-version:references:reply-to:subject:subject:to
:to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
1749931922; x=1750018322; bh=tGodLXKsXGkE49m9GDrmkwPnYb0ZccFTV+I
5FifZ+AI=; b=o+BmSjqVoje5ulgPAsEnCx/GOVj0KLO8akOdUyYzl6DPW8lvGjx
oOX2XXvczLu7Ycih8av5PEQ8ax2PzkEBB5++vvt9JKXxmyzsb7ncqScil7mMZtgW
EYZUTgm/JafoNbbBXi+qs6VchuE868dNXfGbdqPmPDifH4rDLsXfY4HivDs0B8a7
qKC4+LhTt2+fje3cgrPLVR+h0vqo3S0Qtc5cCbQ1xlocX/5qmKNJjZhxEjKWwMv1
z5YS205q4USFTGpEe6eROpvKH6OsefNQtUqHBM5Vv6ShJXuuOmbGwWrqak+5im7/
aZ3lBQ/rh1vjEg4SRIxaPvbZcF4BDKROCzQ==
X-ME-Sender: <xms:kddNaHN3BDVZMlz5cM4mZOIrA-dPrXWiAS3rEInrWC2dn04MH3F63Q>
<xme:kddNaB9KMrbVyebDR4BJTBWXe7XSh1zBUaQJTGMVPaoCn1PogEWDe6sk8TDfYGq5H
ZrWd3hvNawE_knG8hg>
X-ME-Received: <xmr:kddNaGTB9e9Yd6iQeVMpP352nkk20iX9liDv_wnCIphsmsYk7vre1OlAstrT84f5WzWNRacrEyrAYzQieKXYo2Pi2XGIcJhH_n_C>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddvudejhecutefuodetggdotefrod
ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp
uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg
hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesmhdtreertddttden
ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg
hnrdgtohhmqeenucggtffrrghtthgvrhhnpeejudefvdeijeeukedttdegudegffevjeeh
heeiueelgfffhfelffehfeevhfdvgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrh
grmhepmhgrihhlfhhrohhmpehsthgvvhgvnhesshhtvggsrghlihgvnhdrtghomhdpnhgs
pghrtghpthhtohephedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepvghlihiise
hgnhhurdhorhhgpdhrtghpthhtoheprhhplhhuihhmsehgmhgrihhlrdgtohhmpdhrtghp
thhtohepjeekjeejfeesuggvsggsuhhgshdrghhnuhdrohhrghdprhgtphhtthhopehlrg
hrshhisehgnhhushdrohhrghdprhgtphhtthhopehirghnsehirghnkhgvlhhlihhnghdr
ohhrgh
X-ME-Proxy: <xmx:kddNaLubdjz2nZ62Ne4lh3PKOuFscwxYXnmXxr9JygPRLbWcM6pgjA>
<xmx:kddNaPcUEkVtsIyk0sWt4JfTxEfJhSLZHAMDFOJGBKf7B_Lq0gRChQ>
<xmx:kddNaH1zI66buNnhd8wl_IrVsuT0G7jjSTEp6sgMTCeZ50UiD6uACw>
<xmx:kddNaL8LtE7oo3ZaGaX_UsBxcnktkdj5C_68SkKvWcngnhQMdt-NAQ>
<xmx:ktdNaAV8CYGTJA1jGoxo0g472ozip3Fln6mp0KtcAIxyCfQoFkOCZCfP>
Feedback-ID: ie8a146a7:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat,
14 Jun 2025 16:12:01 -0400 (EDT)
From: Steven Allen <steven@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <86msaaqo3u.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN>
<86ikkysgck.fsf@HIDDEN> <87ecvm9u63.fsf@HIDDEN>
<86msaaqo3u.fsf@HIDDEN>
Date: Sat, 14 Jun 2025 13:12:00 -0700
Message-ID: <87o6uq84mn.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN,
Ian Kelling <ian@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
--=-=-=
Content-Type: text/plain
Content-Disposition: inline
Ian (CCed): I've run into an issue where `accept-process-output' is
always waiting an extra 10ms (READ_OUTPUT_DELAY_INCREMENT) after
receiving output as long as the PROCESS passed to it is nil. This
appears to be related to the change you made in bug#20978 and I'm
wondering if you can shed more light on when `accept-process-output'
should wait for more input.
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: Steven Allen <steven@HIDDEN>
>> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN
>> Date: Sat, 14 Jun 2025 09:15:00 -0700
>>
>> Eli Zaretskii <eliz@HIDDEN> writes:
>>
>> > We wait the second time because there could be more input, so I don't
>> > see how this could be wrong in general. I also don't quite see the
>> > "gotcha!" in your analysis below:
>>
>> There is no available input. There might be more input in 10ms
>
> That's what I meant by "there could be more input".
I'd rather make use of what we know: we don't know if we'll get more
data in the near future but we *do* know that we've just received some
data that the caller (of `accept-process-output`) may want to handle. So
we might as well return to the caller and give them a chance to do
something about it.
The exception to this rule is adaptive read buffering where explicitly
_don't_ want to yield too little data at a time if we expect we'll
receive more in the near future. I agree we should wait a bit longer in
that case (the attached patch should handle that).
>> My understanding is that `accept-process-output' is supposed to return
>> immediately when output has been read BOTH when PROCESS is nil and when
>> it is non-nil.
>
> That's true, but wait_reading_process_output is called not just by
> accept-process-output, it is called by several other callers, and in
> that case it is not necessarily true that we need to exit immediately
> upon the first available channel. Keep in mind that reading from a
> subprocess also invokes a filter function, if there is one, so it
> might make sense in some cases to read from more than one source,
> perhaps even from all of the ones that have output ready to be read.
>
>> Looking at the history, it looks like the code in question was changed
>> in bug#20978. However, from what I can tell, we should only apply
>> adaptive read buffering when `process-adaptive-read-buffering' is
>> non-nil. I think we may need to change the check in question to:
>>
>> if (!process_output_skip && (!wait_proc || wait_proc == XPROCESS (proc)))
>> wait = MINIMUM;
>
> That could indeed fly, but process-adaptive-read-buffering is non-nil
> by default. Does url set it to nil?
It is nil by default as of bug#75574 (8a669b6be523e043423b81571a8c94cb49ccc8e5).
> In any case, I think the addition of process-adaptive-read-buffering
> test should only affect the case when wait_proc is zero.
Looking at this more, I'm becoming more and more convinced that this was
a bug inadvertently introduced by
12a2691fb254db20607b2067a12b795a6d7c6649. That patch was trying to make
adaptive read buffering work again but it went too far and applied the
delay everywhere. I've attached what I think is the correct patch, but
I've also CCed Ian (author of #20978 and the associated patch) in hopes
of getting a bit more context here.
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
filename=0001-Exit-accept-process-output-early-when-we-get-output-.patch
From 8ab170b8ad4718e6d5d12bbc85e9bcda02fffc0d Mon Sep 17 00:00:00 2001
From: Steven Allen <steven@HIDDEN>
Date: Sat, 14 Jun 2025 10:05:05 -0700
Subject: [PATCH] Exit accept-process-output early when we get output from any
process
Previously, the caller requested to wait on output from any process (not
a specific process), we'd always end up waiting
10ms (READ_OUTPUT_DELAY_INCREMENT) one final time before returning, even
when adaptive read buffering was disabled.
* src/process.c (wait_reading_process_output): skip adaptive read
buffering delays when we're (a) not reading from any specific process
and (b) adaptive read buffering is disabled. (Bug#78773)
---
src/process.c | 16 ++++++----------
1 file changed, 6 insertions(+), 10 deletions(-)
diff --git a/src/process.c b/src/process.c
index e61ec425f7e..81a4aa32dcf 100644
--- a/src/process.c
+++ b/src/process.c
@@ -5671,13 +5671,12 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
process_output_skip = 0;
}
- /* If we've got some output and haven't limited our timeout
- with adaptive read buffering, limit it. */
- if (got_some_output > 0 && !process_skipped
- && (timeout.tv_sec
- || timeout.tv_nsec > READ_OUTPUT_DELAY_INCREMENT))
- timeout = make_timespec (0, READ_OUTPUT_DELAY_INCREMENT);
-
+ /* If we've got some output and aren't waiting on a process
+ via adaptive read buffering, switch our waiting mode to
+ MINIMUM vacuum up any data remaining to be read then
+ exit. */
+ if (got_some_output > 0 && !process_skipped)
+ wait = MINIMUM;
if (NILP (wait_for_cell) && just_wait_proc >= 0
&& timespec_valid_p (timer_delay)
@@ -5974,9 +5973,6 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
got_some_output = nread;
if (nread > 0)
{
- /* Vacuum up any leftovers without waiting. */
- if (wait_proc == XPROCESS (proc))
- wait = MINIMUM;
/* Since read_process_output can run a filter,
which can call accept-process-output,
don't try to read from any other processes
--
2.49.0
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 20:01:18 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 16:01:18 2025 Received: from localhost ([127.0.0.1]:44715 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQX41-0002HZ-B6 for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 16:01:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55472) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQX3x-0002G7-TR for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 16:01:15 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQX3r-00032K-Mh; Sat, 14 Jun 2025 16:01:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=IXj/sgr/4EuMr7pN4M6jp3SPfuq3Xsd5qi3IOQQI3WM=; b=JTOzreNd7f6D 1KE5p/gMNVNeY8uPW/LIXUN9g+5rS6agLi12ujd6jDoa2rd1DcfReVIKZntXVouXYLv7xQAbO3zKu XBrlQSGc4ztjUYUGsbF+GyjwIpwxwdMK8v0ChauyAYa/xpH6PSV0PnCAWetRSRPwZPyYIer51xN2D ajU6EbM1Gf3cO+UtDfYD4TIYlmDHZbyyJkvNoxzKA7a0sj2/Vf3l4YvpOtHMckn7qVr2s0fnokLxA DqB773ZVpzFHhLZSVR5NYzY+fjqmTB3mqTUYAjt461QpwhOAa4YDmafReS9zDxawcJtRNTG1CPq9L +FARzWwMlvfAAC3EYT0rkw==; Date: Sat, 14 Jun 2025 23:01:03 +0300 Message-Id: <86frg2qeio.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <87v7oy867x.fsf@HIDDEN> (message from Steven Allen on Sat, 14 Jun 2025 12:37:38 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> <87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN> <8734c3h6xp.fsf@HIDDEN> <86frg2sfp2.fsf@HIDDEN> <87bjqq9tob.fsf@HIDDEN> <86ldpuqmpg.fsf@HIDDEN> <871prm9o75.fsf@HIDDEN> <86ikkyqisb.fsf@HIDDEN> <87v7oy867x.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Sat, 14 Jun 2025 12:37:38 -0700 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> To be clear, calling `accept-process-output' with a nil process is the > >> exception, not the norm (8 out of 105 call sites). > > > > That might be, but in this particular case we changed the call to give > > it the nil argument for a reason, and it will be hard to convince me > > to go back on that decision. > > It was changed in 93e1248c2085dfb675d7ed916ec5621e3fe6e2c6 as a blind > attempt to fix a hard to reproduce bug. The commit message was: > > * lisp/url/url.el (url-retrieve-synchronously): Use > `accept-process-output' on a null process. That is the safer, more > conventional way of achieving non-blocking sleep-for (bug#49897). > > I understand if you want to leave well enough alone, but I also want to > find SOME way to fix this bug. Which leads us to... > > > So I suggest to explore other ways of solving this first. > > This is what I've been trying to do. There are only two ways to fix > this: > > 1. Change `accept-process-output' to not unconditionally wait an > additional 10ms after receiving output when PROCESS is nil "just in > case" there's some additional data to read. > > 2. Change `url-retrieve-synchronously' pass a non-nil PROCESS to > `accept-process-output' thereby avoiding the additional 10ms wait. > > I've been trying to go down the first path (I think that *is* the > correct path) but fixing it will necessarily require > `accept-process-output` to return immediately after receiving output > when PROCESS is nil, instead of waiting around in case some additional > output arrives. Taking a step back, you never explained how come waiting for 10 msec causes some operation to take a full second. Could you please describe what happens in more details, so that such a significant effect of such a small delay could be understood without knowing what emacs-syncthing is or does? One aspect of this which I'd like to understand is how general is this problem and what kind of use patterns will bump into it. Because if it turns out that just emacs-syncthing is affected, a simple solution would be for emacs-syncthing to have its own variant of url-retrieve-synchronously, in which case it can do anything it likes with its variant of that function. By contrast, if we want to modify the code in Emacs, we need at least to understand what use cases need that and consider their importance vs the other kinds, which might suffer degradation in performance due to such a change.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 19:37:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 15:37:53 2025
Received: from localhost ([127.0.0.1]:44695 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uQWhL-0006Rj-Gd
for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 15:37:52 -0400
Received: from fhigh-b6-smtp.messagingengine.com ([202.12.124.157]:51505)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <steven@HIDDEN>)
id 1uQWhI-0006QV-Ee
for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 15:37:49 -0400
Received: from phl-compute-02.internal (phl-compute-02.phl.internal
[10.202.2.42])
by mailfhigh.stl.internal (Postfix) with ESMTP id 8880325400CA;
Sat, 14 Jun 2025 15:37:42 -0400 (EDT)
Received: from phl-mailfrontend-01 ([10.202.2.162])
by phl-compute-02.internal (MEProxy); Sat, 14 Jun 2025 15:37:42 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com;
h=cc:cc:content-type:content-type:date:date:from:from
:in-reply-to:in-reply-to:message-id:mime-version:references
:reply-to:subject:subject:to:to; s=fm3; t=1749929862; x=
1750016262; bh=eJl5mfFheJ0QFLwxexgFT1IVzHOWaFXceI8Ph3YW4Ec=; b=b
Q75vds8qhZOgbCpQb1SVAJO+bBNHFPqPqzVxRehTEh34k4Z0M3H03VwttIDuo8Om
f0U7jvo4Rsblhe/bnLdhmwZB6Q1jU5Vba4zhWW9NSDG70i5AY9FCEYWFh7sd7o3C
OuTe/HbPyBUa3e3opU49evSUs8tCBXMBty/MGtPHkqne9pvMONJ0JKIcHTkLVqGu
jcdPQwHzPWVqOy44EDY9EJEpCMP/9F2iWBSZQKPLqAe3IAv9Cn9OPSAQd4HOj8t3
zHotzeBW6RhEyQ/ORzg6sQcIjBeCxRCnJZwSv7xQQy71C7S7E63ZY/hepGLBkq0N
G8eTreoVUm5jZoThA2IoA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-type:content-type:date:date
:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
:message-id:mime-version:references:reply-to:subject:subject:to
:to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
1749929862; x=1750016262; bh=eJl5mfFheJ0QFLwxexgFT1IVzHOWaFXceI8
Ph3YW4Ec=; b=YBCovt+RbjioIZiAi1eJSs4aZ4dE2t3ctiQKIjEfpvA9UrpKkyI
Lf++2VsjqNfHmGvAUTj4UmRT+F4BMQ/OGPpxBOvstO8Ocv2+bZuu3uXxDmAZY0DQ
VQl6xkHoaaVEzmkT7hJoRouBcl5hyq/tDuxtT/XKGJ8CPmSnFQ/gGyiQJ50vsRM0
syJ1c6+Xlu6m6RTw0uTo3U8yOq5fDQanF4A5lv5FhJd/N0dam6BG4daV/T1CiT/N
fLxGV4eTobN5SYMmtKMTCmprtoyl4RhZezxDyfXKpdDgT8kDWgT6nA70GYF+Hpa9
tkMdVbJRgboBysFyKXTNYweHbCsds7RdM1Q==
X-ME-Sender: <xms:hs9NaArJhvY2L8fO9gS3yV3h0rziO4QLxhMWe_W0pYQuF8ITerIQ0A>
<xme:hs9NaGqxMFkLxX0l7QD0KQXHTqDA82PX3KFDKRhSqAgkNw1_Oyu3bWh2qNosSUmnw
dFdp3UvHavGatrdVTw>
X-ME-Received: <xmr:hs9NaFMeOSrJR6jZ9QXQK6x2GK-PsmmR4oxw5pSSrTkaBcErh-iQBs6Ir8mRx35GnBAXSrVUe6PlrHW5IcsaRElrMFRqz_dgPVrZ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddvudeilecutefuodetggdotefrod
ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp
uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg
hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden
ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg
hnrdgtohhmqeenucggtffrrghtthgvrhhnpedvkeehkeegleehheeggfduleektefhhffg
ueffteekgedtvdefuddutddtjeejvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrh
grmhepmhgrihhlfhhrohhmpehsthgvvhgvnhesshhtvggsrghlihgvnhdrtghomhdpnhgs
pghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepvghlihiise
hgnhhurdhorhhgpdhrtghpthhtoheprhhplhhuihhmsehgmhgrihhlrdgtohhmpdhrtghp
thhtohepjeekjeejfeesuggvsggsuhhgshdrghhnuhdrohhrghdprhgtphhtthhopehlrg
hrshhisehgnhhushdrohhrgh
X-ME-Proxy: <xmx:hs9NaH7n935_CBwj7zI-Rg1CxWnSrx5Kzq4gMKpRZKB6HY71DQYepQ>
<xmx:hs9NaP6I9LHSZFEq8EPOPZIDlAQT4x_2_ne0ob-taO8vn8XqgAXzhw>
<xmx:hs9NaHgkQYc9QzvsQ9YnNyvS_mKVC7FotWVABDeySnZPdmtdDPkm9w>
<xmx:hs9NaJ7ovT6y9LlliYrdW-ckEYzBNd_Eto2jCcmFj6iFj_-K5TCZEg>
<xmx:hs9NaNZ5BKNm7ymxQbzZGuFaJ124ZT_Y6NbU8ASlIwhb3IIPcpSgaw-u>
Feedback-ID: ie8a146a7:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat,
14 Jun 2025 15:37:41 -0400 (EDT)
From: Steven Allen <steven@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <86ikkyqisb.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN>
<87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN>
<8734c3h6xp.fsf@HIDDEN> <86frg2sfp2.fsf@HIDDEN>
<87bjqq9tob.fsf@HIDDEN> <86ldpuqmpg.fsf@HIDDEN>
<871prm9o75.fsf@HIDDEN> <86ikkyqisb.fsf@HIDDEN>
Date: Sat, 14 Jun 2025 12:37:38 -0700
Message-ID: <87v7oy867x.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: Steven Allen <steven@HIDDEN>
>> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN
>> Date: Sat, 14 Jun 2025 11:23:58 -0700
>>
>> Eli Zaretskii <eliz@HIDDEN> writes:
>>
>> > No, I mean other callers of the same low-level functionality, for
>> > example, url-retrieve.
>> >
>> > Once again, the C function where you propose changes is used in a wide
>> > variety of scenarios, and several clients of it could be active at the
>> > same time.
>>
>> In that case, `accept-process-output' will process output for those
>> other processes as usual (calling filters/sentinels, etc.) but won't
>> return call until we have data that's relevant to this call to
>> `url-retrieve-synchronously'. We haven't set JUST-THIS-ONE in this call
>> to `accept-process-output' so we won't simply *ignore* all other process
>> output.
>>
>> To be clear, calling `accept-process-output' with a nil process is the
>> exception, not the norm (8 out of 105 call sites).
>
> That might be, but in this particular case we changed the call to give
> it the nil argument for a reason, and it will be hard to convince me
> to go back on that decision.
It was changed in 93e1248c2085dfb675d7ed916ec5621e3fe6e2c6 as a blind
attempt to fix a hard to reproduce bug. The commit message was:
* lisp/url/url.el (url-retrieve-synchronously): Use
`accept-process-output' on a null process. That is the safer, more
conventional way of achieving non-blocking sleep-for (bug#49897).
I understand if you want to leave well enough alone, but I also want to
find SOME way to fix this bug. Which leads us to...
> So I suggest to explore other ways of solving this first.
This is what I've been trying to do. There are only two ways to fix
this:
1. Change `accept-process-output' to not unconditionally wait an
additional 10ms after receiving output when PROCESS is nil "just in
case" there's some additional data to read.
2. Change `url-retrieve-synchronously' pass a non-nil PROCESS to
`accept-process-output' thereby avoiding the additional 10ms wait.
I've been trying to go down the first path (I think that *is* the
correct path) but fixing it will necessarily require
`accept-process-output` to return immediately after receiving output
when PROCESS is nil, instead of waiting around in case some additional
output arrives.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 18:29:09 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 14:29:08 2025 Received: from localhost ([127.0.0.1]:43675 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQVcq-0004yY-00 for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 14:29:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58562) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQVcl-0004wa-N6 for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 14:29:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQVcf-0001YJ-QK; Sat, 14 Jun 2025 14:28:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=0WYLpoKJNLKFBs5rLNv/QnraNFjmYNZMvjwMnrpwmoo=; b=niD7Uzo0QC9R 0zt0UhWz990I/36SQLnfo2CHWYGrB9YCSatc9mogvmWKvvISfhmsznck4zM+DBKsWc5ULVOtSq0em 14rsE9qu7z9sdS7Vyjd4HGgMJ49A83gTEne0Ev9+qlDV4RX34XFjNPmKyISSkHuB6RnRBz3r3ChAH CJVDxwMH7RC8E+VWKMIu575ZZ/Zn8e8mFxwD/tRW9Ill2ob680TiqCjMdYoeXeQohBCmfBevMLuRd YpKzThyQfg8SCfcC4NlITDIP509266vpXvowQd+hEJH02sg8HhIMLrADSprZAu4GBDzyEhhHhXFZ5 Uxpgu3DJGBR5jaEVqUza0g==; Date: Sat, 14 Jun 2025 21:28:52 +0300 Message-Id: <86ikkyqisb.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <871prm9o75.fsf@HIDDEN> (message from Steven Allen on Sat, 14 Jun 2025 11:23:58 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> <87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN> <8734c3h6xp.fsf@HIDDEN> <86frg2sfp2.fsf@HIDDEN> <87bjqq9tob.fsf@HIDDEN> <86ldpuqmpg.fsf@HIDDEN> <871prm9o75.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Sat, 14 Jun 2025 11:23:58 -0700 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > No, I mean other callers of the same low-level functionality, for > > example, url-retrieve. > > > > Once again, the C function where you propose changes is used in a wide > > variety of scenarios, and several clients of it could be active at the > > same time. > > In that case, `accept-process-output' will process output for those > other processes as usual (calling filters/sentinels, etc.) but won't > return call until we have data that's relevant to this call to > `url-retrieve-synchronously'. We haven't set JUST-THIS-ONE in this call > to `accept-process-output' so we won't simply *ignore* all other process > output. > > To be clear, calling `accept-process-output' with a nil process is the > exception, not the norm (8 out of 105 call sites). That might be, but in this particular case we changed the call to give it the nil argument for a reason, and it will be hard to convince me to go back on that decision. So I suggest to explore other ways of solving this first.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 18:24:11 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 14:24:11 2025 Received: from localhost ([127.0.0.1]:43566 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQVY2-0004KL-Ou for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 14:24:11 -0400 Received: from fout-b7-smtp.messagingengine.com ([202.12.124.150]:52327) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uQVY0-0004Jp-DA for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 14:24:09 -0400 Received: from phl-compute-05.internal (phl-compute-05.phl.internal [10.202.2.45]) by mailfout.stl.internal (Postfix) with ESMTP id 5055411400C4; Sat, 14 Jun 2025 14:24:02 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Sat, 14 Jun 2025 14:24:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1749925442; x= 1750011842; bh=oW/9/UdA3/8cdYA546eWz/LWTNCpLvJm5hXcjdOUtys=; b=R nIcQIwEU5+Y0lvTzUSmkgIbJSSUpHGshoP0GBM8KlZ7+kP2AME1gKZdyflmFi5uc vKMZvZcWC5+tWmfNn/c8gc4GB9f4+O9JaJJbeYy9fFBm9Ahh+TMMxydbBuGFQ4jO CGAK1P6kBAp0ywl4z949jUS/LcDEgIsLYnOt6dmkmhIlWxK15/YL5eQQONEG5TEg LW0pQ0lcGz3m3Jxm3cTsLA11Fx1nVT9HkeYwBdosc4FlDlpGrjgeiarG0jU5W54x IvgTl0u2BlpnVnQew/GNyDpj1EFQ6hI8fnEwCbBdA2+trOZnWGEz+tENzvoio7Ba AmX4j/ncuCvSx1K/VtPeg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1749925442; x=1750011842; bh=oW/9/UdA3/8cdYA546eWz/LWTNCpLvJm5hX cjdOUtys=; b=RuBNNiCGGQSIsBlSwmY9k66G0owWs2BSX6UwXtKbN+NoXQMw4kY x8WQf/vAqA7ySZbveUVgntG1W+J5B1dPgejan1OU+l1VEK2Q2F+HlYCKiWpgasJ3 zJYDURxtlqxbogOdse6TBjdk9Ozpdc13SbdCpCu4alfAu6Q4C943oIwUTdD+xMCe APDsj+U9hkw3aBWqVgSnowjErhzxtlOyU6HWSoOYncu/GT61Tt2UdzpayPF3fcr6 v+OuABqLWyVOQMdkTidERGC5bIXWFK4qCUN+L/76OC/x2fto54LB1tULWQOzuoT8 OzFr/gu7Jw2KkY8JoWLFxzFOz2c6onS06/Q== X-ME-Sender: <xms:Qb5NaM_dZRxldNZlCN5T6P69Wie9rIC9Xx10DtmvfLlje_QkYbUxZg> <xme:Qb5NaEuHceHjKn4eHimJ5uCmD1EJ72tdE5SjutkNyE9Ora_CQRJ-t55v7zIdBg0An x742hZzMh2LhmihmRI> X-ME-Received: <xmr:Qb5NaCBVaLf4PkvUdvTpczi6SijYsJw9h-8ggNjT910K0AJiUlpBvqcE-AJ1sra_oh12yr261nbrRjuFFbkMik6-c_5h-4ClHMs3> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddvudehgecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg hnrdgtohhmqeenucggtffrrghtthgvrhhnpedvkeehkeegleehheeggfduleektefhhffg ueffteekgedtvdefuddutddtjeejvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehsthgvvhgvnhesshhtvggsrghlihgvnhdrtghomhdpnhgs pghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepvghlihiise hgnhhurdhorhhgpdhrtghpthhtoheprhhplhhuihhmsehgmhgrihhlrdgtohhmpdhrtghp thhtohepjeekjeejfeesuggvsggsuhhgshdrghhnuhdrohhrghdprhgtphhtthhopehlrg hrshhisehgnhhushdrohhrgh X-ME-Proxy: <xmx:Qb5NaMdqSlIkJ9_6W_9jsddVqLfKz8mQizWqgUqevRKjomW7OL9lZQ> <xmx:Qb5NaBMFViNFSXixFB0LnJcXobqVg_LgjcWD_V80Seuir2rG2N9Dkg> <xmx:Qb5NaGnXmfyNbTnXiqkix_LH8pQLf0Qhnfi0jV7e6OtrRFgOw_956w> <xmx:Qb5NaDufctow2OaO2OEREpOicvb-sLP-qMDyRhflSiP_5I4SPnNCJQ> <xmx:Qr5NaNtW3rSQ6wyLJl4Cg3LKx-aixifEmeU2Mc71zvMdRlEls1j3bxYp> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 14 Jun 2025 14:24:01 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <86ldpuqmpg.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> <87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN> <8734c3h6xp.fsf@HIDDEN> <86frg2sfp2.fsf@HIDDEN> <87bjqq9tob.fsf@HIDDEN> <86ldpuqmpg.fsf@HIDDEN> Date: Sat, 14 Jun 2025 11:23:58 -0700 Message-ID: <871prm9o75.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Steven Allen <steven@HIDDEN> >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN >> Date: Sat, 14 Jun 2025 09:25:40 -0700 >> >> Eli Zaretskii <eliz@HIDDEN> writes: >> >> > What if url-retrieve was called for more than a single connection? >> > Won't your FIRST patch cause a regression in that case? >> >> I assume you're talking about redirects here? That's the only >> multiple-process case I know about. In this case, it'll just return back >> to the top of the `url-retrieve-synchronously' loop (same as it would >> before) and wait on the next process. > > No, I mean other callers of the same low-level functionality, for > example, url-retrieve. > > Once again, the C function where you propose changes is used in a wide > variety of scenarios, and several clients of it could be active at the > same time. In that case, `accept-process-output' will process output for those other processes as usual (calling filters/sentinels, etc.) but won't return call until we have data that's relevant to this call to `url-retrieve-synchronously'. We haven't set JUST-THIS-ONE in this call to `accept-process-output' so we won't simply *ignore* all other process output. To be clear, calling `accept-process-output' with a nil process is the exception, not the norm (8 out of 105 call sites).
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 17:04:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 13:04:24 2025 Received: from localhost ([127.0.0.1]:42188 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQUIp-0001f2-Nk for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 13:04:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50682) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQUIn-0001de-7k for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 13:04:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQUIh-0000B5-4a; Sat, 14 Jun 2025 13:04:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=l/7CxTBa1NMrn+bZu6w4yW/58toob7u9qQn0uC/BIrk=; b=P05U4Oq5v2kf nUJFDHlqmINbr+fNLTQtV6Bq0EdDQtBTKrG8QnnnTUhEff4aiBj7ZHGWxX5qYVuXwgpqijkDzB8n3 MtQxWwypLLOAWlf8VlMaHh9lXmi6V2tYBThClqExhQWP5lCw5+K8EYAmBTFpzKJzIJdfRtHrVU36T /CvjBrQguPppb64CqhADofovFDODO3MmyR2DW820SWFCJaOt9mVOxfgerea5UzoAxp7lNzmeciRMU 2oOsFAIl73U1rkGFW/xLonO9DIxcFIbJBCd0S8+z7WKD64/G/L1AjMtSv1m9hv01uLxWjUo5u/Rjz ihrDeBLKejXfn7F3v4rhqA==; Date: Sat, 14 Jun 2025 20:04:11 +0300 Message-Id: <86ldpuqmpg.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <87bjqq9tob.fsf@HIDDEN> (message from Steven Allen on Sat, 14 Jun 2025 09:25:40 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> <87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN> <8734c3h6xp.fsf@HIDDEN> <86frg2sfp2.fsf@HIDDEN> <87bjqq9tob.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Sat, 14 Jun 2025 09:25:40 -0700 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > What if url-retrieve was called for more than a single connection? > > Won't your FIRST patch cause a regression in that case? > > I assume you're talking about redirects here? That's the only > multiple-process case I know about. In this case, it'll just return back > to the top of the `url-retrieve-synchronously' loop (same as it would > before) and wait on the next process. No, I mean other callers of the same low-level functionality, for example, url-retrieve. Once again, the C function where you propose changes is used in a wide variety of scenarios, and several clients of it could be active at the same time.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 16:34:10 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 12:34:10 2025 Received: from localhost ([127.0.0.1]:42151 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQTpY-00058Z-KH for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 12:34:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54340) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQTpU-00056n-UB for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 12:34:06 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQTpP-0005So-9V; Sat, 14 Jun 2025 12:33:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=jWYsTJeMR9drOuYLQND2Ondt2n1Hv1ubQkIjv1gvNf8=; b=J45bJGMnXzFQ hWsVA1fq+ZLSiDHZh06YgxgrM6R9/EfF+a36kWsYyq3a7LGdMoP3Z2/QpyS9MB43dSEAAT+OKtGxL 5rcaYCtZD0CexsbczmOsjgAOHISaOs2CZ8hAJXBaK5+5hTtOJebr8M3BxppfpCyjzgk++i1af+INM F2anU+cJac2t3RTRP7Nx4awwYjRivGuyyQkOuaAcl1QWiKXhDSR/1kkOh5oNYSOOa4nJrEuMXG24/ k6hnWHcnpc7PbU4vlPq5JhAb4EHoAnOns3WL2Iq9I9/ZXnibCp3bcXPWv+Ovz22Lt1W0272IGMtyT s8qGD11ywku882E0OYECWw==; Date: Sat, 14 Jun 2025 19:33:57 +0300 Message-Id: <86msaaqo3u.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <87ecvm9u63.fsf@HIDDEN> (message from Steven Allen on Sat, 14 Jun 2025 09:15:00 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <86ikkysgck.fsf@HIDDEN> <87ecvm9u63.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Sat, 14 Jun 2025 09:15:00 -0700 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > We wait the second time because there could be more input, so I don't > > see how this could be wrong in general. I also don't quite see the > > "gotcha!" in your analysis below: > > There is no available input. There might be more input in 10ms That's what I meant by "there could be more input". > My understanding is that `accept-process-output' is supposed to return > immediately when output has been read BOTH when PROCESS is nil and when > it is non-nil. That's true, but wait_reading_process_output is called not just by accept-process-output, it is called by several other callers, and in that case it is not necessarily true that we need to exit immediately upon the first available channel. Keep in mind that reading from a subprocess also invokes a filter function, if there is one, so it might make sense in some cases to read from more than one source, perhaps even from all of the ones that have output ready to be read. > Looking at the history, it looks like the code in question was changed > in bug#20978. However, from what I can tell, we should only apply > adaptive read buffering when `process-adaptive-read-buffering' is > non-nil. I think we may need to change the check in question to: > > if (!process_output_skip && (!wait_proc || wait_proc == XPROCESS (proc))) > wait = MINIMUM; That could indeed fly, but process-adaptive-read-buffering is non-nil by default. Does url set it to nil? In any case, I think the addition of process-adaptive-read-buffering test should only affect the case when wait_proc is zero.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 16:25:51 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 12:25:51 2025 Received: from localhost ([127.0.0.1]:42111 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQThW-0003rS-J9 for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 12:25:51 -0400 Received: from fhigh-b4-smtp.messagingengine.com ([202.12.124.155]:54835) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uQThT-0003qu-Qr for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 12:25:48 -0400 Received: from phl-compute-03.internal (phl-compute-03.phl.internal [10.202.2.43]) by mailfhigh.stl.internal (Postfix) with ESMTP id 2B2C82540108; Sat, 14 Jun 2025 12:25:42 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-03.internal (MEProxy); Sat, 14 Jun 2025 12:25:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1749918342; x= 1750004742; bh=MITjktcqlpChF5fZSKAaW/EJvOrnKivSx11+LHvKHxY=; b=T Wpqt0FedT7Bq63GRWwD0AbU4ZGnL2f4opBaVUQMTlaAxWd47MKKQPWFEBe3/9iFx Paar/IH5lRdBEqsCuBRd+Itkq1g6laceN6K3HojkNyi02PGgHR3ePFLMeXWEATK2 ekzbVEGl7Er4Ygt4nBxtNuPwvlmVS8wtCXdJ3VyLY0iBagiK+LxrnvYUQ2krbdFk o/y2oYqw4XsLGfGtuE4csOkI8p1mUoSLCtYMmSGXuy0+GubDmRGJL7OpmDGa6tiO PuTZQHk0PDQUbYnDKfvczEi8W2M3dqz3WjsZp3AQwtn9BILqe0iIAEpQswyXcf2N 80t+NZeqN4UVdztj65J3A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1749918342; x=1750004742; bh=MITjktcqlpChF5fZSKAaW/EJvOrnKivSx11 +LHvKHxY=; b=CaynswSs5iuxR/hBwMgiT0wSZAkeJymhxaWjQ8g2Bc00i0Gqs31 ionCS81Uc0QriWljKxiBlgj32S0e8SUUvDKcxHnIcsBP1ujtj2aXplYihlpw6ZJp IM21VNceNCF1Zo3oEMu4fvjf2ogwme4q4v4+KSqb+3eGzJAI3pHjnKl45xkifSc2 ilvQSzHVuz1TUOJ8Mo47GbMlzM6NUjUhZbjWGGkYJGmALIg9j5k7V/zRFE7xpGLk mwEjk7TM4J7fsYnjOn4DmMCJJ9kRcLirq7w62W5DIP34EYvSI8bJPxey3EanIcwB vBN1ejBHK7AorHmJrGTp11X5Lng2Sq5riiQ== X-ME-Sender: <xms:haJNaEAMwBJr9qD7AXFngxIDTjHq5zxKVTW9WTr2WVt_7X5QELcDdg> <xme:haJNaGjTvJlDi-qwhh8YAk_TTytwbiDkjMp-XUu--8b8G0iYKXhjjje6QDsE-B_tO tlqk3ylGEsax1tsCwQ> X-ME-Received: <xmr:haJNaHnLlSa9EQLoshsM1xHsWVGd4eovFH7qTjpWEIJnPddp5sbpMzY9UhPxaC3GKowZiv_m8b2AZisuGBJgB86gYs6plrM> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddvudeftdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg hnrdgtohhmqeenucggtffrrghtthgvrhhnpedvkeehkeegleehheeggfduleektefhhffg ueffteekgedtvdefuddutddtjeejvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehsthgvvhgvnhesshhtvggsrghlihgvnhdrtghomhdpnhgs pghrtghpthhtohepgedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepvghlihiise hgnhhurdhorhhgpdhrtghpthhtoheprhhplhhuihhmsehgmhgrihhlrdgtohhmpdhrtghp thhtohepjeekjeejfeesuggvsggsuhhgshdrghhnuhdrohhrghdprhgtphhtthhopehlrg hrshhisehgnhhushdrohhrgh X-ME-Proxy: <xmx:haJNaKzEBcmj_zcrQGSdGd1Q3EjcG1BHIukgeux4RzE47Qfzj0qyXA> <xmx:haJNaJQJ-3iKegL6pttPP5KKs2Jv3JbzUnPI93VgRRh4fgyekWm8bA> <xmx:haJNaFao1Lq9eXN0j3clCGbnPvkljKhuIfMgu5taEJVUANaEZ_c_0A> <xmx:haJNaCSCrtE-PBD6cpiD5XrM5aurP0cV5hElnKHFTueII_tFOxpvHQ> <xmx:hqJNaIycR9lQDu9rREwOJFGj3Z7E9jBfa0LwY_cGNKyCI0rOTqN5__yR> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 14 Jun 2025 12:25:41 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <86frg2sfp2.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> <87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN> <8734c3h6xp.fsf@HIDDEN> <86frg2sfp2.fsf@HIDDEN> Date: Sat, 14 Jun 2025 09:25:40 -0700 Message-ID: <87bjqq9tob.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Steven Allen <steven@HIDDEN> >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN >> Date: Fri, 13 Jun 2025 10:45:06 -0700 >> >> First, the actual bug: When called with no PROCESS, >> `accept-process-output' would wait an additional 10ms >> (READ_OUTPUT_DELAY_INCREMENT) after successfully reading data from some >> (any) process. `url-retrieve-synchronously' was slow for localhost calls >> because 10ms is a long time for fast requests. This bug would add 10ms >> of latency to every request. > > But it's an unneeded delay only if there's no more output from the > subprocess during those 10 ms, isn't that so? And we don't really > know if all the process output was read, do we? (discussed in the other email) >> My SECOND patch (0772d96d, the one to process.c) fixes this >> underlying bug by setting the timeout to 0 in this case: when we manage >> to read data from some process and aren't waiting on any specific >> process. > > But what if we are also waiting for output from other processes? Your > patch will slow those down, won't it? In you case, you only care > about a single process, but that's not so in general. (discussed in the other email) >> Next steps: >> >> 1. The SECOND patch fixes the actual bug and should be installed to fix >> the underlying bug (assuming I correctly diagnosed the problem/solution, >> but I'm pretty sure I did). > > I disagree for now. I need to understand better why you think it's > TRT in the general case. (discussed in the other email) >> 2. Personally, I would install the FIRST patch as well as it avoids >> returning to `url-retrieve-synchronously' until we've actually read >> relevant data. > > What if url-retrieve was called for more than a single connection? > Won't your FIRST patch cause a regression in that case? I assume you're talking about redirects here? That's the only multiple-process case I know about. In this case, it'll just return back to the top of the `url-retrieve-synchronously' loop (same as it would before) and wait on the next process.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 16:15:15 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 12:15:15 2025 Received: from localhost ([127.0.0.1]:41875 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQTXG-0002ab-3U for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 12:15:14 -0400 Received: from fout-b6-smtp.messagingengine.com ([202.12.124.149]:45009) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uQTXB-0002Vp-66 for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 12:15:11 -0400 Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfout.stl.internal (Postfix) with ESMTP id 414AA11400B9; Sat, 14 Jun 2025 12:15:03 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-01.internal (MEProxy); Sat, 14 Jun 2025 12:15:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1749917703; x= 1750004103; bh=sdGgE/6u+RnYK/rvA46KYPBFQu4fHdpBdqs9wapcbyY=; b=E bwLIB+uYbjRSeKGRTamcSh00nAJrNAfpJwaWXzMUSMT/zWmPWnVZHPHqq5k+zYVL wOGyo+8WsmuHjZTw32L0NFkkW1OoyF4SUmInYiuspIH7nfwzoEUY5ta668M8pxZk 47+3N21IN3TPnCvqpvqj9OO3AeagTyzAoPcfnCZ1H64I7UU+37jmJL246MSsxwvY eKq0reVsUiuACpHz9s1zM9IpT81k9kOOULpp6uQ/2O8kb8ao3XHEoQwL8MCQ2GBj 5ZQf31lu+ersVGN3Zs2skCZy8f2DsIBcgD9NozBypF+l0HoMyab54aps4yG9lsDO 4DwasFj9M+5Y2ntcy8iSQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1749917703; x=1750004103; bh=sdGgE/6u+RnYK/rvA46KYPBFQu4fHdpBdqs 9wapcbyY=; b=g1EX53yQmM/EhhXnoyw7xcHjgHtrMwcp/L00HBpIoHqYm5eSAyw NhbRfkODQrhVu55Bv5YvFz7f2qAbFtmybpvShtT5YNP+IJia4JB59GB6p9WAc4+D TNSBtgoq1JPEIfmX0+XYHOPd/ihy+/dQr6fj02FE/aizy9E+hBbkwMxuZW0b8BWt cIczArORI0Raz2L59Lch/R7eTD0EW8w2sf5OHGMy1Lag0gMRNLyI+it54XJcizIz Jf/jL6Dgs6wXaGXN3ORJ1dqR472Gd+JaCdlz2wZByDkAkUwp2ldbXLDaEcqZws/i F95OZdxBCxdGk05m2WsTX5PSP36gTci4k5g== X-ME-Sender: <xms:BqBNaCvhWZPYkZi--XPCihlADc9cSQBchFck5nHMWRO5hlPfWFLCuA> <xme:BqBNaHedK6PMl8ETghmJG_cTNwxkr1wWi6TVRn-nQvX0aoA1v57uVnHyzH7elJW4b KLQ9NqY05c5epYKj0U> X-ME-Received: <xmr:BqBNaNxNH_zGp_toJlvV4UL5sRXVbcFk7tnlcTee7eKIhf-DaE1j2qr5hSGItR189Lqn3tKFAprU-jiFbt0BNsU_kV7H-ZY> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddvuddvkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg hnrdgtohhmqeenucggtffrrghtthgvrhhnpeejffefuddvieefgeevkeeggfelhefhffeg leetffetuedttefggeffheeufeektdenucffohhmrghinhepghhnuhdrohhrghenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsthgvvhgvnhes shhtvggsrghlihgvnhdrtghomhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtph houhhtpdhrtghpthhtohepvghlihiisehgnhhurdhorhhgpdhrtghpthhtoheprhhplhhu ihhmsehgmhgrihhlrdgtohhmpdhrtghpthhtohepjeekjeejfeesuggvsggsuhhgshdrgh hnuhdrohhrghdprhgtphhtthhopehlrghrshhisehgnhhushdrohhrgh X-ME-Proxy: <xmx:BqBNaNM6-0zTbMcU3TJd4NRWhi953TNx7Mzoe4z8AiRrTCOlk_g1gA> <xmx:BqBNaC9ime5S_JUQ68EyFEvwm3DFTNFBTzHx7t02bQj4oR23CnRjoA> <xmx:BqBNaFWkvysqCAjtH89tEc7yYJRa4uQ_ETUhxN5dthlbnzmQRUAtFw> <xmx:BqBNaLfnNnUDtjHl9OoOqcUWIGObEFVs8hFj_z6bDruohtChMxKyKA> <xmx:B6BNaDel2Y4ngPGJh047ORtAzlUK1-vABuP8Aji8WpOG0LLvzAyK6eeA> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 14 Jun 2025 12:15:02 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <86ikkysgck.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <86ikkysgck.fsf@HIDDEN> Date: Sat, 14 Jun 2025 09:15:00 -0700 Message-ID: <87ecvm9u63.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Steven Allen <steven@HIDDEN> >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN >> Date: Thu, 12 Jun 2025 13:54:01 -0700 >> >> Ok, I think I've figured it out. We loop twice and wait on a timeout >> (READ_OUTPUT_DELAY_INCREMENT) the second loop because we're already done >> reading. > > We wait the second time because there could be more input, so I don't > see how this could be wrong in general. I also don't quite see the > "gotcha!" in your analysis below: There is no available input. There might be more input in 10ms, but we do know that there's no more available right now because we poll pselect with a zero timeout when `wait' is `MINIMAL'. The following check prevents us from exiting the loop (in this case) when there are file descriptors with waiting data, regardless of what the timeout is: https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=bec823b107ef7d3b51b8e430ccab82c81bd63d24#n5810 There is still a design question of how this SHOULD behave (discussed at the end). >> The first time through, we hit the following, recording that we've >> gotten some output: >> >> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5974 > > OK. > >> We hit the following, setting our timeout to READ_OUTPUT_DELAY_INCREMENT: >> >> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5679 > > This is to _shorten_ the wait timeout: > > /* If we've got some output and haven't limited our timeout > with adaptive read buffering, limit it. */ > if (got_some_output > 0 && !process_skipped > && (timeout.tv_sec > || timeout.tv_nsec > READ_OUTPUT_DELAY_INCREMENT)) > timeout = make_timespec (0, READ_OUTPUT_DELAY_INCREMENT); > > So I don't quite see how this could be a problem here. Am I missing > something? Nothing is wrong here, I'm explaining why we're adding exactly 10ms to every call to `accept-process-output' with a nil PROCESS. The fact that it only decreases the timeout also explains why your patch to reduce the timeout in `url-retrieve-synchronously' makes this faster (as long as the new timeout is less than 10ms). >> Then we hit the select call and wait the full timeout (nothing to read) >> and we finally exit here (because our timeout has passed): >> >> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5832 > > The "full timeout" being the value of READ_OUTPUT_DELAY_INCREMENT, is > that right? Then why is it not TRT, since it _shortens_ the wait? Yes. > >> I think what we need to do is modify: >> >> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5978 >> >> from >> >> if (wait_proc == XPROCESS (proc)) >> wait = MINIMUM; >> >> to >> >> if (!wait_proc || wait_proc == XPROCESS (proc)) >> wait = MINIMUM; >> >> So that we set the timeout to 0 here: >> >> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5439 >> >> And exit early. > > Why? That code was there for a reason. What if we are waiting for > other processes as well? > > What did I miss in the above description that explains why the current > code is wrong? My understanding is that `accept-process-output' is supposed to return immediately when output has been read BOTH when PROCESS is nil and when it is non-nil. If this is not the case, then: 1. We now understand why passing a nil process to `accept-process-output' is slower than passing a specific process. 2. The fix to the original bug report is to apply the first patch I sent to have `url-retrieve-synchronously' wait on a specific process. Looking at the history, it looks like the code in question was changed in bug#20978. However, from what I can tell, we should only apply adaptive read buffering when `process-adaptive-read-buffering' is non-nil. I think we may need to change the check in question to: if (!process_output_skip && (!wait_proc || wait_proc == XPROCESS (proc))) wait = MINIMUM; But I'm very much not sure about that.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 11:52:55 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 07:52:55 2025 Received: from localhost ([127.0.0.1]:36650 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQPRO-0002AE-PG for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 07:52:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38032) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQPRI-00028o-Tw for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 07:52:51 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQPRD-0006Jf-B9; Sat, 14 Jun 2025 07:52:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=gIGIfaQpjlXWrFPInCBLf6YRXYWV65xs0lFTgb5R1ug=; b=oEW7ERwzezsi S/Cu1rgl8P+zQfA3HYr3vP9ZZ2XjfuxoPge3JwEwXijCh0d8UTSbY4Me6nPyfdSrZeyQGdQpQ3Kww B6HxodThvDz0aYlY1HVeyxsB7VF9Tn7P8NdTaiGWnkkaTpHLjtjQ0RxN5oqA8+iOh5BvZfsw8dVvx 0NPS0WKeIg0MmIQh2HwnrcGdDSI8BEGMdvSk8YcQyFTv6PAoe7TdGgKhAs9cEXLBYvA8JpiFFomhS FJaDbWlZl3Oi4ghEEHC+XR+58eJjHMtrvkz5v1tm3ABSzULrknqJfDPrQ6lkWdPCOl09BW/V9yHZg kCCgqOsdiWp8hnx3LHgBYw==; Date: Sat, 14 Jun 2025 14:52:41 +0300 Message-Id: <86frg2sfp2.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <8734c3h6xp.fsf@HIDDEN> (message from Steven Allen on Fri, 13 Jun 2025 10:45:06 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> <87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN> <8734c3h6xp.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Fri, 13 Jun 2025 10:45:06 -0700 > > First, the actual bug: When called with no PROCESS, > `accept-process-output' would wait an additional 10ms > (READ_OUTPUT_DELAY_INCREMENT) after successfully reading data from some > (any) process. `url-retrieve-synchronously' was slow for localhost calls > because 10ms is a long time for fast requests. This bug would add 10ms > of latency to every request. But it's an unneeded delay only if there's no more output from the subprocess during those 10 ms, isn't that so? And we don't really know if all the process output was read, do we? > My SECOND patch (0772d96d, the one to process.c) fixes this > underlying bug by setting the timeout to 0 in this case: when we manage > to read data from some process and aren't waiting on any specific > process. But what if we are also waiting for output from other processes? Your patch will slow those down, won't it? In you case, you only care about a single process, but that's not so in general. > Next steps: > > 1. The SECOND patch fixes the actual bug and should be installed to fix > the underlying bug (assuming I correctly diagnosed the problem/solution, > but I'm pretty sure I did). I disagree for now. I need to understand better why you think it's TRT in the general case. > 2. Personally, I would install the FIRST patch as well as it avoids > returning to `url-retrieve-synchronously' until we've actually read > relevant data. What if url-retrieve was called for more than a single connection? Won't your FIRST patch cause a regression in that case? > This is how `url-retrieve-synchronously' behaved before > bug#49897 changed it to work around then undiagnosed bugs in > `accept-process-output'. Exactly. I'm unwilling to risk reintroducing bugs we already fixed.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 11:45:13 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 07:45:13 2025 Received: from localhost ([127.0.0.1]:36498 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQPJw-0001Vo-OU for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 07:45:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35070) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQPJu-0001UU-7Q for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 07:45:11 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQPJo-0005cL-PF; Sat, 14 Jun 2025 07:45:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=6e9v6rP8UyG0joFroRc/31bUDwtcWivqZUSmxlMVw8k=; b=RjlNq+PRKNqG/6E60axD 9tHLcRVWrmvL9JL8B3DBTH4tTZoaDCqKC9Dj/DXZN4Yxj4+4eM+/UrKtKmdiWmv8tLt4Xx0rqsAg6 leyH9k6ROPuTwZKSXzEIj/CwVt2U81GQ8b/QrddfZ6hWq45iEYlNaoUKbpdeatjXpUp1QpUJPo5PK yCvJAwMEX6iFnJP1FAEJmfpVjedCLSbJUmdZmtFx7Gu2HdWGrwkoy/hjiFiKnBiVCXgP/rMrHj8EP MW89UpS7cr5jJl0KelhrI7NV2dypHNCJLvWJrhJoCQ7JKv7YPQ1s3cHi+35Cd25LpSsm8yuBoLIAp z9bvHgf3VGOhUw==; Date: Sat, 14 Jun 2025 14:45:02 +0300 Message-Id: <86h60isg1t.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Robert Pluim <rpluim@HIDDEN> In-Reply-To: <874iwjam37.fsf@HIDDEN> (message from Robert Pluim on Fri, 13 Jun 2025 13:59:40 +0200) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <87bjqs9fxc.fsf@HIDDEN> <86jz5fex3r.fsf@HIDDEN> <874iwjam37.fsf@HIDDEN> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN, steven@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Robert Pluim <rpluim@HIDDEN> > Cc: steven@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Fri, 13 Jun 2025 13:59:40 +0200 > > >>>>> On Fri, 13 Jun 2025 13:48:08 +0300, Eli Zaretskii <eliz@HIDDEN> said: > > >> Iʼm wondering why weʼre setting the timeout to zero and calling > >> `pselect' again, which means recomputing the wait set again, if I > >> havenʼt gotten lost in the twisty maze of ifʼs and global variables > >> 😀. Weʼve got input, why canʼt we just `break'? There might be some > >> work we donʼt do, but that will get done the next time > >> `wait_reading_process_output' is called. > > Eli> Try looking at the Git history of this code, maybe it will tell you > Eli> how this code evolved, and bring up past discussions and bug reports > Eli> which caused us to make the code as it is today. > > Yes, itʼs gnarly. I think the following would be ok, and it passes the > test case for Bug#20978. Please explain why you think this change is okay, by talking us through what will happen with it in the case in pint. I generally don't like to touch this code with a 3-mile pole, and if this means the current use case will run slowly, so be it. So I need a clear evidence that (a) the current code is wrong (not just sub-optimal, but wrong), and (b) that it can never affect other cases handled by wait_reading_process_output, of which there are a legion. > But I canʼt detect any difference in Stevenʼs test case, so maybe we > let sleeping pselectʼs lie. You are saying that Stevenʼs patch also doesn't break bug#20978? If so, why did you post a different one? And I don't yet understand why short-circuiting like Steven suggests is TRT.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 14 Jun 2025 11:38:46 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 14 07:38:46 2025 Received: from localhost ([127.0.0.1]:36378 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQPDi-000129-7l for submit <at> debbugs.gnu.org; Sat, 14 Jun 2025 07:38:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34928) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQPDf-00011W-8D for 78773 <at> debbugs.gnu.org; Sat, 14 Jun 2025 07:38:44 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQPDZ-0004to-Dz; Sat, 14 Jun 2025 07:38:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=CjLDmf74OXMTKBToF+XlnJebrhHw/06h8BUtAZhciZw=; b=Bsbre8+JG6NR 43qylcXhPGnxzQIaQtmIHP8AItJT7ucTnEzkJefTt1Lq1Yhv5/io3YaDfZsX0XDj4J1SH9R8/EtAc sCNNQ4dHa0iY4tON/qUBj8d5XsMWyLhaVGsNeLY8nDUavt6a2buxRmuhQjSbpFJxfDnO/vNNFqKAp GqTv2ZpcDJgf1shFTB2pwIXiJoGBW19Yckp9KyDP3G8eI1kDV4foAOOQWKCzU9EZs45L/RmJM7b9E m1HO4GW/Bwm1n3YasSBI76vF2DdZCY8cEEIO4oV3g+UiVZ1kiDaQvY2sElBjNQQmGachtuMrVgAuB d1fLcUV8xAM5zkLdy77PCA==; Date: Sat, 14 Jun 2025 14:38:35 +0300 Message-Id: <86ikkysgck.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <87o6usadg6.fsf@HIDDEN> (message from Steven Allen on Thu, 12 Jun 2025 13:54:01 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Thu, 12 Jun 2025 13:54:01 -0700 > > Ok, I think I've figured it out. We loop twice and wait on a timeout > (READ_OUTPUT_DELAY_INCREMENT) the second loop because we're already done > reading. We wait the second time because there could be more input, so I don't see how this could be wrong in general. I also don't quite see the "gotcha!" in your analysis below: > The first time through, we hit the following, recording that we've > gotten some output: > > https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5974 OK. > We hit the following, setting our timeout to READ_OUTPUT_DELAY_INCREMENT: > > https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5679 This is to _shorten_ the wait timeout: /* If we've got some output and haven't limited our timeout with adaptive read buffering, limit it. */ if (got_some_output > 0 && !process_skipped && (timeout.tv_sec || timeout.tv_nsec > READ_OUTPUT_DELAY_INCREMENT)) timeout = make_timespec (0, READ_OUTPUT_DELAY_INCREMENT); So I don't quite see how this could be a problem here. Am I missing something? > Then we hit the select call and wait the full timeout (nothing to read) > and we finally exit here (because our timeout has passed): > > https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5832 The "full timeout" being the value of READ_OUTPUT_DELAY_INCREMENT, is that right? Then why is it not TRT, since it _shortens_ the wait? > I think what we need to do is modify: > > https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5978 > > from > > if (wait_proc == XPROCESS (proc)) > wait = MINIMUM; > > to > > if (!wait_proc || wait_proc == XPROCESS (proc)) > wait = MINIMUM; > > So that we set the timeout to 0 here: > > https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5439 > > And exit early. Why? That code was there for a reason. What if we are waiting for other processes as well? What did I miss in the above description that explains why the current code is wrong?
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 17:45:18 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 13:45:18 2025 Received: from localhost ([127.0.0.1]:49291 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQ8Ss-0000nQ-0o for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 13:45:18 -0400 Received: from fhigh-a5-smtp.messagingengine.com ([103.168.172.156]:42101) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uQ8So-0000kq-5v for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 13:45:16 -0400 Received: from phl-compute-06.internal (phl-compute-06.phl.internal [10.202.2.46]) by mailfhigh.phl.internal (Postfix) with ESMTP id D311F11400FD; Fri, 13 Jun 2025 13:45:08 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-06.internal (MEProxy); Fri, 13 Jun 2025 13:45:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1749836708; x= 1749923108; bh=qhmmOxugbg4tq7C680s7fnwgB2CzS2hAM695HAhBVRA=; b=l QxCdJvOFlKHYrUzY25NJgk98ggW4L+Ohr7e7offfeepDVwBhArOz65emNGEYLxfG x/d5jziuyzdNKWCcOPPVcOEOIXV0tlVaJ29UjN1DTKRHk1Wx5x4f7Gsu8WZID01x bn3KGYNjSPwshA/JmoHUvs7ooG1s2rNuphwxwZUeJH9UOxHenEKqvL4qNk0C+nDA yf2KZj935qo8N2DTVbevwINyUO0TWqjGtNjt4o2kzQTBQjOTBV22p01gnHJh/2nu elNMOSRddCVgJRpvFH4TSNqhwgE+/RKtgL1SrvOUgUyesu7b0dOJTA91SHLSS+JJ 0yMt8OOSKRYl8SWsg+sgg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1749836708; x=1749923108; bh=qhmmOxugbg4tq7C680s7fnwgB2CzS2hAM69 5HAhBVRA=; b=nqQLxhEjCvxlfJSheJJAnwf0cSDUKLHH8ZKbjITie7KVuw5p9wP EgyNpOVkm2JeIozMFATeNPZzAXfw+KlixbYHOascX5RNemctT2AywNbKwZxkMztr fA5G8VjNsbajcrkN1u9wMSwLYlkGPkq2uXgr7dGaqp/iwR5SzBxHv3eaujFJRoZ8 kMJvz6a+R1U8eB/iPs0E9AjzD+sr+OdFoxaA1FfrHlRUScCg3zaBUlIw0SZZH373 imZ0X+HAIRyiGNn3qlmKw5GV41ezbvHC+egFAF6igPBLQPBh5aUJ8Gx+ZKBCCTtO mtaVg7MG3PrWXQojROB/Nf4a/6d60nGhWJA== X-ME-Sender: <xms:pGNMaJur97vYfILpYDfDDCE9Ou1_gQN2vigKWip2CgJyLoL4UqvKcg> <xme:pGNMaCdve3HePHTqggVBQbYdpZVek-wBZaXY1oc_-F8mvxczWaGmnEzGji_ksavgr F6ocnf-dzhwozNLJYE> X-ME-Received: <xmr:pGNMaMy1v3hMDuHL207W1jLx-12CPs-yx7eoqtmuhgjichYBWDUj0F7NzUyY> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddukeehkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg hnrdgtohhmqeenucggtffrrghtthgvrhhnpeejffefuddvieefgeevkeeggfelhefhffeg leetffetuedttefggeffheeufeektdenucffohhmrghinhepghhnuhdrohhrghenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsthgvvhgvnhes shhtvggsrghlihgvnhdrtghomhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtph houhhtpdhrtghpthhtohepvghlihiisehgnhhurdhorhhgpdhrtghpthhtoheprhhplhhu ihhmsehgmhgrihhlrdgtohhmpdhrtghpthhtohepjeekjeejfeesuggvsggsuhhgshdrgh hnuhdrohhrghdprhgtphhtthhopehlrghrshhisehgnhhushdrohhrgh X-ME-Proxy: <xmx:pGNMaAMc6Wvgr3rbFcPpFCN8RY6AcazkUUzbkkfurrqtiS3LFOmY6A> <xmx:pGNMaJ9jcwq7hO23yu_lmmWadVhF8P7276-gf3q4qsEzxDEaIKwMyA> <xmx:pGNMaAXhHFjlUZ-l0oy8vdK7OVHYasyBeid2mswi_ZCnVCVtqHcyWw> <xmx:pGNMaKfer6_RIKPMbe8QLY9LqUWJhm8GuTFkK3a_O3jQIf_kRl950A> <xmx:pGNMaKfbQkoHpb7JQ9iJ_7ic4U4CQJKvzJRF_hndWglQa7lnShZ9RSKg> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 13 Jun 2025 13:45:08 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <8634c3ej85.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> <87ecvnhf96.fsf@HIDDEN> <8634c3ej85.fsf@HIDDEN> Date: Fri, 13 Jun 2025 10:45:06 -0700 Message-ID: <8734c3h6xp.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Steven Allen <steven@HIDDEN> >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN >> Date: Fri, 13 Jun 2025 07:45:25 -0700 >> >> >> Eli Zaretskii <eliz@HIDDEN> writes: >> >> >> From: Steven Allen <steven@HIDDEN> >> >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN >> >> Date: Thu, 12 Jun 2025 12:20:30 -0700 >> >> >> >> Eli Zaretskii <eliz@HIDDEN> writes: >> >> >> >> >> I'll do that today. I have a suspicion that fast requests waiting on a >> >> >> single process exit early here: >> >> >> >> >> >> https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5562 >> >> > >> >> > The condition for that block is >> >> > >> >> > if (wait_proc >> >> > && ! EQ (wait_proc->status, Qrun) >> >> > && ! connecting_status (wait_proc->status)) >> >> > >> >> > And the comment says "Don't wait for output from a non-running >> >> > process." Is the case here that the network connection was already >> >> > closed when we read from the sub-process? I thought that was not the >> >> > case. >> >> >> >> With my patch, it does hit that case, but on the third loop through so >> >> it's not exiting early. >> > >> > How come? What is wait_proc->status in that case? >> >> Note: this is WITH my original patch (to url.el), the one that makes it >> only wait on one process. >> >> In that case, the status is "run" the 1-2 loops through, then "exit" on >> the last loop (triggering this branch). > > So your patch is only useful when the sub-process exits soon after > producing its single chunk of output, is that right? Not quite. That condition will trigger on the last call to `accept-process-output` when a PROCESS is specified, regardless of how short the total response was. But there's a more to the story; let's start at the top. First, the actual bug: When called with no PROCESS, `accept-process-output' would wait an additional 10ms (READ_OUTPUT_DELAY_INCREMENT) after successfully reading data from some (any) process. `url-retrieve-synchronously' was slow for localhost calls because 10ms is a long time for fast requests. This bug would add 10ms of latency to every request. My SECOND patch (0772d96d, the one to process.c) fixes this underlying bug by setting the timeout to 0 in this case: when we manage to read data from some process and aren't waiting on any specific process. My FIRST patch (4529f63f, the one to `url-retrieve-synchronously') worked around this bug by specifying the process to wait on. With that patch I happened to hit the branch in question because the requests were fast enough to complete in one call to `accept-process-output'. However, the FIRST patch would still have worked around the underlying bug (the issue fixed in the SECOND patch) because the FIRST patch specifies the process to wait on and the underlying bug is only triggered when the process is NOT specified. Next steps: 1. The SECOND patch fixes the actual bug and should be installed to fix the underlying bug (assuming I correctly diagnosed the problem/solution, but I'm pretty sure I did). 2. Personally, I would install the FIRST patch as well as it avoids returning to `url-retrieve-synchronously' until we've actually read relevant data. This is how `url-retrieve-synchronously' behaved before bug#49897 changed it to work around then undiagnosed bugs in `accept-process-output'. However, that's very much a maintainer decision because it might cause the bug in `accept-process-output' that motivated this change (bug#49861) to rear its head again.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 15:48:35 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 11:48:34 2025 Received: from localhost ([127.0.0.1]:48519 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQ6dt-0008GP-Iy for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 11:48:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59350) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQ6dq-0008G8-3M for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 11:48:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQ6dk-0007B8-CV; Fri, 13 Jun 2025 11:48:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Pd0/u3oVFDn/dRF1HtT3OU8xGNvSO2YFxlVn8Sm6eNE=; b=bJDcWKVFqc4B qOFg9qD3jIBoyId+H9DzdGzEWoopoi1oL4Sgm/ytxXdXO7fW3IbNkMPjQTK/FzAmsdYO5M4FuD0bU 8h/7WFOIMrpAck/aBVmfKeqSf+XTeMf0Mus3o50j0+5GjJ35u0/CdHLVTd1DZYeOfJ5Bs9jJxACqn GN4eaJHp6AFpJ9NGHCJdZRGCQd56LDC7tAbmSP93li2ICULtQ5pbm0o0r6TVAtXBsjbqn8tPvkCF0 4gszft5jCH0QWvt08jz5zUhsjlPUKb6fgWkcSZWnDkC0QGLURGWJOU5g8WOygguiJm7ImU3s43O0Q AOAmzCtvaMaMc+nyEnM6RQ==; Date: Fri, 13 Jun 2025 18:47:54 +0300 Message-Id: <8634c3ej85.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <87ecvnhf96.fsf@HIDDEN> (message from Steven Allen on Fri, 13 Jun 2025 07:45:25 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> <87ecvnhf96.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Fri, 13 Jun 2025 07:45:25 -0700 > > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> From: Steven Allen <steven@HIDDEN> > >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > >> Date: Thu, 12 Jun 2025 12:20:30 -0700 > >> > >> Eli Zaretskii <eliz@HIDDEN> writes: > >> > >> >> I'll do that today. I have a suspicion that fast requests waiting on a > >> >> single process exit early here: > >> >> > >> >> https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5562 > >> > > >> > The condition for that block is > >> > > >> > if (wait_proc > >> > && ! EQ (wait_proc->status, Qrun) > >> > && ! connecting_status (wait_proc->status)) > >> > > >> > And the comment says "Don't wait for output from a non-running > >> > process." Is the case here that the network connection was already > >> > closed when we read from the sub-process? I thought that was not the > >> > case. > >> > >> With my patch, it does hit that case, but on the third loop through so > >> it's not exiting early. > > > > How come? What is wait_proc->status in that case? > > Note: this is WITH my original patch (to url.el), the one that makes it > only wait on one process. > > In that case, the status is "run" the 1-2 loops through, then "exit" on > the last loop (triggering this branch). So your patch is only useful when the sub-process exits soon after producing its single chunk of output, is that right?
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 15:31:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 11:31:38 2025
Received: from localhost ([127.0.0.1]:48433 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uQ6NW-0005k8-5i
for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 11:31:38 -0400
Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]:58556)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.84_2) (envelope-from <rpluim@HIDDEN>) id 1uQ6NT-0005io-I4
for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 11:31:36 -0400
Received: by mail-wr1-x42f.google.com with SMTP id
ffacd0b85a97d-3a53359dea5so1471776f8f.0
for <78773 <at> debbugs.gnu.org>; Fri, 13 Jun 2025 08:31:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1749828689; x=1750433489; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
:reply-to; bh=PKgEUeEfrG16uV/5FaHTW4h9GLSKNe0wiWl7nUSMkJE=;
b=GrfkmnmmL9oRobAe5kRivT5Tdo9AzXt9ZTWGCENRWD9gBd3mynkRnXgj0AgpaNLfa+
HxirhAIM85TuJQ+gEDsrOuq1kN8MkBORaxnDgpl9jnWbsNVztU+YeHwdFqqsZnX+Df9K
wPRuEXDPAnJp4lMfw5dKqhTQXjSOCKxneBOLIgEyAqUtElP/WI766Y4g+3k+j96T7Gfw
MDZhb9ZaPBYqAz+AZW7WzflWPczcwZFX8QMcTkrNuAyfk9hN/HfBn1nU/95W17tGTM+s
8IEu0rM2doF+UVAT61GW3ZTcSBAWb+DogFC+yGe0z7cM7cZWV+iQ4psmenxN37/wMdIT
9ioA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1749828689; x=1750433489;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=PKgEUeEfrG16uV/5FaHTW4h9GLSKNe0wiWl7nUSMkJE=;
b=pjuJcrqLiA3+r5kqTcDRba3wvh0uHnd5tMb5b3faZfdl3Z3IbKnYY9uV0MHr4Hs+lr
t7nYmk6w9Z7FSs/vXfNiFlb6xsLXm7n75IAKvPXHF63HenfAJSst+EXZNT0i9VZLTQ+4
yiJ2AyDAPpBkTeqNBi4LhEV6WsbQZd/OULeifaZf8v7J9xzpGvsYUnxZInzZRqbqgA1S
SIh8XejcyBzkIScsxw2F8zK/pcaHhS2zc9UqmADHACq3vkBA/Ri2/MYtFgt7rpfbQNhZ
iX9ujsYW/tMi5nR6xPfNbHIudkssuLOvuNY9Xz4639FygMafhKJgq3hQRWXalOskk2FY
lciQ==
X-Forwarded-Encrypted: i=1;
AJvYcCUyXx960/f0l/jHluBCrVybBvvnnD0QUCfln77+rhFrF7oDgeLwOkk3ouqlgjNrhqhvnj+2lg==@debbugs.gnu.org
X-Gm-Message-State: AOJu0YzjmJjlfcRIUBeYt688DWtr7mxOjRWeYkHBDdHhzLQAWPoVptBt
mMtcC5HmG5HEwJ1mNmkLcmpjq6C3zO3r94mQS2xIZ//dNGIvPmZ0r6z8
X-Gm-Gg: ASbGncvqDmz4svPUPzc5i2AbazgzX1Yl8863ZPN69EaWPa3T9OkXqzTUrQQCMy7Kt44
Be9a1dr6rCp6PrJw4fd/418PLPgTXKII93gvLi+v+eC656e889hbCcm6xseUhw9kdFN8IThgfZR
YBXCUdXQwPqc/MB+3jmv3M1EBGNWhVKcWIf8m0/GSISP2FQw4KA9WJfnQ/tCAfBC4LT18gQGGV9
tUKViLVJcSd4wFc/+dotyZkIHhFqWtJy8w6FpnZtlWCNMNfj6NYbzW3OpFk/wIK1O4VKEHm+IRz
+Zbp3ObXMTYCptzuHNqZP7lTxoY5rKvC2IWBUhwe0amYl7U0hA==
X-Google-Smtp-Source: AGHT+IEthoYFNSJDBRjC2Gy0D3m3ZZlYJb4KUFe6N/WcqReo4fThG74WDQpggOw+z2uND7J77b+U+Q==
X-Received: by 2002:a05:6000:41dd:b0:3a4:e609:dc63 with SMTP id
ffacd0b85a97d-3a57237190fmr206018f8f.20.1749828689071;
Fri, 13 Jun 2025 08:31:29 -0700 (PDT)
Received: from rltb ([2a01:e0a:3f3:fb51:df39:e830:abf3:ba4f])
by smtp.gmail.com with ESMTPSA id
ffacd0b85a97d-3a568a72cd3sm2716064f8f.32.2025.06.13.08.31.28
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Fri, 13 Jun 2025 08:31:28 -0700 (PDT)
From: Robert Pluim <rpluim@HIDDEN>
To: Steven Allen <steven@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <875xgzhdwe.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN>
<87bjqs9fxc.fsf@HIDDEN> <86jz5fex3r.fsf@HIDDEN>
<874iwjam37.fsf@HIDDEN> <87bjqrhezw.fsf@HIDDEN>
<87zfeb8z72.fsf@HIDDEN> <875xgzhdwe.fsf@HIDDEN>
Date: Fri, 13 Jun 2025 17:31:27 +0200
Message-ID: <87v7oz8xps.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
>>>>> On Fri, 13 Jun 2025 08:14:41 -0700, Steven Allen <steven@HIDDEN=
m> said:
>>=20
>> Since I can=CA=BCt definitively say it won=CA=BCt break anything, we=
should go
>> with the minimal patch, ie yours.
Steven> Your approach is correct, I think. It just may be less performa=
nt as the
Steven> user will now need to accept output twice when we could have ju=
st
Steven> "finished the job" the first time around.
We=CA=BCll let Eli decide =F0=9F=98=BA
Robert
--=20
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 15:14:51 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 11:14:51 2025 Received: from localhost ([127.0.0.1]:48198 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQ67H-0000LZ-GH for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 11:14:51 -0400 Received: from fhigh-a4-smtp.messagingengine.com ([103.168.172.155]:48989) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uQ67F-0000LI-0n for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 11:14:49 -0400 Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfhigh.phl.internal (Postfix) with ESMTP id 8973911400F3; Fri, 13 Jun 2025 11:14:43 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-10.internal (MEProxy); Fri, 13 Jun 2025 11:14:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1749827683; x=1749914083; bh=Gee0h8IYCkvUEA535JMAlirqEcK0p6sY fv/NmP0f5L0=; b=ka2Nn3D+ZnquIaLtde+tn1D0oHb/j1xcKUSLKG0tcO2Ltxxr nFRUEhdVYQEWXLl8uYqBqmzPykqvEnYAZohc3/ra9RdclVtlfowGVxLodWB27gCj /YRkiB3kNJRdPNNmYmapNXuf/diWdH0SQwLQVc0BpTtYmRVtl4KkFzMPXhnR3aax n5BidPRFLQRUe4MAYSj4szEbHOXBJzyByu0CMQm+YrojLLmmIZgXuLbYO6zew9r6 +R5aw9uKsRDlFuyr4LV5j78HFw/yGZq4qVg8z+R5q+QuKZV3SazF3DIHd6jx9jYa ZzbTrGSjBTZaQ5mDcNT3qh/VUo3uFwf7MknGug== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1749827683; x= 1749914083; bh=Gee0h8IYCkvUEA535JMAlirqEcK0p6sYfv/NmP0f5L0=; b=M nA0ddv8yxG8WCwojl79T8xvdhcr8JMOJzBMoIGzBtR7Nw+gRETcyTTeSAaWQF0Fb 96CTTv8sgWr+dopgxul85WO/mUar5uSRs07TmaEo6qTdkWN5SQ2ldkegMtZZS4IL Apms/935lhBaub/oZ2DrOUzrE7dOHGTTOkaK1AsHe3eOZdXdtyU/9fOt7PulmKsu 0qWdPrDMswHkrO2bLznRGrPuHEFHxS1Qp05RXzwbXGE3rH935E4BljzeoUuyL2Xd /RAfifhncbsP2Di+0tWv4o+eWwvF5Mo+BQLPNmBKv+8GrbOTsS8cLu2HyrJRsUAg MAH7SdA560q0I0mSHUl1Q== X-ME-Sender: <xms:Y0BMaO5UfOV6zBk183K8X6Wek4AUMk0wOeaC93PTaNt-ozjez-U1MA> <xme:Y0BMaH4ZnGHBiqUGkN8Hzoa4mxblT0i70SIU6SxK9t3I-5dK9QwSlxk2OujiCu0DK MswaYRkPj69-wTQ904> X-ME-Received: <xmr:Y0BMaNdm5onDKMIO7mUql6ccQaPDBRZOq_A3LGEFpjk1uHHx9COnEQCrv9yEupBuQiEeHTppLVzTTku38USyoW8ZFmPc080> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddukedvkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgfgsehtqhertddttdej necuhfhrohhmpefuthgvvhgvnhcutehllhgvnhcuoehsthgvvhgvnhesshhtvggsrghlih gvnhdrtghomheqnecuggftrfgrthhtvghrnhepudevlefgtdekheekgffhtdegheduhfeg tdfgffefteetgfetkeeivddthefhkeetnecuffhomhgrihhnpehgnhhurdhorhhgnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepshhtvghvvghn sehsthgvsggrlhhivghnrdgtohhmpdhnsggprhgtphhtthhopeegpdhmohguvgepshhmth hpohhuthdprhgtphhtthhopehrphhluhhimhesghhmrghilhdrtghomhdprhgtphhtthho pegvlhhiiiesghhnuhdrohhrghdprhgtphhtthhopeejkeejjeefseguvggssghughhsrd hgnhhurdhorhhgpdhrtghpthhtoheplhgrrhhsihesghhnuhhsrdhorhhg X-ME-Proxy: <xmx:Y0BMaLJfljzk1L9bB4_npvl32fleSclJWH-w4iNm5OwbMbeCB-XPuQ> <xmx:Y0BMaCJvXLaFijO9wgOnvNs8iabehR2aO1lkJ1ABt806XmkW0uWTtg> <xmx:Y0BMaMx5ciFYYxwhcjhWrPOpUt0ePqcjzPmHFRbR80EAXxg2t6SQXw> <xmx:Y0BMaGK12LlsKjEmhwgBUDT7uQONCcVQ1r6XwF1PSm0UL3D1tbi4Aw> <xmx:Y0BMaLpKhoUIK7RcsU1_KtDCjuaSjPG6g49Qj0FR7R6uKJOZPWZaBxRx> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 13 Jun 2025 11:14:42 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Robert Pluim <rpluim@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <87zfeb8z72.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <87bjqs9fxc.fsf@HIDDEN> <86jz5fex3r.fsf@HIDDEN> <874iwjam37.fsf@HIDDEN> <87bjqrhezw.fsf@HIDDEN> <87zfeb8z72.fsf@HIDDEN> Date: Fri, 13 Jun 2025 08:14:41 -0700 Message-ID: <875xgzhdwe.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Robert Pluim <rpluim@HIDDEN> writes: >>>>>> On Fri, 13 Jun 2025 07:50:59 -0700, Steven Allen <steven@HIDDEN= om> said: > > Steven> I think this patch will cause us to the final drain loop when= waiting on > Steven> a single connection. It's probably fine, but I assume this dr= ain loop > Steven> existed for a reason? > > Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/p= rocess.c?h=3Dbec823b107ef7d3b51b8e430ccab82c81bd63d24#n5525 > > It will, but we=CA=BCll already have read from the process at line 5972 or > thereabouts. > > Since I can=CA=BCt definitively say it won=CA=BCt break anything, we shou= ld go > with the minimal patch, ie yours. Your approach is correct, I think. It just may be less performant as the user will now need to accept output twice when we could have just "finished the job" the first time around.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 14:59:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 10:59:39 2025
Received: from localhost ([127.0.0.1]:47779 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uQ5sZ-0007Vk-36
for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 10:59:39 -0400
Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]:59857)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.84_2) (envelope-from <rpluim@HIDDEN>) id 1uQ5sW-0007VT-RK
for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 10:59:37 -0400
Received: by mail-wm1-x334.google.com with SMTP id
5b1f17b1804b1-450ce671a08so13960025e9.3
for <78773 <at> debbugs.gnu.org>; Fri, 13 Jun 2025 07:59:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1749826771; x=1750431571; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
:reply-to; bh=WlR6410Qbghbsx7GOU4ekNOt6WMmI3dB6AaCg9Mj9Bk=;
b=J/+n5HOn3QTypKlYM5oGVmdxE6uMQMhKtrnG8WkHIcbeoRy50VprM/AeuQiqAZAV78
Q+YUS9j3K36sVbB1NDZqxG4Ry92YbPeHeCQdoFhpcBVbxizNNZAzAwLA6CDZDWMivUei
qdEnbHyxObSh4/dft5hOvV3Yw8Mdjx4q33YVamt1lEvV7+XDg9JLlUh8pYfhtFedsY3N
g8VOpME/E6MNbW08di6eicBl4e1In06ZzubCuQKJdM1ZSQzcacFEvsUxFzldMYWzQb2x
LnsChBOxBvyI11bym1aVPWAvgdsGdY8Cgj+7tbwK6HyF41DapAsMSUQciikaOiEffz1B
yOpw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1749826771; x=1750431571;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=WlR6410Qbghbsx7GOU4ekNOt6WMmI3dB6AaCg9Mj9Bk=;
b=MizwbR+hdtC+Gu0cSjTrWcoZdpkuDVy5uZClCrSBnifP6PZ1q6FOMUUDTH0KAZjo9R
O8Fs/9enLMdGvs6dAjCgoif2fRu373vQn2Sgca3EB5s1t4CzPloQHPNSGOSgXwdwcc4m
/xeVGnWv/cot5ZMOi7JbSaweaSWoH3l9gBDKP6bHAMiL2xE3SYkUXp2+TcnlMATDA/vD
iiLlKafX9UM5kRzPTTpfkRD3UqK7O8FsDuFdY4HCRweOKW1AWCIvnz/8nR8M5kqJL8MG
H+1tzBxxBnEnwhcZTMoqT/d989aI/NYmnLTcQoh6s2FclkUGIQG3e8B/dUWJvYEKrtFH
AqNA==
X-Forwarded-Encrypted: i=1;
AJvYcCXYRohkzW0Ss03/VM7bgBQ5aEnKn18gXSki0URMPMwhUAP+jj3pl1wRRZKVRTXBrj8T7OAqKA==@debbugs.gnu.org
X-Gm-Message-State: AOJu0YwWGZk54olLLBseF3mMusGzObMuTDlsWfT+noysNYkAEgHFMITx
C8qcxSsquySGyZm7ixpVmZz0iEQbHBjQKu/GB2SgftR+nQc3OJHfdpgeIN2kSwTs
X-Gm-Gg: ASbGncsu06rdObMfZlOSVP+L9EC0js8Mvl7eUfTSC+pbVGZTj6Z8h5CS/aGh0XXg24j
k2/hiMugwtr7w9xpujQHRIPzOJynPkSMkXiGHmh03f7p03Ln+pbbq4HOdnOuZt+l9DRVy13NSqa
nY7qKkUzNqI2xIiylO/F4oz+vWXZqxXIAec6lYbico5W05OmpYUIdkDFKu9M6qtadX6YTqgTkAb
Mqu8JL4B54J5ev2xmMyhEXmUEYZr184ecC42KNbrtZGJ/dp3MzBumVgBYCKCLHBZ22H4Pwr9vsd
M+NalfqgN9ho9eni3RX0ApzpGBa/TYAw8zzTQ0y9PE51DqxeVg==
X-Google-Smtp-Source: AGHT+IEUPylrGlUjs+p/QAywlAtzx0NMWJeoLDOqPpYaEX+91lINIappnT+Qpcpqsd1u1GjKXOZMNA==
X-Received: by 2002:a05:600c:4f09:b0:43d:8ea:8d7a with SMTP id
5b1f17b1804b1-45334af5c89mr30044275e9.28.1749826770398;
Fri, 13 Jun 2025 07:59:30 -0700 (PDT)
Received: from rltb ([2a01:e0a:3f3:fb51:df39:e830:abf3:ba4f])
by smtp.gmail.com with ESMTPSA id
ffacd0b85a97d-3a568a7fa9dsm2584708f8f.44.2025.06.13.07.59.29
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Fri, 13 Jun 2025 07:59:29 -0700 (PDT)
From: Robert Pluim <rpluim@HIDDEN>
To: Steven Allen <steven@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <87bjqrhezw.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN>
<87bjqs9fxc.fsf@HIDDEN> <86jz5fex3r.fsf@HIDDEN>
<874iwjam37.fsf@HIDDEN> <87bjqrhezw.fsf@HIDDEN>
Date: Fri, 13 Jun 2025 16:59:29 +0200
Message-ID: <87zfeb8z72.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
>>>>> On Fri, 13 Jun 2025 07:50:59 -0700, Steven Allen <steven@HIDDEN=
m> said:
Steven> I think this patch will cause us to the final drain loop when w=
aiting on
Steven> a single connection. It's probably fine, but I assume this drai=
n loop
Steven> existed for a reason?
Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/pro=
cess.c?h=3Dbec823b107ef7d3b51b8e430ccab82c81bd63d24#n5525
It will, but we=CA=BCll already have read from the process at line 5972 or
thereabouts.
Since I can=CA=BCt definitively say it won=CA=BCt break anything, we should=
go
with the minimal patch, ie yours.
Robert
--=20
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 14:51:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 10:51:09 2025
Received: from localhost ([127.0.0.1]:47549 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uQ5kL-0006tE-3h
for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 10:51:09 -0400
Received: from fout-a1-smtp.messagingengine.com ([103.168.172.144]:35637)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <steven@HIDDEN>)
id 1uQ5kI-0006sh-ND
for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 10:51:07 -0400
Received: from phl-compute-01.internal (phl-compute-01.phl.internal
[10.202.2.41])
by mailfout.phl.internal (Postfix) with ESMTP id 1764B13806AB;
Fri, 13 Jun 2025 10:51:01 -0400 (EDT)
Received: from phl-mailfrontend-01 ([10.202.2.162])
by phl-compute-01.internal (MEProxy); Fri, 13 Jun 2025 10:51:01 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com;
h=cc:cc:content-transfer-encoding:content-type:content-type
:date:date:from:from:in-reply-to:in-reply-to:message-id
:mime-version:references:reply-to:subject:subject:to:to; s=fm3;
t=1749826261; x=1749912661; bh=88FqpJjfKy6u8R1+Z8ZkPkp4ab4l420q
2n59gxff5Tk=; b=L/37h56QFMWZoHkZ0sZYYVKKJWUsxbnIBuZEZKtJAoGqrcMD
8GfXJBWo1JxoINdlw/0sxGPBy0zFyxW3Bwwps1U4J9PnKau+RHyo7U6phdTghoJ1
UHqSeLrAcROYfv9xYKnChfHBd/cqDyscZKRVbN/RvIfNAA0r1RVR7IboK0k+NVdL
4bCFtA6PkXpO0Bl4yDhPxGRLurGO1h6DEhcr9/41MUUiN8r1BDL1YqsxpbFzTzd0
9F7vt1VdUmlvmX2pJlE9HegcrsSITl10979b+yiEEkZrTSCrSaa+zHn+UcpqBYH7
tUi8B/Q05L+FtD7O8RQ9GcsfaBA/+UA/xjQkMw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:subject:subject:to:to:x-me-proxy
:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1749826261; x=
1749912661; bh=88FqpJjfKy6u8R1+Z8ZkPkp4ab4l420q2n59gxff5Tk=; b=B
7G+9YYeUyAzcvWMLwGET1Q3cbegou8N2Hr+8gsguofys4FDxhadD6m00mzJjU0D7
iodc5QiaI2/qDbAzp/vi8W9RnkCRPIhDEA04NkfWepUX33qljyRuCB0E9VjSVf+t
ccWcwgYui8akc8jYeHTQwQyZrtUqUgKhZ0VsjJwm2tkLKHsSQgVr8jQtzdvGhEkc
qhTkKNHwQdGDfbLRKGPuXecA9c1DHz6r2zv8Vq789PB4+jknybI+NBnIy7Wz4St8
NspApHm3F67o42zdNDqROXuXZitJfSK+LSJcGmdBL1vVTqtdFjVkIfZsqzt71gKC
20UpFnnCPhGXZodyQHQqw==
X-ME-Sender: <xms:1DpMaMfN6kAGbmvyAhTEjaUeuwmmj-evjbx6Zcai-u5K1VyfYeLVJQ>
<xme:1DpMaOODRCY-WyRnHQ2xWM3DlkWFq5Z_OQWaLeUp2BM_0mJICeNKtP8c-Ay8rsxVx
tB2D8TnQUsIlCQhSlU>
X-ME-Received: <xmr:1DpMaNhxTkDSdTnB2o4vQQzdWH5xUfY3A9OisEw2AhtZLKgnpoO4f6EGqr78w2-wR8DCiUgmAIfA7HwlsofZNB7XAtaKPZw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddukedvfecutefuodetggdotefrod
ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp
uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg
hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgfgsehtqhertddttdej
necuhfhrohhmpefuthgvvhgvnhcutehllhgvnhcuoehsthgvvhgvnhesshhtvggsrghlih
gvnhdrtghomheqnecuggftrfgrthhtvghrnhepudevlefgtdekheekgffhtdegheduhfeg
tdfgffefteetgfetkeeivddthefhkeetnecuffhomhgrihhnpehgnhhurdhorhhgnecuve
hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepshhtvghvvghn
sehsthgvsggrlhhivghnrdgtohhmpdhnsggprhgtphhtthhopeegpdhmohguvgepshhmth
hpohhuthdprhgtphhtthhopehrphhluhhimhesghhmrghilhdrtghomhdprhgtphhtthho
pegvlhhiiiesghhnuhdrohhrghdprhgtphhtthhopeejkeejjeefseguvggssghughhsrd
hgnhhurdhorhhgpdhrtghpthhtoheplhgrrhhsihesghhnuhhsrdhorhhg
X-ME-Proxy: <xmx:1DpMaB_1sWM0F71JcxHWPEF3qDBW6-EBKc6QNI_jxli4NH7pnHNFWw>
<xmx:1DpMaIv81mtR2XsSjKvOuaEMFlOFn6osjHBiOpymlL3vvQkxhQhpCw>
<xmx:1DpMaIEFeexSkp8uTQ9qgvH3RiYAZrTcVxVHjNp2MZ9DRvJp8zKbLQ>
<xmx:1DpMaHPPAs4On2dxRGax5ayhrDF9zJQnuMturKtym8azfrk2Fk0pGA>
<xmx:1TpMaNPs-ftwMdbeDENDeFiYBWU7tAuGrcufxADamPH_mxw_kSGTDwsm>
Feedback-ID: ie8a146a7:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
13 Jun 2025 10:51:00 -0400 (EDT)
From: Steven Allen <steven@HIDDEN>
To: Robert Pluim <rpluim@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <874iwjam37.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN>
<87bjqs9fxc.fsf@HIDDEN> <86jz5fex3r.fsf@HIDDEN>
<874iwjam37.fsf@HIDDEN>
Date: Fri, 13 Jun 2025 07:50:59 -0700
Message-ID: <87bjqrhezw.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
Robert Pluim <rpluim@HIDDEN> writes:
>>>>>> On Fri, 13 Jun 2025 13:48:08 +0300, Eli Zaretskii <eliz@HIDDEN> sai=
d:
>
> >> I=CA=BCm wondering why we=CA=BCre setting the timeout to zero and =
calling
> >> `pselect' again, which means recomputing the wait set again, if I
> >> haven=CA=BCt gotten lost in the twisty maze of if=CA=BCs and globa=
l variables
> >> =F0=9F=98=80. We=CA=BCve got input, why can=CA=BCt we just `break'=
? There might be some
> >> work we don=CA=BCt do, but that will get done the next time
> >> `wait_reading_process_output' is called.
>
> Eli> Try looking at the Git history of this code, maybe it will tell =
you
> Eli> how this code evolved, and bring up past discussions and bug rep=
orts
> Eli> which caused us to make the code as it is today.
>
> Yes, it=CA=BCs gnarly. I think the following would be ok, and it passes t=
he
> test case for Bug#20978.
>
> But I can=CA=BCt detect any difference in Steven=CA=BCs test case, so may=
be we
> let sleeping pselect=CA=BCs lie.
I think this patch will cause us to the final drain loop when waiting on
a single connection. It's probably fine, but I assume this drain loop
existed for a reason?
https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=3Db=
ec823b107ef7d3b51b8e430ccab82c81bd63d24#n5525
> Robert
> --=20
> diff --git a/src/process.c b/src/process.c
> index e61ec425f7e..0119330f7fe 100644
> --- a/src/process.c
> +++ b/src/process.c
> @@ -5934,6 +5934,7 @@ wait_reading_process_output (intmax_t time_limit, i=
nt nsecs, int read_kbd,
> channel_start
> =3D process_prioritize_lower_fds ? 0 : last_read_channel + 1;
>=20=20
> + bool break_now =3D false;
> for (channel =3D channel_start, wrapped =3D false;
> !wrapped || (channel < channel_start && channel <=3D max_desc);
> channel++)
> @@ -5975,8 +5976,11 @@ wait_reading_process_output (intmax_t time_limit, =
int nsecs, int read_kbd,
> if (nread > 0)
> {
> /* Vacuum up any leftovers without waiting. */
> - if (wait_proc =3D=3D XPROCESS (proc))
> - wait =3D MINIMUM;
> + if (!wait_proc || wait_proc =3D=3D XPROCESS (proc))
> + {
> + wait =3D MINIMUM;
> + break_now =3D true;
> + }
> /* Since read_process_output can run a filter,
> which can call accept-process-output,
> don't try to read from any other processes
> @@ -6119,6 +6123,8 @@ wait_reading_process_output (intmax_t time_limit, i=
nt nsecs, int read_kbd,
> }
> }
> } /* End for each file descriptor. */
> + if (break_now) /* Don't rerun select. */
> + break;
> } /* End while exit conditions not met. */
>=20=20
> unbind_to (count, Qnil);
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 14:45:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 10:45:37 2025 Received: from localhost ([127.0.0.1]:47475 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQ5ez-0006YK-1v for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 10:45:37 -0400 Received: from fhigh-a3-smtp.messagingengine.com ([103.168.172.154]:47049) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uQ5ev-0006Y4-J2 for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 10:45:34 -0400 Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfhigh.phl.internal (Postfix) with ESMTP id 03C7A1140163; Fri, 13 Jun 2025 10:45:28 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Fri, 13 Jun 2025 10:45:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1749825927; x= 1749912327; bh=wIBHhEdoi+Av/lYNU+MRQTNnMnqFJ4IqF+/gtozwJ0Q=; b=L ce6LkgNfogdac+rF5cRwR78MJhKKJdI1Lhy2pnGziLSBwTIFigK3vby+ahL44w5+ qYx6Dm60oDuzVmFvYrqjNoLsIOV4tFhlYW9t/vPXeIuYV5cU/wavUv8Bj9hZQaJA k5MwsyaRPGytO1RWgVyAYyQ2tEhjwgB6YTyIGgv2cWGQaIlZ9+jVaZd+H0Degt3+ vcIUXCL/u4N4JzYma7Tuvo2ac40C704N/AdZ6RzgcOOKTh+kXaknrW4krpALRdnZ qccCW5cgxKJaOuJgg5ezj4uF0eO/xMnvf32ofWGWHF6NiNb1ol9XbHl6kBEvcYo5 v8YKJWurOZ4b8MCq2iGhA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1749825927; x=1749912327; bh=wIBHhEdoi+Av/lYNU+MRQTNnMnqFJ4IqF+/ gtozwJ0Q=; b=mEwCLQVnkjRhmcLjEUZ7itTgDhqCBkEPNNQ9zK6NsN5KOiszO8a LRDLct9lpijbMQlk6m/LuwdJJnZtYq/IsQPB1Dmlbn71DDlsN0kSAbhvDlRA60re tu8Uo7/+za3CkIWQSiHhjOy5Ir4rZsIR/71NGXVdfz28RJ3I1sAUBPJoUMgL6x9u QGYKA9sBIU2kjREFz4m2U0GBkvkJvOZfkCjYjtFTTXV09z3dUX1UFJhcwYtxFhIH 4tJPg9sn5PWomq/ScIS3RFGZ3OVBT1au9DLlIL+0A7MeWEAhpSicUWvM7zZVE/xq 4G+tC3klvLoxmgOsueLc3uy617oUjeZHY7Q== X-ME-Sender: <xms:hzlMaGju874aX898SksfKid3Tg6MrWGxiMVql9-zULkZ-mEQ82V42Q> <xme:hzlMaHAO5LVUtZPLlsZK8K0SqynDEPz5xDp7pB_0ZKRPyDYYuHanezoorR3xWDk2D oCEGx-IwoDvO4t47Ac> X-ME-Received: <xmr:hzlMaOEds73FYWdvCC3e_3WkECnpcEXNuwBHakj5Tgl9WhLZD6LMlLS2WvrKSizfqS2EMRLZ864W0mKh5b8RBGQ0G2vzx2k> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddukedvfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg hnrdgtohhmqeenucggtffrrghtthgvrhhnpeejffefuddvieefgeevkeeggfelhefhffeg leetffetuedttefggeffheeufeektdenucffohhmrghinhepghhnuhdrohhrghenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsthgvvhgvnhes shhtvggsrghlihgvnhdrtghomhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtph houhhtpdhrtghpthhtohepvghlihiisehgnhhurdhorhhgpdhrtghpthhtoheprhhplhhu ihhmsehgmhgrihhlrdgtohhmpdhrtghpthhtohepjeekjeejfeesuggvsggsuhhgshdrgh hnuhdrohhrghdprhgtphhtthhopehlrghrshhisehgnhhushdrohhrgh X-ME-Proxy: <xmx:hzlMaPRKa6BAXytMMP3-QDTAgJz4N3zrhwCYbd0qjjq1ig47TiT6cQ> <xmx:hzlMaDzmOaAJ5IYISBWYtzT-yjcO5XfP2dt3z8VpIlf7F0C5SxK_iQ> <xmx:hzlMaN6LtwXgE6tODcm61U4CjGNX8MGJwsavldONNqGxqynu144LfQ> <xmx:hzlMaAyTFT7ositsGME4jEKmQfr6B5kc8RReALAgDGjgDpJLx-KDEg> <xmx:hzlMaLTANM28gAJJz7nguAWu3g0f3j9KRV4aBScigATJA-FO6BZSuMAd> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 13 Jun 2025 10:45:27 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <86plf8duap.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <86plf8duap.fsf@HIDDEN> Date: Fri, 13 Jun 2025 07:45:25 -0700 Message-ID: <87ecvnhf96.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Steven Allen <steven@HIDDEN> >> Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN >> Date: Thu, 12 Jun 2025 12:20:30 -0700 >> >> Eli Zaretskii <eliz@HIDDEN> writes: >> >> >> I'll do that today. I have a suspicion that fast requests waiting on a >> >> single process exit early here: >> >> >> >> https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5562 >> > >> > The condition for that block is >> > >> > if (wait_proc >> > && ! EQ (wait_proc->status, Qrun) >> > && ! connecting_status (wait_proc->status)) >> > >> > And the comment says "Don't wait for output from a non-running >> > process." Is the case here that the network connection was already >> > closed when we read from the sub-process? I thought that was not the >> > case. >> >> With my patch, it does hit that case, but on the third loop through so >> it's not exiting early. > > How come? What is wait_proc->status in that case? Note: this is WITH my original patch (to url.el), the one that makes it only wait on one process. In that case, the status is "run" the 1-2 loops through, then "exit" on the last loop (triggering this branch).
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 11:59:52 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 07:59:52 2025
Received: from localhost ([127.0.0.1]:43307 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uQ34Z-00034S-VP
for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 07:59:52 -0400
Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]:54565)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.84_2) (envelope-from <rpluim@HIDDEN>) id 1uQ34V-00034A-T5
for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 07:59:49 -0400
Received: by mail-wm1-x334.google.com with SMTP id
5b1f17b1804b1-453398e90e9so3353415e9.1
for <78773 <at> debbugs.gnu.org>; Fri, 13 Jun 2025 04:59:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1749815981; x=1750420781; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
:reply-to; bh=+wV7vjA31XJOQLX7GGlqVgGVPlt+Y7Brtfr7zLQqRfU=;
b=fZG0ZIsTZ3IoG5KmRqjf2ivXeOrFhCq2xV8r7RDR9xRB//EYkHLl6w3tjeeV2mNA7F
PbCrZudbLacqyKZSplJTfPgOBn03mROlZglwqLWGdNCjqL6wzjllz76qSzXVWfIJFjID
CVhDcd1U+G/BjWrZJc1kSLqghqF1EffwwG80CCQkuC3weVOLYhLi1R3eA/C0B5kFiPRD
LH6Z2tooq3my3CTKrm57ySw9v9mtst4eogewT3jU13490cIWA0dIeJ/2Lh25+xTJmlTW
zdlLDtCT1ku/BKHexL7WCOTxMJ9jaabhSk9C3YEopXqzim8cvpFbB0q14ZHRV91/vu13
RG3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1749815981; x=1750420781;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=+wV7vjA31XJOQLX7GGlqVgGVPlt+Y7Brtfr7zLQqRfU=;
b=Y4Q5Qm1jTClytZyTpcsaBYvGGwf+y4h9ziltZs1zck9kBng5E2hEVxCl+sdbyT9Mw9
L2L9QzxR/xA2SK7YtJj6PpWnjSwe2rk+GAWuxAxM8EtKGyW65wqCT3Qxw77CWtC2+zBY
o7Db+rSKdTEg/z+l3XdYg4p+vzqrfY64Rgf9fznWs9sdpt73G01s++0ItMHyTtemuGrK
I1CoqxjZu2u96XLbRNfEyPzujJn1heUrB1CGxp19K4B0UDgyAyiDgksZp+QfwxqBqQH9
rxNzIdnTfCHVaNQw6+GGDHQ165RpKXfGE7GBtqvMC7x9BIG5hDaDYavIyxMdcA1PVyJH
vZZA==
X-Forwarded-Encrypted: i=1;
AJvYcCXx8f5bL7lsBM5LetUQ7J6W7I8sbn5cJCwZGQhGnoMMvGrZ2Ihmsl2+MxW/PeoqIVUjhTWo9A==@debbugs.gnu.org
X-Gm-Message-State: AOJu0Yyevppo1al1IW54I5BLmUu49m4NpZ+7sNHAY1WaYrDuNOdH7YEJ
2rzBodWZfNuKgQqTpVbzh35oTK00TVb55FBa740JoxnD6cAo3oFdDIAs
X-Gm-Gg: ASbGncvoRu1Fgn3+i0c8cdmZYsQxU45dicHNJEVklXM1ijb0u2QLd3odN2VymA5EsEp
9gj4HHCss3QPklOFw4wHiJ+1qFBZYttqQlal3f4EDIxr5MqERqLvQuqEDMMNyYvEgtdvrTq+agn
KPoknpfCpuCZzAGUp2ru+UgS1anBLIZ+Wp8Zr/LYJ3XTikQCrHslEcJCD3jt5soojjgeVLdAJfI
CRyfhBYacihxmFrqbdHP5N6k0wgoOnrreWOgN18z3cf7qs/urGeaXhPVWd8JM78oJbDaGZPPhyD
fwhAzOEMjk36LgM/EmBtbAV5rx/WME1/0jJDMcizCrLBWZYilvV8dBd54T5J
X-Google-Smtp-Source: AGHT+IGXIPVnQDaEz7/VhoxAt4XsTlRAuaSdMeotkRq3yGWQACAic12UeZq3IkFgbNiPqLBIToAdMg==
X-Received: by 2002:a05:600c:4f55:b0:43c:f629:66f4 with SMTP id
5b1f17b1804b1-453349acb61mr32003125e9.0.1749815981282;
Fri, 13 Jun 2025 04:59:41 -0700 (PDT)
Received: from rltb ([2a01:e0a:3f3:fb51:c51f:cac0:9d7b:73dd])
by smtp.gmail.com with ESMTPSA id
5b1f17b1804b1-4532e2618f0sm49791665e9.37.2025.06.13.04.59.40
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Fri, 13 Jun 2025 04:59:40 -0700 (PDT)
From: Robert Pluim <rpluim@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <86jz5fex3r.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN>
<87bjqs9fxc.fsf@HIDDEN> <86jz5fex3r.fsf@HIDDEN>
Date: Fri, 13 Jun 2025 13:59:40 +0200
Message-ID: <874iwjam37.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN, steven@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
>>>>> On Fri, 13 Jun 2025 13:48:08 +0300, Eli Zaretskii <eliz@HIDDEN> said:
>> I=CA=BCm wondering why we=CA=BCre setting the timeout to zero and ca=
lling
>> `pselect' again, which means recomputing the wait set again, if I
>> haven=CA=BCt gotten lost in the twisty maze of if=CA=BCs and global =
variables
>> =F0=9F=98=80. We=CA=BCve got input, why can=CA=BCt we just `break'? =
There might be some
>> work we don=CA=BCt do, but that will get done the next time
>> `wait_reading_process_output' is called.
Eli> Try looking at the Git history of this code, maybe it will tell you
Eli> how this code evolved, and bring up past discussions and bug repor=
ts
Eli> which caused us to make the code as it is today.
Yes, it=CA=BCs gnarly. I think the following would be ok, and it passes the
test case for Bug#20978.
But I can=CA=BCt detect any difference in Steven=CA=BCs test case, so maybe=
we
let sleeping pselect=CA=BCs lie.
Robert
--=20
diff --git a/src/process.c b/src/process.c
index e61ec425f7e..0119330f7fe 100644
--- a/src/process.c
+++ b/src/process.c
@@ -5934,6 +5934,7 @@ wait_reading_process_output (intmax_t time_limit, int=
nsecs, int read_kbd,
channel_start
=3D process_prioritize_lower_fds ? 0 : last_read_channel + 1;
=20
+ bool break_now =3D false;
for (channel =3D channel_start, wrapped =3D false;
!wrapped || (channel < channel_start && channel <=3D max_desc);
channel++)
@@ -5975,8 +5976,11 @@ wait_reading_process_output (intmax_t time_limit, in=
t nsecs, int read_kbd,
if (nread > 0)
{
/* Vacuum up any leftovers without waiting. */
- if (wait_proc =3D=3D XPROCESS (proc))
- wait =3D MINIMUM;
+ if (!wait_proc || wait_proc =3D=3D XPROCESS (proc))
+ {
+ wait =3D MINIMUM;
+ break_now =3D true;
+ }
/* Since read_process_output can run a filter,
which can call accept-process-output,
don't try to read from any other processes
@@ -6119,6 +6123,8 @@ wait_reading_process_output (intmax_t time_limit, int=
nsecs, int read_kbd,
}
}
} /* End for each file descriptor. */
+ if (break_now) /* Don't rerun select. */
+ break;
} /* End while exit conditions not met. */
=20
unbind_to (count, Qnil);
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 10:48:19 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 06:48:19 2025 Received: from localhost ([127.0.0.1]:41819 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uQ1xL-0003FE-59 for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 06:48:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35268) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uQ1xI-0003Ew-QP for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 06:48:17 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uQ1xD-0003M6-8m; Fri, 13 Jun 2025 06:48:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=VopnovTtQsgbGimNFkfHFT9KDAOe76rIkpOpBjxXEl0=; b=FOsJlm/gs0LnuwjRjMsF cfMCYQZ2MMME37BO5EyBmRXJ7BTM0YeXxcvF8ItUYZ1Ybx+ZbwyNRwphcdlJg49cr03oq2whZeFSc lNjlRXkelv9dRQRADRHIammMw+45riqNFCxnoW67+4nIGcaIExoojP75pRaKvUT/iPPj8T3jttZzX 2smNUq/pjxPEymB7dQpa7MhedKe7Cv7zczUBIeYsW+J0N5X46694SfscSHnH7ihK4RKpc1Hu5EgSg pJUbNdG7xtdRj6tp7PimbVbNq5x+IWoUmA57KVS63MgkUZJaNj0asVv3TobNxaqWMQ35dXySHw8Vc RouKhlvfygu3sw==; Date: Fri, 13 Jun 2025 13:48:08 +0300 Message-Id: <86jz5fex3r.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Robert Pluim <rpluim@HIDDEN> In-Reply-To: <87bjqs9fxc.fsf@HIDDEN> (message from Robert Pluim on Fri, 13 Jun 2025 10:58:07 +0200) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN> <87bjqs9fxc.fsf@HIDDEN> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN, steven@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Robert Pluim <rpluim@HIDDEN> > Cc: Eli Zaretskii <eliz@HIDDEN>, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Fri, 13 Jun 2025 10:58:07 +0200 > > >>>>> On Thu, 12 Jun 2025 13:54:01 -0700, Steven Allen <steven@HIDDEN> said: > > Steven> Ok, I think I've figured it out. We loop twice and wait on a timeout > Steven> (READ_OUTPUT_DELAY_INCREMENT) the second loop because we're already done > Steven> reading. > > Steven> The first time through, we hit the following, recording that we've > Steven> gotten some output: > > Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5974 > > > Steven> We hit the following, setting our timeout to READ_OUTPUT_DELAY_INCREMENT: > > Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5679 > > Steven> Then we hit the select call and wait the full timeout (nothing to read) > Steven> and we finally exit here (because our timeout has passed): > > Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5832 > > Steven> *** > > Steven> I think what we need to do is modify: > > Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5978 > > Steven> from > > Steven> if (wait_proc == XPROCESS (proc)) > Steven> wait = MINIMUM; > > Steven> to > > Steven> if (!wait_proc || wait_proc == XPROCESS (proc)) > Steven> wait = MINIMUM; > > Steven> So that we set the timeout to 0 here: > > Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5439 > > Steven> And exit early. > > That analysis looks correct to me. It works for the original test > case, at least. I didn't yet have time to look into this closely enough to have an opinion. > Iʼm wondering why weʼre setting the timeout to zero and calling > `pselect' again, which means recomputing the wait set again, if I > havenʼt gotten lost in the twisty maze of ifʼs and global variables > 😀. Weʼve got input, why canʼt we just `break'? There might be some > work we donʼt do, but that will get done the next time > `wait_reading_process_output' is called. Try looking at the Git history of this code, maybe it will tell you how this code evolved, and bring up past discussions and bug reports which caused us to make the code as it is today.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 08:58:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 04:58:23 2025
Received: from localhost ([127.0.0.1]:39587 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uQ0Es-0008K1-Ht
for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 04:58:23 -0400
Received: from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d]:48397)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.84_2) (envelope-from <rpluim@HIDDEN>) id 1uQ0Ep-0008JB-PB
for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 04:58:16 -0400
Received: by mail-wm1-x32d.google.com with SMTP id
5b1f17b1804b1-450cfb6a794so11046875e9.1
for <78773 <at> debbugs.gnu.org>; Fri, 13 Jun 2025 01:58:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1749805089; x=1750409889; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
:reply-to; bh=kAwHz/t9LjItrcQzLdHbqvTWfVUk6LKIoMMHQaI3Y/E=;
b=cHLtF/07aMPDy0kzhxo19Y8ae/jD5hBePrOPUdXk2QonumNsLzu3tge9k5HfIE5umP
TwY6lobR17bZ83iF7zZgjmX/OMlf7C1tIq/mqFdTquRLLv6X2sbDl/c6tCrXUy2Vx60I
J/5+TWd3W5TjRUb2Ql6O82iQE5RIgIokk9m7lbgbTncOZOplSBW6xSMDDqgF33pDfn6i
ENofu2RUYaQGcg78ArxJDkgrps4h/1VgW3ouPipxvJie5RM6OFh/4gr4OlxkpnVmPPBH
4tD9PamP8DnWWdnX5OlY4vcM4BiBaLcJYNSX0NSADKtUtfrye0OIHd6NwfOp+QFZlBgj
8gRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1749805089; x=1750409889;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=kAwHz/t9LjItrcQzLdHbqvTWfVUk6LKIoMMHQaI3Y/E=;
b=XFEcie6L3mco5b140NqxddJO/Z62n2uyNtSJ7XA/BKI0s9Yj0taXyagwD2BTHWkML3
sqQpuQnF6s2EV2fTTYKIQWbfsOGBJOiOCxPhIGjKmJpifzTpGaON5S4msh9kOJ1SjX/z
R1qX1w8SVSqqSaf0cRJTdwfIfMlehrJlbV1GyTuf4yQ+U/HWZRhPwEpJwHvklfrKDWQ1
o3a+wRxhY/7+3lDiPrLU0iPUCVTXpX82Dx/NYnGz/9G3+Qmt3bbMAv1KhJDlWbG98NXm
M2UKTkrdxZHmJFlewVDLUFxB9CGjHfciVPu9shb4wDTUtrE2i/i9K7tTmQBb9Q77krjp
zwgQ==
X-Forwarded-Encrypted: i=1;
AJvYcCV2Atw0j6+F+2nGnfgcqz0wOdBEXlGVSoMX58EUYxIoiAyKhO9Cokm1R4cn2iXOSDdLSA9Z7w==@debbugs.gnu.org
X-Gm-Message-State: AOJu0YwU5aM/mjLgHmGuoxwsVw/qimcEFFo+VIFlWP2N73iMruOwb3eL
VB4itlvASgHwRIovHKW6C7bo/judo/WQHSgJtAYlV1R0Ug3myQZBd4mg
X-Gm-Gg: ASbGnctrRK+go5pN1D2dTQDnjy71zMUCe1qifTOTrj0rOqJm2ocUVgX6FzwL6FzCmgD
BYpb8Tnob8SklAU0/VFyAOddDf2v7sNMetohL+SI6aAYQfIarr/3k74axY8EF626osbeai+JLRv
noJQgO78L+RPxwGrmKe1WHpsn+f/7DCt6Mz3AsmuzbqqHWUVNCqm2lYH6D0i0jRy0xcSSMnvuR3
nDFyJno8Y2fx3QAP/o0Zt/hEYZ9bASF4nTYtFlkQFAADzH0AgUYGb2CysFUGgwulXTCdiSYD5Bp
k8ApcJQ1jbzprkEBDKSJL568yLqZEE4eaeghjD8O4D9NCyT/eA==
X-Google-Smtp-Source: AGHT+IGf1w7OFlH7QzLbWylvs7oXR5afKc3OoBVoU0V142784OTcwrnWL6Bsyr9ktVU88QJKSJEMYQ==
X-Received: by 2002:a05:600c:138b:b0:442:ccf9:e6f2 with SMTP id
5b1f17b1804b1-45334b630ffmr22233415e9.16.1749805088752;
Fri, 13 Jun 2025 01:58:08 -0700 (PDT)
Received: from rltb ([2a01:e0a:3f3:fb51:c51f:cac0:9d7b:73dd])
by smtp.gmail.com with ESMTPSA id
5b1f17b1804b1-453305a0d9dsm39052945e9.21.2025.06.13.01.58.08
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Fri, 13 Jun 2025 01:58:08 -0700 (PDT)
From: Robert Pluim <rpluim@HIDDEN>
To: Steven Allen <steven@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <87o6usadg6.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN> <87o6usadg6.fsf@HIDDEN>
Date: Fri, 13 Jun 2025 10:58:07 +0200
Message-ID: <87bjqs9fxc.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, Eli Zaretskii <eliz@HIDDEN>, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
>>>>> On Thu, 12 Jun 2025 13:54:01 -0700, Steven Allen <steven@HIDDEN=
m> said:
Steven> Ok, I think I've figured it out. We loop twice and wait on a ti=
meout
Steven> (READ_OUTPUT_DELAY_INCREMENT) the second loop because we're alr=
eady done
Steven> reading.
Steven> The first time through, we hit the following, recording that we=
've
Steven> gotten some output:
Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/pro=
cess.c?h=3D81a3e4e51167be51c63eae682331210bc62f7280#n5974
Steven> We hit the following, setting our timeout to READ_OUTPUT_DELAY_=
INCREMENT:
Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/pro=
cess.c?h=3D81a3e4e51167be51c63eae682331210bc62f7280#n5679
Steven> Then we hit the select call and wait the full timeout (nothing =
to read)
Steven> and we finally exit here (because our timeout has passed):
Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/pro=
cess.c?h=3D81a3e4e51167be51c63eae682331210bc62f7280#n5832
Steven> ***
Steven> I think what we need to do is modify:
Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/pro=
cess.c?h=3D81a3e4e51167be51c63eae682331210bc62f7280#n5978
Steven> from
Steven> if (wait_proc =3D=3D XPROCESS (proc))
Steven> wait =3D MINIMUM;
Steven> to
Steven> if (!wait_proc || wait_proc =3D=3D XPROCESS (proc))
Steven> wait =3D MINIMUM;
Steven> So that we set the timeout to 0 here:
Steven> https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/pro=
cess.c?h=3D81a3e4e51167be51c63eae682331210bc62f7280#n5439
Steven> And exit early.
That analysis looks correct to me. It works for the original test
case, at least.
I=CA=BCm wondering why we=CA=BCre setting the timeout to zero and calling
`pselect' again, which means recomputing the wait set again, if I
haven=CA=BCt gotten lost in the twisty maze of if=CA=BCs and global variabl=
es
=F0=9F=98=80. We=CA=BCve got input, why can=CA=BCt we just `break'? There m=
ight be some
work we don=CA=BCt do, but that will get done the next time
`wait_reading_process_output' is called.
Robert
--=20
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 13 Jun 2025 06:34:22 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 13 02:34:22 2025 Received: from localhost ([127.0.0.1]:37499 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uPxzY-0006WY-Gh for submit <at> debbugs.gnu.org; Fri, 13 Jun 2025 02:34:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43908) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uPxzT-0006Ub-EQ for 78773 <at> debbugs.gnu.org; Fri, 13 Jun 2025 02:34:17 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uPxzN-0004Q2-IL; Fri, 13 Jun 2025 02:34:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=qDLmA2kuvSkN2wlh+874YbTq+m7PhNT+XNz7xxPRFQY=; b=eR3Zaf4ZadoV Ca7UYX7OsWdahcx6oyzu6Zbf6rwaITBelVm1jDqv+Ni+6pPTt0NVwtoKw1EnHKJs9FnY/m5dx0H59 PnCEBfGSXyoKCnlZNjj9dBy3DCC8qLCK5k3EHRdKbBnXExgMJBK1xRyy+ZiEP2+kvv5WEYsunXG1I LjAvfu25wJrM63KNbBhNQmwvILjHX7ZDhbqumdZEc9tG2hP286HOnlNZEghctNV4KNzxN5VqlnttR H6/em5k4w+pJEcuaB5VJmg6ygyBub9WjAJgvRu3wZJq542/n5AKwt+P5vZeQ+ORG/tnOaCjy4h2gA 40MzuAz+KaWzJ7dCQa0lsg==; Date: Fri, 13 Jun 2025 09:34:06 +0300 Message-Id: <86plf8duap.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN> In-Reply-To: <87v7p0ahs1.fsf@HIDDEN> (message from Steven Allen on Thu, 12 Jun 2025 12:20:30 -0700) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> <87v7p0ahs1.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Steven Allen <steven@HIDDEN> > Cc: rpluim@HIDDEN, 78773 <at> debbugs.gnu.org, larsi@HIDDEN > Date: Thu, 12 Jun 2025 12:20:30 -0700 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> I'll do that today. I have a suspicion that fast requests waiting on a > >> single process exit early here: > >> > >> https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5562 > > > > The condition for that block is > > > > if (wait_proc > > && ! EQ (wait_proc->status, Qrun) > > && ! connecting_status (wait_proc->status)) > > > > And the comment says "Don't wait for output from a non-running > > process." Is the case here that the network connection was already > > closed when we read from the sub-process? I thought that was not the > > case. > > With my patch, it does hit that case, but on the third loop through so > it's not exiting early. How come? What is wait_proc->status in that case?
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 12 Jun 2025 20:54:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 16:54:13 2025
Received: from localhost ([127.0.0.1]:32843 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uPow8-0006Q8-Bs
for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 16:54:12 -0400
Received: from fhigh-b1-smtp.messagingengine.com ([202.12.124.152]:57641)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <steven@HIDDEN>)
id 1uPow5-0006Pl-4W
for 78773 <at> debbugs.gnu.org; Thu, 12 Jun 2025 16:54:09 -0400
Received: from phl-compute-06.internal (phl-compute-06.phl.internal
[10.202.2.46])
by mailfhigh.stl.internal (Postfix) with ESMTP id 5581D2540092;
Thu, 12 Jun 2025 16:54:03 -0400 (EDT)
Received: from phl-mailfrontend-02 ([10.202.2.163])
by phl-compute-06.internal (MEProxy); Thu, 12 Jun 2025 16:54:03 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com;
h=cc:cc:content-type:content-type:date:date:from:from
:in-reply-to:in-reply-to:message-id:mime-version:references
:reply-to:subject:subject:to:to; s=fm3; t=1749761643; x=
1749848043; bh=FynnKFuVSidWqpxJgQvO20Jb2ANTX4zw0IEILpYaSGc=; b=n
aiQEjvdmd+GOas/OXNmEWnOJ4ZFemEcggYnFZ/mCsobwXEnl8Uae4ZDjrQ+7iRY6
6nJKm8A2dQbMJgmzYYTOip7OOdR72jCCs1Q4qV28zhP68UBmoRFkY3N3Jvx1uCus
TaHl1PdrTMz+7tEl0Q7lfOxJmqiHO4+2pkfFMbiSNHyD+tNjwjrdgKr8fqb5CUhM
XFcpfdhNegGFJtrkvzDyYkJz5N/jKTQEv6eNtHH6bw3/N/strETFmLSFnj+dvsNQ
w6z6JQduGocBzfKip1acnrILYuDGAMo98woZZv2MhYJyr5GUoBVhB0GbuUpIaQrw
D5P5GxRAgNbcsvSUuO/Dg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-type:content-type:date:date
:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
:message-id:mime-version:references:reply-to:subject:subject:to
:to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
1749761643; x=1749848043; bh=FynnKFuVSidWqpxJgQvO20Jb2ANTX4zw0IE
ILpYaSGc=; b=WUZzcKohsm7KBHjA7pbNLN303S527GEZ1c/cDw4sGxOczfz4Tq1
SgI4s4SpHUNiLwdAUvT9YefxTzhYdE1Vn+SuPBEudnL4wMhcOVrcSthHfQBWHx2r
PWmo57G89R8T14+uZKrD+nnVspSN7A0oKObWcnHhnnbAXE+6M+hXNzLlNQ0foT8r
50Tl7XCQ1c7ph8WojgGuqf24a0Y5eoI993S5wGMcVwyaDxpmvnVwEF1Llf/FFcHA
2PEBZ6Gso61XBpTe7stViqqxA32vCpak/I73P5sskJqg4tSi8ERmpqRJBJ/MWdIa
uPwWqUd8Mhe5TPyt3g2s1CUoa+1wTQMvkiw==
X-ME-Sender: <xms:aj5LaMxRUcQqPthn-VSXH9RHkcdTsCzLHmLLBwkjYcYmVeFvw172cA>
<xme:aj5LaAQ4Iz5mY3XKP5xqhO2Bx5hvly97DlCKFEJp_u0gApu05CDOr962-hXZDQ25F
ab8H_maZ6d5JTq5pDU>
X-ME-Received: <xmr:aj5LaOXJYiyaAmqIH3QuCgGqE3fME2Uz-VOGYOOzSjUk9V5FV1Tb-n_BPD8FJHKSanSScn-UDXWbYFWb7If15bH-KD-0m9c>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugdduiedtkecutefuodetggdotefrod
ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp
uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg
hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesmhdtreertddttden
ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg
hnrdgtohhmqeenucggtffrrghtthgvrhhnpeeitedvhfffuefhtdeutdegfeegieeiteev
vefgueefieevvdehfeejffffgfevjeenucffohhmrghinhepghhnuhdrohhrghenucevlh
hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsthgvvhgvnhes
shhtvggsrghlihgvnhdrtghomhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtph
houhhtpdhrtghpthhtohepvghlihiisehgnhhurdhorhhgpdhrtghpthhtoheprhhplhhu
ihhmsehgmhgrihhlrdgtohhmpdhrtghpthhtohepjeekjeejfeesuggvsggsuhhgshdrgh
hnuhdrohhrghdprhgtphhtthhopehlrghrshhisehgnhhushdrohhrgh
X-ME-Proxy: <xmx:aj5LaKggc6HuHmqOALkIwBI7yigx1aOZ7rvAWvpRirMyM6ZynU_nWg>
<xmx:aj5LaOCbfzTt08qZyPQ2OeHVOdwZ8oeF9ZV-1-fPn2t4gW7lukVOeg>
<xmx:aj5LaLI6x3O1caM90Whpf-eJsbUdH4YJx3nRph3DHHzcYv1DHplkXQ>
<xmx:aj5LaFB7RTVH5vlDdIRQ95yiqhYHJYY83SGcZ2W4AFO2ChsJPG2_Sw>
<xmx:az5LaL9Qn9A9X18D71HIvEBdajsc4ZYO_Db05Ws09TME19XmCnVeyS9l>
Feedback-ID: ie8a146a7:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
12 Jun 2025 16:54:02 -0400 (EDT)
From: Steven Allen <steven@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <87v7p0ahs1.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN>
<87v7p0ahs1.fsf@HIDDEN>
Date: Thu, 12 Jun 2025 13:54:01 -0700
Message-ID: <87o6usadg6.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
--=-=-=
Content-Type: text/plain
Steven Allen <steven@HIDDEN> writes:
> Eli Zaretskii <eliz@HIDDEN> writes:
>
>>> From: Steven Allen <steven@HIDDEN>
>>> Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN
>>> Date: Thu, 12 Jun 2025 11:02:41 -0700
>>>
>>> > Perhaps instrumenting wait_reading_process_output with printf's would
>>> > help to understand the control flow in the case of nil and non-nil
>>> > PROCESS argument?
>>>
>>> I'll do that today. I have a suspicion that fast requests waiting on a
>>> single process exit early here:
>>>
>>> https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5562
>>
>> The condition for that block is
>>
>> if (wait_proc
>> && ! EQ (wait_proc->status, Qrun)
>> && ! connecting_status (wait_proc->status))
>>
>> And the comment says "Don't wait for output from a non-running
>> process." Is the case here that the network connection was already
>> closed when we read from the sub-process? I thought that was not the
>> case.
>
> With my patch, it does hit that case, but on the third loop through so
> it's not exiting early.
>
> I also tried setting the timeout in `accept-process-output' to nil and
> that got me the same speedup as my fix but ended up looping 23 times so
> there's something really strange going on with the timeout here.
Ok, I think I've figured it out. We loop twice and wait on a timeout
(READ_OUTPUT_DELAY_INCREMENT) the second loop because we're already done
reading.
The first time through, we hit the following, recording that we've
gotten some output:
https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5974
We hit the following, setting our timeout to READ_OUTPUT_DELAY_INCREMENT:
https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5679
Then we hit the select call and wait the full timeout (nothing to read)
and we finally exit here (because our timeout has passed):
https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5832
***
I think what we need to do is modify:
https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5978
from
if (wait_proc == XPROCESS (proc))
wait = MINIMUM;
to
if (!wait_proc || wait_proc == XPROCESS (proc))
wait = MINIMUM;
So that we set the timeout to 0 here:
https://cgit.git.savannah.gnu.org/cgit/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5439
And exit early.
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
filename=0001-Exit-accept-process-output-early-when-we-get-output-.patch
From 0772d96d50734fff14a17b2f207661418f5c814c Mon Sep 17 00:00:00 2001
From: Steven Allen <steven@HIDDEN>
Date: Thu, 12 Jun 2025 13:48:08 -0700
Subject: [PATCH] Exit accept-process-output early when we get output from any
process
Previously, the caller requested to wait on output from any process (not
a specific process), we'd always end up waiting
10ms (READ_OUTPUT_DELAY_INCREMENT) one final time before returning.
* src/process.c (wait_reading_process_output): abort waiting early if we
either receive output from the process we were waiting on OR we weren't
waiting on a process.
---
src/process.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/process.c b/src/process.c
index e61ec425f7e..7b8fd1357e3 100644
--- a/src/process.c
+++ b/src/process.c
@@ -5975,7 +5975,7 @@ wait_reading_process_output (intmax_t time_limit, int nsecs, int read_kbd,
if (nread > 0)
{
/* Vacuum up any leftovers without waiting. */
- if (wait_proc == XPROCESS (proc))
+ if (!wait_proc || wait_proc == XPROCESS (proc))
wait = MINIMUM;
/* Since read_process_output can run a filter,
which can call accept-process-output,
--
2.49.0
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 12 Jun 2025 19:20:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 15:20:41 2025 Received: from localhost ([127.0.0.1]:60625 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uPnTd-0008Tv-CN for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 15:20:41 -0400 Received: from fhigh-b8-smtp.messagingengine.com ([202.12.124.159]:54475) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uPnTZ-0008TQ-V4 for 78773 <at> debbugs.gnu.org; Thu, 12 Jun 2025 15:20:38 -0400 Received: from phl-compute-06.internal (phl-compute-06.phl.internal [10.202.2.46]) by mailfhigh.stl.internal (Postfix) with ESMTP id 55C7C25402F1; Thu, 12 Jun 2025 15:20:32 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-06.internal (MEProxy); Thu, 12 Jun 2025 15:20:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-type:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to; s=fm3; t=1749756032; x= 1749842432; bh=Ue/ifrVf8tJ7LVewsm/tpQZk9qn0Np6uF2zM3Mkfc7w=; b=Q bFNo1L28z2zGVDZnCaaWTeGMKJX+Pz9lEuOdmAXZLJpGLoXUfVNolqrvnGML7liS bqhRU4BH3OH+9e2BLxAQoQ/uQ+T5Fo1SERgIBzDJueHh1LlyKV9HW7XKStSvLdmB OW2c0ytNyzhEJ+tM+bkArUNs77SR+JKdgJOKXaQYWV0loEc/4SJa1HhFKkGPnYL/ CRMAxvflz0RDgiFndjCVWxhavfSZfXkRVDR9ruUTn2URK9XlDAJLXx5aDeeoQ1/C k0D3BSVkG6t5JDf1jFWtORoLYJTw8ec7/x8Yiwp+GEATriVandSfnPEa1rolyKBn Lj7YXcZn1AV6WKpiyzRFQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1749756032; x=1749842432; bh=Ue/ifrVf8tJ7LVewsm/tpQZk9qn0Np6uF2z M3Mkfc7w=; b=BXbFO/dJq904y5RWhTiOXL5KoJjgexTfdqs8IQ9pbYr59oaBNKU DZ5l+FqL83n7RZ5dfo4nsmKX+x4W0jM4TxTWF3lXPMOGuy8w8ZNz+L+n5nYHXc7t hCd/SQ2GXkCpsZ7e0ax/IaH+p8+7cSE0KeSuuw2R6sj6/cR8hSWdZBlzUUdt/EhM QW7mWzF14EXGLFcpNs2hyOshewoGkkfP/zHsU/LNZy+87JL0gHPlELoc9PNwVNwG oeCK+ikQb8knVjFxmwbxaIUbknULb/LiFTcA8ZQN7vmLkMbndx3qNLf/fxmwAwSJ wlRuUoyRHLgL6w8HF/f+sCgK36cB/4BC/OQ== X-ME-Sender: <xms:fyhLaCGbUowj0mxDdRA2vkDqhhwEaRK4SYIR00zspVK_iQLiZ221-g> <xme:fyhLaDXP7d7tbtRw7-MM4UoBC7jOsu8Lp6kQEwOyo8I8xEKI6kaXGTIsUda2XnxVU siLLGnrNrsJBJlw9j4> X-ME-Received: <xmr:fyhLaMIvp4U6qU_l5WWHv2Pm138NTZgVvLxvBrzr_ctZ181I04aKlqVwaYW8oNvd3LnOfF7dgYfhYpjadEN0MbTATTnklbw> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugdduheeklecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg hnrdgtohhmqeenucggtffrrghtthgvrhhnpeejffefuddvieefgeevkeeggfelhefhffeg leetffetuedttefggeffheeufeektdenucffohhmrghinhepghhnuhdrohhrghenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsthgvvhgvnhes shhtvggsrghlihgvnhdrtghomhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmhhtph houhhtpdhrtghpthhtohepvghlihiisehgnhhurdhorhhgpdhrtghpthhtoheprhhplhhu ihhmsehgmhgrihhlrdgtohhmpdhrtghpthhtohepjeekjeejfeesuggvsggsuhhgshdrgh hnuhdrohhrghdprhgtphhtthhopehlrghrshhisehgnhhushdrohhrgh X-ME-Proxy: <xmx:fyhLaMEN06SSqkwaCFPAQPrAS_x8eV9BiKNgs0JmuCnBrdWgeZW5Iw> <xmx:fyhLaIUFaBRDjuxGI2pDVLimp9JebVSXJ7qWWXMH4HrF5UG5vpU0xA> <xmx:fyhLaPNzwhK1EHYk-6VRquHydBIwguvIIL-j2e2d-xbJgy8vOiFdkA> <xmx:fyhLaP1BmcG_TpzGLYMius-IwEFGgYQZjzL6L5Odh9CKqZ4TSguXlw> <xmx:gChLaK0ujBBbj4SvOhTyHk2m11KZeYNsv_QVOX5Bg9KyuMknSuQ02-JS> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 12 Jun 2025 15:20:31 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <86zfecerva.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> <87y0twdeim.fsf@HIDDEN> <86zfecerva.fsf@HIDDEN> Date: Thu, 12 Jun 2025 12:20:30 -0700 Message-ID: <87v7p0ahs1.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Steven Allen <steven@HIDDEN> >> Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN >> Date: Thu, 12 Jun 2025 11:02:41 -0700 >> >> > Perhaps instrumenting wait_reading_process_output with printf's would >> > help to understand the control flow in the case of nil and non-nil >> > PROCESS argument? >> >> I'll do that today. I have a suspicion that fast requests waiting on a >> single process exit early here: >> >> https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5562 > > The condition for that block is > > if (wait_proc > && ! EQ (wait_proc->status, Qrun) > && ! connecting_status (wait_proc->status)) > > And the comment says "Don't wait for output from a non-running > process." Is the case here that the network connection was already > closed when we read from the sub-process? I thought that was not the > case. With my patch, it does hit that case, but on the third loop through so it's not exiting early. I also tried setting the timeout in `accept-process-output' to nil and that got me the same speedup as my fix but ended up looping 23 times so there's something really strange going on with the timeout here.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 12 Jun 2025 18:29:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 14:29:18 2025
Received: from localhost ([127.0.0.1]:60314 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uPmfu-0004vp-Ei
for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 14:29:18 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:36226)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uPmfr-0004vb-Lb
for 78773 <at> debbugs.gnu.org; Thu, 12 Jun 2025 14:29:16 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1uPmfm-0006E1-43; Thu, 12 Jun 2025 14:29:10 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=Af6wgzBN4MRUYabkAB/9Y0DUSzakS+tFNWI0+eMzQg8=; b=KB89Tz0APyMJ
h5wQdXG96oAawwEi+m22YlhoUzKkWBF6xwhvQhutAKfMLVzuC1DSX+hzVnQklvtxBzQ8UGVd0zzp0
6nYOQQyqj5yw1cKXhek5GzVfndJDaUsz+G8z/PybNZS2ySLzogYmBU7Z5wsZa/V/107gLNZyagWvV
NV6YyyY0DuJoguogqMjcaePsZQxKlhN0DnW4HGk64Ut9BMVNYzZFeQlAlLlvdimNQHXs12k3LZhns
r013wRr9GMDx+KM4XMTph9dIz53RM834RzoM49kNxZVzy/ltXbE8lp66jvXIabBipwjONsXtK309F
ywCKSc4eyvO3u2sSFVd9zA==;
Date: Thu, 12 Jun 2025 21:28:57 +0300
Message-Id: <86zfecerva.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Steven Allen <steven@HIDDEN>
In-Reply-To: <87y0twdeim.fsf@HIDDEN> (message from Steven Allen on Thu,
12 Jun 2025 11:02:41 -0700)
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
<87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN>
<87y0twdeim.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, rpluim@HIDDEN, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Steven Allen <steven@HIDDEN>
> Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN
> Date: Thu, 12 Jun 2025 11:02:41 -0700
>
> > Perhaps instrumenting wait_reading_process_output with printf's would
> > help to understand the control flow in the case of nil and non-nil
> > PROCESS argument?
>
> I'll do that today. I have a suspicion that fast requests waiting on a
> single process exit early here:
>
> https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h=81a3e4e51167be51c63eae682331210bc62f7280#n5562
The condition for that block is
if (wait_proc
&& ! EQ (wait_proc->status, Qrun)
&& ! connecting_status (wait_proc->status))
And the comment says "Don't wait for output from a non-running
process." Is the case here that the network connection was already
closed when we read from the sub-process? I thought that was not the
case.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 12 Jun 2025 18:02:52 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 14:02:52 2025 Received: from localhost ([127.0.0.1]:60200 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uPmGJ-0003C9-FA for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 14:02:51 -0400 Received: from fout-b2-smtp.messagingengine.com ([202.12.124.145]:47423) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <steven@HIDDEN>) id 1uPmGH-0003Bo-Gi for 78773 <at> debbugs.gnu.org; Thu, 12 Jun 2025 14:02:50 -0400 Received: from phl-compute-02.internal (phl-compute-02.phl.internal [10.202.2.42]) by mailfout.stl.internal (Postfix) with ESMTP id 8692F11401D1; Thu, 12 Jun 2025 14:02:43 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-02.internal (MEProxy); Thu, 12 Jun 2025 14:02:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm3; t=1749751363; x=1749837763; bh=Nz1pAdw9T8ZoJyoRqn0N8Q2VQeLvAp0W taPsTsALrWQ=; b=qfVq2WwkqxMK6TLif4rywZHTkG/oEV8zBdbvffSn+kNxZWQo AFhxdpeL36FAbQWvQlGhndjL5ISzZSDzrS9Bcs9NYPlTGqhIZdUUzOMITWCVjk0r j4W6IPDsL8Y+jjwZv9OBrjw+NkiMRsc6XMo/UtNwIGgu1QkJkYP1kzE+m7KY47qm XsohzdKiZSiooNuCGacDRpmD1sIw05CJQe0p0BF8OwiU2ass7eNzxxJcF8HSrGRo XoRWufyzpLqWGRvJUMLTx2Wv133jPuf5zYNy1XPd920pl0fH7Vvv/IJ9ytM3ZaO8 wlbT4o32mAyxak572PMJQVBD3aFPihMqDFlgtw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1749751363; x= 1749837763; bh=Nz1pAdw9T8ZoJyoRqn0N8Q2VQeLvAp0WtaPsTsALrWQ=; b=h P1L85GWxo9YwFbxxC7xb0awXDoSbrKeeSjJd3isH/wi+cAGsyySzeaNZfqkHFyr6 itxhbZJVfzcLY4CkU5xiffaLq4oED7My/rPQFDHL3y6H5dt7n597QuGCQb4VnjFN rePa5EPRzGpytt6BuCqWa+HHcy793GvZHNqqi7HAaQONso7kVVrWcMAClxz1OVoc gAkB+R7GvCInZVl8PGbjlN6kWCCz+SBrA8u1U5bWpOJBk7JU6JKHQldYZU1BHAnt 2iUKhAt9tvGeXWqH7SQ+y+gYm8QJle6U9IB3fdUM4px8qJwiVc49HiYF+FLg9VfS YdbVEws0iHgyxQsHRR9uA== X-ME-Sender: <xms:QhZLaAHF_TzU1xILInU4meff1Rc19Hkl2hlOSMcfD8KfLtGw3Vnoqg> <xme:QhZLaJXj7blx2BK0jydkStB9ev-Te5gyWVGcOzN3r3rzHK3DQVLD_9W38cN8wUqdg wDSf-r59E0atIsR89c> X-ME-Received: <xmr:QhZLaKIcft-E_fC32RmeioJ4YmZbZLnkUV1WSeshlmTYsllDCAXmeIqAGDGYmqCwtQcfF-rhovw8rEWlyINf4B3GXAeFPCE> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugdduheejgecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgfgsehtqhertddttdej necuhfhrohhmpefuthgvvhgvnhcutehllhgvnhcuoehsthgvvhgvnhesshhtvggsrghlih gvnhdrtghomheqnecuggftrfgrthhtvghrnhepudevlefgtdekheekgffhtdegheduhfeg tdfgffefteetgfetkeeivddthefhkeetnecuffhomhgrihhnpehgnhhurdhorhhgnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepshhtvghvvghn sehsthgvsggrlhhivghnrdgtohhmpdhnsggprhgtphhtthhopeegpdhmohguvgepshhmth hpohhuthdprhgtphhtthhopegvlhhiiiesghhnuhdrohhrghdprhgtphhtthhopehrphhl uhhimhesghhmrghilhdrtghomhdprhgtphhtthhopeejkeejjeefseguvggssghughhsrd hgnhhurdhorhhgpdhrtghpthhtoheplhgrrhhsihesghhnuhhsrdhorhhg X-ME-Proxy: <xmx:QhZLaCEpp5w2uTOeRy1Z29bU9N1i4A4k7E9d7fDXNlaMp3rVaE2VSA> <xmx:QhZLaGXtz6ZY3F64sgm7MLlz0uEVBvqxNtoFGxTTbjjRKMyQstCmlw> <xmx:QhZLaFNu08WzD4tZwmemEtrD7GFdwWCI_uC8DonbYkksFy7kxgUBOA> <xmx:QhZLaN0cRlG91ooUIsbUN9YaK0CmNhX_j8dqJ0ZLyd4Uu5LV4ASDkw> <xmx:QxZLaJC77It73QNJI8hQa1ov8Dkr4klujCw1qz__HLpu0CY-4y4C0jtb> Feedback-ID: ie8a146a7:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 12 Jun 2025 14:02:42 -0400 (EDT) From: Steven Allen <steven@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN>, Robert Pluim <rpluim@HIDDEN> Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections In-Reply-To: <86ldpxfc68.fsf@HIDDEN> References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> <86ldpxfc68.fsf@HIDDEN> Date: Thu, 12 Jun 2025 11:02:41 -0700 Message-ID: <87y0twdeim.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> The fact that it=CA=BCs localhost shouldn=CA=BCt matter. The code in que= stion is >> convoluted, as you well know, so figuring out what=CA=BCs going on needs= us >> to have a good understanding of which cases trigger this >> behaviour. But the evidence so far points at the handling of the >> timeout. > > In the case of a remote host, the speedup was marginal, so I think > the localhost does matter, somehow. It looks like a latency thing. It's not that it's localhost specific, it's that the effect is more pronounced on low-latency connections because something, somewhere is adding a tiny delay (likely the timeout). When I test against my router (10ms latency, how awful!), I get a 1.25x speedup (1.2x speedup with just your timeout change). I've also tested with IP addresses and reproduced the same results, so it's not related to DNS lookups. > Perhaps instrumenting wait_reading_process_output with printf's would > help to understand the control flow in the case of nil and non-nil > PROCESS argument? I'll do that today. I have a suspicion that fast requests waiting on a single process exit early here: https://https.git.savannah.gnu.org/cgit/git/emacs.git/tree/src/process.c?h= =3D81a3e4e51167be51c63eae682331210bc62f7280#n5562 >> But the evidence so far points at the handling of the timeout. I can say that we're definitely NOT waiting for the timeout: increasing the timeout (to, e.g., 1-2s) doesn't slow things down. I suspect having a really short timeout is causing us to abort wait_reading_process_output early before it hits some expensive part.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 12 Jun 2025 17:27:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 13:27:25 2025
Received: from localhost ([127.0.0.1]:60044 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uPli0-0000lq-Ow
for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 13:27:25 -0400
Received: from fhigh-a6-smtp.messagingengine.com ([103.168.172.157]:37201)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <steven@HIDDEN>)
id 1uPlhx-0000lQ-Io
for 78773 <at> debbugs.gnu.org; Thu, 12 Jun 2025 13:27:22 -0400
Received: from phl-compute-05.internal (phl-compute-05.phl.internal
[10.202.2.45])
by mailfhigh.phl.internal (Postfix) with ESMTP id C50101140215;
Thu, 12 Jun 2025 13:27:15 -0400 (EDT)
Received: from phl-mailfrontend-02 ([10.202.2.163])
by phl-compute-05.internal (MEProxy); Thu, 12 Jun 2025 13:27:15 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com;
h=cc:cc:content-type:content-type:date:date:from:from
:in-reply-to:in-reply-to:message-id:mime-version:references
:reply-to:subject:subject:to:to; s=fm3; t=1749749235; x=
1749835635; bh=bVgSSrGMo+vnzx7nAMxa7Q/0vIbgFeZyxYf2j+SgWtA=; b=a
wYYcXgK6HPoERVCp72MWkCbX62gDBY5Na98sc3jxE4paLbG1Jzk+Sk3xBkXfG7l8
svqmj30NqsdyAt486zPZ0B+91+L8GxuYciaRWAcmz6W7LnSt4dhU6VVsSKCeoJsi
v7yiyWuFpyt2qL7U0FTQS3TGyXZYaLaa1hYzqkVuQfEv6isOkygapRSxywlj17i8
z54qYElUx4OZoYJmNEHfKUM3PaY3xENXFT/QgC3g8oD3HGTQubn7FAan1t9VzTt8
/7J2+TA9/sWzYxIomBH5RvzAdFX4Pqa9Ov172lE2mhh2aluOJYjaOqVt6riL8a/1
q/EhZtZ1yVDTb5l5lxrIg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-type:content-type:date:date
:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
:message-id:mime-version:references:reply-to:subject:subject:to
:to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
1749749235; x=1749835635; bh=bVgSSrGMo+vnzx7nAMxa7Q/0vIbgFeZyxYf
2j+SgWtA=; b=B6t3gokCduYAamu/MoT97j1tFFO92MWGZMLA6d1hVeU2xoRJXZu
9mHxnbgEbKomqQIRUaCswwxbiXXMvuReLs+Yja1a2cPed27grvDvr7gxuoKcckoN
QXgjsl3qW2kjNnOfaFGBJXix7TjOzFxfMxG7fNWrZOHcyESA3VKRStgyXqfm4Hix
5EfLMoQbUd+FcPLCbGSS4VTqIeHLC46Obo+ozpZn5f/qhtL/6Hfvim/yIJcU4WtM
zP7sgNDEa7Bo6dAus0p2y4b5hHw1VZc3s556G0zmNtTapK5JJRP+5L4+5v34G10F
zR+NZvuFhQY6dcvmG5MWe7kSE0MWY1pSwtA==
X-ME-Sender: <xms:8w1LaMfHtm9-Yq_LknIic1DCc6J7-6oh29pWTIJEHVeyvUGdVcoZZQ>
<xme:8w1LaOP1uc1vRM1_awWUiwq0TWtNA9b0WPvaFaK-ANZF1gyl8MJueY50l0GfdwHPY
WQAf4Chvuf9J9NkMvE>
X-ME-Received: <xmr:8w1LaNj7uykStxQ_zfCHE79KrqupaMOV0z9J8m1h5sUvHIRbi1MiKZB7Wf-elrcsCwpZhDgb8tGSvyY58cbuidjDEyq_7GI>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugdduheeijecutefuodetggdotefrod
ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp
uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg
hnthhsucdlqddutddtmdenucfjughrpefhvfevufgjfhffkfggtgesthdtredttddttden
ucfhrhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivg
hnrdgtohhmqeenucggtffrrghtthgvrhhnpeehgeffteevudefuedvfefhgeduffethfel
uedtvddutdejgfetgeetgfeitdevgfenucffohhmrghinhepuddvjedrtddrtddrudenuc
evlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsthgvvhgv
nhesshhtvggsrghlihgvnhdrtghomhdpnhgspghrtghpthhtohepgedpmhhouggvpehsmh
htphhouhhtpdhrtghpthhtohepvghlihiisehgnhhurdhorhhgpdhrtghpthhtoheprhhp
lhhuihhmsehgmhgrihhlrdgtohhmpdhrtghpthhtohepjeekjeejfeesuggvsggsuhhgsh
drghhnuhdrohhrghdprhgtphhtthhopehlrghrshhisehgnhhushdrohhrgh
X-ME-Proxy: <xmx:8w1LaB-3UWlUGvsR3D4rGYZAmSEQG1-jj0TLjKLYLQN7tFdF5ExDDw>
<xmx:8w1LaIuWKDGN7Nb690joQUf23-qYD1HO106Wlphwnz2lVQs6X0kOfA>
<xmx:8w1LaIGXkzhHYwpieN0lhwvhmS4GceJLG1XwSkPnCnXknHDv1SXuyg>
<xmx:8w1LaHP5fqwPV-IUqnW1XV90hVroh7WyxCc7kp1AtFr4fWxhQHWGQQ>
<xmx:8w1LaNNmwum5fro56zmaoQKTUWe_vCcYJKyt600jJzmu11lgezOkq68g>
Feedback-ID: ie8a146a7:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
12 Jun 2025 13:27:14 -0400 (EDT)
From: Steven Allen <steven@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>, Robert Pluim <rpluim@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <8634c5h30i.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
Date: Thu, 12 Jun 2025 10:27:13 -0700
Message-ID: <871proeuq6.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
Eli Zaretskii <eliz@HIDDEN> writes:
>> With this one change, this code goes back to being as slow as it was before.
>
> What happens if you leave the 1st argument of accept-process-output at
> its current nil value, but change the 2nd argument to be 0.005 instead
> of 0.05 (i.e., 10 times smaller)?
Now testing with:
emacs -Q --load=url.el --batch --eval \
"(benchmark 100 '(url-retrieve-synchronously \"http://127.0.0.1:8000\"))"
- Changing the timeout from 0.05 to 0.005 drops the time from 1.25
seconds to 0.63 seconds (~2x faster).
- My patch drops it to 0.056 (22x faster in a clean environment).
However note: changing the timeout from 0.05 to 0.005 makes the wait
loop spin 6 times for higher latency (non-localhost, ~40ms) requests whereas it
loops exactly once with the default timeout.
> Also, how many other sub-processes (of any kind, not just network
> processes) do you have in that session when you are testing this
> issue?
From list-processes, 13-14 when testing this originally (including network
processes, listening sockets, etc.).
>> However, the patch in question also changed the rest of
>> `url-retrieve-synchronously' so I'm hoping the issue lies elsewhere?
>
> Unfortunately, I doubt that we will get any useful answers to this
> question. We need to understand better why asking for output from a
> single process has such a dramatic effect in your case with localhost
> requests. If it happens only with localhost requests, perhaps we
> could make some changes only for that case.
Testing with a server with a 30ms latency (tested with "emacs -Q"), I get:
1. A 1.4x speedup with my patch.
2. A 1.3x speedup by simply changing the timeout.
But there's a lot more noise in the higher-latency tests.
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 12 Jun 2025 11:10:35 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 07:10:34 2025 Received: from localhost ([127.0.0.1]:56843 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uPfpK-0002Wn-Em for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 07:10:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42258) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uPfpI-0002W6-D2 for 78773 <at> debbugs.gnu.org; Thu, 12 Jun 2025 07:10:33 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uPfpC-0004r3-AI; Thu, 12 Jun 2025 07:10:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=tm0rVmm18pcFN8P+GaMLSvijCMxHGm/Hf9RkFBRIN+4=; b=NhrGIJuidVAjnN1G2cIM ReYzxKakniN6ulsQRbk/gZ094uVMxpN1qwIQUp5bY9G2u2A+5JjiOmXMEaAVyQJvxs/fmsPCwzm8f FKg+UPLj9O/wGxVPPKQOHXYtZB2KmQP1GcLxl1Ue6R+sZacPhDDEitRbPjxZ/Yaq96MaOs/8gVwf9 JxJZgkJ+EHLxjauqpCcwuXJhxFNzxzvrh0DCW7fxUYqVwAElFcstt3chp8CFE+yc3yxeoIK36spDc XmUTOl1KjwncbX6uOU/ellNOek+/qrzEMjs8ftihs1lMbAsJE1juIAiF1F4M84RF+gtx/aj7Qznpo 0cp8W4e29Hd6jQ==; Date: Thu, 12 Jun 2025 14:10:23 +0300 Message-Id: <86ldpxfc68.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Robert Pluim <rpluim@HIDDEN> In-Reply-To: <87frg59wsa.fsf@HIDDEN> (message from Robert Pluim on Thu, 12 Jun 2025 10:41:41 +0200) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN> <87frg59wsa.fsf@HIDDEN> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN, steven@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Robert Pluim <rpluim@HIDDEN> > Cc: Steven Allen <steven@HIDDEN>, 78773 <at> debbugs.gnu.org, > larsi@HIDDEN > Date: Thu, 12 Jun 2025 10:41:41 +0200 > > >>>>> On Thu, 12 Jun 2025 09:45:17 +0300, Eli Zaretskii <eliz@HIDDEN> said: > > Eli> What happens if you leave the 1st argument of accept-process-output at > Eli> its current nil value, but change the 2nd argument to be 0.005 instead > Eli> of 0.05 (i.e., 10 times smaller)? > > That speeds it up, although not as dramatically > > Eli> Also, how many other sub-processes (of any kind, not just network > Eli> processes) do you have in that session when you are testing this > Eli> issue? > > The answer to that is important, since there are a couple of loops in > `wait_reading_process_output' that depend on that, and can trigger DNS > or TLS operations as a result. But Iʼve just tested this in 'emacs -Q' > with zero processes, and I see the speedup. If this happens with a single sub-process, it's strange. It means that we somehow wait for the full timeout even though some output from the single sub-process arrives, which should have ended the wait. Or what am I missing? > Eli> Unfortunately, I doubt that we will get any useful answers to this > Eli> question. We need to understand better why asking for output from a > Eli> single process has such a dramatic effect in your case with localhost > Eli> requests. If it happens only with localhost requests, perhaps we > Eli> could make some changes only for that case. > > Eli> Robert, any ideas or suggestions? > > The fact that itʼs localhost shouldnʼt matter. The code in question is > convoluted, as you well know, so figuring out whatʼs going on needs us > to have a good understanding of which cases trigger this > behaviour. But the evidence so far points at the handling of the > timeout. In the case of a remote host, the speedup was marginal, so I think the localhost does matter, somehow. Perhaps instrumenting wait_reading_process_output with printf's would help to understand the control flow in the case of nil and non-nil PROCESS argument?
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at 78773) by debbugs.gnu.org; 12 Jun 2025 08:41:52 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 04:41:52 2025
Received: from localhost ([127.0.0.1]:56182 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uPdVP-0006P4-P2
for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 04:41:52 -0400
Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]:49456)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.84_2) (envelope-from <rpluim@HIDDEN>) id 1uPdVN-0006Oc-Lc
for 78773 <at> debbugs.gnu.org; Thu, 12 Jun 2025 04:41:50 -0400
Received: by mail-wr1-x42b.google.com with SMTP id
ffacd0b85a97d-3a365a6804eso394619f8f.3
for <78773 <at> debbugs.gnu.org>; Thu, 12 Jun 2025 01:41:49 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1749717703; x=1750322503; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id
:reply-to; bh=XrJ2pQecjJn1tKoq9OI5hM0FHaF3cDox5Daf0gaOjsI=;
b=nffA8udnwjB5sshlXwdV+7zIazcQzw7Ljxje5gcCQ5UovppDA+IXw3VpEWl4mkh5LE
WE0PLnLjspNGYdzCMqTmbKxDK+4ciBhX6bpso7t22vFPZDpxG8sZuoGFFmImWVkNc4hd
kQ8RqdoC4BDM0kA/iWHS+7CFeq0apQsTRhfNzYvHAirsLfKh3+fs0VfaliRyGoqUd1/Z
Tr9k3u646saF4b9sC3cpZ2DZqtmqTwOj8bEdcgXR4H9BLa1E83YmXzAQKbhM9eJaJJpC
WH8GbKziLjImq9nFsh5mkGo37d+qAiG6REEJ6HeA9VLNpfuCMfjktviywe4/PWDWnngy
NMoQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1749717703; x=1750322503;
h=content-transfer-encoding:mime-version:message-id:date:references
:in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=XrJ2pQecjJn1tKoq9OI5hM0FHaF3cDox5Daf0gaOjsI=;
b=DU8IZQZ7fV8yZYxjuEDQVRjpehds/xOr5gpEXAd2ETUyjE7tvrr7msF6EUUo6rL0i3
EJJBoW4DbNuKM9lOJX0ViNPx7m718ClXd2Rq8kcXAbwvfrisXX5vah3VOT3bcSj8cLss
C6mGaicZijctfY7YUhWCpU9jwmBnaJKe41z5OvueS3DMrV/kzBxHhrJ7IRqUkRYOc9NV
4ZuilC+ufpPH9Aeq76jX+ynmFf/C2gXGtQRx0zTsw/Fi9MboXCYXdgl9s/r6swIaYB/W
hvYmt3Wstg+miqfxppJ2zEh2iPZ+0FVwiFz7STOrTqTAU5ubupvLzZHpcGBRBJsfJVu0
rebg==
X-Forwarded-Encrypted: i=1;
AJvYcCUjLqP00YPqaXZ3fGxkuK4RlauzOBclQXstyP4KwaGs8Zq9UIlnET5yyyLGRohNeIw+ZciS2w==@debbugs.gnu.org
X-Gm-Message-State: AOJu0Yy+vza2AnE2CO30Oiv5lHw1GG5WPKDMlnAwzvlIl68w1ccv1+s0
HGd5E9X1zN4Ptal9Rpr7w41PojR9ll2NQtcxzriJg6IMonbWThhjKUcD7msiyWEF
X-Gm-Gg: ASbGncsdIZtw56Pig92KfFkg9i3GLqx49CmKDOSS20AC7+KNTl1jZlsilXmq3BkmeGT
PWG47jq9pWGL0R4u6T0P46RsaFEmKq7fRH8JsZkVgesywOAauYXB2ChGp2MiaywCyIfbAk+Nk+h
OkUXyMo3fYXcklzRT4MgElKiyDtV+rRhwNkQ2Cg0KcIrA3RraYlxVoy9eG6w+7fEFv/XAkY5J5i
N0NJC7RUDai9P1qDQEyeV0TaVVGdGQJjI0iUXc3VmjgMpnnodNwIm+NZfL63nZVEw4mg0z19J9Y
KknSL14WJWaC2tSfHSTA9dnJZCJwoY8XQTz7Y5lGGShGBAs1aQ==
X-Google-Smtp-Source: AGHT+IGFXV4fL+7jg3uacAsPFNonGQV+sNwyTu23dYDfI0ZZMtoiIsG4ShUBfGSQqsGzrpx6usq4dA==
X-Received: by 2002:a05:6000:2283:b0:3a3:7ba5:93a5 with SMTP id
ffacd0b85a97d-3a55869cd37mr5292303f8f.26.1749717703021;
Thu, 12 Jun 2025 01:41:43 -0700 (PDT)
Received: from rltb ([2a01:e0a:3f3:fb51:949a:873a:3ff3:56b6])
by smtp.gmail.com with ESMTPSA id
ffacd0b85a97d-3a561b65ab2sm1324278f8f.96.2025.06.12.01.41.42
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Thu, 12 Jun 2025 01:41:42 -0700 (PDT)
From: Robert Pluim <rpluim@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for
low-latency connections
In-Reply-To: <8634c5h30i.fsf@HIDDEN>
References: <8734c51u0i.fsf@HIDDEN> <8634c5h30i.fsf@HIDDEN>
Date: Thu, 12 Jun 2025 10:41:41 +0200
Message-ID: <87frg59wsa.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 78773
Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN, Steven Allen <steven@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
(I dropped dick chiang from the CC=CA=BCs)
>>>>> On Thu, 12 Jun 2025 09:45:17 +0300, Eli Zaretskii <eliz@HIDDEN> said:
Eli> What happens if you leave the 1st argument of accept-process-outpu=
t at
Eli> its current nil value, but change the 2nd argument to be 0.005 ins=
tead
Eli> of 0.05 (i.e., 10 times smaller)?
That speeds it up, although not as dramatically
Eli> Also, how many other sub-processes (of any kind, not just network
Eli> processes) do you have in that session when you are testing this
Eli> issue?
The answer to that is important, since there are a couple of loops in
`wait_reading_process_output' that depend on that, and can trigger DNS
or TLS operations as a result. But I=CA=BCve just tested this in 'emacs -Q'
with zero processes, and I see the speedup.
>> **Historical context**
>>=20
>> `url-retrieve-synchronously' was changed to wait on "nil" in:
>>=20
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D49897,
>>=20
>> Motivated by this bug report:
>>=20
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D49861
>>=20
>> However, the patch in question also changed the rest of
>> `url-retrieve-synchronously' so I'm hoping the issue lies elsewhere?
Eli> Unfortunately, I doubt that we will get any useful answers to this
Eli> question. We need to understand better why asking for output from=
a
Eli> single process has such a dramatic effect in your case with localh=
ost
Eli> requests. If it happens only with localhost requests, perhaps we
Eli> could make some changes only for that case.
Eli> Robert, any ideas or suggestions?
The fact that it=CA=BCs localhost shouldn=CA=BCt matter. The code in questi=
on is
convoluted, as you well know, so figuring out what=CA=BCs going on needs us
to have a good understanding of which cases trigger this
behaviour. But the evidence so far points at the handling of the
timeout.
Robert
--=20
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.Received: (at 78773) by debbugs.gnu.org; 12 Jun 2025 06:45:31 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 02:45:31 2025 Received: from localhost ([127.0.0.1]:55503 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1uPbgo-0005Ht-EO for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 02:45:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47332) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1uPbgl-0004xZ-S3 for 78773 <at> debbugs.gnu.org; Thu, 12 Jun 2025 02:45:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1uPbgf-0000Ty-Fi; Thu, 12 Jun 2025 02:45:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=6WgUHcbL6EIGgvPRIAJgGD1HhM7axmxywI0n/DySrtg=; b=Ciy08OQHewcl EYz1piHWxPv/y5HEMyceofnHi3j+5oa1lecpmxzNl/MknyZmzolSqeqQUwZkAamfUOEUeS1lNmMK8 fSbFBO8p3/pcFq31yzU29NdGFEzJLkMSFvVp8cw+O4wqBasZps7ZeXANweGkqbVBwZJQxlB3rv8FH gyGjJMcIuM5sgTFzRhsKwctT4wrTuKz3vS9Tvsmp1/68Jg+oS7W1IG0X5iodJS8SsRxcwrxR3QOrl um5Q0q3r3PTyB44Xwn5OihXkZ4xkSLr3MiljMDuLOxC79OV+HyeMZLCNHnIhKQS611M4e1NeYGAOn 7Vy0nnF5CSl7HI3q33TVOw==; Date: Thu, 12 Jun 2025 09:45:17 +0300 Message-Id: <8634c5h30i.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Steven Allen <steven@HIDDEN>, Robert Pluim <rpluim@HIDDEN> In-Reply-To: <8734c51u0i.fsf@HIDDEN> (bug-gnu-emacs@HIDDEN) Subject: Re: bug#78773: [PATCH] Speedup url-retrieve-synchronously for low-latency connections References: <8734c51u0i.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 78773 Cc: 78773 <at> debbugs.gnu.org, larsi@HIDDEN, dick.r.chiang@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Cc: Lars Ingebrigtsen <larsi@HIDDEN>, dick <dick.r.chiang@HIDDEN> > Date: Wed, 11 Jun 2025 21:08:45 -0700 > From: Steven Allen via "Bug reports for GNU Emacs, > the Swiss army knife of text editors" <bug-gnu-emacs@HIDDEN> > > **Present context:** > > I'm running into an issue in emacs-syncthing [1] where making a few > localhost network requests are taking a full second (blocking Emacs) > when they should be instantaneous. While digging into > `url-retrieve-synchronously', I found that waiting on the correct > network process instead of passing nil to `accept-process-output' made > these network requests instantaneous (although I have no idea why). See > the attached patch. > > [1]: https://github.com/KeyWeeUsr/emacs-syncthing > > To test this patch, you can: > > 1. Run a simple web server on localhost. I usually use > > python -m http.server --bind 127.0.0.1 8000 > > 2. Evaluate: (benchmark 100 '(url-retrieve-synchronously "http://127.0.0.1:8000")) > > With this patch, this form is ~16x faster on my machine. I've also > tested this against a remote machine with a ~45ms latency and found a > 1.25x speedup. > > I've confirmed that this isn't busy-waiting by modifying this code to > print a message each time it loops: it loops the same number of times > with or without my patch. > > I've confirmed that this speedup is strictly due to passing the target > process to `accept-process-output' by applying my patch then changing > JUST "proc" to "nil": > > ;; ms, so split the difference. > (accept-process-output proc 0.05)) > > to > > ;; ms, so split the difference. > (accept-process-output nil 0.05)) > > With this one change, this code goes back to being as slow as it was before. What happens if you leave the 1st argument of accept-process-output at its current nil value, but change the 2nd argument to be 0.005 instead of 0.05 (i.e., 10 times smaller)? Also, how many other sub-processes (of any kind, not just network processes) do you have in that session when you are testing this issue? > **Historical context** > > `url-retrieve-synchronously' was changed to wait on "nil" in: > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49897, > > Motivated by this bug report: > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49861 > > However, the patch in question also changed the rest of > `url-retrieve-synchronously' so I'm hoping the issue lies elsewhere? Unfortunately, I doubt that we will get any useful answers to this question. We need to understand better why asking for output from a single process has such a dramatic effect in your case with localhost requests. If it happens only with localhost requests, perhaps we could make some changes only for that case. Robert, any ideas or suggestions?
bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
Received: (at submit) by debbugs.gnu.org; 12 Jun 2025 04:09:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 12 00:09:24 2025
Received: from localhost ([127.0.0.1]:54799 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1uPZFj-0007ME-BA
for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 00:09:23 -0400
Received: from lists.gnu.org ([2001:470:142::17]:41560)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <steven@HIDDEN>)
id 1uPZFg-0007Km-3J
for submit <at> debbugs.gnu.org; Thu, 12 Jun 2025 00:09:21 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <steven@HIDDEN>)
id 1uPZFF-0000ZI-19
for bug-gnu-emacs@HIDDEN; Thu, 12 Jun 2025 00:08:53 -0400
Received: from fout-b1-smtp.messagingengine.com ([202.12.124.144])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <steven@HIDDEN>)
id 1uPZFB-0008GQ-GT
for bug-gnu-emacs@HIDDEN; Thu, 12 Jun 2025 00:08:52 -0400
Received: from phl-compute-05.internal (phl-compute-05.phl.internal
[10.202.2.45])
by mailfout.stl.internal (Postfix) with ESMTP id 12B261140175;
Thu, 12 Jun 2025 00:08:48 -0400 (EDT)
Received: from phl-mailfrontend-01 ([10.202.2.162])
by phl-compute-05.internal (MEProxy); Thu, 12 Jun 2025 00:08:48 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stebalien.com;
h=cc:cc:content-type:content-type:date:date:from:from
:in-reply-to:message-id:mime-version:reply-to:subject:subject:to
:to; s=fm3; t=1749701327; x=1749787727; bh=xCarY22GUUeg9O7eLpsAJ
MZVbSv1EAQk8gx6oifdD9c=; b=TJ2hWSEHRZ6hDbjtk5qC1tFNENvqAms5c+pmu
MLJ0HZz8I8aplng0JkkRpUEXaMIMrzzkGz2oNvSdJv5FBoAelNCsv7s7anGp1oji
lLNrcgDFhDU9a83eWSJTawcSYmsoCUwUhLJyN1yO6VBQbONI+gWCRQLRaB0th7Tj
cY76VHRaQ1CteZ3+4jpzXCqo5osfW8r0ibnx7e2EG7wwPoRyuPcPYn7mki/ioSPm
BYgYka8p3ETFDN2/Q9QUaltaMaUNhINAWlOMF2ZGlY+MmlC082FpdVQCU8gbTiMx
2Dpvz4/Q8qS9qYlMIfNy/IqAefc55bRtfjFWpXdosqJ50/gcw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-type:content-type:date:date
:feedback-id:feedback-id:from:from:in-reply-to:message-id
:mime-version:reply-to:subject:subject:to:to:x-me-proxy
:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1749701327; x=
1749787727; bh=xCarY22GUUeg9O7eLpsAJMZVbSv1EAQk8gx6oifdD9c=; b=o
iu8vBTgFpkJrMbyWdXq3D9rXxmHa/zFNJ/NY2diELIFlErSdwIHEjzv3LgYqUWxp
HEsR/9y/9Z8fZdegKdMqmlF5xNTYvx4v7iCgJhWzJ70OiqsjH6IXD0m+0qiVq6G3
z0TX0FXoLsHqa2fCFozf4LqYAUjcqUHEj2zZYqSMBNG1NcmR89ApHsJhjc7KEd2c
eOyJHtHUKF1RgL86k0KyyirnuYhbSllqTCG6q6l8IaJkHe9d67faopMP0noA8R3O
46ZQqMMmTl7BdoUM8Eb8tHJm+doEvw6Yv11SHgZhnrST/GZSRO7Wgg9gfk8SIm8c
54uAUBYO5d/IrGHgKiuaw==
X-ME-Sender: <xms:z1JKaPHglSxv5cUb3tyNULK1B3y1nKeC9Cf-ry6yCBKSCmN9bSMuXQ>
<xme:z1JKaMVoC0ibJ7SpW_hy2EgBJHs43tiU4mRjDOClP7FkOTSckK3jyp182d8-FruRg
6HXq9vfGx-3iMhOw74>
X-ME-Received: <xmr:z1JKaBLy-E-w_7OwE1AzhFNJyDsDurVI8RdqQJj_6d9MPK7eKkleZ30ERLEft6S0gDsN1Her0hZ0ZhZnvE_uBuTWZqSn3A0>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtddugddugedtjecutefuodetggdotefrod
ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp
uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg
hnthhsucdlqddutddtmdenucfjughrpefhvfevufffkfggtgesmhdtreertddttdenucfh
rhhomhepufhtvghvvghnucetlhhlvghnuceoshhtvghvvghnsehsthgvsggrlhhivghnrd
gtohhmqeenucggtffrrghtthgvrhhnpeeftedvfeeggfekvddtvdegkeefkeeggfegveef
gfeihfdugfdvgedttdejtdehffenucffohhmrghinhepghhithhhuhgsrdgtohhmpdduvd
ejrddtrddtrddupdhgnhhurdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr
rghmpehmrghilhhfrhhomhepshhtvghvvghnsehsthgvsggrlhhivghnrdgtohhmpdhnsg
gprhgtphhtthhopeefpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegsuhhgqdhg
nhhuqdgvmhgrtghssehgnhhurdhorhhgpdhrtghpthhtohepughitghkrdhrrdgthhhirg
hnghesghhmrghilhdrtghomhdprhgtphhtthhopehlrghrshhisehgnhhushdrohhrgh
X-ME-Proxy: <xmx:z1JKaNGp9r8bg7igF7ozibwsgLJIMK_275W0owIJA9RShuE2smcc1A>
<xmx:z1JKaFXYh1_u93BLAmQHRUyX_hw1lYTQgDMd549BFZ1bac4UXJGSvQ>
<xmx:z1JKaIMWdl4VHJpKY7Y2-nnbLtTc2OU9NlOjFXkF_8HkOg-083my6Q>
<xmx:z1JKaE3Z4h0qLcXqwTaSMCsOcqWfxnsBWm62a2CBsD7VTgrijra2uA>
<xmx:z1JKaHa0Wqx5rLFKH1EkjIaXGlvXNGoe19cFTHj5rajoZ_Y8X-7h9u12>
Feedback-ID: ie8a146a7:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
12 Jun 2025 00:08:47 -0400 (EDT)
From: Steven Allen <steven@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: [PATCH] Speedup url-retrieve-synchronously for low-latency connections
X-Debbugs-Cc:
Date: Wed, 11 Jun 2025 21:08:45 -0700
Message-ID: <8734c51u0i.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
Received-SPF: pass client-ip=202.12.124.144; envelope-from=steven@HIDDEN;
helo=fout-b1-smtp.messagingengine.com
X-Spam_score_int: -15
X-Spam_score: -1.6
X-Spam_bar: -
X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
NUMERIC_HTTP_ADDR=1.242, RCVD_IN_DNSWL_LOW=-0.7,
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001,
SPF_HELO_PASS=-0.001, SPF_PASS=-0.001,
WEIRD_PORT=0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.7 (/)
X-Debbugs-Envelope-To: submit
Cc: Lars Ingebrigtsen <larsi@HIDDEN>, dick <dick.r.chiang@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.3 (/)
--=-=-=
Content-Type: text/plain
Tags: patch
**Present context:**
I'm running into an issue in emacs-syncthing [1] where making a few
localhost network requests are taking a full second (blocking Emacs)
when they should be instantaneous. While digging into
`url-retrieve-synchronously', I found that waiting on the correct
network process instead of passing nil to `accept-process-output' made
these network requests instantaneous (although I have no idea why). See
the attached patch.
[1]: https://github.com/KeyWeeUsr/emacs-syncthing
To test this patch, you can:
1. Run a simple web server on localhost. I usually use
python -m http.server --bind 127.0.0.1 8000
2. Evaluate: (benchmark 100 '(url-retrieve-synchronously "http://127.0.0.1:8000"))
With this patch, this form is ~16x faster on my machine. I've also
tested this against a remote machine with a ~45ms latency and found a
1.25x speedup.
I've confirmed that this isn't busy-waiting by modifying this code to
print a message each time it loops: it loops the same number of times
with or without my patch.
I've confirmed that this speedup is strictly due to passing the target
process to `accept-process-output' by applying my patch then changing
JUST "proc" to "nil":
;; ms, so split the difference.
(accept-process-output proc 0.05))
to
;; ms, so split the difference.
(accept-process-output nil 0.05))
With this one change, this code goes back to being as slow as it was before.
**Historical context**
`url-retrieve-synchronously' was changed to wait on "nil" in:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49897,
Motivated by this bug report:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49861
However, the patch in question also changed the rest of
`url-retrieve-synchronously' so I'm hoping the issue lies elsewhere?
In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, cairo version
1.18.4) of 2025-06-11 built on Laptop
Repository revision: 87244ef1661f9e9d7c08f65047551e8a34cd92b2
Repository branch: makepkg
Windowing system distributor 'The X.Org Foundation', version 11.0.12101016
System Description: Arch Linux
Configured using:
'configure
'CPPFLAGS=-I/run/user/1000/build/emacs-git/src/mps-git/build/include '
'LDFLAGS=-L/run/user/1000/build/emacs-git/src/mps-git/build/lib -Wl,-O1
-Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
-Wl,-z,pack-relative-relocs -flto=auto' --prefix=/usr --sysconfdir=/etc
--libexecdir=/usr/lib --localstatedir=/var --mandir=/usr/share/man
--with-gameuser=:games --with-modules --without-m17n-flt
--without-selinux --without-pop --without-gconf --disable-gc-mark-trace
--with-mps=yes --enable-link-time-optimization
--with-native-compilation=yes --with-xinput2 --with-x-toolkit=no
--without-toolkit-scroll-bars --without-xaw3d --without-gsettings
--with-cairo-xcb --without-xft --with-sound=no --with-tree-sitter
--without-gpm --without-compress-install
'--program-transform-name=s/\([ec]tags\)/\1.emacs/'
'CFLAGS=-march=native -mtune=native -O3 -pipe -fno-plt -fexceptions
-Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security
-fstack-clash-protection -fcf-protection -fomit-frame-pointer
-fno-math-errno -fno-trapping-math -fno-math-errno -fno-trapping-math
-flto=auto''
--=-=-=
Content-Type: text/patch
Content-Disposition: attachment;
filename=0001-Speedup-url-retrieve-synchronously-for-low-latency-c.patch
From 4529f63f3c55b40fd43642592cd4dc7162766c7f Mon Sep 17 00:00:00 2001
From: Steven Allen <steven@HIDDEN>
Date: Wed, 11 Jun 2025 20:19:01 -0700
Subject: [PATCH] Speedup url-retrieve-synchronously for low-latency
connections
On my machine, this provides a ~16x speedup for localhost request and a
1.28x speedup for a 50ms latency connection.
* lisp/url/url.el (url-retrieve-synchronously): Wait for output from the
target process instead of waiting for output from any process.
---
lisp/url/url.el | 23 +++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/lisp/url/url.el b/lisp/url/url.el
index 090a952cf4c..c8dc002b0ac 100644
--- a/lisp/url/url.el
+++ b/lisp/url/url.el
@@ -270,17 +270,20 @@ url-retrieve-synchronously
(kill-buffer proc-buffer))
;; Accommodate hack in commit 55d1d8b.
(setq proc-buffer redirect-buffer)))
- (when-let* ((proc (get-buffer-process proc-buffer)))
- (when (memq (process-status proc)
+ (if-let* ((proc (get-buffer-process proc-buffer)))
+ (if (memq (process-status proc)
'(closed exit signal failed))
- ;; Process sentinel vagaries occasionally cause
- ;; url-retrieve to fail calling callback.
- (unless data-buffer
- (url-debug 'retrieval "Dead process %s" url)
- (throw 'done 'exception))))
- ;; Querying over consumer internet in the US takes 100
- ;; ms, so split the difference.
- (accept-process-output nil 0.05)))
+ ;; Process sentinel vagaries occasionally cause
+ ;; url-retrieve to fail calling callback.
+ (unless data-buffer
+ (url-debug 'retrieval "Dead process %s" url)
+ (throw 'done 'exception))
+ ;; Querying over consumer internet in the US takes 100
+ ;; ms, so split the difference.
+ (accept-process-output proc 0.05))
+ ;; This case shouldn't happen but this keeps us from
+ ;; spinning and locking up Emacs if it does.
+ (accept-process-output nil 0.05))))
;; Kill the process buffer on redirects.
(when (and data-buffer
(not (eq data-buffer proc-buffer)))
--
2.49.0
--=-=-=--
Steven Allen <steven@HIDDEN>:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#78773; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.