Received: (at 64735) by debbugs.gnu.org; 14 Sep 2023 05:41:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 14 01:41:40 2023 Received: from localhost ([127.0.0.1]:36572 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qgf6h-0001jj-UP for submit <at> debbugs.gnu.org; Thu, 14 Sep 2023 01:41:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56714) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qgf6f-0001jU-7c for 64735 <at> debbugs.gnu.org; Thu, 14 Sep 2023 01:41:39 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qgf6S-00083R-UQ; Thu, 14 Sep 2023 01:41:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=CFuYYF71FoRm+7xntSM48ncTkuKifQE0ZqqGJ+bJEd8=; b=YwNZsL35mCFkOshehrsN Xxqua8iG1fvsCx4nGJe6ptj8d8g+Xvo0rjMrq9VY+PppgZywmsXL+UwUBfO7nB5t84oxg9l/sDd1R KrN7C3NsYX3CU4xObxVjyzDWgJpemf2aAUyXvQ5oeQq+l0ObX+r+/011+zjiNm87FCYDGzvo6BOwp tHc/T1J4CIHbuSl8cvtfCFnBG8fQkwf1RYWNHRVfpIq0BNmhWGx+XhDbRHBqjSRexzYyOsltbB7nk cM/YYNJqq1e6ud++5mbSo34hm16n3hgLHYDOJNOwBMQXvrvDHSoYmkgMkd/6h7guEnNmdefCYMMNA NZGSCEyNi/FaGw==; Date: Thu, 14 Sep 2023 08:41:11 +0300 Message-Id: <83bke5mhvs.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <e46f996a-0bed-8796-23d7-c9c8b7d428be@HIDDEN> (message from Dmitry Gutov on Wed, 13 Sep 2023 23:38:29 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> <831qf3pd1y.fsf@HIDDEN> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN> <83sf7jnq0m.fsf@HIDDEN> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN> <83ledanvzw.fsf@HIDDEN> <bc4bb3de-d433-7249-cb53-2f312656589d@HIDDEN> <83r0n2m7qz.fsf@HIDDEN> <26afa109-9ba3-78a3-0e68-7585ae8e3a19@HIDDEN> <83il8dna30.fsf@HIDDEN> <e46f996a-0bed-8796-23d7-c9c8b7d428be@HIDDEN> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Wed, 13 Sep 2023 23:38:29 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > > I think these questions are slightly premature. We should first have > > the implementation of that filter, and then look for candidates that > > could benefit from it. > > The implementation in that patch looks almost complete to me, unless you > have any further comments. Fine, then please post a complete patch with all the bells and whistles, and let's have it reviewed more widely. (I suggest a new bug report, as this one is already prohibitively long to follow, includes unrelated issues, and I fear some people will ignore patches posted to it). I think there are a few subtleties we still need to figure out. > The main difference would be the change in > the dispatch comparison from > > if (p->filter == Qinternal_default_process_filter) > > to > > if (p->filter == Qbuffer) Btw, both of the above are mistakes: you cannot compare Lisp objects as if they were simple values. You must use EQ. > This function gives PROCESS the filter function FILTER. If FILTER > is ‘nil’, it gives the process the default filter, which inserts > the process output into the process buffer. If FILTER is ‘t’, > Emacs stops accepting output from the process, unless it’s a > network server process that listens for incoming connections. > > What can we add? > > If FILTER is ‘buffer’, it works like the default one, only a bit faster. > > ? If FILTER is the symbol ‘buffer’, it works like the default filter, but makes some shortcuts to be faster: it doesn't adjust markers and the process mark (something else?). Of course, the real text will depend on what the final patch will look like: I'm not yet sure I understand which parts of internal-default-process-filter you want to keep in this alternative filter. (If you intend to keep all of them, it might be better to replace internal-default-process-filter completely, perhaps first with some variable exposed to Lisp which we could use to see if the new one causes issues.) > > My tendency is to change only callers which > > are in many cases expected to get a lot of stuff from a subprocess, so > > shell buffers are probably out. But we could discuss that later. > > When I'm thinking of start-file-process-shell-command, I have in mind > project--files-in-directory, which currently uses > process-file-shell-command. Though I suppose most cases would be more > easily converted to use make-process (like xref-matches-in-files uses > process-file for launching a shell pipeline already). > > I was also thinking about Flymake backends because those work in the > background. The outputs are usually small, but can easily grow in rare > cases, without particular limit. Flymake also runs in the background, > meaning whatever extra work it has to do (or especially GC pressure), > affects the delays when editing. I think we will have to address these on a case by case basis. The issues and aspects are not trivial and sometimes subtle. We might even introduce knobs to allow different pipe sizes if there's no one-fits-all value for a specific function using these primitives. > >> I'm not sure what negatives to test for, though. Raising the limit 10x > >> is unlikely to lead to an OOM, but I guess some processes could grow > >> higher latency?.. > > > > With a large buffer and small subprocess output we will ask the OS for > > a large memory increment for no good reason. Then the following GC > > will want to compact the gap, which means it will be slower. > > I wonder what scenario that might become apparent in. Launching many > small processes at once? Can't think of a realistic test case. One process suffices. The effect might not be significant, but slowdowns due to new features are generally considered regressions.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 13 Sep 2023 20:38:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 13 16:38:57 2023
Received: from localhost ([127.0.0.1]:36183 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qgWdQ-0007et-Se
for submit <at> debbugs.gnu.org; Wed, 13 Sep 2023 16:38:57 -0400
Received: from wout3-smtp.messagingengine.com ([64.147.123.19]:57285)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qgWdJ-0007ea-Ku
for 64735 <at> debbugs.gnu.org; Wed, 13 Sep 2023 16:38:51 -0400
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
by mailout.west.internal (Postfix) with ESMTP id 3C2D832009A4;
Wed, 13 Sep 2023 16:38:34 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute5.internal (MEProxy); Wed, 13 Sep 2023 16:38:34 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm3; t=
1694637513; x=1694723913; bh=sELkUu/ROWCusXj+gqW6rAA6N72rSwsQEOe
VjtxuYlQ=; b=hO5uJVW2YRocSDOQrqWSaNxW8PvO84YbuLiSBz5dDyyF24X/I48
aO/i4bg6WXC4GPCT821FwrqrrGawaYGG0741XCz1A139pOllBNvWOzYo/8QmPt/l
FTFq+AvVnCkzD1ICQpL++7USCOTwQSdAz6gKcAutkoqsocdDWrDjhoYxLx6zTJEW
rpH/uUuoccsQopp5iBMZAhyV6rx5dnWYjEQ5Kx196PWC8RqFGiCk6y3c2nfM8Ovi
LSg/3gkBZRN1g6Q8wi/ubjnN60nutlnf2vsRgbLrAVgyDsau96vfGb8pHXcqCPYR
dQnWUdfd1us2WAFCdNbSDe1kjPgOA6Rirnw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=
1694637513; x=1694723913; bh=sELkUu/ROWCusXj+gqW6rAA6N72rSwsQEOe
VjtxuYlQ=; b=MWbwc3sdT2+wEXP1oj7L85dkyFDomklnwXR2zTqz4cLeCfyltZ0
MFU7AnFbYc4z5bcBDImEHx+OgD/zJUe7Ycxzzi8/Jf4yoNFdfRmipp7x35c6azhy
daQFCktp39T61ep8ZZCT+EFpibPIh3mEaTDTFvzU+LVG0cBOy2gv6aipkm7/+PaL
XqDLSG8p439wYyKUFYB24tl+BKEfs2ArGmQXJgE07y9zkqRAPrhSBdyxoEJQXmLz
zTx9B6q7cIiDcBEjZUg+TquwlTYdZKayomDTAPOq8DuMn7RyL4sMtDscI6K7LX29
bekbJnBoz7/5730Gut30yUFsO3ymWowiUwA==
X-ME-Sender: <xms:yB0CZdjb0rccsMy9i7NsO1LAe6nEoI8DLEPmqTWlo4IDFCbNtj-ZAg>
<xme:yB0CZSDHYflUKZqY3CfNG4GjHOOcBuJEk9nnqSecoBJo3Zi3xctm6o6CHilk8UNAj
-zsxi0DqWQ_763C1U8>
X-ME-Received: <xmr:yB0CZdHgaFBTtMBUixvHTGjHFBJjnKDAJw8oLJVr3Zl1EwUJ98XzwmiUf3N313k>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeikedgudehvdcutefuodetggdotefrod
ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh
necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd
enucfjughrpefkffggfgfuvfevfhfhjggtgfesthekredttdefjeenucfhrhhomhepffhm
ihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrg
htthgvrhhnpefhffehleejffegffeugefhkeektdffgfehjedvgeejtedtudehueffgffg
feejheenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe
gumhhithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:yB0CZSTrMjq76mgZk76BtwqvguMhCWQY08sOWPofEdDh4P027VhHGQ>
<xmx:yB0CZaw6ZMAFpjJ6oEQ8FC5iGQkHhZ0YHQSn48lCs8QU6C_FGY1fBw>
<xmx:yB0CZY4f2rYjmnU2hfn8oakPKe8fioz6HzBF1zSYTVqwxyYXizfZbg>
<xmx:yR0CZe-o5bKMcYKmVrJHvsiqSfTqEasrl8W8q1kmTxM_RMY2YhMTzQ>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
13 Sep 2023 16:38:31 -0400 (EDT)
Message-ID: <e46f996a-0bed-8796-23d7-c9c8b7d428be@HIDDEN>
Date: Wed, 13 Sep 2023 23:38:29 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN>
<d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
<83bkfs2tw5.fsf@HIDDEN>
<18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN>
<83il8lxjcu.fsf@HIDDEN>
<2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN>
<8334zmtwwi.fsf@HIDDEN>
<e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN>
<83tts0rkh5.fsf@HIDDEN>
<e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN>
<831qf3pd1y.fsf@HIDDEN>
<28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN>
<83sf7jnq0m.fsf@HIDDEN>
<5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN>
<83ledanvzw.fsf@HIDDEN>
<bc4bb3de-d433-7249-cb53-2f312656589d@HIDDEN>
<83r0n2m7qz.fsf@HIDDEN>
<26afa109-9ba3-78a3-0e68-7585ae8e3a19@HIDDEN>
<83il8dna30.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <83il8dna30.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.2 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.2 (---)
On 13/09/2023 22:32, Eli Zaretskii wrote:
>>> We could change the ones we care about, though.
>>
>> Which ones do we care about? I've found a bunch of 'make-process' calls
>> without :filter specified (flymake backends, ). Do we upgrade them all?
>>
>> The difference is likely not critical in most of them, but the change
>> would likely result in small reduction of GC pressure in the
>> corresponding Emacs sessions.
>>
>> We'll also need to version-guard the ones that are in ELPA.
>>
>> We don't touch the implementations of functions like start-file-process,
>> right?
>>
>> What about the callers of functions like
>> start-file-process-shell-command who want to take advantage of the
>> improvement? Are we okay with them all having to call
>> (set-process-filter proc 'buffer) on the returned process value?
>
> I think these questions are slightly premature. We should first have
> the implementation of that filter, and then look for candidates that
> could benefit from it.
The implementation in that patch looks almost complete to me, unless you
have any further comments. The main difference would be the change in
the dispatch comparison from
if (p->filter == Qinternal_default_process_filter)
to
if (p->filter == Qbuffer)
, I think. Of course I can re-submit the amended patch, if you like.
Regarding documentation, though. How will we describe that new value?
The process filter is described like this in the manual:
This function gives PROCESS the filter function FILTER. If FILTER
is ‘nil’, it gives the process the default filter, which inserts
the process output into the process buffer. If FILTER is ‘t’,
Emacs stops accepting output from the process, unless it’s a
network server process that listens for incoming connections.
What can we add?
If FILTER is ‘buffer’, it works like the default one, only a bit faster.
?
> My tendency is to change only callers which
> are in many cases expected to get a lot of stuff from a subprocess, so
> shell buffers are probably out. But we could discuss that later.
When I'm thinking of start-file-process-shell-command, I have in mind
project--files-in-directory, which currently uses
process-file-shell-command. Though I suppose most cases would be more
easily converted to use make-process (like xref-matches-in-files uses
process-file for launching a shell pipeline already).
I was also thinking about Flymake backends because those work in the
background. The outputs are usually small, but can easily grow in rare
cases, without particular limit. Flymake also runs in the background,
meaning whatever extra work it has to do (or especially GC pressure),
affects the delays when editing.
>>> We could also discuss changing
>>> the default value, but that would require measurements in as many
>>> cases as we can afford.
>>
>> If you have some particular scenarios in mind, and what to look out for,
>> I could test them out at least on one platform.
>
> Didn't think about that enough to have scenarios.
>
>> I'm not sure what negatives to test for, though. Raising the limit 10x
>> is unlikely to lead to an OOM, but I guess some processes could grow
>> higher latency?..
>
> With a large buffer and small subprocess output we will ask the OS for
> a large memory increment for no good reason. Then the following GC
> will want to compact the gap, which means it will be slower.
I wonder what scenario that might become apparent in. Launching many
small processes at once? Can't think of a realistic test case.
Anyway, if you prefer to put off the discussion about changing the
default, that's fine by me. Or split into a separate bug.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 13 Sep 2023 19:32:33 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 13 15:32:33 2023 Received: from localhost ([127.0.0.1]:36083 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qgVbE-0005v1-GQ for submit <at> debbugs.gnu.org; Wed, 13 Sep 2023 15:32:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47812) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qgVbA-0005um-0J for 64735 <at> debbugs.gnu.org; Wed, 13 Sep 2023 15:32:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qgVay-0005eK-2F; Wed, 13 Sep 2023 15:32:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=z8u8XtDibBmHLeS/Myf7pIOYA7LEmdU1n04fZ78sMR8=; b=E4I7XGiZc84y 4ylu6wx/ATF1g+TLl8O+bg8kRgfFwZL2Aiskf+rTCal/7rE1j9ydFKPpn5dv4oovuIZ3XFHfaxHWb un35wOyU8v8DleuGJE5/tXjo/5SSQ+F1LqVhuwV0eM8+FuxAXn7Am45rCoIt3rPBxzJSVlVdXx6Nv 57xGZjeMwjQjUho1668Qa1EwV/HF5rRn59CYkK3m0jf8f7YmYZjSmJWt8fqN+WCD1l5YS1IqMejI1 Bnxm4SHeD7UQ1CLVO2z2I+E7nO09VceUf2Jz7Wwc05kqoWfm1mnj2M1trkoBUD0CUGhkcPH4tw1mc inuUkJIGBTiy0AEVUSHpxg==; Date: Wed, 13 Sep 2023 22:32:03 +0300 Message-Id: <83il8dna30.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <26afa109-9ba3-78a3-0e68-7585ae8e3a19@HIDDEN> (message from Dmitry Gutov on Wed, 13 Sep 2023 20:27:09 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> <831qf3pd1y.fsf@HIDDEN> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN> <83sf7jnq0m.fsf@HIDDEN> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN> <83ledanvzw.fsf@HIDDEN> <bc4bb3de-d433-7249-cb53-2f312656589d@HIDDEN> <83r0n2m7qz.fsf@HIDDEN> <26afa109-9ba3-78a3-0e68-7585ae8e3a19@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Wed, 13 Sep 2023 20:27:09 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > >> Are you sure? IIUC the fact that is passes 'curbuf' as the last argument > >> to 'decode_coding_c_string' means that decoding happens inside the > >> buffer. This has been my explanation for the performance improvement anyway. > > > > Yes, you are right, sorry. > > So we're not going to try the gap-based approach? Okay. decode_coding_c_string does that internally. > >> The would be the safest alternative. OTOH, this way we'd pass up on the > >> opportunity to make all existing asynchronous processes without custom > >> filters, a little bit faster in one fell swoop. > > > > We could change the ones we care about, though. > > Which ones do we care about? I've found a bunch of 'make-process' calls > without :filter specified (flymake backends, ). Do we upgrade them all? > > The difference is likely not critical in most of them, but the change > would likely result in small reduction of GC pressure in the > corresponding Emacs sessions. > > We'll also need to version-guard the ones that are in ELPA. > > We don't touch the implementations of functions like start-file-process, > right? > > What about the callers of functions like > start-file-process-shell-command who want to take advantage of the > improvement? Are we okay with them all having to call > (set-process-filter proc 'buffer) on the returned process value? I think these questions are slightly premature. We should first have the implementation of that filter, and then look for candidates that could benefit from it. My tendency is to change only callers which are in many cases expected to get a lot of stuff from a subprocess, so shell buffers are probably out. But we could discuss that later. > > We could also discuss changing > > the default value, but that would require measurements in as many > > cases as we can afford. > > If you have some particular scenarios in mind, and what to look out for, > I could test them out at least on one platform. Didn't think about that enough to have scenarios. > I'm not sure what negatives to test for, though. Raising the limit 10x > is unlikely to lead to an OOM, but I guess some processes could grow > higher latency?.. With a large buffer and small subprocess output we will ask the OS for a large memory increment for no good reason. Then the following GC will want to compact the gap, which means it will be slower.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 13 Sep 2023 17:27:29 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 13 13:27:29 2023 Received: from localhost ([127.0.0.1]:35891 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qgTeC-0005nk-Qo for submit <at> debbugs.gnu.org; Wed, 13 Sep 2023 13:27:29 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:45405) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qgTe9-0005nW-FP for 64735 <at> debbugs.gnu.org; Wed, 13 Sep 2023 13:27:27 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id DB7DF3200989; Wed, 13 Sep 2023 13:27:13 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 13 Sep 2023 13:27:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1694626033; x=1694712433; bh=VDCkBYEU3Ebxlnylz5EUpmpYkukUVkNSGuj 0ISVOiwk=; b=pg/+pvH8NcfvYCEDSx9sWJzy6L4LLntKqM41m8wyAHsrrKEsLYS vTOLIDFZEeGk7o6CdzsaJHj6pD1LNSElqEACMgjDZqG50VtGB2v3UDczT53R8vjg s7wB/kKDYOfWPfm94VIzL8HywPtGDgAzU0F2tHnmOz0QB/NxevrQRdVGq2u/OWnq 4zhJRWo+ACrzh12zrL+SXh+ShrshUBtmOHYGdTrJyH4SypTZHTwlkOTrUBEMbWWv UwK9qifyerpq6Ux0gYEO2dXJzpw/6wIl/oincYjFedD6bwB6htUu0g3JlP8zaS18 KLvXHJk6Llsqqny+/yFTllPwjdu1aMG7Zyw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1694626033; x=1694712433; bh=VDCkBYEU3Ebxlnylz5EUpmpYkukUVkNSGuj 0ISVOiwk=; b=b+Rrx2dFfS4hBpqLkjPfcE5RNnodJe2Y43NJAwfuo+R6hdEfXm3 wN8fxpj/tD84o+LiozB/Y0VnYfErBG6WyjALiy4a0J+DvgEkRxa9l+7rhtHxqZVM MMgcg4sPTEQDS+95sibneUjORFuXksJvMdNYBHUy3vesvWdkbH7DB36hKPy3Ha2w QOff9T5jF40dtMlLo51kHR7enwm3Ha5neESreKkWIWQvnSIHEWHQASnmxdZQO/ek XWTX0kKNYJHA4nYmOVx0Qaukk9K2giDA2q6UDRKFxU6qgXpsBHJY2yMcqUWuDPNp q2lqDSykXA0THwZtddBqCoe2At+iwrUvWeg== X-ME-Sender: <xms:8fABZdk72gfkc29K7dPOyakkOfrApHo3392Glyihzw2bFY6eqToDHg> <xme:8fABZY2NsguHzoyN6GKLvgD-mnF6kU_GsXcIppAvz3n0MDHiyUNFHtCtdJ21p9SUm caYrQja0oI7TpjR7GI> X-ME-Received: <xmr:8fABZTpLjyFBF2MMcoh3HwoG-N_WbojzZGvVZOLvqDbSXRu3roSx90lCDeGKlkw> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeikedguddufecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhm ihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrg htthgvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeev ledvveenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:8fABZdmP9PHEubJ73eyFT-0fMEGktkOD7A_AgwxBWBTSa_x-rYSg7g> <xmx:8fABZb35rcRaDrsX0sTkRHq7fXakk-uUepHuX7Mm0CeUOOH8a1CoXA> <xmx:8fABZcu_la1HUzwIzRitWfXFMr_bOzDZqsz9DgPa247hDBLM1eCnTQ> <xmx:8fABZcwzbBX-8OSDZoIFoH2BKW2WMUIXwjXlWV42upj8VUjJQzkIig> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 13 Sep 2023 13:27:11 -0400 (EDT) Message-ID: <26afa109-9ba3-78a3-0e68-7585ae8e3a19@HIDDEN> Date: Wed, 13 Sep 2023 20:27:09 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> <831qf3pd1y.fsf@HIDDEN> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN> <83sf7jnq0m.fsf@HIDDEN> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN> <83ledanvzw.fsf@HIDDEN> <bc4bb3de-d433-7249-cb53-2f312656589d@HIDDEN> <83r0n2m7qz.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83r0n2m7qz.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.2 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.2 (---) On 13/09/2023 18:07, Eli Zaretskii wrote: >> Date: Wed, 13 Sep 2023 17:27:49 +0300 >> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov <dmitry@HIDDEN> >> >>>> Does the patch from my last patch-bearing email look similar enough to >>>> what you're describing? >>>> >>>> The one called read_and_insert_process_output.diff >>> >>> No, not entirely: it still produces a Lisp string when decoding is >>> needed, and then inserts that string into the buffer. >> >> Are you sure? IIUC the fact that is passes 'curbuf' as the last argument >> to 'decode_coding_c_string' means that decoding happens inside the >> buffer. This has been my explanation for the performance improvement anyway. > > Yes, you are right, sorry. So we're not going to try the gap-based approach? Okay. >>> Sure, but in this case we don't need any filtering. It's basically >>> the same idea as internal-default-process-filter: we just need to >>> insert the process output into a buffer, and optionally decode it. >> >> Pretty much. But that raises the question of what to do with the >> existing function internal-default-process-filter. > > Nothing. It will remain as the default filter. Okay, if you are sure. >>> We can provide a special filter identified by a symbol. Such a filter >>> will not be Lisp-callable, it will exist for the cases where we need >>> to insert the output into the process buffer. >> >> The would be the safest alternative. OTOH, this way we'd pass up on the >> opportunity to make all existing asynchronous processes without custom >> filters, a little bit faster in one fell swoop. > > We could change the ones we care about, though. Which ones do we care about? I've found a bunch of 'make-process' calls without :filter specified (flymake backends, ). Do we upgrade them all? The difference is likely not critical in most of them, but the change would likely result in small reduction of GC pressure in the corresponding Emacs sessions. We'll also need to version-guard the ones that are in ELPA. We don't touch the implementations of functions like start-file-process, right? What about the callers of functions like start-file-process-shell-command who want to take advantage of the improvement? Are we okay with them all having to call (set-process-filter proc 'buffer) on the returned process value? >> Should we also discuss increasing the default of >> read-process-output-max? Even increasing it 10x (not necessarily 100x) >> creates a noticeable difference, especially combined with the proposed >> change. > > That should be limited to specific cases where we expect to see a lot > of stuff coming from the subprocess. So it would be okay to bump it in particular functions? Okay. > We could also discuss changing > the default value, but that would require measurements in as many > cases as we can afford. If you have some particular scenarios in mind, and what to look out for, I could test them out at least on one platform. I'm not sure what negatives to test for, though. Raising the limit 10x is unlikely to lead to an OOM, but I guess some processes could grow higher latency?..
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 13 Sep 2023 15:08:15 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 13 11:08:15 2023 Received: from localhost ([127.0.0.1]:35647 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qgRTT-0005H9-A0 for submit <at> debbugs.gnu.org; Wed, 13 Sep 2023 11:08:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56124) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qgRTQ-0005Gq-MI for 64735 <at> debbugs.gnu.org; Wed, 13 Sep 2023 11:08:14 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qgRTF-0005tL-8O; Wed, 13 Sep 2023 11:08:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=P9+tH0UISD2MLZiMZ3tAEzZoaPa7jA2EjoJzp7FNr/Y=; b=WYWJL3LF6GnT VCDXFBve8y/JJN1UL5CFoIT9qElM22BosGeEbsMlIKyCUdGmebjk1MLnXITWSC8avRT+PUJzp5Mak qwzivg3YN0esahY7e6DZ5ubpRGQt4qB4mv0zmC98DQ/1+X0hd7zR99Q/qbV69qtF/R5wNUjZCq+9T CM6+ndYLJXFrhJo0K50aZdFsDGH+ei1xrE1gmRfcyNzN6f2ALM4kRgeSZxEcxQhFT9fwXVuHpd8d3 od+Nt3Xwao9AgRP5XNVaeLX/+RUlrJIdfaEdP5a0jEw/JbRslaysyJ2qk2oIFGtWDcUjqqodMskYI bN3+T2oWoftS1M7P7IhlZw==; Date: Wed, 13 Sep 2023 18:07:48 +0300 Message-Id: <83r0n2m7qz.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <bc4bb3de-d433-7249-cb53-2f312656589d@HIDDEN> (message from Dmitry Gutov on Wed, 13 Sep 2023 17:27:49 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> <831qf3pd1y.fsf@HIDDEN> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN> <83sf7jnq0m.fsf@HIDDEN> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN> <83ledanvzw.fsf@HIDDEN> <bc4bb3de-d433-7249-cb53-2f312656589d@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Wed, 13 Sep 2023 17:27:49 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > >> Does the patch from my last patch-bearing email look similar enough to > >> what you're describing? > >> > >> The one called read_and_insert_process_output.diff > > > > No, not entirely: it still produces a Lisp string when decoding is > > needed, and then inserts that string into the buffer. > > Are you sure? IIUC the fact that is passes 'curbuf' as the last argument > to 'decode_coding_c_string' means that decoding happens inside the > buffer. This has been my explanation for the performance improvement anyway. Yes, you are right, sorry. > > Sure, but in this case we don't need any filtering. It's basically > > the same idea as internal-default-process-filter: we just need to > > insert the process output into a buffer, and optionally decode it. > > Pretty much. But that raises the question of what to do with the > existing function internal-default-process-filter. Nothing. It will remain as the default filter. > Looking around, it doesn't seem to be used with advice (a good thing: > the proposed change would break that), but it is called directly in some > packages like magit-blame, org-assistant, with-editor, wisi, sweeprolog, > etc. I suppose we'd just keep it around unchanged. Yes. > > We can provide a special filter identified by a symbol. Such a filter > > will not be Lisp-callable, it will exist for the cases where we need > > to insert the output into the process buffer. > > The would be the safest alternative. OTOH, this way we'd pass up on the > opportunity to make all existing asynchronous processes without custom > filters, a little bit faster in one fell swoop. We could change the ones we care about, though. > Should we also discuss increasing the default of > read-process-output-max? Even increasing it 10x (not necessarily 100x) > creates a noticeable difference, especially combined with the proposed > change. That should be limited to specific cases where we expect to see a lot of stuff coming from the subprocess. We could also discuss changing the default value, but that would require measurements in as many cases as we can afford.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 13 Sep 2023 14:28:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 13 10:28:13 2023
Received: from localhost ([127.0.0.1]:35535 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qgQqi-0001Pr-Ro
for submit <at> debbugs.gnu.org; Wed, 13 Sep 2023 10:28:13 -0400
Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:55661)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qgQqb-0001P9-5Y
for 64735 <at> debbugs.gnu.org; Wed, 13 Sep 2023 10:28:12 -0400
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by mailout.west.internal (Postfix) with ESMTP id C4A3332007BE;
Wed, 13 Sep 2023 10:27:53 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute4.internal (MEProxy); Wed, 13 Sep 2023 10:27:54 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm3; t=
1694615273; x=1694701673; bh=qw2CUkx912C0EhU21xm0F0yGgOIkoPNpXaq
0dSd83CU=; b=gdYalQe+frSoSrAKBZoRc6hx11c7rEThP5nn9LASD57AjVT1tda
2+3ktvA+S6RlwvXsxvcNe/DJSFa5kPO+fbgmlGnRkPQuqfJ61oLzJNzkCyx+g7e4
kx1FLKf5TE9YkXqJING17bTVFdKPxxM4hyPbV/f1INeXSuSSRLBOi4QlPfJmipLy
dDLsqYGWzsfp+WqbyxG4raHHrccGOIBei18G8MASOK9at1a/BYPj5u/HoybPYh/J
fH0Gw/dleI4D9lmibsk7UxErry8X4TuTr81VmjEs70k2wTprnUNo7iCUauDV1bdE
Ij6x3cbJosj/RFDHKkowaFP3nxBMjLMcPlg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=
1694615273; x=1694701673; bh=qw2CUkx912C0EhU21xm0F0yGgOIkoPNpXaq
0dSd83CU=; b=Sz8bHqbZoWoVobpfkxe8AEylesT6As0o+iHqKhr0GbB7rlzt5yi
M/OEw3ISn+x0R2kder2XAve0l5gYbthg/SQaapoZvRXohLCV5WKTf2ZQLDbjBxxQ
ZKFIAv2naipmpkiKGP+k3OiVv7k5rAkAp8+MfEu2V4fHOa8xcDmkB1WNtySsUQ7K
nyStORqb6pAAEdvb8RHTQrE/KPnqGaRHkuy7blIAilKxzNZQaDF2dx0Cygmz8rpU
ATLkcGEqkohM/Xn2cRVhUXHRUhWzq7BGdnKFNMwgBbY40jTobIryBoSRrx8qKIE3
6FFBhZE3Ua1zKBpNtDtNfZmaEqm6nfjfZAQ==
X-ME-Sender: <xms:6MYBZeQ7ZWScsPsfo4h7QFMkhoYKlKAQ_53nyoUKbpJIQtuYpn2XfQ>
<xme:6MYBZTx0D-nDkDhwazXK8PaygeqRGDnZWY-WwazKe1NNNKX8L9oNloPNlwqorZgVs
nidxOWf1nEKxdl68LE>
X-ME-Received: <xmr:6MYBZb29skSOr8GQhZrZMrFmP1eVKEbFYzjf8igf3fFPyNMOR6bgeVDveCFeIaE>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeikedgjeejucetufdoteggodetrfdotf
fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi
thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth
htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel
vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug
hmihhtrhihsehguhhtohhvrdguvghv
X-ME-Proxy: <xmx:6cYBZaADsuvmlKrlrsibUd6arx__vTyOejC0cs7BSZb16D5mPfuVJw>
<xmx:6cYBZXgP0Z1Y2ZeAyStaAG6Ipa4PqydzXzs-JgsZKkCJFC5O8RbxVA>
<xmx:6cYBZWoCcNnfwob944naC1zDadiq4vhTOOLfT2x-N8kCKEpc2zdZdw>
<xmx:6cYBZUuL6q1w75Lt0AqNAsO-l-ZnjHFm6PrOR8GXgi4rPtbskID6Wg>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
13 Sep 2023 10:27:51 -0400 (EDT)
Message-ID: <bc4bb3de-d433-7249-cb53-2f312656589d@HIDDEN>
Date: Wed, 13 Sep 2023 17:27:49 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN>
<d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
<83bkfs2tw5.fsf@HIDDEN>
<18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN>
<83il8lxjcu.fsf@HIDDEN>
<2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN>
<8334zmtwwi.fsf@HIDDEN>
<e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN>
<83tts0rkh5.fsf@HIDDEN>
<e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN>
<831qf3pd1y.fsf@HIDDEN>
<28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN>
<83sf7jnq0m.fsf@HIDDEN>
<5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN>
<83ledanvzw.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <83ledanvzw.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.2 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.2 (---)
On 13/09/2023 14:38, Eli Zaretskii wrote:
>> Date: Tue, 12 Sep 2023 23:27:49 +0300
>> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
>> 64735 <at> debbugs.gnu.org
>> From: Dmitry Gutov <dmitry@HIDDEN>
>>
>>> That's why I said "or writing a new filter function".
>>> read_and_dispose_of_process_output will have to call this new filter
>>> differently, passing it the raw text read from the subprocess, where
>>> read_and_dispose_of_process_output current first decodes the text and
>>> produces a Lisp string from it. Then the filter would need to do
>>> something similar to what insert-file-contents does: insert the raw
>>> input into the gap, then call decode_coding_gap to decode that
>>> in-place.
>>
>> Does the patch from my last patch-bearing email look similar enough to
>> what you're describing?
>>
>> The one called read_and_insert_process_output.diff
>
> No, not entirely: it still produces a Lisp string when decoding is
> needed, and then inserts that string into the buffer.
Are you sure? IIUC the fact that is passes 'curbuf' as the last argument
to 'decode_coding_c_string' means that decoding happens inside the
buffer. This has been my explanation for the performance improvement anyway.
If it still generated a Lisp string, I think that would mean that we
could save the general shape of internal-default-process-filter and just
improve its implementation for the same measured benefit.
> Did you look at what insert-file-contents does? If not I suggest to
> have a look, starting from this comment:
>
> /* Here, we don't do code conversion in the loop. It is done by
> decode_coding_gap after all data are read into the buffer. */
>
> and ending here:
>
> if (CODING_MAY_REQUIRE_DECODING (&coding)
> && (inserted > 0 || CODING_REQUIRE_FLUSHING (&coding)))
> {
> /* Now we have all the new bytes at the beginning of the gap,
> but `decode_coding_gap` can't have them at the beginning of the gap,
> so we need to move them. */
> memmove (GAP_END_ADDR - inserted, GPT_ADDR, inserted);
> decode_coding_gap (&coding, inserted);
> inserted = coding.produced_char;
> coding_system = CODING_ID_NAME (coding.id);
> }
> else if (inserted > 0)
> {
> /* Make the text read part of the buffer. */
> eassert (NILP (BVAR (current_buffer, enable_multibyte_characters)));
> insert_from_gap_1 (inserted, inserted, false);
>
> invalidate_buffer_caches (current_buffer, PT, PT + inserted);
> adjust_after_insert (PT, PT_BYTE, PT + inserted, PT_BYTE + inserted,
> inserted);
> }
That does look different. I'm not sure how long it would take me to
adapt this code (if you have an alternative patch to suggest right away,
please go ahead), but if this method turns out to be faster, it sounds
like we could improve the performance of 'call_process' the same way.
That would be a win-win.
>> The result there, though, is that a "filter" (in the sense that
>> make-process uses that term) is not used at all.
>
> Sure, but in this case we don't need any filtering. It's basically
> the same idea as internal-default-process-filter: we just need to
> insert the process output into a buffer, and optionally decode it.
Pretty much. But that raises the question of what to do with the
existing function internal-default-process-filter.
Looking around, it doesn't seem to be used with advice (a good thing:
the proposed change would break that), but it is called directly in some
packages like magit-blame, org-assistant, with-editor, wisi, sweeprolog,
etc. I suppose we'd just keep it around unchanged.
>>> And if we want to use it in production, we should
>>> probably work on adding that special default filter which inserts and
>>> decodes directly into the buffer, because that will probably lower the
>>> GC pressure and thus has hope of being faster. Or even replace the
>>> default filter implementation with that new one.
>>
>> But a filter must be a Lisp function, which can't help but accept only
>> Lisp strings (not C string) as argument. Isn't that right?
>
> We can provide a special filter identified by a symbol. Such a filter
> will not be Lisp-callable, it will exist for the cases where we need
> to insert the output into the process buffer.
The would be the safest alternative. OTOH, this way we'd pass up on the
opportunity to make all existing asynchronous processes without custom
filters, a little bit faster in one fell swoop.
> Any Lisp callback could
> then access the process output as the text of that buffer, no Lisp
> strings needed. I thought this was a worthy goal; apologies if I
> misunderstood.
Sorry, I was just quibbling about the terminology, to make sure we are
on the same page on what is being proposed. If the patch and evidence
look good to people, that is. And I'd like to explore that improvement
venue to the max.
But note that it has limitations as well (e.g. filter is the only way to
get in-process callbacks from the process, and avoiding it for best
performance will require external callback such as timers), so if
someone has any better ideas how to improve GC time to a comparable
extent but keep design unchanged, that's also welcome.
Should we also discuss increasing the default of
read-process-output-max? Even increasing it 10x (not necessarily 100x)
creates a noticeable difference, especially combined with the proposed
change.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 13 Sep 2023 11:39:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Sep 13 07:39:10 2023
Received: from localhost ([127.0.0.1]:60954 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qgOD8-0005IF-BL
for submit <at> debbugs.gnu.org; Wed, 13 Sep 2023 07:39:10 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:43088)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qgOD6-0005I2-0v
for 64735 <at> debbugs.gnu.org; Wed, 13 Sep 2023 07:39:09 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qgOCu-0008Af-KX; Wed, 13 Sep 2023 07:38:56 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=OOjaOQ+avF9SXZYieipNqFMRzI2rBjiRDZy4dy6P2eg=; b=ojck29XUVHmF
e+sw+h+nysdFLqIXKpWlq7eMxDOLIQk7PvdeZdmO7dmGE3w4mR+8zLvnAQSJpCDe5JQIiZFEyN9pV
l426RwV/Salel2pYNG5XvhXjUQZr4rxqpTbyx4fJ14ZjEV5yTdEtb3Z+XmvaXVyzoW/2lYr5JGOf0
qPDUDoeKtt50GMNzrHBCpm6O1VLGdHcAmLzIoQMdx6TNfYJkGk0QJUBRs76+2yqS4Yyka+sLo//ET
HUpupbW29uDs+U1LNRe4L1M/qlWQ5/xsE1sZZQkxfhWDuXg72k73z5DLkFWz1AQ99mao256+FiJCx
b7Q6LUwr8/8erxNhinv7Dw==;
Date: Wed, 13 Sep 2023 14:38:43 +0300
Message-Id: <83ledanvzw.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN> (message from
Dmitry Gutov on Tue, 12 Sep 2023 23:27:49 +0300)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN>
<834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN>
<d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
<83bkfs2tw5.fsf@HIDDEN>
<18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN>
<83il8lxjcu.fsf@HIDDEN>
<2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN>
<8334zmtwwi.fsf@HIDDEN>
<e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN>
<83tts0rkh5.fsf@HIDDEN>
<e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN>
<831qf3pd1y.fsf@HIDDEN>
<28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN>
<83sf7jnq0m.fsf@HIDDEN> <5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Date: Tue, 12 Sep 2023 23:27:49 +0300
> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
> 64735 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry@HIDDEN>
>
> > That's why I said "or writing a new filter function".
> > read_and_dispose_of_process_output will have to call this new filter
> > differently, passing it the raw text read from the subprocess, where
> > read_and_dispose_of_process_output current first decodes the text and
> > produces a Lisp string from it. Then the filter would need to do
> > something similar to what insert-file-contents does: insert the raw
> > input into the gap, then call decode_coding_gap to decode that
> > in-place.
>
> Does the patch from my last patch-bearing email look similar enough to
> what you're describing?
>
> The one called read_and_insert_process_output.diff
No, not entirely: it still produces a Lisp string when decoding is
needed, and then inserts that string into the buffer.
Did you look at what insert-file-contents does? If not I suggest to
have a look, starting from this comment:
/* Here, we don't do code conversion in the loop. It is done by
decode_coding_gap after all data are read into the buffer. */
and ending here:
if (CODING_MAY_REQUIRE_DECODING (&coding)
&& (inserted > 0 || CODING_REQUIRE_FLUSHING (&coding)))
{
/* Now we have all the new bytes at the beginning of the gap,
but `decode_coding_gap` can't have them at the beginning of the gap,
so we need to move them. */
memmove (GAP_END_ADDR - inserted, GPT_ADDR, inserted);
decode_coding_gap (&coding, inserted);
inserted = coding.produced_char;
coding_system = CODING_ID_NAME (coding.id);
}
else if (inserted > 0)
{
/* Make the text read part of the buffer. */
eassert (NILP (BVAR (current_buffer, enable_multibyte_characters)));
insert_from_gap_1 (inserted, inserted, false);
invalidate_buffer_caches (current_buffer, PT, PT + inserted);
adjust_after_insert (PT, PT_BYTE, PT + inserted, PT_BYTE + inserted,
inserted);
}
> The result there, though, is that a "filter" (in the sense that
> make-process uses that term) is not used at all.
Sure, but in this case we don't need any filtering. It's basically
the same idea as internal-default-process-filter: we just need to
insert the process output into a buffer, and optionally decode it.
> > And if we want to use it in production, we should
> > probably work on adding that special default filter which inserts and
> > decodes directly into the buffer, because that will probably lower the
> > GC pressure and thus has hope of being faster. Or even replace the
> > default filter implementation with that new one.
>
> But a filter must be a Lisp function, which can't help but accept only
> Lisp strings (not C string) as argument. Isn't that right?
We can provide a special filter identified by a symbol. Such a filter
will not be Lisp-callable, it will exist for the cases where we need
to insert the output into the process buffer. Any Lisp callback could
then access the process output as the text of that buffer, no Lisp
strings needed. I thought this was a worthy goal; apologies if I
misunderstood.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 12 Sep 2023 20:28:07 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 12 16:28:07 2023 Received: from localhost ([127.0.0.1]:60141 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qg9zS-0002HZ-Pj for submit <at> debbugs.gnu.org; Tue, 12 Sep 2023 16:28:07 -0400 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:59149) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qg9zP-0002H3-E2 for 64735 <at> debbugs.gnu.org; Tue, 12 Sep 2023 16:28:05 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id A32465C01E8; Tue, 12 Sep 2023 16:27:52 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 12 Sep 2023 16:27:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1694550472; x=1694636872; bh=GpU9UjNNZ2rCNxsVAsYpUrwlWi5xiIiqvyH gHpFpMzY=; b=de3WkuPF7jTAzFqET5DYh7rVgda2jXwi0vZWMACNX0gYIuKlBKI 1BgJ6aKJNFy8mjre5dvoLb1GkiINlRZxWbbK3QHZ8r72KZiZCgvDHuw6etfaDH/t uepMZqc6Ho/9WnP6t4CBQmuF8DTMuZDe1rm20YkheWSNYYaxiqtPjDNOi9ujvPB/ Afr5+WKrhmHxQeYh0kkawSMi3RZNYAx/uSIgVPTHWMjlQXH+pGr6+VUV1S1L1RR8 hmxFENpWBf2eKg9gsELARiS8PUs4ZEITIUG0IQo1l9tu7y/4mCR9m71cokBEt8xv lFl8cEDuHOETCGssiLxtReCj2e6bk2wE9OQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1694550472; x=1694636872; bh=GpU9UjNNZ2rCNxsVAsYpUrwlWi5xiIiqvyH gHpFpMzY=; b=z3Gyux7C8NG2xZLIHaTEOdgwJYAJNJVp2fCF3oIxtH0CKHFcHRl pyai1QAI/YLyt7qsuTa7TwMZe9XUCUtHDdUu6FWewJzNMXv5h9fcVfR4gub90Km1 L3Vw3468stMtsGrdHL6z/JO9uYY45FHkkj+P40bxRnuw8zYjZ8lpXlfoc09gfMZ8 Z2fAWb3mqI94+amX3Zy6FshlV9hkGneHjXM6Ntmdzz8jWgT4qEO2wdl+ageC8SAB 0yd7SvrblPSiENtUsFJYvUWLW/xI+XbNjrrk8RyEx/FYQdHc6ZMQpEg5+QbzpIZF koqFs8vdQru/sR0f0trhaEKZ5FI7jZEf1Pw== X-ME-Sender: <xms:yMkAZYBvmMPLofYHck7SuufQ8UJoYRFI_09BrzVGV8wFQTKnYwva9A> <xme:yMkAZaiVCk7XjUvDavzrX30IQY3ptTLuCA10XPTbBqVB5gNykvb3k17SoRr_Evo47 eiV7kgE0oKHBTrc0V4> X-ME-Received: <xmr:yMkAZbkAN1u-QvDVS7C0MdYsYJwQ94cE50H6r_t-2fG5WsS7TyIJL9jR8Rvn_Sc> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeiiedgudehtdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhm ihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrg htthgvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeev ledvveenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:yMkAZezwgJ6-zp2a5sUgeP_TAD8qYjy1YM_xeEfqUOpeg2beHIAiYg> <xmx:yMkAZdRgLlMnY2um4-BXa7ft1hVw_h_TJJ2u4_ue9HowhpRlgo6G7A> <xmx:yMkAZZZOFLKaOEHyOw3qTEJ4K1bZ78UsGEr_OxgoCuWmWNqlfsiG7A> <xmx:yMkAZfeww_xjE_UbYyilTAlFPfGFyeJ5ytHz12VkcwLkCx1JmBRsqA> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 12 Sep 2023 16:27:51 -0400 (EDT) Message-ID: <5c493f86-0af5-256f-41a7-7d886ab4c5e4@HIDDEN> Date: Tue, 12 Sep 2023 23:27:49 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> <831qf3pd1y.fsf@HIDDEN> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN> <83sf7jnq0m.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83sf7jnq0m.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.2 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.2 (---) On 12/09/2023 22:35, Eli Zaretskii wrote: >> Date: Tue, 12 Sep 2023 21:48:37 +0300 >> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov <dmitry@HIDDEN> >> >>> then we could try extending >>> internal-default-process-filter (or writing a new filter function >>> similar to it) so that it inserts the stuff into the gap and then uses >>> decode_coding_gap, >> >> Can that work at all? By the time internal-default-process-filter is >> called, we have already turned the string from char* into Lisp_Object >> text, which we then pass to it. So consing has already happened, IIUC. > > That's why I said "or writing a new filter function". > read_and_dispose_of_process_output will have to call this new filter > differently, passing it the raw text read from the subprocess, where > read_and_dispose_of_process_output current first decodes the text and > produces a Lisp string from it. Then the filter would need to do > something similar to what insert-file-contents does: insert the raw > input into the gap, then call decode_coding_gap to decode that > in-place. Does the patch from my last patch-bearing email look similar enough to what you're describing? The one called read_and_insert_process_output.diff The result there, though, is that a "filter" (in the sense that make-process uses that term) is not used at all. >>> which converts inserted bytes in-place -- that, at >>> least, will be correct and will avoid consing intermediate temporary >>> strings from the process output, then decoding them, then inserting >>> them. Other than that, the -2 and -3 variants are very close >>> runners-up of -5, so maybe I'm missing something, but I see no reason >>> be too excited here? I mean, 0.89 vs 0.92? really? >> >> The important part is not 0.89 vs 0.92 (that would be meaningless >> indeed), but that we have an _asyncronous_ implementation of the feature >> that works as fast as the existing synchronous one (or faster! if we >> also bind read-process-output-max to a large value, the time is 0.72). >> >> The possible applications for that range from simple (printing progress >> bar while the scan is happening) to more advanced (launching a >> concurrent process where we pipe the received file names concurrently to >> 'xargs grep'), including visuals (xref buffer which shows the >> intermediate search results right away, updating them gradually, all >> without blocking the UI). > > Hold your horses. Emacs only reads output from sub-processes when > it's idle. So printing a progress bar (which makes Emacs not idle) > with the asynchronous implementation is basically the same as having > the synchronous implementation call some callback from time to time > (which will then show the progress). Obviously there is more work to be done, including further desgin and benchmarking. But unlike before, at least the starting performance (before further features are added) is not worse. Note that the variant -5 is somewhat limited since it doesn't use a filter - that means that no callbacks a issued while the output is arriving, meaning that if it's taken as base, whatever refreshes would have to be initiated from somewhere else. E.g. from a timer. > As for piping to another process, this is best handled by using a > shell pipe, without passing stuff through Emacs. And even if you do > need to pass it through Emacs, you could do the same with the > synchronous implementation -- only the "xargs" part needs to be > asynchronous, the part that reads file names does not. Right? Yes and no: if both steps are asynchronous, the final output window could be displayed right away, rather than waiting for the first step (or both) to be finished. Which can be a meaningful improvement for some (and still is an upside of 'M-x rgrep'). > Please note: I'm not saying that the asynchronous implementation is > not interesting. It might even have advantages in some specific use > cases. So it is good to have it. It just isn't a breakthrough, > that's all. Not a breakthrough, of course, just a lower-level insight (hopefully). I do think it would be meaningful to manage to reduce the runtime of a real-life program (which includes other work) by 10-20% solely by reducing GC pressure in a generic facility like process output handling. > And if we want to use it in production, we should > probably work on adding that special default filter which inserts and > decodes directly into the buffer, because that will probably lower the > GC pressure and thus has hope of being faster. Or even replace the > default filter implementation with that new one. But a filter must be a Lisp function, which can't help but accept only Lisp strings (not C string) as argument. Isn't that right?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 12 Sep 2023 19:36:07 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 12 15:36:07 2023 Received: from localhost ([127.0.0.1]:60110 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qg9B9-0006kc-7S for submit <at> debbugs.gnu.org; Tue, 12 Sep 2023 15:36:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42098) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qg9B6-0006k6-B4 for 64735 <at> debbugs.gnu.org; Tue, 12 Sep 2023 15:36:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qg9Av-0007Vo-9J; Tue, 12 Sep 2023 15:35:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=R9F4/GmMtcUGFgKQiY8I7hTyYDm96X+ZJs5UYdgq0I8=; b=Bk4ek8Bg4dC4 RRyJgQlY7Kso0cBjzliB/rG+0tv4lBZvqzsXbeccKVuoRWjIQsg0WE03Aenfe+Egi2NkKB/z1yobF RImbY+Sr7m+c+bVaLY2MJwKZzx0dMT2J2/34DwduSSkthv3hiFGisvv4HSYGsNZxSvBlk2cjIOj0H pnyp8qwjBheplAgYZMlXjpStMJ9FSw4zAkwsmMs04zmu2bf1tQtCXwk6qDIC7jOlhc0rk8ajgMj8V AaoZH4f2lGpaUJ6QPKanvHnpVywG3wIke1EiNBtnCrxvzq+jeaFLghX30cHQsbph7Uv/k8aHk8HGm UQWgRmDdxnKEk6rfBcOY3w==; Date: Tue, 12 Sep 2023 22:35:37 +0300 Message-Id: <83sf7jnq0m.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN> (message from Dmitry Gutov on Tue, 12 Sep 2023 21:48:37 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> <831qf3pd1y.fsf@HIDDEN> <28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Tue, 12 Sep 2023 21:48:37 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > > then we could try extending > > internal-default-process-filter (or writing a new filter function > > similar to it) so that it inserts the stuff into the gap and then uses > > decode_coding_gap, > > Can that work at all? By the time internal-default-process-filter is > called, we have already turned the string from char* into Lisp_Object > text, which we then pass to it. So consing has already happened, IIUC. That's why I said "or writing a new filter function". read_and_dispose_of_process_output will have to call this new filter differently, passing it the raw text read from the subprocess, where read_and_dispose_of_process_output current first decodes the text and produces a Lisp string from it. Then the filter would need to do something similar to what insert-file-contents does: insert the raw input into the gap, then call decode_coding_gap to decode that in-place. > > which converts inserted bytes in-place -- that, at > > least, will be correct and will avoid consing intermediate temporary > > strings from the process output, then decoding them, then inserting > > them. Other than that, the -2 and -3 variants are very close > > runners-up of -5, so maybe I'm missing something, but I see no reason > > be too excited here? I mean, 0.89 vs 0.92? really? > > The important part is not 0.89 vs 0.92 (that would be meaningless > indeed), but that we have an _asyncronous_ implementation of the feature > that works as fast as the existing synchronous one (or faster! if we > also bind read-process-output-max to a large value, the time is 0.72). > > The possible applications for that range from simple (printing progress > bar while the scan is happening) to more advanced (launching a > concurrent process where we pipe the received file names concurrently to > 'xargs grep'), including visuals (xref buffer which shows the > intermediate search results right away, updating them gradually, all > without blocking the UI). Hold your horses. Emacs only reads output from sub-processes when it's idle. So printing a progress bar (which makes Emacs not idle) with the asynchronous implementation is basically the same as having the synchronous implementation call some callback from time to time (which will then show the progress). As for piping to another process, this is best handled by using a shell pipe, without passing stuff through Emacs. And even if you do need to pass it through Emacs, you could do the same with the synchronous implementation -- only the "xargs" part needs to be asynchronous, the part that reads file names does not. Right? Please note: I'm not saying that the asynchronous implementation is not interesting. It might even have advantages in some specific use cases. So it is good to have it. It just isn't a breakthrough, that's all. And if we want to use it in production, we should probably work on adding that special default filter which inserts and decodes directly into the buffer, because that will probably lower the GC pressure and thus has hope of being faster. Or even replace the default filter implementation with that new one. > > About inserting into the buffer: what we do is insert into the gap, > > and when the gap becomes full, we enlarge it. Enlarging the gap > > involves: (a) enlarging the chunk of memory allocated to buffer text > > (which might mean we ask the OS for more memory), and (b) moving the > > characters after the gap to the right to free space for inserting more > > stuff. This is pretty fast, but still, with a large pipe buffer and a > > lot of output, we do this many times, so it could add up to something > > pretty tangible. It's hard to me to tell whether this is > > significantly faster than consing strings and inserting them, only > > measurements can tell. > > See the benchmark tables and the POC patch in my previous email. Using a > better filter function would be ideal, but it seems like that's not > going to fit the current design. Happy to be proven wrong, though. I see no reason why reading subprocess output couldn't use the same technique as insert-file-contents does.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 12 Sep 2023 18:48:56 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 12 14:48:56 2023 Received: from localhost ([127.0.0.1]:60054 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qg8RT-0002wM-9H for submit <at> debbugs.gnu.org; Tue, 12 Sep 2023 14:48:55 -0400 Received: from out5-smtp.messagingengine.com ([66.111.4.29]:58265) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qg8RQ-0002vo-7B for 64735 <at> debbugs.gnu.org; Tue, 12 Sep 2023 14:48:54 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 48F855C0186; Tue, 12 Sep 2023 14:48:42 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Tue, 12 Sep 2023 14:48:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1694544522; x=1694630922; bh=5y1qtQbgVfNrNKwFJo2jBPpTbQGgQXAYB5Q OS8leH/U=; b=HWKaG07aW/f1tgLXn2Ug6+SwKe4ltw4b4sIbXA25OLFw60gIKhG 76jSeIRaHhuDymhoin2oPcSOaHJEEYyMrhv7unJLD4a6uUjicfRqZV7zxYvhavJM 0e1yc/28xMRJEhHlwIo68yt0cmuwcTa1wRaqA4AeYXZFMy7MMAac+Kzr5shBs7Lu c+43so436oAtjeDziTJHmEDUTp12G0aef591An84mD6V7jJT/ovZ/HA1Eo05Kcef 3eo8YxY59AYhs9b0kh4uugHnqBuRNFqY5oSooCnXEixfU0tRRlRJBimbGv7JoHA3 crEhPCFTNO1nYUH3bIKvQ5tgKIIG+lQ0GNw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1694544522; x=1694630922; bh=5y1qtQbgVfNrNKwFJo2jBPpTbQGgQXAYB5Q OS8leH/U=; b=U14jQBM+z728FKFDRsZvSrKwU+N3/J3SmgmFi3Z+1aBX+F/yPGG lq20G/0/LvobW54VJbS2gt32R5lYUPViqQC90UBFZcvtbIzOXkrkl4RxJ2IjWJWO 1/zeb/6vHhC0l+VP3ViAYT/R+Iblhh/A3uPHvq6BRKe326+CxSYISKrFqEKqirt5 0nmDS8tRhXm3Hnb2OOrWsrYxbvINhTm4lkOUOPuOr2PKrQ2WHgGQ9FtWMuG7WgNB IWYG28st632nGt4s9zXxrP3oOKQF82cTWuWwmFT+/Jsm/Lhs3c64rGVQUfUpm4pk mrzUStNNzJNWkUxmyMq8U7Wec0zaJTWOVgg== X-ME-Sender: <xms:ibIAZRmzDP1aJG8n-QYI9roYjjNI9Yh_QlXNANpW1NfyUK5ISwuOWQ> <xme:ibIAZc3TUjgFhq1wKip7BjftGomEFUpaUxYcyJuAEOHTSOkmx1By9tscmM4SNRwUk IMcEOqzCCJnohSw-eY> X-ME-Received: <xmr:ibIAZXq06mOxyDMFf8uNkX3Zs0iprWRVsr7sNkuzQmaTA0d0Qj4cxkzyiM7k1m0> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeiiedgudeftdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhm ihhtrhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrg htthgvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeev ledvveenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:ibIAZRngb9oxyGphMXXfxI9VOwatkAJ3pgtBl0x_-fMTXTorcfYBLQ> <xmx:ibIAZf2kSAZsVunFrNyGNWZ0-tDvCXtcfT3E_mtwzOkJOFHU5jvcfg> <xmx:ibIAZQsqTPGkqSzNdvaDQ9gT5MUKI5fhzpWS_eCtc2sXETQrDBygqw> <xmx:irIAZQymLTdxnikXCIymByTJ9dkX5amclxgclGuXr36fAtJS0p7DOA> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 12 Sep 2023 14:48:40 -0400 (EDT) Message-ID: <28a7916e-92d5-77ab-a61e-f85b59ac76b1@HIDDEN> Date: Tue, 12 Sep 2023 21:48:37 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> <831qf3pd1y.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <831qf3pd1y.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.2 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.2 (---) On 12/09/2023 19:32, Eli Zaretskii wrote: >> Date: Tue, 12 Sep 2023 17:23:53 +0300 >> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov <dmitry@HIDDEN> >> >> - Dropping most of the setup in read_and_dispose_of_process_output >> (which creates some consing too) and calling >> Finternal_default_process_filter directly (call_filter_directly.diff), >> when it is the filter to be used anyway. >> >> - Going around that function entirely, skipping the creation of a Lisp >> string (CHARS -> TEXT) and inserting into the buffer directly (when the >> filter is set to the default, of course). Copied and adapted some code >> from 'call_process' for that (read_and_insert_process_output.diff). >> >> Neither are intended as complete proposals, but here are some >> comparisons. Note that either of these patches could only help the >> implementations that don't set up process filter (the naive first one, >> and the new parallel number 5 above). >> >> For testing, I used two different repo checkouts that are large enough >> to not finish too quickly: gecko-dev and torvalds-linux. >> >> master >> >> | Function | gecko-dev | linux | >> | find-directory-files-recursively | 1.69 | 0.41 | >> | find-directory-files-recursively-2 | 1.16 | 0.28 | >> | find-directory-files-recursively-3 | 0.92 | 0.23 | >> | find-directory-files-recursively-5 | 1.07 | 0.26 | >> | find-directory-files-recursively (rpom 409600) | 1.42 | 0.35 | >> | find-directory-files-recursively-2 (rpom 409600) | 0.90 | 0.25 | >> | find-directory-files-recursively-5 (rpom 409600) | 0.89 | 0.24 | >> >> call_filter_directly.diff (basically, not much difference) >> >> | Function | gecko-dev | linux | >> | find-directory-files-recursively | 1.64 | 0.38 | >> | find-directory-files-recursively-5 | 1.05 | 0.26 | >> | find-directory-files-recursively (rpom 409600) | 1.42 | 0.36 | >> | find-directory-files-recursively-5 (rpom 409600) | 0.91 | 0.25 | >> >> read_and_insert_process_output.diff (noticeable differences) >> >> | Function | gecko-dev | linux | >> | find-directory-files-recursively | 1.30 | 0.34 | >> | find-directory-files-recursively-5 | 1.03 | 0.25 | >> | find-directory-files-recursively (rpom 409600) | 1.20 | 0.35 | >> | find-directory-files-recursively-5 (rpom 409600) | (!!) 0.72 | 0.21 | >> >> So it seems like we have at least two potential ways to implement an >> asynchronous file listing routine that is as fast or faster than the >> synchronous one (if only thanks to starting the parsing in parallel). >> >> Combining the last patch together with using the very large value of >> read-process-output-max seems to yield the most benefit, but I'm not >> sure if it's appropriate to just raise that value in our code, though. >> >> Thoughts? > > I'm not sure what exactly is here to think about. Removing portions > of read_and_insert_process_output, or bypassing it entirely, is not > going to fly, because AFAIU it basically means we don't decode text, > which can only work with plain ASCII file names, and/or don't move the > markers in the process buffer, which also cannot be avoided. That one was really a test to see whether the extra handling added any meaningful consing to affect GC. Removing it didn't make a difference, table number 2, so no. > If you > want to conclude that inserting the process's output into a buffer > without consing Lisp strings is faster (which I'm not sure, see below, > but it could be true), That's what my tests seem to show, see table 3 (the last one). > then we could try extending > internal-default-process-filter (or writing a new filter function > similar to it) so that it inserts the stuff into the gap and then uses > decode_coding_gap, Can that work at all? By the time internal-default-process-filter is called, we have already turned the string from char* into Lisp_Object text, which we then pass to it. So consing has already happened, IIUC. > which converts inserted bytes in-place -- that, at > least, will be correct and will avoid consing intermediate temporary > strings from the process output, then decoding them, then inserting > them. Other than that, the -2 and -3 variants are very close > runners-up of -5, so maybe I'm missing something, but I see no reason > be too excited here? I mean, 0.89 vs 0.92? really? The important part is not 0.89 vs 0.92 (that would be meaningless indeed), but that we have an _asyncronous_ implementation of the feature that works as fast as the existing synchronous one (or faster! if we also bind read-process-output-max to a large value, the time is 0.72). The possible applications for that range from simple (printing progress bar while the scan is happening) to more advanced (launching a concurrent process where we pipe the received file names concurrently to 'xargs grep'), including visuals (xref buffer which shows the intermediate search results right away, updating them gradually, all without blocking the UI). > About inserting into the buffer: what we do is insert into the gap, > and when the gap becomes full, we enlarge it. Enlarging the gap > involves: (a) enlarging the chunk of memory allocated to buffer text > (which might mean we ask the OS for more memory), and (b) moving the > characters after the gap to the right to free space for inserting more > stuff. This is pretty fast, but still, with a large pipe buffer and a > lot of output, we do this many times, so it could add up to something > pretty tangible. It's hard to me to tell whether this is > significantly faster than consing strings and inserting them, only > measurements can tell. See the benchmark tables and the POC patch in my previous email. Using a better filter function would be ideal, but it seems like that's not going to fit the current design. Happy to be proven wrong, though.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 12 Sep 2023 16:33:12 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 12 12:33:12 2023 Received: from localhost ([127.0.0.1]:59864 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qg6K7-00055Z-VZ for submit <at> debbugs.gnu.org; Tue, 12 Sep 2023 12:33:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52164) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qg6K3-00055I-MB for 64735 <at> debbugs.gnu.org; Tue, 12 Sep 2023 12:33:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qg6Js-0007xZ-9R; Tue, 12 Sep 2023 12:32:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=VZwx7DhGquSA4ytEqqiIcDZjXhsS4uUJ1h0/YBNoRXM=; b=Tsc1X9aST/DO QIAxsu/o3H/W4q3JkVNIL7Ci38gMm/VYHYJ3XA0a9rT8RsCcIxO5TtIYB54Zuggfuy0b6oUvqwnr0 JuLec/i6QKwQdEE6p/DpSgW13BsVZ+CWCpQe/+byCkoifOoqW8juKkfIKvjWdezLwWne7/8EUE746 /wrMbwzKDHsPVDr+CuosAe4dnWb+xbTKGI2wC5YxRhKD3Xoxt6DSwaBX2jcykB/fpDN/4xoQ2wGIo XwwtuO4r4Qvq4UZDcxDzHWgoDVQwsv+e+xIjXCOnW4RQhOxQnWVkRf4NzByaMOJhggiRcMDx3bvT/ a3RWAbFIzRpk48Z+NFpslQ==; Date: Tue, 12 Sep 2023 19:32:41 +0300 Message-Id: <831qf3pd1y.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> (message from Dmitry Gutov on Tue, 12 Sep 2023 17:23:53 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Tue, 12 Sep 2023 17:23:53 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > - Dropping most of the setup in read_and_dispose_of_process_output > (which creates some consing too) and calling > Finternal_default_process_filter directly (call_filter_directly.diff), > when it is the filter to be used anyway. > > - Going around that function entirely, skipping the creation of a Lisp > string (CHARS -> TEXT) and inserting into the buffer directly (when the > filter is set to the default, of course). Copied and adapted some code > from 'call_process' for that (read_and_insert_process_output.diff). > > Neither are intended as complete proposals, but here are some > comparisons. Note that either of these patches could only help the > implementations that don't set up process filter (the naive first one, > and the new parallel number 5 above). > > For testing, I used two different repo checkouts that are large enough > to not finish too quickly: gecko-dev and torvalds-linux. > > master > > | Function | gecko-dev | linux | > | find-directory-files-recursively | 1.69 | 0.41 | > | find-directory-files-recursively-2 | 1.16 | 0.28 | > | find-directory-files-recursively-3 | 0.92 | 0.23 | > | find-directory-files-recursively-5 | 1.07 | 0.26 | > | find-directory-files-recursively (rpom 409600) | 1.42 | 0.35 | > | find-directory-files-recursively-2 (rpom 409600) | 0.90 | 0.25 | > | find-directory-files-recursively-5 (rpom 409600) | 0.89 | 0.24 | > > call_filter_directly.diff (basically, not much difference) > > | Function | gecko-dev | linux | > | find-directory-files-recursively | 1.64 | 0.38 | > | find-directory-files-recursively-5 | 1.05 | 0.26 | > | find-directory-files-recursively (rpom 409600) | 1.42 | 0.36 | > | find-directory-files-recursively-5 (rpom 409600) | 0.91 | 0.25 | > > read_and_insert_process_output.diff (noticeable differences) > > | Function | gecko-dev | linux | > | find-directory-files-recursively | 1.30 | 0.34 | > | find-directory-files-recursively-5 | 1.03 | 0.25 | > | find-directory-files-recursively (rpom 409600) | 1.20 | 0.35 | > | find-directory-files-recursively-5 (rpom 409600) | (!!) 0.72 | 0.21 | > > So it seems like we have at least two potential ways to implement an > asynchronous file listing routine that is as fast or faster than the > synchronous one (if only thanks to starting the parsing in parallel). > > Combining the last patch together with using the very large value of > read-process-output-max seems to yield the most benefit, but I'm not > sure if it's appropriate to just raise that value in our code, though. > > Thoughts? I'm not sure what exactly is here to think about. Removing portions of read_and_insert_process_output, or bypassing it entirely, is not going to fly, because AFAIU it basically means we don't decode text, which can only work with plain ASCII file names, and/or don't move the markers in the process buffer, which also cannot be avoided. If you want to conclude that inserting the process's output into a buffer without consing Lisp strings is faster (which I'm not sure, see below, but it could be true), then we could try extending internal-default-process-filter (or writing a new filter function similar to it) so that it inserts the stuff into the gap and then uses decode_coding_gap, which converts inserted bytes in-place -- that, at least, will be correct and will avoid consing intermediate temporary strings from the process output, then decoding them, then inserting them. Other than that, the -2 and -3 variants are very close runners-up of -5, so maybe I'm missing something, but I see no reason be too excited here? I mean, 0.89 vs 0.92? really? About inserting into the buffer: what we do is insert into the gap, and when the gap becomes full, we enlarge it. Enlarging the gap involves: (a) enlarging the chunk of memory allocated to buffer text (which might mean we ask the OS for more memory), and (b) moving the characters after the gap to the right to free space for inserting more stuff. This is pretty fast, but still, with a large pipe buffer and a lot of output, we do this many times, so it could add up to something pretty tangible. It's hard to me to tell whether this is significantly faster than consing strings and inserting them, only measurements can tell.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 12 Sep 2023 14:26:41 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 12 10:26:41 2023 Received: from localhost ([127.0.0.1]:59719 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qg4Lg-0007P7-Hx for submit <at> debbugs.gnu.org; Tue, 12 Sep 2023 10:26:41 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:58487) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qg4Lf-0007Ot-11 for 64735 <at> debbugs.gnu.org; Tue, 12 Sep 2023 10:26:39 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 34AD95C0145; Tue, 12 Sep 2023 10:26:29 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Tue, 12 Sep 2023 10:26:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1694528789; x=1694615189; bh=kb nA+MxBx0YBIlUbx+xdq3+WRUhtmwdEJkf0eiGO1I8=; b=mirJCNK2fVrZnIxqbh CSDZExhedn3+0Lcre+qEbWbjO7sbM3L6FR4/CJV1W89DrRaO5EzlzcrheVWXQBEp KlZ7aA1lTpSB+qfdOIOJqGWq5nACoubDehzwlE+hpjaqKwHSGEgAZe2JAIf3lnrr BplNRC2Vv0w/GTJpyG08GGcZFToHa6wXTf9UqGfxCoJsDVPHgg3qUzd9wjswLgTm 7TaGcKE4C3RgDchBSvc1evseTCi6Ma77CWRAetif5GP94QVQ5uWWwdt91LALL/0T N5rnd+YiecqgInm9VzHlb0OmcuGTxzBRXoeacZekUGz/IVw5ujV7d6Z23NHK8MGs FVEg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1694528789; x=1694615189; bh=kbnA+MxBx0YBI lUbx+xdq3+WRUhtmwdEJkf0eiGO1I8=; b=TSx/Spgn5YVuc1tAaugnwsWK5yioH knRCDDp6gmfrog8/qAlk3HG66SSJPDtqcR6IO75GjbjwgFcEmt6p+mn/hJWL7QYS 643bPw2bejp+EkMndE2A2xQxnqPjFqvYXd7QNhK8FBxzAyuf/MlueAd+h9BIphsZ 3RvWMTqOXxgTX80uBKE/cIMykijgneOM/sTFZB0pm780FppbJQGwurOd6M7pys29 zXIXbUGBCP0ea9d8AdZa4xUY1n1WZkcub//dtwtTyJZ1q/kdp+z5dZB7NDSHggWU fi4HH0NvGKFDRZ52sxAyHmOoHue84GG5mbwfOi2LRs0TZ6vdvW/1IgprQ== X-ME-Sender: <xms:FXUAZbU2TQEUA4ujJ713719hStz0pMCHeQx-llrVgFfBjylVLWZXyw> <xme:FXUAZTkF2ZmEm8UfZjDTv9iTWk8z5u6PBa64z1hbY_paowXYI5aAj_oPJysN1WX47 WrSkzsW1OwmVrSkgBA> X-ME-Received: <xmr:FXUAZXbHBLJNfY4fBCW0jKcfBqS1ED6hlhQvJbkDmoO8Du7mUEvo5T1D1D5mF4A> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeiiedgjeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptgfkffggfgfuhffvvehfjgesmhdtreertdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeiheegkeetgffghefhgeeiveeuudegvdeuteffhfettdelleehkeffledvuddt leenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:FXUAZWVd6JCMNRqNnVzIYKskgFKbx_PilmjmdQwOVNqgiJek9fqcLQ> <xmx:FXUAZVlGOGK3T9UMEHUe7S4G8UUkNU4sLgsSBZ6KPdUBBHeZE5jZbg> <xmx:FXUAZTeGHOrqArrHe-GlEYbXMonOEmSUa2uzEs8y5gAtHWCSqavkRQ> <xmx:FXUAZbjbpFl-GrK9X5laaExaTMxCDPbe9K1DZJFahw6MOFBTbgp_Qg> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 12 Sep 2023 10:26:27 -0400 (EDT) Content-Type: multipart/mixed; boundary="------------pNrGtcmBObRwwOXzSA0eJQDE" Message-ID: <e38f060a-b55e-86c7-73e5-630ec76013d5@HIDDEN> Date: Tue, 12 Sep 2023 17:26:26 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US From: Dmitry Gutov <dmitry@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> In-Reply-To: <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN> X-Spam-Score: -2.2 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.2 (---) This is a multi-part message in MIME format. --------------pNrGtcmBObRwwOXzSA0eJQDE Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 12/09/2023 17:23, Dmitry Gutov wrote: > Neither are intended as complete proposals, but here are some > comparisons. Note that either of these patches could only help the > implementations that don't set up process filter (the naive first one, > and the new parallel number 5 above). Sorry, forgot to attach the patches. --------------pNrGtcmBObRwwOXzSA0eJQDE Content-Type: text/x-patch; charset=UTF-8; name="call_filter_directly.diff" Content-Disposition: attachment; filename="call_filter_directly.diff" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3NyYy9wcm9jZXNzLmMgYi9zcmMvcHJvY2Vzcy5jCmluZGV4IDA4Y2I4 MTBlYzEzLi5iZGJlOGQ5NjA2NCAxMDA2NDQKLS0tIGEvc3JjL3Byb2Nlc3MuYworKysgYi9z cmMvcHJvY2Vzcy5jCkBAIC02MjI3LDcgKzYyMjcsMTUgQEAgcmVhZF9wcm9jZXNzX291dHB1 dCAoTGlzcF9PYmplY3QgcHJvYywgaW50IGNoYW5uZWwpCiAgICAgIGZyaWVuZHMgZG9uJ3Qg ZXhwZWN0IGN1cnJlbnQtYnVmZmVyIHRvIGJlIGNoYW5nZWQgZnJvbSB1bmRlciB0aGVtLiAg Ki8KICAgcmVjb3JkX3Vud2luZF9jdXJyZW50X2J1ZmZlciAoKTsKIAotICByZWFkX2FuZF9k aXNwb3NlX29mX3Byb2Nlc3Nfb3V0cHV0IChwLCBjaGFycywgbmJ5dGVzLCBjb2RpbmcpOwor ICBpZiAocC0+ZmlsdGVyID09IFFpbnRlcm5hbF9kZWZhdWx0X3Byb2Nlc3NfZmlsdGVyKQor ICAgIHsKKyAgICAgIExpc3BfT2JqZWN0IHRleHQ7CisgICAgICBkZWNvZGVfY29kaW5nX2Nf c3RyaW5nIChjb2RpbmcsICh1bnNpZ25lZCBjaGFyICopIGNoYXJzLCBuYnl0ZXMsIFF0KTsK KyAgICAgIHRleHQgPSBjb2RpbmctPmRzdF9vYmplY3Q7CisgICAgICBGaW50ZXJuYWxfZGVm YXVsdF9wcm9jZXNzX2ZpbHRlciAocHJvYywgdGV4dCk7CisgICAgfQorICBlbHNlCisgICAg cmVhZF9hbmRfZGlzcG9zZV9vZl9wcm9jZXNzX291dHB1dCAocCwgY2hhcnMsIG5ieXRlcywg Y29kaW5nKTsKIAogICAvKiBIYW5kbGluZyB0aGUgcHJvY2VzcyBvdXRwdXQgc2hvdWxkIG5v dCBkZWFjdGl2YXRlIHRoZSBtYXJrLiAgKi8KICAgVmRlYWN0aXZhdGVfbWFyayA9IG9kZWFj dGl2YXRlOwo= --------------pNrGtcmBObRwwOXzSA0eJQDE Content-Type: text/x-patch; charset=UTF-8; name="read_and_insert_process_output.diff" Content-Disposition: attachment; filename="read_and_insert_process_output.diff" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3NyYy9wcm9jZXNzLmMgYi9zcmMvcHJvY2Vzcy5jCmluZGV4IDA4Y2I4 MTBlYzEzLi41ZGI1NjY5MmZlMSAxMDA2NDQKLS0tIGEvc3JjL3Byb2Nlc3MuYworKysgYi9z cmMvcHJvY2Vzcy5jCkBAIC02MTEyLDYgKzYxMTIsMTEgQEAgcmVhZF9hbmRfZGlzcG9zZV9v Zl9wcm9jZXNzX291dHB1dCAoc3RydWN0IExpc3BfUHJvY2VzcyAqcCwgY2hhciAqY2hhcnMs CiAJCQkJICAgIHNzaXplX3QgbmJ5dGVzLAogCQkJCSAgICBzdHJ1Y3QgY29kaW5nX3N5c3Rl bSAqY29kaW5nKTsKIAorc3RhdGljIHZvaWQKK3JlYWRfYW5kX2luc2VydF9wcm9jZXNzX291 dHB1dCAoc3RydWN0IExpc3BfUHJvY2VzcyAqcCwgY2hhciAqYnVmLAorCQkJCSAgICBzc2l6 ZV90IG5yZWFkLAorCQkJCXN0cnVjdCBjb2Rpbmdfc3lzdGVtICpwcm9jZXNzX2NvZGluZyk7 CisKIC8qIFJlYWQgcGVuZGluZyBvdXRwdXQgZnJvbSB0aGUgcHJvY2VzcyBjaGFubmVsLAog ICAgc3RhcnRpbmcgd2l0aCBvdXIgYnVmZmVyZWQtYWhlYWQgY2hhcmFjdGVyIGlmIHdlIGhh dmUgb25lLgogICAgWWllbGQgbnVtYmVyIG9mIGRlY29kZWQgY2hhcmFjdGVycyByZWFkLApA QCAtNjIyNyw3ICs2MjMyLDEwIEBAIHJlYWRfcHJvY2Vzc19vdXRwdXQgKExpc3BfT2JqZWN0 IHByb2MsIGludCBjaGFubmVsKQogICAgICBmcmllbmRzIGRvbid0IGV4cGVjdCBjdXJyZW50 LWJ1ZmZlciB0byBiZSBjaGFuZ2VkIGZyb20gdW5kZXIgdGhlbS4gICovCiAgIHJlY29yZF91 bndpbmRfY3VycmVudF9idWZmZXIgKCk7CiAKLSAgcmVhZF9hbmRfZGlzcG9zZV9vZl9wcm9j ZXNzX291dHB1dCAocCwgY2hhcnMsIG5ieXRlcywgY29kaW5nKTsKKyAgaWYgKHAtPmZpbHRl ciA9PSBRaW50ZXJuYWxfZGVmYXVsdF9wcm9jZXNzX2ZpbHRlcikKKyAgICByZWFkX2FuZF9p bnNlcnRfcHJvY2Vzc19vdXRwdXQgKHAsIGNoYXJzLCBuYnl0ZXMsIGNvZGluZyk7CisgIGVs c2UKKyAgICByZWFkX2FuZF9kaXNwb3NlX29mX3Byb2Nlc3Nfb3V0cHV0IChwLCBjaGFycywg bmJ5dGVzLCBjb2RpbmcpOwogCiAgIC8qIEhhbmRsaW5nIHRoZSBwcm9jZXNzIG91dHB1dCBz aG91bGQgbm90IGRlYWN0aXZhdGUgdGhlIG1hcmsuICAqLwogICBWZGVhY3RpdmF0ZV9tYXJr ID0gb2RlYWN0aXZhdGU7CkBAIC02MjM2LDYgKzYyNDQsNDYgQEAgcmVhZF9wcm9jZXNzX291 dHB1dCAoTGlzcF9PYmplY3QgcHJvYywgaW50IGNoYW5uZWwpCiAgIHJldHVybiBuYnl0ZXM7 CiB9CiAKK3N0YXRpYyB2b2lkIHJlYWRfYW5kX2luc2VydF9wcm9jZXNzX291dHB1dCAoc3Ry dWN0IExpc3BfUHJvY2VzcyAqcCwgY2hhciAqYnVmLAorCQkJCSAgICBzc2l6ZV90IG5yZWFk LAorCQkJCSAgICBzdHJ1Y3QgY29kaW5nX3N5c3RlbSAqcHJvY2Vzc19jb2RpbmcpCit7Cisg IGlmICghbnJlYWQgfHwgTklMUCAocC0+YnVmZmVyKSB8fCAhQlVGRkVSX0xJVkVfUCAoWEJV RkZFUiAocC0+YnVmZmVyKSkpCisgICAgOworICBlbHNlIGlmIChOSUxQIChCVkFSIChYQlVG RkVSKHAtPmJ1ZmZlciksIGVuYWJsZV9tdWx0aWJ5dGVfY2hhcmFjdGVycykpCisJICAgJiYg ISBDT0RJTkdfTUFZX1JFUVVJUkVfREVDT0RJTkcgKHByb2Nlc3NfY29kaW5nKSkKKyAgICB7 CisgICAgICBpbnNlcnRfMV9ib3RoIChidWYsIG5yZWFkLCBucmVhZCwgMCwgMCwgMCk7Cisg ICAgICBzaWduYWxfYWZ0ZXJfY2hhbmdlIChQVCAtIG5yZWFkLCAwLCBucmVhZCk7CisgICAg fQorICBlbHNlCisgICAgewkJCS8qIFdlIGhhdmUgdG8gZGVjb2RlIHRoZSBpbnB1dC4gICov CisgICAgICBMaXNwX09iamVjdCBjdXJidWY7CisgICAgICBpbnQgY2FycnlvdmVyID0gMDsK KyAgICAgIHNwZWNwZGxfcmVmIGNvdW50MSA9IFNQRUNQRExfSU5ERVggKCk7CisKKyAgICAg IFhTRVRCVUZGRVIgKGN1cmJ1ZiwgY3VycmVudF9idWZmZXIpOworICAgICAgLyogV2UgY2Fu bm90IGFsbG93IGFmdGVyLWNoYW5nZS1mdW5jdGlvbnMgYmUgcnVuCisJIGR1cmluZyBkZWNv ZGluZywgYmVjYXVzZSB0aGF0IG1pZ2h0IG1vZGlmeSB0aGUKKwkgYnVmZmVyLCB3aGlsZSB3 ZSByZWx5IG9uIHByb2Nlc3NfY29kaW5nLnByb2R1Y2VkIHRvCisJIGZhaXRoZnVsbHkgcmVm bGVjdCBpbnNlcnRlZCB0ZXh0IHVudGlsIHdlCisJIFRFTVBfU0VUX1BUX0JPVEggYmVsb3cu ICAqLworICAgICAgc3BlY2JpbmQgKFFpbmhpYml0X21vZGlmaWNhdGlvbl9ob29rcywgUXQp OworICAgICAgZGVjb2RlX2NvZGluZ19jX3N0cmluZyAocHJvY2Vzc19jb2RpbmcsCisJCQkg ICAgICAodW5zaWduZWQgY2hhciAqKSBidWYsIG5yZWFkLCBjdXJidWYpOworICAgICAgdW5i aW5kX3RvIChjb3VudDEsIFFuaWwpOworCisgICAgICBURU1QX1NFVF9QVF9CT1RIIChQVCAr IHByb2Nlc3NfY29kaW5nLT5wcm9kdWNlZF9jaGFyLAorCQkJUFRfQllURSArIHByb2Nlc3Nf Y29kaW5nLT5wcm9kdWNlZCk7CisgICAgICBzaWduYWxfYWZ0ZXJfY2hhbmdlIChQVCAtIHBy b2Nlc3NfY29kaW5nLT5wcm9kdWNlZF9jaGFyLAorCQkJICAgMCwgcHJvY2Vzc19jb2Rpbmct PnByb2R1Y2VkX2NoYXIpOworICAgICAgY2FycnlvdmVyID0gcHJvY2Vzc19jb2RpbmctPmNh cnJ5b3Zlcl9ieXRlczsKKyAgICAgIGlmIChjYXJyeW92ZXIgPiAwKQorCW1lbWNweSAoYnVm LCBwcm9jZXNzX2NvZGluZy0+Y2FycnlvdmVyLAorCQlwcm9jZXNzX2NvZGluZy0+Y2Fycnlv dmVyX2J5dGVzKTsKKyAgICB9Cit9CisKIHN0YXRpYyB2b2lkCiByZWFkX2FuZF9kaXNwb3Nl X29mX3Byb2Nlc3Nfb3V0cHV0IChzdHJ1Y3QgTGlzcF9Qcm9jZXNzICpwLCBjaGFyICpjaGFy cywKIAkJCQkgICAgc3NpemVfdCBuYnl0ZXMsCg== --------------pNrGtcmBObRwwOXzSA0eJQDE--
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 12 Sep 2023 14:24:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 12 10:24:10 2023
Received: from localhost ([127.0.0.1]:59705 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qg4JG-0007L7-1b
for submit <at> debbugs.gnu.org; Tue, 12 Sep 2023 10:24:10 -0400
Received: from out4-smtp.messagingengine.com ([66.111.4.28]:45243)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qg4JD-0007Kt-Co
for 64735 <at> debbugs.gnu.org; Tue, 12 Sep 2023 10:24:08 -0400
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
by mailout.nyi.internal (Postfix) with ESMTP id 632F25C028B;
Tue, 12 Sep 2023 10:23:57 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute6.internal (MEProxy); Tue, 12 Sep 2023 10:23:57 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm3; t=
1694528637; x=1694615037; bh=QqKv4EB9n4HHUgnDi8ixSAF6v1Yo+A/OUTO
pnI8ETVg=; b=iv6rbAJMZ+6wQrUVJui0ruD2UGNZO1XnpaBrW+/f37XDGq8YZsW
D/CKSJVO7Zae7ZxUzTVPt7dlb3hzDroyJg94EskN8uKZupTwQwDRI9gh7oN4xqhT
AwMRSaC3FTSmsokKAQWQzA7ImLNWX8bHtMnvTpvFiNlELAlkP7f2FsCZAs8h6wst
MaE6K8oack3WnfDQQlaIP6/qAubT8wadiGYzlwIzj3Yyf4dHglpKuXiJUAR2tS2D
lL30YogJ0VxSLCt/CNJDovbu/JOqJgSj5laxf42mxllPIFeb5Pv8CwgIKYKDDf8t
Dss/aB83Qa6Lfma9Spmrb/NHOBCorLiM7jQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
1694528637; x=1694615037; bh=QqKv4EB9n4HHUgnDi8ixSAF6v1Yo+A/OUTO
pnI8ETVg=; b=JECeLGxyo/l8XSSy2tujwMEjfNqCCfbsvg+7AakIUptJE9kpNEh
Cz5tpaVx0JxZuNMj7BdUyQ4hApDW2VED7iBYg4VkXBPRF2jl3XJ/UMPd5/BhC+dr
BHfGV1vqT0SpHb0ITXMNetX4afeAKXvIPJb3KQ2ZIbtg8bDkL8WdrxAf45zrnVyM
lHAqqZItk1ZUt8Q38K5EQhePwAJltJSrPpT+WIOsqtUWFPYSEMeBBAQgW9+x2WfW
nqdTpzYuapoTmK7gWPpzj7w5yZ3ZWTTl/9ou9chG2WGjRyA9OL5KOe3T1g7R85zX
uIcxSFy3qJ1FhMXxWwDI+yf5hy26HaCynIg==
X-ME-Sender: <xms:fXQAZShaSr8HT3UPR3Qk5-1wo8MOTZGEhyAYpsi1RIvMMuwaNUgejg>
<xme:fXQAZTBvflQrvq0GQzsx2Evc99w4yStQDlwwN_U8KlocfM9DFPz9OG6NT-98UYyFS
_YlV5a2K__Ptdw3bCY>
X-ME-Received: <xmr:fXQAZaHPWqkmmRivGivmm4MKJzkB1SVZQFc6kSEWWBE7bSzqj0o5cdkS5I3vTvo>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeiiedgjeehucetufdoteggodetrfdotf
fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi
thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth
htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel
vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug
hmihhtrhihsehguhhtohhvrdguvghv
X-ME-Proxy: <xmx:fXQAZbR5hP0S0f4XehlQkkG7QFaOBReGnhDE1-MJf4hmLjgd2Xh_2A>
<xmx:fXQAZfxN83x-hQD7tKXrO7eIJ2dZLQjerGwZcACXVfcddjbzQcxhsw>
<xmx:fXQAZZ4437inoggqoU049fzS0lNr85ssAQ4hmccDxa7ybZY5NNoSvg>
<xmx:fXQAZf_Tbq_Rt0Aqr4SrsB4-3vR195kvEbDVF3TOHZThcl2d-9pCTg>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
12 Sep 2023 10:23:55 -0400 (EDT)
Message-ID: <e6dd2d5c-9a88-6e18-f1e2-544a9cb1beaf@HIDDEN>
Date: Tue, 12 Sep 2023 17:23:53 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN>
<83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN>
<83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN>
<834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN>
<d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
<83bkfs2tw5.fsf@HIDDEN>
<18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN>
<83il8lxjcu.fsf@HIDDEN>
<2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN>
<8334zmtwwi.fsf@HIDDEN>
<e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN>
<83tts0rkh5.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <83tts0rkh5.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.2 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.2 (---)
On 11/09/2023 14:57, Eli Zaretskii wrote:
>> So there is also a second recording for
>> find-directory-files-recursively-2 with read-process-output-max=409600.
>> It does improve the performance significantly (and reduce the number of
>> GC pauses). I guess what I'm still not clear on, is whether the number
>> of GC pauses is fewer because of less consing (the only column that
>> looks significantly different is the 3rd: VECTOR-CELLS), or because the
>> process finishes faster due to larger buffers, which itself causes fewer
>> calls to maybe_gc.
> I think the latter.
It might be both.
To try to analyze how large might per-chunk overhead be (CPU and GC-wise
combined), I first implemented the same function in yet another way that
doesn't use :filter (so that the default filter is used). But still
asynchronously, with parsing happening concurrently to the process:
(defun find-directory-files-recursively-5 (dir regexp &optional
include-directories _p follow-symlinks)
(cl-assert (null _p) t "find-directory-files-recursively can't accept
arbitrary predicates")
(with-temp-buffer
(setq case-fold-search nil)
(cd dir)
(let* ((command
(append
(list "find" (file-local-name dir))
(if follow-symlinks
'("-L")
'("!" "(" "-type" "l" "-xtype" "d" ")"))
(unless (string-empty-p regexp)
(list "-regex" (concat ".*" regexp ".*")))
(unless include-directories
'("!" "-type" "d"))
'("-print0")
))
(remote (file-remote-p dir))
(proc
(if remote
(let ((proc (apply #'start-file-process
"find" (current-buffer) command)))
(set-process-sentinel proc (lambda (_proc _state)))
(set-process-query-on-exit-flag proc nil)
proc)
(make-process :name "find" :buffer (current-buffer)
:connection-type 'pipe
:noquery t
:sentinel (lambda (_proc _state))
:command command)))
start ret)
(setq start (point-min))
(while (accept-process-output proc)
(goto-char start)
(while (search-forward "\0" nil t)
(push (buffer-substring-no-properties start (1- (point))) ret)
(setq start (point))))
ret)))
This method already improved the performance somewhat (compared to
find-directory-files-recursively-2), but not too much. So I tried these
next two steps:
- Dropping most of the setup in read_and_dispose_of_process_output
(which creates some consing too) and calling
Finternal_default_process_filter directly (call_filter_directly.diff),
when it is the filter to be used anyway.
- Going around that function entirely, skipping the creation of a Lisp
string (CHARS -> TEXT) and inserting into the buffer directly (when the
filter is set to the default, of course). Copied and adapted some code
from 'call_process' for that (read_and_insert_process_output.diff).
Neither are intended as complete proposals, but here are some
comparisons. Note that either of these patches could only help the
implementations that don't set up process filter (the naive first one,
and the new parallel number 5 above).
For testing, I used two different repo checkouts that are large enough
to not finish too quickly: gecko-dev and torvalds-linux.
master
| Function | gecko-dev | linux |
| find-directory-files-recursively | 1.69 | 0.41 |
| find-directory-files-recursively-2 | 1.16 | 0.28 |
| find-directory-files-recursively-3 | 0.92 | 0.23 |
| find-directory-files-recursively-5 | 1.07 | 0.26 |
| find-directory-files-recursively (rpom 409600) | 1.42 | 0.35 |
| find-directory-files-recursively-2 (rpom 409600) | 0.90 | 0.25 |
| find-directory-files-recursively-5 (rpom 409600) | 0.89 | 0.24 |
call_filter_directly.diff (basically, not much difference)
| Function | gecko-dev | linux |
| find-directory-files-recursively | 1.64 | 0.38 |
| find-directory-files-recursively-5 | 1.05 | 0.26 |
| find-directory-files-recursively (rpom 409600) | 1.42 | 0.36 |
| find-directory-files-recursively-5 (rpom 409600) | 0.91 | 0.25 |
read_and_insert_process_output.diff (noticeable differences)
| Function | gecko-dev | linux |
| find-directory-files-recursively | 1.30 | 0.34 |
| find-directory-files-recursively-5 | 1.03 | 0.25 |
| find-directory-files-recursively (rpom 409600) | 1.20 | 0.35 |
| find-directory-files-recursively-5 (rpom 409600) | (!!) 0.72 | 0.21 |
So it seems like we have at least two potential ways to implement an
asynchronous file listing routine that is as fast or faster than the
synchronous one (if only thanks to starting the parsing in parallel).
Combining the last patch together with using the very large value of
read-process-output-max seems to yield the most benefit, but I'm not
sure if it's appropriate to just raise that value in our code, though.
Thoughts?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 12 Sep 2023 13:11:17 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 12 09:11:17 2023 Received: from localhost ([127.0.0.1]:56473 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qg3Ai-0004Ll-Ok for submit <at> debbugs.gnu.org; Tue, 12 Sep 2023 09:11:17 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:59111) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qg3Ah-0004LY-5Y for 64735 <at> debbugs.gnu.org; Tue, 12 Sep 2023 09:11:16 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 5C18B5C0107; Tue, 12 Sep 2023 09:11:05 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 12 Sep 2023 09:11:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1694524265; x=1694610665; bh=9qMf1VTEcKVk+StAaAG1jAFIdhxK42vLIjq czYrcMj4=; b=gXgmiyO246hmw/gVzKmY0WT8ljXdavcl+xd5cGuGlbuZE2ECUhx OLg3eHnz5b1+ozBsJwdZ0WXhsEbK+UPwLgUWFeeIGI5IVr07HaczPPdm0f6NvjU8 n7TQG5KnDhV0G4SIyKk7MmL0vCdvS1MZr1cthTUa5WjMdFFuUF7y1b2MDcFfg9+v aE0OBWHnMxsB9q6zIP3CVpWaDZD772V93ipVdl2M8CSK8Pj3Y81cqeJItg2m1S7q cGW/fRZeRcdOvorH+WWsJCCjtda6ejhVlSwx7zQKi/P5nywHO4ZtMpUBXKWYcJIK 5mECrqjN/KwZqyjkZgfV3tv48GwnnxW569A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1694524265; x=1694610665; bh=9qMf1VTEcKVk+StAaAG1jAFIdhxK42vLIjq czYrcMj4=; b=moZM/cie22ToghgRysBm0c8ZUbjOO1maZ67blmN8CeUQ/M3bxTh UQ6djtLBAnB8zJTYH8VAUQDW+nDSKRcD3EaegU5i7f+5flG7AIfS6sYGWltKQMO7 FrEOnQnxE7QZYJFKc24NRCuMJU82UytplekNWjVxZ7kiAHlkHNuSXW1Tz4JvlJM9 BwAkBjZUrgghGpi/+/JrM8qxZNo6jCq0BMQrXFm+jqnnUdOfCVRhFZkfVvO5K1/9 DoplGS+wqszxY45D87B/qfPooaz1hAJdu+gHLiRDvq58Qzj8qerGmaM3v2v+LuuK lJ/qkcZFZLlB9Yga1xsjcTsRiLds741tx5g== X-ME-Sender: <xms:aGMAZZZWnKIIIezTkrER4a6DSRVqi2PkVVogZY6Sbzr1AG6Ko_nTqw> <xme:aGMAZQa-meAXpjh0aGyhzk2ACkt-SkZ5bEY3UbvjYRQZuSNQ7DUWpw4FKDvVYoKgl RImOu-KjBF3cqhj3ZE> X-ME-Received: <xmr:aGMAZb-kFxsfI7Yybwb9D54-rP6HOm8LFSfzQuoyVRQL_DXY78LVVBkoDE5FNgI> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeiiedgiedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:aGMAZXorDI4JbnrsVU0oIUNG3o-ubl9JiiXdCUafERSG9d0l1D1v8g> <xmx:aGMAZUrj-YZrCvA-YWK5P5GiC6pg_wQmazeRm4k3VO-xhR-ntIvGUA> <xmx:aGMAZdTeuC-ODbNl0q98GibrxNMGzy99_3mRc6jCKQ4B8VbEqrQH9g> <xmx:aWMAZfXKhr-sZkRy3I5q54UR9U-gw2lzVAkcaitX1vobTVY1UEWx_A> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 12 Sep 2023 09:11:03 -0400 (EDT) Message-ID: <f157382c-4941-75ff-0937-930d3021d059@HIDDEN> Date: Tue, 12 Sep 2023 16:11:01 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <14b26ff0-9d9b-3600-bf6e-133db4031622@HIDDEN> <83ledbpqmr.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83ledbpqmr.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.2 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.2 (---) On 12/09/2023 14:39, Eli Zaretskii wrote: >> Date: Tue, 12 Sep 2023 02:06:50 +0300 >> Cc:luangruo@HIDDEN,sbaugh@HIDDEN,yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov<dmitry@HIDDEN> >> >>> No, we don't wait until it's zero, we perform GC on the first >>> opportunity that we_notice_ that it crossed zero. So examining how >>> negative is the value of consing_until_gc when GC is actually >>> performed could tell us whether we checked the threshold with high >>> enough frequency, and comparing these values between different runs >>> could tell us whether the shorter time spend in GC means really less >>> garbage or less frequent checks for the need to GC. >> Good point, I'm attaching the same outputs with "last value of >> consing_until_gc" added to every line. >> >> There are some pretty low values in the "read-process-output-max 409600" >> part of the experiment, which probably means runtime staying in C >> accumulating the output into the (now larger) buffer? Not sure. > No, I think this means we really miss some GC opportunities, and we > cons quite a lot more strings between GC cycles due to that. Or possibly same number of strings but longer ones? > I guess > this happens because we somehow cons many strings in code that doesn't > call maybe_gc or something. Yes, staying in some C code that doesn't call maybe_gc for a while. I think we're describing the same thing, only I was doing that from the positive side (less frequent GCs = better performance in this scenario), and you from the negative one (less frequent GCs = more chances for an OOM to happen in some related but different scenario).
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 12 Sep 2023 11:40:16 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 12 07:40:16 2023 Received: from localhost ([127.0.0.1]:56326 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qg1kV-0007Xu-9h for submit <at> debbugs.gnu.org; Tue, 12 Sep 2023 07:40:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39480) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qg1kQ-0007XG-9q for 64735 <at> debbugs.gnu.org; Tue, 12 Sep 2023 07:40:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qg1k6-0002fT-AA; Tue, 12 Sep 2023 07:39:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=wcFFnMb+m4sTdfLtKnVv9R8+ktdQFlm0Gd/TEQvajQM=; b=k9NmyED/BhFq 10y5eqsGvz+8VW0iC4mFvmqq1Xida7XpRQdq1rb9DS4X9mE/XtiWIt/Kp54OAJVkn5qIKrF/k/e02 lJB2OHKKZiZsI4f+HbCqE4vW1WMaWjLDPEADnbM24rqWqE7snLIR3BHaX9TvGPc2RiPuagc5bQ6Ir L2K3nA9L7LBjkOrv2dQx4las2wzaxpgCi5NOvimg7Q3SV8OKiuxCMmH30I6tohZLBghr1HEecoV/j +jZKzCJRAlo/z2bIEBLN5bC/5GRJzcZ2LlDgGKvtdLGjrizTRjfMcHAzs2Rq145JAY13wE0ExnM+x iTN5ZzX1DdSHfoJzjdA71g==; Date: Tue, 12 Sep 2023 14:39:24 +0300 Message-Id: <83ledbpqmr.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <14b26ff0-9d9b-3600-bf6e-133db4031622@HIDDEN> (message from Dmitry Gutov on Tue, 12 Sep 2023 02:06:50 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> <14b26ff0-9d9b-3600-bf6e-133db4031622@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Tue, 12 Sep 2023 02:06:50 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > > No, we don't wait until it's zero, we perform GC on the first > > opportunity that we _notice_ that it crossed zero. So examining how > > negative is the value of consing_until_gc when GC is actually > > performed could tell us whether we checked the threshold with high > > enough frequency, and comparing these values between different runs > > could tell us whether the shorter time spend in GC means really less > > garbage or less frequent checks for the need to GC. > > Good point, I'm attaching the same outputs with "last value of > consing_until_gc" added to every line. > > There are some pretty low values in the "read-process-output-max 409600" > part of the experiment, which probably means runtime staying in C > accumulating the output into the (now larger) buffer? Not sure. No, I think this means we really miss some GC opportunities, and we cons quite a lot more strings between GC cycles due to that. I guess this happens because we somehow cons many strings in code that doesn't call maybe_gc or something.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 11 Sep 2023 23:07:07 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 19:07:07 2023 Received: from localhost ([127.0.0.1]:55087 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qfpzm-0002WQ-F8 for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 19:07:07 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:59347) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qfpzi-0002Vt-Os for 64735 <at> debbugs.gnu.org; Mon, 11 Sep 2023 19:07:04 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 51A0A5C00C3; Mon, 11 Sep 2023 19:06:53 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Mon, 11 Sep 2023 19:06:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1694473613; x=1694560013; bh=0F hjaCLYH0J+BiznP34bJ0U1c7h84T2s2gXbd3fQUA0=; b=EQB0CXLV84MNmvrHWA P8M2DtSRAJbL36478imu7qFYttgDXebCq+papZYPQVgHqOS9x2CBpXZPGz1+QfHn rHAG4v1hafj3Q6Ig3SyiVPZn69uW8VCFCxW9+gC+5BcY/jgRboh188zliSchVuIt Qxnj70V5eG6reFBAaUKpz2Zh/DFoJvPKwOhY6kFNJuugiBTi/is/MSI2S92H9sLy hpKLeeEpxnicKCraM9RBDje9VMPUOWoKmvabjNiVUJrE73K+42h29fOx0l8HKcGc 10VOw9+ZNqZkgYzFUUrOK2kEzlCWzr5pe0Xk9+ICExcvvIftCfywf9DngxDHxnEz ENtg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1694473613; x=1694560013; bh=0FhjaCLYH0J+B iznP34bJ0U1c7h84T2s2gXbd3fQUA0=; b=EfHaOo8SH2cS4tiE2YeR9C0jgaYG/ EnPlQ8hWL26KTGflOndIh3/P8473razPBYM91orNjqX8hWTYlgd9eDN7eXM9FbDH mZ+OxbBLQqC8AzmeUSSdJsZIcETU/XRgppB0/i19jFx3h0UXC3eHQElLfUg1Aehd CH1Sdt0noTs1n7yDmRNrZ7/QkNDpmi4hYmxlSmkOlDgUCGG132bmXyEOCXVFWTL4 kp+U30Gg9XryKco1cpuzVpZB7JSMCwEtjOSGT+U2avNoL3tFls1iukibD3Aobgwx Wu64Q4j056KH1b0umSgZA2QBlktUYKmECnYO5FLBN97s8cpESuyJfNhWg== X-ME-Sender: <xms:jJ3_ZEtIl2AVBLG9ncZ9JE31ezrTL-cBrdB2uxIPmk62LJv3SKmHaA> <xme:jJ3_ZBdGdwJYxr3XjlqyWEZ1AVczMXH1KbEX7Wq4YdpWoQfkPNSvthqbEbG3ROTaI b2pX1Nui3CVLtjAUUM> X-ME-Received: <xmr:jJ3_ZPzZVbyFTybKsjN_xsJtKn-uSA2fJxWjCThrexSQN1uSZ4ppvEh_Ifqhbh8> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeihedgudejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptgfkffggfgfuvfevfhfhjgesmhdtreertdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeehteekgfetieeujeeuvddtvdelteffleejteduvdefffejieehheeuteffveei jeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:jZ3_ZHOkmROmp37frTj1p2-GAPTOqZ6WJmZKL33YBCZ_lbiMzBOfGA> <xmx:jZ3_ZE8bcvLcf0NGmASesW_v96hVdjysdmPj9RRMasy1kUbfEl8I6g> <xmx:jZ3_ZPU22e7nNduE4jYv0YMXqIEpMHHAxTEvE_dP0TC6BvxIm-71sg> <xmx:jZ3_ZDamM7WPmYrAsk5T0TW1CkABaNfbgJ1HfcBSEV778uRyFXHS3Q> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 11 Sep 2023 19:06:51 -0400 (EDT) Content-Type: multipart/mixed; boundary="------------I3MK6f5qXVUuZUbk3Nf06WpH" Message-ID: <14b26ff0-9d9b-3600-bf6e-133db4031622@HIDDEN> Date: Tue, 12 Sep 2023 02:06:50 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> <83tts0rkh5.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83tts0rkh5.fsf@HIDDEN> X-Spam-Score: -2.2 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.2 (---) This is a multi-part message in MIME format. --------------I3MK6f5qXVUuZUbk3Nf06WpH Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 11/09/2023 14:57, Eli Zaretskii wrote: >> Date: Mon, 11 Sep 2023 03:02:55 +0300 >> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov <dmitry@HIDDEN> >> >>> You could record its value in a local variable at the entry to >>> garbage_collect, and the expose that value to Lisp. >> >> That also doesn't seem to give much, given that the condition for >> entering 'maybe_garbage_collect' is (consing_until_gc < 0). I.e. we wait >> until it's down to 0, then garbage-collect. > > No, we don't wait until it's zero, we perform GC on the first > opportunity that we _notice_ that it crossed zero. So examining how > negative is the value of consing_until_gc when GC is actually > performed could tell us whether we checked the threshold with high > enough frequency, and comparing these values between different runs > could tell us whether the shorter time spend in GC means really less > garbage or less frequent checks for the need to GC. Good point, I'm attaching the same outputs with "last value of consing_until_gc" added to every line. There are some pretty low values in the "read-process-output-max 409600" part of the experiment, which probably means runtime staying in C accumulating the output into the (now larger) buffer? Not sure. >> What's in there? First of all, for find-directory-files-recursively-3, >> there are 0 garbage collections between the beginning of the function >> and when we start parsing the output (no GCs while the process is >> writing to the buffer synchronously). I guess inserting output in a >> buffer doesn't increase consing, so there's nothing to GC? > > No, we just don't count increasing size of buffer text in the "consing > since GC" counter. Basically, buffer text is never "garbage", except > when a buffer is killed. That makes sense. Perhaps it hints at a faster design for calling process asynchronously as well (more on another experiment later). >> Next: for find-directory-files-recursively-2, the process only finishes >> at the end, when all GC cycles are done for. I suppose that also means >> we block the process's output while Lisp is running, and also that >> whatever GC events occur might coincide with the chunks of output coming >> from the process, and however many of them turn out to be in total. > > We don't block the process when GC runs. We do stop reading from the > process, so if and when the pipe fills, the OS will block the process. Right. But the effect is almost the same, including the potential side-effect that (IIUC) when a process is waiting like that, it's just suspended and not rushing ahead using the CPU/disk/etc resources to the max. That's an orthogonal train of thought, sorry. --------------I3MK6f5qXVUuZUbk3Nf06WpH Content-Type: text/plain; charset=UTF-8; name="gcs2b.txt" Content-Disposition: attachment; filename="gcs2b.txt" Content-Transfer-Encoding: base64 ZmluZC1kaXJlY3RvcnktZmlsZXMtcmVjdXJzaXZlbHktMgoKVXNlcyBtYWtlLXByb2Nlc3Mg YW5kIDpmaWx0ZXIgdG8gcGFyc2UgdGhlIG91dHB1dApjb25jdXJyZW50bHkgd2l0aCB0aGUg cHJvY2Vzcy4KCldpdGggKHJlYWQtcHJvY2Vzcy1vdXRwdXQtbWF4IDQwOTYpOgoKc3RhcnQg bm93CmN1Z2MgLTI1NjAgY291bnRzIGRpZmYgKDQyMzQ1IDMgMTQwOTcgMiA0MDg1ODMgMTk2 NiA0NDc5KQpjdWdjIC00NDQ5IGNvdW50cyBkaWZmICgyNDU5OSAxIDM0MiAwIDg4MzI0NyAw IDYyNjYpCmN1Z2MgLTEwMCBjb3VudHMgZGlmZiAoMjQwNzAgMSAzNTQgMCA5NzczODcgMCA2 MDA5KQpjdWdjIC0xMTYgY291bnRzIGRpZmYgKDI3MjY2IDEgMjc4IDAgOTQwNzIzIDAgNzQ4 NSkKY3VnYyAtOTUgY291bnRzIGRpZmYgKDI3NDg2IDEgMjcwIDAgMTAxNDU5MSAwIDc1ODYp CmN1Z2MgLTExNyBjb3VudHMgZGlmZiAoMjcxNTcgMSAyOTQgMCAxMTIxMDY1IDAgNzMyOSkK Y3VnYyAtMTQ2IGNvdW50cyBkaWZmICgyODIzMyAxIDMxNiAwIDExODU1MjcgMCA3NTYyKQpj dWdjIC0xNDMgY291bnRzIGRpZmYgKDMwNTk3IDEgMzU0IDAgMTIxNzMyMCAwIDgxNDcpCmN1 Z2MgLTQ4MDcgY291bnRzIGRpZmYgKDI1OTI1IDEgMzgwIDAgMTQ3NDYxOCAwIDY0MDcpCmN1 Z2MgLTEyNyBjb3VudHMgZGlmZiAoMzMzNDQgMSAzNjggMCAxMzQxNDUzIDAgODk2NSkKY3Vn YyAtMTc3IGNvdW50cyBkaWZmICgzNDc4NSAxIDQ3OCAwIDE0MzQ0MzIgMCA4ODQyKQpjdWdj IC0yODAxIGNvdW50cyBkaWZmICgzNzA2OSAxIDQ2NCAwIDE0Nzc4MjUgMCA5Njc1KQpjdWdj IC0yMyBjb3VudHMgZGlmZiAoNDA4MTcgMSA0NDggMCAxNDc4NDQ1IDAgMTA5OTkpCmN1Z2Mg LTEyMTUgY291bnRzIGRpZmYgKDQ0NTI2IDEgNTAwIDAgMTUwMzYwNCAwIDExOTY0KQpjdWdj IC00MTg5IGNvdW50cyBkaWZmICg0MjMwNSAxIDQ2OCAwIDE3MDE5ODkgMCAxMTM1NCkKY3Vn YyAtNDcxNSBjb3VudHMgZGlmZiAoMzY2NDQgMSA1MzIgMCAyMDM2Nzc4IDAgOTA4MikKY3Vn YyAtODUgY291bnRzIGRpZmYgKDM4MjM0IDEgNTQyIDAgMjEzMTc1NiAwIDk1MzUpCmN1Z2Mg LTg2MSBjb3VudHMgZGlmZiAoNDE2MzIgMSA1NzggMCAyMTg4MTg2IDAgMTA0NzQpCmN1Z2Mg LTExNyBjb3VudHMgZGlmZiAoNDYwMjkgMSA1ODAgMCAyMjExNjg1IDAgMTE5MjEpCmN1Z2Mg LTM4IGNvdW50cyBkaWZmICg1MDM1MyAxIDcyOCAwIDIyODAzODggMCAxMjU2OCkKY3VnYyAt MjUzNyBjb3VudHMgZGlmZiAoNTcxNjggMSA4ODggMCAyMjg2MzgxIDAgMTM5NzQpCmN1Z2Mg LTM2NzYgY291bnRzIGRpZmYgKDYxNTcwIDEgOTI0IDAgMjM0MTQwMiAwIDE1MjQ2KQpjdWdj IC0xNzQgY291bnRzIGRpZmYgKDU2NTA0IDEgOTI0IDAgMjY4OTMwMCAwIDEzNTAyKQpjdWdj IC0xMDAxIGNvdW50cyBkaWZmICg1NzA2NiAxIDg0MiAwIDI4NTUwMjggMCAxNDA5OCkKY3Vn YyAtMTQ2IGNvdW50cyBkaWZmICg1NzcxNiAxIDkxNiAwIDMwNjMyMzggMCAxMzg5MSkKY3Vn YyAtMTQ4IGNvdW50cyBkaWZmICg2Mjg2OCAxIDk4MiAwIDMxMzkxMTEgMCAxNTI0NCkKY3Vn YyAtMTczMCBjb3VudHMgZGlmZiAoNjQ4MDkgMSA4NTYgMCAzMjgzODU1IDAgMTY1MzUpCmN1 Z2MgLTE2MiBjb3VudHMgZGlmZiAoNjkxODMgMSA4NzAgMCAzMzk0MDMxIDAgMTc5MDIpCjxm aW5pc2hlZFxuPgp0b3RhbCBjaHVua3MgNjY1MgpFbGFwc2VkIHRpbWU6IDEuMjMzMDE2cyAo MC42Njg4MTlzIGluIDI4IEdDcykKCkFuZCBoZXJlJ3Mgd2l0aCAocmVhZC1wcm9jZXNzLW91 dHB1dC1tYXggNDA5NjAwKToKCnN0YXJ0IG5vdwpjdWdjIC0xMiBjb3VudHMgZGlmZiAoNTkx NjAgNSAyMjU0NyAxMTYgMTU1NDM0IDIwNDYgMjEwMykKY3VnYyAtMTU0MDAxIGNvdW50cyBk aWZmICgxODY3MSAxIDE2IDAgMTAzNDUzOCAwIDYxNzIpCmN1Z2MgLTEwMCBjb3VudHMgZGlm ZiAoMjAyNTAgMSAxNCAwIDEwMDM5NjYgMCA2NzA4KQpjdWdjIC0xOTAyOTQgY291bnRzIGRp ZmYgKDE5NjIzIDEgMTYgMCAxMjQ0NDQxIDAgNjQ4OSkKY3VnYyAtNTggY291bnRzIGRpZmYg KDI2MTYwIDEgMTQgMCAxMDE1MTI4IDAgODY3OCkKY3VnYyAtMjkzMDY3IGNvdW50cyBkaWZm ICgyMjczNyAxIDE2IDAgMTQyNjg3NCAwIDc1MjcpCmN1Z2MgLTkyIGNvdW50cyBkaWZmICgy ODMwOCAxIDE0IDAgMTE2MDIxMyAwIDkzOTQpCmN1Z2MgLTI1IGNvdW50cyBkaWZmICgyMTYy MCAxIDE2IDAgMTUzNTY4NiAwIDcxNTMpCmN1Z2MgLTIxIGNvdW50cyBkaWZmICgyMzI1MSAx IDE2IDAgMTU1NDcyMCAwIDc2OTgpCmN1Z2MgLTE0MyBjb3VudHMgZGlmZiAoMjk5ODggMSAx NiAwIDE0NjI2MzkgMCA5OTQzKQpjdWdjIC0xMTcgY291bnRzIGRpZmYgKDI4ODI3IDEgMTYg MCAxNjIyNTYyIDAgOTU1NikKY3VnYyAtMjYgY291bnRzIGRpZmYgKDMzOTU5IDEgMTYgMCAx NjA2ODE1IDAgMTEyNjYpCmN1Z2MgLTE3IGNvdW50cyBkaWZmICgzNzQ3NiAxIDE2IDAgMTYz OTg1MyAwIDEyNDM5KQpjdWdjIC0yNTA5OTIgY291bnRzIGRpZmYgKDMxMzQ1IDEgMTggMCAy MDgxNjYzIDAgMTAzODMpCmN1Z2MgLTI4OTE0MiBjb3VudHMgZGlmZiAoMjk5MDQgMSAxOCAw IDI0NDg0MTAgMCA5OTAxKQpjdWdjIC0yOTAyMjcgY291bnRzIGRpZmYgKDMwNjc1IDEgMTgg MCAyNDQ4MTU2IDAgMTAxNTkpCmN1Z2MgLTI2NDMxNSBjb3VudHMgZGlmZiAoMzU0MTggMSAx OCAwIDI0NDY1MDggMCAxMTc0MSkKY3VnYyAtMzIgY291bnRzIGRpZmYgKDQxNzQxIDEgMTgg MCAyMzQzOTAwIDAgMTM4NDcpCmN1Z2MgLTIyMDEgY291bnRzIGRpZmYgKDQ0NTIzIDEgMTEy IDAgMjQ3ODMxMCAwIDE0MjM5KQpjdWdjIC0xNTY3MyBjb3VudHMgZGlmZiAoNDk2MjIgMSAx NzAgMCAyNTI4MjIxIDAgMTU1OTIpCmN1Z2MgLTQwMjY3IGNvdW50cyBkaWZmICg0MTk5MCAx IDU4IDAgMjk3MjAxNSAwIDEzNjkzKQpjdWdjIC0xNTkgY291bnRzIGRpZmYgKDQxMDEwIDEg MjIgMCAzMTc3OTk0IDAgMTM1ODApCmN1Z2MgLTQyIGNvdW50cyBkaWZmICg0NzYwMiAxIDE1 NiAwIDMyNTk4MzMgMCAxNTAwOSkKY3VnYyAtMzU4ODg0IGNvdW50cyBkaWZmICg0Mzc0MCAx IDM0IDAgMzY4NzE0NSAwIDE0NDM2KQpjdWdjIC0yMiBjb3VudHMgZGlmZiAoNTU1OTggMSAy MCAwIDM0OTQxOTAgMCAxODQ1NCkKY3VnYyAtMTI3MCBjb3VudHMgZGlmZiAoNjAxMjggMSAx OTAgMCAzNjgzNDYxIDAgMTg5ODApCjxmaW5pc2hlZFxuPgp0b3RhbCBjaHVua3MgMjczCkVs YXBzZWQgdGltZTogMC45MzI2MjVzICgwLjYwODcxM3MgaW4gMjYgR0NzKQo= --------------I3MK6f5qXVUuZUbk3Nf06WpH Content-Type: text/plain; charset=UTF-8; name="gcs3b.txt" Content-Disposition: attachment; filename="gcs3b.txt" Content-Transfer-Encoding: base64 ZmluZC1kaXJlY3RvcnktZmlsZXMtcmVjdXJzaXZlbHktMwoKVXNlcyBwcm9jZXNzLWZpbGUs IHBhcnNlcyB0aGUgYnVmZmVyCndpdGggc2VhcmNoLWZvcndhcmQgYXQgdGhlIGVuZC4KCnN0 YXJ0IG5vdwo8cHJvY2VzcyBmaW5pc2hlZCwgbm93IHBhcnNpbmc+CmN1Z2MgLTEyOSBjb3Vu dHMgZGlmZiAoMTc2NjcgMSAxNTY1IDEgNzc5MDgxIDkzIDEwNDY1KQpjdWdjIC0xMzkgY291 bnRzIGRpZmYgKDEyNzg5IDEgMTIgMCA5MTIzNjQgMCAxMjY5NikKY3VnYyAtODQgY291bnRz IGRpZmYgKDEzNDk2IDEgMTIgMCAxMDI4MDYwIDAgMTM0MDMpCmN1Z2MgLTMzIGNvdW50cyBk aWZmICgxNDExMiAxIDEyIDAgMTE2ODUyMiAwIDE0MDE5KQpjdWdjIC0xNTMgY291bnRzIGRp ZmYgKDE0MzU0IDEgMTIgMCAxMzQ3MjQxIDAgMTQyNjEpCmN1Z2MgLTcyIGNvdW50cyBkaWZm ICgxNzAwNSAxIDEyIDAgMTQwMTA3NSAwIDE2OTEyKQpjdWdjIC0xNyBjb3VudHMgZGlmZiAo MjA4MTAgMSAxMiAwIDE0MDMzOTYgMCAyMDcxNykKY3VnYyAtOTQgY291bnRzIGRpZmYgKDE4 NTE2IDEgMTIgMCAxNzkyNTA4IDAgMTg0MjMpCmN1Z2MgLTEyMCBjb3VudHMgZGlmZiAoMTc5 ODEgMSAxMiAwIDIxMDg0NTggMCAxNzg4OCkKY3VnYyAtNTAgY291bnRzIGRpZmYgKDIyMDkw IDEgMTIgMCAyMTY5ODM1IDAgMjE5OTcpCmN1Z2MgLTEzNiBjb3VudHMgZGlmZiAoMjY3NDkg MSAxMiAwIDIyMzEwMzcgMCAyNjY1NikKY3VnYyAtOTMgY291bnRzIGRpZmYgKDI1MzAwIDEg MTIgMCAyNjg3ODQzIDAgMjUyMDcpCmN1Z2MgLTcyIGNvdW50cyBkaWZmICgyNjE2NSAxIDEy IDAgMzA0NjE0MCAwIDI2MDcyKQpjdWdjIC0xNDIgY291bnRzIGRpZmYgKDMwOTY4IDEgMTIg MCAzMjA1MzA2IDAgMzA4NzUpCkVsYXBzZWQgdGltZTogMC45MzgxODBzICgwLjMxNDYzMHMg aW4gMTQgR0NzKQo= --------------I3MK6f5qXVUuZUbk3Nf06WpH--
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 11 Sep 2023 11:57:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 11 07:57:42 2023 Received: from localhost ([127.0.0.1]:52273 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qffXx-0002UP-Ka for submit <at> debbugs.gnu.org; Mon, 11 Sep 2023 07:57:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43040) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qffXu-0002U9-Id for 64735 <at> debbugs.gnu.org; Mon, 11 Sep 2023 07:57:40 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qffXk-0004cx-5q; Mon, 11 Sep 2023 07:57:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=ApsTiErd2jgr2dnSg7gSl250EvRb98aN/jZKQ2h1of8=; b=UE7elJ/Jjkdl 5X2B2ZvL0opu+nsl9Zxwl0T9oEoO7fI8Ke9PraQkw2MWMruj8HSEAq/57pIT2bsGCww06y/r0s11U IK0BfXCG/nPX8A1QTg5et86IUg9NA10qFJlAXrYFKsMfi6/Vak92JQ5vjF1HAt0zk/rmBPdCNzUAK rYgFc5H/ElKrmO+9Hx90QxG9ZwnexQjMKkm/ip7jTupL3JTytlg36/hsRwBYhcgLWnfi878vgxr6L rf3vHs1S6CltmFkGQoPoRCP4HFKHXNDb24JIY+Y2uLU/362rQDxEfsyjYV23HLN4vV9vlN3tpX5v3 Xm+POh5xKWqI+onV57/DGg==; Date: Mon, 11 Sep 2023 14:57:10 +0300 Message-Id: <83tts0rkh5.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> (message from Dmitry Gutov on Mon, 11 Sep 2023 03:02:55 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> <8334zmtwwi.fsf@HIDDEN> <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Mon, 11 Sep 2023 03:02:55 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > > You could record its value in a local variable at the entry to > > garbage_collect, and the expose that value to Lisp. > > That also doesn't seem to give much, given that the condition for > entering 'maybe_garbage_collect' is (consing_until_gc < 0). I.e. we wait > until it's down to 0, then garbage-collect. No, we don't wait until it's zero, we perform GC on the first opportunity that we _notice_ that it crossed zero. So examining how negative is the value of consing_until_gc when GC is actually performed could tell us whether we checked the threshold with high enough frequency, and comparing these values between different runs could tell us whether the shorter time spend in GC means really less garbage or less frequent checks for the need to GC. > What's in there? First of all, for find-directory-files-recursively-3, > there are 0 garbage collections between the beginning of the function > and when we start parsing the output (no GCs while the process is > writing to the buffer synchronously). I guess inserting output in a > buffer doesn't increase consing, so there's nothing to GC? No, we just don't count increasing size of buffer text in the "consing since GC" counter. Basically, buffer text is never "garbage", except when a buffer is killed. > Next: for find-directory-files-recursively-2, the process only finishes > at the end, when all GC cycles are done for. I suppose that also means > we block the process's output while Lisp is running, and also that > whatever GC events occur might coincide with the chunks of output coming > from the process, and however many of them turn out to be in total. We don't block the process when GC runs. We do stop reading from the process, so if and when the pipe fills, the OS will block the process. > So there is also a second recording for > find-directory-files-recursively-2 with read-process-output-max=409600. > It does improve the performance significantly (and reduce the number of > GC pauses). I guess what I'm still not clear on, is whether the number > of GC pauses is fewer because of less consing (the only column that > looks significantly different is the 3rd: VECTOR-CELLS), or because the > process finishes faster due to larger buffers, which itself causes fewer > calls to maybe_gc. I think the latter.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 11 Sep 2023 00:03:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Sep 10 20:03:13 2023
Received: from localhost ([127.0.0.1]:51546 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qfUOW-0002Hz-L2
for submit <at> debbugs.gnu.org; Sun, 10 Sep 2023 20:03:13 -0400
Received: from out3-smtp.messagingengine.com ([66.111.4.27]:42585)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qfUOR-0002Hj-LQ
for 64735 <at> debbugs.gnu.org; Sun, 10 Sep 2023 20:03:10 -0400
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
by mailout.nyi.internal (Postfix) with ESMTP id 801EA5C0067;
Sun, 10 Sep 2023 20:02:58 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute6.internal (MEProxy); Sun, 10 Sep 2023 20:02:58 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-type:content-type:date:date:from:from:in-reply-to
:in-reply-to:message-id:mime-version:references:reply-to:sender
:subject:subject:to:to; s=fm3; t=1694390578; x=1694476978; bh=fL
aeDQZpvI2i8RppxU+bKtKe1kdbtfqOkUEZdEflxMk=; b=rwqJdtXVmdI+kt+I+H
7FZ2dGe68GkUjA5uywEoXFwC2T/skhew4EuuV77lH7JWwDVNlkBI40uLMCiwsJGo
hWz/iqLFaopwgaZxdAnnuuUezEBZMJtn8oUN0qMOd3RhgXHlw5xtv95MHnC8CqlE
RSVLvp7VahwfpHK+g5zh+ZxlZXXqi9r3u41OB8yrnQnQy3IiLqelQUGZ0shBjo3p
IwnjBxhLNEcc2+oWhWCu5l/4e4hvKD/InX2BlBpdgT+/y/alzu6J8vZ6kbKQUAmC
IXi9RZA7zDa6R5Fax7CY2lRVDKUBYQQCJdPPt3ZICwaC1+SYYrFyyy5axNZEWT6t
Ga0g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-type:content-type:date:date
:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
:message-id:mime-version:references:reply-to:sender:subject
:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
:x-sasl-enc; s=fm1; t=1694390578; x=1694476978; bh=fLaeDQZpvI2i8
RppxU+bKtKe1kdbtfqOkUEZdEflxMk=; b=JSys0Nq1YX2/n65YFOHX3m4xl3iQC
dT0AbL3ZKkfOpZSoJ64YyayLCHM3Z3InFlh9Pp7uOjtZVU0//4i/3f+cchL1JsoJ
1sf8YKdLn0fphwWkPK78oiY5YbH16K3dxeaj4AtlqtZYg695hjUym1VR+WjTwM0g
q0kvurBHieI/K2pa1SYuSvZyLwDsK9EV1PcUv14r71XK0Gwfx7cpE+8b/6dseMoH
UAkyEy5nsW/VxhWM2uEqvcW/iabH+ndXukVqXGoq9Ph7ULlXKGHuUDlSFxLm2HoQ
BUCVUjVg3xHCURJH8Th0zcO8gM4yHIxBug0mOb2n6RwdtBlqPv8f7fc/g==
X-ME-Sender: <xms:MVn-ZIHtGrSzxVxTNWrT1NrA-ZjN3bPUKfq7atRgdczeQINbzMn2kg>
<xme:MVn-ZBUiu_mT0UA4U2J1g6LhAyBilvW8dnNYnvJoC-M2dNozboAA6p1xSuYa_-oPS
o-HKV4ZzIC1c5ByDEI>
X-ME-Received: <xmr:MVn-ZCK5CA5-ZE7TZL1DHXAHqvexjUWNK78P6ev775XwZwiyzICePdmHxSBuNSU>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeifedgvdelucetufdoteggodetrfdotf
fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
cujfgurheptgfkffggfgfuvfevfhfhjgesmhdtreertdefjeenucfhrhhomhepffhmihht
rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
gvrhhnpeehteekgfetieeujeeuvddtvdelteffleejteduvdefffejieehheeuteffveei
jeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh
hithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:MVn-ZKHLRjx_Ibfc1W49BXSHEpVIjqWxrmX9afbGiBA7v9ev-IzXTQ>
<xmx:MVn-ZOUdx9f-sFdnFBja6aSOEzmnb8FUHnz3VIWGo2mJkHJ0MKUyvg>
<xmx:MVn-ZNMUpmq6yzvtGu8dhxE5nO8HbUouBWID5TgZQbqT_KDsDSS69A>
<xmx:Mln-ZNQhf9tPvYfzGSqkxlrJ5yDmFMYtO7DKOFLqi3PQmkNYAvAj7A>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun,
10 Sep 2023 20:02:56 -0400 (EDT)
Content-Type: multipart/mixed; boundary="------------mo9wiY0Ph8bca03YUu6cB50g"
Message-ID: <e480d2fb-0912-c9d6-66c8-d11e07482545@HIDDEN>
Date: Mon, 11 Sep 2023 03:02:55 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN>
<83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN>
<83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN>
<83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN>
<834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN>
<d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
<83bkfs2tw5.fsf@HIDDEN>
<18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN>
<83il8lxjcu.fsf@HIDDEN>
<2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN>
<8334zmtwwi.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <8334zmtwwi.fsf@HIDDEN>
X-Spam-Score: -2.2 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.2 (---)
This is a multi-part message in MIME format.
--------------mo9wiY0Ph8bca03YUu6cB50g
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
On 10/09/2023 08:33, Eli Zaretskii wrote:
>> Date: Sun, 10 Sep 2023 04:30:24 +0300
>> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
>> 64735 <at> debbugs.gnu.org
>> From: Dmitry Gutov <dmitry@HIDDEN>
>>
>> On 08/09/2023 09:35, Eli Zaretskii wrote:
>>> I think you'd need to expose consing_until_gc to Lisp, and then you
>>> can collect the data from Lisp.
>>
>> I can expose it to Lisp and print all three from post-gc-hook, but the
>> result just looks like this:
>>
>> gc-pct 0.1 gc-thr 800000 cugc 4611686018427387903
>>
>> Perhaps I need to add a hook which runs at the beginning of GC? Or of
>> maybe_gc even?
>
> You could record its value in a local variable at the entry to
> garbage_collect, and the expose that value to Lisp.
That also doesn't seem to give much, given that the condition for
entering 'maybe_garbage_collect' is (consing_until_gc < 0). I.e. we wait
until it's down to 0, then garbage-collect. What we could perhaps do is
add another hook (or a printer) at the beginning of maybe_gc, but either
would result in lots and lots of output.
>> Alternatively, (memory-use-counts) seems to retain some counters which
>> don't get erased during garbage collection.
>
> Maybe using those will be good enough, indeed.
I added this instrumentation:
(defvar last-mem-counts '(0 0 0 0 0 0 0))
(defun gc-record-after ()
(let* ((counts (memory-use-counts))
(diff (cl-map 'list
(lambda (old new) (- new old))
last-mem-counts counts)))
(setq last-mem-counts counts)
(message "counts diff %s" diff)))
(add-hook 'post-gc-hook #'gc-record-after)
so that after each garbage collection we print the differences in all
counters (CONSES FLOATS VECTOR-CELLS SYMBOLS STRING-CHARS INTERVALS
STRINGS).
And a message call when the process finishes.
And made those recordings during the benchmark runs of two different
listing methods (one using make-process, another using process-file) to
list all files in a large directory (there are ~200000 files there).
The make-process one I also ran with a different (large) value of
read-process-output-max. Results attached.
What's in there? First of all, for find-directory-files-recursively-3,
there are 0 garbage collections between the beginning of the function
and when we start parsing the output (no GCs while the process is
writing to the buffer synchronously). I guess inserting output in a
buffer doesn't increase consing, so there's nothing to GC?
Next: for find-directory-files-recursively-2, the process only finishes
at the end, when all GC cycles are done for. I suppose that also means
we block the process's output while Lisp is running, and also that
whatever GC events occur might coincide with the chunks of output coming
from the process, and however many of them turn out to be in total.
So there is also a second recording for
find-directory-files-recursively-2 with read-process-output-max=409600.
It does improve the performance significantly (and reduce the number of
GC pauses). I guess what I'm still not clear on, is whether the number
of GC pauses is fewer because of less consing (the only column that
looks significantly different is the 3rd: VECTOR-CELLS), or because the
process finishes faster due to larger buffers, which itself causes fewer
calls to maybe_gc.
And, of course, what else could be done to reduce the time spent in GC
in the asynchronous case.
--------------mo9wiY0Ph8bca03YUu6cB50g
Content-Type: text/plain; charset=UTF-8; name="gcs2.txt"
Content-Disposition: attachment; filename="gcs2.txt"
Content-Transfer-Encoding: base64
ZmluZC1kaXJlY3RvcnktZmlsZXMtcmVjdXJzaXZlbHktMgoKVXNlcyBtYWtlLXByb2Nlc3Mg
YW5kIDpmaWx0ZXIgdG8gcGFyc2UgdGhlIG91dHB1dApjb25jdXJyZW50bHkgd2l0aCB0aGUg
cHJvY2Vzcy4KCldpdGggKHJlYWQtcHJvY2Vzcy1vdXRwdXQtbWF4IDQwOTYpOgoKc3RhcnQg
bm93CmNvdW50cyBkaWZmICg3NTg0MCAxMyAzMTE3NyA2MCAzNDM0NDMgMzQ5NiA0NzQ4KQpj
b3VudHMgZGlmZiAoNDE5NDYgMSA0NjAgMCAxMjI2NDk0IDAgODQyNSkKY291bnRzIGRpZmYg
KDQzMTY1IDEgNDUwIDAgMTI4NDIxNCAwIDg5NTEpCmNvdW50cyBkaWZmICg0MzUxMyAxIDM2
NCAwIDEzNDMzMTYgMCAxMDEyNSkKY291bnRzIGRpZmYgKDQzMjAwIDEgMzg0IDAgMTQ3OTA0
OCAwIDk3NjYpCmNvdW50cyBkaWZmICg0NjIyMCAxIDQyOCAwIDE1Mjg4NjMgMCAxMDI0MikK
Y291bnRzIGRpZmYgKDQzMTI1IDEgNDYyIDAgMTc2NzA2OCAwIDg3OTApCmNvdW50cyBkaWZm
ICg0OTExOCAxIDQ1OCAwIDE3MjMyNzEgMCAxMDgzMikKY291bnRzIGRpZmYgKDUzMTU2IDEg
NTcyIDAgMTc4OTkxOSAwIDEwNzc0KQpjb3VudHMgZGlmZiAoNTc3NTUgMSA1NDggMCAxNzgz
Mjg2IDAgMTI2MDApCmNvdW50cyBkaWZmICg2MjE3MSAxIDU1NCAwIDE3OTUyMTYgMCAxMzk5
NSkKY291bnRzIGRpZmYgKDYyMDIwIDEgNTUwIDAgMTk2MzI1NSAwIDEzOTk2KQpjb3VudHMg
ZGlmZiAoNTQ1NTkgMSA2MTYgMCAyMzg3MzA4IDAgMTA3MDApCmNvdW50cyBkaWZmICg1NjQy
OCAxIDYzNCAwIDI1MTMyMTkgMCAxMTA5NSkKY291bnRzIGRpZmYgKDYyNjExIDEgNjU4IDAg
MjUxMDc1NiAwIDEyODY0KQpjb3VudHMgZGlmZiAoNjc1NjAgMSA3MDggMCAyNTc0MzEyIDAg
MTM4OTkpCmNvdW50cyBkaWZmICg3ODE1NCAxIDkyOCAwIDI1NzIyNzMgMCAxNDcxNCkKY291
bnRzIGRpZmYgKDg2Nzk0IDEgOTc2IDAgMjUyMDkxNSAwIDE3MDA0KQpjb3VudHMgZGlmZiAo
NzgxMTIgMSA4NzQgMCAyOTQzNTQ4IDAgMTUzNjcpCmNvdW50cyBkaWZmICg3OTQ0MyAxIDg5
NCAwIDMxMzg5NDggMCAxNTU1OSkKY291bnRzIGRpZmYgKDgxODYxIDEgOTg0IDAgMzM0Mzc2
NCAwIDE1MjYwKQpjb3VudHMgZGlmZiAoODc3MjQgMSAxMDMwIDAgMzQzMDk2OSAwIDE2NjUw
KQpjb3VudHMgZGlmZiAoODg1MzIgMSA5MDIgMCAzNTkxMDUyIDAgMTg0ODcpCmNvdW50cyBk
aWZmICg5MjA4MyAxIDk1MiAwIDM3NjkyOTAgMCAxOTA2NSkKPGZpbmlzaGVkXG4+CkVsYXBz
ZWQgdGltZTogMS4zNDQ0MjJzICgwLjc0NzEyNnMgaW4gMjQgR0NzKQoKQW5kIGhlcmUncyB3
aXRoIChyZWFkLXByb2Nlc3Mtb3V0cHV0LW1heCA0MDk2MDApOgoKc3RhcnQgbm93CmNvdW50
cyBkaWZmICg1Nzk2NyAxIDQwNDAgMSA5ODE5MTIgMTA2IDc3MzEpCmNvdW50cyBkaWZmICgz
MjA3NSAxIDIwIDAgMTkxOTA5NiAwIDEwNTYwKQpjb3VudHMgZGlmZiAoNDM0MzEgMSAxOCAw
IDIyNTkzMTQgMCAxNDM3MSkKY291bnRzIGRpZmYgKDQ2MzM1IDEgMTggMCAyNDI2MjkwIDAg
MTUzMzkpCmNvdW50cyBkaWZmICgzMTg3MiAxIDE4IDAgMjQ0NzYzOSAwIDEwNTE4KQpjb3Vu
dHMgZGlmZiAoNDY1MjcgMSAxOCAwIDIzMjgwNDIgMCAxNTQwMykKY291bnRzIGRpZmYgKDQy
NDY4IDEgMTggMCAyMDk5OTc2IDAgMTQwNTApCmNvdW50cyBkaWZmICg0ODY0OCAxIDE4IDAg
MjMwMjcxMyAwIDE2MTEwKQpjb3VudHMgZGlmZiAoNTA0MDQgMSAyMCAwIDMyNjA5MjEgMCAx
NjY2OSkKY291bnRzIGRpZmYgKDQwMTQ3IDEgMjAgMCAzMjY0NDYzIDAgMTMyNTEpCmNvdW50
cyBkaWZmICg0ODExOCAxIDIwIDAgMzI2MTcyNSAwIDE1OTA4KQpjb3VudHMgZGlmZiAoNjA3
MzIgMSAyODIgMCAyNzkxMDAzIDAgMTY3ODUpCmNvdW50cyBkaWZmICg3MTMyOSAxIDUwNiAw
IDI3NjIyMzcgMCAxNzQ4NykKY291bnRzIGRpZmYgKDYxNDU1IDEgMzQyIDAgMzE5Mjc3MSAw
IDE2MjcxKQpjb3VudHMgZGlmZiAoNDkwMzUgMSAzMCAwIDM2NjM3MTUgMCAxNjA4NSkKY291
bnRzIGRpZmYgKDU4NjUxIDEgMjM2IDAgMzc4Mzg4OCAwIDE2NjgzKQpjb3VudHMgZGlmZiAo
NTcxMzIgMSAyNCAwIDQ1NTc2ODggMCAxODg2MikKY291bnRzIGRpZmYgKDcxMzE5IDEgMjQg
MCA0NzY5ODkxIDAgMjM1OTEpCjxmaW5pc2hlZFxuPgpFbGFwc2VkIHRpbWU6IDAuODkwNzEw
cyAoMC41NDY0ODZzIGluIDE4IEdDcykK
--------------mo9wiY0Ph8bca03YUu6cB50g
Content-Type: text/plain; charset=UTF-8; name="gcs3.txt"
Content-Disposition: attachment; filename="gcs3.txt"
Content-Transfer-Encoding: base64
ZmluZC1kaXJlY3RvcnktZmlsZXMtcmVjdXJzaXZlbHktMwoKVXNlcyBwcm9jZXNzLWZpbGUs
IHBhcnNlcyB0aGUgYnVmZmVyCndpdGggc2VhcmNoLWZvcndhcmQgYXQgdGhlIGVuZC4KCnN0
YXJ0IG5vdwo8cHJvY2VzcyBmaW5pc2hlZCwgbm93IHBhcnNpbmc+CmNvdW50cyBkaWZmICg2
Mjc3MSA1IDI2NjI5IDYzIDQ1ODIxMSAzMjIzIDgwMzgpCmNvdW50cyBkaWZmICgxNzA0NSAx
IDEyIDAgMTI4ODE1MyAwIDE2OTQ5KQpjb3VudHMgZGlmZiAoMTgzMDEgMSAxMiAwIDE0MzIx
NjUgMCAxODIwNSkKY291bnRzIGRpZmYgKDE3NjQzIDEgMTIgMCAxNzE2Mjk0IDAgMTc1NDcp
CmNvdW50cyBkaWZmICgyMTkxNyAxIDEyIDAgMTcyNjQ2MiAwIDIxODIxKQpjb3VudHMgZGlm
ZiAoMjU4ODggMSAxMiAwIDE3NzczNzEgMCAyNTc5MikKY291bnRzIGRpZmYgKDIxNzQzIDEg
MTIgMCAyMzQ1MTQzIDAgMjE2NDcpCmNvdW50cyBkaWZmICgyNDAzNSAxIDEyIDAgMjU2MTQ5
MSAwIDIzOTM5KQpjb3VudHMgZGlmZiAoMzAwMjggMSAxMiAwIDI1OTMwNjkgMCAyOTkzMikK
Y291bnRzIGRpZmYgKDI5NjI3IDEgMTIgMCAzMDQxMzA3IDAgMjk1MzEpCmNvdW50cyBkaWZm
ICgzMDE0MCAxIDEyIDAgMzQ3OTIwOSAwIDMwMDQ0KQpjb3VudHMgZGlmZiAoMzUxODEgMSAx
MiAwIDM2OTA0ODAgMCAzNTA4NSkKRWxhcHNlZCB0aW1lOiAwLjk0MzA5MHMgKDAuMzUxNzk5
cyBpbiAxMiBHQ3MpCg==
--------------mo9wiY0Ph8bca03YUu6cB50g--
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 10 Sep 2023 05:34:02 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Sep 10 01:34:02 2023 Received: from localhost ([127.0.0.1]:48783 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qfD58-0000qZ-7o for submit <at> debbugs.gnu.org; Sun, 10 Sep 2023 01:34:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40588) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qfD54-0000q4-T3 for 64735 <at> debbugs.gnu.org; Sun, 10 Sep 2023 01:34:00 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qfD4v-0003Nz-IN; Sun, 10 Sep 2023 01:33:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=B5RnSeg5gnFEVm4o4/zT3TKzHS+gtBC+WlH+nPO64a4=; b=blgjFS5FZIEW 7eUO9KoUjpQc3PmNMTBxDD8JJ/KLoDbeOtZFFp7w8tTe8Jjp60SVEYzep9W7KSVr4PBLvFqABlGrJ AabZbT/zZti8SUDxV4i8Fn7TiTsKAHqp/yHI4HhLCSZ0f8rj0k2ECD7TUc6nT0Me+vVWokIzGOtlr sjAOGANg7KkPvJGs01L90ZST18x7Tgqt8tmDC0eHQYI+3oVMFJhPyw9FhHkRKLCrPHJDrvZ1zfZlw mEPVvkVhrhSoCtIwCXywPtFIqvzUWbCfjG2/QMRPzfCNVWYig25f61nF8AWiIDQI7Srf/tBRUvO03 P5qYrRHngC8GBhTthom6sA==; Date: Sun, 10 Sep 2023 08:33:33 +0300 Message-Id: <8334zmtwwi.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> (message from Dmitry Gutov on Sun, 10 Sep 2023 04:30:24 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Sun, 10 Sep 2023 04:30:24 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > On 08/09/2023 09:35, Eli Zaretskii wrote: > > I think you'd need to expose consing_until_gc to Lisp, and then you > > can collect the data from Lisp. > > I can expose it to Lisp and print all three from post-gc-hook, but the > result just looks like this: > > gc-pct 0.1 gc-thr 800000 cugc 4611686018427387903 > > Perhaps I need to add a hook which runs at the beginning of GC? Or of > maybe_gc even? You could record its value in a local variable at the entry to garbage_collect, and the expose that value to Lisp. > Alternatively, (memory-use-counts) seems to retain some counters which > don't get erased during garbage collection. Maybe using those will be good enough, indeed. > And as for MS Windows, IIRC all find-based solution are currently slow > equally, so we're unlikely to make things worse there anyway. I was actually thinking about *BSD and macOS.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 10 Sep 2023 01:30:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Sep 09 21:30:42 2023 Received: from localhost ([127.0.0.1]:48660 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qf9He-00036P-18 for submit <at> debbugs.gnu.org; Sat, 09 Sep 2023 21:30:42 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:59513) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qf9HZ-000368-Ah for 64735 <at> debbugs.gnu.org; Sat, 09 Sep 2023 21:30:40 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id F2DF1320085B; Sat, 9 Sep 2023 21:30:27 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Sat, 09 Sep 2023 21:30:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1694309427; x=1694395827; bh=cLJDhPyxPDrxRcQ1y2gfepKp3soinuXDF3w olOn9Ax0=; b=dXKbGqIif99K2SUjRssTAJom3y/Z7qJCu97wj5bhSEmeYU+yPwf SP//ePN5HoqD19JVHzF5oLlZWv76EIyVZnibE2YjzdC9VoIgVCmN5qgoCzAbI+8s g2LQ1Iqt8OwxPD20jtX8BytDRMOoehGBni5rlHv537pNpec3xdS+C8VStNJIEoFw 3a8qcMr5i6YMHHuuKxYgbTceOS5KBzOvaaFaMSi6aRU7hrWp9b/tjhTtex4cDkmL +7Q7OG9x35k5N4HmhigwtyyeOOn5Uox5efqJFt03HMSXAaiMN/C9ilTyF2eSDNV6 JLZDffQDVYgY9b+bwmKmksEx8hG4Cr+JvZg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1694309427; x=1694395827; bh=cLJDhPyxPDrxRcQ1y2gfepKp3soinuXDF3w olOn9Ax0=; b=zPc36bjldH1Cdk9++EGP3pQBlrrpqxFJgsLaOg6JQ0K7gxLqPTh Eexs6XWkpyM5KnHnWpLyko9t/e3v6kQcJE/rruG/QnfumIPjKFX/Z7+jmyg3gZhz CDDbM4+ThysQAB7e2JjLgbejWQom1nN7TeMCRn/f3wLf6FwLdA1saFb01RlK32qv aQI8xzdEiFePMv1ksMO6H/mWljeJyYcx5cHroJUd3BcP9H+RUUzYZQ0YVrTC/5Jf 644n7VPHgtiqsw4eLKvAITtaE8WKmUS7eKL+TApMKnZiZbo+nfGUirEarmaeWl6k efl1//x84fFn8vbEmUgDSUF8693dLxCAXIQ== X-ME-Sender: <xms:Mxz9ZNWQbHo8zh5cllnb-ykHDAbRNMzBePwpHJPHHGPoxehSTPqQzw> <xme:Mxz9ZNkkqyy9DOQt_7dCXOfsj6A6rRXB7p9WOYyog1cyJjLHN_gxxxhtHBvEtsaYb MxFygpGfdJcLT3Bc2M> X-ME-Received: <xmr:Mxz9ZJb-ZeW_qa9JTe2i_sYLvYf3KSwSMJ9qwZaI0JeqITw20IcXX-dVewfPvcM> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeitddggeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephfeutdekveeggeetteekfeejffegudduudfhueevleeftdffffeggeeivddv jeelnecuffhomhgrihhnpehgnhhurdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenuc frrghrrghmpehmrghilhhfrhhomhepughmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:Mxz9ZAWWU1pWa0AEAaEd-6qczIjOTMJ8m70kIbUrw1AwoHOW3-22tg> <xmx:Mxz9ZHkiP22_7zvxhnYy3OBmZitfOHM5eRnE5IMeMZnEcXe-kBpxMQ> <xmx:Mxz9ZNf_THOAB0dVGbNcya4v3WXEwYGwsU1ckIW05ZnfyNUu72g39g> <xmx:Mxz9ZFjpjhjyFzXL97GzhwMc7CSqSfGv_HKQ5QesJqG7ncgYj6b9xQ> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 9 Sep 2023 21:30:25 -0400 (EDT) Message-ID: <2e21ec81-8e4f-4c02-ea15-43bd6da3daa7@HIDDEN> Date: Sun, 10 Sep 2023 04:30:24 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> <83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> <83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> <83il8lxjcu.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83il8lxjcu.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.2 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.2 (---) On 08/09/2023 09:35, Eli Zaretskii wrote: >>> These questions can only be answered by dumping the values of the 2 GC >>> thresholds and of consing_until_gc for each GC cycle. It could be >>> that we are consing more Lisp memory, or it could be that one of the >>> implementations provides fewer opportunities for Emacs to call >>> maybe_gc. Or it could be some combination of the two. >> Do you think the outputs of >> https://elpa.gnu.org/packages/emacs-gc-stats.html could help? > I think you'd need to expose consing_until_gc to Lisp, and then you > can collect the data from Lisp. I can expose it to Lisp and print all three from post-gc-hook, but the result just looks like this: gc-pct 0.1 gc-thr 800000 cugc 4611686018427387903 Perhaps I need to add a hook which runs at the beginning of GC? Or of maybe_gc even? Alternatively, (memory-use-counts) seems to retain some counters which don't get erased during garbage collection. >> All examples which use make-process call it with :connection-type 'pipe. >> >> The one that calls process-file (the "synchronous" impl) also probably >> does, but I don't see that in the docstring. > Yes, call-process uses pipes. So finding the optimum boils down to > running various scenarios. It is also possible that the optimum will > be different on different systems, btw. Sure, but I'd like to improve the state of affairs in at least the main one. And as for MS Windows, IIRC all find-based solution are currently slow equally, so we're unlikely to make things worse there anyway.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 8 Sep 2023 06:36:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 08 02:36:24 2023
Received: from localhost ([127.0.0.1]:42030 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qeV6N-00035q-HJ
for submit <at> debbugs.gnu.org; Fri, 08 Sep 2023 02:36:24 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:37808)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qeV6J-00035X-AY
for 64735 <at> debbugs.gnu.org; Fri, 08 Sep 2023 02:36:21 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qeV6B-0007SI-7z; Fri, 08 Sep 2023 02:36:11 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=rlsXkYvfXeSdQet6A3eWOCfH2P3C49eUVv3ZL7mxz/0=; b=Szbrau3yHD2L
OmDsJZmh7ebyYqOrRUkQN1zunnAZdgkkSD0Vejw15meINIrNwb8LWHP4gk0Kzl1UKqbeWoUMd5Q1J
0Hc7Gqwi3rh0ya0pBUNBj8wxMx3dxVPwZOxFPIZprvDw5NtgekBhFrm/rxAzpLlvlhtPhuwMJ+oJv
K3cJkYiXjPFi7Bjfn39awRHBobOhIOvIF/2Xl7ybo9+W1b8IWDbGwAZgW1/Vuzj6WExmPf9SdvoC5
wmo7oZcTTXK7PBBIgW1rmjiQEJ0wRP6McchSFnfUp72U6js7Jo7whu4cBOT74DBMhZy7VHVNWGXsq
+hJ4CQP09McT/kSujXSxRw==;
Date: Fri, 08 Sep 2023 09:35:45 +0300
Message-Id: <83il8lxjcu.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN> (message from
Dmitry Gutov on Fri, 8 Sep 2023 03:53:37 +0300)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN>
<83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN>
<83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN>
<83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN>
<834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN>
<d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
<83bkfs2tw5.fsf@HIDDEN> <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Date: Fri, 8 Sep 2023 03:53:37 +0300
> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
> 64735 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry@HIDDEN>
>
> >> (("with-find 4096" . "Elapsed time: 1.737742s (1.019624s in 28 GCs)")
> >> ("with-find 40960" . "Elapsed time: 1.515376s (0.942906s in 26 GCs)")
> >> ("with-find 409600" . "Elapsed time: 1.458987s (0.948857s in 26 GCs)")
> >> ("with-find 1048576" . "Elapsed time: 1.363882s (0.888599s in 24 GCs)")
> >> ("with-find-p 4096" . "Elapsed time: 1.202522s (0.745758s in 19 GCs)")
> >> ("with-find-p 40960" . "Elapsed time: 1.005221s (0.640815s in 16 GCs)")
> >> ("with-find-p 409600" . "Elapsed time: 0.855483s (0.591208s in 15 GCs)")
> >> ("with-find-p 1048576". "Elapsed time: 0.825936s (0.623876s in 16 GCs)")
> >> ("with-find-sync 4096" . "Elapsed time: 0.848059s (0.272570s in 7 GCs)")
> >> ("with-find-sync 409600"."Elapsed time: 0.912932s (0.339230s in 9 GCs)")
> >> ("with-find-sync 1048576"."Elapsed time: 0.880479s (0.303047s in 8 GCs)"
> >> ))
> >>
> >> What was puzzling for me, overall, is that if we take "with-find 409600"
> >> (the fastest among the asynchronous runs without parallelism) and
> >> "with-find-sync", the difference in GC time (which is repeatable),
> >> 0.66s, almost covers all the difference in performance. And as for
> >> "with-find-p 409600", it would come out on top! Which it did in Ihor's
> >> tests when GC was disabled.
> >>
> >> But where does the extra GC time come from? Is it from extra consing in
> >> the asynchronous call's case? If it is, it's not from all the chunked
> >> strings, apparently, given that increasing max string's size (and
> >> decreasing their number by 2x-6x, according to my logging) doesn't
> >> affect the reported GC time much.
> >>
> >> Could the extra time spent in GC just come from the fact that it's given
> >> more opportunities to run, maybe? call_process stays entirely in C,
> >> whereas make-process, with its asynchronous approach, goes between C and
> >> Lisp even time it receives input. The report above might indicate so:
> >> with-find-p have ~20 garbage collection cycles, whereas with-find-sync -
> >> only ~10. Or could there be some other source of consing, unrelated to
> >> the process output string, and how finely they are sliced?
> >
> > These questions can only be answered by dumping the values of the 2 GC
> > thresholds and of consing_until_gc for each GC cycle. It could be
> > that we are consing more Lisp memory, or it could be that one of the
> > implementations provides fewer opportunities for Emacs to call
> > maybe_gc. Or it could be some combination of the two.
>
> Do you think the outputs of
> https://elpa.gnu.org/packages/emacs-gc-stats.html could help?
I think you'd need to expose consing_until_gc to Lisp, and then you
can collect the data from Lisp.
> Otherwise, I suppose I need to add some fprintf's somewhere. Would the
> beginning of maybe_gc inside lisp.h be a good place for that?
I can only recommend the fprintf method if doing this from Lisp is
impossible for some reason.
> >> If we get back to increasing read-process-output-max, which does help
> >> (apparently due to reducing the number we switch between reading from
> >> the process and doing... whatever else), the sweet spot seems to be
> >> 1048576, which is my system's maximum value. Anything higher - and the
> >> perf goes back to worse -- I'm guessing something somewhere resets the
> >> value to default? Not sure why it doesn't clip to the maximum allowed,
> >> though.
> >>
> >> Anyway, it would be helpful to be able to decide on as high as possible
> >> value without manually reading from /proc/sys/fs/pipe-max-size. And what
> >> of other OSes?
> >
> > Is this with pipes or with PTYs?
>
> All examples which use make-process call it with :connection-type 'pipe.
>
> The one that calls process-file (the "synchronous" impl) also probably
> does, but I don't see that in the docstring.
Yes, call-process uses pipes. So finding the optimum boils down to
running various scenarios. It is also possible that the optimum will
be different on different systems, btw.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 8 Sep 2023 00:53:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 07 20:53:51 2023
Received: from localhost ([127.0.0.1]:41775 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qePks-0002XM-Vx
for submit <at> debbugs.gnu.org; Thu, 07 Sep 2023 20:53:51 -0400
Received: from out4-smtp.messagingengine.com ([66.111.4.28]:60991)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qePkq-0002X6-DE
for 64735 <at> debbugs.gnu.org; Thu, 07 Sep 2023 20:53:49 -0400
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
by mailout.nyi.internal (Postfix) with ESMTP id 43CA95C01AE;
Thu, 7 Sep 2023 20:53:41 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute6.internal (MEProxy); Thu, 07 Sep 2023 20:53:41 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm3; t=
1694134421; x=1694220821; bh=vEWUKj+DkYzcQEJei0wqPj0y5tPhO96g1TN
Ff1lUa+k=; b=Zq4KNiWFWfKpHCq0oQfGGLYra1GUWY1LWvxjEleQk0PWG9bBJ3Z
0++jG6rPKIga1a0VK/0A/zUJ+Yyeg+aJLb7lnz2kBDEh9xE5zheHKBDMjj7ZGJt1
x/9ALLYeSxU58IQGAQw1aj2PGcmutmNRP/4somdRBh6q2rDSy9q3AIA0K9P2m88m
Xb8aCEVLcVBxUTsb5SctrqDQQiSHcE3wEhfbxDdXmoUGTF+d/LkcIl0Hb5/Czb8y
BxAF8p0sQla6ru1qjlNdJX6QITjRLSW6YPwBuVU2BkF4Df1nCZw1GtLz/RAPOnuO
tsp6EefpGeZpyASn/akoRXRO8E8k4ZWM2Yg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
1694134421; x=1694220821; bh=vEWUKj+DkYzcQEJei0wqPj0y5tPhO96g1TN
Ff1lUa+k=; b=hefmK20OOl0Bamt4J7mgijNrbm9/k7bGllggAAQZgM8kHmuphcQ
gSrvu94LCQ7seAyMKtut3rmZZFz+XgTh59uNqQLRoXfwMz5YZAWfW/otKzIADleG
qc8WhGK7EWZdCDGghT0RBPQvTYDRkbxwflfRt3w6D8ndQkfmUmR8nUROaURQ5vur
w2IESNe7VDPaHyc2lFsEPyBjuWrwaIGMFXtwAqWYQikhg2AzmMIbet4mYGibI2uz
2wEn4ixy9hWJXAS3/igV65j7sT4wa/rjsB5l1R9+BG/ugWK35WOfHgBh9jj1uoq9
/D6f37BkiVvF60EJ1HhqL/6xGv5+slL5c8g==
X-ME-Sender: <xms:lHD6ZEsB-JEzmEYCxPpMsMFfktoRWxbmMjdoMwFFdveWYKH5iZDYng>
<xme:lHD6ZBcL3Hoe8c8vc5TON4tkXqwVLU9OmG7HcmKgNiv94Sskwo44Gof65m2cdQOGT
uGuX66-AvqfCgeQjLs>
X-ME-Received: <xmr:lHD6ZPy6OySssv8t2NR5HWN4tuoT-ui_RsYgUdB4-KP88DKJRsXj_FUEnePk7sk>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehiedggeduucetufdoteggodetrfdotf
fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi
thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth
htvghrnhephfeutdekveeggeetteekfeejffegudduudfhueevleeftdffffeggeeivddv
jeelnecuffhomhgrihhnpehgnhhurdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenuc
frrghrrghmpehmrghilhhfrhhomhepughmihhtrhihsehguhhtohhvrdguvghv
X-ME-Proxy: <xmx:lHD6ZHNxoEQsRZl6yXdv5Al60kdfHzn8itIvwzFI-EkB5C4kS6yH6g>
<xmx:lHD6ZE9kheqsOOqY3O_jLqAwKbq5zRuawC138YUYH-FQGoMgqRUoVg>
<xmx:lHD6ZPVLpS7uNaWRVlmkaJ8SLYr8Z-splI3DwCJlk2cKFyDG-bCAyQ>
<xmx:lXD6ZDZbjbitJg-pKKQxuqApwiL9PbWFumloX3XQf9CKAbdmgavadA>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
7 Sep 2023 20:53:39 -0400 (EDT)
Message-ID: <18a0b4d8-32bd-3ecd-8db4-32608a1ebba7@HIDDEN>
Date: Fri, 8 Sep 2023 03:53:37 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN>
<83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN>
<83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN>
<83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN>
<834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN>
<d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
<83bkfs2tw5.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <83bkfs2tw5.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.2 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.2 (---)
Let's try to investigate this some more, if we can.
On 31/07/2023 14:38, Eli Zaretskii wrote:
>> Date: Sun, 30 Jul 2023 04:35:49 +0300
>> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
>> 64735 <at> debbugs.gnu.org
>> From: Dmitry Gutov <dmitry@HIDDEN>
>>
>> In this context, junks objects are objects that will need to be
>> collected by garbage collector very soon because they are just a
>> byproduct of a function's execution (but aren't used in the return
>> value, for example). The more of them a function creates, the more work
>> it will be, supposedly, for the GC.
>>
>> Heap is perhaps the wrong term (given that C has its own notion of
>> heap), but I meant the memory managed by the Lisp runtime.
>>
>> And chunks are the buffered strings that get passed to the process
>> filter. Chunks of the process' output. By default, these chunks are 4096
>> characters long, but the comparisons tweak that value by 10x and 100x.
>
> If the subprocess output is inserted into a buffer, its effect on the
> GC will be different. (Not sure if this is relevant to the issue at
> hand, as I lost track of the many variants of the function that were
> presented.)
Yes, one of the variants inserts into the buffer (one that uses a
synchronous process call and also, coincidentally, spends less time in
GC), and the asynchronous work from a process filter.
>>> If I read what you wrote superficially, without delving into the
>>> details (which I can't understand), you are saying that the overall
>>> amount of consing is roughly the same.
>>
>> What is "amount of consing"? Is it just the number of objects? Or does
>> their size (e.g. string length) affect GC pressure as well?
>
> In general, both, since we have 2 GC thresholds, and GC is actually
> done when both are exceeded. So the effect will also depend on how
> much Lisp memory is already allocated in the Emacs process where these
> benchmarks are run.
All right.
>>> This is consistent with the
>>> fact that the GC times change only very little. So I don't think I
>>> see, on this level, what puzzles you in this picture.
>>
>> Now that you pointed that out, the picture is just more puzzling. While
>> 0.1s in GC is not insignificant (it's 10% of the whole runtime), it does
>> seem to have been more of a fluke, and on average the fluctuations in GC
>> time are smaller.
>>
>> Here's an extended comparison:
>>
>> (("with-find 4096" . "Elapsed time: 1.737742s (1.019624s in 28 GCs)")
>> ("with-find 40960" . "Elapsed time: 1.515376s (0.942906s in 26 GCs)")
>> ("with-find 409600" . "Elapsed time: 1.458987s (0.948857s in 26 GCs)")
>> ("with-find 1048576" . "Elapsed time: 1.363882s (0.888599s in 24 GCs)")
>> ("with-find-p 4096" . "Elapsed time: 1.202522s (0.745758s in 19 GCs)")
>> ("with-find-p 40960" . "Elapsed time: 1.005221s (0.640815s in 16 GCs)")
>> ("with-find-p 409600" . "Elapsed time: 0.855483s (0.591208s in 15 GCs)")
>> ("with-find-p 1048576". "Elapsed time: 0.825936s (0.623876s in 16 GCs)")
>> ("with-find-sync 4096" . "Elapsed time: 0.848059s (0.272570s in 7 GCs)")
>> ("with-find-sync 409600"."Elapsed time: 0.912932s (0.339230s in 9 GCs)")
>> ("with-find-sync 1048576"."Elapsed time: 0.880479s (0.303047s in 8 GCs)"
>> ))
>>
>> What was puzzling for me, overall, is that if we take "with-find 409600"
>> (the fastest among the asynchronous runs without parallelism) and
>> "with-find-sync", the difference in GC time (which is repeatable),
>> 0.66s, almost covers all the difference in performance. And as for
>> "with-find-p 409600", it would come out on top! Which it did in Ihor's
>> tests when GC was disabled.
>>
>> But where does the extra GC time come from? Is it from extra consing in
>> the asynchronous call's case? If it is, it's not from all the chunked
>> strings, apparently, given that increasing max string's size (and
>> decreasing their number by 2x-6x, according to my logging) doesn't
>> affect the reported GC time much.
>>
>> Could the extra time spent in GC just come from the fact that it's given
>> more opportunities to run, maybe? call_process stays entirely in C,
>> whereas make-process, with its asynchronous approach, goes between C and
>> Lisp even time it receives input. The report above might indicate so:
>> with-find-p have ~20 garbage collection cycles, whereas with-find-sync -
>> only ~10. Or could there be some other source of consing, unrelated to
>> the process output string, and how finely they are sliced?
>
> These questions can only be answered by dumping the values of the 2 GC
> thresholds and of consing_until_gc for each GC cycle. It could be
> that we are consing more Lisp memory, or it could be that one of the
> implementations provides fewer opportunities for Emacs to call
> maybe_gc. Or it could be some combination of the two.
Do you think the outputs of
https://elpa.gnu.org/packages/emacs-gc-stats.html could help?
Otherwise, I suppose I need to add some fprintf's somewhere. Would the
beginning of maybe_gc inside lisp.h be a good place for that?
>> If we get back to increasing read-process-output-max, which does help
>> (apparently due to reducing the number we switch between reading from
>> the process and doing... whatever else), the sweet spot seems to be
>> 1048576, which is my system's maximum value. Anything higher - and the
>> perf goes back to worse -- I'm guessing something somewhere resets the
>> value to default? Not sure why it doesn't clip to the maximum allowed,
>> though.
>>
>> Anyway, it would be helpful to be able to decide on as high as possible
>> value without manually reading from /proc/sys/fs/pipe-max-size. And what
>> of other OSes?
>
> Is this with pipes or with PTYs?
All examples which use make-process call it with :connection-type 'pipe.
The one that calls process-file (the "synchronous" impl) also probably
does, but I don't see that in the docstring.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 31 Jul 2023 11:38:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 31 07:38:28 2023
Received: from localhost ([127.0.0.1]:51935 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qQREJ-0002cL-P6
for submit <at> debbugs.gnu.org; Mon, 31 Jul 2023 07:38:28 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:36488)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qQREH-0002c9-8I
for 64735 <at> debbugs.gnu.org; Mon, 31 Jul 2023 07:38:26 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qQREA-00043j-MV; Mon, 31 Jul 2023 07:38:18 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=e3t9o9cqVCQ4loUKEOavC7d3QlIDyiAJPPlZ64FunRM=; b=Zo7XCgOthxeh
1Y3ZfXbGPvOttiM/m1fvCB0mlxT7PzMZiKXvcochspr+2Xf8Os2EYzlFQB5P6qnkHLlDDGbk3uT5L
xlmGdMlPAHI+1aF/taopwAFYSJ1xpCj6GiPj7bTEtLatFNSYaJtMuSvBcETsxI61whgE6HGoJdu0X
1DkWfF5JsJ+7/smra/QfsXZShxC0w0vjiuhe1TwUIdY4pxZ8ubr815O/O2uXjF/kSZoFwLkbhAjm+
Gd9RDwYlRjs6zX8d8suECn+ncsjMNaPj6AqWoYSfeIWh48Ii3sbt/F+Y1h6Tt7VI23drkb9yevu5w
gC13C55q1IeVGLFSUcIz9g==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qQRE9-0006ZH-UE; Mon, 31 Jul 2023 07:38:18 -0400
Date: Mon, 31 Jul 2023 14:38:18 +0300
Message-Id: <83bkfs2tw5.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN> (message from
Dmitry Gutov on Sun, 30 Jul 2023 04:35:49 +0300)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN>
<83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN>
<83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN>
<83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN>
<834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN> <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Date: Sun, 30 Jul 2023 04:35:49 +0300
> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
> 64735 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry@HIDDEN>
>
> In this context, junks objects are objects that will need to be
> collected by garbage collector very soon because they are just a
> byproduct of a function's execution (but aren't used in the return
> value, for example). The more of them a function creates, the more work
> it will be, supposedly, for the GC.
>
> Heap is perhaps the wrong term (given that C has its own notion of
> heap), but I meant the memory managed by the Lisp runtime.
>
> And chunks are the buffered strings that get passed to the process
> filter. Chunks of the process' output. By default, these chunks are 4096
> characters long, but the comparisons tweak that value by 10x and 100x.
If the subprocess output is inserted into a buffer, its effect on the
GC will be different. (Not sure if this is relevant to the issue at
hand, as I lost track of the many variants of the function that were
presented.)
> > If I read what you wrote superficially, without delving into the
> > details (which I can't understand), you are saying that the overall
> > amount of consing is roughly the same.
>
> What is "amount of consing"? Is it just the number of objects? Or does
> their size (e.g. string length) affect GC pressure as well?
In general, both, since we have 2 GC thresholds, and GC is actually
done when both are exceeded. So the effect will also depend on how
much Lisp memory is already allocated in the Emacs process where these
benchmarks are run.
> > This is consistent with the
> > fact that the GC times change only very little. So I don't think I
> > see, on this level, what puzzles you in this picture.
>
> Now that you pointed that out, the picture is just more puzzling. While
> 0.1s in GC is not insignificant (it's 10% of the whole runtime), it does
> seem to have been more of a fluke, and on average the fluctuations in GC
> time are smaller.
>
> Here's an extended comparison:
>
> (("with-find 4096" . "Elapsed time: 1.737742s (1.019624s in 28 GCs)")
> ("with-find 40960" . "Elapsed time: 1.515376s (0.942906s in 26 GCs)")
> ("with-find 409600" . "Elapsed time: 1.458987s (0.948857s in 26 GCs)")
> ("with-find 1048576" . "Elapsed time: 1.363882s (0.888599s in 24 GCs)")
> ("with-find-p 4096" . "Elapsed time: 1.202522s (0.745758s in 19 GCs)")
> ("with-find-p 40960" . "Elapsed time: 1.005221s (0.640815s in 16 GCs)")
> ("with-find-p 409600" . "Elapsed time: 0.855483s (0.591208s in 15 GCs)")
> ("with-find-p 1048576". "Elapsed time: 0.825936s (0.623876s in 16 GCs)")
> ("with-find-sync 4096" . "Elapsed time: 0.848059s (0.272570s in 7 GCs)")
> ("with-find-sync 409600"."Elapsed time: 0.912932s (0.339230s in 9 GCs)")
> ("with-find-sync 1048576"."Elapsed time: 0.880479s (0.303047s in 8 GCs)"
> ))
>
> What was puzzling for me, overall, is that if we take "with-find 409600"
> (the fastest among the asynchronous runs without parallelism) and
> "with-find-sync", the difference in GC time (which is repeatable),
> 0.66s, almost covers all the difference in performance. And as for
> "with-find-p 409600", it would come out on top! Which it did in Ihor's
> tests when GC was disabled.
>
> But where does the extra GC time come from? Is it from extra consing in
> the asynchronous call's case? If it is, it's not from all the chunked
> strings, apparently, given that increasing max string's size (and
> decreasing their number by 2x-6x, according to my logging) doesn't
> affect the reported GC time much.
>
> Could the extra time spent in GC just come from the fact that it's given
> more opportunities to run, maybe? call_process stays entirely in C,
> whereas make-process, with its asynchronous approach, goes between C and
> Lisp even time it receives input. The report above might indicate so:
> with-find-p have ~20 garbage collection cycles, whereas with-find-sync -
> only ~10. Or could there be some other source of consing, unrelated to
> the process output string, and how finely they are sliced?
These questions can only be answered by dumping the values of the 2 GC
thresholds and of consing_until_gc for each GC cycle. It could be
that we are consing more Lisp memory, or it could be that one of the
implementations provides fewer opportunities for Emacs to call
maybe_gc. Or it could be some combination of the two.
> If we get back to increasing read-process-output-max, which does help
> (apparently due to reducing the number we switch between reading from
> the process and doing... whatever else), the sweet spot seems to be
> 1048576, which is my system's maximum value. Anything higher - and the
> perf goes back to worse -- I'm guessing something somewhere resets the
> value to default? Not sure why it doesn't clip to the maximum allowed,
> though.
>
> Anyway, it would be helpful to be able to decide on as high as possible
> value without manually reading from /proc/sys/fs/pipe-max-size. And what
> of other OSes?
Is this with pipes or with PTYs?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 30 Jul 2023 01:36:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 29 21:36:02 2023
Received: from localhost ([127.0.0.1]:49110 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qPvLl-0001DU-Kb
for submit <at> debbugs.gnu.org; Sat, 29 Jul 2023 21:36:02 -0400
Received: from out2-smtp.messagingengine.com ([66.111.4.26]:44143)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qPvLi-0001D9-Mf
for 64735 <at> debbugs.gnu.org; Sat, 29 Jul 2023 21:35:59 -0400
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
by mailout.nyi.internal (Postfix) with ESMTP id 747DF5C009A;
Sat, 29 Jul 2023 21:35:53 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute3.internal (MEProxy); Sat, 29 Jul 2023 21:35:53 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm2; t=
1690680953; x=1690767353; bh=YgxU7i11/XCn0vDsz5JhY3gGnIHKv6/VfGQ
6I0trznE=; b=VvKi7r4z/9L5coIKMDCVXvr2+rVOq3hH1Fa/5+L/9blEMwDS9b6
nkv5yZBBufAn1t1kNY9NiNzkGFKXcaQAEz9YuMpbAWg+vpt+9IEdYXTdXQv3ALv5
Z7xM1M4b5h9PTeqf2i88/hfTIKh1XCxpx/fDLEy/vltlfIu8cnU7F++BzRi2aE25
tBr/p8BzhcPUUOpDi4ZxEcyLUPHRcW2e1b2iRvWLkZCjQzou5HRmFtOzzP3PdRyR
ed58C+TlvctmE/MKX2R21dZjHfCguUdLFj01ZAhehrOtGthJnQ0xns6jek8QwuvN
65i4j2PZWkKmXvkIKjeCbx+AXa07SnC6oZA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=
1690680953; x=1690767353; bh=YgxU7i11/XCn0vDsz5JhY3gGnIHKv6/VfGQ
6I0trznE=; b=yn7ctk9vTJeIXvLKYh7VfMfLSWHGPu9Wu/NqkvziZuacn2bGchO
LoL236dS7gzG295xQiGCmLugTK7yiII4yvlw5RQe1o2Rx7/d9k5DdcpBSzmCN9zX
U7HZjlws/vOwDsW/mGw5nSzd7jEtYIPayxJ6oe0rJ7ZCdsdt5TtEXY/YKeKBPd1J
n+G8BPtU19ecm8o4XpCJWhcbWG8wUVBoDTKZEx9JDS8rlr69f9W48pje2vuk/yQB
zvXlPkBw0k85gA6C5IDRArWQYb2f5K/sAqcpnBhgHUOG/sVElaB/UP5t/iein7wF
utEk1IcQuG+1F58No+BfSKzAb1mSd27nJNg==
X-ME-Sender: <xms:eb7FZDtbJO4O1H1bh7rU4icrFzvrJ6jLb-2d76aK5_LsN0n1Dgq6Cg>
<xme:eb7FZEf2y5sJxKotlRUWgJ4y_KybVcVn7SenGnjzq0JYPLbRF-0in6LiCOAilHWbY
jJ8KtTlv18MA42PiS0>
X-ME-Received: <xmr:eb7FZGz-7ydrGwB2eg5MkZgFAFWzaHzaRJoxh1FyCzwR9g1spYP9FdYkhRTUe2g>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrieelgdegjecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht
rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv
veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh
hithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:eb7FZCPINxi_N4pebyZyQ0kQ6i6P3jsJEHwxpaoc7ohsw8XMXjE09w>
<xmx:eb7FZD9tF4TfPAJBFGYD45CRzUqwVipC1Uj88qbjWGFK2jyCUCrszw>
<xmx:eb7FZCW6GiORjrPvx_nlV61l17kI17u7tJIOphN7QgTlN8Dk5a4Nqg>
<xmx:eb7FZGZfEOjI2W3kiyQ6ltuA-DcETIaCk6YG_MirtPgTCeFmwSvptA>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat,
29 Jul 2023 21:35:51 -0400 (EDT)
Message-ID: <d319bd54-3f0d-a205-3412-ef3961074416@HIDDEN>
Date: Sun, 30 Jul 2023 04:35:49 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN>
<83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN>
<83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN>
<83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN>
<834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
<83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
<59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
<83pm4bi6qa.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <83pm4bi6qa.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.8 (/)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)
On 29/07/2023 09:15, Eli Zaretskii wrote:
>> Date: Sat, 29 Jul 2023 03:12:34 +0300
>> From: Dmitry Gutov <dmitry@HIDDEN>
>> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
>> 64735 <at> debbugs.gnu.org
>>
>> It seems like per-chunk overhead is non-trivial, and affects GC somehow
>> (but not in a way that just any string would).
>>
>> In this test, by default, the output produces ~6000 strings and passes
>> them to the filter function. Meaning, read_and_dispose_of_process_output
>> is called about 6000 times, producing the overhead of roughly 0.2s.
>> Something in there must be producing extra work for the GC.
>>
>> This line seems suspect:
>>
>> list3 (outstream, make_lisp_proc (p), text),
>>
>> Creates 3 conses and one Lisp object (tagged pointer). But maybe I'm
>> missing something bigger.
>
> I don't understand what puzzles you here. You need to make your
> descriptions more clear to allow others to follow your logic. You use
> terms you never explain: "junk objects", "number of strings in the
> heap", "per-chunk overhead" (what is "chunk"?), which is a no-no when
> explaining complex technical stuff to others.
In this context, junks objects are objects that will need to be
collected by garbage collector very soon because they are just a
byproduct of a function's execution (but aren't used in the return
value, for example). The more of them a function creates, the more work
it will be, supposedly, for the GC.
Heap is perhaps the wrong term (given that C has its own notion of
heap), but I meant the memory managed by the Lisp runtime.
And chunks are the buffered strings that get passed to the process
filter. Chunks of the process' output. By default, these chunks are 4096
characters long, but the comparisons tweak that value by 10x and 100x.
> If I read what you wrote superficially, without delving into the
> details (which I can't understand), you are saying that the overall
> amount of consing is roughly the same.
What is "amount of consing"? Is it just the number of objects? Or does
their size (e.g. string length) affect GC pressure as well?
> This is consistent with the
> fact that the GC times change only very little. So I don't think I
> see, on this level, what puzzles you in this picture.
Now that you pointed that out, the picture is just more puzzling. While
0.1s in GC is not insignificant (it's 10% of the whole runtime), it does
seem to have been more of a fluke, and on average the fluctuations in GC
time are smaller.
Here's an extended comparison:
(("with-find 4096" . "Elapsed time: 1.737742s (1.019624s in 28 GCs)")
("with-find 40960" . "Elapsed time: 1.515376s (0.942906s in 26 GCs)")
("with-find 409600" . "Elapsed time: 1.458987s (0.948857s in 26 GCs)")
("with-find 1048576" . "Elapsed time: 1.363882s (0.888599s in 24 GCs)")
("with-find-p 4096" . "Elapsed time: 1.202522s (0.745758s in 19 GCs)")
("with-find-p 40960" . "Elapsed time: 1.005221s (0.640815s in 16 GCs)")
("with-find-p 409600" . "Elapsed time: 0.855483s (0.591208s in 15 GCs)")
("with-find-p 1048576". "Elapsed time: 0.825936s (0.623876s in 16 GCs)")
("with-find-sync 4096" . "Elapsed time: 0.848059s (0.272570s in 7 GCs)")
("with-find-sync 409600"."Elapsed time: 0.912932s (0.339230s in 9 GCs)")
("with-find-sync 1048576"."Elapsed time: 0.880479s (0.303047s in 8 GCs)"
))
What was puzzling for me, overall, is that if we take "with-find 409600"
(the fastest among the asynchronous runs without parallelism) and
"with-find-sync", the difference in GC time (which is repeatable),
0.66s, almost covers all the difference in performance. And as for
"with-find-p 409600", it would come out on top! Which it did in Ihor's
tests when GC was disabled.
But where does the extra GC time come from? Is it from extra consing in
the asynchronous call's case? If it is, it's not from all the chunked
strings, apparently, given that increasing max string's size (and
decreasing their number by 2x-6x, according to my logging) doesn't
affect the reported GC time much.
Could the extra time spent in GC just come from the fact that it's given
more opportunities to run, maybe? call_process stays entirely in C,
whereas make-process, with its asynchronous approach, goes between C and
Lisp even time it receives input. The report above might indicate so:
with-find-p have ~20 garbage collection cycles, whereas with-find-sync -
only ~10. Or could there be some other source of consing, unrelated to
the process output string, and how finely they are sliced?
Changing process-adaptive-read-buffering to nil didn't have any effect here.
If we get back to increasing read-process-output-max, which does help
(apparently due to reducing the number we switch between reading from
the process and doing... whatever else), the sweet spot seems to be
1048576, which is my system's maximum value. Anything higher - and the
perf goes back to worse -- I'm guessing something somewhere resets the
value to default? Not sure why it doesn't clip to the maximum allowed,
though.
Anyway, it would be helpful to be able to decide on as high as possible
value without manually reading from /proc/sys/fs/pipe-max-size. And what
of other OSes?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 29 Jul 2023 06:14:34 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 29 02:14:34 2023 Received: from localhost ([127.0.0.1]:46377 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qPdDl-0007p5-Ob for submit <at> debbugs.gnu.org; Sat, 29 Jul 2023 02:14:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60452) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qPdDj-0007oq-Vj for 64735 <at> debbugs.gnu.org; Sat, 29 Jul 2023 02:14:33 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qPdDd-0007PE-2o; Sat, 29 Jul 2023 02:14:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=SKQqTmH/Rt0eZcq836DttsASeUiyw+hd9kw/kzzepCc=; b=lnDPcJx+vkIw 8evJXHSkO1q+WakRkdOut/NfjSUy86218Oe6ht9sV3kj/fDDfXT7juSWS6ouCSZwptNICQOR+nUiu wYSpB/pvw89mMhE2uF/NJL3vhrw3V98p+U4PU5Umyn0zeK/iXSI4lgOR/ByuygAdNrt60s76XjLGe bpBvot4DPRadVWx/V7j+imWO4GTfj5xq6AmnGtt14YOdHZ4au96WBHhZ2xPdoIZIuRZCjaZtV2hjl mi44EUNGAqo0NbiXHpEl6ByxOnNEkYBSndo2IZV6YGpHJXQaXiQZ0Pr4oU0ozC4MERtwmGLORDtfp vr6PFpidv/iH9N1OVocF7g==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qPdDS-0006vw-Qs; Sat, 29 Jul 2023 02:14:18 -0400 Date: Sat, 29 Jul 2023 09:15:09 +0300 Message-Id: <83pm4bi6qa.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> (message from Dmitry Gutov on Sat, 29 Jul 2023 03:12:34 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN> <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN> <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Sat, 29 Jul 2023 03:12:34 +0300 > From: Dmitry Gutov <dmitry@HIDDEN> > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > > It seems like per-chunk overhead is non-trivial, and affects GC somehow > (but not in a way that just any string would). > > In this test, by default, the output produces ~6000 strings and passes > them to the filter function. Meaning, read_and_dispose_of_process_output > is called about 6000 times, producing the overhead of roughly 0.2s. > Something in there must be producing extra work for the GC. > > This line seems suspect: > > list3 (outstream, make_lisp_proc (p), text), > > Creates 3 conses and one Lisp object (tagged pointer). But maybe I'm > missing something bigger. I don't understand what puzzles you here. You need to make your descriptions more clear to allow others to follow your logic. You use terms you never explain: "junk objects", "number of strings in the heap", "per-chunk overhead" (what is "chunk"?), which is a no-no when explaining complex technical stuff to others. If I read what you wrote superficially, without delving into the details (which I can't understand), you are saying that the overall amount of consing is roughly the same. This is consistent with the fact that the GC times change only very little. So I don't think I see, on this level, what puzzles you in this picture.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 29 Jul 2023 00:12:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 28 20:12:47 2023
Received: from localhost ([127.0.0.1]:46293 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qPXZe-0005ym-V9
for submit <at> debbugs.gnu.org; Fri, 28 Jul 2023 20:12:47 -0400
Received: from out3-smtp.messagingengine.com ([66.111.4.27]:60679)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qPXZc-0005yZ-LE
for 64735 <at> debbugs.gnu.org; Fri, 28 Jul 2023 20:12:45 -0400
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
by mailout.nyi.internal (Postfix) with ESMTP id 7F2A75C00CF;
Fri, 28 Jul 2023 20:12:39 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute1.internal (MEProxy); Fri, 28 Jul 2023 20:12:39 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm2; t=
1690589559; x=1690675959; bh=kDAy/oC8PTT3tibtcSafhP6m7ELCJpuOacX
162MYb58=; b=ZuGxXyaG6Dv3YzuwVPbxRtuxdzofsnn8W9cyhGWh7RwvLVJw+/A
2+n5nly/4Ag/VclJoEU1uf3X9EAE7OKci6WiQJ65wqdS3/QN2ZDWZgGXdjKEb7LZ
KYaL0I6nVZKzNdiNSrYRiJtXw6ZhVybN+hwmT4ssrC0sPOJ1hYDGyRjEma5B6QE1
Gq9LD20rxk1FYlLUCLAEAsiQUwA4UrsY6vAVmI4hjx4VbpPCfv9ZNZx1w+is/9Hr
aDm45+QAoD2WEpWVFigpmvWvrtnfi60gb0x1PONcLW4mnwJcWHEqPLj1+FXvlT/l
OovEHp9F8jgrvMb7nkiZTDTEoLTI0c7WfgA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=
1690589559; x=1690675959; bh=kDAy/oC8PTT3tibtcSafhP6m7ELCJpuOacX
162MYb58=; b=NzWC8Lh00Qrblh9/livnDpRMs50pFyYQXlKBGtE9JzmrdG8NO6I
O/GB3oULeR9JybJiYXn+dxof/SuJLpe1dawD3SWrIBV844HbbV7og4OPBXQrU2jl
hdE+Rtqudm1a7b63wSGD2JsK38r2amFdqXsfAp11t0nejw5FOmpNYtiwwAWh2op2
rRdLcjiaZNXJBa6Ne5aXNtVHnIaS7srX0eobHoNlDS+BOs2dnjFc7g0Cxl5CA2Gz
XzwEPFbu7uTYpHTh1Cp2tWS28uo6QZ6DuiFP98iBwsiuQ6GV5uQZe3JvqPc13HLv
4NURigtoSQ1+2urLFBL8WkB15WDWt6ojyfw==
X-ME-Sender: <xms:dlnEZCHkll446RUyZgP323ZlD5OYgELky48fXDZqPUThcdEIPz1y9g>
<xme:dlnEZDXklgu24TyMF4Olx_9cc7oF8X7ANqYRlnh1eqlKtQf-P7ohjzQHfOwXC_YTx
CI-Qhk_R7BRi5hJRfY>
X-ME-Received: <xmr:dlnEZMIgg84QFHMAK6YhNvu27zk6bRGQHR0MdNyD1012pZ89GTYt9o_RKU6bkhw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrieejgdefudcutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpefkffggfgfuhffvvehfjggtgfesthejredttdefjeenucfhrhhomhepffhmihht
rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
gvrhhnpeejgeeivdefgffhudetueevjefgfeelleetvddtfeevkeehveffueeuhfehjeff
hfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh
hithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:dlnEZMGq_nCBj17H4vI4kRvRywiXOJu9yUsWcaLzhCFCRRyoBo6M5g>
<xmx:dlnEZIXeB-7K0yCJEyVW67x0roURLa4BZuPHkeeETKkQqyuFUktYww>
<xmx:dlnEZPPYx2Qc6zNBYn5-66LwZJaHXzUCWbyHCUTOp1k3ePxZ-VLrpg>
<xmx:d1nEZPRJSUuROj1prQLzfwlYGTAsS6jfCMj6qN1InRjCFi8gJ1-MyA>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
28 Jul 2023 20:12:36 -0400 (EDT)
Message-ID: <59c30342-a7e0-d83b-a128-0faae4cbd633@HIDDEN>
Date: Sat, 29 Jul 2023 03:12:34 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
From: Dmitry Gutov <dmitry@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN>
<35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
In-Reply-To: <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.8 (/)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)
On 27/07/2023 16:30, Dmitry Gutov wrote:
> I can imagine that the filter-based approach necessarily creates more
> strings (to pass to the filter function). Maybe we could increase those
> strings' size (thus reducing the number) by increasing the read buffer
> size?
To go further along this route, first of all, I verified that the input
strings are (almost) all the same length: 4096. And they are parsed into
strings with length 50-100 characters, meaning the number of "junk"
objects due to the process-filter approach probably shouldn't matter too
much, given that the number of strings returned is 40-80x more.
But then I ran these tests with different values of
read-process-output-max, which exactly increased those strings' size,
proportionally reducing their number. The results were:
> (my-bench-rpom 1 default-directory "")
=>
(("with-find-p 4096" . "Elapsed time: 0.945478s (0.474680s in 6 GCs)")
("with-find-p 40960" . "Elapsed time: 0.760727s (0.395379s in 5 GCs)")
("with-find-p 409600" . "Elapsed time: 0.729757s (0.394881s in 5 GCs)"))
where
(defun my-bench-rpom (count path regexp)
(setq path (expand-file-name path))
(list
(cons "with-find-p 4096"
(let ((read-process-output-max 4096))
(benchmark count (list 'find-directory-files-recursively-2
path regexp))))
(cons "with-find-p 40960"
(let ((read-process-output-max 40960))
(benchmark count (list 'find-directory-files-recursively-2
path regexp))))
(cons "with-find-p 409600"
(let ((read-process-output-max 409600))
(benchmark count (list 'find-directory-files-recursively-2
path regexp))))))
...with the last iteration showing consistently the same or better
performance than the "sync" version I benchmarked previously.
What does that mean for us? The number of strings in the heap is
reduced, but not by much (again, the result is a list with 43x more
elements). The combined memory taken up by these intermediate strings to
be garbage-collected, is the same.
It seems like per-chunk overhead is non-trivial, and affects GC somehow
(but not in a way that just any string would).
In this test, by default, the output produces ~6000 strings and passes
them to the filter function. Meaning, read_and_dispose_of_process_output
is called about 6000 times, producing the overhead of roughly 0.2s.
Something in there must be producing extra work for the GC.
This line seems suspect:
list3 (outstream, make_lisp_proc (p), text),
Creates 3 conses and one Lisp object (tagged pointer). But maybe I'm
missing something bigger.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 27 Jul 2023 13:31:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 27 09:31:11 2023
Received: from localhost ([127.0.0.1]:40957 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qP15D-0003zO-85
for submit <at> debbugs.gnu.org; Thu, 27 Jul 2023 09:31:11 -0400
Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:47203)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qP15A-0003zA-54
for 64735 <at> debbugs.gnu.org; Thu, 27 Jul 2023 09:31:10 -0400
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
by mailout.west.internal (Postfix) with ESMTP id 4C05D32008FD;
Thu, 27 Jul 2023 09:31:01 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute3.internal (MEProxy); Thu, 27 Jul 2023 09:31:01 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm2; t=
1690464660; x=1690551060; bh=waeVfQIki+HZTOaa0uly1FtCecdYCc2KDOT
pFnDrxV4=; b=Fu6XNhQYD8DQZv2Rrc53t+xhHmsXy/dr72czO3ZrOQ9qoSHxcbv
vlMcZQLk65YhehhBjarDnat/Z7JexZRcTpW31QW/SxxutizG4H/uMQmWaKOKkx04
23wpgkEErDNp0/UN7o9UHDxgnWYtVAP2ZkDFKtdwd24Jje/akV7iecL9FJ+ARWFE
ozU46BWyNDGnnVN9QdgnXNR0cry+0uRdsF2r98kk+gXCTE0AVmHqM/raiJoXlonA
Bm48FY6uA2ec16hsbSq+NcoswvB8szNjcvCNf3dtY1IKARxkbbb8ThG1YJQuoerr
NST6E0kqgJxLG8MzAIhcTHVoEDE3zIbfcNA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=
1690464660; x=1690551060; bh=waeVfQIki+HZTOaa0uly1FtCecdYCc2KDOT
pFnDrxV4=; b=uhtOiyAutyrsDZ0qrAkGgXec43QBKknbbyaLiddqmDkiVaep+By
b/HzlNt09C5buv3E5ejuaSTxEpPLjb2kO3xigbif2AQuz7/nxM2GlR8V6PkknGvf
wNjRHCTL7IdkLSMCJJFMZamXN0gyStlE2epEv0/eGwvoinFhgiBBOrVgN464gllF
/7kU5zs/lqMr+zuQDZHfqyUzLEwv1wSGIS/LbKeFfhjoE3lKVRLF64NONoBGneWX
qTyZ4koCyZ7c9Vqe+WdYvBK35qSsTzqcVSAcofH6FJzDJtJ1jJRwXot/DAwTcoxK
UymW93WD+zRhIAGC86H8kX+8mkkZt9xdxaQ==
X-ME-Sender: <xms:lHHCZFvV3vnfhQDhMW3kU8XVj_-DfGeTQvGveFSeL8TnNjRjSd9AlQ>
<xme:lHHCZOe1anl5OdT5p9spdsjhinQaAd9TqoHot_heb7NDpYasVuI2GmVFzvDjyU3jV
X6or2bBcYbBg0wbi9s>
X-ME-Received: <xmr:lHHCZIxVMYxmLpjfVH1HPG_3-tI21orEYwWOsxTNgpjN4EU0XjXhN0oSIOhU_4Q>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrieeggdeflecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht
rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv
veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh
hithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:lHHCZMOg0I2Sb5JJhhedW5LzwY33bMZuqUZQioBgLHHP4IY7LmiBVQ>
<xmx:lHHCZF9pS1xY1h8aOdEldaJaDSo_E8GZvxEx-FujDAEHBbxqy4NqCg>
<xmx:lHHCZMXTQROsZcirsy-4GjbT_zv8atkCEA2ecFTa0qG7A67lixJg3A>
<xmx:lHHCZAYNJToI0_1foWtsJcx21ZYMPmF3l8Nb-ZDePtef-S5NxvmOZQ>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu,
27 Jul 2023 09:30:58 -0400 (EDT)
Message-ID: <35f8b664-0241-9f96-1aa0-20ca51b2d34c@HIDDEN>
Date: Thu, 27 Jul 2023 16:30:56 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <83v8e6lyi4.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.8 (/)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)
On 27/07/2023 08:22, Eli Zaretskii wrote:
>> Date: Thu, 27 Jul 2023 03:41:29 +0300
>> Cc: Eli Zaretskii <eliz@HIDDEN>, luangruo@HIDDEN, sbaugh@HIDDEN,
>> 64735 <at> debbugs.gnu.org
>> From: Dmitry Gutov <dmitry@HIDDEN>
>>
>>> I have modified `directory-files-recursively' to avoid O(N^2) `nconc'
>>> calls + bypassing regexp matches when REGEXP is nil.
>>
>> Sounds good. I haven't examined the diff closely, but it sounds like an
>> improvement that can be applied irrespective of how this discussion ends.
>
> That change should be submitted as a separate issue and discussed in
> detail before we decide we can make it.
Sure.
>>> If we forget about GC, Elisp version can get fairly close to GNU find.
>>> And if we do not perform regexp matching (which makes sense when the
>>> REGEXP is ""), Elisp version is faster.
>>
>> We can't really forget about GC, though.
>
> But we could temporarily lift the threshold while this function runs,
> if that leads to significant savings.
I mean, everything's doable, but if we do this for this function, why
not others? Most long-running code would see an improvement from that
kind of change (the 'find'-based solutions too).
IIRC the main drawback is running out of memory in extreme conditions or
on low-memory platforms/devices. It's not like this feature is
particularly protected from this.
>> But the above numbers make me hopeful about the async-parallel solution,
>> implying that the parallelization really can help (and offset whatever
>> latency we lose on pselect), as soon as we determine the source of extra
>> consing and decide what to do about it.
>
> Isn't it clear that additional consing comes from the fact that we
> first insert the Find's output into a buffer or produce a string from
> it, and then chop that into individual file names?
But we do that in all 'find'-based solutions: the synchronous one takes
buffer text and chops it into strings. The first asynchronous does the
same. The other ("with-find-p") works from a process filter, chopping up
strings that get passed to it.
But the amount of time spent in GC is different, with most of the
difference in performance attributable to it: if we subtract time spent
in GC, the runtimes are approximately equal.
I can imagine that the filter-based approach necessarily creates more
strings (to pass to the filter function). Maybe we could increase those
strings' size (thus reducing the number) by increasing the read buffer
size? I haven't found a relevant variable, though.
Or if there was some other callback that runs after the next chunk of
output arrives from the process, we could parse it from the buffer. But
the insertion into the buffer would need to be made efficient
(apparently internal-default-process-filter currently uses the same
sequence of strings as the other filters for input, with the same amount
of consing).
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 27 Jul 2023 09:28:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 27 05:28:09 2023
Received: from localhost ([127.0.0.1]:40761 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOxI0-0003Ay-MX
for submit <at> debbugs.gnu.org; Thu, 27 Jul 2023 05:28:09 -0400
Received: from mout01.posteo.de ([185.67.36.65]:34351)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qOxHy-0003AT-ED
for 64735 <at> debbugs.gnu.org; Thu, 27 Jul 2023 05:28:07 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout01.posteo.de (Postfix) with ESMTPS id 5357B240028
for <64735 <at> debbugs.gnu.org>; Thu, 27 Jul 2023 11:27:59 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1690450080; bh=Ytd8VeSAvDPI0pZncAHkopW37IDlkzXaSTuFZBDLBSA=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=UZIc4HoR+pQI+aMOoSRzZ9ejnXSvibXMSBwegGokQiXpH6jx0/ghEu73ZGnnainhK
sSR7XSArREeddIioxcbp1X2l0YuMbzeylTan32ZivAJIVG9n5PFdPZdf08JHmPyXaN
u7naM8/PGZPU2xstIauSJMOycl7lXVKI0QSBYZYdPApDsFTY5pEJ41HbuSNDidLTiZ
ctTz3jVMoLxN7Pr8Je5iQxP/ExDDEi8EnzSs4GlSimRGFvuiZYZta/6yoSwH6z8HWc
F9bpnJRn3qc75IfXMEGj9RFD+8J5LqhITfwOTbGHiyLx/qJRPbJAEb8+ZARKHG08hV
QSPV1h9T43iSQ==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4RBQRL3pxJz9rxQ;
Thu, 27 Jul 2023 11:27:58 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <83edktn3k4.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN>
<87lef17ok8.fsf@localhost> <83edktn3k4.fsf@HIDDEN>
Date: Thu, 27 Jul 2023 09:28:11 +0000
Message-ID: <87a5vh7lg4.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, dmitry@HIDDEN, 64735 <at> debbugs.gnu.org,
sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Eli Zaretskii <eliz@HIDDEN> writes:
> And if this is about testing REGEXP for being a trivial one, adding
> such a test to the existing code is trivial, and hardly justifies an
> objection to what I wrote.
I was replying to your interpretations on why I included "no-regexp"
test.
I agree that we should not use this test as comparison with GNU find.
But I also wanted to say that adding the trivial REGEXP test will be
useful. Especially because it is easy. Should I prepare a patch?
>> Isn't `file-name-all-completions' more limited and cannot accept
>> arbitrary regexp?
>
> No, see completion-regexp-list.
That would be equivalent to forcing `include-directories' being t.
In any case, even if we ignore INCLUDE-DIRECTORIES, there is no gain:
(my-bench 10 "/usr/src/linux/" "")
(("built-in non-recursive no filename handler alist" . "Elapsed time: 5.780714s (4.352086s in 6 GCs)")
("built-in non-recursive no filename handler alist + completion-regexp-list" . "Elapsed time: 5.739315s (4.359772s in 6 GCs)"))
(let ((gc-cons-threshold most-positive-fixnum))
(my-bench 10 "/usr/src/linux/" ""))
(("built-in non-recursive no filename handler alist" . "Elapsed time: 1.267828s")
("built-in non-recursive no filename handler alist + completion-regexp-list" . "Elapsed time: 1.275656s"))
In the test, I removed all the `string-match' calls and instead let-bound (completion-regexp-list (list regexp))
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 27 Jul 2023 08:47:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 27 04:47:16 2023
Received: from localhost ([127.0.0.1]:40717 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOweR-00023y-UJ
for submit <at> debbugs.gnu.org; Thu, 27 Jul 2023 04:47:16 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:53756)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qOweP-00023i-6p
for 64735 <at> debbugs.gnu.org; Thu, 27 Jul 2023 04:47:14 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qOweJ-00064Z-8j; Thu, 27 Jul 2023 04:47:07 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=FGidqSqyf3NQoMOuBTjJkJ1H2ZvLfT/qCzbWv9CTy9g=; b=YHi74+EpbofA
X/ZZe1fH7BGp53f0FOFpoRkl1tv0noSjQJgUJBM12id2RJTzt3o0sQ2K5NiCV1Dgc8JHjlEttxaAv
vbFXteHQSMNpAkT1aVSXl+m0tPKpqfi9hyeNu8JuqADRyV6ABCZYoC8cj/WE3mk2ueneSFZ+EFKr/
UBTltMmkv/dSTT+HiUBy9jjdaQE9EgvsgnmmMIK4amkQwq1x7No7492ZwEa1lLej4F6KNbQ53QGY2
stXuFPGRbKYyo/o1Z5GKXOuGBlcmtLDeev6r0eZXLNesDL6hY1fyJmquXlpB55xKr58kPAFS9ole/
+pefz/1jjGzPITJkFBZIxQ==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qOweI-0004Zi-OT; Thu, 27 Jul 2023 04:47:07 -0400
Date: Thu, 27 Jul 2023 11:47:55 +0300
Message-Id: <83edktn3k4.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <87lef17ok8.fsf@localhost> (message from Ihor Radchenko on Thu,
27 Jul 2023 08:20:55 +0000)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN>
<87lef17ok8.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, dmitry@HIDDEN, 64735 <at> debbugs.gnu.org,
sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: Dmitry Gutov <dmitry@HIDDEN>, luangruo@HIDDEN,
> sbaugh@HIDDEN, 64735 <at> debbugs.gnu.org
> Date: Thu, 27 Jul 2023 08:20:55 +0000
>
> Eli Zaretskii <eliz@HIDDEN> writes:
>
> >> Skipping regexp matching entirely, though, will make this benchmark
> >> farther removed from real-life usage: this thread started from being
> >> able to handle multiple ignore entries when listing files (e.g. in a
> >> project).
> >
> > Agreed. From my POV, that variant's purpose was only to show how much
> > time is spent in matching file names against some include or exclude
> > list.
>
> Yes and no.
>
> It is not uncommon to query _all_ the files in directory and something
> as simple as
>
> (when (and (not (member regexp '("" ".*"))) (string-match regexp file))...)
>
> can give considerable speedup.
I don't understand what you are saying. The current code already
checks PREDICATE for being nil, and if it is, does nothing about
filtering.
And if this is about testing REGEXP for being a trivial one, adding
such a test to the existing code is trivial, and hardly justifies an
objection to what I wrote.
> > There's a possibility of pushing this filtering into
> > file-name-all-completions, but I'm not sure that will be faster. We
> > should try that and measure the results, I think.
>
> Isn't `file-name-all-completions' more limited and cannot accept
> arbitrary regexp?
No, see completion-regexp-list.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 27 Jul 2023 08:20:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 27 04:20:51 2023
Received: from localhost ([127.0.0.1]:40684 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOwEt-0001Pf-5i
for submit <at> debbugs.gnu.org; Thu, 27 Jul 2023 04:20:51 -0400
Received: from mout01.posteo.de ([185.67.36.65]:39391)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qOwEr-0001PS-FP
for 64735 <at> debbugs.gnu.org; Thu, 27 Jul 2023 04:20:50 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout01.posteo.de (Postfix) with ESMTPS id 8A7C4240029
for <64735 <at> debbugs.gnu.org>; Thu, 27 Jul 2023 10:20:43 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1690446043; bh=8CK4frDJIpaTtKrKYiSTXrB+ElVpBcM6E1a0mmb6o1Q=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=ealKtDjJUROjtxIE65nvxGkX0PytgtR2EbRtkqybupTbEWkAXuNQVasRQiY+9nFRW
+VPXxHaX6Ai7sxii/5KdlMhAdBcNj1TfCeqoChGHwIBq4Ot+ACZzuMML6lgVRSltRK
+jBXXsKEa3vlCfSnGmEuMDnlv1BsLHgfz2GVGi2boxO6xan0ZUCatFuhCNk0QQXH8z
QfJFwCnA7Vi+62x3Rlwe9JjNsoEvYFeQnJCUF3PnW+e9LjAmFYQQeKqw77kEDo9ytv
abRInWx0iGu5R0h55oj3qfYw8T6RDr8mQykKqkWvzpIo0ODKA/dVqtlv/yS7hpWF6p
ZyHWycvM6b+Rw==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4RBNxk3V4Tz6tvc;
Thu, 27 Jul 2023 10:20:42 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <83v8e6lyi4.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
<878rb3m43b.fsf@localhost>
<eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> <83v8e6lyi4.fsf@HIDDEN>
Date: Thu, 27 Jul 2023 08:20:55 +0000
Message-ID: <87lef17ok8.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, Dmitry Gutov <dmitry@HIDDEN>, 64735 <at> debbugs.gnu.org,
sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Eli Zaretskii <eliz@HIDDEN> writes:
>> > I have modified `directory-files-recursively' to avoid O(N^2) `nconc'
>> > calls + bypassing regexp matches when REGEXP is nil.
>>
>> Sounds good. I haven't examined the diff closely, but it sounds like an
>> improvement that can be applied irrespective of how this discussion ends.
>
> That change should be submitted as a separate issue and discussed in
> detail before we decide we can make it.
I will look into it. This was mostly a quick and dirty rewrite without
paying too match attention to file order in the result.
>> Skipping regexp matching entirely, though, will make this benchmark
>> farther removed from real-life usage: this thread started from being
>> able to handle multiple ignore entries when listing files (e.g. in a
>> project).
>
> Agreed. From my POV, that variant's purpose was only to show how much
> time is spent in matching file names against some include or exclude
> list.
Yes and no.
It is not uncommon to query _all_ the files in directory and something
as simple as
(when (and (not (member regexp '("" ".*"))) (string-match regexp file))...)
can give considerable speedup.
Might be worth adding such optimization.
>> So any solution for that (whether we use it on all or just
>> some platforms) needs to be able to handle those. And it doesn't seem
>> like directory-files-recursively has any alternative solution for that
>> other than calling string-match on every found file.
>
> There's a possibility of pushing this filtering into
> file-name-all-completions, but I'm not sure that will be faster. We
> should try that and measure the results, I think.
Isn't `file-name-all-completions' more limited and cannot accept
arbitrary regexp?
>> We can't really forget about GC, though.
>
> But we could temporarily lift the threshold while this function runs,
> if that leads to significant savings.
Yup. Also, GC times and frequencies will vary across different Emacs
sessions. So, we may not want to rely on it when comparing the
benchmarks from different people.
>> But the above numbers make me hopeful about the async-parallel solution,
>> implying that the parallelization really can help (and offset whatever
>> latency we lose on pselect), as soon as we determine the source of extra
>> consing and decide what to do about it.
>
> Isn't it clear that additional consing comes from the fact that we
> first insert the Find's output into a buffer or produce a string from
> it, and then chop that into individual file names?
To add to it, I also tried to implement a version of
`directory-files-recursively' that first inserts all the files in buffer
and then filters them using `re-search-forward' instead of calling
`string-match' on every file name string.
That ended up being slower compared to the current `string-match' approach.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 27 Jul 2023 05:21:58 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 27 01:21:58 2023 Received: from localhost ([127.0.0.1]:40515 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qOtRk-00050u-61 for submit <at> debbugs.gnu.org; Thu, 27 Jul 2023 01:21:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55860) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qOtRa-00050P-19 for 64735 <at> debbugs.gnu.org; Thu, 27 Jul 2023 01:21:54 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qOtRS-0003GL-Qb; Thu, 27 Jul 2023 01:21:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Em312YoA8zr106LqmGsn/fJfxHg7M4cadCjL2P8DwXk=; b=ZMEqN/1IAyur LkgqKB7Uf+WCadyvjpLJZtC7jdNhbGE2zBMfMxjgWYTeTpuY/dl4qLiu3CSqBqgrPQHy1RZyvJGcR gG84Xd+e4wHu55yVnrLTXHW+9RCgKq1o/zCBkbYXQNm45B/3k+y61vpEb/uIJ7d76tBMEqP6Hh2vg 8yH++UZd9oZWbZAWXtAMo3dZOXCRSAMKwcQ/YcINHOypd676pIp07/BdRb4/cXnCTHh+g8DVlpmmp x60My5MLUebGo96IJKk093ZlQDMOXHCcVqrZL+SiDRTE6eOW0c8rEpDeZCsW2OA9z5L6Ht7tOczH/ shpxAV+4PGE21W4nyuxbAQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qOtRR-0001kO-QM; Thu, 27 Jul 2023 01:21:38 -0400 Date: Thu, 27 Jul 2023 08:22:27 +0300 Message-Id: <83v8e6lyi4.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> (message from Dmitry Gutov on Thu, 27 Jul 2023 03:41:29 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost> <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost> <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN> X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Thu, 27 Jul 2023 03:41:29 +0300 > Cc: Eli Zaretskii <eliz@HIDDEN>, luangruo@HIDDEN, sbaugh@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > > I have modified `directory-files-recursively' to avoid O(N^2) `nconc' > > calls + bypassing regexp matches when REGEXP is nil. > > Sounds good. I haven't examined the diff closely, but it sounds like an > improvement that can be applied irrespective of how this discussion ends. That change should be submitted as a separate issue and discussed in detail before we decide we can make it. > Skipping regexp matching entirely, though, will make this benchmark > farther removed from real-life usage: this thread started from being > able to handle multiple ignore entries when listing files (e.g. in a > project). Agreed. From my POV, that variant's purpose was only to show how much time is spent in matching file names against some include or exclude list. > So any solution for that (whether we use it on all or just > some platforms) needs to be able to handle those. And it doesn't seem > like directory-files-recursively has any alternative solution for that > other than calling string-match on every found file. There's a possibility of pushing this filtering into file-name-all-completions, but I'm not sure that will be faster. We should try that and measure the results, I think. > > If we forget about GC, Elisp version can get fairly close to GNU find. > > And if we do not perform regexp matching (which makes sense when the > > REGEXP is ""), Elisp version is faster. > > We can't really forget about GC, though. But we could temporarily lift the threshold while this function runs, if that leads to significant savings. > But the above numbers make me hopeful about the async-parallel solution, > implying that the parallelization really can help (and offset whatever > latency we lose on pselect), as soon as we determine the source of extra > consing and decide what to do about it. Isn't it clear that additional consing comes from the fact that we first insert the Find's output into a buffer or produce a string from it, and then chop that into individual file names?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 27 Jul 2023 00:41:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 26 20:41:44 2023
Received: from localhost ([127.0.0.1]:40346 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOp4a-0006AG-8l
for submit <at> debbugs.gnu.org; Wed, 26 Jul 2023 20:41:44 -0400
Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:57969)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qOp4X-0006A3-Aw
for 64735 <at> debbugs.gnu.org; Wed, 26 Jul 2023 20:41:42 -0400
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
by mailout.west.internal (Postfix) with ESMTP id ADF0332007E8;
Wed, 26 Jul 2023 20:41:34 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute3.internal (MEProxy); Wed, 26 Jul 2023 20:41:35 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm2; t=
1690418494; x=1690504894; bh=ySSA/l4LP/FOYqC9lH92WgOaNrzVFPoeleu
zEfCWzU8=; b=V7xeKrzYYuyM+v0dzAsopQQoH5F8g2yxXDCMKPZxXRmysbGcdsq
503guhJVqtapQ2EpxYXj6DyYbi2ZAgFtMrhqKhOd0S/U7OyWqylfLLxzwl0xkfgh
xhS9EFbqcVotOray3uLn1VF2ehqXuE2/wyoOHeBU1+6o24gxZFifeE5RckAoMGYT
ZAZkKXDRc84khWtTKzipWSEgmry86ozmBH41mApAZXovt/tm3LtCaKNSx3oStUHz
RigcoW7k42whJYat/S2q8ZAgYHL6dc66Q++mpFyftXEmEC3I9HQ1fwym5Oz7kzR0
itTonfrNssKTSUD3QS+mYHP4LbwkXSusHZw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=
1690418494; x=1690504894; bh=ySSA/l4LP/FOYqC9lH92WgOaNrzVFPoeleu
zEfCWzU8=; b=OWDzB9Fj1vHq4elpHc/hlg7vMvGF56o9KnrXCFxpedxA+h/SYTW
alO8zo/BaGnduo9NByaPOKY+HjkeyHn8ni6JVukGR8bITqtSk4td6nZYHIhQtcux
pBMIDjKT3F8YXyOgDLoL5BXIWHYjCxnPY++c40LKdSARbvqMIaXL307y6/+80QKM
rwZt0F41EySsBCJJI1ouQSG8DWHuDxckNYWHjkIrans4V6OCCqzibCi7T0yTmRQb
8AkmJx4dCyzYxiDcuDkwk0GZIC/MDbOB2Z6Yv+weG+Ojl6wqcZWXL3FO1yC6xO48
c5xzh2ZYBo0cfe9K6kh6tdaiGeHQ3L940SA==
X-ME-Sender: <xms:Pb3BZJCoXgkP7-Pz5mHTaNkY7d7DaMsHpClXOHDxR1-b3pc-A_pzhg>
<xme:Pb3BZHhBdZVVKEQwIZmnCC4mT8c2pCoviMsuqnrARwpYFjayaFHwTYzSyChTjrZMj
As07OLK0_iWZt2qpnY>
X-ME-Received: <xmr:Pb3BZEkIv13_fs2HGHDucJSKntsWcPz9D9GNWGFc3NB3VJaz3fCp78jUGDJmq40>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrieefgdefiecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht
rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
gvrhhnpeehleeflefhheeujeduteejveeggeejjefhleeljeettdfhjedthfejgfeivdeu
leenucffohhmrghinhephihhvghtihhlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptd
enucfrrghrrghmpehmrghilhhfrhhomhepughmihhtrhihsehguhhtohhvrdguvghv
X-ME-Proxy: <xmx:Pb3BZDyJ98F-MHiF2x3FO5ddNtHqppc0R75FFIK42Xu9rks60IT5EA>
<xmx:Pb3BZOQaRcOXTN08yADSVuAziXsFu_Hrrbzsd_5w6vFAVke9_DyV_w>
<xmx:Pb3BZGbnqz36LCkCj-nkv9sB209P4mN8W8P-oUQq4gUgnN4MKE-yxA>
<xmx:Pr3BZMeTR2miPyYq3Oc19prcgeXw2J6A31lE7Eifh_sBCcyUslrkHg>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed,
26 Jul 2023 20:41:32 -0400 (EDT)
Message-ID: <eba1c6f1-6ad6-d7eb-b43d-a4173622ce99@HIDDEN>
Date: Thu, 27 Jul 2023 03:41:29 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Ihor Radchenko <yantar92@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN> <878rb3m43b.fsf@localhost>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <878rb3m43b.fsf@localhost>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.8 (/)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)
On 26/07/2023 12:09, Ihor Radchenko wrote:
>> It's also something to note that, GC-wise, numbers 1 and 2 are not the
>> worst: the time must be spent somewhere else.
> Indeed. I did more detailed analysis in
> https://yhetil.org/emacs-devel/87cz0p2xlc.fsf@localhost/
>
> Main contributors in the lisp versions are (in the order from most
> significant to less significant) (1) file name handlers; (2) regexp
> matching of the file names; (3) nconc calls in the current
> `directory-files-recursively' implementation.
>
> I have modified `directory-files-recursively' to avoid O(N^2) `nconc'
> calls + bypassing regexp matches when REGEXP is nil.
Sounds good. I haven't examined the diff closely, but it sounds like an
improvement that can be applied irrespective of how this discussion ends.
Skipping regexp matching entirely, though, will make this benchmark
farther removed from real-life usage: this thread started from being
able to handle multiple ignore entries when listing files (e.g. in a
project). So any solution for that (whether we use it on all or just
some platforms) needs to be able to handle those. And it doesn't seem
like directory-files-recursively has any alternative solution for that
other than calling string-match on every found file.
> Here are the results (using the attached modified version of your
> benchmark file):
>
> (my-bench 10 "/usr/src/linux/" "")
> (("built-in" . "Elapsed time: 7.285597s (3.853368s in 6 GCs)")
> ("built-in no filename handler alist" . "Elapsed time: 5.855019s (3.760662s in 6 GCs)")
> ("built-in non-recursive no filename handler alist" . "Elapsed time: 5.817639s (4.326945s in 7 GCs)")
> ("built-in non-recursive no filename handler alist + skip re-match" . "Elapsed time: 2.708306s (1.871665s in 3 GCs)")
> ("with-find" . "Elapsed time: 6.082200s (4.262830s in 7 GCs)")
> ("with-find-p" . "Elapsed time: 4.325503s (3.058647s in 5 GCs)")
> ("with-find-sync" . "Elapsed time: 3.267648s (1.903655s in 3 GCs)"))
Nice.
> (let ((gc-cons-threshold most-positive-fixnum))
> (my-bench 10 "/usr/src/linux/" ""))
> (("built-in" . "Elapsed time: 2.754473s")
> ("built-in no filename handler alist" . "Elapsed time: 1.322443s")
> ("built-in non-recursive no filename handler alist" . "Elapsed time: 1.235044s")
> ("built-in non-recursive no filename handler alist + skip re-match" . "Elapsed time: 0.750275s")
> ("with-find" . "Elapsed time: 1.438510s")
> ("with-find-p" . "Elapsed time: 1.200876s")
> ("with-find-sync" . "Elapsed time: 1.349755s"))
>
> If we forget about GC, Elisp version can get fairly close to GNU find.
> And if we do not perform regexp matching (which makes sense when the
> REGEXP is ""), Elisp version is faster.
We can't really forget about GC, though.
But the above numbers make me hopeful about the async-parallel solution,
implying that the parallelization really can help (and offset whatever
latency we lose on pselect), as soon as we determine the source of extra
consing and decide what to do about it.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 26 Jul 2023 09:09:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 26 05:09:28 2023
Received: from localhost ([127.0.0.1]:47556 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOaWO-0006rX-3T
for submit <at> debbugs.gnu.org; Wed, 26 Jul 2023 05:09:28 -0400
Received: from mout02.posteo.de ([185.67.36.66]:40895)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qOaWI-0006qf-3P
for 64735 <at> debbugs.gnu.org; Wed, 26 Jul 2023 05:09:23 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout02.posteo.de (Postfix) with ESMTPS id CC654240105
for <64735 <at> debbugs.gnu.org>; Wed, 26 Jul 2023 11:09:15 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1690362555; bh=dfRxu3WjvbG3Z1ab4VqywW3Zm8wPksFVH7XOINpcQmQ=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=h+j0SfJCpqNj4RdF0Kg9nAuvk/5Bv04nuVjuTbbr5Sgp2Imzj7zUzFhQKq4qbm4Ex
WZL6z3ydQLtNn0x8dUaDTtwkrrXudrIvbi1293n0QOSOfhjnCjgd7uV82/IQcykGKi
KJtw2oA4VsX7orOi6Ub9Q4YspFaNQavkxy7i4h5O7LmcrUiC18WKZuil+ky9xUFaAd
GZNLIcScYXywjD8bZJil9YRpOXAKkRWslNVS1b/Ht/aXZ/bkPxrHKCVoZPv1UVJze5
mAGm6YQdXrHchbzuq/i9aWdWDaVDLXugY3zsTuPpgAlstWTqnnbAKafupq0YFv43P6
vZODBNltCkn2Q==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R9p4B4gBkz6tn4;
Wed, 26 Jul 2023 11:09:14 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
<87351cs8no.fsf@localhost>
<35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
Date: Wed, 26 Jul 2023 09:09:28 +0000
Message-ID: <878rb3m43b.fsf@localhost>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--=-=-=
Content-Type: text/plain
Dmitry Gutov <dmitry@HIDDEN> writes:
>> (my-bench 10 "/usr/src/linux/" "")
>>
>> (("built-in" . "Elapsed time: 7.034326s (3.598539s in 14 GCs)")
>> ("built-in no filename handler alist" . "Elapsed time: 5.907194s (3.698456s in 15 GCs)")
>> ("with-find" . "Elapsed time: 6.078056s (4.052791s in 16 GCs)")
>> ("with-find-p" . "Elapsed time: 4.496762s (2.739565s in 11 GCs)")
>> ("with-find-sync" . "Elapsed time: 3.702760s (1.715160s in 7 GCs)"))
>
> Thanks, for the extra data point in particular. Easy to see how it
> compares to the most efficient use of 'find', right (on GNU/Linix, at
> least)?
>
> It's also something to note that, GC-wise, numbers 1 and 2 are not the
> worst: the time must be spent somewhere else.
Indeed. I did more detailed analysis in
https://yhetil.org/emacs-devel/87cz0p2xlc.fsf@localhost/
Main contributors in the lisp versions are (in the order from most
significant to less significant) (1) file name handlers; (2) regexp
matching of the file names; (3) nconc calls in the current
`directory-files-recursively' implementation.
I have modified `directory-files-recursively' to avoid O(N^2) `nconc'
calls + bypassing regexp matches when REGEXP is nil.
Here are the results (using the attached modified version of your
benchmark file):
(my-bench 10 "/usr/src/linux/" "")
(("built-in" . "Elapsed time: 7.285597s (3.853368s in 6 GCs)")
("built-in no filename handler alist" . "Elapsed time: 5.855019s (3.760662s in 6 GCs)")
("built-in non-recursive no filename handler alist" . "Elapsed time: 5.817639s (4.326945s in 7 GCs)")
("built-in non-recursive no filename handler alist + skip re-match" . "Elapsed time: 2.708306s (1.871665s in 3 GCs)")
("with-find" . "Elapsed time: 6.082200s (4.262830s in 7 GCs)")
("with-find-p" . "Elapsed time: 4.325503s (3.058647s in 5 GCs)")
("with-find-sync" . "Elapsed time: 3.267648s (1.903655s in 3 GCs)"))
(let ((gc-cons-threshold most-positive-fixnum))
(my-bench 10 "/usr/src/linux/" ""))
(("built-in" . "Elapsed time: 2.754473s")
("built-in no filename handler alist" . "Elapsed time: 1.322443s")
("built-in non-recursive no filename handler alist" . "Elapsed time: 1.235044s")
("built-in non-recursive no filename handler alist + skip re-match" . "Elapsed time: 0.750275s")
("with-find" . "Elapsed time: 1.438510s")
("with-find-p" . "Elapsed time: 1.200876s")
("with-find-sync" . "Elapsed time: 1.349755s"))
If we forget about GC, Elisp version can get fairly close to GNU find.
And if we do not perform regexp matching (which makes sense when the
REGEXP is ""), Elisp version is faster.
--=-=-=
Content-Type: text/plain
Content-Disposition: inline; filename=find-bench.el
;; -*- lexical-binding: t; -*-
(defun find-directory-files-recursively (dir regexp &optional include-directories _p follow-symlinks)
(cl-assert (null _p) t "find-directory-files-recursively can't accept arbitrary predicates")
(with-temp-buffer
(setq case-fold-search nil)
(cd dir)
(let* ((command
(append
(list "find" (file-local-name dir))
(if follow-symlinks
'("-L")
'("!" "(" "-type" "l" "-xtype" "d" ")"))
(unless (string-empty-p regexp)
(list "-regex" (concat ".*" regexp ".*")))
(unless include-directories
'("!" "-type" "d"))
'("-print0")
))
(remote (file-remote-p dir))
(proc
(if remote
(let ((proc (apply #'start-file-process
"find" (current-buffer) command)))
(set-process-sentinel proc (lambda (_proc _state)))
(set-process-query-on-exit-flag proc nil)
proc)
(make-process :name "find" :buffer (current-buffer)
:connection-type 'pipe
:noquery t
:sentinel (lambda (_proc _state))
:command command))))
(while (accept-process-output proc))
(let ((start (goto-char (point-min))) ret)
(while (search-forward "\0" nil t)
(push (concat remote (buffer-substring-no-properties start (1- (point)))) ret)
(setq start (point)))
ret))))
(defun find-directory-files-recursively-2 (dir regexp &optional include-directories _p follow-symlinks)
(cl-assert (null _p) t "find-directory-files-recursively can't accept arbitrary predicates")
(cl-assert (not (file-remote-p dir)))
(let* (buffered
result
(proc
(make-process
:name "find" :buffer nil
:connection-type 'pipe
:noquery t
:sentinel (lambda (_proc _state))
:filter (lambda (proc data)
(let ((start 0))
(when-let (end (string-search "\0" data start))
(push (concat buffered (substring data start end)) result)
(setq buffered "")
(setq start (1+ end))
(while-let ((end (string-search "\0" data start)))
(push (substring data start end) result)
(setq start (1+ end))))
(setq buffered (concat buffered (substring data start)))))
:command (append
(list "find" (file-local-name dir))
(if follow-symlinks
'("-L")
'("!" "(" "-type" "l" "-xtype" "d" ")"))
(unless (string-empty-p regexp)
(list "-regex" (concat ".*" regexp ".*")))
(unless include-directories
'("!" "-type" "d"))
'("-print0")
))))
(while (accept-process-output proc))
result))
(defun find-directory-files-recursively-3 (dir regexp &optional include-directories _p follow-symlinks)
(cl-assert (null _p) t "find-directory-files-recursively can't accept arbitrary predicates")
(cl-assert (not (file-remote-p dir)))
(let ((args `(,(file-local-name dir)
,@(if follow-symlinks
'("-L")
'("!" "(" "-type" "l" "-xtype" "d" ")"))
,@(unless (string-empty-p regexp)
(list "-regex" (concat ".*" regexp ".*")))
,@(unless include-directories
'("!" "-type" "d"))
"-print0")))
(with-temp-buffer
(let ((status (apply #'process-file
"find"
nil
t
nil
args))
(pt (point-min))
res)
(unless (zerop status)
(error "Listing failed"))
(goto-char (point-min))
(while (search-forward "\0" nil t)
(push (buffer-substring-no-properties pt (1- (point)))
res)
(setq pt (point)))
res))))
(defun directory-files-recursively-strip-nconc
(dir regexp
&optional include-directories predicate
follow-symlinks)
"Return list of all files under directory DIR whose names match REGEXP.
This function works recursively. Files are returned in \"depth
first\" order, and files from each directory are sorted in
alphabetical order. Each file name appears in the returned list
in its absolute form.
By default, the returned list excludes directories, but if
optional argument INCLUDE-DIRECTORIES is non-nil, they are
included.
PREDICATE can be either nil (which means that all subdirectories
of DIR are descended into), t (which means that subdirectories that
can't be read are ignored), or a function (which is called with
the name of each subdirectory, and should return non-nil if the
subdirectory is to be descended into).
If FOLLOW-SYMLINKS is non-nil, symbolic links that point to
directories are followed. Note that this can lead to infinite
recursion."
(let* ((result nil)
(dirs (list dir))
(dir (directory-file-name dir))
;; When DIR is "/", remote file names like "/method:" could
;; also be offered. We shall suppress them.
(tramp-mode (and tramp-mode (file-remote-p (expand-file-name dir)))))
(while (setq dir (pop dirs))
(dolist (file (file-name-all-completions "" dir))
(unless (member file '("./" "../"))
(if (directory-name-p file)
(let* ((leaf (substring file 0 (1- (length file))))
(full-file (concat dir "/" leaf)))
;; Don't follow symlinks to other directories.
(when (and (or (not (file-symlink-p full-file))
follow-symlinks)
;; Allow filtering subdirectories.
(or (eq predicate nil)
(eq predicate t)
(funcall predicate full-file)))
(push full-file dirs))
(when (and include-directories
(string-match regexp leaf))
(setq result (nconc result (list full-file)))))
(when (and regexp (string-match regexp file))
(push (concat dir "/" file) result))))))
(sort result #'string<)))
(defun my-bench (count path regexp)
(setq path (expand-file-name path))
;; (let ((old (directory-files-recursively path regexp))
;; (new (find-directory-files-recursively-3 path regexp)))
;; (dolist (path old)
;; (unless (member path new) (error "! %s not in" path)))
;; (dolist (path new)
;; (unless (member path old) (error "!! %s not in" path))))
(list
(cons "built-in" (benchmark count (list 'directory-files-recursively path regexp)))
(cons "built-in no filename handler alist" (let (file-name-handler-alist) (benchmark count (list 'directory-files-recursively path regexp))))
(cons "built-in non-recursive no filename handler alist" (let (file-name-handler-alist) (benchmark count (list 'directory-files-recursively-strip-nconc path regexp))))
(cons "built-in non-recursive no filename handler alist + skip re-match" (let (file-name-handler-alist) (benchmark count (list 'directory-files-recursively-strip-nconc path nil))))
(cons "with-find" (benchmark count (list 'find-directory-files-recursively path regexp)))
(cons "with-find-p" (benchmark count (list 'find-directory-files-recursively-2 path regexp)))
(cons "with-find-sync" (benchmark count (list 'find-directory-files-recursively-3 path regexp)))))
(provide 'find-bench)
--=-=-=
Content-Type: text/plain
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
--=-=-=--
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 26 Jul 2023 02:35:27 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 25 22:35:27 2023 Received: from localhost ([127.0.0.1]:47276 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qOUN4-00057x-BI for submit <at> debbugs.gnu.org; Tue, 25 Jul 2023 22:35:27 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:50853) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qOUN1-00057h-Ua for 64735 <at> debbugs.gnu.org; Tue, 25 Jul 2023 22:35:24 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id CD7C53200940; Tue, 25 Jul 2023 22:35:17 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 25 Jul 2023 22:35:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690338917; x=1690425317; bh=K2hPmBI5ORt1P1I9b12hUA1B568rm3T9q0y z9cpe+X8=; b=hqOVxsaevI7VjZBjyMluqipQ43mvmjmvYQFmxvL1sKIzhhApGCK YvPXJeAbzG9wT9Wxk9VHpBLiJLV6iPTqIFRIpRxo81GW7bF+gjMSgvffAcTI1mf3 5Lc2Nh/ikntnsiVxPmPQL2ECaYvai1uxZb2SjOtpotmwo1er9S0/v7WbQham1U+Y 2JcNwCnrNxCMCPvOpsZvrLe1jnqF6XTbTXe7/7hs5NEuiNgEBxRMijmlSTRbY5lX 5ntBurE0qSszYyI8mW5BBLiyx9oFL2by3oJEQNJrBCehB99Deo3gKue5ZVidkgNJ 846PtxljZeGIAsV+VunDuwCpVylPJDf4VPg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690338917; x=1690425317; bh=K2hPmBI5ORt1P1I9b12hUA1B568rm3T9q0y z9cpe+X8=; b=v0jIIhIi/Gbg1RCOs10eN+mls5i3QjtsFWmOXSRlbBceS8FclZn g+llICkQmtOpfsPAWmNivIGb9jmQswq2pKFByjp2X93ki60QK8GbYzFUnWAcWTx5 UefiAgfeaoTx8GnFnXKCwcu7r8Y4uE7BxNccQ7V2hbAbUGRDn2ySGkVTk2m8rJK8 iJGm868PydrsLXC5RabX01bnHvhXP1rptqWYDkLYtO9DfcJAQd8Jzi9I+hDQiY7y eIUbGCkmQm+PRzA67C6Y5U9Y1aR4wdhzEwIpoWKrMdN8CXr726XpySbBQWIILxpP W20j8jGBkHTO61zR3IAEkTXoE1/qSFM1Jlg== X-ME-Sender: <xms:ZIbAZPcWvsxIiaGQ90GVFlGdbq-h1X2zIC0gmRLqWfEkSIVrOCEAvQ> <xme:ZIbAZFPNO5s9AqP_6A8lU3lV2u4fcs3hGNWI0Ec4Psf2yMYdz6I5HiFI-dlJnwymM uJxMEfNNBtiXu-MxpE> X-ME-Received: <xmr:ZIbAZIhW0wwDkD-nf0LU78gT1M0-WNy2RYJJHRCJQ_7l17gm3RidfEVGthS4Vs8> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedriedugdehlecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:ZIbAZA90_U8SgFAQEKeVWZPhE8Vh1JsIq3tRcbJR6nli3I_3ZNGgEA> <xmx:ZIbAZLuoRwHnXMPWi3L9Mrt4D5MhvGtFM_NhS1qviecxbEldIUkW2A> <xmx:ZIbAZPF1ZSTA7mAKKsW7h6DrFGRV4zKVDsj1TGRGG_1SWzRhv2Mr9Q> <xmx:ZYbAZLJI0lVu8yg35YGFpOthT6ILxNscs5CYQejiUe7gDfvCKpv2-Q> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 25 Jul 2023 22:35:14 -0400 (EDT) Message-ID: <910b6a80-687e-9ffb-03fe-263dfdc10031@HIDDEN> Date: Wed, 26 Jul 2023 05:35:13 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <83351bq1ds.fsf@HIDDEN> <491724c3-534b-a498-8e48-c8d94531875c@HIDDEN> <83zg3jo18g.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83zg3jo18g.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 26/07/2023 05:28, Eli Zaretskii wrote: >> Date: Wed, 26 Jul 2023 04:56:20 +0300 >> Cc:luangruo@HIDDEN,sbaugh@HIDDEN,yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov<dmitry@HIDDEN> >> >> Your other idea (spending time in text conversion) also sounds >> plausible, but I don't know whether this much overhead can be explained >> by it. And don't we have to convert any process's output to our internal >> encoding anyway, on any platform? > We do, but you-all probably run your tests on a system where the > external encoding is UTF-8, right? That is much faster. I do. I suppose that transcoding can/uses the short-circuit approach, avoiding extra copying when the memory representations match. It should be possible to measure the encoding's overhead by checking how big the output is, testing our code on a smaller string, and multiplying. Or, more roughly, by piping it to "iconv -f Windows-1251 -t UTF-8" and measuring how long it will take to finish (if our encoding takes longer, that could point to an optimization opportunity as well).
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 26 Jul 2023 02:28:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 25 22:28:35 2023
Received: from localhost ([127.0.0.1]:47265 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOUGR-0004wi-2b
for submit <at> debbugs.gnu.org; Tue, 25 Jul 2023 22:28:35 -0400
Received: from wnew4-smtp.messagingengine.com ([64.147.123.18]:34069)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qOUGP-0004wT-Ay
for 64735 <at> debbugs.gnu.org; Tue, 25 Jul 2023 22:28:33 -0400
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by mailnew.west.internal (Postfix) with ESMTP id E7F932B0016E;
Tue, 25 Jul 2023 22:28:26 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute4.internal (MEProxy); Tue, 25 Jul 2023 22:28:27 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm2; t=
1690338506; x=1690345706; bh=74xdhRfrhO14sFCCEbwcAaJMdU3fv/eJbHD
ie3m3Nng=; b=f0o+1Whq+8VKbm+kPc95nPEJCc4ZatfJ6ZyV0Pba678BFiBfk1D
sxiclAYFCyn76whDaP37DSWTIvh6bJE8K3N//dX1IRLn+GPDVnmmQnhTJNhP4gsu
eMYBY9MRJa9xJCx6nuYo6JbJIe875y9R+in9KG6iKA0kYifljeF4btijmZpoag40
hHj1+MWbFwsgcXST7VD2ZUdi1DLN0RoANmBBgjC8ohRU/dlul+dkGVMORdH6Q102
80GWWg+CPPo+quhb/eH2bzrIQ3w7S/sMcpwUw7/ylQQFtpk/wlGK6nFU0CN+oxOV
lb/JtKnKnqnRY841l+MS3a9/O7daR/h6CCg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=
1690338506; x=1690345706; bh=74xdhRfrhO14sFCCEbwcAaJMdU3fv/eJbHD
ie3m3Nng=; b=ARXXeCKp7e1hGaLdcusSKL+4LEeNrTqqcOiKjvMV12tOe316Ns1
pwJ3HJOIyrKh2xOnYfoY8WFtex0Lo94sN1nCkVk078JwA0+WftVY3fNf5rxwj20w
8oJYSneoGQwQX6M85l+e1KUKTFUOpOlvy9BIb0gKIO4W2M18k+EB1EPO3lDC6DR9
WqWiQROQsprpn8FwS17mcCHAA6BmDALJA/pYhwGkKZ2Q4/u4aQSDjZOGgz5NEVNC
G4Hg9mduRh0LupbwAuRpbQbIufLhyeS+Fw0oipqFst7o2hdO5NVMsHwVBzk8vr28
ytLyCYUfltfFFb/Pi5X28NT1EZSUN/Ai2hw==
X-ME-Sender: <xms:wITAZFsIbwY4PNutBkUQqwDCOFiSPS46_uIVnuTCtGF-4Rd7jwHHpA>
<xme:wITAZOdsomScLaeKPkqZgxr09M_i6v1Z6eP9uFipWT1_Z-Vla3zo-H2XdKGm9bg0v
m3DLYSJWDuhca--Ypg>
X-ME-Received: <xmr:wITAZIxAVHWpcMirWdkgGx5Y-aOneus_AjQzdylhPo3tm9s2KSx3BlmRu9NAJPo>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedriedugdehjecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht
rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv
veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh
hithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:wITAZMOv0NSX_-GjlylDSfuBDFOAYz_6wJp7UY0eeNv12WOMx4ZVrg>
<xmx:wITAZF_My3irSKZLTAytpdf8szf_7U98SWaVKNq9q_BTtRIDXUlJCA>
<xmx:wITAZMX63dJqiF-mLgK80apOT85Rqmz0Xqw9jUou6ZvL-eswFDUO-A>
<xmx:yoTAZKK1uoCJaJ7m1rkSHY8N7m1Bv2Yvw7LmV7W7FQTYRckUSRZ5f_ms0ZQ>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
25 Jul 2023 22:28:14 -0400 (EDT)
Message-ID: <7163164f-28a3-13b0-ffa8-eb1fc8b0b569@HIDDEN>
Date: Wed, 26 Jul 2023 05:28:13 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: sbaugh@HIDDEN
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <877cqniyyk.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <877cqniyyk.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.8 (/)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>,
yantar92@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)
On 25/07/2023 22:16, sbaugh@HIDDEN wrote:
> Dmitry Gutov <dmitry@HIDDEN> writes:
>>> (my-bench 1 default-directory "")
>>
>> (("built-in" . "Elapsed time: 1.601649s (0.709108s in 22 GCs)")
>> ("with-find" . "Elapsed time: 1.792383s (1.135869s in 38 GCs)")
>> ("with-find-p" . "Elapsed time: 1.248543s (0.682827s in 20 GCs)")
>> ("with-find-sync" . "Elapsed time: 0.922291s (0.343497s in 10 GCs)"))
>
> Tangent, but:
>
> Ugh, wow, call-process really is a lot faster than make-process. I see
> now why people disliked my idea of replacing call-process with something
> based on make-process, this is a big difference...
More like forewarned. Do we want to exchange 25% of performance for
extra reactivity? We might. But we'd probably put that behind a pref and
have to maintain two implementations.
> There's zero reason it has to be so slow... maybe I should try to make a
> better make-process API and implementation which is actually fast.
> (without worrying about being constrained by compatibility with
> something that's already dog-slow)
I don't know if the API itself is at fault. The first step should be to
investigate which part of the current one is actually slow, I think.
But then, of course, if improved performance really requires a change in
the API, we can switch to some new one too (which having to maintain at
least two implementations for a number of years).
BTW, looking at the difference between the with-find-* approaches'
performance, it seems like most of it comes down to GC.
Any chance we're doing extra copying of strings even when we don't have
to, or some inefficient copying -- compared to the sync implementation?
E.g. we could use the "fast" approach at least when the :filter is not
specified (which is the case in the first impl, "with-find"). The manual
says this:
The default filter simply outputs directly to the process buffer.
Perhaps it's worth looking at.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 26 Jul 2023 02:27:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 25 22:27:37 2023 Received: from localhost ([127.0.0.1]:47260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qOUFU-0004v6-Lz for submit <at> debbugs.gnu.org; Tue, 25 Jul 2023 22:27:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42178) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qOUFS-0004ut-H1 for 64735 <at> debbugs.gnu.org; Tue, 25 Jul 2023 22:27:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qOUFM-0005Jf-Tl; Tue, 25 Jul 2023 22:27:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=B/5atKyJeUF44bRLmMU4vqV3Rl6keHwhGJZKHqekt94=; b=Aoq/mhwcFQCJ rHs20NRF0ch2ITQzerVB5kLwW5mPWFyZIlQrxWS7Mo4IrLPDTRdA3KSboOwCsr9dfb/Il9KAIW5Ey jFZlFV2Z6+wEG0JLs3oA2vhVk0xhMQQuyJyQu9+dgx6w6afb8pRsCkWJBhgJBvJQe1sVNV2B9+Phz EmgP696UkS5XIDZ7EMmjOF963ruSgVr86qPMWuX2agpbXmvk7lpQDXSIM0dl2sJITItPHlDqIOdIZ QPlH2CO61BYZYlVGlwShAGRqVaoqf7uI8AieOenWx5LT0MJ+pIqwDUpQYa+lSNmXLaG6gGjYeOCLU gk7g6VMlWVXIUVHDmsa/bA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qOUFM-0007Qm-Cd; Tue, 25 Jul 2023 22:27:28 -0400 Date: Wed, 26 Jul 2023 05:28:15 +0300 Message-Id: <83zg3jo18g.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <491724c3-534b-a498-8e48-c8d94531875c@HIDDEN> (message from Dmitry Gutov on Wed, 26 Jul 2023 04:56:20 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN> <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <83351bq1ds.fsf@HIDDEN> <491724c3-534b-a498-8e48-c8d94531875c@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Wed, 26 Jul 2023 04:56:20 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > Your other idea (spending time in text conversion) also sounds > plausible, but I don't know whether this much overhead can be explained > by it. And don't we have to convert any process's output to our internal > encoding anyway, on any platform? We do, but you-all probably run your tests on a system where the external encoding is UTF-8, right? That is much faster.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 26 Jul 2023 01:56:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 25 21:56:32 2023
Received: from localhost ([127.0.0.1]:47250 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOTlP-00048x-Qs
for submit <at> debbugs.gnu.org; Tue, 25 Jul 2023 21:56:32 -0400
Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:38135)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qOTlO-00048l-2I
for 64735 <at> debbugs.gnu.org; Tue, 25 Jul 2023 21:56:30 -0400
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
by mailout.west.internal (Postfix) with ESMTP id D2DA732008C0;
Tue, 25 Jul 2023 21:56:23 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute2.internal (MEProxy); Tue, 25 Jul 2023 21:56:24 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm2; t=
1690336583; x=1690422983; bh=Hlobxmo9o5+6CYqNyUY7Jmyztu8RkRGhtjr
sFcEFgwk=; b=fzjTWdkQpzSzDghfy3iudzekjoVZ3TUwKMSS2suPCK/daaMWwB5
7yBJW3O6z7VwvwSx+k3CiQHzmHJeEgIBtqmMSaWO3hvS+DVsTTUaGuC2eugLUh1h
9S5qXpFyqJDj6KDgR+pswMF54LTk2eJnSp5kVy+Vu6fhr8cjwhyQLNLQn0uv33gC
CtIMS/YimZG6hrM2Qx5JjGgyNkYsbsRdJPPfYdyD8E37rHuOCqNLpwGIj8pZu14J
PWnAMQkwyJFzs9iVVZYNgyyV9Hac+PkLiebxbbI585QuP9bUqGMQ+S/JvIW/457r
AsDTdQONvtu/VDJrOp766mmb9YRVRRcqE0g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=
1690336583; x=1690422983; bh=Hlobxmo9o5+6CYqNyUY7Jmyztu8RkRGhtjr
sFcEFgwk=; b=CTxSgj9swJ0AurQZ2caWn0O3EWnxcq3g9prj8P4CPIVsF/SvWfS
hPipstfdZMI4jWzVvTtYqaRYcSQ9OfeaQvmCyez4hFNOcI9iIrxtBLXsncCA5107
jiIcOEKnIobLZ6FBpj513PNCl+9oiYIM0kpCqKZTZTezwH1zp4HeSl+J8tkixgMx
ztWEU7T5IIZRlQxSXd8C+3jg1csdRmSucCRq3q3rKg693+KcPRsZsqZw/S/3o3hK
4fvNUiVHF+56KzWuBkmFmusLiemXB2NA9w4C9miS2gLCT0Fh/fvvp/XoZ0zGdxIV
OgESjWFRZR0kBp/BCwmd3ZwyTi2BFFmjN6A==
X-ME-Sender: <xms:R33AZC1PrMbpUCtQKHTmJRaQBqXtOiilSTyrjgd5cUAQQzwla28VnQ>
<xme:R33AZFFA8rHvbgH8mI-EpuJqL57IybvACoj5RNLcgm1iNYWYqsqcBGxDjYAD94kMT
iaS4ucJJJzYtCb5R28>
X-ME-Received: <xmr:R33AZK6A-unrcbcPj3-KN-JLt6N1f8u1a-UVnp7SFOtkJT1_ig0_spZ2g5RxVW8>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedriedugdehudcutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht
rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv
veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh
hithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:R33AZD2KkHh67FFWFuu0SMSxdek4OZbmaqQ3J8MxbKfVQo8SitH4TA>
<xmx:R33AZFFuPcSx5oxZtd_Dtos-gBMUmmH2EYwlbkoCIo0GarsptcTMxw>
<xmx:R33AZM9a1zAvyNCst_H-FZFKPVMmtQkYRLJIA5vcToVjpfunuLgEog>
<xmx:R33AZFCrvKPPl8Q06xu8fV-sFu-sDmqr56NkDPXXJ_b0vV9Ho3-zWg>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
25 Jul 2023 21:56:22 -0400 (EDT)
Message-ID: <491724c3-534b-a498-8e48-c8d94531875c@HIDDEN>
Date: Wed, 26 Jul 2023 04:56:20 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <83351bq1ds.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <83351bq1ds.fsf@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.8 (/)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)
On 25/07/2023 21:42, Eli Zaretskii wrote:
> Almost no change on Windows:
>
> (("built-in" . "Elapsed time: 1.218750s (0.078125s in 5 GCs)")
> ("with-find" . "Elapsed time: 8.984375s (0.109375s in 7 GCs)")
> ("with-find-p" . "Elapsed time: 8.718750s (0.046875s in 3 GCs)")
> ("with-find-sync" . "Elapsed time: 8.921875s (0.046875s in 3 GCs)"))
>
> I'm beginning to suspect the implementation of pipes (and IPC in
> general). How else can such slowdown be explained?
Seems so (I'm not well-versed in the lower level details, alas).
Your other idea (spending time in text conversion) also sounds
plausible, but I don't know whether this much overhead can be explained
by it. And don't we have to convert any process's output to our internal
encoding anyway, on any platform?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 26 Jul 2023 01:51:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 25 21:51:27 2023
Received: from localhost ([127.0.0.1]:47245 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOTgV-00041a-3H
for submit <at> debbugs.gnu.org; Tue, 25 Jul 2023 21:51:27 -0400
Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:57915)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qOTgT-00041K-77
for 64735 <at> debbugs.gnu.org; Tue, 25 Jul 2023 21:51:25 -0400
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
by mailout.west.internal (Postfix) with ESMTP id DA7CE32007E8;
Tue, 25 Jul 2023 21:51:18 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute5.internal (MEProxy); Tue, 25 Jul 2023 21:51:19 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-transfer-encoding:content-type:content-type:date
:date:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to; s=fm2; t=
1690336278; x=1690422678; bh=LjT85s9eYxb/9vf3gzkWEl6ej7yi2/gbGhI
PdMab67Y=; b=mmErWlp8QxXlYm2PPav8POsTd55cqx4sf/29z2EsLLwiJquNESY
eFBiOLHnMANubeUdoG1jruqPRM5t1dLEV3Vdj5ThAs9S3/HRrn5TvybAC3Uvl2os
+aVhdkRvbCAecsRN3U3ZNhWygmEVUYEH/XTjuSx910fSkqLqyJ1siUXfNqirexp4
+uOCFeyTZroMjSS3J1aOcqB1Yn17pFeJDHqMBryPL+5Ag6t2GsvocRnnHkmqmdNC
xhduNIjFDXzddpaf/3K/0VZ2CBtsQCwb9Ffm2DRqVHIz1IEicF1nFfcC+2mnhvak
7SlzcuygQWdgWVgB4Ra/h7CfYjpxIHlKXfA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-transfer-encoding
:content-type:content-type:date:date:feedback-id:feedback-id
:from:from:in-reply-to:in-reply-to:message-id:mime-version
:references:reply-to:sender:subject:subject:to:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=
1690336278; x=1690422678; bh=LjT85s9eYxb/9vf3gzkWEl6ej7yi2/gbGhI
PdMab67Y=; b=VztAE3mPMlWLZajzNCDgoiqvYd4J51omnP+SJWuqg+/5OLOhqPb
UctflD3tSz9fEjiKerTSfOZi55kF2IwItma/uWrdXAAWmHRsv/Usc/3fTuC7Uw8a
dwrE/BcbC2MGpk1LQEYXF2K6kNBGNKkcq5W8d913BbnmdELm+WpSu66QBgDAUDYi
lo9pUfdkXkHk8vNljRICG31YBlmcriXyVbTCYFEARTDvn+9J6vmetLkn19RG85Ax
86Qx74NgDeXKrnlz19STJIuA+sofN+hOBVbmm0085lsEJnTLDevEqVMp91TssEEj
8aPKMGqEKxV2fF82ppyojaVEjQ8/ZqRtvSA==
X-ME-Sender: <xms:FnzAZAJXqDST6BpQn9ytSdcFfYqq73NreuZQIxZ2qihJVRtDvd8xBg>
<xme:FnzAZAL0puMViViAbIBhvOeGdw2Gg7Z8wZBDMV--gfk4CF8it0Gwk_daGSA590bar
5L85kATNropiMFYpAM>
X-ME-Received: <xmr:FnzAZAsJiJL86dUe1i_V_wn9XNSjfTZjJWym9o68pcb9XpCzMgcKhqTjZmFQoD0>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedriedugdeglecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht
rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv
veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh
hithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:FnzAZNYnE8HjswovkcoVX5rB9uYjYnHL9vvIX0uqcY5wUcJUo3faeA>
<xmx:FnzAZHb-ffTBMLvAB-8D6E7qS2sG_4vtRWN1olLgQJPkya204rM7EQ>
<xmx:FnzAZJD48bXt8EHC8PIC70YJQ5enC0P9RcasHsNtiLQtwzAwofiD4w>
<xmx:FnzAZDHgCgAitm64VpI0zTOEdVzlR5WxNqkLDymOMvR8gHVTcRQeXQ>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue,
25 Jul 2023 21:51:16 -0400 (EDT)
Message-ID: <35163e56-607d-9c5b-e3e8-5d5b548b3cb7@HIDDEN>
Date: Wed, 26 Jul 2023 04:51:15 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Ihor Radchenko <yantar92@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> <87351cs8no.fsf@localhost>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <87351cs8no.fsf@localhost>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.8 (/)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)
On 25/07/2023 11:22, Ihor Radchenko wrote:
> Dmitry Gutov<dmitry@HIDDEN> writes:
>
>> I'm attaching an extended benchmark, one that includes a "synchronous"
>> implementation as well. Please give it a spin as well.
> GNU/Linux SSD
>
> (my-bench 10 "/usr/src/linux/" "")
>
> (("built-in" . "Elapsed time: 7.034326s (3.598539s in 14 GCs)")
> ("built-in no filename handler alist" . "Elapsed time: 5.907194s (3.698456s in 15 GCs)")
> ("with-find" . "Elapsed time: 6.078056s (4.052791s in 16 GCs)")
> ("with-find-p" . "Elapsed time: 4.496762s (2.739565s in 11 GCs)")
> ("with-find-sync" . "Elapsed time: 3.702760s (1.715160s in 7 GCs)"))
Thanks, for the extra data point in particular. Easy to see how it
compares to the most efficient use of 'find', right (on GNU/Linix, at
least)?
It's also something to note that, GC-wise, numbers 1 and 2 are not the
worst: the time must be spent somewhere else.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 25 Jul 2023 19:16:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 25 15:16:27 2023
Received: from localhost ([127.0.0.1]:46993 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qONWF-0001be-L8
for submit <at> debbugs.gnu.org; Tue, 25 Jul 2023 15:16:27 -0400
Received: from s.wrqvtbkv.outbound-mail.sendgrid.net ([149.72.123.24]:45740)
by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from
<bounces+21787432-b76c-64735=debbugs.gnu.org@HIDDEN>)
id 1qONWD-0001bQ-PI
for 64735 <at> debbugs.gnu.org; Tue, 25 Jul 2023 15:16:26 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=catern.com;
h=from:subject:in-reply-to:references:mime-version:to:cc:content-type:
content-transfer-encoding:cc:content-type:from:subject:to;
s=s1; bh=IBR4/bLNhmMuVWLT8kELUMGk3qDwcpDbnzCKfKDjpnc=;
b=Oi6sgk3f8o5xl4ny2SNda8LXfuTPOYqHkAdJ9CfoZwFjPEWeQpMsrGfvrkRWUsiIfeFq
30wE1kM5nxJ9vFT1fFlx4SQvvNml2kD0VLraQjVq7yD1XqXD1Oe4kko6Nq2kchk15vT3Qj
MLxtZ1SJtd9NZyA8MgBEmkEbZzj1WaURQ1rPnjQ+vr7pmBYXRyHEAPsZMA3bm5g/bBFnqZ
W7an40y63uE6dkwoBTdV2TgseTg+4ZrODFB1MT4uS+r/QNQKV/O+0Uuxl/uBonJWOq0dwp
qzZwkY5HpT+dK/hHSgUBa5E9kcxITJTWYgGkE5PyLbtpRFh3W5rVlpk1YLNSDbSA==
Received: by filterdrecv-d7bbbc8bf-ftxgm with SMTP id
filterdrecv-d7bbbc8bf-ftxgm-1-64C01F84-38
2023-07-25 19:16:20.292981975 +0000 UTC m=+6550593.487377294
Received: from earth.catern.com (unknown) by geopod-ismtpd-canary-0 (SG)
with ESMTP id VmMM01OkQD2enD_ee9hN2g
Tue, 25 Jul 2023 19:16:20.261 +0000 (UTC)
X-Comment: SPF check N/A for local connections - client-ip=::1; helo=localhost;
envelope-from=sbaugh@HIDDEN; receiver=gutov.dev
Received: from localhost (localhost [IPv6:::1])
by earth.catern.com (Postfix) with ESMTPSA id B92F860171;
Tue, 25 Jul 2023 15:16:19 -0400 (EDT)
From: sbaugh@HIDDEN
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> (Dmitry Gutov's
message of "Tue, 25 Jul 2023 05:41:13 +0300")
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN>
<83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN>
<83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN>
<83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN>
<834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
<83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
Date: Tue, 25 Jul 2023 19:16:20 +0000 (UTC)
Message-ID: <877cqniyyk.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
X-SG-EID: =?us-ascii?Q?ZgbRq7gjGrt0q=2FPjvxk7wM0yQFRdOkTJAtEbkjCkHbJBtyc=2F2Had=2FAF8pYFPW7?=
=?us-ascii?Q?yam57WgVZDCYyakw+TpVDVX8szhc1PBagwIzxv6?=
=?us-ascii?Q?ScccyHq9TPxXj4qzq=2FdfKFVQXqNYGms7xDZf0gQ?=
=?us-ascii?Q?12Bhwv=2FYP3SgUFrLrpRJkMTJzm9YGIn=2FtKEbM2X?=
=?us-ascii?Q?IbAik9MWN5NfyDVL9haS8cz80tYxWa110gA=3D=3D?=
To: Dmitry Gutov <dmitry@HIDDEN>
X-Entity-ID: d/0VcHixlS0t7iB1YKCv4Q==
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Spam-Score: 1.2 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: Dmitry Gutov <dmitry@HIDDEN> writes: >> (my-bench 1
default-directory
"") > > (("built-in" . "Elapsed time: 1.601649s (0.709108s in 22 GCs)") >
("with-find" . "Elapsed time: 1.792383s (1.135869s in [...]
Content analysis details: (1.2 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
1.2 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in
bl.spamcop.net
[Blocked - see <https://www.spamcop.net/bl.shtml?149.72.123.24>]
-0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[149.72.123.24 listed in wl.mailspike.net]
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay
lines -0.0 T_SCC_BODY_TEXT_LINE No description available.
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>,
yantar92@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.2 (/)
Dmitry Gutov <dmitry@HIDDEN> writes:
>> (my-bench 1 default-directory "")
>
> (("built-in" . "Elapsed time: 1.601649s (0.709108s in 22 GCs)")
> ("with-find" . "Elapsed time: 1.792383s (1.135869s in 38 GCs)")
> ("with-find-p" . "Elapsed time: 1.248543s (0.682827s in 20 GCs)")
> ("with-find-sync" . "Elapsed time: 0.922291s (0.343497s in 10 GCs)"))
Tangent, but:
Ugh, wow, call-process really is a lot faster than make-process. I see
now why people disliked my idea of replacing call-process with something
based on make-process, this is a big difference...
There's zero reason it has to be so slow... maybe I should try to make a
better make-process API and implementation which is actually fast.
(without worrying about being constrained by compatibility with
something that's already dog-slow)
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 25 Jul 2023 18:41:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 25 14:41:31 2023
Received: from localhost ([127.0.0.1]:46953 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qOMyR-0000hw-6i
for submit <at> debbugs.gnu.org; Tue, 25 Jul 2023 14:41:31 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:41030)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qOMyP-0000hk-A8
for 64735 <at> debbugs.gnu.org; Tue, 25 Jul 2023 14:41:30 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qOMyJ-00012k-BS; Tue, 25 Jul 2023 14:41:23 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=zOgpaUojH+o3WAWSjrNnxm2jFWE0iIVrGL5NIq2ALGA=; b=ETeIdlPEpLQ8
ZzcrVeUyjj1YVdtoRitBso4m4JYd6uWcRok218IyM9wQrYZlAWlteEWR6/qmj3dKpGThbNONzvR8o
ERxElQ0uox+ZXprA91TIuRLuS+ug35up9rxZDy4VYqpgqhjVIywLgYJupoHw4JTTV/WVtFIS+JRE2
by+rEIbjZNUduT9YUed5VpbGTaZ6SeboftTVVePvYFJpjYWoVYE4PezSpe9oNbbw1J1PaSZuL7nJ/
QbIXD6FQ6kkSOoSJgxCsLl+Ht+6yZzHh9JE62nX3VmcjvbOSdf5iUPNn0hk/QGnIxC32BabQNdkmK
q1ogdLtCP45WpyFoo04IIA==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qOMyI-0004rK-Fn; Tue, 25 Jul 2023 14:41:22 -0400
Date: Tue, 25 Jul 2023 21:42:07 +0300
Message-Id: <83351bq1ds.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN> (message from
Dmitry Gutov on Tue, 25 Jul 2023 05:41:13 +0300)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Date: Tue, 25 Jul 2023 05:41:13 +0300
> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
> 64735 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry@HIDDEN>
>
> >> One thing to try it changing the -with-find implementation to use a
> >> synchronous call, to compare (e.g. using 'process-file'). And repeat
> >> these tests on GNU/Linux too.
> >
> > This still uses pipes, albeit without the pselect stuff.
>
> I'm attaching an extended benchmark, one that includes a "synchronous"
> implementation as well. Please give it a spin as well.
>
> Here (GNU/Linux) the reported numbers look like this:
>
> > (my-bench 1 default-directory "")
>
> (("built-in" . "Elapsed time: 1.601649s (0.709108s in 22 GCs)")
> ("with-find" . "Elapsed time: 1.792383s (1.135869s in 38 GCs)")
> ("with-find-p" . "Elapsed time: 1.248543s (0.682827s in 20 GCs)")
> ("with-find-sync" . "Elapsed time: 0.922291s (0.343497s in 10 GCs)"))
Almost no change on Windows:
(("built-in" . "Elapsed time: 1.218750s (0.078125s in 5 GCs)")
("with-find" . "Elapsed time: 8.984375s (0.109375s in 7 GCs)")
("with-find-p" . "Elapsed time: 8.718750s (0.046875s in 3 GCs)")
("with-find-sync" . "Elapsed time: 8.921875s (0.046875s in 3 GCs)"))
I'm beginning to suspect the implementation of pipes (and IPC in
general). How else can such slowdown be explained?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 25 Jul 2023 08:21:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 25 04:21:58 2023
Received: from localhost ([127.0.0.1]:44610 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qODIs-0002lR-1c
for submit <at> debbugs.gnu.org; Tue, 25 Jul 2023 04:21:58 -0400
Received: from mout01.posteo.de ([185.67.36.65]:41269)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qODIq-0002lE-FL
for 64735 <at> debbugs.gnu.org; Tue, 25 Jul 2023 04:21:57 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout01.posteo.de (Postfix) with ESMTPS id CE7F424002A
for <64735 <at> debbugs.gnu.org>; Tue, 25 Jul 2023 10:21:50 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1690273310; bh=tKB26pFHgJY4X6yxRUui3D+FBB7mc/QFxj66gJ1hNVY=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=bTGwfG85sho6USLiIg03rNXUVIZyanqU4V5H2/f4em4Kymv6otlHehLKLPb2uKgU+
g/XqYKQvPZwuhNN2Ix9LhZ78b2REuI7YQOS0Q8KQdwxK8dWQ7fYHrwLd5qJRIrJ/CU
9eh7fJQrQbhWjFHZEMtQiGmRIxZmdxB+HCuovlH5GQOVu6Lur0lmBXfeMRfDq1XMTs
7KGXm9asq+cQWDGUykTOM2qDMwYX8Td+F9o8BYra7HGVB1A+M4u6ntLN2BxtHyLuhI
pnaTvV1B7A3u+5dm58VYR5twBxZqSHSqTVUN9yDnGPsrYHy8zouWUDkO7/TM4i9hfr
Ft7ez6u3T8F5w==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R993x74d2z9s2b;
Tue, 25 Jul 2023 10:21:49 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN>
<87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
<69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
Date: Tue, 25 Jul 2023 08:22:03 +0000
Message-ID: <87351cs8no.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Dmitry Gutov <dmitry@HIDDEN> writes:
> I'm attaching an extended benchmark, one that includes a "synchronous"
> implementation as well. Please give it a spin as well.
GNU/Linux SSD
(my-bench 10 "/usr/src/linux/" "")
(("built-in" . "Elapsed time: 7.034326s (3.598539s in 14 GCs)")
("built-in no filename handler alist" . "Elapsed time: 5.907194s (3.698456s in 15 GCs)")
("with-find" . "Elapsed time: 6.078056s (4.052791s in 16 GCs)")
("with-find-p" . "Elapsed time: 4.496762s (2.739565s in 11 GCs)")
("with-find-sync" . "Elapsed time: 3.702760s (1.715160s in 7 GCs)"))
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 25 Jul 2023 02:41:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 24 22:41:28 2023
Received: from localhost ([127.0.0.1]:44431 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qO7zL-0001V2-Pu
for submit <at> debbugs.gnu.org; Mon, 24 Jul 2023 22:41:28 -0400
Received: from out3-smtp.messagingengine.com ([66.111.4.27]:59715)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <dmitry@HIDDEN>) id 1qO7zG-0001Um-OD
for 64735 <at> debbugs.gnu.org; Mon, 24 Jul 2023 22:41:26 -0400
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by mailout.nyi.internal (Postfix) with ESMTP id 88AB95C00C4;
Mon, 24 Jul 2023 22:41:17 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute4.internal (MEProxy); Mon, 24 Jul 2023 22:41:17 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
:cc:content-type:content-type:date:date:from:from:in-reply-to
:in-reply-to:message-id:mime-version:references:reply-to:sender
:subject:subject:to:to; s=fm2; t=1690252877; x=1690339277; bh=Kd
D5CaufEZfzbvLLgSfE9fmyv5EqLYURgdScJ9kdKTE=; b=nw49x9lOqHPJVGu/0W
vn4Bg5spnMUpe2mZIYelT6T4PNb9nHIVKpNL11HwHeLl0kq2TLtXEWwexqo66lTZ
KxSoJ2XpxSZRmh71Rlao+NKChcjw3ebbRfyEwGMjqSOSYChA6GJBCvpThnJn4I2x
AQLKIt4f0BE8jXZ4gVK/Ct+n5GXUNDzpCe368+dZUYEvyeEj500pXtAv9vW1luKx
5RehGpk71dM0zPNId7fGrdkxHEL8w77qdjXWIMlWWfl4tUctjmim3RGaQRkq9qmp
LXCvRIcL8lK8pp6+uzp2F2uY9Q6HODPtOXSujrjCZMt8C5zaaxeMGpXwSvrkRj0I
RWzA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:cc:content-type:content-type:date:date
:feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
:message-id:mime-version:references:reply-to:sender:subject
:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
:x-sasl-enc; s=fm3; t=1690252877; x=1690339277; bh=KdD5CaufEZfzb
vLLgSfE9fmyv5EqLYURgdScJ9kdKTE=; b=1YfQoGQ04+1oX/FM8HljVelrMPR5G
/MJmum4B88dNdr7NJAa2WbbbBCYHOh6Ka5B7FX0rGUK3oOKAkhEm3ogF5RWcfi5M
r/MPI9uva9xc57MqDow0m6tnKu5I7nFC+d9rm3YTSvMSpx4mv4w0GtF7vuZ7eUq4
4brCsKd10JrVuSJJwOpbYnB1B6vr+DROVl+ViwFCTULWRxOb1zRgp/ilxBUdykoC
mxCTNNVYWQQgO872AeG1as7D1f6suIt7aWZFS2rJ/G538PCFotXZ4J088ZJtIIHE
bVmaDuqDqY6vJk3fiYJtZLSxditXZwm+mRRFo66AGUfaXtGgVF57VxzvQ==
X-ME-Sender: <xms:TDa_ZFDAOBtHlTE1Q-R4LImjLap_m0NFq4W1biYqRBQpL4TcQw2xVA>
<xme:TDa_ZDjep4U2kFJ6rhiSfw23oxyhGW3nNjEp3TCwg5E_LFgMSg_XNT_GPievTQhXI
tlpsSTcbnZvJ7FDYfg>
X-ME-Received: <xmr:TDa_ZAlGIB0bk6kSJl1sxGZDtc5k1y5qFNqwIl--HlszgfvHl4wTq_E3tUNLxqY>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheelgdeitdcutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpegtkfffgggfuffvvehfhfgjsehmtderredtfeejnecuhfhrohhmpeffmhhithhr
hicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrthhtvg
hrnhepheetkefgteeiueejuedvtddvleetffeljeetuddvfeffjeeiheehueetffevieej
necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepughmih
htrhihsehguhhtohhvrdguvghv
X-ME-Proxy: <xmx:TDa_ZPxT2QhHl74mquHDzRC1wSNl01OchEkU55VQuBQZyVeE0N3SrA>
<xmx:TDa_ZKTukDZxUuYO8oQb_hrABguy4Gl2gDQwPe2-5Mb-8_C-KTimjQ>
<xmx:TDa_ZCaQMyrTbTamyVMdVtHd9aMDz6b6yhcbYb1eu8soWVAyGVLDBg>
<xmx:TTa_ZIeUU8zww7l9ywF1ywGBP53DoEHDKHW6u6PLQg9sv71f4v0ilQ>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon,
24 Jul 2023 22:41:15 -0400 (EDT)
Content-Type: multipart/mixed; boundary="------------jKC7NdUtouvK1j69QsODn7eD"
Message-ID: <69a98e2a-5816-d36b-9d04-8609291333cd@HIDDEN>
Date: Tue, 25 Jul 2023 05:41:13 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.13.0
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
Content-Language: en-US
To: Eli Zaretskii <eliz@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> <83a5vlsanw.fsf@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <83a5vlsanw.fsf@HIDDEN>
X-Spam-Score: -0.8 (/)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)
This is a multi-part message in MIME format.
--------------jKC7NdUtouvK1j69QsODn7eD
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
On 24/07/2023 16:26, Eli Zaretskii wrote:
>> Date: Mon, 24 Jul 2023 15:55:13 +0300
>> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
>> 64735 <at> debbugs.gnu.org
>> From: Dmitry Gutov <dmitry@HIDDEN>
>>
>>>> 1. 'find' itself is much slower there. There is room for improvement in
>>>> the port.
>>>
>>> I think it's the filesystem, not the port (which I did myself in this
>>> case).
>>
>> But directory-files-recursively goes through the same filesystem,
>> doesn't it?
>
> It does (more or less; see below). But I was not trying to explain
> why Find is slower than directory-files-recursively, I was trying to
> explain why Find on Windows is slower than Find on GNU/Linux.
Understood. But we probably don't need to worry about the differences
between platforms as much as about choosing the best option for each
platform (or not choosing the worst, at least). So I'm more interested
about how the find-based solution is more than 4x slower than the
built-in one on MS Windows.
> If you are asking why directory-files-recursively is so much faster on
> Windows than Find, then the main factors I can think about are:
>
> . IPC, at least in how we implement it in Emacs on MS-Windows, via a
> separate thread and OS-level events between them to signal that
> stuff is available for reading, whereas
> directory-files-recursively avoids this overhead completely;
> . Find uses Posix APIs: 'stat', 'chdir', 'readdir' -- which on
> Windows are emulated by wrappers around native APIs. Moreover,
> Find uses 'char *' for file names, so calling native APIs involves
> transparent conversion to UTF-16 and back, which is what native
> APIs accept and return. By contrast, Emacs on Windows calls the
> native APIs directly, and converts to UTF-16 from UTF-8, which is
> faster. (This last point also means that using Find on Windows
> has another grave disadvantage: it cannot fully support non-ASCII
> file names, only those that can be encoded by the current
> single-byte system codepage.)
I seem to remember that Wine, which also does a similar dance of
translating library and system calls, is often very close to the native
performance for many programs. So this could be a problem, but
necessarily a significant one.
Although text encoding conversion seems like a prime suspect, if the
problem is here.
>>>> 2. The process output handling is worse.
>>>
>>> Not sure what that means.
>>
>> Emacs's ability to process the output of a process on the particular
>> platform.
>>
>> You said:
>>
>> Btw, the Find command with pipe to some other program, like wc,
>> finishes much faster, like 2 to 4 times faster than when it is run
>> from find-directory-files-recursively. That's probably the slowdown
>> due to communications with async subprocesses in action.
>
> I see this slowdown on GNU/Linux as well.
>
>> One thing to try it changing the -with-find implementation to use a
>> synchronous call, to compare (e.g. using 'process-file'). And repeat
>> these tests on GNU/Linux too.
>
> This still uses pipes, albeit without the pselect stuff.
I'm attaching an extended benchmark, one that includes a "synchronous"
implementation as well. Please give it a spin as well.
Here (GNU/Linux) the reported numbers look like this:
> (my-bench 1 default-directory "")
(("built-in" . "Elapsed time: 1.601649s (0.709108s in 22 GCs)")
("with-find" . "Elapsed time: 1.792383s (1.135869s in 38 GCs)")
("with-find-p" . "Elapsed time: 1.248543s (0.682827s in 20 GCs)")
("with-find-sync" . "Elapsed time: 0.922291s (0.343497s in 10 GCs)"))
--------------jKC7NdUtouvK1j69QsODn7eD
Content-Type: text/x-emacs-lisp; charset=UTF-8; name="find-bench.el"
Content-Disposition: attachment; filename="find-bench.el"
Content-Transfer-Encoding: base64
KGRlZnVuIGZpbmQtZGlyZWN0b3J5LWZpbGVzLXJlY3Vyc2l2ZWx5IChkaXIgcmVnZXhwICZv
cHRpb25hbCBpbmNsdWRlLWRpcmVjdG9yaWVzIF9wIGZvbGxvdy1zeW1saW5rcykKICAoY2wt
YXNzZXJ0IChudWxsIF9wKSB0ICJmaW5kLWRpcmVjdG9yeS1maWxlcy1yZWN1cnNpdmVseSBj
YW4ndCBhY2NlcHQgYXJiaXRyYXJ5IHByZWRpY2F0ZXMiKQogICh3aXRoLXRlbXAtYnVmZmVy
CiAgICAoc2V0cSBjYXNlLWZvbGQtc2VhcmNoIG5pbCkKICAgIChjZCBkaXIpCiAgICAobGV0
KiAoKGNvbW1hbmQKCSAgICAoYXBwZW5kCgkgICAgIChsaXN0ICJmaW5kIiAoZmlsZS1sb2Nh
bC1uYW1lIGRpcikpCgkgICAgIChpZiBmb2xsb3ctc3ltbGlua3MKCQkgJygiLUwiKQoJICAg
ICAgICcoIiEiICIoIiAiLXR5cGUiICJsIiAiLXh0eXBlIiAiZCIgIikiKSkKCSAgICAgKHVu
bGVzcyAoc3RyaW5nLWVtcHR5LXAgcmVnZXhwKQoJICAgICAgIChsaXN0ICItcmVnZXgiIChj
b25jYXQgIi4qIiByZWdleHAgIi4qIikpKQoJICAgICAodW5sZXNzIGluY2x1ZGUtZGlyZWN0
b3JpZXMKCSAgICAgICAnKCIhIiAiLXR5cGUiICJkIikpCgkgICAgICcoIi1wcmludDAiKQoJ
ICAgICApKQoJICAgKHJlbW90ZSAoZmlsZS1yZW1vdGUtcCBkaXIpKQoJICAgKHByb2MKCSAg
ICAoaWYgcmVtb3RlCgkJKGxldCAoKHByb2MgKGFwcGx5ICMnc3RhcnQtZmlsZS1wcm9jZXNz
CgkJCQkgICAiZmluZCIgKGN1cnJlbnQtYnVmZmVyKSBjb21tYW5kKSkpCgkJICAoc2V0LXBy
b2Nlc3Mtc2VudGluZWwgcHJvYyAobGFtYmRhIChfcHJvYyBfc3RhdGUpKSkKCQkgIChzZXQt
cHJvY2Vzcy1xdWVyeS1vbi1leGl0LWZsYWcgcHJvYyBuaWwpCgkJICBwcm9jKQoJICAgICAg
KG1ha2UtcHJvY2VzcyA6bmFtZSAiZmluZCIgOmJ1ZmZlciAoY3VycmVudC1idWZmZXIpCgkJ
CSAgICA6Y29ubmVjdGlvbi10eXBlICdwaXBlCgkJCSAgICA6bm9xdWVyeSB0CgkJCSAgICA6
c2VudGluZWwgKGxhbWJkYSAoX3Byb2MgX3N0YXRlKSkKCQkJICAgIDpjb21tYW5kIGNvbW1h
bmQpKSkpCiAgICAgICh3aGlsZSAoYWNjZXB0LXByb2Nlc3Mtb3V0cHV0IHByb2MpKQogICAg
ICAobGV0ICgoc3RhcnQgKGdvdG8tY2hhciAocG9pbnQtbWluKSkpIHJldCkKCSh3aGlsZSAo
c2VhcmNoLWZvcndhcmQgIlwwIiBuaWwgdCkKCSAgKHB1c2ggKGNvbmNhdCByZW1vdGUgKGJ1
ZmZlci1zdWJzdHJpbmctbm8tcHJvcGVydGllcyBzdGFydCAoMS0gKHBvaW50KSkpKSByZXQp
CgkgIChzZXRxIHN0YXJ0IChwb2ludCkpKQoJcmV0KSkpKQoKKGRlZnVuIGZpbmQtZGlyZWN0
b3J5LWZpbGVzLXJlY3Vyc2l2ZWx5LTIgKGRpciByZWdleHAgJm9wdGlvbmFsIGluY2x1ZGUt
ZGlyZWN0b3JpZXMgX3AgZm9sbG93LXN5bWxpbmtzKQogIChjbC1hc3NlcnQgKG51bGwgX3Ap
IHQgImZpbmQtZGlyZWN0b3J5LWZpbGVzLXJlY3Vyc2l2ZWx5IGNhbid0IGFjY2VwdCBhcmJp
dHJhcnkgcHJlZGljYXRlcyIpCiAgKGNsLWFzc2VydCAobm90IChmaWxlLXJlbW90ZS1wIGRp
cikpKQogIChsZXQqIChidWZmZXJlZAogICAgICAgICByZXN1bHQKICAgICAgICAgKHByb2MK
CSAgKG1ha2UtcHJvY2VzcwogICAgICAgICAgIDpuYW1lICJmaW5kIiA6YnVmZmVyIG5pbAoJ
ICAgOmNvbm5lY3Rpb24tdHlwZSAncGlwZQoJICAgOm5vcXVlcnkgdAoJICAgOnNlbnRpbmVs
IChsYW1iZGEgKF9wcm9jIF9zdGF0ZSkpCiAgICAgICAgICAgOmZpbHRlciAobGFtYmRhIChw
cm9jIGRhdGEpCiAgICAgICAgICAgICAgICAgICAgIChsZXQgKChzdGFydCAwKSkKICAgICAg
ICAgICAgICAgICAgICAgICAod2hlbi1sZXQgKGVuZCAoc3RyaW5nLXNlYXJjaCAiXDAiIGRh
dGEgc3RhcnQpKQogICAgICAgICAgICAgICAgICAgICAgICAgKHB1c2ggKGNvbmNhdCBidWZm
ZXJlZCAoc3Vic3RyaW5nIGRhdGEgc3RhcnQgZW5kKSkgcmVzdWx0KQogICAgICAgICAgICAg
ICAgICAgICAgICAgKHNldHEgYnVmZmVyZWQgIiIpCiAgICAgICAgICAgICAgICAgICAgICAg
ICAoc2V0cSBzdGFydCAoMSsgZW5kKSkKICAgICAgICAgICAgICAgICAgICAgICAgICh3aGls
ZS1sZXQgKChlbmQgKHN0cmluZy1zZWFyY2ggIlwwIiBkYXRhIHN0YXJ0KSkpCiAgICAgICAg
ICAgICAgICAgICAgICAgICAgIChwdXNoIChzdWJzdHJpbmcgZGF0YSBzdGFydCBlbmQpIHJl
c3VsdCkKICAgICAgICAgICAgICAgICAgICAgICAgICAgKHNldHEgc3RhcnQgKDErIGVuZCkp
KSkKICAgICAgICAgICAgICAgICAgICAgICAoc2V0cSBidWZmZXJlZCAoY29uY2F0IGJ1ZmZl
cmVkIChzdWJzdHJpbmcgZGF0YSBzdGFydCkpKSkpCgkgICA6Y29tbWFuZCAoYXBwZW5kCgkg
ICAgICAgICAgICAgKGxpc3QgImZpbmQiIChmaWxlLWxvY2FsLW5hbWUgZGlyKSkKCSAgICAg
ICAgICAgICAoaWYgZm9sbG93LXN5bWxpbmtzCgkJICAgICAgICAgJygiLUwiKQoJICAgICAg
ICAgICAgICAgJygiISIgIigiICItdHlwZSIgImwiICIteHR5cGUiICJkIiAiKSIpKQoJICAg
ICAgICAgICAgICh1bmxlc3MgKHN0cmluZy1lbXB0eS1wIHJlZ2V4cCkKCSAgICAgICAgICAg
ICAgIChsaXN0ICItcmVnZXgiIChjb25jYXQgIi4qIiByZWdleHAgIi4qIikpKQoJICAgICAg
ICAgICAgICh1bmxlc3MgaW5jbHVkZS1kaXJlY3RvcmllcwoJICAgICAgICAgICAgICAgJygi
ISIgIi10eXBlIiAiZCIpKQoJICAgICAgICAgICAgICcoIi1wcmludDAiKQoJICAgICAgICAg
ICAgICkpKSkKICAgICh3aGlsZSAoYWNjZXB0LXByb2Nlc3Mtb3V0cHV0IHByb2MpKQogICAg
cmVzdWx0KSkKCihkZWZ1biBmaW5kLWRpcmVjdG9yeS1maWxlcy1yZWN1cnNpdmVseS0zIChk
aXIgcmVnZXhwICZvcHRpb25hbCBpbmNsdWRlLWRpcmVjdG9yaWVzIF9wIGZvbGxvdy1zeW1s
aW5rcykKICAoY2wtYXNzZXJ0IChudWxsIF9wKSB0ICJmaW5kLWRpcmVjdG9yeS1maWxlcy1y
ZWN1cnNpdmVseSBjYW4ndCBhY2NlcHQgYXJiaXRyYXJ5IHByZWRpY2F0ZXMiKQogIChjbC1h
c3NlcnQgKG5vdCAoZmlsZS1yZW1vdGUtcCBkaXIpKSkKICAobGV0ICgoYXJncyBgKCwoZmls
ZS1sb2NhbC1uYW1lIGRpcikKCSAgICAgICAgLEAoaWYgZm9sbG93LXN5bWxpbmtzCgkJICAg
ICAgJygiLUwiKQoJICAgICAgICAgICAgJygiISIgIigiICItdHlwZSIgImwiICIteHR5cGUi
ICJkIiAiKSIpKQoJICAgICAgICAsQCh1bmxlc3MgKHN0cmluZy1lbXB0eS1wIHJlZ2V4cCkK
CSAgICAgICAgICAgIChsaXN0ICItcmVnZXgiIChjb25jYXQgIi4qIiByZWdleHAgIi4qIikp
KQoJICAgICAgICAsQCh1bmxlc3MgaW5jbHVkZS1kaXJlY3RvcmllcwoJICAgICAgICAgICAg
JygiISIgIi10eXBlIiAiZCIpKQoJICAgICAgICAiLXByaW50MCIpKSkKICAgICh3aXRoLXRl
bXAtYnVmZmVyCiAgICAgIChsZXQgKChzdGF0dXMgKGFwcGx5ICMncHJvY2Vzcy1maWxlCiAg
ICAgICAgICAgICAgICAgICAgICAgICAgICJmaW5kIgogICAgICAgICAgICAgICAgICAgICAg
ICAgICBuaWwKICAgICAgICAgICAgICAgICAgICAgICAgICAgdAogICAgICAgICAgICAgICAg
ICAgICAgICAgICBuaWwKICAgICAgICAgICAgICAgICAgICAgICAgICAgYXJncykpCiAgICAg
ICAgICAgIChwdCAocG9pbnQtbWluKSkKICAgICAgICAgICAgcmVzKQogICAgICAgICh1bmxl
c3MgKHplcm9wIHN0YXR1cykKICAgICAgICAgIChlcnJvciAiTGlzdGluZyBmYWlsZWQiKSkK
ICAgICAgICAoZ290by1jaGFyIChwb2ludC1taW4pKQogICAgICAgICh3aGlsZSAoc2VhcmNo
LWZvcndhcmQgIlwwIiBuaWwgdCkKICAgICAgICAgIChwdXNoIChidWZmZXItc3Vic3RyaW5n
LW5vLXByb3BlcnRpZXMgcHQgKDEtIChwb2ludCkpKQogICAgICAgICAgICAgICAgcmVzKQog
ICAgICAgICAgKHNldHEgcHQgKHBvaW50KSkpCiAgICAgICAgcmVzKSkpKQoKKGRlZnVuIG15
LWJlbmNoIChjb3VudCBwYXRoIHJlZ2V4cCkKICAoc2V0cSBwYXRoIChleHBhbmQtZmlsZS1u
YW1lIHBhdGgpKQogIDs7IChsZXQgKChvbGQgKGRpcmVjdG9yeS1maWxlcy1yZWN1cnNpdmVs
eSBwYXRoIHJlZ2V4cCkpCiAgOzsgICAgICAgKG5ldyAoZmluZC1kaXJlY3RvcnktZmlsZXMt
cmVjdXJzaXZlbHktMyBwYXRoIHJlZ2V4cCkpKQogIDs7ICAgKGRvbGlzdCAocGF0aCBvbGQp
CiAgOzsgICAgICh1bmxlc3MgKG1lbWJlciBwYXRoIG5ldykgKGVycm9yICIhICVzIG5vdCBp
biIgcGF0aCkpKQogIDs7ICAgKGRvbGlzdCAocGF0aCBuZXcpCiAgOzsgICAgICh1bmxlc3Mg
KG1lbWJlciBwYXRoIG9sZCkgKGVycm9yICIhISAlcyBub3QgaW4iIHBhdGgpKSkpCiAgKGxp
c3QKICAgKGNvbnMgImJ1aWx0LWluIiAoYmVuY2htYXJrIGNvdW50IChsaXN0ICdkaXJlY3Rv
cnktZmlsZXMtcmVjdXJzaXZlbHkgcGF0aCByZWdleHApKSkKICAgKGNvbnMgIndpdGgtZmlu
ZCIgKGJlbmNobWFyayBjb3VudCAobGlzdCAnZmluZC1kaXJlY3RvcnktZmlsZXMtcmVjdXJz
aXZlbHkgcGF0aCByZWdleHApKSkKICAgKGNvbnMgIndpdGgtZmluZC1wIiAoYmVuY2htYXJr
IGNvdW50IChsaXN0ICdmaW5kLWRpcmVjdG9yeS1maWxlcy1yZWN1cnNpdmVseS0yIHBhdGgg
cmVnZXhwKSkpCiAgIChjb25zICJ3aXRoLWZpbmQtc3luYyIgKGJlbmNobWFyayBjb3VudCAo
bGlzdCAnZmluZC1kaXJlY3RvcnktZmlsZXMtcmVjdXJzaXZlbHktMyBwYXRoIHJlZ2V4cCkp
KSkpCg==
--------------jKC7NdUtouvK1j69QsODn7eD--
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 24 Jul 2023 13:25:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 24 09:25:55 2023
Received: from localhost ([127.0.0.1]:42167 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNvZS-0007f3-T2
for submit <at> debbugs.gnu.org; Mon, 24 Jul 2023 09:25:55 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:36468)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qNvZQ-0007ep-Du
for 64735 <at> debbugs.gnu.org; Mon, 24 Jul 2023 09:25:53 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNvZK-0007Rn-AS; Mon, 24 Jul 2023 09:25:46 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=H1lJ48cWW5Ttpe9JSy1LcYxPiI0BF08jN5Bcy/d9fHA=; b=fZikmWSoqQE9
+WZLdKcjYBf+EgULAO+HnLS41hKKEcmOZEFcQcKbYHS7EEjuiMnLfW+Uy+wuLMUFXWwCW1Wnduvro
m7aQfKuWESRdNpOU1Tc8ESj6EnMVFo6AqsUMpAJsscYBiou4+DPGlNa1gcRIrAsxPWOji63vR+cE9
dkbyGPFvj4F35bLZtSDdXO/p0k1oKLq2tdopJLnUuYnMymcWwRY9m1wE9nFxapYkebsS01AR1eCbS
qvfT4j2IWbWothjagQ8yoF6H8PzOJyaWhAuOuOX0KfRFzKKJJokH1K/pqUbZy288s+c6sDELb35Lb
mel4fn/DsWjJ5XJKeAQhqw==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNvZJ-0007dw-BU; Mon, 24 Jul 2023 09:25:45 -0400
Date: Mon, 24 Jul 2023 16:26:27 +0300
Message-Id: <83a5vlsanw.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> (message from
Dmitry Gutov on Mon, 24 Jul 2023 15:55:13 +0300)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN>
<c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN>
<2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN>
<83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN>
<937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Date: Mon, 24 Jul 2023 15:55:13 +0300
> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
> 64735 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry@HIDDEN>
>
> >> 1. 'find' itself is much slower there. There is room for improvement in
> >> the port.
> >
> > I think it's the filesystem, not the port (which I did myself in this
> > case).
>
> But directory-files-recursively goes through the same filesystem,
> doesn't it?
It does (more or less; see below). But I was not trying to explain
why Find is slower than directory-files-recursively, I was trying to
explain why Find on Windows is slower than Find on GNU/Linux.
If you are asking why directory-files-recursively is so much faster on
Windows than Find, then the main factors I can think about are:
. IPC, at least in how we implement it in Emacs on MS-Windows, via a
separate thread and OS-level events between them to signal that
stuff is available for reading, whereas
directory-files-recursively avoids this overhead completely;
. Find uses Posix APIs: 'stat', 'chdir', 'readdir' -- which on
Windows are emulated by wrappers around native APIs. Moreover,
Find uses 'char *' for file names, so calling native APIs involves
transparent conversion to UTF-16 and back, which is what native
APIs accept and return. By contrast, Emacs on Windows calls the
native APIs directly, and converts to UTF-16 from UTF-8, which is
faster. (This last point also means that using Find on Windows
has another grave disadvantage: it cannot fully support non-ASCII
file names, only those that can be encoded by the current
single-byte system codepage.)
> >> 2. The process output handling is worse.
> >
> > Not sure what that means.
>
> Emacs's ability to process the output of a process on the particular
> platform.
>
> You said:
>
> Btw, the Find command with pipe to some other program, like wc,
> finishes much faster, like 2 to 4 times faster than when it is run
> from find-directory-files-recursively. That's probably the slowdown
> due to communications with async subprocesses in action.
I see this slowdown on GNU/Linux as well.
> One thing to try it changing the -with-find implementation to use a
> synchronous call, to compare (e.g. using 'process-file'). And repeat
> these tests on GNU/Linux too.
This still uses pipes, albeit without the pselect stuff.
> >> 3. Something particular to the project being used for the test.
> >
> > I don't think I understand this one.
>
> This described the possibility where the disparity between the
> implementations' runtimes was due to something unusual in the project
> structure, if you tested different projects between Windows and
> GNU/Linux, making direct comparison less useful. It's the least likely
> cause, but still sometimes a possibility.
I have on my Windows system a d:/usr/share tree that is very similar
to (albeit somewhat smaller than) a typical /usr/share tree on Posix
systems. I tried with that as well, and the results were similar.
> > The ezwinports is the version I'm using here. But maybe someone came
> > up with a better one: after all, I did my port many years ago (because
> > the native ports available back then were abysmally slow).
>
> We should also look at the exact numbers. If you say that "| wc"
> invocation is 2-4x faster than what's reported in the benchmark, then it
> takes about 2-4 seconds. Which is still oddly slower than your reported
> numbers for directory-files-recursively.
Yes, so there are additional factors at work, at least with this port
of Find.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 24 Jul 2023 12:55:25 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 24 08:55:25 2023 Received: from localhost ([127.0.0.1]:42086 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNv5x-0006rS-CP for submit <at> debbugs.gnu.org; Mon, 24 Jul 2023 08:55:25 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:44141) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qNv5u-0006rA-8h for 64735 <at> debbugs.gnu.org; Mon, 24 Jul 2023 08:55:24 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 1C2775C0182; Mon, 24 Jul 2023 08:55:17 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 24 Jul 2023 08:55:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690203317; x=1690289717; bh=+UK8rdOiJ+qouJOeYUXXhbJJVrZFhFeylYi g/OjKxVU=; b=JVx1O7Uq+n65oMZb6YgbjYlePyHK6/BW08zYocmm6tcN1rOndAR xQy+t70v80vtwIO2eCk0w6g4nGVpCommCcGjW9fz1qtAGZu7WllKSutRRE5DpJ0r LW1GoHJhv8vJ7TEztPmrlktNDAeIskXQba3zeF7k3T/bW8TBVPOdIxCrGS9UI1VN /8yLurUX4RRL09DzLYOolvMoz3oKZlXjESqW7YL3/xB9vQ8psEdE5ap2W2Ho+VOb AwwbqpgWh25XyN7okRnTQDNG6VSqUF0XJKD0yK4NyrDsvEGZBNwv1MNtj+ANhVoV uqzKLl0qrKJytf3FDt1h+aglu8cFezlaXZA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690203317; x=1690289717; bh=+UK8rdOiJ+qouJOeYUXXhbJJVrZFhFeylYi g/OjKxVU=; b=nX+RgStoztPdk+MDbt2OiY3I66SJh0/jZ9zwKhCfL5IBgEiLgfU kWtta3PIPHycEqQmP0Hi5WNOf0h/ITLM3JxmG7sClRDrnL5iO1HBBVaj/36T3vP6 ggf/R5DMv+wFdo94eJYac8jo/0hsUrfDY6Hh8u3jwA9y1hiC027Ek4uGR/G/MXvn lMqwR82/vwlW0GG2lIiu6FoHO7RcpMngfoBBQbOESW3WyYiNSYLTN/WnNsTPZuAg jzpuLyKE+2nmqzYCGKrv6hAjf8VVCgCOyAAF5hC7OEmFEMPkAutaRKmFWdQG9SJW MoSLYoxSfJb/uGNOzNHI+DM4XABulof7Lig== X-ME-Sender: <xms:tHS-ZLvTYh1hkXMHy4N-0EEWhIzuiBnhnc04nbCavBFC6rtLaWkAyw> <xme:tHS-ZMdEqO-MLRRGOmHDwPEZOVxmWPf2Z2zxB8UHvwj0fI_3kel-Yz3bNPzA3365Z WBhfTjApFK192_YMXo> X-ME-Received: <xmr:tHS-ZOxqRCmR7uyfuBZ0L5HVLIJ1Hu7OvXIL5JVSphE9yp-IrUAyve3kKudKbbY> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheekgdehiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:tHS-ZKP6pIgYynlXjULXdtVCI0Kay16QasasA5DOACv6ND9hmxJquQ> <xmx:tHS-ZL_t9At6xRVT1LXX2u4DOFKwF945Cb0OoIfJdE8adWBvQqbDig> <xmx:tHS-ZKVw3XNRC7775_E2gXtBZRsYYbV79cdboy38nrV3EDPAW0J_Bg> <xmx:tXS-ZOZlAyFFj6wjbACp6EySokauYvk7GVTBhKih5T4p8zsZJsUVhw> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 24 Jul 2023 08:55:15 -0400 (EDT) Message-ID: <937c3b8e-7742-91b7-c2cf-4cadd0782f0c@HIDDEN> Date: Mon, 24 Jul 2023 15:55:13 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> <834jlttv1p.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <834jlttv1p.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 24/07/2023 14:20, Eli Zaretskii wrote: >> Date: Sun, 23 Jul 2023 22:27:26 +0300 >> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov <dmitry@HIDDEN> >> >> On 23/07/2023 20:56, Eli Zaretskii wrote: >>>> And, ideally, do all the relevant benchmarking when proposing the change. >>> Of course. Although the benchmarks until now already show quite a >>> variability. >> >> Speaking of your MS Windows results that are unflattering to 'find', it >> might be worth it to do a more varied comparison, to determine the >> OS-specific bottleneck. >> >> Off the top of my head, here are some possibilities: >> >> 1. 'find' itself is much slower there. There is room for improvement in >> the port. > > I think it's the filesystem, not the port (which I did myself in this > case). But directory-files-recursively goes through the same filesystem, doesn't it? > But I'd welcome similar tests on other Windows systems with > other ports of Find. Just remember to measure this particular > benchmark, not just Find itself from the shell, as the times are very > different (as I reported up-thread). Concur. >> 2. The process output handling is worse. > > Not sure what that means. Emacs's ability to process the output of a process on the particular platform. You said: Btw, the Find command with pipe to some other program, like wc, finishes much faster, like 2 to 4 times faster than when it is run from find-directory-files-recursively. That's probably the slowdown due to communications with async subprocesses in action. One thing to try it changing the -with-find implementation to use a synchronous call, to compare (e.g. using 'process-file'). And repeat these tests on GNU/Linux too. That would help us gauge the viability of using an asynchronous process to get the file listing. But also, if one was just looking into reimplementing directory-files-recursively using 'find' (to create an endpoint with swappable implementations, for example), 'process-file' is a suitable substitute because the original is also currently synchronous. >> 3. Something particular to the project being used for the test. > > I don't think I understand this one. This described the possibility where the disparity between the implementations' runtimes was due to something unusual in the project structure, if you tested different projects between Windows and GNU/Linux, making direct comparison less useful. It's the least likely cause, but still sometimes a possibility. >> To look into the possibility #1, you can try running the same command in >> the terminal with the output to NUL and comparing the runtime to what's >> reported in the benchmark. > > Output to the null device is a bad idea, as (AFAIR) Find is clever > enough to detect that and do nothing. I run "find | wc" instead, and > already reported that it is much faster. Now I see it, thanks. >> I actually remember, from my time on MS Windows about 10 years ago, that >> some older ports of 'find' and/or 'grep' did have performance problems, >> but IIRC ezwinports contained the improved versions. > > The ezwinports is the version I'm using here. But maybe someone came > up with a better one: after all, I did my port many years ago (because > the native ports available back then were abysmally slow). We should also look at the exact numbers. If you say that "| wc" invocation is 2-4x faster than what's reported in the benchmark, then it takes about 2-4 seconds. Which is still oddly slower than your reported numbers for directory-files-recursively.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 24 Jul 2023 11:20:28 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 24 07:20:28 2023 Received: from localhost ([127.0.0.1]:41902 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNtc4-00054y-AL for submit <at> debbugs.gnu.org; Mon, 24 Jul 2023 07:20:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51848) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNtc0-00054j-MH for 64735 <at> debbugs.gnu.org; Mon, 24 Jul 2023 07:20:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNtbu-0000Ew-Go; Mon, 24 Jul 2023 07:20:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=FR7LxRVIVf1c4Yu40K/82nWcs2J1WHFdSaBPl4M5EMo=; b=Q++I7mtygtr1 R94QZt2UvEfTmogOVS0Wgk+rxFlo8N/OL18dTXR/pZJ5lfUu4vydgspd/+8ma471i3d2bLm5VpRAB At/fz6DW0LzHsDnKNK1qxgVM8PzbrRumQSpEvhhV0nEi9Z6j7WhCY94aPJLG6/yQWRkhj4ryUWrrn 1P2Z8XNUeVjnrTmsSGxdoTOjMCAE7A73A+3933ociQ+04t2PGslHEY6je0JIA7ywmffRwLR3q9M2+ 2RHk9ExwuvMPibxegXjsLlfpj6Np2F/QulkfTpJ1/2purjRwwdRkIjoPo6Vr24KxkoVyTYW2xGlzh OdhfcXmyph/9dHRHvF4CWQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNtbj-000096-LU; Mon, 24 Jul 2023 07:20:18 -0400 Date: Mon, 24 Jul 2023 14:20:50 +0300 Message-Id: <834jlttv1p.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> (message from Dmitry Gutov on Sun, 23 Jul 2023 22:27:26 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Sun, 23 Jul 2023 22:27:26 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > On 23/07/2023 20:56, Eli Zaretskii wrote: > >> And, ideally, do all the relevant benchmarking when proposing the change. > > Of course. Although the benchmarks until now already show quite a > > variability. > > Speaking of your MS Windows results that are unflattering to 'find', it > might be worth it to do a more varied comparison, to determine the > OS-specific bottleneck. > > Off the top of my head, here are some possibilities: > > 1. 'find' itself is much slower there. There is room for improvement in > the port. I think it's the filesystem, not the port (which I did myself in this case). But I'd welcome similar tests on other Windows systems with other ports of Find. Just remember to measure this particular benchmark, not just Find itself from the shell, as the times are very different (as I reported up-thread). > 2. The process output handling is worse. Not sure what that means. > 3. Something particular to the project being used for the test. I don't think I understand this one. > To look into the possibility #1, you can try running the same command in > the terminal with the output to NUL and comparing the runtime to what's > reported in the benchmark. Output to the null device is a bad idea, as (AFAIR) Find is clever enough to detect that and do nothing. I run "find | wc" instead, and already reported that it is much faster. > I actually remember, from my time on MS Windows about 10 years ago, that > some older ports of 'find' and/or 'grep' did have performance problems, > but IIRC ezwinports contained the improved versions. The ezwinports is the version I'm using here. But maybe someone came up with a better one: after all, I did my port many years ago (because the native ports available back then were abysmally slow).
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 24 Jul 2023 09:32:10 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 24 05:32:10 2023 Received: from localhost ([127.0.0.1]:41778 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNrvC-00022o-RY for submit <at> debbugs.gnu.org; Mon, 24 Jul 2023 05:32:10 -0400 Received: from mout.gmx.net ([212.227.17.21]:58251) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qNrv8-00022H-AQ for 64735 <at> debbugs.gnu.org; Mon, 24 Jul 2023 05:32:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1690191109; x=1690795909; i=michael.albinus@HIDDEN; bh=4Gh4/eltQqhWdolKvPTExVf3zsQY4Qle5Xzn8RSisoc=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=tITyChjwzJIi07SUoNi6srb1bYQt034lUusidH9zz00abb6eyyl+bmKQ+nSzoDGNrhmIRLV gqd5gixVWKu7aG8Tos4vbapoTrovs0STpJojFzExHEOQL1+vCdZumD41pg1rVKVuDoP2FzP0+ wZAXa/CiO/PAVqtvx35fAxY3PntidPoBQcqJoU8UFMXb54MEeeZs5R7HE8lyyI6gtlF4yxRsv /MeeVjbzsJLUFeamKWEzu4bCQpNQJBcJYEAkCfjsdUxdRaR8e0/+0LYFRz81TBKQ1vGDhbH/Q ArPR3qzjBnvkE2McT0Kz3qrw24IxNTxp6sWExJxHHvWNYdgoy2Hw== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MdebB-1pp5CG3BxZ-00ZeEG; Mon, 24 Jul 2023 11:31:49 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87bkg13eig.fsf@localhost> (Ihor Radchenko's message of "Mon, 24 Jul 2023 08:22:31 +0000") References: <iermszrwqj4.fsf@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> <87pm4jrxra.fsf@localhost> <871qgyzw1v.fsf@HIDDEN> <87mszl3gpj.fsf@localhost> <871qgxyc2n.fsf@HIDDEN> <87bkg13eig.fsf@localhost> Date: Mon, 24 Jul 2023 11:31:48 +0200 Message-ID: <87v8e9wt8b.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:y75v4HIHR7VRZT1UKzxkxMrAIUQglKuuhVGuKmCAgTtyaoQ7xTq 1kVaweepLaz4FAejv3ykZTvRniIJ5T5jj6FSV41beBoOuOPYN86ku3XYH1LFy14Tli8XCkE QypcIl1JyJPezr6JZ58VVT4f9187zectrYcewZTypR/wVDdWb/96UvaLFi9yh5LN6a5vNuF lAaBGisVaOL0U7KowFHmw== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:KH9YYJVTzMw=;8TShp4QnKk6kqxrfHS3AFPTJA/0 UaNbFi7jwx4am+r4qkyGSTJPMufXfJc6SgxTTMDSTPoVfiTCEgs6Er5yVMwp4L2T2LASHgpHn exvvVd7WwH+/EhkRCuG5Xocp+5oVu6SdY1XsFsUAnqIwI0NJZIE5jno1l1zolVKmcbsIsZkvu aVdgmF257s94Uu5lkYKqh5ZxbjxQiI4mdpDDwRI6yTgMb24UnKzX4uPz2wfjXA8fP4LljXsmT aE4ZPdnaifaArbZirlFrZMFJsnlnEeV5maD1npxw3rftDdg3O/sFVwE67VjaKa9eZh37KC/e+ 2OSEojRnNRogBxcwZso9uFN0ZeoLXiQI0DOg6JVcIrR9rWCBzcZbnLrx8oMuBaOsCf2s9Zj0v CZfBo7RHdpnRYl+TdMdPT3kWt8l3DJy2ZpYiVhE3MD+5cL/MjR6wPoN3j+0H5W1icYE+jHYce qonEHggovhtSExkCbBMCjUaSBpBV625qlOLhDYmG80iyxhF7tJb5j1KBn0dbWK/E6eFZsNi3+ 2OcZeR7gTC1TIlKFuJYpFanqs7MT3qUdoXa7Z9taeBhfqRyEjQAve0AV+Afrng/wx3v1OmSzO 4C96osvVpl6QyKEu0DPJXPiddzF+k7P9YrHEDL8fV6zC2/iMCS4bOgzMB6sA1NmS57u/3Vgr9 vgId/mqtKTa2r1zwQXOwEAbchVRdVxafA97mKvlhKGetIRCa2HKuRsvM7oh23SvfY3snY1Nvc e+5wgPxlIvX3Q7wyHnJ/EHr9BlvKq4BQGFXlL/mAUn+J5aWi+t8D094YIxN027+CYe8xwGcTn ddpi7yBjNKRRpXuHss7Enk2bPEUDpVS1VFRiZPWvu/+quukMDyNxhgsqvGx8AH1h6X3CBHNnB DXmImxbk7SDKI610mzJGw06SNHZUzmfRF+bgdzjBGgVao3g8gG8pzmQgd6IO3sUEaOpkGHN4f KwcgmQ== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Ihor Radchenko <yantar92@HIDDEN> writes: Hi Ihor, > Why so? `directory-files-recursively' is already supported by Tramp via > `directory-files'. It isn't supported by Tramp yet. Tramp hasn't heard ever about. > But at least for some backends `directory-files-recursively' may be > implemented more efficiently. If other backends do not implement it, > `directory-files' will be used. Of course, and I did propose to add it. I just wanted to avoid an inflation of proposals for primitive operations being supported by Tramp. And yes, not all backends need to implement an own version of `directory-files-recursively'. But this could happen for other primitive operations, so we must always think about whether it is worth to add a primitive to Tramp. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 24 Jul 2023 08:22:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 24 04:22:26 2023 Received: from localhost ([127.0.0.1]:41719 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNqpm-00007Z-5b for submit <at> debbugs.gnu.org; Mon, 24 Jul 2023 04:22:26 -0400 Received: from mout01.posteo.de ([185.67.36.65]:39523) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNqpk-00007F-5X for 64735 <at> debbugs.gnu.org; Mon, 24 Jul 2023 04:22:24 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 99C39240029 for <64735 <at> debbugs.gnu.org>; Mon, 24 Jul 2023 10:22:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690186938; bh=t3nN9O44R+GphrJ/Vetgc1UsnNZoruTezJzMr49lUnw=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=TfpRFw6T5pwzqw6aHFmZjqM2IteuI0ekI2vq9SeOWCCeAoILEhEZA9ohNYvqQHJuZ hKS0/kuEZbaza9z2XplRhjmOsxA3iKP66ZRRQhMG4TEIhputICymoamEa80nw2MUzC PJ5Rw8it/vtdmBd5VJbi5mPYNdlXydQ8eDek3WtWT3q2rKwQ0CfsGmF5k8H+L3oeSH 7dSaJ7gjaUoFpbVm6uPw/WBJhY3SrczPtm0st1rDakhQDv2H3fI/Py/u2g51Uo3xsM 2q3OyPmiLGHW1SVC78EeSsK9kb+I8XiWBIgmYEefqxwABvxe7e3+6RDAlQZPY0pUDL iRWSl1CFGbFqQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R8Y6x5Mp6z9s6g; Mon, 24 Jul 2023 10:22:17 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <871qgxyc2n.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> <87pm4jrxra.fsf@localhost> <871qgyzw1v.fsf@HIDDEN> <87mszl3gpj.fsf@localhost> <871qgxyc2n.fsf@HIDDEN> Date: Mon, 24 Jul 2023 08:22:31 +0000 Message-ID: <87bkg13eig.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Michael Albinus <michael.albinus@HIDDEN> writes: >> So, it is doable, and not difficult. Good to know. > > Technically it isn't difficult. But don't forget: > > - We support already ~80 primitive operations. > > - A new primitive operation must be handled by all Tramp backends, which > could require up to 10 different implementations. Why so? `directory-files-recursively' is already supported by Tramp via `directory-files'. But at least for some backends `directory-files-recursively' may be implemented more efficiently. If other backends do not implement it, `directory-files' will be used. > - I'm the only Tramp maintainer, for decades. I hope that the above approach with only some backends implementing such support will not add too much of maintenance burden. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 24 Jul 2023 07:59:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 24 03:59:53 2023 Received: from localhost ([127.0.0.1]:41688 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNqTx-0007vj-CG for submit <at> debbugs.gnu.org; Mon, 24 Jul 2023 03:59:53 -0400 Received: from mout.gmx.net ([212.227.15.18]:49967) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qNqTs-0007vQ-Gt for 64735 <at> debbugs.gnu.org; Mon, 24 Jul 2023 03:59:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1690185570; x=1690790370; i=michael.albinus@HIDDEN; bh=NrcYiXhIBO5kFifKga3t0wzZvLSK3OvxqhLKNVE6hPE=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=DjCmnmqur4BrqAE9Z6FmV80ASfjgDOEFuU18JZAu0gh2UP11jpEp5JTj6/gHSno7QEZYrOj DF7gZjbbw+n1VzaV22uD01SeXVDpE7wFATl1MZqENLOipC5Ldz1FUwmCOxYP0YwzUeTKr7ERh VMFpfTcCUPm7wp6moMX1cdB+bnqEuq1kpPr5e2gWbbST+NJo5QXZMNBhgJ8Glb2fuzx+UtmDJ t8eDg5AkHWlO9kxGC4qq8hF5wMVN5ma0wANiXDkBKlx5BoYcwTvygh/bE8UBEsG0mTuEn4ju7 J3PmNZWbkWgNXj0jTLhMCinV4i5pCW7BX0Rq4hUcEFxJVBN+8fbw== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MryT9-1pajsV1pOX-00nvdL; Mon, 24 Jul 2023 09:59:30 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87mszl3gpj.fsf@localhost> (Ihor Radchenko's message of "Mon, 24 Jul 2023 07:35:04 +0000") References: <iermszrwqj4.fsf@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> <87pm4jrxra.fsf@localhost> <871qgyzw1v.fsf@HIDDEN> <87mszl3gpj.fsf@localhost> Date: Mon, 24 Jul 2023 09:59:28 +0200 Message-ID: <871qgxyc2n.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:R2RVGbsVISVb4UN2+3ZNwMXNFzJtLCNuVlwW06WXf1teeOh49lN wLZDxygE+Kfa0RhA4PXHVtvv3LX439wtqfDpNoOhh0udSQjYwygK/y/h+GpeLRFoARo4T30 utGs//jEQSu3kz595t4XUawMzrvBLWqs1AIFjNDWDPLcXjm0M7hDFZNRT5HNz7y5ZAduV4B STECC87wnDC8tih+VK+2w== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:K2aHPcvDJ0I=;Lkv1uknEpgucN+RDWWF7irbiKrV 68OCuu7hCHMZ4Ye0k0UOyJAnfg0tZkcLrnCgBFuEKj4EDj0rZxwInG9JtKRfDS5acb7oy14ZX 4cBnP2uYYuqNZFaQ+0HC3ydcEE/llEv2PFAwgRb+klMoju1vOAeAnZuxfNOpaqJndg+uTCVlo EgaE5vtcOrgeXIvyfdMSQRx90mouG1J4V+La7KO7mqdYqHG6c/sjJBGtSEhalPKZEFSf5+HYB knbRxp00s7cw4gNSS7H1NI3K3T3EyejdPjssLWgUk6fBZ7SeoRniuFXd0j39e0rwCrYtCOhQv i/kHdmYZMA3E0+r82+U+0NHpteFJe0xnDRNezvXSyz1r/fyk6Ww8iTYcEcVz3jP4wx9ON4kGc B+2GawJN0Qk1hk2FYL6qr63OY48tExGLbKB5WwpzvzaeUHMsBkorJlD6hEbJSPRGS9KK+xdsE nORRzUNS/u4C+tZnDg1qm1emMzVxnockW4GICh+uGaMPKyPANXqc/bXx7LlxI97LL3ljYCzWb /QtZjVGG0EYIPgplSQfbNpXpW+TrYLwp0LuBbLleMdzTkeV+BPW6pcKqugUeg5UOgLAM61VcA 82B1RhRwkzGMq3cMpYOHbt9qeLdbMLY6rhCZf12fXreSdl9e9KYTdr1ATf4xlEl1A+7Z78hnS TBQVSrykUj8EzyzCjkCUiwFYidaFohk7aHS9NCHCIsUT4fJMbUPTvtLk/Y7acG7JArktPWtXh XWkbpqaGT2vuD7TT0fcu5USzfSyQHB1Rkaqu0z+VzrQDFM52QNOrUVDdNvAJ6NSvIMlzwx1Tj o04uybE8kqyviuW+bZAwmlgj1iwWPmWcSQ1GgT14BohPeZ3ZGBffSSJpBilqySaSrioWGqzsG AdEnE22xsH19wjC4aWvPR/QVWP/+0BjULJ/bftUdi8Cf5ZuM4k3zkqTjSJeA7+YliVtMCf+/h frFiUA== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Ihor Radchenko <yantar92@HIDDEN> writes: Hi Ihor, >> Tramp is just a stupid library, w/o own intelligence. It offers >> alternative implementations for the set of primitive operations listed in >> (info "(elisp) Magic File Names") > > This makes me wonder we can simply add a "find" file handler that will > use find as necessary when GNU find executable is available. > >>> GNU find is faster simply because it is running on remote machine itself. >>> But AFAIU, if TRAMP could convert repetitive network request for each >>> directory into a single request, it would speed things up significantly. >> >> If you want something like this, you must add directory-files-recursively >> to that list of primitive operations. > > So, it is doable, and not difficult. Good to know. Technically it isn't difficult. But don't forget: - We support already ~80 primitive operations. - A new primitive operation must be handled by all Tramp backends, which could require up to 10 different implementations. - I'm the only Tramp maintainer, for decades. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 24 Jul 2023 07:35:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jul 24 03:35:01 2023 Received: from localhost ([127.0.0.1]:41650 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNq5t-00078v-8a for submit <at> debbugs.gnu.org; Mon, 24 Jul 2023 03:35:01 -0400 Received: from mout01.posteo.de ([185.67.36.65]:57435) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNq5q-00078e-CZ for 64735 <at> debbugs.gnu.org; Mon, 24 Jul 2023 03:34:59 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 4E8F3240029 for <64735 <at> debbugs.gnu.org>; Mon, 24 Jul 2023 09:34:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690184092; bh=OqthtdZWfqnOhu5N5YjqAQkhCfk3lm6674KrvpkvZuk=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=YRqvcU6n4nC4yu8+bwSof9K8XgdwIw2fdRvu/C7rx6RwQC2fmmSv4BpVWHUN7rmOo XHnogDHl9tTuDsFcoH7n6FF/tCk7kcNKjGfuU5RF+ssrudVp9SX8u8tV4EFTyfDD/p 5b4/INDSNo620tdI4ylhYLUhxA+d1PyyHlBbpngJcyRoFvJQW/qJBvQRa2Z7uQP9Pv 9aU0/Aw+mC+5exnE96XXf+F9ytCdLYdYVMtvjm/6OjLQuUiyu/yrIsaW4AAKEbzt6e MbWYJnJ8MgwXAQ6FKCRsqNkhJnk9j8yF95hUfP3cugkCZPV1PLaICQ/KM5rTToUvd2 Wo1lH3zWRxtFg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R8X4C011Xz9rxd; Mon, 24 Jul 2023 09:34:50 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <871qgyzw1v.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> <87pm4jrxra.fsf@localhost> <871qgyzw1v.fsf@HIDDEN> Date: Mon, 24 Jul 2023 07:35:04 +0000 Message-ID: <87mszl3gpj.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Michael Albinus <michael.albinus@HIDDEN> writes: > Tramp is just a stupid library, w/o own intelligence. It offers > alternative implementations for the set of primitive operations listed in > (info "(elisp) Magic File Names") This makes me wonder we can simply add a "find" file handler that will use find as necessary when GNU find executable is available. >> GNU find is faster simply because it is running on remote machine itself. >> But AFAIU, if TRAMP could convert repetitive network request for each >> directory into a single request, it would speed things up significantly. > > If you want something like this, you must add directory-files-recursively > to that list of primitive operations. So, it is doable, and not difficult. Good to know. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 19:44:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 15:44:37 2023 Received: from localhost ([127.0.0.1]:41259 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNf0P-0005KT-2b for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 15:44:37 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:42355) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qNf0K-0005KA-28 for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 15:44:35 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 052885C00BF; Sun, 23 Jul 2023 15:44:26 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Sun, 23 Jul 2023 15:44:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690141466; x=1690227866; bh=AkWN7gpo2ZAb/gBv8GCIY3kbasv77uvb2RK T2gVvpQY=; b=h259HOPBWiGLsDZPiomiGJulL0HdMHEhCQFpblSkvy3Xeu7VXUu XoKxuTlSnuSsL5jLIDe9UOsNlYptSQ2EZqUzotqgwNHoV05AF1sBvTPfStuYhviW 96znj/wvQHPGxe1n0nUFDeUR8iOKikd+kaxxz2ritjkIDd6ySfRQdi8rZZbCtEtz 0aXQASYl3plwdA8yj4A3jsQ9THUQaOHEyZ6EhdrMNWhi1MQ1wWDTgO1o4/naGF1u dxqj29ATu31mn/JHIP4F2k/dQfMuGdYgPOCYSU2ghdUJPyoH8U0jAOWtBO87R97L 7kz7v+wKPyK9w5QStDu3878e52NU9uSsiBw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690141466; x=1690227866; bh=AkWN7gpo2ZAb/gBv8GCIY3kbasv77uvb2RK T2gVvpQY=; b=SwUOVS2kD8RASWGsyNzymKyjYF60PHZ20cx0ju7hMreeYfEzk1g 1TgVJEVqUkPtz+a3iIRFSmNeg7XanRBhj0w28VbnT0q0k7Ca3yJf2jBsn9XZYICD JhYItvs6bgIhmNLp0AROcLmEKaNHjncBJfdUWFZ5Msd9hmW1ewpV4AZfF+kqImc0 ZZ4ZFHx6NGHVMWIuy9MXIRoOo9ip7dagoeGLSCEFwRFVp5vZWGRbPoFt7l56yUCM xB67TgLKv5JnDK/7+DYoEspc9WtPSXCMk1+DGSa10Cp9lyrmwai2CndhZPJoiUA5 19wsoSH7nR6UTFlbbbX+BJQcTr3vKBQ04Ww== X-ME-Sender: <xms:GYO9ZFVSR0ZlIafqJ0z84kpeBZR8XFiD3-RyO57n4AAC-Meifd3xKw> <xme:GYO9ZFmIqOQxvWG_BAd9lQge5Jdaf247x0DLr--AE5E8vGfQUYVgfSfs6f2HiYYEJ cRj77ccXs3d72wiDmM> X-ME-Received: <xmr:GYO9ZBZcrp38zkQoxX7uzMnlt-Pwp7_ItpB-bbKckYcBsp54DnKQZtkgIY3rlqw> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheeigddufeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:GYO9ZIVlJAdDmN47Tta6WjDzDEsP3OsnEp_k_4gIIXMDBlf8NqzGzw> <xmx:GYO9ZPl7lOYUk2kGicCXqgPV1lwVbyrc2Z2nrCzyepFx4LsZxYIpPA> <xmx:GYO9ZFcbDU37HrdXKJBLPBjCXTzxvyhp9Uk0Uyyx3dcVwJVVmkpxSg> <xmx:GoO9ZNj05vSkTltz-i2QJ4oiRVSdM99TY6eduSIyUZpvKFudnC78tA> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 23 Jul 2023 15:44:24 -0400 (EDT) Message-ID: <2c03fa11-6c97-cbdd-567e-b5dba85b7f77@HIDDEN> Date: Sun, 23 Jul 2023 22:44:23 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <e2f1e366-71f6-15b2-7515-8bce1cb43dc3@HIDDEN> <83a5vmtrog.fsf@HIDDEN> <1ff691d6-67fe-7aa5-75bc-435438e59152@HIDDEN> <837cqqtolx.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <837cqqtolx.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 23/07/2023 22:27, Eli Zaretskii wrote: >> Far be it from me to insist, though, but if we indeed reimplemented all >> the good parts of 'find', that would make the new function a suitable >> replacement/improvement, at least on local hosts (instead of it just >> being used as a fallback). > The basic problem here is this: the regexp or pattern to filter out > ignorables is specified as a Lisp string, which is in the internal > Emacs representation of characters. So to compare file names we > receive either from Find or from a C API, we need either to decode the > file names we receive (which in practice means they should be Lisp > strings), or encode the regexp and use its C string payload. Yes, the latter sounds more fiddly, but it seems to be *the* way toward find's performance levels. But only the benchmarks can tell whether the hassle is worthwhile.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 19:27:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 15:27:37 2023 Received: from localhost ([127.0.0.1]:41252 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNejx-0004vy-9t for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 15:27:37 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:59089) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qNeju-0004vk-M1 for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 15:27:35 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 7BA745C0043; Sun, 23 Jul 2023 15:27:29 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Sun, 23 Jul 2023 15:27:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690140449; x=1690226849; bh=FbeRyXKYgT9IIX2gOp6H/B6wY2557U1eNIG Rf40ubkE=; b=Flt1Uu1OXcY0GgyALc/oLzFONhpD7wviJYWbAQZXsQtAVjOaxTg pU6ctedUSXdGuJ0GNko/RbHHwDhVDHz+XfYagKLgkfMvahef+yf2UImJY6W3TGsv 2+PyogyCu2DoP8GLZhj3A78rIg/Bcgvg9wEbzpCnL4y4k6V76eRdTIDubjxpvajq U5HYfH2jb9pHEXxxI8avniyeaDVbPNZdpFhDaG8Zjxmt4SZiIFa+eBfJJEaQKtR8 LTRipdYvs9fhmWE5EDOn248wAHyTxf33h8OyZcR52TAU6NxdOwnWbplUSmtL0NzA C1ZKuCb+Q5Fl6xtTk9dyUUkVTBrTJotacpw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690140449; x=1690226849; bh=FbeRyXKYgT9IIX2gOp6H/B6wY2557U1eNIG Rf40ubkE=; b=iPFKlGRxaGkkFiBCuahTf0kxvrLb8gyC7OvlpZ6cvYuDvyIQI3X Z76gkiumO0qXLZtyad78onQDBcxdqMcrhywUgMBOOQvaSeD/NESr67C4/WRUjxUb aX5KxX+GAOyw3oZ76DPMIivTpaqkxncgRvO7RZRbBWNIKI/GM9e8zYzvtDp4tLui X372XFfHEMI5Rbi2VwKRHo+3keq2oKQmmPisKNp8H41ddLZmTsNxtzfxtwtpdOst 6cijhKTDWo5MkSowIr9TtnBLGdxF0Q6mzyUDpNVlpKeiLL8+vUK+ZIl7K59vTbJk ifHSIkdMiUs3vhebvbKUSzHI+u3kKHiKR7A== X-ME-Sender: <xms:IX-9ZIEXeMOIKmnTOA44Zd5cUTaRx3vBqbMQoZFO0AyKy-DQRonurA> <xme:IX-9ZBUEIvD_6gKUOEObRiNEhvuLpop5PegM_33A1V5NdW4eAx4AJ0IdBgz2aVDWv QkeXAePwm_tzGaPlIU> X-ME-Received: <xmr:IX-9ZCLz4r5iQ6RoypR1tPrNS57XSBDT8fXTgfhyn8pLUkzO1duqZkMPVizvHCU> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheeigddufeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:IX-9ZKF9TRFoQJFDZszZN4XtjjFmXCF1ptw3PanIossa_fz27JokQw> <xmx:IX-9ZOXnjKdlJos62Cxx-ImVV_sa9FPNAtvqB_4C9avgevjv0ssqnw> <xmx:IX-9ZNMWWc16-RSGfklP82-VIeJfZwVw9o2PBJyL4BhgszzEI0TB2Q> <xmx:IX-9ZNRj0AzlKuvbSug9PBQilVXiLsFUn1ZYoTP-xb6Oq5FwbDKcog> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 23 Jul 2023 15:27:27 -0400 (EDT) Message-ID: <83bd4246-ac41-90ec-1df3-02d0bd59ca44@HIDDEN> Date: Sun, 23 Jul 2023 22:27:26 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83bkg2tsu6.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 23/07/2023 20:56, Eli Zaretskii wrote: >> And, ideally, do all the relevant benchmarking when proposing the change. > Of course. Although the benchmarks until now already show quite a > variability. Speaking of your MS Windows results that are unflattering to 'find', it might be worth it to do a more varied comparison, to determine the OS-specific bottleneck. Off the top of my head, here are some possibilities: 1. 'find' itself is much slower there. There is room for improvement in the port. 2. The process output handling is worse. 3. Something particular to the project being used for the test. To look into the possibility #1, you can try running the same command in the terminal with the output to NUL and comparing the runtime to what's reported in the benchmark. I actually remember, from my time on MS Windows about 10 years ago, that some older ports of 'find' and/or 'grep' did have performance problems, but IIRC ezwinports contained the improved versions.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 19:27:08 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 15:27:08 2023 Received: from localhost ([127.0.0.1]:41248 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNejT-0004vH-Qy for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 15:27:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47780) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNejR-0004un-Nc for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 15:27:06 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNejM-0004di-BW; Sun, 23 Jul 2023 15:27:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=bcrmXRIWQz+FPlz0O3TCToeHfSucAGZnDgu6yX3G6Hc=; b=BFGYpifczX9y TsPSJGbb8q67YkMg+uYwrlDNOsGUNhTx7Ay3BZkh4YDZGGsekfu0JqyOPvjwncqSAWyxjn+7n25r0 N5qmNwPc5zOQ0JhsVNqnOa1QxHXr9oTXp0nBb+Vwr+poS5fsgzeopv33z977aEI6l5vKt92tZYAbU uwhNXV/VbVcE/n4QYONQh/sur0Ono/9GWLn8HxKCDRgDnHSmG6dh0ATnlrVvnY6T/AHpgUK5sJCX7 AD52tISoAGAh5kfeVtDfZ2pIDN4wL3DwU5Lq2Yp6KpbTeywzV5mGDyOxD1dOnNOCAC3J01TC08vJ9 pmhgOsb/3aSetrfAQ9RR6w==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNejJ-0002Uj-Lz; Sun, 23 Jul 2023 15:26:59 -0400 Date: Sun, 23 Jul 2023 22:27:38 +0300 Message-Id: <837cqqtolx.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <1ff691d6-67fe-7aa5-75bc-435438e59152@HIDDEN> (message from Dmitry Gutov on Sun, 23 Jul 2023 22:07:17 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <e2f1e366-71f6-15b2-7515-8bce1cb43dc3@HIDDEN> <83a5vmtrog.fsf@HIDDEN> <1ff691d6-67fe-7aa5-75bc-435438e59152@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Sun, 23 Jul 2023 22:07:17 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > On 23/07/2023 21:21, Eli Zaretskii wrote: > >> Date: Sun, 23 Jul 2023 20:58:24 +0300 > >> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > >> 64735 <at> debbugs.gnu.org > >> From: Dmitry Gutov <dmitry@HIDDEN> > >> > >> A possible advantage of doing it earlier, is that if filtering happens > >> in C code you could do it before allocating Lisp strings > > > > That's not what happens today. And it isn't easy to do what you > > suggest, since the file names we get from the C APIs need to be > > decoded, and that is awkward at best with C strings. > > It is what happens today when 'find' is used, though. No, I was talking about what file-name-all-completions does. > Far be it from me to insist, though, but if we indeed reimplemented all > the good parts of 'find', that would make the new function a suitable > replacement/improvement, at least on local hosts (instead of it just > being used as a fallback). The basic problem here is this: the regexp or pattern to filter out ignorables is specified as a Lisp string, which is in the internal Emacs representation of characters. So to compare file names we receive either from Find or from a C API, we need either to decode the file names we receive (which in practice means they should be Lisp strings), or encode the regexp and use its C string payload.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 19:07:30 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 15:07:29 2023 Received: from localhost ([127.0.0.1]:41237 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNeQT-0004U0-Gn for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 15:07:29 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:37789) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qNeQQ-0004Tm-ID for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 15:07:28 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 286345C00D9; Sun, 23 Jul 2023 15:07:21 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Sun, 23 Jul 2023 15:07:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690139241; x=1690225641; bh=1Ee063bk7od1pJ2gcn7SesVGEGGjWNmniE9 Jcu5v7m8=; b=IzYHnh0g9fTS9g8inzsB6sY+yk59Zo/ejiu6Zw0Oi7rBf3adDch Kys243WsDsWQJbv+mv52aPUmuuHq4OlvP9xCvbN+jh+HpCZtVndqkRBkTh+ow8zB WH+rmVQ9Vh/Wup/yg9cb+1nnk3EoW2aNgA32Jyz9C/gdJ1GyTjcuGs4egsIxpFTV 1iRKqxSicmjgPDzc7CkE1ByJAt3MysN69VzL6A35PonxzX+zWL+gp02/lpdY/X9T DLpm6qIxIw5b/+kqaiCNGWjiO9iIOdh53EgW8Gee5O6tDvlSBw4gHIcwF6nu3Jc5 /Ozkd3NO4bYz/ejEKc4vu/IfO/YNdijeWhQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690139241; x=1690225641; bh=1Ee063bk7od1pJ2gcn7SesVGEGGjWNmniE9 Jcu5v7m8=; b=pikfNHvJ/C8Q37IvfGOTOVyrhV9xL9YOinXm/Ck1xov6WnkS/Ol yA0xNAA5l11mEgviejbNUpV/Gemt8YtZplEIWbTirFewE1S8XGtrttw68VyZZFzT iGBZuuKREaGcoroG3HxUsgP+sC7dlnjUiHq6IHlIhWHyZf3V04wr+1xSIsN3BPh0 Imkbj51yV7e6AaN1TDNXhYOvlLmGaTm7lSPC13YkiO+HOGVHYnHBa1Ek29W/HXX2 xGbYpAcv7Hs9W1vYO5Owmp+O9F5BDfc9qSgEG36643LwFn+FZGNXIQq9EdhXbgr3 +xRinCDJgysz9zFSSCe9kkYq9Rg6sB7cpNw== X-ME-Sender: <xms:aHq9ZCZrBb-XpiL-zPaZhQbrILw44Nw-aNj6Y_Dw_N2MaRlSfSI1_w> <xme:aHq9ZFbfkDbtnHDZXrhmmeg4iJavBqdqxBVzbDE4R3N-UYXVrYbSZW2G25zbjWBr9 CLsuIz2NnAyIHfNDG0> X-ME-Received: <xmr:aHq9ZM-z_tI7LYeHq83Mu7feHzmi0dkPnR1vyPJgunzSynQmQXS6GmTFId8x7bw> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheeigddufedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:aHq9ZEr1tTXyL5yCdv-CgCxAG60vlqAatFt8W1pgrMn4RX59OEkiog> <xmx:aHq9ZNqwVeG13IH8oAYdopHGb_Ch8ZbpdgV3Lm6LY4-vkKkPPBb0Uw> <xmx:aHq9ZCTFPoQvFMQNS6jTVoYounQcEtT5CRgSR30d88huum3QfySNaQ> <xmx:aXq9ZEUrZciyM6iv1VJq2Pt0FV7CNeIgJboxnYqo3pL0_p-uzkUOMQ> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 23 Jul 2023 15:07:19 -0400 (EDT) Message-ID: <1ff691d6-67fe-7aa5-75bc-435438e59152@HIDDEN> Date: Sun, 23 Jul 2023 22:07:17 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <e2f1e366-71f6-15b2-7515-8bce1cb43dc3@HIDDEN> <83a5vmtrog.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83a5vmtrog.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 23/07/2023 21:21, Eli Zaretskii wrote: >> Date: Sun, 23 Jul 2023 20:58:24 +0300 >> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov <dmitry@HIDDEN> >> >> A possible advantage of doing it earlier, is that if filtering happens >> in C code you could do it before allocating Lisp strings > > That's not what happens today. And it isn't easy to do what you > suggest, since the file names we get from the C APIs need to be > decoded, and that is awkward at best with C strings. It is what happens today when 'find' is used, though. Far be it from me to insist, though, but if we indeed reimplemented all the good parts of 'find', that would make the new function a suitable replacement/improvement, at least on local hosts (instead of it just being used as a fallback).
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 18:20:52 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 14:20:52 2023 Received: from localhost ([127.0.0.1]:41224 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNdhL-0003Kk-S1 for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 14:20:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50558) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNdhG-0003K8-Be for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 14:20:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNdhA-00054K-EK; Sun, 23 Jul 2023 14:20:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=3huSKIquAcjLowpeC6+P1Pg0dbYPqwCPUEtw4jbLDSA=; b=mn4AMLItPZok B06cZciTNH5Ii00Y8GjHJxEccvc8pTj2izmZcAVCpxvOnGKzirbZ5ewIqhOBrrcIUAi/M8IxiIDST 1tQwtkpuDid2rM7S3s1ldlnLSlBuVE0pnZ0jcSGy1fm2H9Zj2n/Uzk6MnNFXWfRuLsxb9OfX4W/GU JQA1JSb47+QqmKIMMy06nu/36K+gA7YOKow26buYMB+/zCsRJ6Xi0TR21i+QSBZJyqPApJcKrXek0 MOLCdZcSAqxkudjt3PibZ/KkWKTZvIHohB6qhRgTVRvD8l7UNBEVUptmNrb8U6z7XA3fyn5eCubcq uFIWbiRQaDttV8hDMEzWXQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNdh9-0004dY-Tu; Sun, 23 Jul 2023 14:20:40 -0400 Date: Sun, 23 Jul 2023 21:21:19 +0300 Message-Id: <83a5vmtrog.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <e2f1e366-71f6-15b2-7515-8bce1cb43dc3@HIDDEN> (message from Dmitry Gutov on Sun, 23 Jul 2023 20:58:24 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> <e2f1e366-71f6-15b2-7515-8bce1cb43dc3@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Sun, 23 Jul 2023 20:58:24 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > A possible advantage of doing it earlier, is that if filtering happens > in C code you could do it before allocating Lisp strings That's not what happens today. And it isn't easy to do what you suggest, since the file names we get from the C APIs need to be decoded, and that is awkward at best with C strings.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 17:58:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 13:58:37 2023 Received: from localhost ([127.0.0.1]:41217 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNdLo-0002kE-Ki for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 13:58:36 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:53169) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qNdLm-0002k1-Fx for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 13:58:35 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 90C7732002E8; Sun, 23 Jul 2023 13:58:28 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Sun, 23 Jul 2023 13:58:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690135108; x=1690221508; bh=yq5fEfKbuJB8fN1QdIXxmmOWKMAnPKDk3xx 91X4qLt8=; b=buaEjVLROhPLwUFElEehba9yNmMWUo/wtSepVFOEctGBi4OOm4l lxu/0Tmzh6byGD4cZmjCdpmT4wlepWrtuAhjGpRpce3Tv6cdqfIdHcZslAdJvA2l FoF6dDqAwytxLDGyAYz/sJ5FDhgi4HGFto+7lyw7g5DGfeCP2o6IWgthOgvCmUhv tG6VDH4ZpfF2HHdbQiHcm0tYk0uJcBSaducl5JEagtNRAvqYFhQxGtuVFdWiTHsO jWc+5hiu+Q3K35rKLjIxEtrTEQEya3SKci1xJdVXCk8p5k78g/NT6tAveHBrnEQI hXcoczgC+mLVruB3STwDX0N4aLkYeu+FW/w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690135108; x=1690221508; bh=yq5fEfKbuJB8fN1QdIXxmmOWKMAnPKDk3xx 91X4qLt8=; b=PwngWDJTsSyuBzU+4mjnAaf7dxP+Epof66BOlUTmRnaShKvPaQd 5bm35qAQNOa4pFJr3YJc9c8cUwQDIuhj5lYWXGFuFU7/rm91BAe9i692rk9pJLzW Ayz1d7/1pdiBvBGdIVrZbv6MYwAKWPPdLOObLpdFG+d8rqgm9mMQXNDtlscO4uYb 4gVZ2U0bO2KASYoMy3SHwj53DGh+Uckkuvp/m79hJP6nl2y1JxfBvb9XKvDFv467 Uf4EbD9plnkdCetz1kuw1qzvgExNVTvJYkKeS5GHX7ClqoegKK3zaGgwrs6JIfa0 uLojRX4IHucoNe2WVxIoO6kkOcbZW5BLM3w== X-ME-Sender: <xms:Q2q9ZEp7mHU1XR21mAIoSc7JzHz2yuZIrOZb1tS0XtEEJ94HZkxvVA> <xme:Q2q9ZKoW3ywYptSFbbsV387biu5C3cM6Vv2Y2UQAUMmvvVETHwmQIx0SUSmMdZExx dqJ571Thx1xTyQIra8> X-ME-Received: <xmr:Q2q9ZJO6cdEAzd-tZuJXl2E1udhG_upcuTLWccC_JJbAhP6HEMpiRW3kLl-dTcE> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheeigdduudekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:Q2q9ZL7ucxnReHZzsKP2W7opEtibVLZzRuD6SgsYMQPb5XJki5whXQ> <xmx:Q2q9ZD4fRxZGZSPnuU9bHh2FEfhyDa8bXqB80ZrtG13wivaxAuAr8A> <xmx:Q2q9ZLidaHRGHSzEwYqs5pmG-62gl7ORYHr_zg1nUX0bY1kg0lbkNQ> <xmx:RGq9ZFkCsvRcBQADp_QZjxq723Yc0lSwtwm3P6_maRuUdIUfs00MdA> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 23 Jul 2023 13:58:26 -0400 (EDT) Message-ID: <e2f1e366-71f6-15b2-7515-8bce1cb43dc3@HIDDEN> Date: Sun, 23 Jul 2023 20:58:24 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> <83bkg2tsu6.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83bkg2tsu6.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 23/07/2023 20:56, Eli Zaretskii wrote: >> Date: Sun, 23 Jul 2023 20:46:19 +0300 >> Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov <dmitry@HIDDEN> >> >> On 23/07/2023 14:18, Eli Zaretskii wrote: >>>> Date: Sun, 23 Jul 2023 13:46:30 +0300 >>>> Cc:luangruo@HIDDEN,sbaugh@HIDDEN,yantar92@HIDDEN, >>>> 64735 <at> debbugs.gnu.org >>>> From: Dmitry Gutov<dmitry@HIDDEN> >>>> >>>> On 23/07/2023 08:11, Eli Zaretskii wrote: >>>>> Even better: compute completion-regexp-list so that IGNOREs are >>>>> filtered by file-name-all-completions in the first place. >>>> We don't have lookahead in Emacs regexps, so I'm not sure it's possible >>>> to construct regexp that says "don't match entries A, B and C". >>> Well, maybe just having a way of telling file-name-all-completions to >>> negate the sense of completion-regexp-list would be enough to make >>> that happen? >> >> Some way to do that is certainly possible (e.g. a new option and >> corresponding code, maybe; maybe not), it's just that the person >> implementing it should consider the performance of the resulting solution. > > I agree. However, if we are going to implement filtering of file > names, I don't think it matters where in the pipeline to perform the > filtering. A possible advantage of doing it earlier, is that if filtering happens in C code you could do it before allocating Lisp strings, thereby lowering the resulting GC pressure at the outset.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 17:55:52 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 13:55:52 2023 Received: from localhost ([127.0.0.1]:41212 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNdJ7-0002f4-0V for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 13:55:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42778) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNdJ2-0002ed-F1 for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 13:55:48 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNdIv-0007OL-Dw; Sun, 23 Jul 2023 13:55:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=9loBuXdpOhwiQ4Q5Eh3mjURms7w4P6xpZYnIg0br91Y=; b=ip9d/HnpmSQR 6grqUFo7E72LVA8tWXeWNejoX9wEqdSpGWA7G5j+m67/6e12d/iB3vWKHtLO8MG26VjRvRjQkCFwy /rYMD7jB400jgJXHZAFDUG3EolwHhlIYOL70OphHAm7Nj0uG+I9cXSvgZhlct81G/WZrxnkXBheAT 7xH4Z+mlR2WJlh7JhVonTNPyYg8Ha1Itj9rLtWKwVSNDAD+MhtvkoJJUkQDBDIpS73VEhIAP/ZU6m Fzj2IsB3o25RkH3heSe316pkYtPZWZATr9ruj0+YEKLOY3s4ic3MyJ7YF5vsICK5IQzzpAWTB4o00 J/GqshgOra+0FNOuSPMnYQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNdIu-0006qx-S6; Sun, 23 Jul 2023 13:55:37 -0400 Date: Sun, 23 Jul 2023 20:56:17 +0300 Message-Id: <83bkg2tsu6.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> (message from Dmitry Gutov on Sun, 23 Jul 2023 20:46:19 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Sun, 23 Jul 2023 20:46:19 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > On 23/07/2023 14:18, Eli Zaretskii wrote: > >> Date: Sun, 23 Jul 2023 13:46:30 +0300 > >> Cc:luangruo@HIDDEN,sbaugh@HIDDEN,yantar92@HIDDEN, > >> 64735 <at> debbugs.gnu.org > >> From: Dmitry Gutov<dmitry@HIDDEN> > >> > >> On 23/07/2023 08:11, Eli Zaretskii wrote: > >>> Even better: compute completion-regexp-list so that IGNOREs are > >>> filtered by file-name-all-completions in the first place. > >> We don't have lookahead in Emacs regexps, so I'm not sure it's possible > >> to construct regexp that says "don't match entries A, B and C". > > Well, maybe just having a way of telling file-name-all-completions to > > negate the sense of completion-regexp-list would be enough to make > > that happen? > > Some way to do that is certainly possible (e.g. a new option and > corresponding code, maybe; maybe not), it's just that the person > implementing it should consider the performance of the resulting solution. I agree. However, if we are going to implement filtering of file names, I don't think it matters where in the pipeline to perform the filtering. The advantage of using completion-regexp-list is that the matching is done in C, so is probably at least a tad faster. > And, ideally, do all the relevant benchmarking when proposing the change. Of course. Although the benchmarks until now already show quite a variability.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 17:46:36 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 13:46:36 2023 Received: from localhost ([127.0.0.1]:41207 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNdAB-0002KR-So for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 13:46:36 -0400 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:44269) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qNdA7-0002K8-5V for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 13:46:34 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 8CEDD320047A; Sun, 23 Jul 2023 13:46:24 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Sun, 23 Jul 2023 13:46:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690134384; x=1690220784; bh=FqUSWgxGmq/9h/KPhMqPbHj179uxlDJNrBF 6nqcmeWE=; b=ZvcYRTWTT78J6NpgMgioyUZDNfdxFrIfAVIvmY4KDa3eK5e+L0j q26wp2IYrf2BgSoEL7Fp89RztmV7ygSZeWzRETPtm0VZzNkeKBWMQgLdFRPRdk4H MzooqXDxNVUHRePJ1VuyEDVfhdtOD31s3xVdFS5/ckj2KcvkWqquND7rF4lpkkwq 7YPgHPPZLXcDH8NVt4JBXim4lK9Zk1S1FvGMj4T+tpRCELKpnj1YgP1xZyFcTtcL e/coFMxqM9IFtcs7q5QjzNhIDNIDVwZ8URY7lpCp9UzbXIe1CgBzogjhftEjQaMQ oi+a0ZCLKW5UMsHp83A93KQIRk5kRa/cnyw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690134384; x=1690220784; bh=FqUSWgxGmq/9h/KPhMqPbHj179uxlDJNrBF 6nqcmeWE=; b=dh1KSvWPEp5ad+dT7FVHfiisoOPgZriIMXlimij1fJkJw5KGvNc EedSrApmLFb46+QXAKdyShsJ5aMcD8JfQSH0khw0TUkdgt3oTr8S6Fojc9fbsn4h +fAWrjiHfEpg7pMo8Nie+fCrvVaY4nJnkvubrVrcNkl/h3nUEFQJWxzTdeXJCb9B fqD1QzzGqA+4hLJEaVmoWWy3/ogk7kKW4iPp8DP/gPMkr/9xTAW9h+RzmnTl0Bh5 E+NmatvV5Br/PLY7QwzJzjIKRJgqNVOjXinX7dJkgLIQ+BHNTgwWvRFBqWqpG1WL 4crYIrJ1l/29S+ZdoqpK9xWm38ZmBBdKfTg== X-ME-Sender: <xms:b2e9ZClVaTmCrDWe7wQSOotBfPS4_3SdL34lX3TrYmgZw1wAIhcqsA> <xme:b2e9ZJ00HmxzBvT6lOEs3b57P9-mJk6QfZu1o-TIS42rgo1jNK5fdQhPrc9EjV-PU esieVBASchWtZorWEI> X-ME-Received: <xmr:b2e9ZArNPcowzuQmZ7XxtCdsiN-TC1fKgijIAKzIcpC9KBpGJkcZ_HRmA3HDRQc> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheeigdduudeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:b2e9ZGm1opgBur1tW1CnZ8g65V2TTUIox_VtxEaRZ2fSgThS1Ida_A> <xmx:b2e9ZA1YU7NOlqHG1l-Sa7gnvgflDu06S8eupemGdp_VIxrcJuFPrA> <xmx:b2e9ZNvHei6yD18Hj60paC96VSRWcH0ndwZQ-e_MZJsvdqIEgFy0Ig> <xmx:cGe9ZNwtRA2vkev7Hz1kdeWVsJLxoIEfrPcdWxIbAdzjOOrG_tJgrw> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 23 Jul 2023 13:46:22 -0400 (EDT) Message-ID: <2d844a34-857d-3d59-b897-73372baac480@HIDDEN> Date: Sun, 23 Jul 2023 20:46:19 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> <83sf9eub9d.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83sf9eub9d.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 23/07/2023 14:18, Eli Zaretskii wrote: >> Date: Sun, 23 Jul 2023 13:46:30 +0300 >> Cc:luangruo@HIDDEN,sbaugh@HIDDEN,yantar92@HIDDEN, >> 64735 <at> debbugs.gnu.org >> From: Dmitry Gutov<dmitry@HIDDEN> >> >> On 23/07/2023 08:11, Eli Zaretskii wrote: >>> Even better: compute completion-regexp-list so that IGNOREs are >>> filtered by file-name-all-completions in the first place. >> We don't have lookahead in Emacs regexps, so I'm not sure it's possible >> to construct regexp that says "don't match entries A, B and C". > Well, maybe just having a way of telling file-name-all-completions to > negate the sense of completion-regexp-list would be enough to make > that happen? Some way to do that is certainly possible (e.g. a new option and corresponding code, maybe; maybe not), it's just that the person implementing it should consider the performance of the resulting solution. And, ideally, do all the relevant benchmarking when proposing the change.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 14:32:28 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 10:32:28 2023 Received: from localhost ([127.0.0.1]:41120 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNa8J-0005OI-Nk for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 10:32:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36750) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNa8H-0005O5-4a for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 10:32:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNa8A-00040A-SK; Sun, 23 Jul 2023 10:32:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=6kBE+t1hb6wxHid8ElAB4wMt7B/PKiGc0eWJ1JMsptw=; b=BM8BXOSo0gI0 v9KR2tj+2anlvcK6T27anzBHpiJw77oEL8gee+36eKVr2BXHia/LuF5rs3uTkU9t6HIerK2/kXQw2 Nf6fRn+JMhzM40LSiATv+VF5DoF9n0nL6Q7LX84c1umwAAGpBsuBNn3C2FYjovbgeBRzhulhAWJ/B YM+MfU2PvanS4ffL6jiOx8A6WD5lXXy1BuiCnsOQDUx7eljIA08rQ3w4i1Id6NHfCg6xU96Nh4Di8 lI6t2qnpDOk8656PKrxOzcmUbImsL3cJNaUvDm7coAq03VXUww3VgstrwbmhOksCoZO2FVbIKrJEA PRtj7PqnQHH2IsSKzmYmuw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNa80-0003xc-Ns; Sun, 23 Jul 2023 10:32:09 -0400 Date: Sun, 23 Jul 2023 17:32:48 +0300 Message-Id: <83fs5eu29b.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87o7k2n33g.fsf@localhost> (message from Ihor Radchenko on Sun, 23 Jul 2023 13:56:35 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> <83y1j7t19d.fsf@HIDDEN> <87r0ozq7ze.fsf@localhost> <83v8ebszcy.fsf@HIDDEN> <87v8ean99h.fsf@localhost> <83o7k2u70z.fsf@HIDDEN> <87sf9en5ta.fsf@localhost> <83jzuqu52l.fsf@HIDDEN> <87o7k2n33g.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sun, 23 Jul 2023 13:56:35 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > Anything's possible, but when a function produces text, like file > > names, then the natural thing is either to return them as strings or > > to insert them into some buffer. > > Do you mean to re-use process buffer and process API, but for internal > asynchronous C functions (rather than sub-processes)? Not necessarily a process buffer, no. Just some temporary buffer. We already do stuff like that for some C primitives.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 13:56:32 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 09:56:32 2023 Received: from localhost ([127.0.0.1]:41070 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNZZY-0004PW-0R for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 09:56:32 -0400 Received: from mout01.posteo.de ([185.67.36.65]:44633) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNZZU-0004PP-OZ for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 09:56:30 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 3298F240028 for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 15:56:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690120583; bh=ICVlmVsAbXZFmLBOrdichBsEf/CTjZIapsEi9luil1A=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=naxVmHADHuL0vrLV4lJgfy/VgmmnYa/r6OtNxObsJatan53UTpMwlfoILnMGlBx0R t7vMyk4J63xx9fCIdDB2guO0c5lXlu8U1bDi3IDNecBpveuhw+oC4So/pvHO9zh30B OdNk56rUWr1G8RlOzAs6uvsGGeWRA93M9RPAdNYBDYWTb4DfLpHPqCMyD3+hNwL1n4 sbYo8GbUMhwi1UkjviKGjHnTVoqV0LChs7aX/byVQpPc6Ppory7EgI9X433fGn37HW WPowW+p3Z0JkAx0p/ITdfqksuTR3F/Yn+wRh4uoYyrErFDwcVKoYCU0WDmDpOp6u/+ 7Mrwlu2MuVgxg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R84Zt05Hvz9s87; Sun, 23 Jul 2023 15:56:21 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83jzuqu52l.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> <83y1j7t19d.fsf@HIDDEN> <87r0ozq7ze.fsf@localhost> <83v8ebszcy.fsf@HIDDEN> <87v8ean99h.fsf@localhost> <83o7k2u70z.fsf@HIDDEN> <87sf9en5ta.fsf@localhost> <83jzuqu52l.fsf@HIDDEN> Date: Sun, 23 Jul 2023 13:56:35 +0000 Message-ID: <87o7k2n33g.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: > Anything's possible, but when a function produces text, like file > names, then the natural thing is either to return them as strings or > to insert them into some buffer. Do you mean to re-use process buffer and process API, but for internal asynchronous C functions (rather than sub-processes)? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 13:31:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 09:31:40 2023 Received: from localhost ([127.0.0.1]:39025 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNZBU-0003A1-B8 for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 09:31:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47542) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNZBR-00039k-2h for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 09:31:38 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNZBL-0001xm-BT; Sun, 23 Jul 2023 09:31:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=8WthDV1koexSuSPHEquKSnF5vOF3UGGDXZ8fsypYCgo=; b=LxBi3A775/0U otmEzaLmxGg7+Rg7Ubkq9bLwdVrGIhR6EwTavE/BD/J7M0be73U64NnBLfcrDe/CgBiFmqLaKdNdo Nmpm3ne0b6RfhUqhTn5uy3hCA3OAcPy0jH+MXqCWaFkxS733NDIcgTglCg0fMbH4VYvCIFtsubkhS zthq/3WHrvl2qcbfw75RlDXd/PwisQUAB5kG+DxChEZpDrNGyb9Sb1AuKZ3RILlq9p2SvsfYkTWwo V3n5aDKgJfo/K7kB2PC1T+Pzs4V2/ZbZXpi+jyK8QIhxD+i2GovTrJCtibkojTDyt6caiHGJJMTR7 3TNvyAnLGEojNc4IjdiumA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNZBF-00040i-5N; Sun, 23 Jul 2023 09:31:25 -0400 Date: Sun, 23 Jul 2023 16:32:02 +0300 Message-Id: <83jzuqu52l.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87sf9en5ta.fsf@localhost> (message from Ihor Radchenko on Sun, 23 Jul 2023 12:57:53 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> <83y1j7t19d.fsf@HIDDEN> <87r0ozq7ze.fsf@localhost> <83v8ebszcy.fsf@HIDDEN> <87v8ean99h.fsf@localhost> <83o7k2u70z.fsf@HIDDEN> <87sf9en5ta.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sun, 23 Jul 2023 12:57:53 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> > It could be "call as soon as we got 100 file names", for example. The > >> > number can even be a separate parameter passed to the API. > >> > >> Will consing the filename strings also be delayed until the callback is invoked? > > > > No. I don't think it's possible (or desirable). We could keep them > > in some malloc'ed buffer, of course, but what's the point? This would > > only be justified if somehow creation of Lisp strings proved to be a > > terrible bottleneck, which would leave me mightily surprised. > > Thanks for the clarification! > Then, would it make sense to have such a callback API more general? (not > just for listing directory files). > > For example, the callbacks might be attached to a list variable that > will accumulate the async results. Then, the callbacks will be called on > that list, similar to how process sentinels are called when a chunk of > output is arriving to the process buffer. Anything's possible, but when a function produces text, like file names, then the natural thing is either to return them as strings or to insert them into some buffer.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 12:57:54 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 08:57:54 2023 Received: from localhost ([127.0.0.1]:39001 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNYeo-0002N3-6F for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 08:57:54 -0400 Received: from mout01.posteo.de ([185.67.36.65]:44825) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNYel-0002Mq-MB for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 08:57:53 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 21E1B240029 for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 14:57:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690117066; bh=cBj0+v06C819vrM2g7+8Pj3Zjd6RLU8eX2iUpx/YHAA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=YfRvBdoaM4wk+9AnAjptjqJ/nSI5XZX2m39GT8uhnbTLkTQuEaBmAVfnmA+Le5yDO /pdYiJ9RmNR/xtag+3w9/Bvj3uds2pXsslwQtSO59zf1A5Vd241WVLtR0niBiwBZI4 QAU/3HpoHS8wAiN2XiqysyAEGaATYTOxRy5jJMzKmGJKXp30aWWaaJ11LmJUCZIz52 h1G1GJzf0T3eOQXAUNV9np6FXuCzUfODGyn24ZwIbUPf2bp8C1vPYoGIlu8ySnKgcY VTSBdpYo3vDrq0bcNBFG18vQ4/8T6o3aMpk1JypRPv/gQituQ0wGEloIbmfjmbGX1d S5reKcwuJkNBg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R83HD2psgz6v2X; Sun, 23 Jul 2023 14:57:43 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83o7k2u70z.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> <83y1j7t19d.fsf@HIDDEN> <87r0ozq7ze.fsf@localhost> <83v8ebszcy.fsf@HIDDEN> <87v8ean99h.fsf@localhost> <83o7k2u70z.fsf@HIDDEN> Date: Sun, 23 Jul 2023 12:57:53 +0000 Message-ID: <87sf9en5ta.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> > It could be "call as soon as we got 100 file names", for example. The >> > number can even be a separate parameter passed to the API. >> >> Will consing the filename strings also be delayed until the callback is invoked? > > No. I don't think it's possible (or desirable). We could keep them > in some malloc'ed buffer, of course, but what's the point? This would > only be justified if somehow creation of Lisp strings proved to be a > terrible bottleneck, which would leave me mightily surprised. Thanks for the clarification! Then, would it make sense to have such a callback API more general? (not just for listing directory files). For example, the callbacks might be attached to a list variable that will accumulate the async results. Then, the callbacks will be called on that list, similar to how process sentinels are called when a chunk of output is arriving to the process buffer. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 12:49:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 08:49:26 2023 Received: from localhost ([127.0.0.1]:38992 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNYWb-0002Br-Pj for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 08:49:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59200) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNYWX-0002Bb-DY for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 08:49:24 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNYWQ-0003zj-Cc; Sun, 23 Jul 2023 08:49:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=uX6QBxVov59s1+6CP6IDoiWIBCUltr7wAD2iiMwPFCY=; b=L6Qd+9W868iz gRXWfpzwxoCc3nwk4XipON9CEXxLZWcuYaeq27YaZIrjnDXTA027YqhROJCzGyezc36DZpdnw3AQy 8I8eAOba4Jrq6PkhVk3ST9wnTUdCHAPNHpKeZDevKhJcMNcJw0lSnb60NjTMhIm/tS/TmeA5O+B9w rVqEIcQqYbFayQ3zXEVVlOKvkOt/luMvBUwxkJuRdOTq9nAvN9cXhkEdze11QYEUo+gonB3/rRTfv 7XKZDo6Xdhiqea4C27mg/pRuZyqATAyQwzy4WRODjrtXrhoW1OgAIHu2sOIyd5CXHgqU+gwvrDabU iC2t9uFCJkCaqUQUY3wfrQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNYWJ-0007q6-Ir; Sun, 23 Jul 2023 08:49:07 -0400 Date: Sun, 23 Jul 2023 15:49:48 +0300 Message-Id: <83o7k2u70z.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87v8ean99h.fsf@localhost> (message from Ihor Radchenko on Sun, 23 Jul 2023 11:43:22 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> <83y1j7t19d.fsf@HIDDEN> <87r0ozq7ze.fsf@localhost> <83v8ebszcy.fsf@HIDDEN> <87v8ean99h.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sun, 23 Jul 2023 11:43:22 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> But what is the strategy that should be used to call the CALLBACK? > >> You clearly had something other than "call as soon as we got another > >> file name" in mind. > > > > It could be "call as soon as we got 100 file names", for example. The > > number can even be a separate parameter passed to the API. > > Will consing the filename strings also be delayed until the callback is invoked? No. I don't think it's possible (or desirable). We could keep them in some malloc'ed buffer, of course, but what's the point? This would only be justified if somehow creation of Lisp strings proved to be a terrible bottleneck, which would leave me mightily surprised.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 11:50:39 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 07:50:39 2023 Received: from localhost ([127.0.0.1]:38892 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNXbi-0006PH-Sp for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 07:50:39 -0400 Received: from mout.gmx.net ([212.227.15.19]:33733) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qNXbg-0006P4-3g for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 07:50:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1690113022; x=1690717822; i=michael.albinus@HIDDEN; bh=CDK8rNQ8n+aCQFo8/XW6B1venrAIDbQIy7cmMh/3a+0=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=kl+A/mW29FcsTFfZ3LEKCrhG7LjYOQ0bND4UzD+bFr8BhW+9tSniW+yrVYU1SBhyrWFwJ8J cLIwIAEy6wH2G19QLBTHwxzmwapGYqqMJ7/f4XbdtM/vBTKa4D17Te1OY97y1XX0vyjaAOqpm JoX9WIj8v4mZN75mOWr7J1/nszqCXXV2nmniNodiYRNjllSFIK2S8Sy0Q2oJvfyApOA2sYA6F CkXNGFJVWvL7sAgXq3dKHoLWfODCb4VI6hkjFt5FT5hS+Dovi03W6A/BeA9/vViSbvjrOis0W AnK2s9uy4GsRER0dTG9t+speNROh/LRquxp0FjRs1sLABwVyxPsA== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1N1OXZ-1pvSkv1igN-012prw; Sun, 23 Jul 2023 13:50:22 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87pm4jrxra.fsf@localhost> (Ihor Radchenko's message of "Sun, 23 Jul 2023 05:40:41 +0000") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> <87pm4jrxra.fsf@localhost> Date: Sun, 23 Jul 2023 13:50:20 +0200 Message-ID: <871qgyzw1v.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:Rx3NFDdga11fN1b0+hG/6bcqKTQMxI+bFChq2MWbsUXVROcpMoc 7aeWfR0vyiuAwsGGTQqfC/4QDHD6bztEi7vrX0+PnlACSfkHQC1hLsFpIycZzb1RXfcIl4E ITcMqdhBhlCvrXmbiNBUFUjQbEfhT5totBQUrMsRmjZZKvjTYX+GnGyu1MsSKykht4UAT6M PN7V962ScHWSl41yf0w2A== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:JL50sloy2i4=;rCx/XO2hQJfjVu6tAouGt84zNMi q5qEEmEUJ2mYvp2f/va2ISUDDoDO9hDMm3M/qhiuQoZQKO1IzpZG8Y9hT6aixZDP4AnFRaVbD kAYIvdOTlUqX3EAe40dI7/oH/ysAYR1knBQ41ljGML+UiRWSrkCjTwgN1ddKX8aCOlp99AGYm WF8h93FzTjsjy79uDeG1AoG9YhYa14SBUAuo+FYSO+du9HiVpIhqIU0QNTfPYsEJltsW++MpQ SzVJNqjalqRRMpiLI/5+Z64GARtYAfZ2qeTJoPT7d1KswJ5spW8HHp+VtXJepeDFrmLlPhXKq KNgQPDNQ314ww6BD2z+0V1Paj2H1RLMpi/2x6+x6VfqbTuzYPU+b2jb9f6GMC4H7YLL2RrAT3 ETaAhSDE1pA0YbMT8PPTsY4+pQ9wtJcm0BCOFWBOdyzGLf21AWbgmTcQgp+jRYhVN/hxjm13I bsXuQOjIiHaEIomkz54QP9DmdE57TaW5LG9oa+ZvhGJQABmgql4lZUBkExevwbJZEUP09qbSV ECjAIV24CAPDs+H0C7v+uJgdD25Z7WNNF8xjRxRQwh29eJcTOoEsge/+AbikSE/yX+xaZDjOU sPHrDJmSWc0hI2m077I6MLPbJ1Q4Wabbau+n2I9FNjeB+eEVNJ8Pu55C07zNcidOAH17qfyH4 47PklClm+OimUqMcI5/h/WLgnfbUEoskN1B8QkPtzizZiYdhhu1JdaK6gSMvqalQzEvxNtr4I a+M5hz4j+PA60CQam16rqP9/swphIocskfy2uygHa3o/ndclC1znawmnpSOPxKN690iRZK5l9 ne618w29wJ9kxEorQWCYNP6xMDhC4mpEyVMRO6loAniNH/eWAhP+6f2wJrAl81tPd+EWbu2QD qYJdKdO8O4GOitJLiBllFWUVxGP3EtDSR33Oc687zDhEpfSDXgjOmnSAEJuzbmweWyeP8NjFK MbQ8l8OmETXoNk2OULYxknZQXlA= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Ihor Radchenko <yantar92@HIDDEN> writes: Hi Ihor, >> On remote host, I can see that `find-lisp-find-files' must use >> tramp entries in `file-name-handler-alist'. Although, it will likely not >> be usable then - running GNU find on remote host is going to be >> unbeatable compared to repetitive TRAMP queries for file listing. > > That said, Michael, may you please provide some insight about TRAMP > directory listing queries. May they be more optimized when we need to > query recursively rather than per directory? Tramp is just a stupid library, w/o own intelligence. It offers alternative implementations for the set of primitive operations listed in (info "(elisp) Magic File Names") There's no optimization wrt to bundling several operations into a more suited remote command. > GNU find is faster simply because it is running on remote machine itself. > But AFAIU, if TRAMP could convert repetitive network request for each > directory into a single request, it would speed things up significantly. If you want something like this, you must add directory-files-recursively to that list of primitive operations. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 11:44:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 07:44:46 2023
Received: from localhost ([127.0.0.1]:38848 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNXW2-00067T-5x
for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 07:44:46 -0400
Received: from mout.gmx.net ([212.227.17.22]:50907)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <michael.albinus@HIDDEN>) id 1qNXW0-00067F-2G
for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 07:44:44 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de;
s=s31663417; t=1690112670; x=1690717470; i=michael.albinus@HIDDEN;
bh=1a+NW7ugdQ/3HVhlZJk7ylEVb6Sfu4llkirb+q+P4rY=;
h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date;
b=L7fc4sK8BfREX/qB5ZwEyO9WU/9v5K+h7Eri9Va7lt1SGKaqf0Z9iSJH58OmR66Gz5wSSDb
AjDMo4Ny/d/9+cEW/nL9jaDUG1dh19tBoyUdv9GZxESh5wZCvndLcMTRDqQ32Ei7BKDDdeh/E
FgQSZJzgs1PdJJ278Q6i/4wtUk9rTD6Sk+atnB9XfyY9NI9zfZjtmKEgi8e6YPIJMjcnbkGXE
C/p6AGABrK9t3Y2seiTWslHdeDLnysG054o7auD8XCirjuE2hSYGCtu2zGlPE/NHQo9wslFc2
JZx5jbz2H/dvQ4Dj30lVKPGnSv/fshVB/2qoU+EugswX8OB2HywQ==
X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a
Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx104
[212.227.17.168]) with ESMTPSA (Nemesis) id 1MKbg4-1qbv860BCu-00Kzqg; Sun, 23
Jul 2023 13:44:30 +0200
From: Michael Albinus <michael.albinus@HIDDEN>
To: Spencer Baugh <sbaugh@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <ierwmyru0r2.fsf@HIDDEN> (Spencer Baugh's message of
"Sat, 22 Jul 2023 16:53:05 -0400")
References: <iermszrwqj4.fsf@HIDDEN> <83v8edzb31.fsf@HIDDEN>
<87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost>
<87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost>
<87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN>
<878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN>
<87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN>
<87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN>
<ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN>
<87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN>
<87cz0jj25g.fsf@HIDDEN> <83wmyr7sbq.fsf@HIDDEN>
<ierwmyru0r2.fsf@HIDDEN>
Date: Sun, 23 Jul 2023 13:44:28 +0200
Message-ID: <875y6azwbn.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K1:RtX3AJPAFvxVmGaea4ktSBDmAWhoCvKfotBeglCWyt8Cl7Tlkbn
DZ+xaXWrGfpsmRexHmHxFM+yI4s6Ab211WHQFC1BJ3CgwOiB3yGOO/8gdG/eaYXt87/cd97
/DVpEQOeve8JNKsqxWWK37KJphLhuaRz5nscmuB5u1tKX1ovguN8Qe0C8ivOXfZuIVcrvPf
owcfxAn2Ezy/C+wGBUYqQ==
X-Spam-Flag: NO
UI-OutboundReport: notjunk:1;M01:P0:3xh9gDaq5JU=;zjbC969cWaKqfEQ46b3/DnW1fJf
t5q6oPu22QmvMgYpOUYZO772sEmHYTrdS9EZTuuHIEiS6r4XEjMc394lOqY/UZ2Yr6ZyNM3mX
Ik3aq+3sK34ITMBzFGEVFJheVgjNte5xKbWgGeSaSx37vKJC/9/cFRQbHNKXky7Ze21ZsD0tM
1crfntNxTQnqgMMYEIlm98LuLI87RyJI9pZXGEjlRkSXlMi7/s1uSdA3J+al9Y73XVpg8qQ2Q
RxJtgFMZzYrghjasbrmwoloZ38qsLqTtWBvBWAfsrXFX/vJ9BUQ31nzdrG5YG7FCQVIKmYGN1
UsVmRA+GdPs5cx8sz3zRftLRHV6OlfI3EI3cBIcBoNQeynEoAhBSHSgdVSUF5VWUJk34ZSGcR
ROrXyLlzvPiZTCg8+dJOIQh490zo0NcGUTHQnxjYekgd5YEWLQusZVVB+auyPLaMh5lWfPqRO
d7ph4oC5sBgh1jY9d01XKO0hSIkDKCjwxx5i3xqqVit5TW+haTUW2dnHuR2BspJO7t/f4OeEJ
2Yjk+geP/X9Xkik91nZn0WguZvwnB3KyoO9QBYCffcP/d+ZAtZAJUbxXauGddGoeTdjMQgjv9
HIozorDNtuCdIF8cODPH4SbiPYdNxktjNR3sfkd9599enZZRtEvps5ZUTsaVDoMU0X2KOvZXS
jzCTLa53Gt9ZMyeb3dMayEdsAP/+5vsCEBZ5qz6K9wPmdhbEkbOuVS+8YLTY/DLJTIm2OY9K2
m6PgdiER44h+Mn0xkB6x6ueoL47WOX7NUw4ICNxoqtk+wR1ZjxrD/++LsiOIASLHUaavVHsG3
MX1Tx7Vh2gPFRUFzYkmAxMQlt7o/lmjSaLl9WPZj/glPd+4Ah+r6UTCVQN1SY+L2bYeYxDvA/
bNInyF4/ZqOM377mEd0rbRxCQG2liPCHK45egkRQsS2PqPMs+TEp+K6aYBqUSjdAs1YZJiN2Z
bQCo6OHF5RI/Z1QlUA6HQZL6iUA=
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 64735
Cc: yantar92@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN,
Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
Spencer Baugh <sbaugh@HIDDEN> writes:
Hi Spencer,
> I mean having Emacs read output from the process and turn them into
> strings while find is still running and walking the directory tree. So
> the two parts are running in parallel. This, specifically:
Just as POC, I have modified your function slightly that it runs with
both local and remote directories.
--8<---------------cut here---------------start------------->8---
(defun find-directory-files-recursively (dir regexp &optional include-directories _predicate follow-symlinks)
(let* (buffered
result
(remote (file-remote-p dir))
(file-name-handler-alist (and remote file-name-handler-alist))
(proc
(make-process
:name "find" :buffer nil
:connection-type 'pipe
:noquery t
:sentinel #'ignore
:file-handler remote
:filter (lambda (proc data)
(let ((start 0))
(when-let ((end (string-search "\0" data start)))
(push (concat buffered (substring data start end)) result)
(setq buffered "")
(setq start (1+ end))
(while-let ((end (string-search "\0" data start)))
(push (substring data start end) result)
(setq start (1+ end))))
(setq buffered (concat buffered (substring data start)))))
:command (append
(list "find" (file-local-name dir))
(if follow-symlinks
'("-L")
'("!" "(" "-type" "l" "-xtype" "d" ")"))
(unless (string-empty-p regexp)
"-regex" (concat ".*" regexp ".*"))
(unless include-directories
'("!" "-type" "d"))
'("-print0")
))))
(while (accept-process-output proc))
(if remote (mapcar (lambda (file) (concat remote file)) result) result)))
--8<---------------cut here---------------end--------------->8---
This returns on my laptop
--8<---------------cut here---------------start------------->8---
(my-bench 100 "~/src/tramp" "")
(("built-in" . "Elapsed time: 99.177562s (3.403403s in 107 GCs)")
("with-find" . "Elapsed time: 83.432360s (2.820053s in 98 GCs)"))
(my-bench 100 "/ssh:remotehost:~/src/tramp" "")
(("built-in" . "Elapsed time: 128.406359s (34.981183s in 1850 GCs)")
("with-find" . "Elapsed time: 82.765064s (4.155410s in 163 GCs)"))
--8<---------------cut here---------------end--------------->8---
Of course the other problems still remain. For example, you cannot know
whether on a given host (local or remote) find supports all
arguments. On my NAS, for example, we have
--8<---------------cut here---------------start------------->8---
[~] # find -h
BusyBox v1.01 (2022.10.27-23:57+0000) multi-call binary
Usage: find [PATH...] [EXPRESSION]
Search for files in a directory hierarchy. The default PATH is
the current directory; default EXPRESSION is '-print'
EXPRESSION may consist of:
-follow Dereference symbolic links.
-name PATTERN File name (leading directories removed) matches PATTERN.
-print Print (default and assumed).
-type X Filetype matches X (where X is one of: f,d,l,b,c,...)
-perm PERMS Permissions match any of (+NNN); all of (-NNN);
or exactly (NNN)
-mtime TIME Modified time is greater than (+N); less than (-N);
or exactly (N) days
--8<---------------cut here---------------end--------------->8---
Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 11:43:20 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 07:43:20 2023 Received: from localhost ([127.0.0.1]:38844 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNXUd-000654-NP for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 07:43:20 -0400 Received: from mout02.posteo.de ([185.67.36.66]:58097) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNXUa-00064n-Ll for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 07:43:18 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 6007D240104 for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 13:43:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690112590; bh=EprmvCXSJ/iOtbRGNLhnACUSKqCsfdR3S/u16gsSWyo=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=r7ha4COXbDsaW5xacSz2vbExKt1206Xe0gARCR1Xpn6vT/KIWBvSr2xGhH5T4Ny0U ffUzVJE7A+OxtKxbYwW/pelPtvefI1mrjGRaThD3sE4OEWhE3pb3fOydg8ReJA+sE0 YGgF80fe4rLXHg4Fhwo669uxeayRXvdVFjhfkA6eqn49TRFHFZEKIGyWAO5dVErFVv Q2u8ewi/Rw2fIX/c78iZ+HeLklVhWz4Yy9JRqIbcCajsoCTzQ0PtRGj4S7jm1dqCcW ZfyNXiLUZM/vhk4T/G4SbWQ5E+u49tKCuU1oT6g7QeavJr/Xy4NSJ1yBotNgvaNKbu WBPWNVpr0lO7g== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R81d90Vb1z9rym; Sun, 23 Jul 2023 13:43:08 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83v8ebszcy.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> <83y1j7t19d.fsf@HIDDEN> <87r0ozq7ze.fsf@localhost> <83v8ebszcy.fsf@HIDDEN> Date: Sun, 23 Jul 2023 11:43:22 +0000 Message-ID: <87v8ean99h.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> But what is the strategy that should be used to call the CALLBACK? >> You clearly had something other than "call as soon as we got another >> file name" in mind. > > It could be "call as soon as we got 100 file names", for example. The > number can even be a separate parameter passed to the API. Will consing the filename strings also be delayed until the callback is invoked? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 11:17:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 07:17:53 2023 Received: from localhost ([127.0.0.1]:38733 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNX60-0002mM-OM for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 07:17:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45700) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNX5w-0002m7-D1 for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 07:17:50 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNX5q-0001KP-Pa; Sun, 23 Jul 2023 07:17:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=881VaWWR6/6hmLW60V6Rrk0SLr4RLKmpV1Q6Z3WqHT8=; b=KplbbSoe/fhE fm6pqJPDba1OcI83HvAVflMsD6RWHOZ/pe57MV0WwF3zUa+UKgGo16NHIKyypfCxOyffxflqSqSFJ KoRODxjdSB7qfkqZOFkBBmFHbEcjFE1XglBbFaCUlD4x0Wp71KOsFoE/0oOkC4zUKXCO5Tu9fciYF EdC0jLc9g1P79I3CQLaUmO8bzbUkYrSL3XhZKMCj7VwoRFrS4TQhf32j6rYab27AL8lUWBB217Ozc VvFZ6AOYLZoMwJfDpv6eVt4R8ArawTZyEmVV6iqgnq1jLTdcSjAaV1RCQRuZsI/RPlExvwOVQkPmh utFdpqCE9Zv8WU3puaMRWw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNX5q-0001Pt-8j; Sun, 23 Jul 2023 07:17:42 -0400 Date: Sun, 23 Jul 2023 14:18:22 +0300 Message-Id: <83sf9eub9d.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> (message from Dmitry Gutov on Sun, 23 Jul 2023 13:46:30 +0300) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Sun, 23 Jul 2023 13:46:30 +0300 > Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, > 64735 <at> debbugs.gnu.org > From: Dmitry Gutov <dmitry@HIDDEN> > > On 23/07/2023 08:11, Eli Zaretskii wrote: > > Even better: compute completion-regexp-list so that IGNOREs are > > filtered by file-name-all-completions in the first place. > > We don't have lookahead in Emacs regexps, so I'm not sure it's possible > to construct regexp that says "don't match entries A, B and C". Well, maybe just having a way of telling file-name-all-completions to negate the sense of completion-regexp-list would be enough to make that happen?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 10:46:44 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 06:46:43 2023 Received: from localhost ([127.0.0.1]:38602 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNWbr-0001oT-Jp for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 06:46:43 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:40321) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qNWbp-0001oG-IO for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 06:46:42 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 79D1C32002E8; Sun, 23 Jul 2023 06:46:34 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Sun, 23 Jul 2023 06:46:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690109194; x=1690195594; bh=OVh2saTZlwwRL/aJOIcnqc6iHo8cueNIxyt Qm0eHLU4=; b=fxLL6paXptYloogOeih4wkjYUat6Qbu5Ek9cm+wMjwZ8vdFvj/A eT2PTfLoGqxhTSm0r0nWFoQ8es+OKoV9VTjuf7WR9egPj3KJKg5ZdBdAtGK1Xnxu JSWZaJfUGuYmITNpEKFmagpLa2lpBxgxvXK+7I0GwW3+9XDHEXGMzo1WRy+Jqw11 QjYem72iLG7FDDb8AkxiYYSfmYGwiK1FMWhCpIyghISbRW7R8I7ZithAOknGxxm+ w8HZF1w9e2StpookLBPYwljKGpUX65FzAA3iscoVi7VKZQiep1VFrHeX6K++vNGc V7WOH28+zLBiZHBeAeoUhvYMV0U1NA/wujw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690109194; x=1690195594; bh=OVh2saTZlwwRL/aJOIcnqc6iHo8cueNIxyt Qm0eHLU4=; b=bjrfmGuHrbGrhvhiT0/DZWWDta4lFRblQU5tr1JFcrhsJrLJqzK V5rRkUIrMhuGiV/BOOZLvxVdgaOPT/08U8jIdPxO0g7DaIZ2BcgW4IT9X81zCQb4 6I69Yu5TgXg3Jaaom25muvkvu/qASL3aEnyNx9qgqQ32jcWke7HfythmsvJUWBVr z+2dRtiI57lX9Kyl/qHGPajFWg4AhSE3RtzoC4sEMHX8XAhCp0TGeoRI/wNl3Paa yNPYBzdpKGTzGb7j9rHFgggg4i0YaMOxz0koKFsVluwExc813h5nxG1h1qNhq+pl f0JqQJa3BZoSuy4JtSFA/1F79D3mh8uhKKg== X-ME-Sender: <xms:CQW9ZH9r7rVqsgdQYLgrZ8hti7Henk_zJ6LudVs_MNO0TkDBDC7yQQ> <xme:CQW9ZDv7IS6siXnyAOJesQtBOxYu-NY-RjvYaNqRZSGsCUB0RDiAieoD0HW88nX30 4bKQVutzZm7czBW7pE> X-ME-Received: <xmr:CQW9ZFBS8-fjOCPOlY89v9NZB8ewF8c7se977lL4zcyTRfIbfAdiOMNvJcDWhlg> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheeigdefudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:CQW9ZDdYMeUUcwzhbQENCX7c-qTsOS04D8H3mFHSETz66ADo4SfOMw> <xmx:CQW9ZMO9Oyyq7sAFPEutCe2bOLDbCNU7U1Xr8a7b-8pdBC6U6KMKHw> <xmx:CQW9ZFmalwwkuZ4x0dvHHPm41fBZjE4h5kApedUCfHOHHLlj-9T2aw> <xmx:CQW9ZBqS7UPBLiq2wVPIAX7InMUozOCsyec3TzofKvmUsq64YJMG_w> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 23 Jul 2023 06:46:32 -0400 (EDT) Message-ID: <c8eab12f-1978-7385-c142-c590642d35f0@HIDDEN> Date: Sun, 23 Jul 2023 13:46:30 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> <83sf9f6wm0.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83sf9f6wm0.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 23/07/2023 08:11, Eli Zaretskii wrote: > Even better: compute completion-regexp-list so that IGNOREs are > filtered by file-name-all-completions in the first place. We don't have lookahead in Emacs regexps, so I'm not sure it's possible to construct regexp that says "don't match entries A, B and C".
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 10:20:36 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 06:20:36 2023 Received: from localhost ([127.0.0.1]:38494 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNWCa-0000ta-52 for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 06:20:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40440) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNWCW-0000tM-Nt for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 06:20:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNWCQ-00018n-SC; Sun, 23 Jul 2023 06:20:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=ELRLkdMoVTbXQqvXXCkSUP9O4geSC2i600VTeWLpWo0=; b=kfcnGO8JJ/4I e7QaaWhh9cwyt+6Nzb8hdXL7z62JzqG/KUHSf+2nIz2zFOhI9rZSC9c4uT3VeKE0dxRjDsnFsdtGt bgWv80Em3NFMYe3bhq/NceO3Ip3QRWLgJQc+Ub9oeQGDiV/wH+Oq+mXuOh0ACVDz5PAt9Nfz0teOv 4kYgS6tn/OGBW5kVYomtrkAYWXXBy0APWO/2ti3Gvjwiqx1eBcN30UvRTP0w4nTpiYaiZQMbfGin0 pbgQKDLOqWRv6b+eefOSA+D4drgYMLpoe1GUuLZ0lCcl2XGLauRP0OpRvvnQTYPoAbORlkkg+QGDF n/QtmPp38lWN3qA9q9+IAA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNWC6-0003yp-9h; Sun, 23 Jul 2023 06:20:19 -0400 Date: Sun, 23 Jul 2023 13:20:45 +0300 Message-Id: <83v8ebszcy.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87r0ozq7ze.fsf@localhost> (message from Ihor Radchenko on Sun, 23 Jul 2023 09:42:45 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> <83y1j7t19d.fsf@HIDDEN> <87r0ozq7ze.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sun, 23 Jul 2023 09:42:45 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > Think about an API that is passed a function, and calls that function > > when appropriate, to perform caller-defined processing of the stuff > > generated by the API's implementation. That function is what I > > referred to as "callback". > > But what is the strategy that should be used to call the CALLBACK? > You clearly had something other than "call as soon as we got another > file name" in mind. It could be "call as soon as we got 100 file names", for example. The number can even be a separate parameter passed to the API.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 09:42:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 05:42:40 2023 Received: from localhost ([127.0.0.1]:38341 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNVbr-000896-Rm for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:42:40 -0400 Received: from mout02.posteo.de ([185.67.36.66]:37367) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNVbq-00088r-22 for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:42:38 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 65B52240104 for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 11:42:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690105352; bh=ZrAFA289wcl72Cj2hrJf7GXH0mUzbvs1bRCt432M54c=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=bGdmukiHk+diLRDsPgn2+DBNZfpK6QjVdfkP4tb4wm1hTCSX6PP0sg5tknFSetplP pXN0oyMH5TTYdxDYE2L4ipvWh+94XXg9oXJJJP1y1ejYorkrJA6GyyB7cdJtBlV3eq SuCbAu7ras+fAQJKDGzv6dx5cCgh7fvj3k095c7spOzrB8H2PQAKcLAVxqy3Uhec26 m1vjoK8NM7DZU8kpAISscF9FzXi8g1jA2XloK9Yx6jzJsTg39ZQiWEprXanrHmn5Kx qrZB9mk/EAizCl2iVMnAFugIg5fdhrJtG+IMyieipFZpm72X+mTvxg1iu2WFc6M8FK 4SJCycV/yHEgA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R7yxz5QTyz6tsj; Sun, 23 Jul 2023 11:42:31 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83y1j7t19d.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> <83y1j7t19d.fsf@HIDDEN> Date: Sun, 23 Jul 2023 09:42:45 +0000 Message-ID: <87r0ozq7ze.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> The only callback paradigm I know of in Emacs is something like process >> sentinels. Do you have something else in mind? > > Think about an API that is passed a function, and calls that function > when appropriate, to perform caller-defined processing of the stuff > generated by the API's implementation. That function is what I > referred to as "callback". But what is the strategy that should be used to call the CALLBACK? You clearly had something other than "call as soon as we got another file name" in mind. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 09:39:18 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 05:39:18 2023 Received: from localhost ([127.0.0.1]:38337 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNVYc-00083c-6M for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:39:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56810) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNVYZ-00083M-5n for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:39:16 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNVYT-0004KL-15; Sun, 23 Jul 2023 05:39:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=IGX+/bxKZxxUf1iz/6sv4rK/YsQAy3SCtB+aTgAZthw=; b=Artk5X431Kux 3nYRi7WLbjwzOy2kX3svtuj45NfRBbeXZQPc4Va6rG36eNREGb/eSCGAcy8oZ7pluWFLQOO+Lgz3O Cx3qh0gIlT11rL5FGdWw3eyMdWU5+LUPZjxuJcttn2EL7dLdoZ/B+AAEw22vijixWlElYVB0dlbYW yZ+gAQojhMM+uH7dowfhZAFZefr8dSXHfjl/Tttm5LIUALBGm/P5iJ3n3DPJC1x+qUtDOqFkRHlpE OOroCJYITj+sseutaEd8Ym8usdVUg5y4gv6Fv46uWWrip3Tu6XihHVGksByAM1dJraeRK59dsGKXM 1NliMtkNxPLSI71BR+fbmQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNVYM-0004Ft-Jl; Sun, 23 Jul 2023 05:39:02 -0400 Date: Sun, 23 Jul 2023 12:39:42 +0300 Message-Id: <83y1j7t19d.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87tttvq8df.fsf@localhost> (message from Ihor Radchenko on Sun, 23 Jul 2023 09:34:20 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> <87tttvq8df.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sun, 23 Jul 2023 09:34:20 +0000 > > The only callback paradigm I know of in Emacs is something like process > sentinels. Do you have something else in mind? Think about an API that is passed a function, and calls that function when appropriate, to perform caller-defined processing of the stuff generated by the API's implementation. That function is what I referred to as "callback".
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 09:34:16 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 05:34:16 2023 Received: from localhost ([127.0.0.1]:38290 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNVTk-0007sS-7e for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:34:16 -0400 Received: from mout02.posteo.de ([185.67.36.66]:53409) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNVTi-0007sA-1z for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:34:14 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 1C195240104 for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 11:34:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690104848; bh=QBw7MsDGIGZlRQu65Gv4tqoLRdlZs/8VxZr+L286Lq4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=QOpb67BzjPtKuQAzMczKdy1c/1JZWvrtxqfav+Go6uSJTTtZM1b3V1p2dqx2bwUuN GquEgBwR6APHIYsqfxZrJ78Cf9uxCVSNzOH73TNSRKNIOP2C/CTjxrENQtLR/2N0rF tyZlwLcNEZsqPR6pwlqFs6wbmphGNxezpqTdh6/BytDQMucpfUZL6HR+5fkhggKh9A znLcQpAMUWLwEy90oJa2nzV7R60elJ07zjitLRKimHHr67Sl1t1akG442ikHat0uDK IPMNbn3TF2JzhCToPvxzFz6Qh/a+oNMN2nB68j3DTkiWC3s+B2iI87FKACvU4YLq0F D6JERKUTSZP3w== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R7ymH0R8rz9rxg; Sun, 23 Jul 2023 11:34:06 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <831qgzuh5n.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> <831qgzuh5n.fsf@HIDDEN> Date: Sun, 23 Jul 2023 09:34:20 +0000 Message-ID: <87tttvq8df.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> Consider (async-directory-files-recursively dir regexp callback) with >> callback being (lambda (file) (start-process "Copy" nil "cp" file "/tmp/")). > > What is async-directory-files-recursively, and why are we talking > about it? I was talking about an implementation of > directory-files-recursively as a primitive in C. That's not async > code. So I don't understand why we are talking about some > hypothetical async implementation. Then, may you elaborate about how you imagine the proposed callback interface? I clearly did not understand what you had in mind. >> `async-directory-files-recursively' may fire CALLBACK very frequently. >> According to the other benchmarks in this thread, a file from directory >> may be retrieved within 10E-6s or even less. Elisp will have to arrange >> the callbacks to run immediately one after other (in a queue). >> Which will not be very different compared to just running callbacks in a >> synchronous loop. > > Regardless of my confusion above, no one said the callback must > necessarily operate on each file as soon as its name was retrieved, > nor even that the callback must be called for each file. The only callback paradigm I know of in Emacs is something like process sentinels. Do you have something else in mind? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 09:16:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 05:16:38 2023
Received: from localhost ([127.0.0.1]:38240 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNVCf-0007LG-Pi
for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:16:38 -0400
Received: from mout02.posteo.de ([185.67.36.66]:59877)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qNVCd-0007Ky-BZ
for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:16:36 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout02.posteo.de (Postfix) with ESMTPS id 4BB4C240105
for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 11:16:28 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1690103788; bh=rF7PD5bqsHqSjJtB3XArwjW93kixk3qz+l8Ud0BWxV8=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=hxItngyDe2KJxT/fqxYwzsmPy/ggMCpwCbH97Nd19epWIY14u9oaZ6kg7A7ChG6jO
fKiLuYgH9RwHlTu58NIFGHIqGENq4QMp/+mhlhabqdOtbbqoH9dKXDSXOqCpfFG8CX
yjU7W19dlHJFIVUjw8cUv02tm+AzTfbkiDO0nOoo2Xoxua4KBdMN51CC1alPAvKROa
aMXXX9ZGie9kzvi9w0wh+DO76jaITQn9O465bwHvQrMRpBVl313jthW+76aMhdsO1G
grTSkZ8W4/3HnNL30Yv7TYwci/iTqt09jZZpd/12sGNMpD5BB60ZRaQUAcxiPmWGSv
O13vcA8wRzqqQ==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R7yMt5X8Cz6v34;
Sun, 23 Jul 2023 11:16:26 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <83zg3nt2h5.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN> <878rb9l1f5.fsf@localhost>
<87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN>
<878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN>
<87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN>
<83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN>
<83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN>
<83sf9g88eh.fsf@HIDDEN> <87cz0jj25g.fsf@HIDDEN>
<83wmyr7sbq.fsf@HIDDEN> <ierwmyru0r2.fsf@HIDDEN>
<878rb7rrtu.fsf@localhost> <83edkz6ohq.fsf@HIDDEN>
<87zg3nqbzu.fsf@localhost> <83zg3nt2h5.fsf@HIDDEN>
Date: Sun, 23 Jul 2023 09:16:40 +0000
Message-ID: <87wmyrq96v.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Eli Zaretskii <eliz@HIDDEN> writes:
>> With emacs -Q, the results are similar in terms of absolute time spent
>> doing GC:
>>
>> (("built-in" . "Elapsed time: 5.706795s (3.332933s in 304 GCs)")
>> ("built-in no handlers" . "Elapsed time: 4.535871s (3.161111s in 301 GCs)")
>> ("with-find" . "Elapsed time: 4.829426s (3.333890s in 274 GCs)"))
>
> Strange. On my system, GC takes about 8% of the run time. Maybe it's
> a function of how many files are retrieved?
Most likely.
(length (directory-files-recursively "/usr/src/linux/" "")) ; => 145489
My test is producing a very long list of files. 10 times for each test
and for each function variant.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 09:13:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 05:13:00 2023
Received: from localhost ([127.0.0.1]:38228 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNV9A-0007Dy-4M
for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:13:00 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:42972)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qNV98-0007DU-4Q
for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:12:58 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNV91-0005tP-Vq; Sun, 23 Jul 2023 05:12:51 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=gu6Y93dE8rkqypWonem802WzrkkpVaRtr3Tke3xuog8=; b=NQE38lWn2tpi
0U4m33pRt79nuiaQi1YAIxZuiT9Z+6KqduM9ExpIxXIdh3EfyYFlBKMszyRuoUVOz/n4rzv77oslY
C3IQpxof7q/A5Sj3vLTqUoIB0GNsNJvyfdJ2BD6kfbzmLx23IveOMS4wkZ9LasEiOJOWiW+AFlPCg
GmGdqxha0dccVfirx7/bjt6MTwHrzZquwxt/aLUZRuNTC93mz2ve0g+0hJ0KMf/7DQXKV1M4s5ok9
YTZMlTrl0BXnmWJHSE+YjupT57PuXk2dNilqU9zgM0llvh+el7q6eVP3J3TBL6pA7UF0AWWgjBDwA
ZBIXT1yR/8qsP+dRdidt4g==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNV8w-0005rG-0F; Sun, 23 Jul 2023 05:12:46 -0400
Date: Sun, 23 Jul 2023 12:13:26 +0300
Message-Id: <83zg3nt2h5.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <87zg3nqbzu.fsf@localhost> (message from Ihor Radchenko on Sun,
23 Jul 2023 08:16:05 +0000)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost>
<87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN>
<87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN>
<ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN>
<87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN>
<87cz0jj25g.fsf@HIDDEN> <83wmyr7sbq.fsf@HIDDEN>
<ierwmyru0r2.fsf@HIDDEN> <878rb7rrtu.fsf@localhost>
<83edkz6ohq.fsf@HIDDEN> <87zg3nqbzu.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, rms@HIDDEN, dmitry@HIDDEN,
> michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
> Date: Sun, 23 Jul 2023 08:16:05 +0000
>
> With emacs -Q, the results are similar in terms of absolute time spent
> doing GC:
>
> (("built-in" . "Elapsed time: 5.706795s (3.332933s in 304 GCs)")
> ("built-in no handlers" . "Elapsed time: 4.535871s (3.161111s in 301 GCs)")
> ("with-find" . "Elapsed time: 4.829426s (3.333890s in 274 GCs)"))
Strange. On my system, GC takes about 8% of the run time. Maybe it's
a function of how many files are retrieved?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 09:10:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 05:10:47 2023 Received: from localhost ([127.0.0.1]:38188 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNV71-000780-6b for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:10:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNV6z-00077o-46 for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 05:10:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNV6s-0005X3-98; Sun, 23 Jul 2023 05:10:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=XyFizPQv1TXWRTtIdqBl0CT17U4awXxv1d0aZh8do2g=; b=b46MJ6HVUqPB K26MRhoJLG8ruNDAu6ViMU9UJe48UxvsI9arFi9DsDOsnL/jV7hrxBjiJlqMH8tGQRfhPFeLUWK1Z iSbDN5zXxYCCazmkPCHzAIYXfiMSFC+HFTfvRA/zXUiKlBCpMtY2uJX1JTr+5Yb6he3Ew7HZB+LKK nVZe5M8SpHPMZ5rFZwwBbV7alaV2tbG1oAus0hxDN9tf+I31tNX2oeMVl0du57CXN8B9MVut36Vc9 Axmht3NoXylQETApefn0gCHTMXM69Ok2MC+NTepFvMtOZ40W3rLSkOij91gKxXFcNF5aK4pyYzya8 vzMQTQVWtR3C4kkJOVI3Eg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNV6e-0005cX-Ch; Sun, 23 Jul 2023 05:10:32 -0400 Date: Sun, 23 Jul 2023 12:11:00 +0300 Message-Id: <831qgzuh5n.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87351frqr7.fsf@localhost> (message from Ihor Radchenko on Sun, 23 Jul 2023 08:11:56 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> <87351frqr7.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sun, 23 Jul 2023 08:11:56 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> But won't the Elisp callback always result in a queue that will > >> effectively be synchronous? > > > > I don't understand the question (what queue?), and understand even > > less what you are trying to say here. Please elaborate. > > Consider (async-directory-files-recursively dir regexp callback) with > callback being (lambda (file) (start-process "Copy" nil "cp" file "/tmp/")). What is async-directory-files-recursively, and why are we talking about it? I was talking about an implementation of directory-files-recursively as a primitive in C. That's not async code. So I don't understand why we are talking about some hypothetical async implementation. > `async-directory-files-recursively' may fire CALLBACK very frequently. > According to the other benchmarks in this thread, a file from directory > may be retrieved within 10E-6s or even less. Elisp will have to arrange > the callbacks to run immediately one after other (in a queue). > Which will not be very different compared to just running callbacks in a > synchronous loop. Regardless of my confusion above, no one said the callback must necessarily operate on each file as soon as its name was retrieved, nor even that the callback must be called for each file.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 08:16:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 04:16:00 2023
Received: from localhost ([127.0.0.1]:37975 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNUG0-0005Xq-Hg
for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 04:16:00 -0400
Received: from mout01.posteo.de ([185.67.36.65]:58511)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qNUFy-0005Pt-9X
for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 04:15:59 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout01.posteo.de (Postfix) with ESMTPS id B6C6E24002A
for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 10:15:52 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1690100152; bh=OEgX9EtsdGK9/dIQ1lkCaREWAVADYBy4nszJGFUkR2o=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=ptMWYC7oFQLEHRQ7O9FbIi0EV2mKDjNxuDuN8r7Z4TxC+KhO5LLJDEp+fdeLC/kal
BsEvKyInofungXFzs2eY56lCLLFQUC0ksnsxldF2h6PrzQ5v6N/Sg6qBZkOylURHTb
KN7VRdWHVs/elrC0ItOjG1C1Xn7coL6onRsr5wj/B3EROa5gLHhwBySmjS9eLznFmh
2afSiCewdXf1yM6Nn7N+h5Su0zt1XkWOa64Exey8gtG8g/9V/+T06ewV3GRAN3Bxww
OUMg1J0TPnNSi2ABjV0fJ/b6a2sIGpksXmz6xq69d1tcvobhkXPvpxT9NgZH680AcP
MUsM/jvJzPnNg==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R7x1z6vk2z6tvy;
Sun, 23 Jul 2023 10:15:51 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <83edkz6ohq.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost>
<87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN>
<87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN>
<ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN>
<87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN>
<87cz0jj25g.fsf@HIDDEN> <83wmyr7sbq.fsf@HIDDEN>
<ierwmyru0r2.fsf@HIDDEN> <878rb7rrtu.fsf@localhost>
<83edkz6ohq.fsf@HIDDEN>
Date: Sun, 23 Jul 2023 08:16:05 +0000
Message-ID: <87zg3nqbzu.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Eli Zaretskii <eliz@HIDDEN> writes:
>> (("built-in" . "Elapsed time: 7.134589s (3.609741s in 10 GCs)")
>> ("built-in no handlers" . "Elapsed time: 6.041666s (3.856730s in 11 GCs)")
>> ("with-find" . "Elapsed time: 6.300330s (4.248508s in 12 GCs)"))
>
> Is this in "emacs -Q"? Why so much time taken by GC? It indicates
> that temporarily raising the GC thresholds could speed up things by a
> factor of 2 or 3.
With emacs -Q, the results are similar in terms of absolute time spent
doing GC:
(("built-in" . "Elapsed time: 5.706795s (3.332933s in 304 GCs)")
("built-in no handlers" . "Elapsed time: 4.535871s (3.161111s in 301 GCs)")
("with-find" . "Elapsed time: 4.829426s (3.333890s in 274 GCs)"))
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 08:11:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 04:11:53 2023 Received: from localhost ([127.0.0.1]:37930 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNUC0-0002qg-Sg for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 04:11:53 -0400 Received: from mout02.posteo.de ([185.67.36.66]:36859) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNUBy-0002qR-4W for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 04:11:51 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 2B705240104 for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 10:11:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690099904; bh=OuBq0iEkxwNimnYc05k4ItVLvYsucmOrhiyXYRC3F9w=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=bALxblN2eu1XB878ibZ90/t+56aSUKZckksgjmYTI+8STC5eDtAP/GwXpb8ckCsIq qhyK9bZm0sfSXhIEI6/Oe/Xp/5pIRcmXNAaw7aa63ah9f51dMXsU9nRn+pbOCN3EzP RMxS+p56P8XOHhRqrGZ7yi93J88CHhPYwhxuQauCvi67ca9dA7BNPt0arceZ1ouCR/ wf5BB/qngoGPe4uBPY4eO1flVIKSjYx6i1mKiTUYF3949y30W7pTySg71yJX9pzdpY G8KppGzdd+4NEf723I6kNz05ZofyCWJ/ol2O24BT3DzKCt2hsvmkH5Tk9UtCVb+G9F Y9GqDqr5/thvQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R7wxB60mfz9ryh; Sun, 23 Jul 2023 10:11:42 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83fs5f6opg.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> <83fs5f6opg.fsf@HIDDEN> Date: Sun, 23 Jul 2023 08:11:56 +0000 Message-ID: <87351frqr7.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> >> Then how will the callback be different from >> >> (mapc #'my-function (directory-files-recursively ...)) >> >> ? >> > >> > It depends on the application. Applications that want to get all the >> > data and only after that process it will not use the callback. But I >> > can certainly imagine an application that inserts the file names, or >> > some of their transforms, into a buffer, and from time to time >> > triggers redisplay to show the partial results. Or an application >> > could write the file names to some disk file or external consumer, or >> > send them to a network process. >> >> But won't the Elisp callback always result in a queue that will >> effectively be synchronous? > > I don't understand the question (what queue?), and understand even > less what you are trying to say here. Please elaborate. Consider (async-directory-files-recursively dir regexp callback) with callback being (lambda (file) (start-process "Copy" nil "cp" file "/tmp/")). `async-directory-files-recursively' may fire CALLBACK very frequently. According to the other benchmarks in this thread, a file from directory may be retrieved within 10E-6s or even less. Elisp will have to arrange the callbacks to run immediately one after other (in a queue). Which will not be very different compared to just running callbacks in a synchronous loop. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 08:05:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 04:05:59 2023
Received: from localhost ([127.0.0.1]:37920 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNU6I-0002gN-TE
for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 04:05:59 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:38038)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qNU6H-0002gB-LV
for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 04:05:58 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNU6B-0001Gi-Ig; Sun, 23 Jul 2023 04:05:51 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=h3I7YMWCjCO4nGQwk5X66WSyEAEiffK2jrYwghV/2dE=; b=kN9SQYVOqmt3
gzK/+NQwZV6O1t/eURbf5fXnDgoESwemNGJzY9q5EtfW5HCCHAbQjaDtQoovoVpIRlp8GXq9PVURj
BXa7s0DPF64lJPQRHdNF5Az6FKGJYHcOG7Ofb5YhLqYbCwjJXLudEFebZXNqbNXOAqDg5fmucsZ/G
qsDNzfrbZjXz1unDtNs4JcswFONSi44JtHux9XAWSUARq6GAlqXGwsGhQpC0MBWOIbA+fbWhAFGX1
j5WaHCaqVNcn8ksy6og2jOfRTNy9UMhWvpeO/ym2TlN7QOH8kGoT4O7iptR0w+axXx2e4MFs6IU4D
25wJpJdkhzfHclv3wI1BeA==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNU65-0007yu-Gz; Sun, 23 Jul 2023 04:05:45 -0400
Date: Sun, 23 Jul 2023 11:06:25 +0300
Message-Id: <83edkz6ohq.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Ihor Radchenko <yantar92@HIDDEN>
In-Reply-To: <878rb7rrtu.fsf@localhost> (message from Ihor Radchenko on Sun,
23 Jul 2023 07:48:45 +0000)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN> <83v8edzb31.fsf@HIDDEN>
<87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN>
<87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN>
<ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN>
<87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN>
<87cz0jj25g.fsf@HIDDEN> <83wmyr7sbq.fsf@HIDDEN>
<ierwmyru0r2.fsf@HIDDEN> <878rb7rrtu.fsf@localhost>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Ihor Radchenko <yantar92@HIDDEN>
> Cc: Eli Zaretskii <eliz@HIDDEN>, sbaugh@HIDDEN, rms@HIDDEN,
> dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
> Date: Sun, 23 Jul 2023 07:48:45 +0000
>
> (my-bench 10 "/usr/src/linux/" "")
>
> (("built-in" . "Elapsed time: 7.134589s (3.609741s in 10 GCs)")
> ("built-in no handlers" . "Elapsed time: 6.041666s (3.856730s in 11 GCs)")
> ("with-find" . "Elapsed time: 6.300330s (4.248508s in 12 GCs)"))
Is this in "emacs -Q"? Why so much time taken by GC? It indicates
that temporarily raising the GC thresholds could speed up things by a
factor of 2 or 3.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 08:01:25 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 04:01:25 2023 Received: from localhost ([127.0.0.1]:37915 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNU1t-0002Zp-3i for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 04:01:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42688) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNU1q-0002Zb-Ut for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 04:01:23 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNU1k-0000aY-3K; Sun, 23 Jul 2023 04:01:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=IO3kwVpRdxe3KTO8kIoGaSYgBDBajNlIK1L1p+sNBZQ=; b=hln9IJciBXgu K39hjY+Pb5jn1HXneMu/xde6ZI0LwzrpDhIsN4ENvfcdeX7ch8eqPweMkcRYX31gjixF7Kklv2VwO oiy8XAhjV5fLVA/B65SAiOBDqW21Dn+54EJU5nLDpq27lYZZBO7po+UdQS7mV7lIN3foxPgyNDo/b HpNdF53U6fd5LBjmnmNtGU7AS5JNQGQ+DYPmH5WO/L1naBl4jTV9dUbkkTnraUCIcRh9XOM/7Lwmt NOjzkSTUc3QWa2ZJYCMg0lhi3plAQ6YpnzIIGS19fJ37xnEU7eXGwZi8aw94fTC6V3pRlWlGcabjL 0mQZ6v3cuOVqeyEL4beCAw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNU1c-0005jx-Ce; Sun, 23 Jul 2023 04:01:09 -0400 Date: Sun, 23 Jul 2023 11:01:47 +0300 Message-Id: <83fs5f6opg.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <875y6brrnk.fsf@localhost> (message from Ihor Radchenko on Sun, 23 Jul 2023 07:52:31 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> <875y6brrnk.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sun, 23 Jul 2023 07:52:31 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> Then how will the callback be different from > >> (mapc #'my-function (directory-files-recursively ...)) > >> ? > > > > It depends on the application. Applications that want to get all the > > data and only after that process it will not use the callback. But I > > can certainly imagine an application that inserts the file names, or > > some of their transforms, into a buffer, and from time to time > > triggers redisplay to show the partial results. Or an application > > could write the file names to some disk file or external consumer, or > > send them to a network process. > > But won't the Elisp callback always result in a queue that will > effectively be synchronous? I don't understand the question (what queue?), and understand even less what you are trying to say here. Please elaborate.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 07:52:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 03:52:26 2023 Received: from localhost ([127.0.0.1]:37892 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNTtB-0002Jd-V1 for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 03:52:26 -0400 Received: from mout02.posteo.de ([185.67.36.66]:57049) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNTtA-0002JL-CI for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 03:52:24 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id CB29A240101 for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 09:52:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690098738; bh=z4aKAFp8VnAm2PusYShpVkSIeEv0GiQgMWrtBdHXfAA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=PDaBsbQtt0DjNttxIXAfhfljM0cRUMVDKUjMUOyyAx/d5qrulcIiQXFhV8vsEE7KJ L5HAwkIiRZ2eRC/9t7RYlzWtVIs3Ss6MW+GS0eOTRYJw3h5gKiO5Wt2KBlkkq1TN8Q raTmoPGNrnqVm9lK5PmUsRG8G/zq9tLfUHImynVodzR/Qt4NNfNVQR8fO83NYNXs5O QmfsEUWYCoZmFPw6f2uV60vgT5MyZrw20KQCJkwLeQezNglhzZVW4R6sqxXAtPWoer gKGrNiAHTZJYQFnt9EP3zoF0beMk4L8IOz5KqWDHPDYu6l3KFqfA4TZZJVPMzT1MCE SkLjMTFSsF5JQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R7wVp1Hzyz6txs; Sun, 23 Jul 2023 09:52:18 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83351g7yn5.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> <83351g7yn5.fsf@HIDDEN> Date: Sun, 23 Jul 2023 07:52:31 +0000 Message-ID: <875y6brrnk.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> Then how will the callback be different from >> (mapc #'my-function (directory-files-recursively ...)) >> ? > > It depends on the application. Applications that want to get all the > data and only after that process it will not use the callback. But I > can certainly imagine an application that inserts the file names, or > some of their transforms, into a buffer, and from time to time > triggers redisplay to show the partial results. Or an application > could write the file names to some disk file or external consumer, or > send them to a network process. But won't the Elisp callback always result in a queue that will effectively be synchronous? Also, another idea could be using iterators - the applications can just request "next" file as needed, without waiting for the full file list. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 07:48:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 03:48:42 2023
Received: from localhost ([127.0.0.1]:37887 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNTpa-0002EH-6M
for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 03:48:42 -0400
Received: from mout01.posteo.de ([185.67.36.65]:35445)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qNTpX-0002E2-7D
for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 03:48:40 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout01.posteo.de (Postfix) with ESMTPS id 664BC240029
for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 09:48:33 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1690098513; bh=kwC49HbA+CFEF5i/2//Ly2M2KXKP7RE5AvzY+NmpUgE=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=VkuHlz/RW/EkqQCWeTtkP0EomoQBCItB1xEoSL/eb/vR3iQBkd+EXop5igkMJuta+
H5EREpkNAL474oBWlBALzx9YAAHBdOREf7R6vZMbrabkqJ5iP2sJnchWafaPd/YJFj
msEtL4L11pT5bV5g756wnfBF24T0A8hJx2IGakqBFqqkIYxmY2ZRQV1YnwIx2trR6+
p8I3B0QwqYrxBqHMG4LhhTL/1jmLKGou1fnZbZPBGnJn2YuyY8IEkbONfapi8RoBk5
DYxZiVG0hXaYRUaYZ23eQsgVZGU7b38tU2HnpskkkJYdsqsUAonwrjMVU9M++Cn0V7
nTPd03EApYDuQ==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R7wQS0lLCz6twG;
Sun, 23 Jul 2023 09:48:31 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Spencer Baugh <sbaugh@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <ierwmyru0r2.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN> <83v8edzb31.fsf@HIDDEN>
<87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN>
<87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN>
<ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN>
<87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN>
<87cz0jj25g.fsf@HIDDEN> <83wmyr7sbq.fsf@HIDDEN>
<ierwmyru0r2.fsf@HIDDEN>
Date: Sun, 23 Jul 2023 07:48:45 +0000
Message-ID: <878rb7rrtu.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN,
Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Spencer Baugh <sbaugh@HIDDEN> writes:
> Can you try this further change on your Windows (and GNU/Linux) box? I
> just tested on a different box and my original change gets:
On GNU/Linux, with slight modifications
(defun my-bench (count path regexp)
(setq path (expand-file-name path))
;; (let ((old (directory-files-recursively path regexp))
;; (new (find-directory-files-recursively path regexp)))
;; (dolist (path old)
;; (should (member path new)))
;; (dolist (path new)
;; (should (member path old))))
(list
(cons "built-in" (benchmark count (list 'directory-files-recursively
path regexp)))
(cons "built-in no handlers"
(let (file-name-handler-alist)
(benchmark count
(list 'directory-files-recursively path
regexp))))
(cons "with-find" (benchmark count (list
'find-directory-files-recursively path regexp)))))
(my-bench 10 "/usr/src/linux/" "")
(("built-in" . "Elapsed time: 7.134589s (3.609741s in 10 GCs)")
("built-in no handlers" . "Elapsed time: 6.041666s (3.856730s in 11 GCs)")
("with-find" . "Elapsed time: 6.300330s (4.248508s in 12 GCs)"))
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 06:15:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 02:15:00 2023
Received: from localhost ([127.0.0.1]:37840 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNSMu-0008Nc-1V
for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 02:15:00 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:39096)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qNSMr-0008NN-KW
for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 02:14:59 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNSMj-0001Be-VO; Sun, 23 Jul 2023 02:14:49 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=877rEBDiCdCpXvd5zPvzgYUFox7/ScyEiBA2QRapeh8=; b=J7lysJzc3f1g
EkDuqXW1rQmPdGgtY2GOmgGvcCcVc+Fnyv6GtU1PeM7Uk7CtZeUXgSzD5ak0jl0G975sSrxocgswE
JZKR8P5wkXH9Fm7xsTnCGg5UGN+NNSZMRoqaPULBgH0KAmI+hiXMbItkKXFrs5ESIxtNjv0dduVU2
/jAiw9ejWe+tjes7L8n3Yn+LWXuLlSXSC1qHjhfgHRqYM7wKeTPpeKxkJ1KFwLTqTLv3CmRd0O8jB
cwWJC0idjoP7y9E2nGr6Os6tYznvTPLnP5HP5VxFgxooOyxO6IH7DwbgYEPDWi4VRzAXNzM5hLiMK
LlpU6mSx4vMrVHRj9RoqsA==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNSMc-0005zf-QX; Sun, 23 Jul 2023 02:14:43 -0400
Date: Sun, 23 Jul 2023 09:15:22 +0300
Message-Id: <83lef76tmt.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Spencer Baugh <sbaugh@HIDDEN>
In-Reply-To: <ierwmyru0r2.fsf@HIDDEN> (message from Spencer Baugh on
Sat, 22 Jul 2023 16:53:05 -0400)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN> <87cz0lmoxy.fsf@localhost>
<83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN>
<87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN>
<83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN>
<83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN>
<83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN>
<83sf9g88eh.fsf@HIDDEN> <87cz0jj25g.fsf@HIDDEN>
<83wmyr7sbq.fsf@HIDDEN> <ierwmyru0r2.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: yantar92@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Spencer Baugh <sbaugh@HIDDEN>
> Cc: sbaugh@HIDDEN, yantar92@HIDDEN, rms@HIDDEN,
> dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
> Date: Sat, 22 Jul 2023 16:53:05 -0400
>
> Can you try this further change on your Windows (and GNU/Linux) box? I
> just tested on a different box and my original change gets:
>
> (("built-in" . "Elapsed time: 4.506643s (2.276269s in 21 GCs)")
> ("with-find" . "Elapsed time: 4.114531s (2.848497s in 27 GCs)"))
>
> while this parallel implementation gets
>
> (("built-in" . "Elapsed time: 4.479185s (2.236561s in 21 GCs)")
> ("with-find" . "Elapsed time: 2.858452s (1.934647s in 19 GCs)"))
>
> so it might have a favorable impact on Windows and your other GNU/Linux
> box.
Almost no effect here on MS-Windows:
(("built-in" . "Elapsed time: 0.859375s (0.093750s in 4 GCs)")
("with-find" . "Elapsed time: 8.437500s (0.078125s in 4 GCs)"))
It was 8.578 sec with the previous version.
(The Lisp version is somewhat faster in this test because I
native-compiled the code for this test.)
On GNU/Linux:
(("built-in" . "Elapsed time: 4.244898s (1.934182s in 56 GCs)")
("with-find" . "Elapsed time: 3.011574s (1.190498s in 35 GCs)"))
Faster by 10% (previous version yielded 3.327 sec).
Btw, I needed to fix the code: when-let needs 2 open parens after it,
not one. The original code signals an error from the filter function
in Emacs 29.
> >> (cl-assert (null _predicate) t "find-directory-files-recursively can't accept arbitrary predicates")
> >
> > It should.
>
> This is where I think a fallback would be useful - it's basically
> impossible to support arbitrary predicates efficiently here, since it
> requires us to put Lisp in control of whether find descends into a
> directory.
There's nothing wrong with supporting this less efficiently.
And there's no need to control where Find descends: you could just
filter out the files from those directories that need to be ignored.
> So I'm thinking I would just fall back to running the old
> directory-files-recursively whenever there's a predicate. Or just not
> supporting this at all...
We cannot not support it at all, because then it will not be a
replacement. Fallback is okay, though I'd prefer a self-contained
function.
> >> (if follow-symlinks
> >> '("-L")
> >> '("!" "(" "-type" "l" "-xtype" "d" ")"))
> >> (unless (string-empty-p regexp)
> >> "-regex" (concat ".*" regexp ".*"))
> >> (unless include-directories
> >> '("!" "-type" "d"))
> >> '("-print0")
> >
> > Some of these switches are specific to GNU Find. Are we going to
> > support only GNU Find?
>
> POSIX find doesn't support -regex, so I think we have to. We could
> stick to just POSIX find if we only allowed globs in
> find-directory-files-recursively, instead of full regexes.
The latter would again be incompatible with
directory-files-recursively, so it isn't TRT, IMO.
One other subtlety is non-ASCII file names: you use -print0 switch to
Find, which produces null bytes, and those could inhibit decoding of
non-ASCII characters. So you may need to bind
inhibit-null-byte-detection to a non-nil value to get correctly
decoded file names you get from Find.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 05:40:44 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 01:40:44 2023 Received: from localhost ([127.0.0.1]:37825 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNRpk-0007FJ-3S for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 01:40:44 -0400 Received: from mout02.posteo.de ([185.67.36.66]:49817) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNRpi-0007F3-AG for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 01:40:42 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 62CC9240103 for <64735 <at> debbugs.gnu.org>; Sun, 23 Jul 2023 07:40:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690090836; bh=/O98D43j93kzOB5g4OjE58C3gA5aOBcP1LlVc5HALB4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=hSoTdkQf5heem9Jgn/awG/POjasM5+9yikb0ETiTuI38/fsWPNrzdY6KgbriddSf1 HUus9PTc0Z+V4yvh9X/KrYCLvPscI86AsoqwsJeo5+ZK4kkCwRkDxU8BKSPHKLipa3 Vs1ZZu4v6XygRm21oRPK/8WXIP9AAg7HILYWkr9e4egxL/4Jdfw3vxLkuOPFvMQoWs EIyzB0k6Yglib9LRA+/SJ/K2qDqhBPUpIMXmmxY4xfeGCVdsgd6PNIXvp3bSkcaL+r 5EtZOnTs9AFwVYpSsQkxrbPXpHl76Ye65Cmi7CGcIdXieS34BbcKpEuYKX6TEW0euF si+9Xm4Tl9JPQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R7sZq2BDhz6v3P; Sun, 23 Jul 2023 07:40:35 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN>, Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87h6pxl6m4.fsf@localhost> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> Date: Sun, 23 Jul 2023 05:40:41 +0000 Message-ID: <87pm4jrxra.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Ihor Radchenko <yantar92@HIDDEN> writes: > On remote host, I can see that `find-lisp-find-files' must use > tramp entries in `file-name-handler-alist'. Although, it will likely not > be usable then - running GNU find on remote host is going to be > unbeatable compared to repetitive TRAMP queries for file listing. That said, Michael, may you please provide some insight about TRAMP directory listing queries. May they be more optimized when we need to query recursively rather than per directory? GNU find is faster simply because it is running on remote machine itself. But AFAIU, if TRAMP could convert repetitive network request for each directory into a single request, it would speed things up significantly. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 05:27:44 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 01:27:44 2023 Received: from localhost ([127.0.0.1]:37815 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNRdA-0006qu-Dx for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 01:27:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34136) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNRd8-0006qi-Tt for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 01:27:43 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNRd3-0003ey-D4; Sun, 23 Jul 2023 01:27:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=ozftes0KIR0dDxaZkvM2gnV4RFqZC4+i5S/16eEB4bA=; b=Ts1y1erjTOLv edKkrpMOkQTCB9pnHozMMtn3W/V/4vvCK1407flW0XazyPqJi6TpS3DfKB7zpgFrJKw+12Su4mv27 SoIcIejVwTujGKIcwIpqzju1u9qpHz1j1YD9lDSMGHhKvCY6lZQjHd4ijbQswqtJ4xaLuJ+25KhL5 bFKJL4cHZIHVTycEqy5Qkoe1iQ6dZMmzrtAf4j7gx5EVsOGdRof6kW7jebNuXH5khz7GoPSwzgrJb JPwFEYBVarG/54UlxbsPETkXZaAya9HZY97zWWKyFLgu5X4hUXZgoZEUc0SURQL1tYhUdw/CvlYbQ DZx+9XPLUsmk4pl40GrUfA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNRcv-0001jI-DD; Sun, 23 Jul 2023 01:27:29 -0400 Date: Sun, 23 Jul 2023 08:28:10 +0300 Message-Id: <83o7k36vth.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: rms@HIDDEN In-Reply-To: <E1qNPJG-0007gO-Km@HIDDEN> (message from Richard Stallman on Sat, 22 Jul 2023 22:59:02 -0400) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <E1qNPJG-0007gO-Km@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Cc: 64735 <at> debbugs.gnu.org > From: Richard Stallman <rms@HIDDEN> > Date: Sat, 22 Jul 2023 22:59:02 -0400 > > > If someone was thinking of doing that, they would be better off > > responding to RMS's earlier request for C programmers to optimize this > > behavior in find. > > > Since, after all, if we do it that way it will benefit remote files as > > well. > > I wonder if some different way of specifying what to ignore might make > a faster implementation possible. Regexps are general but matching > them tends to be slow. Maybe some less general pattern matching could > be sufficient for these features while faster. You are thinking about matching in Find, or about matching in Emacs? If the former, they can probably use 'fnmatch' or somesuch, to match against shell; wildcards. If the latter, we don't have any pattern matching capabilities in Emacs except fixed strings and regexps.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 05:10:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jul 23 01:10:53 2023
Received: from localhost ([127.0.0.1]:37784 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNRMq-0006Oh-MC
for submit <at> debbugs.gnu.org; Sun, 23 Jul 2023 01:10:53 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:54298)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qNRMg-0006OM-II
for 64735 <at> debbugs.gnu.org; Sun, 23 Jul 2023 01:10:51 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNRMX-0007yn-2n; Sun, 23 Jul 2023 01:10:34 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=PP/nBa58ywXZy/Nzu9OSSCrUpR7PO6qUPorW/0DafgM=; b=P4Iu5RaRlxTY
Pyoy7mAe5G1fKg10VGUPNYTlEQNLa/JZi+FnCC+pZAcLdxBMfAy7Nxj+GajuomWpCR0gUYpNSmCqE
9VV9xvP2WP1K6G+JWb6qWaMqmE5n5K769tXHRK8HIT1aYfo5bTGVI1+XK58xg78QThdN75FHJV4fp
oMA7qI9DvtVE59usYID7NGHfNveVTtiKjqJS9K4CtYiIJcdoJbIzMyb+DUz5htpGeAH10YCs6QExv
2taISrNjLjmPua4BI+fSb4Rw0gQMY/q4O4ibuisyyJRVSdDUoZRN1Gopzyn0UQegV8EB/7iK3WhdP
BFFEKofrB/TL20VtLtSkJA==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNRMP-00006F-At; Sun, 23 Jul 2023 01:10:32 -0400
Date: Sun, 23 Jul 2023 08:11:03 +0300
Message-Id: <83sf9f6wm0.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> (message from
Dmitry Gutov on Sun, 23 Jul 2023 00:01:28 +0300)
Subject: Re: bug#64735: 29.0.92;
find invocations are ~15x slower because of ignores
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost>
<ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN>
<fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: luangruo@HIDDEN, sbaugh@HIDDEN, yantar92@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Cc: Ihor Radchenko <yantar92@HIDDEN>, 64735 <at> debbugs.gnu.org
> Date: Sun, 23 Jul 2023 00:01:28 +0300
> From: Dmitry Gutov <dmitry@HIDDEN>
>
> On 22/07/2023 09:39, Po Lu wrote:
> >
> > Programs that use find should fall back to directory-file-recursively
> > when any of the situations above are detected.
>
> Perhaps if someone implements support for IGNORE entries (wildcards) in
> that function, it would be easy enough to do that fallback.
Shouldn't be hard, since it already filters some of them:
(dolist (file (sort (file-name-all-completions "" dir)
'string<))
(unless (member file '("./" "../")) <<<<<<<<<<<<<<<<<<<
Even better: compute completion-regexp-list so that IGNOREs are
filtered by file-name-all-completions in the first place.
Patches welcome.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 23 Jul 2023 02:59:12 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 22:59:12 2023 Received: from localhost ([127.0.0.1]:37723 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNPJP-0002x6-TY for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 22:59:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57856) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <rms@HIDDEN>) id 1qNPJM-0002wq-V3 for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 22:59:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <rms@HIDDEN>) id 1qNPJH-0004Az-17; Sat, 22 Jul 2023 22:59:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=Date:References:Subject:In-Reply-To:To:From: mime-version; bh=IXrGDaWxAQrwyJxip0erJsgL0sUlcn8DTEE7zFxxrCw=; b=i+hl7POsP/jB GblENesjP/9HudIrQ7edKxGDGoX2n5QIEh/5NmSt7r5T373uVtiCEimycJBldEe3Y2moj3iuZ9ESD 0jswU0urWj7gG5BxIHtsuhTcZnGCqrYR5rX0CtB+hONEPbvcF/FajwettvB050Qy2JLjoUPBLm9kv X42gIb9Mj1jzVtrToRDl4qkH+kHkYa4C1CzNC6HYragjZVIFyYeVcrsc2F1gjB5OaMANobgVeAtEh 9AOaYjFhBgWGpC5iLK5TjqoUm5z0ui6LByKO3V1Y3Svb1WkTDvDPN+hUOAeZzR9/XkI+kHiiwDEmg obaP53xDArtOKnao36WHrw==; Received: from rms by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from <rms@HIDDEN>) id 1qNPJG-0007gO-Km; Sat, 22 Jul 2023 22:59:02 -0400 Content-Type: text/plain; charset=Utf-8 From: Richard Stallman <rms@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> In-Reply-To: <ier8rb9ukjq.fsf@HIDDEN> (message from Spencer Baugh on Fri, 21 Jul 2023 15:33:13 -0400) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> Message-Id: <E1qNPJG-0007gO-Km@HIDDEN> Date: Sat, 22 Jul 2023 22:59:02 -0400 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Reply-To: rms@HIDDEN Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > If someone was thinking of doing that, they would be better off > responding to RMS's earlier request for C programmers to optimize this > behavior in find. > Since, after all, if we do it that way it will benefit remote files as > well. I wonder if some different way of specifying what to ignore might make a faster implementation possible. Regexps are general but matching them tends to be slow. Maybe some less general pattern matching could be sufficient for these features while faster. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org)
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 21:01:41 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 17:01:41 2023 Received: from localhost ([127.0.0.1]:37628 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNJjQ-0002B7-Mf for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 17:01:40 -0400 Received: from out2-smtp.messagingengine.com ([66.111.4.26]:48027) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qNJjO-0002Ao-P7 for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 17:01:39 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 5E84A5C008C; Sat, 22 Jul 2023 17:01:32 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Sat, 22 Jul 2023 17:01:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1690059692; x=1690146092; bh=3+viGp4phQ1xFQgQJEcz49ii6AEfod4tzmj vuJyEo0I=; b=oDCM9iwQW6iyHKoHjVffh7S+dzIV4f9C5lEx+rp9+IAe9kj6eAY p5CC2KIKQ5LYy5j1R7CDjkOyMcI9gNN6HwrBEmRyxkzV1TNYI7vzJ1skdrzjBUyH dGxUnX80au/f3CMmWiS7C2lHtNpmF4ISzF8WDf5O4GowKd0NhILv/iYESKivYbUL zOJCIyKagv93AlGT0aVXzx+isoHt/0GtvICntFbXc0vDifQa5cTcNSuxqRE/4XC3 Rh2c1vxXduxzTiRDQbGB7b0eiwQTUWn/0Y7kxfgYsjplX5OuTO0iLlbTgWu3hxeh 0Tv82/chjst3SIl7WZWZ+dksyeEXLTcJpUA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690059692; x=1690146092; bh=3+viGp4phQ1xFQgQJEcz49ii6AEfod4tzmj vuJyEo0I=; b=gsi3S9Q5wzyqNqjogSnZaehus2X4xkxgAI2Ik8inn17+76veW1K jnBG/PYJWvWNkcJIYfkqXeyvq0j8fzYg6199nAcmm7Onlyh6z4b4589cbdz7ejSk 18tGW8OfzVY3bF6CxGIKMn5Crpn3DnKWNrtJm5BazqKCh5FD2AN13DYQ+tx9cJkK EoTrDHqQTMqOHRGpDPBiJn4UT6cCnnGfaRAlS7R7Cm2Yc3daHOJQZMEzP4MVEKHs 8ZBvq/i/syR8bA4Pw2Nnjo8RtisYB3mJT9pTEQzJxwQOO3ZulyPRMemSSksNDZ+X nCONtUW7v1D/X7CMtrO53ULv2SrNyZhnr7A== X-ME-Sender: <xms:rEO8ZHN2JGI0xLvzpcx0j1LZNOHLmRjwbpAhV63MSZ2z4d1UTtWRiA> <xme:rEO8ZB_d67-7L33z_lDBT_HJNa1V2Gm8lkPG3dTW59eVTwfzL5PYSKZ189nSD4gMb rNcAOyyqW6NWmy3JwI> X-ME-Received: <xmr:rEO8ZGRYC1L7dxI0of4T8DDnljBkuHSuPK4xunv6bKKgB-IsDMByDqrFSehmX_c> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrheeggdduheegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:rEO8ZLvmUZnuMmA_mpq40ZGcx1ikRErY4S35kmebn2HkT6PxOCtAjQ> <xmx:rEO8ZPcD0_c-ooKr98x8qm-xwiq7icMZUs4PokK6F4fdVRFRIeprGQ> <xmx:rEO8ZH02A3WrgqJGsSMgrbhhBXgvEfL4HIRQSP8aYuTNcOcvHHV-VA> <xmx:rEO8ZAr5J7reLZ5Ir9hmM72B1CDSuRU_ciyIJqyRl6FgWapmx03AJg> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 22 Jul 2023 17:01:30 -0400 (EDT) Message-ID: <fde3f1c3-628f-591d-4c52-87104b87c10a@HIDDEN> Date: Sun, 23 Jul 2023 00:01:28 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Po Lu <luangruo@HIDDEN>, Spencer Baugh <sbaugh@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87r0p030w0.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87r0p030w0.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: Ihor Radchenko <yantar92@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 22/07/2023 09:39, Po Lu wrote: > Spencer Baugh <sbaugh@HIDDEN> writes: > >> Not to derail too much, but find as a subprocess has one substantial >> advantage over find in Lisp: It can run in parallel with Emacs, so that >> we actually use multiple CPU cores. >> >> Between that, and the remote support part, I personally much prefer find >> to be a subprocess rather than in Lisp. I don't think optimizing >> directory-files-recursively is a great solution. >> >> (Really it's entirely plausible that Emacs could be improved by >> *removing* directory-files-recursively, in favor of invoking find as a >> subprocess: faster, parallelized execution, and better remote support.) > > find is only present in the default installations of Unix-like systems, > so it doesn't work without additional configuration on MS-Windows or > MS-DOS. project.el and rgrep fail to work on USG Unix because they both > use `-path'. > > Programs that use find should fall back to directory-file-recursively > when any of the situations above are detected. Perhaps if someone implements support for IGNORE entries (wildcards) in that function, it would be easy enough to do that fallback.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 20:53:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 16:53:13 2023
Received: from localhost ([127.0.0.1]:37623 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNJbF-0001xB-BH
for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 16:53:13 -0400
Received: from mxout5.mail.janestreet.com ([64.215.233.18]:51239)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <sbaugh@HIDDEN>) id 1qNJbD-0001wy-27
for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 16:53:12 -0400
From: Spencer Baugh <sbaugh@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <83wmyr7sbq.fsf@HIDDEN> (Eli Zaretskii's message of "Sat, 22 Jul
2023 20:46:01 +0300")
References: <iermszrwqj4.fsf@HIDDEN> <87cz0lmoxy.fsf@localhost>
<83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN>
<87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN>
<83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN>
<83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN>
<83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN>
<83sf9g88eh.fsf@HIDDEN> <87cz0jj25g.fsf@HIDDEN>
<83wmyr7sbq.fsf@HIDDEN>
Date: Sat, 22 Jul 2023 16:53:05 -0400
Message-ID: <ierwmyru0r2.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 64735
Cc: yantar92@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: sbaugh@HIDDEN
>> Date: Sat, 22 Jul 2023 17:18:19 +0000 (UTC)
>> Cc: sbaugh@HIDDEN, yantar92@HIDDEN, rms@HIDDEN, dmitry@HIDDEN,
>> michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
>>
>> First my results:
>>
>> (my-bench 100 "~/public_html" "")
>> (("built-in" . "Elapsed time: 1.140173s (0.389344s in 5 GCs)")
>> ("with-find" . "Elapsed time: 0.643306s (0.305130s in 4 GCs)"))
>>
>> (my-bench 10 "~/.local/src/linux" "")
>> (("built-in" . "Elapsed time: 2.402341s (0.937857s in 11 GCs)")
>> ("with-find" . "Elapsed time: 1.544024s (0.827364s in 10 GCs)"))
>>
>> (my-bench 100 "/ssh:catern.com:~/public_html" "")
>> (("built-in" . "Elapsed time: 36.494233s (6.450840s in 79 GCs)")
>> ("with-find" . "Elapsed time: 4.619035s (1.133656s in 14 GCs)"))
>>
>> 2x speedup on local files, and almost a 10x speedup for remote files.
>
> Thanks, that's impressive. But you omitted some of the features of
> directory-files-recursively, see below.
>
>> And my implementation *isn't even using the fact that find can run in
>> parallel with Emacs*. If I did start using that, I expect even more
>> speed gains from parallelism, which aren't achievable in Emacs itself.
>
> I'm not sure I understand what you mean by "in parallel" and why it
> would be faster.
I mean having Emacs read output from the process and turn them into
strings while find is still running and walking the directory tree. So
the two parts are running in parallel. This, specifically:
(defun find-directory-files-recursively (dir regexp &optional include-directories _predicate follow-symlinks)
(cl-assert (null _predicate) t "find-directory-files-recursively can't accept arbitrary predicates")
(cl-assert (not (file-remote-p dir)))
(let* (buffered
result
(proc
(make-process
:name "find" :buffer nil
:connection-type 'pipe
:noquery t
:sentinel (lambda (_proc _state))
:filter (lambda (proc data)
(let ((start 0))
(when-let (end (string-search "\0" data start))
(push (concat buffered (substring data start end)) result)
(setq buffered "")
(setq start (1+ end))
(while-let ((end (string-search "\0" data start)))
(push (substring data start end) result)
(setq start (1+ end))))
(setq buffered (concat buffered (substring data start)))))
:command (append
(list "find" (file-local-name dir))
(if follow-symlinks
'("-L")
'("!" "(" "-type" "l" "-xtype" "d" ")"))
(unless (string-empty-p regexp)
"-regex" (concat ".*" regexp ".*"))
(unless include-directories
'("!" "-type" "d"))
'("-print0")
))))
(while (accept-process-output proc))
result))
Can you try this further change on your Windows (and GNU/Linux) box? I
just tested on a different box and my original change gets:
(("built-in" . "Elapsed time: 4.506643s (2.276269s in 21 GCs)")
("with-find" . "Elapsed time: 4.114531s (2.848497s in 27 GCs)"))
while this parallel implementation gets
(("built-in" . "Elapsed time: 4.479185s (2.236561s in 21 GCs)")
("with-find" . "Elapsed time: 2.858452s (1.934647s in 19 GCs)"))
so it might have a favorable impact on Windows and your other GNU/Linux
box.
>> So can we add something like this (with the appropriate fallbacks to
>> directory-files-recursively), since it has such a big speedup even
>> without parallelism?
>
> We can have an alternative implementation, yes. But it should support
> predicate, and it should sort the files in each directory like
> directory-files-recursively does, so that it's a drop-in replacement.
> Also, I believe that Find does return "." in each directory, and your
> implementation doesn't filter them, whereas
> directory-files-recursively does AFAIR.
>
> And I see no need for any fallback: that's for the application to do
> if it wants.
>
>> (cl-assert (null _predicate) t "find-directory-files-recursively can't accept arbitrary predicates")
>
> It should.
This is where I think a fallback would be useful - it's basically
impossible to support arbitrary predicates efficiently here, since it
requires us to put Lisp in control of whether find descends into a
directory. So I'm thinking I would just fall back to running the old
directory-files-recursively whenever there's a predicate. Or just not
supporting this at all...
>> (if follow-symlinks
>> '("-L")
>> '("!" "(" "-type" "l" "-xtype" "d" ")"))
>> (unless (string-empty-p regexp)
>> "-regex" (concat ".*" regexp ".*"))
>> (unless include-directories
>> '("!" "-type" "d"))
>> '("-print0")
>
> Some of these switches are specific to GNU Find. Are we going to
> support only GNU Find?
POSIX find doesn't support -regex, so I think we have to. We could
stick to just POSIX find if we only allowed globs in
find-directory-files-recursively, instead of full regexes.
>> ))
>> (remote (file-remote-p dir))
>> (proc
>> (if remote
>> (let ((proc (apply #'start-file-process
>> "find" (current-buffer) command)))
>> (set-process-sentinel proc (lambda (_proc _state)))
>> (set-process-query-on-exit-flag proc nil)
>> proc)
>> (make-process :name "find" :buffer (current-buffer)
>> :connection-type 'pipe
>> :noquery t
>> :sentinel (lambda (_proc _state))
>> :command command))))
>> (while (accept-process-output proc))
>
> Why do you call accept-process-output here? it could interfere with
> reading output from async subprocesses running at the same time. To
> come think of this, why use async subprocesses here and not
> call-process?
See my new iteration which does use the async-ness.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 19:06:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 15:06:36 2023
Received: from localhost ([127.0.0.1]:37583 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNHw3-0007Rb-V4
for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 15:06:36 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:41700)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qNHw1-0007RP-De
for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 15:06:34 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNHvu-0008CH-6Q; Sat, 22 Jul 2023 15:06:26 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=NNm6DMgiXIKdfH4b8Iqyn4idxdxjF3oxB9t+OnorII0=; b=dqNOz+Gp8JBa
wWF7J7egrnSYdJxeqtT/0viwYdOzC/Y5CxuBm3aIOdZwtdCGIhp0TGzfOJk/fBnhzwu4+hVQ/TK3k
ZgOfYJX1Z4e5Waz0Rcc5/ysRoCckNONmlU97y8fiG+hPN86KUutpYROWqt2TdWrO/3HhhnszDc8rx
be7ROoz0PiuE+5MIZliftV9rfm6jyDZeNTlCipNpczSI5QxWc/XhiYP5z03q+I7zOt1wznlZGfCLq
bUqJwivYM8Aogz36pAEX7v6jm1ucgfRDh9LwuJwib7xO4AS0vR4jDQ4ImmortMoSbAV02u5v51H6y
p4Wnscu9rT8Zs6sIYJNA7A==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNHvm-0008KD-Ci; Sat, 22 Jul 2023 15:06:18 -0400
Date: Sat, 22 Jul 2023 22:06:55 +0300
Message-Id: <83tttv7okw.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: sbaugh@HIDDEN, sbaugh@HIDDEN
In-Reply-To: <83v8eb7q8d.fsf@HIDDEN> (message from Eli Zaretskii on Sat, 22
Jul 2023 21:31:14 +0300)
Subject: Re: bug#64735: 29.0.92;
find invocations are ~15x slower because of ignores
References: <iermszrwqj4.fsf@HIDDEN> <87edl1scbw.fsf@HIDDEN>
<87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost>
<83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN>
<87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN>
<83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN>
<83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN>
<83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN>
<83sf9g88eh.fsf@HIDDEN> <87cz0jj25g.fsf@HIDDEN> <83wmyr7sbq.fsf@HIDDEN>
<83v8eb7q8d.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, yantar92@HIDDEN, michael.albinus@HIDDEN, rms@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Cc: dmitry@HIDDEN, yantar92@HIDDEN, michael.albinus@HIDDEN, rms@HIDDEN,
> 64735 <at> debbugs.gnu.org
> Date: Sat, 22 Jul 2023 21:31:14 +0300
> From: Eli Zaretskii <eliz@HIDDEN>
>
> My results on MS-Windows are less encouraging:
>
> (my-bench 2 "d:/usr/archive" "")
> (("built-in" . "Elapsed time: 1.250000s (0.093750s in 5 GCs)")
> ("with-find" . "Elapsed time: 8.578125s (0.109375s in 7 GCs)"))
And here's from a GNU/Linux machine, which is probably not very fast:
(my-bench 10 "/usr/lib" "")
(("built-in" . "Elapsed time: 4.410613s (2.077311s in 56 GCs)")
("with-find" . "Elapsed time: 3.326954s (1.997251s in 54 GCs)"))
Faster, but not by a lot.
On this system /usr/lib has 18000 files in 1860 subdirectories.
Btw, the Find command with pipe to some other program, like wc,
finishes much faster, like 2 to 4 times faster than when it is run
from find-directory-files-recursively. That's probably the slowdown
due to communications with async subprocesses in action.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 18:30:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 14:30:54 2023
Received: from localhost ([127.0.0.1]:37571 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNHNV-0006PI-T3
for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 14:30:54 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:59894)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qNHNT-0006Os-3R
for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 14:30:52 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNHNN-00056Z-4q; Sat, 22 Jul 2023 14:30:45 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=wwIJ3hXVgxXXSiWcZNJmiLbgxASrQvLWP9j/bmN2Oy8=; b=f3oSE/PxgaxX
Ssk+ayhioFbald7Zxify3E4izcP52CIcMQKHo56GcV+i051Z1YIzZY2OrKo8MhMVf2l6G3w79U5CD
Ao4vPWM1qaPAIcQ8BE5i9rZklORdSf+r2hc9avyskwX2KrzQyY+h5wEMbl8ul/7j4iiVCdKKY4rFO
xX3JZOq0DRaZALHeWqaKZL1SeO/RKIg1BnEg8K4CbsRoDW2d9kVzE+yPSAdW6tNtMw3NriMyXYKPX
sGLo9LkJqQdkfbtxgP8Tnuzm/9tsYyvmlhf735f6tGAZdY0utmNYSuYtHYBKGsvKrRZMHXuqlxCza
t7CkTsdcr2TJJ6bm6xEHjw==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNHNE-0008Lv-On; Sat, 22 Jul 2023 14:30:37 -0400
Date: Sat, 22 Jul 2023 21:31:14 +0300
Message-Id: <83v8eb7q8d.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: sbaugh@HIDDEN, sbaugh@HIDDEN
In-Reply-To: <83wmyr7sbq.fsf@HIDDEN> (message from Eli Zaretskii on Sat, 22
Jul 2023 20:46:01 +0300)
Subject: Re: bug#64735: 29.0.92;
find invocations are ~15x slower because of ignores
References: <iermszrwqj4.fsf@HIDDEN> <87edl1scbw.fsf@HIDDEN>
<87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost>
<83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN>
<87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN>
<83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN>
<83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN>
<83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN>
<83sf9g88eh.fsf@HIDDEN> <87cz0jj25g.fsf@HIDDEN> <83wmyr7sbq.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, yantar92@HIDDEN, michael.albinus@HIDDEN, rms@HIDDEN,
64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> Cc: sbaugh@HIDDEN, yantar92@HIDDEN, rms@HIDDEN, dmitry@HIDDEN,
> michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
> Date: Sat, 22 Jul 2023 20:46:01 +0300
> From: Eli Zaretskii <eliz@HIDDEN>
>
> > First my results:
> >
> > (my-bench 100 "~/public_html" "")
> > (("built-in" . "Elapsed time: 1.140173s (0.389344s in 5 GCs)")
> > ("with-find" . "Elapsed time: 0.643306s (0.305130s in 4 GCs)"))
> >
> > (my-bench 10 "~/.local/src/linux" "")
> > (("built-in" . "Elapsed time: 2.402341s (0.937857s in 11 GCs)")
> > ("with-find" . "Elapsed time: 1.544024s (0.827364s in 10 GCs)"))
> >
> > (my-bench 100 "/ssh:catern.com:~/public_html" "")
> > (("built-in" . "Elapsed time: 36.494233s (6.450840s in 79 GCs)")
> > ("with-find" . "Elapsed time: 4.619035s (1.133656s in 14 GCs)"))
> >
> > 2x speedup on local files, and almost a 10x speedup for remote files.
>
> Thanks, that's impressive. But you omitted some of the features of
> directory-files-recursively, see below.
My results on MS-Windows are less encouraging:
(my-bench 2 "d:/usr/archive" "")
(("built-in" . "Elapsed time: 1.250000s (0.093750s in 5 GCs)")
("with-find" . "Elapsed time: 8.578125s (0.109375s in 7 GCs)"))
D:/usr/archive is a directory with 372 subdirectories and more than
12000 files in all of them. The disk is SSD, in case it matters, and
I measured this with a warm disk cache.
So I guess whether or not to use this depends on the underlying
system.
Btw, you should not assume that "-type l" will universally work: at
least on MS-Windows some ports of GNU Find will barf when they see it.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 17:45:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 13:45:41 2023
Received: from localhost ([127.0.0.1]:37544 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNGfk-0005EZ-VU
for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 13:45:41 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:40836)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1qNGfi-0005EM-4B
for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 13:45:39 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNGfb-0004iT-Pt; Sat, 22 Jul 2023 13:45:31 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=bKTVOF9y9vUbCMp9Lv9+yw8MxYmMaeqfltn0EszC6lQ=; b=EtqU/6GBD/w1
PYE0VK+VuJg+i+qBVz6675PKmkmh9CxZIAVr2uf0YspSnJM3u8F3ul2+1b4KpDF5onx0av+E7L1ja
Kd+SaVp16qA50D+W1EN4hu0GcFrzhnW3ydplpxQw23cikhUFVlCDyMj5zb1kgAfDf8tX37wCzFIAJ
5VEmeXAOEN/I+65A/73NTKQ0gTqmGwqJ2XQTgQ0qdJSbkdPjsQtVhSNIPI+jwyXq92yDC7RLHyJnZ
fZkPVx20E17vEUbkO/0YnHFAJVuZKdZaSDLxgvdUSpFZAs7wbGlvA1puAU/2kWsyN4AafaJawkVyk
ySNc1qdcIRX8rEy8wBo2xw==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1qNGfU-0005FA-KZ; Sat, 22 Jul 2023 13:45:24 -0400
Date: Sat, 22 Jul 2023 20:46:01 +0300
Message-Id: <83wmyr7sbq.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: sbaugh@HIDDEN
In-Reply-To: <87cz0jj25g.fsf@HIDDEN> (sbaugh@HIDDEN)
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
References: <iermszrwqj4.fsf@HIDDEN> <87edl1scbw.fsf@HIDDEN>
<87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost>
<83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN>
<87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN>
<83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN>
<83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN>
<83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN>
<83sf9g88eh.fsf@HIDDEN> <87cz0jj25g.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: sbaugh@HIDDEN, yantar92@HIDDEN, rms@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: sbaugh@HIDDEN
> Date: Sat, 22 Jul 2023 17:18:19 +0000 (UTC)
> Cc: sbaugh@HIDDEN, yantar92@HIDDEN, rms@HIDDEN, dmitry@HIDDEN,
> michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
>
> First my results:
>
> (my-bench 100 "~/public_html" "")
> (("built-in" . "Elapsed time: 1.140173s (0.389344s in 5 GCs)")
> ("with-find" . "Elapsed time: 0.643306s (0.305130s in 4 GCs)"))
>
> (my-bench 10 "~/.local/src/linux" "")
> (("built-in" . "Elapsed time: 2.402341s (0.937857s in 11 GCs)")
> ("with-find" . "Elapsed time: 1.544024s (0.827364s in 10 GCs)"))
>
> (my-bench 100 "/ssh:catern.com:~/public_html" "")
> (("built-in" . "Elapsed time: 36.494233s (6.450840s in 79 GCs)")
> ("with-find" . "Elapsed time: 4.619035s (1.133656s in 14 GCs)"))
>
> 2x speedup on local files, and almost a 10x speedup for remote files.
Thanks, that's impressive. But you omitted some of the features of
directory-files-recursively, see below.
> And my implementation *isn't even using the fact that find can run in
> parallel with Emacs*. If I did start using that, I expect even more
> speed gains from parallelism, which aren't achievable in Emacs itself.
I'm not sure I understand what you mean by "in parallel" and why it
would be faster.
> So can we add something like this (with the appropriate fallbacks to
> directory-files-recursively), since it has such a big speedup even
> without parallelism?
We can have an alternative implementation, yes. But it should support
predicate, and it should sort the files in each directory like
directory-files-recursively does, so that it's a drop-in replacement.
Also, I believe that Find does return "." in each directory, and your
implementation doesn't filter them, whereas
directory-files-recursively does AFAIR.
And I see no need for any fallback: that's for the application to do
if it wants.
> (cl-assert (null _predicate) t "find-directory-files-recursively can't accept arbitrary predicates")
It should.
> (if follow-symlinks
> '("-L")
> '("!" "(" "-type" "l" "-xtype" "d" ")"))
> (unless (string-empty-p regexp)
> "-regex" (concat ".*" regexp ".*"))
> (unless include-directories
> '("!" "-type" "d"))
> '("-print0")
Some of these switches are specific to GNU Find. Are we going to
support only GNU Find?
> ))
> (remote (file-remote-p dir))
> (proc
> (if remote
> (let ((proc (apply #'start-file-process
> "find" (current-buffer) command)))
> (set-process-sentinel proc (lambda (_proc _state)))
> (set-process-query-on-exit-flag proc nil)
> proc)
> (make-process :name "find" :buffer (current-buffer)
> :connection-type 'pipe
> :noquery t
> :sentinel (lambda (_proc _state))
> :command command))))
> (while (accept-process-output proc))
Why do you call accept-process-output here? it could interfere with
reading output from async subprocesses running at the same time. To
come think of this, why use async subprocesses here and not
call-process?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 17:26:58 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 13:26:58 2023
Received: from localhost ([127.0.0.1]:37530 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNGNd-0004mQ-PR
for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 13:26:58 -0400
Received: from mout02.posteo.de ([185.67.36.66]:49073)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qNGNb-0004mC-CF
for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 13:26:55 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout02.posteo.de (Postfix) with ESMTPS id 6846E240104
for <64735 <at> debbugs.gnu.org>; Sat, 22 Jul 2023 19:26:48 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1690046808; bh=/3KqGjvuZ/WucD2J9KgOIvjIMwrXz8FbfcoMS0yENDI=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=SU7ZoSmTUGk76ZjwuVmtBNEf5ptBHv1R1hPfmClfH+aenjDRt+37YeY/aKlDruKMb
51IClcgJXPSITmibMX6qXs2vcihezawvl5Agqqp7dVQK24IiHQROtOBjM/kO+pHtLt
JrVOwvIVE5RY5I7Tlu25aUFXQC68/9sqlNdhr0R2hXSxrkAACccGsTYw46HA+p5/Gi
vZTpcZXLhg1sZN9bqCaLmaCSTBMZ1cso0FEMIcSFM9OL/c66LhTkAYIBotErDEJygJ
YDniLbgAvUqY5dyGgedXPsAbtWT00toyaAaX7LZxWS4VwTUqPwoGgXe/hhTC4YEBzq
H9cNO7fkQ26ow==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R7YJ71gjnz6txs;
Sat, 22 Jul 2023 19:26:46 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: sbaugh@HIDDEN
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <87cz0jj25g.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost>
<87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN>
<87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN>
<87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN>
<ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN>
<87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN>
<87cz0jj25g.fsf@HIDDEN>
Date: Sat, 22 Jul 2023 17:26:54 +0000
Message-ID: <87tttvsvq9.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: sbaugh@HIDDEN, rms@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
sbaugh@HIDDEN writes:
> I went ahead and implemented it, and I get a 2x speedup even *without*
> running find in parallel with Emacs.
>
> First my results:
>
> (my-bench 100 "~/public_html" "")
> (("built-in" . "Elapsed time: 1.140173s (0.389344s in 5 GCs)")
> ("with-find" . "Elapsed time: 0.643306s (0.305130s in 4 GCs)"))
>
> (my-bench 10 "~/.local/src/linux" "")
> (("built-in" . "Elapsed time: 2.402341s (0.937857s in 11 GCs)")
> ("with-find" . "Elapsed time: 1.544024s (0.827364s in 10 GCs)"))
What about without `file-name-handler-alist'?
> (my-bench 100 "/ssh:catern.com:~/public_html" "")
> (("built-in" . "Elapsed time: 36.494233s (6.450840s in 79 GCs)")
> ("with-find" . "Elapsed time: 4.619035s (1.133656s in 14 GCs)"))
This is indeed expected.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 17:18:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 13:18:29 2023
Received: from localhost ([127.0.0.1]:37526 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qNGFQ-0004a6-FT
for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 13:18:29 -0400
Received: from s.wrqvtzvf.outbound-mail.sendgrid.net ([149.72.126.143]:13978)
by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from
<bounces+21787432-b76c-64735=debbugs.gnu.org@HIDDEN>)
id 1qNGFO-0004Zt-1I
for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 13:18:27 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=catern.com;
h=from:subject:in-reply-to:references:mime-version:to:cc:content-type:
content-transfer-encoding:cc:content-type:from:subject:to;
s=s1; bh=eD958oiQjQIrk0Ld6XP5QZWP3il36+j1PcE1wIQrrRM=;
b=mD0zv01PsNCOs3TTg4SQLP9DQBEgEkYhptmAkpZBN/wSEP8nYe0pY2ZKa2KtNxyfoHFZ
fn7j3RtrVpNfsNytcb890LE6O+Y7ycfFxURednm90k3u4YvqX5jLM3Vy2e4TqGC+j/6rm5
2T0tHbV8PwRnEInateAHQn/Qnxu6/vJ57EL0RicHLxrp9MwDleVlKDhtvCxg1w6Jyu/I3U
QzGOjs92Eu7BkyalVfpY8dgtMlO+yJNUsdIwJGal0CWawXr1qiH3PMt87aZd/wSsAC3s4X
DIkCXSvFyd2MPmxhk1Y3YE7zt4rumiGq6w4JR+YvqXkjSBwP5T8DaIHhN/kxkkeg==
Received: by filterdrecv-84b96456cb-b5mzh with SMTP id
filterdrecv-84b96456cb-b5mzh-1-64BC0F5B-A
2023-07-22 17:18:19.641567595 +0000 UTC m=+6284401.883235998
Received: from earth.catern.com (unknown) by geopod-ismtpd-8 (SG) with ESMTP
id b_UL3K0eRH2wUziXbaGQpw Sat, 22 Jul 2023 17:18:19.449 +0000 (UTC)
X-Comment: SPF check N/A for local connections - client-ip=::1; helo=localhost;
envelope-from=sbaugh@HIDDEN; receiver=gnu.org
Received: from localhost (localhost [IPv6:::1])
by earth.catern.com (Postfix) with ESMTPSA id 1AA0A60077;
Sat, 22 Jul 2023 13:18:19 -0400 (EDT)
From: sbaugh@HIDDEN
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <83sf9g88eh.fsf@HIDDEN> (Eli Zaretskii's message of "Sat, 22 Jul
2023 14:58:46 +0300")
References: <iermszrwqj4.fsf@HIDDEN> <87edl1scbw.fsf@HIDDEN>
<87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost>
<83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN>
<87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN>
<878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN>
<83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost>
<83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN>
<83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN>
<83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN>
<83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN>
<83sf9g88eh.fsf@HIDDEN>
Date: Sat, 22 Jul 2023 17:18:19 +0000 (UTC)
Message-ID: <87cz0jj25g.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
X-SG-EID: =?us-ascii?Q?ZgbRq7gjGrt0q=2FPjvxk7wM0yQFRdOkTJAtEbkjCkHbJf+I76yiUOSLF77crDiv?=
=?us-ascii?Q?v8+uvwSZTFgqtahh2tqM+F7M2mOi6ydlNgAw8AF?=
=?us-ascii?Q?wCkWrnWqEAmfE88qKoTvOwipr7IJTIoYAGJEQeq?=
=?us-ascii?Q?2qPZMhJhYkAh84G8xLIJ4nPFW7SRJ+7EHZ9DEBN?=
=?us-ascii?Q?KsEgNyyJpTWsQpm2Kbqh1cmrcAk3QRKS6TQ=3D=3D?=
To: Eli Zaretskii <eliz@HIDDEN>
X-Entity-ID: d/0VcHixlS0t7iB1YKCv4Q==
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 64735
Cc: sbaugh@HIDDEN, yantar92@HIDDEN, rms@HIDDEN, dmitry@HIDDEN,
michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: sbaugh@HIDDEN
>> Date: Sat, 22 Jul 2023 10:38:37 +0000 (UTC)
>> Cc: Spencer Baugh <sbaugh@HIDDEN>, dmitry@HIDDEN,
>> yantar92@HIDDEN, michael.albinus@HIDDEN, rms@HIDDEN,
>> 64735 <at> debbugs.gnu.org
>>
>> Eli Zaretskii <eliz@HIDDEN> writes:
>> > No, the first step is to use in Emacs what Find does today, because it
>> > will already be a significant speedup.
>>
>> Why bother? directory-files-recursively is a rarely used API, as you
>> have mentioned before in this thread.
>
> Because we could then use it much more (assuming the result will be
> performant enough -- this remains to be seen).
>
>> And there is a way to speed it up which will have a performance boost
>> which is unbeatable any other way: Use find instead of
>> directory-files-recursively, and operate on files as they find prints
>> them.
>
> Not every command can operate on the output sequentially: some need to
> see all of the output, others will need to be redesigned and
> reimplemented to support such sequential mode.
>
> Moreover, piping from Find incurs overhead: data is broken into blocks
> by the pipe or PTY, reading the data can be slowed down if Emacs is
> busy processing something, etc.
I went ahead and implemented it, and I get a 2x speedup even *without*
running find in parallel with Emacs.
First my results:
(my-bench 100 "~/public_html" "")
(("built-in" . "Elapsed time: 1.140173s (0.389344s in 5 GCs)")
("with-find" . "Elapsed time: 0.643306s (0.305130s in 4 GCs)"))
(my-bench 10 "~/.local/src/linux" "")
(("built-in" . "Elapsed time: 2.402341s (0.937857s in 11 GCs)")
("with-find" . "Elapsed time: 1.544024s (0.827364s in 10 GCs)"))
(my-bench 100 "/ssh:catern.com:~/public_html" "")
(("built-in" . "Elapsed time: 36.494233s (6.450840s in 79 GCs)")
("with-find" . "Elapsed time: 4.619035s (1.133656s in 14 GCs)"))
2x speedup on local files, and almost a 10x speedup for remote files.
And my implementation *isn't even using the fact that find can run in
parallel with Emacs*. If I did start using that, I expect even more
speed gains from parallelism, which aren't achievable in Emacs itself.
So can we add something like this (with the appropriate fallbacks to
directory-files-recursively), since it has such a big speedup even
without parallelism?
My implementation and benchmarking:
(defun find-directory-files-recursively (dir regexp &optional include-directories _predicate follow-symlinks)
(cl-assert (null _predicate) t "find-directory-files-recursively can't accept arbitrary predicates")
(with-temp-buffer
(setq case-fold-search nil)
(cd dir)
(let* ((command
(append
(list "find" (file-local-name dir))
(if follow-symlinks
'("-L")
'("!" "(" "-type" "l" "-xtype" "d" ")"))
(unless (string-empty-p regexp)
"-regex" (concat ".*" regexp ".*"))
(unless include-directories
'("!" "-type" "d"))
'("-print0")
))
(remote (file-remote-p dir))
(proc
(if remote
(let ((proc (apply #'start-file-process
"find" (current-buffer) command)))
(set-process-sentinel proc (lambda (_proc _state)))
(set-process-query-on-exit-flag proc nil)
proc)
(make-process :name "find" :buffer (current-buffer)
:connection-type 'pipe
:noquery t
:sentinel (lambda (_proc _state))
:command command))))
(while (accept-process-output proc))
(let ((start (goto-char (point-min))) ret)
(while (search-forward "\0" nil t)
(push (concat remote (buffer-substring-no-properties start (1- (point)))) ret)
(setq start (point)))
ret))))
(defun my-bench (count path regexp)
(setq path (expand-file-name path))
(let ((old (directory-files-recursively path regexp))
(new (find-directory-files-recursively path regexp)))
(dolist (path old)
(should (member path new)))
(dolist (path new)
(should (member path old))))
(list
(cons "built-in" (benchmark count (list 'directory-files-recursively path regexp)))
(cons "with-find" (benchmark count (list 'find-directory-files-recursively path regexp)))))
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 15:29:12 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 11:29:12 2023 Received: from localhost ([127.0.0.1]:37480 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNEXf-0001pb-LU for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 11:29:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45576) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNEXd-0001pD-CO for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 11:29:10 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNEXW-000332-O7; Sat, 22 Jul 2023 11:29:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Rggkwh1/pJ8WUMA6W5kQZr/BtER1xtpzAdD4yE/9Ne8=; b=o6JdU9Cd906v X10wRem8hgXnQWEPK2uT89F7kLlH7ts+M11+Yc9Sb9fkOnGEWz74ZLW7nFIRAkxHtBzk6J+mmZPUa Ry8+/bG7mw9r7jRKb+lQl3oPx3W57Cgt0pWafaMf84sL9TcdYUf5LLcwXCWI/WKaO6M0SejRYvbdI 5Fd7ElUp8U6uxpv0ZEyytJHWKmIhq/dXd1ciOiMNjNCWoCtKqg8pW7w3vwclCT2HHGlf0aETP4hNF L/8LefC0kSCS5rnjdSq/JVfhxOsdAjKSdpfWhdWlZR4agy1ra67wFX/zWiLTLmkL8orWUJtgIYhu1 h84xJRNxIJ15sldsfQiRng==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNEXQ-0002FP-OC; Sat, 22 Jul 2023 11:28:57 -0400 Date: Sat, 22 Jul 2023 18:29:34 +0300 Message-Id: <83351g7yn5.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87mszornlq.fsf@localhost> (message from Ihor Radchenko on Sat, 22 Jul 2023 15:07:45 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> <87mszornlq.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sat, 22 Jul 2023 15:07:45 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> > So I think a primitive that traverses the tree and produces file names > >> > with or without attributes, and can call some callback if needed, > >> > still has its place. > >> > >> Do you mean asynchronous primitive? > > > > No, a synchronous one. > > Then how will the callback be different from > (mapc #'my-function (directory-files-recursively ...)) > ? It depends on the application. Applications that want to get all the data and only after that process it will not use the callback. But I can certainly imagine an application that inserts the file names, or some of their transforms, into a buffer, and from time to time triggers redisplay to show the partial results. Or an application could write the file names to some disk file or external consumer, or send them to a network process.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 15:07:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 11:07:47 2023 Received: from localhost ([127.0.0.1]:37466 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNECw-0001Id-PZ for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 11:07:47 -0400 Received: from mout01.posteo.de ([185.67.36.65]:37631) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNECu-0001IQ-G4 for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 11:07:45 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id A759D240028 for <64735 <at> debbugs.gnu.org>; Sat, 22 Jul 2023 17:07:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690038458; bh=PGIo2UYi1ciachkniOJ0fjVbYtnYL7qfxQpyPwTp0Qs=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=K4UcUHb6Uh1c7uw4hbhwa8UP0TXjVXyUh5nryEjjj5JCWjU24lTBLYZnwN0DXZpwZ Gto2YrrTq/sxTAs5IlMT6sNhYL+lPQKpA5q+hPvnHNxOeJu0IdnuNSDZw6akGkgZBt Y1MEulduz/BHOGKR8iu5OIGdke6bPuCwyHYdKRiJYyElyVSZbqfhbVhjKt0+BLWINo pqgoYrMSBYHq7ppVjpi/Culjs9/X48Ee9eZN73Jmbig2BsnQ7AlyyCXq0R/Hu2VOms yle3SnT/FnKD2gsSAItAdGiPJ84sRitve96dya5wK+HtHxqemnjyf1uh8msYxayao8 CTJRSAWNv2opw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R7VCY4Qknz6tsg; Sat, 22 Jul 2023 17:07:37 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <838rb881ak.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> <838rb881ak.fsf@HIDDEN> Date: Sat, 22 Jul 2023 15:07:45 +0000 Message-ID: <87mszornlq.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Ihor Radchenko <yantar92@HIDDEN> >> Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, >> michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org >> Date: Sat, 22 Jul 2023 14:14:25 +0000 >> >> Eli Zaretskii <eliz@HIDDEN> writes: >> >> > So I think a primitive that traverses the tree and produces file names >> > with or without attributes, and can call some callback if needed, >> > still has its place. >> >> Do you mean asynchronous primitive? > > No, a synchronous one. Then how will the callback be different from (mapc #'my-function (directory-files-recursively ...)) ? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 14:31:55 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 10:31:55 2023 Received: from localhost ([127.0.0.1]:37419 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNDeF-0000JN-BJ for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 10:31:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42254) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNDeC-0000J5-TB for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 10:31:54 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNDe7-0004XA-AJ; Sat, 22 Jul 2023 10:31:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=/yg88qKF3g4MXeeh40OVFNqkGdDc926dGxx6141b9qI=; b=D2rVKk8FwHX5 LkHZGwPJKI+LHeAaxxSjJ9tDdJSZfyjhQp64JbDcbrcd1Y1/b3u1RvVtpINGGBQxKz1p25c4fZVWZ lv2a1LUebIJUMSdANOqLTMJgiXkp39CHxDzxob92y8TziRjAQwVuWh1Q5+U30exX4wbrL8p+J3jrz w/MOkVFRKfWWe2XORz96m+Cjs03Wayji3Q5Gyz9bdmXJLV349/nRx3UbH6uM/a3esQNWwxdZvhGFe D3i4cS0SoOd5SzQ1NJtWeWovaEURdUKjdQrX2hBSzN4MMgYkICljsyZ2Qe9tf1F4r5gR7YoTJSu9l gfg9qJCox3jOnHmUXUATAA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNDe1-00016D-4T; Sat, 22 Jul 2023 10:31:41 -0400 Date: Sat, 22 Jul 2023 17:32:19 +0300 Message-Id: <838rb881ak.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87pm4krq2m.fsf@localhost> (message from Ihor Radchenko on Sat, 22 Jul 2023 14:14:25 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> <87pm4krq2m.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: sbaugh@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, > michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Sat, 22 Jul 2023 14:14:25 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > So I think a primitive that traverses the tree and produces file names > > with or without attributes, and can call some callback if needed, > > still has its place. > > Do you mean asynchronous primitive? No, a synchronous one.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 14:14:23 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 10:14:23 2023 Received: from localhost ([127.0.0.1]:37395 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNDNH-0008Ha-Ex for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 10:14:23 -0400 Received: from mout01.posteo.de ([185.67.36.65]:47837) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qNDNF-0008HK-Fn for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 10:14:22 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id E5E81240029 for <64735 <at> debbugs.gnu.org>; Sat, 22 Jul 2023 16:14:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690035255; bh=WIZexkpDDEDHYR9XRHB19c3FQey8YwWQK60VhlimEdI=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=RIcoeJ8KVwsrIteNQqyHJVBsKhn1JmH4oAoDHVFmW0k3UWeiIjZYEhptpdtRLTHZX bt6+cO6KrNAzusGrNAXt0HRfOcgtjJW/HYcAhof0E2DDdACXv7zzhRDY+z0dH/NSCI YuxpzfYaSnchxd3GAc5MhB9Iy9WUS8S+ZsUfdUORnZIt3u80qejMtcXwvixwMd/tJ0 mdNFOPkyuE9koj++38xX00MsXMncDVHPwlLDjZ78t1nwrvv/MzTASLUovXIWgkAFrg OD/xFFxdHF24xBfK69O998xHTZqDGbhzbce2AX6+8mnnLVU1PF78fa2CxVKzUir5Q0 l2uZeOdd1wvlQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R7T1x3V86z9s3H; Sat, 22 Jul 2023 16:14:13 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83sf9g88eh.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> <83sf9g88eh.fsf@HIDDEN> Date: Sat, 22 Jul 2023 14:14:25 +0000 Message-ID: <87pm4krq2m.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, rms@HIDDEN, sbaugh@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: > So I think a primitive that traverses the tree and produces file names > with or without attributes, and can call some callback if needed, > still has its place. Do you mean asynchronous primitive? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 11:59:33 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 07:59:33 2023 Received: from localhost ([127.0.0.1]:35593 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNBGm-0004EC-NR for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 07:59:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47920) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNBGk-0004Dy-Uv for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 07:59:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNBGd-0007ka-Nu; Sat, 22 Jul 2023 07:59:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Zpat+D4go7rtCXaY2yPAbzi5PQ4ZzVDUz3OKW2hExaM=; b=XN3Pvf9rKELv B4MizkQzWxRttWvfcOrhO4Nwva1mrJCFbZbpiKps0NbpXdN2YxudB4C6G56pE2oCoRAGTpdAKwaeh dNb9jdn0xOUv10Vlm8z7369Ydwe6w3J1rDBcCDZEjl/6WO75wNZS0aXAzPoF43Ydws7rGK0oAAx52 sqA3JjHx1SAomHDHMq6OI6GcrJs8K+qfB0RgkUq79hWzT+fovdT+Xqw9Dhj1ornIJVPS0rqhvIUbV Co83/y/RuMJ5wRiTJjqUzt9nU/SETmoYclWc8D3bH1jsX4pFrLX+ASz991Z6kra8DThKOB+hUhAEJ f2dO27EVKRBLgMuxwfg8Iw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNBGb-0008RB-SW; Sat, 22 Jul 2023 07:59:22 -0400 Date: Sat, 22 Jul 2023 15:00:00 +0300 Message-Id: <83r0p088cf.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: sbaugh@HIDDEN In-Reply-To: <87fs5gi5xe.fsf@HIDDEN> (sbaugh@HIDDEN) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87edl08d1t.fsf@localhost> <87fs5gi5xe.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org > From: sbaugh@HIDDEN > Date: Sat, 22 Jul 2023 10:42:06 +0000 (UTC) > > Ihor Radchenko <yantar92@HIDDEN> writes: > > Spencer Baugh <sbaugh@HIDDEN> writes: > > > >> - we could use our own recursive directory-tree walking implementation > >> (directory-files-recursively), if we found a nice way to pipe its output > >> directly to grep etc without going through Lisp. (This could be nice > >> for project-files, at least) > > > > May you elaborate this idea? > > One of the reasons directory-files-recursively is slow is because it > allocates memory inside Emacs. If we piped its output directly to grep, > that overhead would be removed. How can you do anything in Emacs without allocating memory?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 11:58:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 07:58:24 2023 Received: from localhost ([127.0.0.1]:35588 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNBFg-0004CH-3t for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 07:58:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50762) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qNBFd-0004C1-E1 for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 07:58:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNBFX-0007bw-8W; Sat, 22 Jul 2023 07:58:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=84MqFbzxw4wvGtn4ZxLkS/dj2BZ3tK/v8PlCYFhzn5A=; b=lLUWEPZ9KhHh tlja7emkSn/e2lniSK3zAJXA5Zh+7w5vzqRmxLi6rMtIHx++HzNN7nYU74cnuYTwJDzZpLKFhZvGB rZ3k5ap1cYCYc6REjKrn8zh1TGTA6Pcy+fhyvAxF4KLPA81ZhmcgEPbypdyvf9ww1HCI0Y76/x2tl 0UwMhERUgiA2n92gfrpRbJff0RcPD9XJeitlRnO0K8eSlA09KNfTEvgDR36q7AnCKNohqOIlwHeKH S8pLEk08lLIOGoi0f1Eeel8K8hDykjvGPjMV2XA24l/Bnq+HkxG5sDM9u5I6lVOZs3QfsjOoPdxZU j1o0I/3aIfCgwaQYy025jQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qNBFR-0006sJ-5p; Sat, 22 Jul 2023 07:58:09 -0400 Date: Sat, 22 Jul 2023 14:58:46 +0300 Message-Id: <83sf9g88eh.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: sbaugh@HIDDEN In-Reply-To: <87ilaci637.fsf@HIDDEN> (sbaugh@HIDDEN) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> <87ilaci637.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, yantar92@HIDDEN, rms@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: sbaugh@HIDDEN > Date: Sat, 22 Jul 2023 10:38:37 +0000 (UTC) > Cc: Spencer Baugh <sbaugh@HIDDEN>, dmitry@HIDDEN, > yantar92@HIDDEN, michael.albinus@HIDDEN, rms@HIDDEN, > 64735 <at> debbugs.gnu.org > > Eli Zaretskii <eliz@HIDDEN> writes: > > No, the first step is to use in Emacs what Find does today, because it > > will already be a significant speedup. > > Why bother? directory-files-recursively is a rarely used API, as you > have mentioned before in this thread. Because we could then use it much more (assuming the result will be performant enough -- this remains to be seen). > And there is a way to speed it up which will have a performance boost > which is unbeatable any other way: Use find instead of > directory-files-recursively, and operate on files as they find prints > them. Not every command can operate on the output sequentially: some need to see all of the output, others will need to be redesigned and reimplemented to support such sequential mode. Moreover, piping from Find incurs overhead: data is broken into blocks by the pipe or PTY, reading the data can be slowed down if Emacs is busy processing something, etc. So I think a primitive that traverses the tree and produces file names with or without attributes, and can call some callback if needed, still has its place. > Since this runs the directory traversal in parallel with Emacs, it > has a speed advantage that is impossible to match in > directory-files-recursively. See above: you have an optimistic view of what actually happens in the relevant use cases. > We can fall back to directory-files-recursively when find is not > available. Find is already available today on many platforms, and we are evidently not happy enough with the results. That is the trigger for this discussion, isn't it? We are talking about ways to improve the performance, and I think having our own primitive that can do it is one such way, or at least it is not clear that it cannot be such a way. > > Optimizing the case of a long > > list of omissions should come later, as it is a minor optimization. > > This seems wrong. directory-files-recursively is rarely used, and rgrep > is a very popular command, and this problem with find makes rgrep around > ~10x slower by default. How in any world is that a minor optimization? > Most Emacs users will never realize that they can speed up rgrep > massively by setting grep-find-ignored-files to nil. Indeed, no-one > realized that until I just pointed it out. In my experience, they just > stop using rgrep in favor of other third-party packages like ripgrep, > because "grep is slow". Making grep-find-ignored-files smaller is independent of this particular issue. If we can make it shorter, we should.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 10:42:29 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 06:42:29 2023 Received: from localhost ([127.0.0.1]:35531 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNA49-000838-VS for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 06:42:29 -0400 Received: from s.wrqvwxzv.outbound-mail.sendgrid.net ([149.72.154.232]:27498) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <bounces+21787432-b76c-64735=debbugs.gnu.org@HIDDEN>) id 1qNA3v-00082i-BF for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 06:42:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=catern.com; h=from:subject:in-reply-to:references:mime-version:to:cc:content-type: content-transfer-encoding:cc:content-type:from:subject:to; s=s1; bh=RdHRp03vjHa+1ZfbHGooFdIIB5sGQ+tAY9sAqKbuNss=; b=teAeOAl+Bw5jkoZDMlz0ASTHWOLaY1bSkOJ36erP8BzuWP/jjauhssnapqTMo/9L2kPS 9fti2VD4exrIqLIZVoS2/9axFwCIfJ+pItr/obBuczU/UHlzHL6DTQL07QsWUvzrWdIrLE Z2wZha2O8vKGEi+LOOlKvIhfAwQshQSbfjy2Mdh81qcwJtLYB1iHcPN87iMMXTmt6kkl4X rHIZ09eAqW+5fdsBe8277uamtolaM+PxmrYVCt0hyHS5tGHWOTy38s5zFNfOYTVtTNM9Mr 7vjUBJ5EKjF0qI1r9OyIPr5UpaIe+aQtuIHE5p5HkjISkcbdu71doOzwV0YwRufw== Received: by filterdrecv-66949dbc98-9d8bm with SMTP id filterdrecv-66949dbc98-9d8bm-1-64BBB27E-1 2023-07-22 10:42:06.033193656 +0000 UTC m=+6260549.422128204 Received: from earth.catern.com (unknown) by geopod-ismtpd-3 (SG) with ESMTP id xquYvXL7Q9OuYzrhCwRkmA Sat, 22 Jul 2023 10:42:05.909 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=::1; helo=localhost; envelope-from=sbaugh@HIDDEN; receiver=posteo.net Received: from localhost (localhost [IPv6:::1]) by earth.catern.com (Postfix) with ESMTPSA id 6A48F60166; Sat, 22 Jul 2023 06:42:05 -0400 (EDT) From: sbaugh@HIDDEN Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87edl08d1t.fsf@localhost> (Ihor Radchenko's message of "Sat, 22 Jul 2023 10:18:22 +0000") References: <iermszrwqj4.fsf@HIDDEN> <87edl08d1t.fsf@localhost> Date: Sat, 22 Jul 2023 10:42:06 +0000 (UTC) Message-ID: <87fs5gi5xe.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 X-SG-EID: =?us-ascii?Q?ZgbRq7gjGrt0q=2FPjvxk7wM0yQFRdOkTJAtEbkjCkHbJSRTbRpqF3T0QMtzR7x2?= =?us-ascii?Q?X=2FZGgD15eka72bRqGmmDcanJaKvC7AHYUCAp5gn?= =?us-ascii?Q?zOkw2hqgQFhz3Q5rpmKy3fhBbuFZHXG+B8S3yuJ?= =?us-ascii?Q?kdllctoDdgIHhC8O59lrCy4i+l715yexau9eTYj?= =?us-ascii?Q?2AIlp1P2lZDrXXZfKUS0XTrBDa2mZo2aYfw=3D=3D?= To: Ihor Radchenko <yantar92@HIDDEN> X-Entity-ID: d/0VcHixlS0t7iB1YKCv4Q== Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.2 (/) Ihor Radchenko <yantar92@HIDDEN> writes: > Spencer Baugh <sbaugh@HIDDEN> writes: > >> - we could use our own recursive directory-tree walking implementation >> (directory-files-recursively), if we found a nice way to pipe its output >> directly to grep etc without going through Lisp. (This could be nice >> for project-files, at least) > > May you elaborate this idea? One of the reasons directory-files-recursively is slow is because it allocates memory inside Emacs. If we piped its output directly to grep, that overhead would be removed. On reflection, though, as I've posted elsewhere in this thread: This is a bad idea and is inherently slower than find, because directory-files-recursively does not run in parallel with Emacs (and never will).
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 10:38:46 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 06:38:46 2023 Received: from localhost ([127.0.0.1]:35526 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qNA0c-0007xt-5S for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 06:38:46 -0400 Received: from s.wrqvtbkv.outbound-mail.sendgrid.net ([149.72.123.24]:2592) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <bounces+21787432-b76c-64735=debbugs.gnu.org@HIDDEN>) id 1qNA0Z-0007xd-1a for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 06:38:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=catern.com; h=from:subject:in-reply-to:references:mime-version:to:cc:content-type: content-transfer-encoding:cc:content-type:from:subject:to; s=s1; bh=JmkGW7uOEyQXvznsKLr+w9z4YgZvxGiiW74fdpKaJ4k=; b=ckq71Yddpzk8yJDHWs+5/9quS+fMHiMfRpazBtAW8aW1GWpjmz00eYr3ZvbwZc/SALmZ qBchfmJVYQG/auYq6R070X2mTczO9ok/39tF2seU6jhFQdrCqtcOF5KJ8fI9QIRElq3jF/ dPU0tnz2oOE8qVJWjBdRB9zs76a+gB9+lguESmztetryf66YSESSsCnyJ4DMEzRFidMVAV 3k8o3aSLl6bvqLGuCpuB5bFeYGpP1N0ApP04WVtEQDONzCowYYJlY7in6kBNf4HEc546H7 Ng7/kEmfL1qPP/If0FdeDySmpLeGFEmmPiZlgqRR7zAh18nJVN37N+WyYVZIHWqw== Received: by filterdrecv-d7bbbc8bf-hf4lw with SMTP id filterdrecv-d7bbbc8bf-hf4lw-1-64BBB1AD-F 2023-07-22 10:38:37.481541034 +0000 UTC m=+6260331.354104113 Received: from earth.catern.com (unknown) by geopod-ismtpd-19 (SG) with ESMTP id M4_2X6gUQg2Lbq6FjHaRfA Sat, 22 Jul 2023 10:38:37.286 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=::1; helo=localhost; envelope-from=sbaugh@HIDDEN; receiver=gnu.org Received: from localhost (localhost [IPv6:::1]) by earth.catern.com (Postfix) with ESMTPSA id B145B60166; Sat, 22 Jul 2023 06:38:36 -0400 (EDT) From: sbaugh@HIDDEN Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83h6pwa52z.fsf@HIDDEN> (Eli Zaretskii's message of "Sat, 22 Jul 2023 08:27:32 +0300") References: <iermszrwqj4.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> <83h6pwa52z.fsf@HIDDEN> Date: Sat, 22 Jul 2023 10:38:37 +0000 (UTC) Message-ID: <87ilaci637.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 X-SG-EID: =?us-ascii?Q?ZgbRq7gjGrt0q=2FPjvxk7wM0yQFRdOkTJAtEbkjCkHbIEq3B8VB07aQyhxWj0nf?= =?us-ascii?Q?MRL7FquXRq3RnCJN0YKQQwVDjUc530FyHIavQw4?= =?us-ascii?Q?ZLWEgDhKAvYRiAi9KjAQ3MYqbVeqAIYZTeC7gUk?= =?us-ascii?Q?D1LrFWCCpgAD5zYmeo=2F3VdlOJP1UGGrM2TjQTXa?= =?us-ascii?Q?lnoB+Ey4svepamu3kVHUTOY9bTTjtj=2Fxq2w=3D=3D?= To: Eli Zaretskii <eliz@HIDDEN> X-Entity-ID: d/0VcHixlS0t7iB1YKCv4Q== Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, yantar92@HIDDEN, rms@HIDDEN, dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Spencer Baugh <sbaugh@HIDDEN> >> Cc: Michael Albinus <michael.albinus@HIDDEN>, dmitry@HIDDEN, >> yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, Richard Stallman >> <rms@HIDDEN> >> Date: Fri, 21 Jul 2023 15:33:13 -0400 >> >> Eli Zaretskii <eliz@HIDDEN> writes: >> > The first idea that comes to mind is to reimplement >> > directory-files-recursively in C, modeled on how Find does that. >> >> If someone was thinking of doing that, they would be better off >> responding to RMS's earlier request for C programmers to optimize this >> behavior in find. > > No, the first step is to use in Emacs what Find does today, because it > will already be a significant speedup. Why bother? directory-files-recursively is a rarely used API, as you have mentioned before in this thread. And there is a way to speed it up which will have a performance boost which is unbeatable any other way: Use find instead of directory-files-recursively, and operate on files as they find prints them. Since this runs the directory traversal in parallel with Emacs, it has a speed advantage that is impossible to match in directory-files-recursively. We can fall back to directory-files-recursively when find is not available. > Optimizing the case of a long > list of omissions should come later, as it is a minor optimization. This seems wrong. directory-files-recursively is rarely used, and rgrep is a very popular command, and this problem with find makes rgrep around ~10x slower by default. How in any world is that a minor optimization? Most Emacs users will never realize that they can speed up rgrep massively by setting grep-find-ignored-files to nil. Indeed, no-one realized that until I just pointed it out. In my experience, they just stop using rgrep in favor of other third-party packages like ripgrep, because "grep is slow".
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 10:18:20 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 06:18:20 2023 Received: from localhost ([127.0.0.1]:35498 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qN9gq-0007JA-83 for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 06:18:20 -0400 Received: from mout01.posteo.de ([185.67.36.65]:34431) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qN9gn-0007Iv-6n for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 06:18:19 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 18260240029 for <64735 <at> debbugs.gnu.org>; Sat, 22 Jul 2023 12:18:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1690021091; bh=eYLrO6lc80gVNDdorkRuqWnTN7qgE7++42Xj7c5twtE=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=Vea9mm9cL5+cbLIyEIEI+DLccbLPwF+S8LY2TfYMYM3IoJMoBLMZVUc4smNn8iam3 av9TrUtpgJnAXAW5q/JmK8CWu4++v+ED/58aD2TDDEz7xIFRVyjx7f2f+V4og0zn+z jV3fpe7R6c+IvZiZt5FVc6nUIVVp1KkdfO9Laz41pJhEZ5kNzY/xCGUYCSee33UMpL z3i3CYuZaBOc7W7ylS9AUA4SxkM1DmB+Ea9Y1Nx+q86sHm6iMC6PF9Jkc/uHuz/dUS JYGSqfKCBGxNaQDsd7pQu0CwYEind4bKdGq7BYqSpZ8LIMk0EgeCLZVHeuPuTxFBMy /iHO80x96Q98A== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R7MnZ2j77z6tw5; Sat, 22 Jul 2023 12:18:10 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <iermszrwqj4.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> Date: Sat, 22 Jul 2023 10:18:22 +0000 Message-ID: <87edl08d1t.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Spencer Baugh <sbaugh@HIDDEN> writes: > - we could use our own recursive directory-tree walking implementation > (directory-files-recursively), if we found a nice way to pipe its output > directly to grep etc without going through Lisp. (This could be nice > for project-files, at least) May you elaborate this idea? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 08:18:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 04:18:01 2023 Received: from localhost ([127.0.0.1]:35418 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qN7oP-000481-A0 for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 04:18:01 -0400 Received: from mout.gmx.net ([212.227.17.20]:50245) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qN7oL-00047e-2b for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 04:18:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1690013854; x=1690618654; i=michael.albinus@HIDDEN; bh=TUiBtIN0DG5tYdHIHc8e4CtiWpBMZNEXEOBucQ0tvV0=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=WLkIrG6MbS8OrZaCKR+yqp53PpGLUiGvHjsLYnp33j3BscTsjFEZBgrdvw6JwRBLx/zMmtr zRIw82h53HQb84O3sYYXAZ3OFPG1J+wJhr/UKWl/WykAvupA2uxRyBVO5cwfew/1kXsiKcvLt dK+7/IglGo75vDQbPOSSazUedPuN0uVoS6u/ezQhGPNexjL+wQm9xk1+9V928sX2uIU/hBhhn 9tF5YDv72HwKC4Ha1il81kk06tJW2wEZoDsYV0XW6KOTBq3Ctg3xNT5t70+cpslHVFpFd1pst vbdSrkE4WXfXBi5WOa4hvajhaGbd++MHHEK/ZUvIR35qh2fRUkrQ== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1M2O6Y-1qLURM04re-003rpl; Sat, 22 Jul 2023 10:17:34 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87jzutb14l.fsf@HIDDEN> (Michael Albinus's message of "Fri, 21 Jul 2023 19:55:22 +0200") References: <iermszrwqj4.fsf@HIDDEN> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> Date: Sat, 22 Jul 2023 10:17:33 +0200 Message-ID: <87cz0kbbs2.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:i/W+U1HfSO1RN++UM91BCPNEGx4BoW9e9O3ydLv2wEtyKzgy4G0 F+Et6dcVoDv8OOUCqeqRQUXpQ072aHV6bcFGjT0puc2VhdtJQw8EmbMVHhsOtplMd5ofaK7 th7JJfbrL36XG2kiMNaUIIlSzSB96cBZqjwRZLYItNVfvTsUt66GOen8la1X0DtMpA7N79U q0DgADs4Ed+nv9q5PQMtQ== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:4qie1BiGz6g=;+XAKsG6WLTK0nf3AovYNEvQI3T4 KK58tkmj+c7E1M7HCH/nynKv5zpfM0de+oDVzxis2hOFt6rim3EdTYh2saGF2x+49jRrK/oc4 NwyvCKelidzZiJo0OBqtzSqqZ09/+vgLp0S7cKGwuus/HLD9wW85Fu1ev0AqsK0RiwTBY7mR6 vQYtmMxi7yxDZP+QsT5bqwWDU6ia1rP7Oc0jinpeqsUVnOz89NCu/UByCmMIeZaBuBXJ+LJPc 0NAZBBxQKX2PhAa2AzaDSwi1G9TQUKRh250V3Lte7/Sfz+MhTUfbhKfskpz7iGbo7CX1cgbgN TFvQy/u/qzf5vs5iMs43YkVpXjXOp/go6Om1BPhLu8vSyCbMBCZHf1vu8/UEMTSlVuZUrGger kOyJ+rt+ICSw9EAbOffH4MrD87GE8nI0mn1HKM7EpBFgFwOlmFUkaVRoL9f7VSk0H2qaT5XvH c17DBIUhuzl+qhZdjNzrzwpgoeEsIBFGEOo2FeJjcWnZOob3sUVCjnk/xUntzuI3pHkPuuPEf Y7aIrRvyfuKO0mDNrXbci5RZvmkqV5lyhf65+ca1Fhs+5OYRu07o/pHuIFUuMcrUkFoSNSvb9 or2m0TEp0h6lb0p3AXqz9GSV1IaxQ+gKA+IMaQEhZfPcG9uGEJnVDyKQabiFzurirXaedHvQQ X86ayBN6Kwi8Wy6ECL9nfYjKGzVkd/jjsX1GUGbBS/XQ2X+SElztBrwAOIPrXYx7vI8By0d4W MCdUdphKYJyimcNKhGMPbtWEEPdggTddIfSqRs315zdCgSIKBa8B+MGb86czpCkX2Po8KO5Wc xX1vfANuzad4gguNI0UykYJHbEvd1FHsGKpuy3/RDagfqrJv/E+9YYtn6XVYD7jTO0oKs7vX4 ngxS/anyBbewBofgLbwEIMBoVstG3Vdq6RPfINbjx5Z6nCZzMTCGu+ehwhWiov92IhQ0ylhpy OzmQlQ== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Michael Albinus <michael.albinus@HIDDEN> writes: >> No, I just disagree that those measures should be seen as solutions of >> the performance problems mentioned here. I don't object to installing >> the changes, I only hope that work on resolving the performance issues >> will not stop because they are installed. > > Thanks. I'll install tomorrow. Pushed to master. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 06:40:17 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 02:40:17 2023 Received: from localhost ([127.0.0.1]:35320 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qN6Hp-0007Q4-Gh for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 02:40:17 -0400 Received: from sonic307-56.consmr.mail.ne1.yahoo.com ([66.163.190.31]:34785) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <luangruo@HIDDEN>) id 1qN6Hn-0007Pp-6X for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 02:40:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1690008009; bh=rC5s1XRkh2ESHgHxu6vgvj12iN9HsKCICKMUPJvqZZY=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From:Subject:Reply-To; b=A8wT1PVb7mWN/4MypeZhW/Qd6OAZWGfInzX5IYfMwfowRXr0/2Kc18R/Uq1me7wTpinK0+OK6zICuIP7hp1QMMVcn1yAEHf6FuanBU4InhdOJme8jdBad5b5YL83uviMjpUBVW8oz9ysqJL6KQ52eXTEW4rjvFFneoRI/GTwurCZEljTOX5qyuEPQJTnGTUiKQzyKuV9D/IKngF/wcHjLQzJVO6x5Hg6LNh8XMWml6r64vVQPmOZdDosZbnPd5m+cbjLAH2+0YDLkJX+vV7G/8NWWS+hK0h2eCXIpbDL/O30pWeXhCwFyf2Jo6sZZ0D4TDgMS3zx+NwyB944ujtNFg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1690008009; bh=SHOSC88P2nuZNVRnQSwAc4OzBKZLdsG68g7hGxrujdY=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=sNTJhI/ll7234FHei3fqovSEAy9DebNzl+cbXKs64f1bnnxGmm22Of1xUJ8gr/3031tc195vhPGrhIC5ncEPkvpB+diOr3rnQWXI5w6d5ZQnnYkALvSWx+FeXr6EhKQFkrqyVoNVwvnqfvlHbKkPTQmI4I+SwRmtmBH30x5iwIlDIoMgrxh0UlsT03t9paygxaVZNQz6u7ZbmrDZ7TCng2u7/IBdywEGIQdY9deNYFDbmsWKotz7ICOKcI2mpEJ7I3AUeSkcAj2fRLrG3PSTSIFU58pVw/miWKtelR/t5yCiY5KDw9a8BpDsTqbKT+QzUnKA8R4MWIKoQO3GDOGyWQ== X-YMail-OSG: Boms5wwVM1mMsj07SqS9mCaeU7oGCpRa893uSoqSPnOGzkd7BZNzXV8zmzejSmZ OjzIm.P_otasz8Qb.yT1GUEwg91_uK9CgJht4xKdT2Ji2azdgQZHksoi6B3RIKOmMtWdlC4k.Gxv Q46yICVA8pHjT0ETDlGuNQPTceV0K8nF61VUIrQA5UOk99Kh1o3fczHMUpkiq4rwz93fndoHwbti VSvqq1TbExiuLdcxDFO4YyGd4e4Py7ofrKkQOUUyjfSk2Y2uftvCfGHxeTP1QjccYsSUqiaIj2sJ ASaj7O_GyQGzL3cxQ2Hp7vbBvHxF2fe91ekWkRjIQh3JHLsSyIBkOmu.0ba.P9IWvHygKs7Mnupy gakTJwzJxbHWhIyzluRhGyXPKqWcWixKindbzaQe0Pssrbj9joEZyNtbOg._yc2aI.L1gAa7X3Mz hCZefm7IGgAmvzjBfW65.0nDiktlQxjD_K19A.3dYy6X2nbp_t5OTiC5NT5qyKLut6up6oUK.6oN fRiJt9tBWhfjQrKdUbLkAsn8lzoHMjb_RxY8GHt.0mOiuZQXrtz1QTjycUBZL2gGcuBarASR1Eja M0WD1AIfEJ1dSp9Of9iUAi9IpKMCJI2FKtvsLJhiULLw4CZD5rgt0lQuYxWZ4qfhN17e_VD6yL3n 18Tu8XI5TJRgFoRRGsyHro5Ybb5gPz7BU8pZDy2p2dwnKm._T..9igSrO7x5N9D6h3sMfgd.OrUR 9grAPTTaH1g0fBX9y2HKgarg63Orx5Pr56s0FJNymzicrRJD_mCfTFXSvfNYF.4WwOiakfGkX2XC 5MED4u5wXS8ANyycrg_cbM7hr1DAYirB0w1MK0.2SgKOVPScwGdmRhLeDvXg49RdTG_TBdSm9QOe Ri9LBaueDgGMzgGr9lyz.k4CT.I5wFYUOFbuSvsuI9zN9Y7AgS_wzsuJwkirVYfAMaMqmovhT3LQ EWY9jWgbjtb8BnJ3dWLUyj.KGInCKn2oCcRBr2E5DRzzcag1xnvNLbTgYtYhuBmT0dTEZ4Bj.uCg IZ3WcNHvGXInXYUf4TwFX9g3zqYpHFUPpFIGbV3ao4nmkFl33dLz_RwapXHpJHeSg4P5e9Togfqq VC1aGlN46DVFQqHVyQqT.415LxjRIKkiLkjuS.5TiHsFrsc_.IWs1BVbPgsIfCkV11mL.nnH.4Zk 8tIJde1O.aT6fZQBo3iW95jQdFhHAuKwedHfoLcQNo0NZ8tsXlBwDSxCArzj52iQsn9FdJNYd51L EyUAnXf6MLcJ45Hu_j.nbCHdBPT9DLw8EUgQATVBQ9ykL7nQdOqFM5Zui.0DenW0YDG8BoUzQfMJ km_RCOV.G_gRTgvok4bqaGLLkv0Tg449wZLpHwynru7LZmlkgywj21Rgzrhl6hI1J0qtUUf8j370 eFwRihmQqLxfFMqivGcAklalqZWRnUk9wGheXKiNWc9ctg0R2HSsQzsBM_u7wQfML5Yfxa8cfklr yV81F6M_cBgcM8FVqLE5.nYpTiH0D5k65NeHglpZGi0T8BeufIzC6iiZNTJ2J4ktmn_YyYmmmGxS _aN0xVqrMJ49LDvxWY_hh87dpAKAoPtZifcCICFv4MOuAEL2IqPE03QJq_wL9G1V39IXRb9.rXw_ 7eHfg.XhKkj4bhp4mc2KuZCn8dAE4Tl9CHT3CA2gFMBKd8oWEW4jGdk4rWAV0XjAAXB9qrYQ4whr 4fKwfib9Vm8HmP6UMdSudgJ6_tAST.tCMWHgwoCmPQZhq78jRevvV.82pkcwWA5D9PedOaVylSu4 z6mpzLDCzDDz4M9mbgWF2JiHdKOWpiix7RPInKKZs6JHgVwigXiYbbeaYjgXG71axTjmLACiWGMw Pyff9qibf6QWwzZDdit4g0tw5Om0BJPA0kNHI2Ecytb4n1fk6ibMk8aCCyp7I5QaT41GQtGDlQDV X1tVDCDqEdfUZVtyJKt9AQbxWYT9blWPn4LAmx8QcihRnb.3jfigmFmhxjJ.Z92KpSF7Pc14_t_x xOs5I6nakLJYEwxSZIK5jWHNTI7lqETOYrw384XHBvBCCQ.3tMv1rhupH7HMcSF8cV0LH37cy4jc MH5dE4qHNdMqUEYWy3lg4Yla6IYxZFAE0YhXB4AamkMlHXLwUXihTUuD70jIQmIu2xGdqCdaPKKg .7Jnm0kBL0cKw5i_UGsAIzXA8Figu0._i_SYqIEQcezcZWYAMT1z.XAkYs1uf28XJZtayRzBc6Xn iVyolh9hYBTlQzxd5L.Z9IN8DGP96tw7qbLqt0x34DS0sWDTuAKjRZUUahiggWyt.rPI- X-Sonic-MF: <luangruo@HIDDEN> X-Sonic-ID: 5b9575fe-9afd-4efb-8bb4-8f9ce28d83d2 Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.ne1.yahoo.com with HTTP; Sat, 22 Jul 2023 06:40:09 +0000 Received: by hermes--production-sg3-9dc5f54fc-9mrgp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a55ae7f91a86cab175db34b82a92cc2e; Sat, 22 Jul 2023 06:40:04 +0000 (UTC) From: Po Lu <luangruo@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <ierfs5iwlx3.fsf@HIDDEN> (Spencer Baugh's message of "Thu, 20 Jul 2023 13:08:24 -0400") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> Date: Sat, 22 Jul 2023 14:39:59 +0800 Message-ID: <87r0p030w0.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Mailer: WebService/1.1.21647 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Content-Length: 987 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, Ihor Radchenko <yantar92@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Spencer Baugh <sbaugh@HIDDEN> writes: > Not to derail too much, but find as a subprocess has one substantial > advantage over find in Lisp: It can run in parallel with Emacs, so that > we actually use multiple CPU cores. > > Between that, and the remote support part, I personally much prefer find > to be a subprocess rather than in Lisp. I don't think optimizing > directory-files-recursively is a great solution. > > (Really it's entirely plausible that Emacs could be improved by > *removing* directory-files-recursively, in favor of invoking find as a > subprocess: faster, parallelized execution, and better remote support.) find is only present in the default installations of Unix-like systems, so it doesn't work without additional configuration on MS-Windows or MS-DOS. project.el and rgrep fail to work on USG Unix because they both use `-path'. Programs that use find should fall back to directory-file-recursively when any of the situations above are detected.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 06:35:33 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 02:35:33 2023 Received: from localhost ([127.0.0.1]:35310 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qN6DF-0007JM-Fn for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 02:35:33 -0400 Received: from sonic317-34.consmr.mail.ne1.yahoo.com ([66.163.184.45]:37505) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <luangruo@HIDDEN>) id 1qN6DB-0007J8-W4 for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 02:35:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1690007724; bh=tSj4J9q3molIIO2jERfQxF75sUyV0FmJXnnWpIBn37U=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From:Subject:Reply-To; b=VbA7BpgP2J9O5uXNECboAvU0S9wfYzR4pLC/RTgvjUWrX7+NB+uFczMFgLaKugWK9o6CNX2IgqPkQvwzmY+0k8Gq+/TBoutlggvMKXXSZnakxObYSMeaLZ122PvkIK56Y8MH2xwgpxtO73cOtZFGi2QbVlo3VFJLUzHGcmcujWvcNnGVrL21H5G+IFh2oinV1smJIw8gQXLVs55fxPkdFJFLRT2HC9ogPjm9bJwFctEOk9DT60V9nlBGthFsYmim6bH549OdWdk4kHhuVP94wJoy3KU7z3i6aGYjE7e+vCDh5+BHcLSp8dOPOnDsVq4JLgVGm/RNHEZ2/eY2rghpWQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1690007724; bh=nnXrmyC4hduJU4nBtBkKJ0mqEcFxp/9cnD38p9Y81q4=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=YFqaY86v4XtjGNJv/k4/qc4aPvDZW1OBAm/llSO80+HQs6kpIhfVo9ruUcKg4fW3lcJvSrvzOZUg/HEXRv6NSiltEZh7kCrpmupSEsqoAhWnh422STM/0norp5KIUJmpGPsQewmrclW/kIMg7uEoUYxwePISvlYp9jxW2CVPMFNsUc0ipO7QWV47CwEutIKtLcQ8bNEkNkIOKIj1PSTmhGmOBFl1S/r1XMb7enmLCyhd2HsvPR/bk2eKZqxAULMrY9pbav85wIuT3nLw6GrbKw2JH5tmmT81xtwD6goE3yFSMz401DaaL0xteBMzSu0eBz9c1JUEGvFxluVt0ZLi8Q== X-YMail-OSG: ZCo3DDcVM1mTFKiG2pOjVUNLtm3fQpvxXwEhHtjRBkevNnLIx8Skl0XnF2zaifQ _F6dBDexC2Os4SaFsBtxbgv00bw9ks9TIE4i3E7yrKbsdTRCqWu0uHhzhwkH45Vjm8aeuB6fB0Al AiHt.niXBVFcd92s4g4m8gQkQTHTrKqfCg_j1LGkHlPJKneB2p9.Fw3XgcvLmKRPWKwBRJGh1Za7 qENkupx0.yYnLU01zdEfwGWKL8AMHJOasbJPHdPXtbuzIITHKViovo.7TjQS2ZQtCP2weeEAQl_Y mHxjWs4XnyKxx3FvvMBGHaxT4wDvlAoWbcCrm31yyGF.2SWCri3IXyV3fTVqsA8KsX86GeTRc.Sj P5K6er9p2xZKvflfbJXUqpRapcEidzxiv5ypoHZvk.RBaIjd4_MXAq_Hsq2MzPCCB6v9tCd2qx5a E0Kpvzue499e1JCVvfkOSOeVfc9TwTeSwq4rE20x3Wo2CKutJrEygQAjbGvs4zfCG7rzCVsEVpQK 7n03jfhXl3P2RUpmsCSNvb310vV2TvXArqf_0LM_5aJ7zCLdM8I9F6.lSWuxBTQAJp7.tFMitZsp 444.s4tR94F6ztlJNssl6zN0uCU5Mls5JkzFsGj4Ad3VuroSGKd9adsk4sKraV7re9FYi_SrZx2y mJCuGGCLJHkmYrvLy23dZGHc0EsS6Pi3qreJPHGzxHrfRwVZa9DpJOx_YDN0GsMJSNnUtpPxIxSu uoLLra8stf6VY1ewGvjpV37.Xf2Ln_6MsuQR0FWbaMZHX4m1uVhLb2AgA_XI_bahLd4SiMVopyQm VA2dbPlJyDvKxqFEX.kb7kfZPFs1hRUBY8Q1DNdgUEVLnyDeeBiD07GqJlju7ElFE_4RM0vrH.Um X4tzgW1qm2k.uPd2a5KGMwJ2dcWDHJEvq_9DWlIsQ6I.YY4m8fktUPmLDKMxtGU_yEfTOu0Vun.J kA4NkkmYAl2XSb7v4ercqukLsn0CSNc36d4KnkAlg4jKOcEJdIaG8O6rBZuQu5UqAGxjeeH51TSS rDr9YdXoeeLbdx4NSy85OFEipsBNTG2DRxpluxxrTnDiSUbjzFytCvG9y.LnKZsMatB.LzyfNxf7 l2vzgA20qYb0L7OFA0mRXwhm8raBEg28iamY2dPZu_fwzKXN.L9Br8g53plGcEN67qEgrcnvy2kG N34RObJvu8TJ8YvQeLT.wQ4ZlW7iWZDWyJnzTyB5TzArNh0L0HTW3epQ8Pyn.F5jWF0OS9T6DQNk sd.oJHywDPFZ_4.AxSJz6UkFDvT4qRwmT2DLrmUt_kPgRW__IUNuMSH7ZqKcGHYXSXCUxsHFXqKc ekzrvt_sMTT1N9nmapXKDFtbismCd.XRbX0STSyigu5YzgyNKWqcGJMTIbpdTtRBao7FQSJhOmFR 82n23Z1zE9KC8eD0xUWIFCuGSpkKTTXLduP.jle6C161opz4qAA9sjL01F642QLbTbr2BLhDe82e 7zV4hlAgF9ZsA_5epHK6857FviVf4Ryf9Um15GCyz6zZV8N56Knc5DuVTj3zmjyqhOthDMvUbRt5 PsBjKnxnwAsngrcgwxf98lXynRTPHuVD6rSgGmJyTNQeroLEDDGSfsXJr5BAKqIXtBNZFWKeAGzm iaBoY6TlraY.cNWO.fuwcjbqQL5C8w_cV7GBbzM.WDJ9IASonDWVReJfX1E8ts99s.GcWU909SML Zfm.FEdZnfO34uY6IKmD0DDh8Dqb7D1Gf5gJJuF8IKGPBA971kcXZ7jyORIn3eVBJP2y_q6AOId1 Q7lCjdwaYWAOyEjdKnJ.ORP1XzIZEGxtU8Yah60evu2VFMKJrSGSJvRJREQpT7Dlm_ufhnFvr9y_ .rro10TUQ653WhewlzvMMaPyS5E_yEMO512doG0AhiEQcsXzrAR9BZ8IyR1q1CVrmMt1Apcff15I HYq8W5B2EzjKF8X_tzL9ypBdmtp5C.8seWLRkTWT3SEOLo5K5PRIQccHUdf26.KHcAdGSL_zhonX dqZW_cPFE8gU00NYzO33KFOwrjt64ysPns4u.HQWLCZTT9Vh0tr.dqjU5_LFGNLQncogn9J8pfAz 2NydmzEo5sE7vPyRWdklUtDV4oUe1CJ9ahIaYmImcXJ_GYLn.CflnSdR7X9IUdFDWiJ1P_p41q_h 74K8qhUQGpLbrlzLLjIyGbgEQiXJj9vS7nZ3KEIl2vSnqHsdn0crNDUF_GxBIovclKkVcgFqFZsW wYYLfVGhOUKtK65d44vaYvvoMd.iNZVHs4TvOtDjhgO7pANgD_PXrexinPoXmTtBkyg-- X-Sonic-MF: <luangruo@HIDDEN> X-Sonic-ID: a74230b5-1c24-4fe9-8632-014ce33b17ac Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.ne1.yahoo.com with HTTP; Sat, 22 Jul 2023 06:35:24 +0000 Received: by hermes--production-sg3-9dc5f54fc-zbsxq (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 75a8bfcd853f39f57b16aa0b3aa78f6c; Sat, 22 Jul 2023 06:35:20 +0000 (UTC) From: Po Lu <luangruo@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83351i338m.fsf@HIDDEN> (Eli Zaretskii's message of "Thu, 20 Jul 2023 20:24:41 +0300") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <83351i338m.fsf@HIDDEN> Date: Sat, 22 Jul 2023 14:35:13 +0800 Message-ID: <87v8ec313y.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Mailer: WebService/1.1.21647 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Content-Length: 824 X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, dmitry@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> Cc: Dmitry Gutov <dmitry@HIDDEN>, 64735 <at> debbugs.gnu.org >> From: Spencer Baugh <sbaugh@HIDDEN> >> Date: Thu, 20 Jul 2023 13:08:24 -0400 >> >> (Really it's entirely plausible that Emacs could be improved by >> *removing* directory-files-recursively, in favor of invoking find as a >> subprocess: faster, parallelized execution, and better remote support.) > > No, there's no reason to remove anything that useful from Emacs. If > this or that API is not the optimal choice for some job, it is easy > enough not to use it. Indeed. I would like to add that subprocesses remain unimplemented on MS-DOS, and the way find is currently invoked from project.el and rgrep makes both packages lose on Unix, indicating that correct portable use of find is decidedly non-trivial.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 05:49:31 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 01:49:31 2023 Received: from localhost ([127.0.0.1]:35293 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qN5Ug-0006Gw-Mr for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 01:49:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54898) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qN5Uf-0006Gi-Dy for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 01:49:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qN5UZ-00051X-PF; Sat, 22 Jul 2023 01:49:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=zRn9FZgcZFeJbByOx9xZRrElfIKZubDLYkRSZpIV5HA=; b=DDTQQhwH9FTR piZ9BBrS9QWLusQc27BP7g1/66caNOUhyn5HYNOA2EC9PzJNvgPNtwthxqB9uCisNW1edf7N60PB0 B+p6/o3Z0bBIAQ/Vvcoz3zPXw3YlFoYHnNx5hiZKWH7fGmsbaiK870+Ybg/1rrto1eoeBvm0aPWp6 IQsOywvoHXJtLEuXT54xmfJh6YiSZMkDU2UvhfksH5JgwNt2Ma3hDOnOdbbNRSz4qDJmCpcZn6kPi be+f1eQ72pp5u9UEzssKFGe9tNaHrDf4XU1wek4AD0v48Ouzi4XZwqPHgcaY9sVljgSlju0ofNKUk ee15sNFo9RB2SfIdd5rngQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qN5UP-0005Cu-Ih; Sat, 22 Jul 2023 01:49:13 -0400 Date: Sat, 22 Jul 2023 08:49:51 +0300 Message-Id: <83bkg4a41s.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: rms@HIDDEN, Michael Albinus <michael.albinus@HIDDEN> In-Reply-To: <E1qN2Wz-0008Jo-L8@HIDDEN> (message from Richard Stallman on Fri, 21 Jul 2023 22:39:41 -0400) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <E1qMg6O-0003aa-SZ@HIDDEN> <E1qN2Wz-0008Jo-L8@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Richard Stallman <rms@HIDDEN> > Date: Fri, 21 Jul 2023 22:39:41 -0400 > > I wonder if it is possible to detect many cases in which > the file-name handlers won't actually do anything, and bind > file-name-hander-list to nil for those. I think we already do, but perhaps we could try harder.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 05:27:23 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jul 22 01:27:23 2023 Received: from localhost ([127.0.0.1]:35275 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qN59G-0005VA-M7 for submit <at> debbugs.gnu.org; Sat, 22 Jul 2023 01:27:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38316) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qN59B-0005Uq-U6 for 64735 <at> debbugs.gnu.org; Sat, 22 Jul 2023 01:27:21 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qN595-0004yf-S5; Sat, 22 Jul 2023 01:27:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=XX7Ug0FB9iYtRvRRVmZkR7izyZKOi6hYkCTIpmCHzwQ=; b=MPTodv4gRJaS 3EFB/wTDxCkNv5vmjJwTvk5vvNQkGuG19yHhXwCnHFGxXhTjIuHOAnDV+4Mbni6V9QFRTA+dMM58P BOO36aOkYiIE89vSaGvkRhkS6i8iLxBVjuyYTjV+AheNZESVqne4F9/HWp5d/Wh9nuC3bZyYfe3JY l3BR+q9bLst5nOEhBGHqnDaW6nBapKTrGUO1XNK9JA2HeLcWlCW5mQCqfKgPw1hNiTQYp0qgQK4kO jOZrktgnJMeX7hydNVwOPeyJgYYTyuRwXNFHlIRkGYRJ6pMLKb8fQDgzjBToxbGkq64NMXoKj3XUy 1wOzyznetbyKXUuVVf/DmQ==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qN58r-0007YI-Cl; Sat, 22 Jul 2023 01:26:58 -0400 Date: Sat, 22 Jul 2023 08:27:32 +0300 Message-Id: <83h6pwa52z.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> In-Reply-To: <ier8rb9ukjq.fsf@HIDDEN> (message from Spencer Baugh on Fri, 21 Jul 2023 15:33:13 -0400) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> <ier8rb9ukjq.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, michael.albinus@HIDDEN, rms@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Spencer Baugh <sbaugh@HIDDEN> > Cc: Michael Albinus <michael.albinus@HIDDEN>, dmitry@HIDDEN, > yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, Richard Stallman > <rms@HIDDEN> > Date: Fri, 21 Jul 2023 15:33:13 -0400 > > Eli Zaretskii <eliz@HIDDEN> writes: > > The first idea that comes to mind is to reimplement > > directory-files-recursively in C, modeled on how Find does that. > > If someone was thinking of doing that, they would be better off > responding to RMS's earlier request for C programmers to optimize this > behavior in find. No, the first step is to use in Emacs what Find does today, because it will already be a significant speedup. Optimizing the case of a long list of omissions should come later, as it is a minor optimization.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 22 Jul 2023 02:39:52 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 22:39:52 2023 Received: from localhost ([127.0.0.1]:35128 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qN2XA-00068d-1e for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 22:39:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47780) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <rms@HIDDEN>) id 1qN2X5-00068H-DG for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 22:39:51 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <rms@HIDDEN>) id 1qN2Wz-0001tQ-Sr; Fri, 21 Jul 2023 22:39:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=Date:References:Subject:In-Reply-To:to:From: mime-version; bh=h/FbF2fHgC77hFUoYIIUucSYk9Axy+/OwzXcMW0NHyY=; b=GxNvALdmCe9u B7T4Ov562pFVEOq6/jcrabfFK45k5qQSjsL87+EjX8i8Z+539SSSlxIVnThKJzcjYLyiqXm+4hSSF ZKt+L+GDnWjx0dOlPJbkqqDGVz5P1lCtvTI/4jhXo4p97x0D0i68kOn06oeixPGz/tlkXPfn/LewO kHornn7otcpJErDudfR+oIUjPL168d/5mkq3WU5ZG0Xxkv3TOj7/U6xKjeRW/lnyKyOI3gG+jXy0W 9BaPDFRapp2ESR6FoEOFZSCGRI5anyEnbI1XpmZLEOcIwTOhx1mjntmrpkJbBeFrRRVMy14FZ40/U oiSD1IgfX8DNTlck4T8Ngg==; Received: from rms by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from <rms@HIDDEN>) id 1qN2Wz-0008Jo-L8; Fri, 21 Jul 2023 22:39:41 -0400 Content-Type: text/plain; charset=Utf-8 From: Richard Stallman <rms@HIDDEN> to: sbaugh@HIDDEN, 64735 <at> debbugs.gnu.org In-Reply-To: <E1qMg6O-0003aa-SZ@HIDDEN> (message from Richard Stallman on Thu, 20 Jul 2023 22:42:44 -0400) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <E1qMg6O-0003aa-SZ@HIDDEN> Message-Id: <E1qN2Wz-0008Jo-L8@HIDDEN> Date: Fri, 21 Jul 2023 22:39:41 -0400 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Reply-To: rms@HIDDEN Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] Since people are making a lot of headway on optimizing this in Emacs, I won't trouble the Find maintainers for now. I wonder if it is possible to detect many cases in which the file-name handlers won't actually do anything, and bind file-name-hander-list to nil for those. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org)
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 20:11:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 16:11:40 2023 Received: from localhost ([127.0.0.1]:34970 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMwTT-0004LI-OW for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 16:11:40 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:51263) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <sbaugh@HIDDEN>) id 1qMwTS-0004L2-0M for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 16:11:39 -0400 From: Spencer Baugh <sbaugh@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <871qh1oy2b.fsf@localhost> (Ihor Radchenko's message of "Fri, 21 Jul 2023 19:37:48 +0000") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87ilaepkaw.fsf@localhost> <ierbkg5ukme.fsf@HIDDEN> <871qh1oy2b.fsf@localhost> Date: Fri, 21 Jul 2023 16:11:31 -0400 Message-ID: <ier5y6duirw.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Ihor Radchenko <yantar92@HIDDEN> writes: > Spencer Baugh <sbaugh@HIDDEN> writes: > >>> Does find use multiple CPU cores? >> >> Not on its own, but when it's running as a separate subprocess of Emacs, >> that subprocess can (and will, on modern core-rich hardware) run on a >> different CPU core from Emacs itself. That's a form of parallelism >> which is very achievable for Emacs, and provides a big performance win. > > AFAIU, the way find is called by project.el is synchronous: (1) call > find; (2) wait until it produces all the results; (3) process the > results. In such scenario, there is no gain from subprocess. > > Is any part of Emacs is even using sentinels together with find? rgrep.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 19:56:45 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 15:56:45 2023 Received: from localhost ([127.0.0.1]:34957 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMwF3-0003xZ-Ho for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:56:45 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:40677) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMwF1-0003xM-RZ for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:56:44 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id C03DF5C00FC; Fri, 21 Jul 2023 15:56:38 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 21 Jul 2023 15:56:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1689969398; x=1690055798; bh=hyK0HyT/9j757p2K1pbxym7ydTJH63OM4OQ +m6/jmzg=; b=EPmGIGzAsvYqXQhvtLLkq4eIg4dXeGuupxpivWd4+0PLdYRWB2r N8uNt5zFPYitnfObKm4oT3nyCPxDZZ0Y6WGT5/8YhMtMhzUSkaqUta0w8avLfRc3 FVb5h/B9+4I+Itz9hcnWG2+XBkJpNT9fBipa/7gaAwTyaMvc19clZ8zFVNsNV2Np HoNxO169YAYU7yWbnwpfBFTcy7ouY5nbvt4Mgkb0BeJilgrNlT1uGLt4H+aGK4T7 jgLPFI2nLkGwI30L7xI8qgarvuPZTVwxI9puGV0I6ta7f6WAhN/zZLftjMfxeALH 3iMSFKTIf40sA+PeGhRn3bnB91gdnDbbmtQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689969398; x=1690055798; bh=hyK0HyT/9j757p2K1pbxym7ydTJH63OM4OQ +m6/jmzg=; b=BY83L8kRyp5kg6oI/ZRDpgIr6cNQKLGyWrxr7cdzLVzk945uQwS /WhQFiEn1aW7nL7WbIiBPnyyPc8C9EBqgR7HX7Vtt7UrNCg349nOEp41vJxT4gxx jJpfR7vJO2K19KDrmDFqPkBiYpWoLpkXu5VEmhBIJMhpkNzBDzbGG/qARKe1Q+Xg m2+IxBREYpq91GqouqI2lKZraCjQwZLWrGj6UZcWzbI7R3SbBmlU1va3MVMJRQS9 hUJ4LREdQK3cmiW/5+yyZcpWB4eAvxeag0opqoQfnzu+wzqg1MUzOlOqyY2W/7ju T7K/H5ZRQ03lR02v711rvxTNpSgJREg30Zw== X-ME-Sender: <xms:9uK6ZGQEgWO0bJ4wr64FtHSGHy79kQ31dx8birY6Q_t3H87PNZqvlw> <xme:9uK6ZLzSBlSoBqCAC37UIk_33bCmSejaM-qm7w9KkfzjXAkAL4KTJHwIbWNSw5wwz ou0akCEMaxTgIhEpOI> X-ME-Received: <xmr:9uK6ZD367CxdGc7zPgxcf9za0ykRR0pgHgp6OT4IumHijyEo2kCTo2Q1q6fgA9U> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgddufeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:9uK6ZCDG2OSOBTzg917-HHr7dkZupZYvTVxDyavTFEtUdJUCHQ5j6A> <xmx:9uK6ZPjfENR86-XGrqJ6NbiMYy_RCzXhgo6xCY-Rv0WtPROP5sv-4Q> <xmx:9uK6ZOrI7n0Nn7dOMDHIrDLWnjHaf4NEUy_-9taYmHKr1XzuX1JVVA> <xmx:9uK6ZHZ19sC8IIh525ul_k8c0nQ9MNqPQUglBmm8URK1c8Go2PIs1Q> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 21 Jul 2023 15:56:37 -0400 (EDT) Message-ID: <debbee40-8f60-999e-98a2-7f99f810500a@HIDDEN> Date: Fri, 21 Jul 2023 22:56:35 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN>, Spencer Baugh <sbaugh@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87ilaepkaw.fsf@localhost> <ierbkg5ukme.fsf@HIDDEN> <871qh1oy2b.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <871qh1oy2b.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 21/07/2023 22:37, Ihor Radchenko wrote: > AFAIU, the way find is called by project.el is synchronous For now.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 19:54:13 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 15:54:13 2023 Received: from localhost ([127.0.0.1]:34952 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMwCa-0003u2-CD for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:54:13 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:49719) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMwCV-0003tn-8T for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:54:11 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 1A8ED5C0090; Fri, 21 Jul 2023 15:54:02 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Fri, 21 Jul 2023 15:54:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1689969242; x=1690055642; bh=8mRF5AHmBaLcWh8sMzoKBdXdlWWQdYJHgWG pCGKqgTo=; b=eEswCrQ4nYKMug5OWTkKNlgZNcLyLsi5E/8Q5f1pOWBeXh1CtEM g6TBT2CekwD8LtU8Q5tYcJ8HhzovCIKkagUShdUyIaX5Rjxz0VNrIlJupDqcvvV8 s53Q2ZToRJa4BVVae6CHQUbX17BKjy7OU3kuqBLzS8tHR7rjloTW2eCk7+O7sW2M +P1IU1cPgretTRCo9WtEL43QRqfCTYkbhx2bTK9qzSuPwhpUmno3gDB+90xWT4PU aKSf+Ceduvyn5zCREx71YDrfp8jkMwUrkB004KjdTMufgaxdjXpVbeh9+NPX7mSM g7VZs4grEerjiXA8guTOMH5XPG5TSB/0U3A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689969242; x=1690055642; bh=8mRF5AHmBaLcWh8sMzoKBdXdlWWQdYJHgWG pCGKqgTo=; b=UMAypXmwtfRwPlDeOHuYtgF3oO5foss62miweR6xPMGS8Vz/EX1 7SV7BPHS2zwGOBhnHyf08uOHjgwrKxBPnplIW0jEX1OuVk1HbipIAq8iPmV/8xE/ 2Aw9oggQ0hjXoISQWHa6K2vr4yeIogs0OHLL2Ut8cp/ZHRCfD2CT0LmSUR24TOXs Vn9+Od7DGv0jnde16yx81Z23keKnjSXbvF+VCOsfJRQyKbmHQRVRMrHtnYlgnmWE l4JhI4t6hvwJCnoRH1ZQ5UKME96A/JPRLvdxFDo5ZGDDzhbPAAh/W5tCjVGWrmoc KJPvN3Ky/kE3H7qEpPXUXoZuFII3u0EFP/A== X-ME-Sender: <xms:WeK6ZBKTyiJAAu7kvjhispVDEzyAE7HZk7R2nY9gsvfgwcvLzkOi0w> <xme:WeK6ZNLm8EjhrJ_sUS_1bKE8DWArPVxCaxLPepXhvhmUiTukc2Im-ZyDzv7PhNOcT Fx9magTnIS5TCQhyII> X-ME-Received: <xmr:WeK6ZJtL61pyIwJUA_uCdnogSrzYiQcFxt-eQM1C8hTsh9iOJRrd5G4qEmWVoho> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgddufeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:WeK6ZCbf433taEQW5fmCHE5CGJu54UHWjHcemUgIiTarrjTnT6m5Ag> <xmx:WeK6ZIbrzTsWCFtoj8HRbsE2atExIUX_5sd4G8OvKwt_S6ZtT6BbyQ> <xmx:WeK6ZGBItkxivXztBw10ZI9kH6h1e76D1AKc_2Fn-R2n41bTCcgeLQ> <xmx:WuK6ZHmj_JvDok-HXAtaYH0pw-3hGJSYBE69yY8fNdDsDpaJlSQGxg> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 21 Jul 2023 15:54:00 -0400 (EDT) Message-ID: <ce17fd10-f119-62b7-e69e-dcc76885e5ea@HIDDEN> Date: Fri, 21 Jul 2023 22:53:58 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> <ea2332cd-ec57-794d-d662-ff522a4dd404@HIDDEN> <87351hl0yj.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87351hl0yj.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 21/07/2023 18:48, Ihor Radchenko wrote: > Dmitry Gutov <dmitry@HIDDEN> writes: > >>> Locally, AFAIU, running `find-lisp-find-files' without >>> `file-name-handler-alist' is equivalent to running GNU find. >>> (That was a reply to Eli's message that we cannot disable >>> `file-name-handler-alist') >> >> But it's slower! At least 2x, even with file handlers disabled. >> According to your own measurements, with a modern SSD (not to mention >> all of our users with spinning media). > > Yes, but (1) there is room for optimization; (2) I have a hope that we > can implement better "ignores" when using `find-lisp-find-files', thus > eventually outperforming GNU find (when used with large number of > ignores). There are natural limits to that optimization, if the approach is to generate the full list of files in Lisp, and then filter it out programmatically: every file name will need to be allocated. That's a lot of unnecessary consing. But you're welcome to try it and report back with results. Tramp is easy to disable, so you should be fine in terms of infrastructure.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 19:37:45 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 15:37:45 2023 Received: from localhost ([127.0.0.1]:34939 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMvwf-0003RY-9K for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:37:45 -0400 Received: from mout02.posteo.de ([185.67.36.66]:54303) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMvwb-0003Qz-GK for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:37:42 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id A911D240106 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 21:37:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689968255; bh=PG6jDwRXJXDtlB0rwfiITmTxfc//YDZ9XPhZ+wjGZ6E=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=MThnlsn4Jua+nJNT0byBa6dLtGEhNur/heQQyg/M1n99LYTCzINgay5vBOImtLf2M s1jgEsEQPDo9Sus28zkkEi6iIaMXFEhBUru3w7dEhC3rhQPX7Qg/PM8ki6I5TK/f/c B3sv4Wa1GoUFy/6e1rCiB6ohTeVkOT9yZT/n1YZ4uao/HpFqX2zPqKD7TyWyy6zEib 3AfE7Xhqz4cl6gSyRQIv6P5SHvNJFiTNsIb2mtv7vK5+w2nx51klXUPMJWy94ti2i+ 4+HBHsfXAELuKIkj6s+uBhkdYgvfVLyDAye09R8Wh881+QGlO/AFHKL4Irx889TTea TY0yOd/BUJ2dA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R70FV5zm0z9rxN; Fri, 21 Jul 2023 21:37:34 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <ierbkg5ukme.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87ilaepkaw.fsf@localhost> <ierbkg5ukme.fsf@HIDDEN> Date: Fri, 21 Jul 2023 19:37:48 +0000 Message-ID: <871qh1oy2b.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Spencer Baugh <sbaugh@HIDDEN> writes: >> Does find use multiple CPU cores? > > Not on its own, but when it's running as a separate subprocess of Emacs, > that subprocess can (and will, on modern core-rich hardware) run on a > different CPU core from Emacs itself. That's a form of parallelism > which is very achievable for Emacs, and provides a big performance win. AFAIU, the way find is called by project.el is synchronous: (1) call find; (2) wait until it produces all the results; (3) process the results. In such scenario, there is no gain from subprocess. Is any part of Emacs is even using sentinels together with find? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 19:33:22 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 15:33:22 2023 Received: from localhost ([127.0.0.1]:34928 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMvsP-0003L8-OY for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:33:22 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:34453) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <sbaugh@HIDDEN>) id 1qMvsM-0003Ku-KZ for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:33:19 -0400 From: Spencer Baugh <sbaugh@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83mszp9kl2.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 21 Jul 2023 21:38:01 +0300") References: <iermszrwqj4.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> <83mszp9kl2.fsf@HIDDEN> Date: Fri, 21 Jul 2023 15:33:13 -0400 Message-ID: <ier8rb9ukjq.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, Michael Albinus <michael.albinus@HIDDEN>, Richard Stallman <rms@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Michael Albinus <michael.albinus@HIDDEN> >> Cc: yantar92@HIDDEN, dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, >> sbaugh@HIDDEN >> Date: Fri, 21 Jul 2023 19:55:22 +0200 >> >> I'm open for any proposal in solving the performance problems. But since >> I'm living in the file name handler world for many years, I might not be >> the best source for new ideas. > > The first idea that comes to mind is to reimplement > directory-files-recursively in C, modeled on how Find does that. If someone was thinking of doing that, they would be better off responding to RMS's earlier request for C programmers to optimize this behavior in find. Since, after all, if we do it that way it will benefit remote files as well.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 19:31:46 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 15:31:46 2023 Received: from localhost ([127.0.0.1]:34924 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMvqs-0003Id-A5 for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:31:46 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:37927) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <sbaugh@HIDDEN>) id 1qMvqp-0003IK-Bc for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 15:31:44 -0400 From: Spencer Baugh <sbaugh@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87ilaepkaw.fsf@localhost> (Ihor Radchenko's message of "Thu, 20 Jul 2023 17:25:11 +0000") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> <87ilaepkaw.fsf@localhost> Date: Fri, 21 Jul 2023 15:31:37 -0400 Message-ID: <ierbkg5ukme.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Ihor Radchenko <yantar92@HIDDEN> writes: > Spencer Baugh <sbaugh@HIDDEN> writes: > >>> Sure. It might also be optimized. Without trying to convince find devs >>> to do something about regexp handling. >> >> Not to derail too much, but find as a subprocess has one substantial >> advantage over find in Lisp: It can run in parallel with Emacs, so that >> we actually use multiple CPU cores. > > Does find use multiple CPU cores? Not on its own, but when it's running as a separate subprocess of Emacs, that subprocess can (and will, on modern core-rich hardware) run on a different CPU core from Emacs itself. That's a form of parallelism which is very achievable for Emacs, and provides a big performance win.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 18:37:34 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 14:37:34 2023 Received: from localhost ([127.0.0.1]:34864 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMv0P-0001uS-RG for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 14:37:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34024) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMv0N-0001uE-5V for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 14:37:32 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMv0H-0003Lu-Hm; Fri, 21 Jul 2023 14:37:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=j46TbXnEu0HRty5w3f4yCJVVKLZpOgHI4UAKN4Cc1iM=; b=LTUOIXyBwiTU 7mfsrFQOpet03PfP28l5+EOi+/DitX6FfgKvzj9dKtjUWcbc7kTvu9F2ruEVzcArBzS4NDPKizpj8 QZxufeyyHNi3fgfmmtDiJKyZmkBpeJRpxaTtO1IUxysdk8KR86rLXc1+gp6WJNSYOnehsCOhzbMiU i1vbzxqCOIA6vZLd/vMwT5IYRNg1hbj43HhrxbbS4vV4RTk/xDfg2SDD/9w/OCOW+DXMAAcel/0W8 +H6d0A/CQanWwfgYsnsCGsPRmJdDEvPjTm+OM94t2L4MXs7AEtYKlZn1nAcj5pU/xWfBnsS36aDLp yzmq7oc1b7u5giKkhiuBuw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMv0H-0000Hj-1r; Fri, 21 Jul 2023 14:37:25 -0400 Date: Fri, 21 Jul 2023 21:38:01 +0300 Message-Id: <83mszp9kl2.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> In-Reply-To: <87jzutb14l.fsf@HIDDEN> (message from Michael Albinus on Fri, 21 Jul 2023 19:55:22 +0200) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> <87jzutb14l.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Michael Albinus <michael.albinus@HIDDEN> > Cc: yantar92@HIDDEN, dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, > sbaugh@HIDDEN > Date: Fri, 21 Jul 2023 19:55:22 +0200 > > I'm open for any proposal in solving the performance problems. But since > I'm living in the file name handler world for many years, I might not be > the best source for new ideas. The first idea that comes to mind is to reimplement directory-files-recursively in C, modeled on how Find does that.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 17:55:37 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 13:55:37 2023 Received: from localhost ([127.0.0.1]:34838 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMuLp-0000uo-FF for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 13:55:37 -0400 Received: from mout.gmx.net ([212.227.15.18]:50549) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMuLm-0000uX-KN for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 13:55:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689962124; x=1690566924; i=michael.albinus@HIDDEN; bh=/L+LiFIUDjAWfV3Q0X0jByP4OL6MKkRgeMa0hyzz/t8=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=ESg6Z2vwiX60T95vvtiB1oH1t21Tlx5ZyJq9oNRi3gPdZPlg7DHgGKVtLoJgCoIEbHaIykI lexp0IA9ttD5xNFQ6uPRnJsUvNtcRAzlKdgCdG6TSNSviHjt5tyL7AxW5ZEiVjv2KpaW5a/Pn 1VqrzY3wZzqG/x/UJySS11YGw+uqorMmuo0apU2piSN5ET4HJPQqCuQxOfA/FfuU8Tw/dXauR w+OUK+9R6HmVWTAm6A254V5DxVpExBFu5FDvhHK20EvnQQLTQ4oZwJUPpifWgq8ZhzJ7boPZe 5AtRFNGae45typ7ZD59KaDnG3kWkBSclOGmUZ+jC/vGBLFn7H2fg== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx004 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MFsZ3-1q8OA83Uel-00HKPF; Fri, 21 Jul 2023 19:55:23 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83pm4l9n0o.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 21 Jul 2023 20:45:27 +0300") References: <iermszrwqj4.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> <83pm4l9n0o.fsf@HIDDEN> Date: Fri, 21 Jul 2023 19:55:22 +0200 Message-ID: <87jzutb14l.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:fb479R4z+XEMOEARPv/3zBDiJ9aYxBCHsZTlb3gGS9TApjEbiNx zxGXP84difRSTKd7wPqYVdK96YapA/Z2ULRmcupm7SU3SpM8uhUrQKwQVPumWLNdYbnLLnv 8+pbo8M8lbAuDWCkaH6MwfA2MdG4jp2g6pqm+CqVZZO3rY1EJ145QJUEUcE2rHNawEOL+jB IT92Mi9yfTI/edBnvk2Mg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:6Ydkd2DPM/g=;mvO9zL+qMDJXL5xU7IhS1qpzFY4 sFVqVk0mzloFarcuvnzjWOFqKr6zPPMP+FPeeKGnhPHRLKgcOPHtSOFnvzfA3FmVUdchzUcMp l2jFHC5niXMT8IZVAwRVvUVS5weoxdq/mAWBSTMvKFyiK3pwADMtN4OhO22ZbZ64j0tV29E9w 4vQz48Ft/Gf0p4Sl9O3VaGNyIuBypDYx7lQAXCwy3Hj0/NcUv5eQQJNtKYyg8v5+QjERqBNRG Us4/CVsli9HGCHkW4GaTGdkr7hesXJYx/EntShwB5pL9JbG8ax9BKBp8tesKMAHInRh8ZKLMP 0W7SsgVr92J9aFpnp9B6PG0lu3OHTcTy5bZ9clcHIwYKmYJ0LesQ8nMn7ULh4zOvSLeUsSRj5 bHxI5lCVkx/BvkDTmqudFpc28fDO9s1zLZUMR5G5rTNMBlOhnT4qyHxfhlW76Cz8sPyu2ZFyV EIJbw0X7Pyyn5uL+psVHSjl7JSeZE64H3Fzv73uJIXFKK7nbKj40hVYCksyldNkRkvSLuxhYS 6XfkvXxLbeUrjr+smxLUCexWY1/19jTev0Soc9ANyH4u6qz38rVFQkQJZJIqKUVGkAbTbioLe Cy2bnyeKjleQwLQx4EnPBaez569zEAb7km7xeF9ypXVR9yKzReGZ6LfToAmMmwO9VWrnOdhBN +R5MVdc3f58zQt5W5N4cGNtgX61sk48tNqp/Bzbby4hO33PFKrqvfWxlrbomh3z4lnAUgPuud uafdaIUwZT92IiBKoEGSFoEMxWw5/K+DGrZAO3Nx83H+NUZXQxkBGQBPyM1PNYJt8WBYU6JM4 A2heYrG8+kmB3MRcgugG8R7tAT5zS4gFCNIWLdNAj19esW/ux6IDeF+MAYVyEaJDjtL6C46jT RoCEW+fS2NhJlLKg3X/z6qUTne87NVl4tGCAwHsJkxoUkKyyaTfpqmjf+tY2dQ4r27GF8L4Nq MsbJSo2kAAyV36INOkt9Y7vyWwU= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: Hi Eli, >> Does this mean you disagree to install the two forms I have proposed? >> Although not perfect, they are better than the current status-quo. > > No, I just disagree that those measures should be seen as solutions of > the performance problems mentioned here. I don't object to installing > the changes, I only hope that work on resolving the performance issues > will not stop because they are installed. Thanks. I'll install tomorrow. I'm open for any proposal in solving the performance problems. But since I'm living in the file name handler world for many years, I might not be the best source for new ideas. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 17:45:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 13:45:03 2023 Received: from localhost ([127.0.0.1]:34819 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMuBb-0000Zr-7J for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 13:45:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38064) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMuBX-0000Z3-GP for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 13:45:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMuBR-0001hd-C7; Fri, 21 Jul 2023 13:44:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=5sHJSDe6ZjHMkbtphaUN/q7HV1JVVPWj2jzdv5dCO48=; b=NzzL0FnIsEAZ w/H0rBFOKd/+SxJxrb3/IcDDwUVZVG/vxRv1sfJwkjbSpuq6ZOw+luDXe1bf7+BwUvGsxsll+7ZrB 3Fo/ATs+WybgurppskzA3V2MyUGy2AHaMEVqj7M03hHc5i9AV8s//pJ3ypkD1/0H/iP6dugUWZ5f1 r77XPVbeTJzjbgEQQlSGl6RCng33us0JTSnYW2/o5YQwD/2BkWhdjnvVf0vplrXKQKDxs98pLcQ07 IiPywb5iu6FVYhoNGqH/4EViy6gXFY3ToVkZR5v5D2HKR24/oEob1Nlm7Ez1VU9uYr88FJ8ERlSgz sTkiiu7T/YCbbChiqWvnrg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMuBQ-0003qY-S0; Fri, 21 Jul 2023 13:44:53 -0400 Date: Fri, 21 Jul 2023 20:45:27 +0300 Message-Id: <83pm4l9n0o.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> In-Reply-To: <87pm4lb4fr.fsf@HIDDEN> (message from Michael Albinus on Fri, 21 Jul 2023 18:43:52 +0200) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> <87pm4lb4fr.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Michael Albinus <michael.albinus@HIDDEN> > Cc: Ihor Radchenko <yantar92@HIDDEN>, dmitry@HIDDEN, > 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN > Date: Fri, 21 Jul 2023 18:43:52 +0200 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > Disabling file-name-handlers is the wrong way to solve these > > performance problems. > > Does this mean you disagree to install the two forms I have proposed? > Although not perfect, they are better than the current status-quo. No, I just disagree that those measures should be seen as solutions of the performance problems mentioned here. I don't object to installing the changes, I only hope that work on resolving the performance issues will not stop because they are installed.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 16:44:11 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 12:44:11 2023 Received: from localhost ([127.0.0.1]:34760 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMtEh-0007Gn-3k for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:44:11 -0400 Received: from mout.gmx.net ([212.227.17.20]:46681) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMtEe-0007GJ-3O for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:44:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689957834; x=1690562634; i=michael.albinus@HIDDEN; bh=WNc5Z1pDrLwFNrhF2Jo6gqIxSJpwW9GkdAW+fFYuvP8=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=USTL/jD625nXl7UoDiPC1UWGbMmkWdOFuFrCoempWD11nVb0SaHLDRvZYkiC4BSw9eoq5ZV uzznbgtc96AjLneA85VIMy5828Y32zXL5aOHoHSxEIuu8J/97LQyai6GJiAO7SNt3BYmsNvLN be0mL4K3tpSPw87UWkwszRucK6pZZ8aElXTQ3OFedhugvr0ybrTXelMuubx9Md4LamsHk90X/ ww0lzYn6iIjGK6v+0hae5CoYuc33Y7ERYszFWC9/TY9MQErOxpfTNpEWUhjHjbOmX6gimhPow XrfQGB6Fvc11x3r42fuUgK0i4Fbl+d05Hoj4uVZ2lgTaIPlqBbMA== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1N3KPq-1pvq8W0xzm-010OqC; Fri, 21 Jul 2023 18:43:54 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83tttx9q4v.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 21 Jul 2023 19:38:08 +0300") References: <iermszrwqj4.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> Date: Fri, 21 Jul 2023 18:43:52 +0200 Message-ID: <87pm4lb4fr.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:OGPJuh3/48G58nYpZP0GZZFck2t/YaUVeSvJYgik1C47Jnwk5ZQ 3zyTfgZccPoov8SicmZhVMuY3VPqfabhE6lu9iWcKbr8lqDsZYroBda1xEawDjtJurh9aN0 ZwU4zBO4nJhxpoCfNJlspmOVgj5nBldXlTezABKy70w7lUaOXTthaxqA/+K77hpFU5p+3/9 NjgEqzapiXghTFaqBT6+g== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:g+Hdt2PwdYg=;4zwIPQxifJi/Db8qv1byuvZTD6o vPp41yCPyMFqUMz+z82wq/2si6HlXQSPVI6mG+kqaO28fzJj08PGk1YHDj3OCNqR8sIAwynIo tWApg4xHokYE01YMttcZp5GGZLrkx733rYZ0kbGhDgJSoxtfxqlPPPMhQHfcTSVG6hZGJxb9k +8zwUG8mY/GCCNz8yPVHD0e1NE9c+B2/RaTH+bK5oNaEwck7j7a6caI0l0ZUBl+pjBufRxKKq 5MX9MQoxfVu+B0zm+qc0QnfzPP1OSUhGqeJZP5ZT7QaL8SHNOjqCycsq/JiEDxZY2y0BINazL kFHPBSGA/Mb9nhSKEpR4qvLwBOdp7NwOYFGSRykHRz35L28ETt0mBfyIOZgnVb+yoH5g5cmej J+U6/aiER4QNFJCpq5ZTyjcAr+kc4u7mNUusz35PM6REW9tdRmL0qdai+imy1HUtjzolHv/a1 keYnxlJxIXXMyaGIqmWGSd9kp0akJZD72LyoBRiCo2zPormf/3UyKKHoihVdgFJxXAaHl1U4w //KVbiTXUwpgIT7YHDSA99xtd/ssSxM2NH/xijPbObRO/977BjuxExOwtNeseGDlG6NPJWpCW Ft4nWAhvBEEqsSr6LHWj+K3YQEaYpt5RzVnhCwd+5oCMY5Zh3V/SGHVNdKApuwSDiCViF8nMG 6IsW+xiiGwnUqwqu2VHVikMfOVKHPqmb/euIim9P0v58pR820epByUjicWnz4/m27cJqUDlea e6mOWMbxpnxAF8ab56whbTepfIMPFsAe5CD+ZqXpyHtxiTcrMfJWhk/dztQhkYsDFoL6a7+XB QxpiEQ3MK1M9IAemMp4wxytxm2pCiOzw96KFVcVwKke8NKgWVHNTd7TzHjg8QfU5KLvjY4E3x 1jbaZSK3doa7BVlwdNeHtdEradC3W6D4YPp2b2iKVcY2FMTEKpB3R2UP8DJ/WC/TYiSHhu6Wu ScLU1fglD83ROgbp2MgPs9qCaxk= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Ihor Radchenko <yantar92@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: Hi Eli, > Disabling file-name-handlers is the wrong way to solve these > performance problems. Does this mean you disagree to install the two forms I have proposed? Although not perfect, they are better than the current status-quo. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 16:43:29 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 12:43:29 2023 Received: from localhost ([127.0.0.1]:34756 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMtE0-0007FI-Nj for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:43:29 -0400 Received: from mout02.posteo.de ([185.67.36.66]:58499) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMtDw-0007Ew-TC for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:43:27 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id F4062240103 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 18:43:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689957799; bh=cdXSD+R+tj+Uy4EfRAZSd/RBzmT4HE4W8XTDPRozTEM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=YxRWId09knQZngm75QHnD89Ya2o4PrOXIBIH97Gx2+ZXDeZeWgL7kks6lT/TbaqlA HAFXOM0ZNdg613jF9imoAC8KjgfoS8NWUAr2nHH19A5iD68vHUiozIcdIFZXGuJ6Dp jOQrgYV25r3ZL+5+aBkzMvxc63ixoM3xuAPL9ohJHR6WGX5Iy7mDeW9b8Rn0k+WtOO QzVDkLM22yjdg2VWfScIXdsgq8Cq0hqOKZ5l8vGLMf1R9nOPdI+j361u1qkH4fQG0y HAqht8iU0IVf/+becNU1O6Exw7mTxyTxaz41qHx9wZR2cZ/EfPJy28L/U35IQd8g7w kh9x+B+agfeQQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6wNQ3MCNz9rxV; Fri, 21 Jul 2023 18:43:18 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83tttx9q4v.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> <83tttx9q4v.fsf@HIDDEN> Date: Fri, 21 Jul 2023 16:43:31 +0000 Message-ID: <87fs5hp64s.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: > Disabling file-name-handlers is the wrong way to solve these > performance problems. We are in agreement here. Note that I am talking about optimization. And Michael proposed to provide a way for disabling only the tramp-related handlers, when it is appropriate. >> I am pretty sure that file name handlers are checked behind the scenes >> by many other common operations. > > I'm pretty sure they aren't. But every file-related primitive calls > expand-file-name (it must, by virtue of the Emacs paradigm whereby > each buffer "lives" in a different directory), and that's what you > see, by and large. The end result is the same - file handlers are searched very frequently any time Emacs file API is used. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 16:37:46 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 12:37:46 2023 Received: from localhost ([127.0.0.1]:34733 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMt8U-00075g-9w for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:37:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33036) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMt8O-00075P-Sd for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:37:44 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMt8H-0005CX-T7; Fri, 21 Jul 2023 12:37:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=j63R0ElzseoRPxTXKoYo4/bMg1JDMiOJ44bgcz5ec6M=; b=iw4lcbWZBR8e /Me8xe4O+DxrigjpPDMvL33PIRqGY0KGIBHp60XbUT86kdLp0kBWv/GWSifUA+8dsGvQehJWHZi1q DryC8/jhshXeG6hjj8Z+z3BGjYmEcbqUh0nDREyM7IXiHhWwGnc6EQiQWwQo3UCSi7WX6KU6ZDgqW IPToDJOhNfI8y4fYBe8CR8ZRDplw+GSI+V7CSaSh+sNzpj1vWDjb5pqBrUWKGW1W1YVQZCHpM2nJm IC5zInziSA5k39Viz5ltIOyssbSVP2K1aiUQssdli1zLJLBbw2FEv/9oU5iOVV9Em9/NNUO9p0KJz 1auBYv346ntoV2z5JnmHnw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMt8H-00066l-6s; Fri, 21 Jul 2023 12:37:33 -0400 Date: Fri, 21 Jul 2023 19:38:08 +0300 Message-Id: <83tttx9q4v.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <878rb944wi.fsf@localhost> (message from Ihor Radchenko on Fri, 21 Jul 2023 16:15:41 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> <878rb944wi.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: Michael Albinus <michael.albinus@HIDDEN>, dmitry@HIDDEN, > 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN > Date: Fri, 21 Jul 2023 16:15:41 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > The figures provided in this thread indicate speedups that are modest > > at best, so I'm not sure they justify measures which could cause > > problems (if that indeed could happen). > > Not that modest. Basically, it all depends on how frequently Emacs file API is > being used. If we take `find-lisp-find-files', which triggers more file > handler lookup, the difference becomes more significant: > > (benchmark-run-compiled 1 (find-lisp-find-files "/home/yantar92/.data" "")) > ;; (3.853305824 4 0.9142656910000007) > (let (file-name-handler-alist) (benchmark-run-compiled 1 (find-lisp-find-files "/home/yantar92/.data" ""))) > ;; (1.545292093 4 0.9098995830000014) The above just means that find-lisp is not a good way of emulating Find in Emacs. It is no accident that it is not used too much. > In particular, `expand-file-name' is commonly used in the wild to ensure > that a given path is full. For a single file, it may not add much > overheads, but it is so common that I believe that it would be worth it > to make even relatively small improvements in performance. The Right Way of avoiding unnecessary calls to expand-file-name is to program dedicated primitives that perform more specialized jobs, instead of calling existing primitives in some higher-level code. Then you can avoid these calls altogether once you know that the input file names are already in absolute form. IOW, if a specific job, when implemented in Lisp, is not performant enough, it means implementing it that way is not a good idea. Disabling file-name-handlers is the wrong way to solve these performance problems. > I am pretty sure that file name handlers are checked behind the scenes > by many other common operations. I'm pretty sure they aren't. But every file-related primitive calls expand-file-name (it must, by virtue of the Emacs paradigm whereby each buffer "lives" in a different directory), and that's what you see, by and large.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 16:15:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 12:15:42 2023 Received: from localhost ([127.0.0.1]:34682 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMsn7-0006WU-Vj for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:15:42 -0400 Received: from mout02.posteo.de ([185.67.36.66]:43329) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMsn5-0006WG-4K for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:15:41 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 634B5240101 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 18:15:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689956133; bh=RN4+cEWqYGuroyNt2YKaOielEQOFdXNB3WmdrAO3nrU=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=qvrCFkCp1z8/wldLRLrTjfiZQzjay/H1jGL1fuJ3tmWqpkM4vxUS9iYjakTnnikL1 na++iKE6r6AePtT5NolYs3UDN6QD+If1xojw+HHx3Bsz0XZNzjgC8vLPv7sxmN2JCO bX/P90XBCJ1xzfO+YGBcw0qzGBO5aIJBh03M4VfElTKfUmPFLLmeJqp8k25WT+IPP+ tmY6YuY0CNsPRk9R/rovasmeBGDU5ID7A9OFz9E+gzO3b9BApTq401/4DI59gjE/9X i5siw5gx+oZfmdELA8wdQ7MoRGIy7VMUZezWnXTD2RhgGp3CSjeHz3S1dbZRtFMpFo GjLE4A9MdFEAA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6vmN1mdCz9rxY; Fri, 21 Jul 2023 18:15:32 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83zg3p9s39.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> Date: Fri, 21 Jul 2023 16:15:41 +0000 Message-ID: <878rb944wi.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Michael Albinus <michael.albinus@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> > (length (directory-files-recursively "~/Git" "")) >> > ;; => 113628 >> > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) >> > ;; => (1.597328425 1 0.47237324699997885) >> > (benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-files-recursively "~/Git" ""))) >> > ;; => (1.0012111910000001 1 0.4860752540000135) > ... > The figures provided in this thread indicate speedups that are modest > at best, so I'm not sure they justify measures which could cause > problems (if that indeed could happen). Not that modest. Basically, it all depends on how frequently Emacs file API is being used. If we take `find-lisp-find-files', which triggers more file handler lookup, the difference becomes more significant: (benchmark-run-compiled 1 (find-lisp-find-files "/home/yantar92/.data" "")) ;; (3.853305824 4 0.9142656910000007) (let (file-name-handler-alist) (benchmark-run-compiled 1 (find-lisp-find-files "/home/yantar92/.data" ""))) ;; (1.545292093 4 0.9098995830000014) In particular, `expand-file-name' is commonly used in the wild to ensure that a given path is full. For a single file, it may not add much overheads, but it is so common that I believe that it would be worth it to make even relatively small improvements in performance. I am pretty sure that file name handlers are checked behind the scenes by many other common operations. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 16:08:30 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 12:08:29 2023 Received: from localhost ([127.0.0.1]:34644 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMsg9-0006KE-IX for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:08:29 -0400 Received: from mout.gmx.net ([212.227.15.19]:51917) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMsg7-0006Jx-EZ for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 12:08:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689955695; x=1690560495; i=michael.albinus@HIDDEN; bh=i56p7OaV5d+8wLROiMiof/k1K+E6k0Y+/BJk4dUv2SM=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=GFDqnHCAI2xAYKCSUaakDkwdKX6sp9nevWyu8q21uc2wRFSYKn9vm9dzmKg8INvYww3OAM4 l2d69alue1SsZ71Ob2H9OT4u9gEv5s3vz7agmBGDItBYNxo1IxKsc9iAYaU5q2iQo7kQ3mF1p MpeivcLXUL0NCb4X4bZi0zv2HOHxnl+VxKPsX5OxmwUbhFgaztZdwdwPHODE7dRYc5FhKRwdI EEpfAYsU/ym6a8Lq4TP1IRp3fI76muw1/5Y+lIzSChwUq5DfMwBXo/exVP3ZHrvsEFWAtLQ2U Yy7tEqXO/sz/29VPTilFPIcIGpiU9VdNGpswSMk7LfewbojExK3w== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1M26rD-1qKwjO2mM5-002WQl; Fri, 21 Jul 2023 18:08:15 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83zg3p9s39.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 21 Jul 2023 18:55:54 +0300") References: <iermszrwqj4.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> <83zg3p9s39.fsf@HIDDEN> Date: Fri, 21 Jul 2023 18:08:14 +0200 Message-ID: <87v8edb635.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:S5cxs2b/fHV5MePDrlvge3joUI0ZN4SaKncPOSaWddIY3MtwAsd 9z66YVfkYExDiuLkcA/snL+Pc3BA0Tjg6/M8zAhY1gVa1Frd55KX5EK2dmDG6rkVI0qiW5K M5RK5JPHT9u5Yd013azVcAsxAhSW++6hOKifDeZD7oRGUTmaYAxXqr5kD9dCoaPuYvdUnLA FQCoKzPd/IUPiZKedutrg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:s9Y0A2U2xvI=;R4MPshp++0+Ho3ys5ffaLRiRRU1 PY17MSZ50DOmogQ2b6/D2m7lZqScJLoMaWnNfyggSYh8RaL9wtv4YY85I0S0ZNkRFVRj+sRfJ /CDvV9Krwp2toxGATEbytsFsX6eXq7eERrK2VL1D5eu7hZ/bwDUhIk2wLUnxXWnoaiJaa3Ucs TfzoWkJSrtrHepN2dVat0UGyTcHj1QZiljvuEOo0/Bbf/yes7VFdF+1HWHAmOQIHSz28suYoO 6GdmS71gjc3cq7kdFDOMTg4+zRTd5jePbYuKd8GDLwkrHO8IX3aF0i/AirlKw2qnpURhCa7A+ ye3AaYxmN3et+fIXvAhZqATfH2+58qPu27o9Gp+7OWkwWVv3LtxL/JwZJQTJkEIDglh0nGOR6 LuTty/jp3ZUZjb6my0U+NSPU8tmRjgin98kKU106tK/rRDCycj38W3xB/fvSU8bB8/UK+btdY lrj3kwxbd3Pucc6mGQ6CJ6jgpheSvFBJUcJ04lUYBzPTOO9OdponBUQBrRVtIQxf/BbE0f/Ks mj52w0lie2Ekoc1MMgi5zzzo3UIMkZ4O9a7LcXv61tSMicUQzczysR/K5ke9qbGMD4R7Qle96 u/gQ7TvEzhGeUZa3T8T0fSdbrONff4ebLAAXnM/0HzrfSKicvUwhWJoczzYiZ5F9o48SATO67 XUyOq/akgB5hfz2srFjQuFwZEwBt8LHoTXPg+Aml4aK2u732blQE1K3rgN/PfzsFImOHqGffM IW2iOq7UzM+sEtVIiWhYoH9sQu4WxpGVN//CMPlmdfK50zPrGqYTqXTnWl8wNr3IUpTVIOk+6 LMi4pKAFXeXru+lz4UtZeEoW6quzneewHYYF5sKVRJOQwuhnGex6kSP0ztKJv9hGYgIu4madF en2cBbcYwKfUxZD9sp5LgHms5WUm8A1XOPm8rglykRB+u4PVYAi/gwHJ7J9LTeegj3/asyTRr /T15dw== Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: Hi Eli, >> > (length (directory-files-recursively "~/Git" "")) >> > ;; =3D> 113628 >> > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) >> > ;; =3D> (1.597328425 1 0.47237324699997885) >> > (benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-f= iles-recursively "~/Git" ""))) >> > ;; =3D> (1.0012111910000001 1 0.4860752540000135) >> > (benchmark-run-compiled 1 (without-remote-files (directory-files-recu= rsively "~/Git" ""))) >> > ;; =3D> (1.147276594 1 0.48820330999998873) >> > (inhibit-remote-files) >> > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) >> > ;; =3D> (1.054041615 1 0.4141427399999884) >> >> Thanks a lot! These figures show, that both without-remote-files and >> inhibit-remote-files are useful. Of course this shouldn't stop us to >> find further approaches for performance optimizations. >> >> I'll wait for some days whether there's opposition, before installing >> them in master. > > Can you spell out what you intend to install? I intend to install without-remote-files and inhibit-remote-files, which I have shown upthread. Plus documentation. > The figures provided in this thread indicate speedups that are modest > at best, so I'm not sure they justify measures which could cause > problems (if that indeed could happen). >> > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) >> > ;; =3D> (1.597328425 1 0.47237324699997885) 1.59 seconds. >> > (benchmark-run-compiled 1 (without-remote-files (directory-files-recu= rsively "~/Git" ""))) >> > ;; =3D> (1.147276594 1 0.48820330999998873) 28% performance boost. >> > (inhibit-remote-files) >> > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) >> > ;; =3D> (1.054041615 1 0.4141427399999884) 34% performance boost. I believe it is more than a modest speedup. And without-remote-files mitigates problems which could happen due to let-binding file-name-handler= -alist. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 15:55:39 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 11:55:39 2023 Received: from localhost ([127.0.0.1]:34597 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMsTi-0005zC-MM for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:55:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40948) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMsTg-0005yy-3g for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:55:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMsTQ-0003RM-3z; Fri, 21 Jul 2023 11:55:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Ttl1E6ptYzqju6Jetua3CqswqmmOTyheJ1HRao74IT8=; b=F1jOrrqkw4i7 dZr2tHQ6SBjl+hewiPuasfLSoAFOs0zzybPRLDkaIFkSbP07ps+iRV8wlow5Stk4IRcsDSX3hkvuM dbYsSoaE4Irul5skTwBxspZlvt3P+HakrhKYSSv/yXSHpdqAuT66d1S/iwtdg7R/k/DxGph38s6gv dVVQpuJ++bEvw/Pk4PcWGwc1eBLxMUrA7YfT4fb5HeTzh4fwU0aDpgPko0JN4860TXaDqSN5LnSyb Msf9sePSN2qjQHGAiNOgn1Gnmzx8iffqd4UbeXxoyV0vkakiknwi4V7IBJrW7z4AoyRZiijZo8AHC vz4CDwoG0/xCyvDbaNsf1Q==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMsTO-0000WQ-1H; Fri, 21 Jul 2023 11:55:18 -0400 Date: Fri, 21 Jul 2023 18:55:54 +0300 Message-Id: <83zg3p9s39.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> In-Reply-To: <87zg3pb6yt.fsf@HIDDEN> (message from Michael Albinus on Fri, 21 Jul 2023 17:49:14 +0200) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> <87zg3pb6yt.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Michael Albinus <michael.albinus@HIDDEN> > Cc: Eli Zaretskii <eliz@HIDDEN>, dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, > sbaugh@HIDDEN > Date: Fri, 21 Jul 2023 17:49:14 +0200 > > Ihor Radchenko <yantar92@HIDDEN> writes: > > Hi Ihor, > > >> My point was to show the differences in the approaches. Do you have also > >> numbers using without-remote-files and inhibit-remote-files? > > > > (length (directory-files-recursively "~/Git" "")) > > ;; => 113628 > > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) > > ;; => (1.597328425 1 0.47237324699997885) > > (benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-files-recursively "~/Git" ""))) > > ;; => (1.0012111910000001 1 0.4860752540000135) > > (benchmark-run-compiled 1 (without-remote-files (directory-files-recursively "~/Git" ""))) > > ;; => (1.147276594 1 0.48820330999998873) > > (inhibit-remote-files) > > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) > > ;; => (1.054041615 1 0.4141427399999884) > > Thanks a lot! These figures show, that both without-remote-files and > inhibit-remote-files are useful. Of course this shouldn't stop us to > find further approaches for performance optimizations. > > I'll wait for some days whether there's opposition, before installing > them in master. Can you spell out what you intend to install? The figures provided in this thread indicate speedups that are modest at best, so I'm not sure they justify measures which could cause problems (if that indeed could happen).
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 15:49:31 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 11:49:31 2023 Received: from localhost ([127.0.0.1]:34580 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMsNm-0005pa-LX for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:49:31 -0400 Received: from mout.gmx.net ([212.227.15.18]:35533) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMsNj-0005pJ-6K for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:49:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689954556; x=1690559356; i=michael.albinus@HIDDEN; bh=3AIB1AI5v0K18NoKohqFeeXgcYtFMKryUIGZNx4Yhb8=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=opLyPz0B+cddl+P74OhcmKqN7Pm34hutYbGSbnEiPDPGYHdHzlqIN33ggI0b0ep0NRkBHuG 37xaf13cqTlBZU+cRRiG0vOWtTiZAUw4yvI+7lLHWJsZOGSE1bI4VMhCRAHHseFmkOYJTv2va zODjkYzD3EVSnltfyr1nRQbxmKwlgSne11ZnTnp7x4cQy5SVtLtPCsQHw6mQr+E1GvPB4jtFO XLRj5w6DupQe69AH5XrKs+ElsLJjKb8x4N8MT9NbWJnKzTVCDGnSGJkKG7XPmrOA0mVLV8Pzy cOpWbfiWD+wPmfSRMODXCMEaN6Yp60rVcKIwBe/OC8EWFNS9q3oQ== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1N3siA-1pwNER42b2-00zoGZ; Fri, 21 Jul 2023 17:49:16 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <878rb9l1f5.fsf@localhost> (Ihor Radchenko's message of "Fri, 21 Jul 2023 15:38:54 +0000") References: <iermszrwqj4.fsf@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> <878rb9l1f5.fsf@localhost> Date: Fri, 21 Jul 2023 17:49:14 +0200 Message-ID: <87zg3pb6yt.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:wJXoA08r1VTJ7kNDyJfoKYsmjrzJZOuFNzkJ5fQs9BGi1Z+Bo7n PIVd30+sLN5XYn0J5stFgKbHWI3o81xqSktWZ6Z4K34MJ5C/JNSi7lnvRbcocclyJw1fXqo xIlzkO1T/0hb5ynvyO8bmU4UzwfqusDjG1glLJ6TeuqP4oI5Na0/EnkrSRHcZU4myHJmXI5 V81c4N5d9p5V4vHB/SEHA== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:2DZ0apATdA8=;X4qjKf6dko7WBESta2Pn5vYMfKG bXRmByMLciOAKm3zU52qyg1ov2wNGhSgQDuX5nz6eOuHE2mbBjjwJ1CPqBLB+lRRoiseDUHXg REdH9s2dnxUN5fBDsPOa5xHGbGQ4YDQ0Iu02h8shwQ0O4+ndS5ZAwfRAeOpz/BUS2xtkPGDHA OnPfveiFbKcsWHA8zI8tH9znfD0U2xU6z1QyAf6OC9MbVzKy+ZxbCm5Dmv+FZPebl8TpLa6mq CKgrJWFCwMKwxMJXq7n+naUupWWTVMa2QpYuz5WkF+rlvW6KBhCGxm29Rl3Il5/xFuWUbTg1d DUuGO0pxt/YK4bURKXb6z3xLPqiyIB9R+3DL9fPJ3+yCkwx7MWF4MvrRkbwDO31MtlBykAOyV vvDw9qiOcSPLyQlfjZX2nUpa4lIt/1OCaKylbIlpcFXkf58p/yygc6eBTVyY7I8w1UlSk4nF4 MGHSBUkI7mfMRF1Ktf8LxP+CWZo3iabl+XkvsYsQEGQeXsdgs/QSZCRO1SfXBkc5XIBpDjGBW 9kKh2COFWjNdIwSVbIkljM8TQZOdrYT/fFe+YC4xjm8WkXwSfsSDY6G/89vefNedn66I/H6vz oYb4NWj+xih8rS1ti6m0OieK9r/4a7BUlGLNotU6dSqgGRFXe8TG4d5kt047MtZJQ3N0xYLzN ymhCBSzostt3567qwKTcG/XLGDK/8Q7CBXotM4q1s7oXG2HvhPQstzBFQnZ8Ia0koFW85CRpv AgTQi9wc3a9InZ/XT5QtXgjElcTnLTecCGmOUmt1d65W9GApRX79PTWyZGRS7jJvzZhF4xsy+ 2XoJpum9FnZdO23Z+hZ4MpG5i74qww+vNexxOJRC7iwa6P6SP4Dg+XG1TfC3Hv9nxmOjlM4Wy L5KJZUpIt5nAE4c8p3yCH63JrhuzXKEIzq1yVel6uAJyixQybc+cfpHc9xm41pwpJH9N167/C R4ENrg== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Ihor Radchenko <yantar92@HIDDEN> writes: Hi Ihor, >> My point was to show the differences in the approaches. Do you have also >> numbers using without-remote-files and inhibit-remote-files? > > (length (directory-files-recursively "~/Git" "")) > ;; => 113628 > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) > ;; => (1.597328425 1 0.47237324699997885) > (benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-files-recursively "~/Git" ""))) > ;; => (1.0012111910000001 1 0.4860752540000135) > (benchmark-run-compiled 1 (without-remote-files (directory-files-recursively "~/Git" ""))) > ;; => (1.147276594 1 0.48820330999998873) > (inhibit-remote-files) > (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) > ;; => (1.054041615 1 0.4141427399999884) Thanks a lot! These figures show, that both without-remote-files and inhibit-remote-files are useful. Of course this shouldn't stop us to find further approaches for performance optimizations. I'll wait for some days whether there's opposition, before installing them in master. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 15:48:48 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 11:48:48 2023 Received: from localhost ([127.0.0.1]:34576 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMsN6-0005o6-71 for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:48:48 -0400 Received: from mout01.posteo.de ([185.67.36.65]:51345) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMsN3-0005ns-Iu for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:48:46 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 09815240029 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 17:48:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689954520; bh=14Yz82ehhieUwDUN96d87bZ2yhNl3xXvPVhzN9KPkOw=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=Fy8eqpy2kZCq6EA5XtxRnfC59zYNa9aSENoLSHR6j1Hp8DWP4k0wWXgo3shRSRZ58 bA02MgpbREzzmk+wTm0RgeKcz+hkEManwOzrbFbPHwFCZ+Px5X1dU8fy/Ht2xaJkFK yVka5gAnIGttH4uiXYDJWizX+uFIk6ATUErrYVimN0EFD+THeOPmNzbwTnOpp/TMUZ sAYavke7G5Nc4uJlR+ImHsAEyT++SI8ZiPvU3X0u4FYuLIVBKvLkQ2b1C/BY+SGt1U ZgPyhGxut7Je2Oncoti06zeAwmeOHmvkgkC6Inwn68EkYnsvlWZRgdx8xH2eMODRrX +2q7eDx3iLkXQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6v9M3t1pz9rxV; Fri, 21 Jul 2023 17:48:39 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <ea2332cd-ec57-794d-d662-ff522a4dd404@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> <ea2332cd-ec57-794d-d662-ff522a4dd404@HIDDEN> Date: Fri, 21 Jul 2023 15:48:52 +0000 Message-ID: <87351hl0yj.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Dmitry Gutov <dmitry@HIDDEN> writes: >> Locally, AFAIU, running `find-lisp-find-files' without >> `file-name-handler-alist' is equivalent to running GNU find. >> (That was a reply to Eli's message that we cannot disable >> `file-name-handler-alist') > > But it's slower! At least 2x, even with file handlers disabled. > According to your own measurements, with a modern SSD (not to mention > all of our users with spinning media). Yes, but (1) there is room for optimization; (2) I have a hope that we can implement better "ignores" when using `find-lisp-find-files', thus eventually outperforming GNU find (when used with large number of ignores). -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 15:44:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 11:44:28 2023
Received: from localhost ([127.0.0.1]:34572 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qMsIu-0005bk-9p
for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:44:28 -0400
Received: from mout01.posteo.de ([185.67.36.65]:56355)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qMsIn-0005bK-1r
for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:44:26 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout01.posteo.de (Postfix) with ESMTPS id 235C3240027
for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 17:44:15 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1689954255; bh=hnsj4PIJpZgWLlL5oi0st/WosZwvA/G5bZBEWGeEk5E=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=Rt8Vd1xAQR3KvG2efiJDseOumolWV65Gz4KjCBfZOqwcVXyfj+wQ6HflnioSawUDu
KXS7LMjpZ8gE4dx+31PNfKRNzLBmQjucP+d2eeYKezUwhFwKLKadsFu0cUefeyZP8o
wLYhyg0k/Ye11vgTX7QKVkuO4K7LMpdZQ1GyQn8sCAU02WBljOArpe5wPni/wHlGZG
RWg1e6ybVWjJeSc+y04kgEeZyKPu0eN8cysCOBvXHuQ+sNsnIK6Dix2C/kWlwjPAyx
8hpM6USuNTQRs88mv8gNQY2+IJ7d7e3arCJoIaKZ9vXXTWzL2VHQ5S+qj6VHIbkUvK
6GWSI5owA2WUA==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R6v4G2wYhz6tvm;
Fri, 21 Jul 2023 17:44:14 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Michael Albinus <michael.albinus@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <875y6dcm4r.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN>
<87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN>
<87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN>
<87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN>
<87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN>
<87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN>
<87fs5hmp6i.fsf@localhost> <87zg3pcuvv.fsf@HIDDEN>
<87a5vpmopa.fsf@localhost> <87mszpct5n.fsf@HIDDEN>
<87mszpl7n8.fsf@localhost> <875y6dcm4r.fsf@HIDDEN>
Date: Fri, 21 Jul 2023 15:44:27 +0000
Message-ID: <875y6dl15w.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org,
sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Michael Albinus <michael.albinus@HIDDEN> writes:
>> I am sorry, but I still do not understand how what I proposed can lead
>> to any feature regression. May you please elaborate?
>
> When you invoke a file name handler based on the value of a variable
> like tramp-mode, either all file operations are enabled, or all are
> disabled.
>
> The mechanism with inhibit-file-name-{handlers,operation} allows you to
> determine more fine-grained, which operation is allowed, and which is
> suppressed.
I did not mean to remove the existing mechanisms.
Just wanted to allow additional check _before_ matching filename with a
regexp. (And I demonstrated that such a check is generally faster
compared to invoking regexp search)
Also, note that `inhibit-file-name-handlers' could then be implemented
without a need to match every single handler against
`inhibit-file-name-handlers' list. Emacs could instead have
handler-enabled-p flag that can be trivially let-bound. Checking a flag
is much faster compared to (memq handler inhibit-file-name-handlers).
Of course, the existing mechanism should be left for backward
compatibility.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 15:41:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 11:41:42 2023 Received: from localhost ([127.0.0.1]:34556 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMsGE-0005WS-FI for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:41:42 -0400 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:33213) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMsGC-0005W8-C8 for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:41:41 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id C67523200907; Fri, 21 Jul 2023 11:41:33 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Fri, 21 Jul 2023 11:41:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1689954093; x=1690040493; bh=jFONDpMu3wNqG5wY5OAv/HyJvLf/5+pq9GU fg5Ufzmc=; b=Xdkl0HO15qiSXZb5smXY9MoFhAx49VszbA9RkxTOGOtPrZ1/NF2 eWHYS34uv32DjslXlEiph1ojITbh9tvja8GCoppTlFAuWrFdtdGa71HAP+xcB6Zz 6zwWBQdQLFAmRzSyf09CYyO9dNdEQs8AYfYnvwVnqfP4PfPlOEulggZcjAt5Nc+a 6Tulte7+gEVBHqBbBFpQxBmfpvxEeuNZR1Ti6xlXUly9HOs6t+a4h+g1Cy/UWgLd AaZyqR8bdgaXNhNhvq7cPvkIIiMQXDZfNrc/CFJu3va8+tGU4v8PBxBndAm+lE8y u2AUJgSvSK69QX9q5nI2ZzwIHXRxvAHTtRw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689954093; x=1690040493; bh=jFONDpMu3wNqG5wY5OAv/HyJvLf/5+pq9GU fg5Ufzmc=; b=wAxetCVuQqqFPJfKlWV8TwXsWaGthLUtgGZtSVuQ44RIUJyyX3t fwFEYsTXsDelmMN2EyrQnHmE3j3U8/ZpZ5cORv/1uxnb/8bpVxqoYBC5UjWPQ9RF ru5+1CcxuY4C38ukUetxeBSH6+ph0fyfyhu/tsdC7aUmeS1Vz41KHzB3vQO8qn1S BABhcdTM4Gp4CMZJ1YnAIpe1Kr4lKE1SSNUT/VkSdqzPqrbxtkcNloDWdUwKId2N WjMj2IpcW56QZQJKllf/vX04TjGqJwhlJ3bkjgPGVvyBrC6/pjsCrm4J1nIsVY3S v3ytE8Y+Y5LUyAo8+gF1XEUrXUsS0GRmJ4Q== X-ME-Sender: <xms:LKe6ZFqt0gNDUxOXHRwK-E22bxdpcwj-eAc2kc02QMnGMg-nrLpuyQ> <xme:LKe6ZHrHYAvgLR5r6MEXOlp6FYSBeom-dkmRNcTPbhq2NnH2z6HtfeuzXZdD5XwAC qH2xWZaTJyIAeHqQok> X-ME-Received: <xmr:LKe6ZCPjRHM3tJqyrlQZ6w7DYeyqAqwU88NJibEKMpcNbw6Bqc8R38tzvU8CeJk> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgdekiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:Lae6ZA6UjfugklP0kBPqQ_i0Bg-KaPzdN7rEw1QKHP7c6E3NL_WHCg> <xmx:Lae6ZE5ieCsAJXfj2tcw9i9HlWM1JHglNAHXS35fq3462u-SOmNA2g> <xmx:Lae6ZIiLYm85z3TqklWmL9ciQiWDLvOlmYGJdla_sSNA7wgZV13Pjw> <xmx:Lae6ZEErpMMIcAbxsMpoMRfFZhAZkk8qPCMHIm5oOAfceSGj28N8AQ> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 21 Jul 2023 11:41:31 -0400 (EDT) Message-ID: <ea2332cd-ec57-794d-d662-ff522a4dd404@HIDDEN> Date: Fri, 21 Jul 2023 18:41:29 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> <87h6pxl6m4.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87h6pxl6m4.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 21/07/2023 16:46, Ihor Radchenko wrote: > Dmitry Gutov <dmitry@HIDDEN> writes: > >> On 21/07/2023 16:34, Ihor Radchenko wrote: >>> Dmitry Gutov<dmitry@HIDDEN> writes: >>> >>>> On 21/07/2023 15:58, Ihor Radchenko wrote: >>>>> Dmitry Gutov<dmitry@HIDDEN> writes: >>>>> >>>>>>>> Disabling file-handlers is inconceivable in Emacs. >>>>>>> Indeed. But we are talking about Emacs find vs. GNU find here. >>>>>>> In the scenarios where GNU find can be used, it is also safe to disable >>>>>>> file handlers, AFAIU. >>> So, we agree here? (I've read your reply as counter-argument to mine.) >>> >> >> We don't, IIUC. >> >> To use GNU find on a remote host, you need to have the file handlers >> enabled. > > Let me clarify then. > I was exploring the possibility to replace GNU find with > `find-lisp-find-files'. > > Locally, AFAIU, running `find-lisp-find-files' without > `file-name-handler-alist' is equivalent to running GNU find. > (That was a reply to Eli's message that we cannot disable > `file-name-handler-alist') But it's slower! At least 2x, even with file handlers disabled. According to your own measurements, with a modern SSD (not to mention all of our users with spinning media). > Although, it will likely not > be usable then - running GNU find on remote host is going to be > unbeatable compared to repetitive TRAMP queries for file listing. That's right.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 15:38:54 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 11:38:54 2023 Received: from localhost ([127.0.0.1]:34537 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMsDW-0005QZ-C4 for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:38:54 -0400 Received: from mout01.posteo.de ([185.67.36.65]:51809) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMsDT-0005QG-Ot for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:38:53 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id B31D7240029 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 17:38:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689953925; bh=lg6z0Re0P0faPg21zfW+vKSpJmokcY7HxP4cXbaEXD4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=IXORiBCpjHzEsqvQJ+QUKdGtMFvE1avn6WOb+iRAzeXrZ2oGWt6eIuWCaR5bmhKxV z5Q8JVFkRwV4C889/Xm3zuIOwDO18q/F0PHogcYpAmeYRJn4H83jlEVqsGm2tYcjG7 WsMqoyu2rSXBGEhLOcyil7tjS+KnKQLe5nCC2v9ePYVbiK78Joci9xZPHhMjtlqX82 ax2/ypuQmbgvU81wrZx1BkTRurpkflPsjdpZdgCqe9zmhdBM5oS9j6E+pv0LpQ0FBV 6Se/IrzitJyTy1fzPxw2+LCWMGlV6Gfq6waBUhPUbmLdjtUq3JlJ7t56JwlhJLzU4v J1/T2tvMiQnHA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6txw2W9Lz9rxB; Fri, 21 Jul 2023 17:38:44 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87a5vpcmc7.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost> <87a5vpcmc7.fsf@HIDDEN> Date: Fri, 21 Jul 2023 15:38:54 +0000 Message-ID: <878rb9l1f5.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Michael Albinus <michael.albinus@HIDDEN> writes: >> Would be interesting to see profiler and perf data for more detailed >> breakdown where those 30+ seconds where spent in. > > I have no SSD. And maybe some of the files are NFS-mounted. That's why I asked about profile data (on emacs). > My point was to show the differences in the approaches. Do you have also > numbers using without-remote-files and inhibit-remote-files? (length (directory-files-recursively "~/Git" "")) ;; => 113628 (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) ;; => (1.597328425 1 0.47237324699997885) (benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-files-recursively "~/Git" ""))) ;; => (1.0012111910000001 1 0.4860752540000135) (benchmark-run-compiled 1 (without-remote-files (directory-files-recursively "~/Git" ""))) ;; => (1.147276594 1 0.48820330999998873) (inhibit-remote-files) (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) ;; => (1.054041615 1 0.4141427399999884) -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 15:36:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 11:36:33 2023
Received: from localhost ([127.0.0.1]:34530 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qMsBF-0005MO-AY
for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:36:33 -0400
Received: from mout.gmx.net ([212.227.15.19]:51077)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <michael.albinus@HIDDEN>) id 1qMsBE-0005M8-1A
for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:36:32 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de;
s=s31663417; t=1689953781; x=1690558581; i=michael.albinus@HIDDEN;
bh=o9fGz98QEezYBH21+4TMGkfUCAG0F70cZDiUPEa+Ba0=;
h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date;
b=PSd1TIvHCAvG4JIIcM7saVDMNad9bvGtQcsICdunzGYH7Z4wOmEPpO4nA/g+1iplh6NEDEc
edT4N5zWrOJDgQSFVeo6i+jCI+7yZr6QHY4SRlnuSpfywCDknO09zp84kzDxJ3xQ3PcSHt0FB
d7BCfgNbjsowhiX7TfPtkj098GiVXHFJ8KyqH2T2JbF8dZTO2Q6g11bYWTY3robeQgBwCghwd
KVkNLKcRC982OY9jjRCf+5HJXqbddnQnrPSKRJ8RoSzIToAcp27LVjRR5uKritveJ6L0sao5V
yzfSOs7/AZkmvIrWauCfgAdD5BYUm2LJuvRrscSFHmi89kde4Xqg==
X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a
Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx005
[212.227.17.190]) with ESMTPSA (Nemesis) id 1MhU5R-1pisek1uRn-00ecBB; Fri, 21
Jul 2023 17:36:21 +0200
From: Michael Albinus <michael.albinus@HIDDEN>
To: Ihor Radchenko <yantar92@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <87mszpl7n8.fsf@localhost> (Ihor Radchenko's message of "Fri, 21
Jul 2023 13:24:27 +0000")
References: <iermszrwqj4.fsf@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN>
<87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN>
<87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN>
<87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN>
<87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN>
<874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost>
<87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost>
<87zg3pcuvv.fsf@HIDDEN> <87a5vpmopa.fsf@localhost>
<87mszpct5n.fsf@HIDDEN> <87mszpl7n8.fsf@localhost>
Date: Fri, 21 Jul 2023 17:36:20 +0200
Message-ID: <875y6dcm4r.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K1:k8yMWVKDr9rWQCVe1syGQeKE3vR4NKii5hpFKwa3568XolouOwB
16yC7KYw+BxXAVnPt0AT6ZnM0PhH0kDFo8w9WaXj2f8qwuUt2lZ6+3L48gmLGy+0lR33RHo
C1RgdAnRr8SGMDWUA/v2F5/0ItbwG5UlWai582mhuS9gTqBXOq0D+YbT2lKlDoEcW8PqITj
C+Drmo0hr9Kr40i77sCOg==
X-Spam-Flag: NO
UI-OutboundReport: notjunk:1;M01:P0:gAn+pdGIjnM=;5ZQCkpyKGhnuQmzhetdjlv6MPbq
ERC10c4TWrF+Vey870310W/ViayUFOLPL3Xmzn5CAJyy8bqoE44voqPimQHcQ4B1cNdicGyah
nZHpBzWozi4BBfeTVghP8qE2cXELPPweSRIRhg5i9Ca8Iv3DvvSDjWnsq8/W88nhdpWVdy30h
yq6QWjhQHrI/ngD8KS5p12LgOIL0I2y9MKbUdliWx5Rc32hPnaWzxPmFCA2cKiTO9azJftWoR
fqZbFb9wSXYbSjSVVHaanHZNMgFCecJ6n61J1kd5mLWQX1gQymYtOe/Coa2n6kTad9MAwyfv7
gZDgiS34wf2tnOMKDfo/88lHPuE4tBpM3rd1ika8QX4+1Dh3HzQqmUWv8+V+b8SvAJxZzdtWX
YZ5/oPX3Q64oGPrZCA0/J2sX35qA2BxliqiF5K+sWp0v1cjX9XOWLriH9kM9Ns7KUBAEJhoPM
epXGMck217i42t17H+Yb4VMa9UxjhSwe2Kkhn2srsNI1Z56N8eYBCllXRIdnHqIf0bOYEj64a
WiT+rzz2jBw4Pqzs2uMCrPus2SwlpdiV4094+I/c0+D1njdsI2rpPR1N8xXeQfe1bc/ZrCUR5
IBGWYHVMw9fcmCZfzHavzK8ZqTlDr6PuB6GhxcGZhZOCObLIeQgn+oYG76g+1kW1Z20FhDC4O
cpjZp1vwHLm6JS6vtPZIb7DI3XIlnTmw9GC7aRa747jVs9ceCSJDj1Kx36Ue6/UmRJWpp6l3r
MrDz3YPoxDhXDRCdsUa+8LJSrXv2O8aIO4wBxZ6hPPVnOmyzNxQs3hr8XfxbmFJY8OgZuQe4r
gJbxJwjo5gMgYRln5oW16CA2EYhiDPR17uZdnWSkkErNa6s6MHXwaaJf2FbQEnxDUsdqUXvVw
h3RhA1t9xM4Uib6fumEpjNfwHIYSDxTwKYn1bxTjBnEM6bQd8oQs75xC1PekD6MfAYzv6gyAV
IBO4ig==
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org,
sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
Ihor Radchenko <yantar92@HIDDEN> writes:
Hi Ihor,
>>> And what do you mean by destroy?
>>
>> "Destroy the feature".
>
> I am sorry, but I still do not understand how what I proposed can lead
> to any feature regression. May you please elaborate?
When you invoke a file name handler based on the value of a variable
like tramp-mode, either all file operations are enabled, or all are
disabled.
The mechanism with inhibit-file-name-{handlers,operation} allows you to
determine more fine-grained, which operation is allowed, and which is
suppressed.
Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 15:32:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 11:32:12 2023
Received: from localhost ([127.0.0.1]:34517 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qMs71-0005EI-Io
for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:32:11 -0400
Received: from mout.gmx.net ([212.227.15.19]:57139)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <michael.albinus@HIDDEN>) id 1qMs6w-0005Di-83
for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 11:32:10 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de;
s=s31663417; t=1689953514; x=1690558314; i=michael.albinus@HIDDEN;
bh=BJL3GZoLlk7Mf/rlVLiS7h4DTwp2PrNHLXaMF7ez0yA=;
h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date;
b=RCpO5W4kSjGtARaHskkK33RBsQIwbyiEwUoDxLPXfopVdbsJRxuBOCDeNGGXwB8GZdyjZBi
1PS/DEhSBT3GNfXUI9G5SUMiyRG7SbTccyGw2Mj6xLnDpX599k4mEULoWPU78g3urM4NoOged
kgnmal9YTNTMR9eGpjQ7YugrsYBPMRAxB4wJgkLaHiu4jIEzzHCJIDF47mFkz3ognPsgrNLsx
oOiUlJg79xso6DNLYcwAMyWlWSXehjaTgsoHbHY4+taqjCsWBtp6JFYau9Yx0s1mAoGVpy6PV
NFWY1WpkhWO/GabQWoMixuutlijl6rKlohoKdxC1o8bHv0bv4ZsQ==
X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a
Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx005
[212.227.17.190]) with ESMTPSA (Nemesis) id 1MKbkM-1qdRYM08Tc-00L13q; Fri, 21
Jul 2023 17:31:54 +0200
From: Michael Albinus <michael.albinus@HIDDEN>
To: Ihor Radchenko <yantar92@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <87pm4ll7ox.fsf@localhost> (Ihor Radchenko's message of "Fri, 21
Jul 2023 13:23:26 +0000")
References: <iermszrwqj4.fsf@HIDDEN> <87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN>
<87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN>
<87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN>
<87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN>
<87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN>
<874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost>
<87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost>
<87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN>
<87r0p1cta3.fsf@HIDDEN> <87pm4ll7ox.fsf@localhost>
Date: Fri, 21 Jul 2023 17:31:52 +0200
Message-ID: <87a5vpcmc7.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K1:d5FVuHYZ2I1Gyj98I0uYKnMThYGOfbBDhhDm4aXBx2e9w+FDIge
BeoiAuW8U5KEeAmW2wRVPZpOHopDBPSFT8tIv/KYSwmlS6NI2BZlauvn6ehIdEDmOp330TY
g2SrZ3vfpLbeg5m254pVZ+tbUhUFG/uLSrnzWAFkALNQyH2bSwTqybGLZx5c+UzZr5ZwGeZ
pe6kNpaAId5j3NJIJ36EQ==
X-Spam-Flag: NO
UI-OutboundReport: notjunk:1;M01:P0:xtZmWgotrKw=;7sfwBaWP6qn+2b+R7a2QRpxbp8k
FxUakwmyVTV3D/cM4CiB7nTMz6EkOzQ9w8Zrr5NmNBa7JbSMZ9OF+jzutf74PyhBX+CMLMxRE
LbBrHoA5hom7wUeanSmp7XQ3SUT7T/WZ2ccAhI9kAnF/hcGRU/W/pNw/CORLxzbC03aQ5mUtw
HrgQ32Lb5Bf/LTJT9xfqNgNGLqfGvwSodv8jz4EY4sbL7AcowOlblyu62tLHwfBx7tt6wIT72
s1gkhPKkEoGZgCI0JgIOy4mshTa7mHsIZoeXDEQhGdjJaAy1HKl730yTNl4srMG5PlEsDGpMR
u03eb2rLCCRruQuJjRmX0ibfAc+YPwrLGCUJqxaX04OWCuem0OKdtYusGsTDcWNXG9Be1mrq9
DkWv7WI1xTjd7COgZaU2Kc+zZZJwue+Xm7xnDvlWryCtz8tZMfE90PcIWFTbQtauuHy4kjOiA
PYZuOGAE2JKyXO5R09uIGNLr6/B1b5bhL9/QyRn+lAWO27udArNfXlQb7sxjfRRpFXmi0+Xhr
vfyrpUj2WXX1fzgmmIOuSVWG76wtaSQ6DVpYOf+bdKQZMGZfTFOBMXVcrXrG1wachTZJ+48VQ
qR71cvOJjjk647FZy5y8nBPKmyX7ML9xhS0EIJsl/tVdxz49+DHKQaDzkn2L1JGPKx6rmFoJs
jNVFC0FUdYtYjkIgghEKPlP/e78V0KrAQZwPSdGdMD3twp83i9WSuoYIrqtocXlXEklqE9+ix
pp7bA31L786BLfiUkAMWcl5D48Ru7ysgoGscy/Y6lW21FbHGpOeu/GlwOXGyLQ94juDouGcE1
ZZxzgQJecftlnutF8FdpjZLu9hW3icUL6Qqs1vasa/nnm6NMRtlCj8N9PyAkiboc9L6KG53CD
C27XcEpQLNAgGbKAtt1ZOTdvZ+kXLb66ebMuXZdofL2KIumPBdwbR5T0FJgw9KeDlTNVjT0+n
wJjbcp5/MrGmYpOVe9D6MusA9fM=
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org,
sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
Ihor Radchenko <yantar92@HIDDEN> writes:
Hi Ihor,
> Interesting. Apparently my SSD is skewing the benchmark data on IO:
>
> (length (directory-files-recursively "~/Git" ""))
> ;; =3D> 113628
>
> (benchmark-run-compiled 1 (directory-files-recursively "~/Git" ""))
> ;; =3D> (1.756453226 2 0.7181273930000032)
> (benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-file=
s-recursively "~/Git" "")))
> ;; =3D> (1.202790778 2 0.7401775709999896)
>
> Would be interesting to see profiler and perf data for more detailed
> breakdown where those 30+ seconds where spent in.
I have no SSD. And maybe some of the files are NFS-mounted.
=2D-8<---------------cut here---------------start------------->8---
[albinus@gandalf emacs]$ sudo lshw -class disk
*-disk
description: ATA Disk
product: SK hynix SC311 S
size: 476GiB (512GB)
=2D-8<---------------cut here---------------end--------------->8---
My point was to show the differences in the approaches. Do you have also
numbers using without-remote-files and inhibit-remote-files?
Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:46:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:46:42 2023 Received: from localhost ([127.0.0.1]:60960 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMqSv-0001yJ-PB for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:46:42 -0400 Received: from mout02.posteo.de ([185.67.36.66]:55647) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMqSs-0001we-DI for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:46:40 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 644D4240104 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 15:46:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689947192; bh=kreoqqOdrt4S+qGU5+IDB8131y5kmZ1PQiW6teC/qHs=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=WgT5TMlk4y43lU414cTDOCKcOoyxwO+bLJGvRPjTvZ4JVCQyDDSvrJljYpVk3Qw2Y H1j6Q53b4QLOsFCwBxebJr9XxRTXRVuz7guSczAN240GvXxOw3f0sNw9DWAK6ZSGwZ hsoJG5PfmHoDtwXnAdTmn/6UNER6DSQpt/Q+2NBzULrsk6BdjuNbLNXqX5PhBpl5iT ydXXxriBL/8NvdhtIBeCnkZPGHLaIsXxU5PjxEvtWHnqalxASYLz8XoexWY1kKCdt7 PAyGC3+t5Luz8HnbRgEtQ7/LLpY3uLhjLFdVMPXlrunUi8gWgCC5JXTXaMLBLsgSz4 h+oArHvUw0hgA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6rSR5ZkYz6tvt; Fri, 21 Jul 2023 15:46:31 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> Date: Fri, 21 Jul 2023 13:46:43 +0000 Message-ID: <87h6pxl6m4.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Dmitry Gutov <dmitry@HIDDEN> writes: > On 21/07/2023 16:34, Ihor Radchenko wrote: >> Dmitry Gutov<dmitry@HIDDEN> writes: >> >>> On 21/07/2023 15:58, Ihor Radchenko wrote: >>>> Dmitry Gutov<dmitry@HIDDEN> writes: >>>> >>>>>>> Disabling file-handlers is inconceivable in Emacs. >>>>>> Indeed. But we are talking about Emacs find vs. GNU find here. >>>>>> In the scenarios where GNU find can be used, it is also safe to disable >>>>>> file handlers, AFAIU. >> So, we agree here? (I've read your reply as counter-argument to mine.) >> > > We don't, IIUC. > > To use GNU find on a remote host, you need to have the file handlers > enabled. Let me clarify then. I was exploring the possibility to replace GNU find with `find-lisp-find-files'. Locally, AFAIU, running `find-lisp-find-files' without `file-name-handler-alist' is equivalent to running GNU find. (That was a reply to Eli's message that we cannot disable `file-name-handler-alist') On remote host, I can see that `find-lisp-find-files' must use tramp entries in `file-name-handler-alist'. Although, it will likely not be usable then - running GNU find on remote host is going to be unbeatable compared to repetitive TRAMP queries for file listing. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:37:11 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:37:11 2023 Received: from localhost ([127.0.0.1]:60945 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMqJi-0001de-WA for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:37:11 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:39547) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMqJg-0001dM-6c for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:37:08 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id B0BB8320095C; Fri, 21 Jul 2023 09:37:01 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Fri, 21 Jul 2023 09:37:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1689946621; x=1690033021; bh=c3L5A39AlpayP5p7EpvNHs9rY60AJ9U5PcQ FhoWhpk0=; b=OwcwNql1STncKOtYCb0UavEu3LiKeW82Gvr+VhIIhBIeHrQ/Eam aGYj2GUFK/ozLm/OxggFngcCsWIT7pqOwNBvGaJyy5xmUQkDiG7TfF84+J+gtPv8 bmpR+guSj7oBSbgfJFu10t3KnItujO345eRX5diAFUzwUMIWyUOIKqHdgUpt5L1T 6qKm+ketAY/kye7NdDnFx5FZEkg2EHWMf8WVMCWjn6QLA+pI2d2NUjGga4EDWywW shld4eNUidPbJ3EFU9iK7qcx8YKPFVW61n937zhuou8dkmC/0jmKJYWTcPibWysz oRh8RV5VfgNf9XHZ4MAm3RHyBDAntiouT9w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689946621; x=1690033021; bh=c3L5A39AlpayP5p7EpvNHs9rY60AJ9U5PcQ FhoWhpk0=; b=jgk6N9YqJI7CcZ28CKF7HWDuCcVWGzuBRjFw1jMRtzEYbnqkZAE A0tPt53EvO6SIXdhiou4v8Y3sPLXJdFLrdZegzks3SnpeY94wxPqNr1xVHAihzqM McHE9KQN0kCI80S408mvNLTG4DKmrjTT0nLnWrtpLQjJb4gwV3jqC4eDbZmHCm5Y WbXVJXlBLHPt1buzVrKw8wJUZWHas3PuuwRyN5YEsJjeNKbqizj6lO0q8OKeLV56 h5vPf2VlRcM4tZiJ4aInxyLQxO6J1kUl1jzAkiWe8WBoY60MbucVtiE0W41KKOCJ dR9eppNsk4iWn68wCFWvWGBAlR3bnYTcAbg== X-ME-Sender: <xms:_Im6ZHijxcJWP_goHK_jAq2GZBFVf-xrVagMzhGtV_EJURHyHhDTww> <xme:_Im6ZECGmonvr2NxhZVQwPNuVPBFZFvqvdDv3Abjl0u2_KYUvWk6GcV8tjqm4cfv9 H_IJeTBELH6eLptU_o> X-ME-Received: <xmr:_Im6ZHEByaV4FQgU_0_gOndQYvGWY7w6NK1gqOYpQRpvhBzOCH0h1FOn4zRQz_8> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgdeifecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:_Im6ZESP8MS2f9ZBPAXr7A8EY6sxJ_Nxx6iJb_t17Ic4oKBPfML3SQ> <xmx:_Im6ZEzcF0Ik1fzlljSUVcFOcN0M2poIthBPwQUWe3EdWRRUtta8vA> <xmx:_Im6ZK6wCxNRxm6t8FobkPdZLrHG-dahgXWKx_D-N8-gVulpS4CHtQ> <xmx:_Ym6ZP-mrR7p_-YACgFZEA4ExaRC7LuANjJ5Jo3N4Q6Gcf3K6oKErw> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 21 Jul 2023 09:36:59 -0400 (EDT) Message-ID: <b725f1c7-58bd-8538-39be-5da01a055424@HIDDEN> Date: Fri, 21 Jul 2023 16:36:57 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> <87jzutl76e.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87jzutl76e.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 21/07/2023 16:34, Ihor Radchenko wrote: > Dmitry Gutov<dmitry@HIDDEN> writes: > >> On 21/07/2023 15:58, Ihor Radchenko wrote: >>> Dmitry Gutov<dmitry@HIDDEN> writes: >>> >>>>>> Disabling file-handlers is inconceivable in Emacs. >>>>> Indeed. But we are talking about Emacs find vs. GNU find here. >>>>> In the scenarios where GNU find can be used, it is also safe to disable >>>>> file handlers, AFAIU. > So, we agree here? (I've read your reply as counter-argument to mine.) > We don't, IIUC. To use GNU find on a remote host, you need to have the file handlers enabled.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:34:35 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:34:34 2023 Received: from localhost ([127.0.0.1]:60937 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMqHC-0001ZG-4o for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:34:34 -0400 Received: from mout01.posteo.de ([185.67.36.65]:34515) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMqH6-0001Yy-1D for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:34:32 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 08C92240028 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 15:34:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689946462; bh=X97yuJL+rnGQxdHuTpyMndNoPTCbWbn9FJCbKlIiL7g=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=hP5UnPkC9mycx1V+uig4ePalGMumbKCFpv0xLPwzU1E07D8p1DK5WlyFXABbaOmfj uVVxma3+LG3Q/+RCJofd0PPDh7cIH9Rd+ByvsuQuxZ3DVqRAhu+IGIVLCDNqShsYjf USW7Ex28HNEzQhE1h5hUIXSZYC4NzsIL45UD5wEG65Hey+7jN+DI1a0PfsaY0Gt5/5 c+/J1XIg1BNkzxZ/gxpJypkWQ50xEAzcpTjXFIJvdNIVNhDxYAdrChUZ6585dVMqID JexMDrhGZrJNrQUuiCie2wH6dGb4sPzkcex6bbdvJNOqTvSwwmOinrdjkj7vrEEhp8 HX/SEYceRt8jw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6rBP0JRJz6tyD; Fri, 21 Jul 2023 15:34:20 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> Date: Fri, 21 Jul 2023 13:34:33 +0000 Message-ID: <87jzutl76e.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Dmitry Gutov <dmitry@HIDDEN> writes: > On 21/07/2023 15:58, Ihor Radchenko wrote: >> Dmitry Gutov<dmitry@HIDDEN> writes: >> >>>>> Disabling file-handlers is inconceivable in Emacs. >>>> Indeed. But we are talking about Emacs find vs. GNU find here. >>>> In the scenarios where GNU find can be used, it is also safe to disable >>>> file handlers, AFAIU. So, we agree here? (I've read your reply as counter-argument to mine.) >>> GNU find can be used on a remote machine. In all the same cases as when >>> it can be used on the local one. >> But GNU find does not take into account Emacs' file-handlers for each >> directory when traversing directories. > > Indeed. Such usage always assumes the initial invocation and each > visited directory belong to the same remote host. Which is usually a > correct assumption. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:24:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:24:24 2023 Received: from localhost ([127.0.0.1]:60929 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMq7L-0001LW-MN for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:24:24 -0400 Received: from mout02.posteo.de ([185.67.36.66]:46703) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMq7J-0001LJ-LL for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:24:22 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 21165240104 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 15:24:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689945856; bh=D7yayK0H7HX5SPCQ63yYWvruDCkDamJOU+YQ9EgS1vw=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=kejeiALzT8/+omWQtIE/WJWEKDgo32jdgPR3mzM+SPTc9qT4Oz4Yoq0hHm53Q83q5 7nII7AQv+hwMV/EoS6E73T+8EXXp0gc7LUEyjsoBwRbjVSQmYl6t2idk5TfSiuYwg5 FTfwWiLc4ivq31Uzna56IqJYA2g/grWGyy/UW8N1fB8BYJhja41XgDg7GOyEyaNa4D dtEOlDyFMFIe4G49l7vtwM5Z1hTZ6TPb2kx572eyPrNLqqpsDWd04CbrJmHvAqIvXJ sBWj9aXjVWdj4wu6DmPjqw11YRHvPIPZCC3nsG12P1+HB2gRzo8pqY2o0J8nOhoRNT T5B1fT57rEwTQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6qyl2qpFz6tm4; Fri, 21 Jul 2023 15:24:15 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87mszpct5n.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87zg3pcuvv.fsf@HIDDEN> <87a5vpmopa.fsf@localhost> <87mszpct5n.fsf@HIDDEN> Date: Fri, 21 Jul 2023 13:24:27 +0000 Message-ID: <87mszpl7n8.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Michael Albinus <michael.albinus@HIDDEN> writes: >> And what do you mean by destroy? > > "Destroy the feature". I am sorry, but I still do not understand how what I proposed can lead to any feature regression. May you please elaborate? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:23:25 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:23:25 2023 Received: from localhost ([127.0.0.1]:60920 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMq6O-0001JU-Ss for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:23:25 -0400 Received: from mout02.posteo.de ([185.67.36.66]:35693) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMq6L-0001JB-9U for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:23:23 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 82ED3240101 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 15:23:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689945795; bh=bZ5Hxl/95taiQ8rPNMgMc7+PpRit3CgWxJKAfN07dOU=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=bavoBXHQfIQBsyMj3WbokD8UxAPLwbu5HXyRXKPn/Wnl/60GMCWZNsc/F1qlLxQED sBs01iwxdUc3YnyBU+1YLlV0HYEtxSAjGKoSr4VBVoll6Se9Fpl01WBYOPS94fP5EX ylwuRqULOoDXkbPnza1/PrBmK2U0uq8AuNJBYq5P3PxQ0486afdBJLArEs44HA8MO2 mKb8q6skzXlCJ8irPDSIvybA4i8/F97rY498nwr8EBPLAz444OGD+gpWBK7cMKcmh2 RPZYQgS6Y7x857gkIPRk1RbDVHl96drG+vEJ83y5SfffteNGI58TyFZ26Ow5Ot+Uoa Ii5Gplrm8kYkQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6qxZ58KQz9rxL; Fri, 21 Jul 2023 15:23:14 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87r0p1cta3.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN> <87r0p1cta3.fsf@HIDDEN> Date: Fri, 21 Jul 2023 13:23:26 +0000 Message-ID: <87pm4ll7ox.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Michael Albinus <michael.albinus@HIDDEN> writes: > --8<---------------cut here---------------start------------->8--- > (benchmark-run-compiled 1 (directory-files-recursively "~/src" "")) > (38.133906724000006 13 0.5019186470000001) > (benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-files-recursively "~/src" ""))) > (32.944982886 13 0.5274874450000002) > --8<---------------cut here---------------end--------------->8--- Interesting. Apparently my SSD is skewing the benchmark data on IO: (length (directory-files-recursively "~/Git" "")) ;; => 113628 (benchmark-run-compiled 1 (directory-files-recursively "~/Git" "")) ;; => (1.756453226 2 0.7181273930000032) (benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-files-recursively "~/Git" ""))) ;; => (1.202790778 2 0.7401775709999896) Would be interesting to see profiler and perf data for more detailed breakdown where those 30+ seconds where spent in. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:17:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:17:23 2023
Received: from localhost ([127.0.0.1]:60911 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qMq0Y-00019h-0L
for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:17:22 -0400
Received: from mout02.posteo.de ([185.67.36.66]:56085)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qMq0O-00019M-3T
for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:17:21 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout02.posteo.de (Postfix) with ESMTPS id 8E5DE240105
for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 15:17:06 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1689945426; bh=YVjgHITeyH1oF6ZfygheH9eD2bdEs4ri7WCKlYXrJ1Y=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=lIBF0dr9Y7Ag25IOlo1QBA3kDUkL8pMR7krtr2lPleYJ/tiy6Rpbj0Tqa+eB1ILmG
gSb8OhOmMgMHdmTDYn39+8Lbwlsbv/G/Zgj4yw/U6XpNGTwopqb1ZBns7z7saOsBGn
Bj/LBYBRn8KNP7q4+lDr13dp+apHcw+z5BH19kULb/rHlPKJL7nVDfvkA21bZlTq/J
dzRamagX4YbkrCs4PVSTr2UUXdS3w0r4VK8BN+2ZkcYrMxFYMDX0qB+3RxuKT3uEEj
1B0GiY1WoCJs8u/XqEcJ+Id6e1HtlJqsOZPneFj7K/Ct5xZuGEijYIm0cP0sniYJpo
WiyEtJwBa/aoA==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R6qpT5rtQz6txC;
Fri, 21 Jul 2023 15:17:05 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <83v8edzb31.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN>
<87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN>
<87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN>
<87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN>
<87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN>
<87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN>
<87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost>
<83v8edzb31.fsf@HIDDEN>
Date: Fri, 21 Jul 2023 13:17:17 +0000
Message-ID: <87v8edl7z6.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org,
sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Eli Zaretskii <eliz@HIDDEN> writes:
>> > The question is: what is more costly
>> > (a) matching complex regexp && call function or
>> > (b) call function (lambda (fn) (when (and foo (match-string- ... fn)) ...))
>>
>> (benchmark-run-compiled 10000000 (string-match-p (caar file-name-handler-alist) "/path/to/very/deep/file"))
>> ;; => (1.495432981 0 0.0)
>> (benchmark-run-compiled 10000000 (funcall (lambda (fn) (and nil (string-match-p (caar file-name-handler-alist) fn))) "/path/to/very/deep/file"))
>> ;; => (0.42053276500000003 0 0.0)
>>
>> Looks like even funcall overheads are not as bad as invoking regexp search.
>
> But "nil" is not a faithful emulation of the real test which will have
> to be put there, is it?
It is, at least in some cases. In other cases, it is list lookup, which
is also faster:
(benchmark-run-compiled 10000000 (funcall (lambda (fn) (and (get 'foo 'jka-compr) (string-match-p (caar file-name-handler-alist) fn))) "/path/to/very/deep/file"))
;; => (0.5831819149999999 0 0.0)
Let me go through default handlers one by one:
file-name-handler-alist is a variable defined in fileio.c.
Value
(("..." . jka-compr-handler)
(".." . epa-file-handler)
("..." . tramp-archive-file-name-handler)
("..." . tramp-completion-file-name-handler)
("..." . tramp-file-name-handler)
("\\`/:" . file-name-non-special))
---- 1 -----
(defun jka-compr-handler (operation &rest args)
(save-match-data
(let ((jka-op (get operation 'jka-compr)))
(if (and jka-op (not jka-compr-inhibit))
(apply jka-op args)
(jka-compr-run-real-handler operation args)))))
skips when `get' fails, and also puts unnecessary `save-match-data'
call, which would better be inside if.
---- 2 -----
(defun epa-file-handler (operation &rest args)
(save-match-data
(let ((op (get operation 'epa-file)))
(if (and op (not epa-inhibit))
(apply op args)
(epa-file-run-real-handler operation args)))))
again checks `get' and also epa-inhitbit. (and again,
`save-match-data' only needed for (apply op args)).
Side note: These handlers essentially force double handler lookup
without skipping already processed handlers when they decide that they
need to delegate to defaults.
---- 3 -----
(if (not tramp-archive-enabled)
;; Unregister `tramp-archive-file-name-handler'.
(progn
(tramp-register-file-name-handlers)
(tramp-archive-run-real-handler operation args))
<...>
Note how this tries to remove itself from handler list, by testing a
boolean variable (nil by default!). However, this "self-removal" will
never happen unless we happen to query a file with matching regexp. If
no archive file is accessed during Emacs session (as it is the case for
me), this branch of code will never be executed and I am doomed to have
Emacs checking for regexp in front of this handler forever.
------ 4 ------
(defun tramp-completion-file-name-handler (operation &rest args)
"Invoke Tramp file name completion handler for OPERATION and ARGS.
Falls back to normal file name handler if no Tramp file name handler exists."
(if-let
((fn (and tramp-mode minibuffer-completing-file-name
(assoc operation tramp-completion-file-name-handler-alist))))
(save-match-data (apply (cdr fn) args))
(tramp-run-real-handler operation args)))
is checking for tramp-mode (t by default) and
minibuffer-completing-file-name (often nil).
-------- 5 --------
(defun tramp-file-name-handler (operation &rest args)
"Invoke Tramp file name handler for OPERATION and ARGS.
Fall back to normal file name handler if no Tramp file name handler exists."
(let ((filename (apply #'tramp-file-name-for-operation operation args))
<...>
(if (tramp-tramp-file-p filename) ;; <<--- always nil when tramp-mode is nil
<do staff>
;; When `tramp-mode' is not enabled, or the file name is quoted,
;; we don't do anything.
(tramp-run-real-handler operation args))
this one is more complex, but does nothing when tramp-mode is nil.
--------- 6 -------
file-name-non-special is complex.
The only thing I noticed is that it binds tramp-mode as
(let ((tramp-mode (and tramp-mode (eq method 'local-copy))))
So, other handlers checking for tramp-mode variable early would benefit
if they were able to do so.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:10:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:10:24 2023 Received: from localhost ([127.0.0.1]:60907 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMptn-0000tL-Ng for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:10:24 -0400 Received: from mout.gmx.net ([212.227.17.22]:58745) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMpta-0000sI-8R for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:10:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689944999; x=1690549799; i=michael.albinus@HIDDEN; bh=V2AEHKSmR2LvJ4OjauRfXgJu7JYYdg1Z55WMKPZm7B4=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=kh4yP/pMcDQpTeQ5N/o5jGEqfVE6dN6/HSE0nYkhW01+Xqv7EwbFISMzvsC4lu4QXejW7kK vGNPk4KE4ZkF4kC4QM5dx4J8PaZy6/Lvzn2ApE8b0b+cGtjeJ97dLD0g4LQSF8VsD+f8xU9H9 s75E/SNIrd/x4UqmRiV3usQq/z6DXdIHBoZcvkykyU/I54Y8BHO9EThooHBimLXo9CM1j4TcI xY0ZBnA86bp0whtnUmC5Q9Yd2sLuB97XZ3hM4OjLyHf0Xh/RsgCQrqhKcHp/z8t+j+AaMhP0o PNbmoSgnjTsI29q1pc8svA9VKSmlAOWIto4kJM7FoBF9wJlQf/9A== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MXGr8-1qPiWH2zyI-00Ylno; Fri, 21 Jul 2023 15:09:59 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83y1j9zbf4.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 21 Jul 2023 15:39:11 +0300") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <83y1j9zbf4.fsf@HIDDEN> Date: Fri, 21 Jul 2023 15:09:58 +0200 Message-ID: <87iladcswp.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:dgC8RwxbuQW+qX1PQ4QrhYm3xutSC21WOgpGHUm4L+50Zmzp/Of O7ksIGV2Ky4gLNNc22PwrQ3RlRN5QpfLfxP+ageJ0EfWrS4lxCkHo7/KhwUJuFhkJR7Amf2 KRmQNeD7JkjHg3BCPMKQP/CNtPObGqXq7M1/dagX7kTcLT4yOCXy9IyB7rM98RXKoY3PEyo k77s65khMJY4QC4KJFmLQ== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:3PHJccNSvLQ=;/oEG/t3jAUQzah16TjPUmyxAGt5 MPH0u3miisRs1WDXT3W2djnjyL/jKgTads7I2Vv5DdaDHiQl0r2xb3SAbRuQaJkewauTvQO73 LNzEe4tnhBhhaPhnFvMBGB76u7lPBQuTJIlkQYEk7L7nKAJCTD4Fx6nn/wardWOUQ4J+DN6VS a36+p5wVSZHyogMErbpXW5Aq0Bt/71bTz+jrSqh4FDWzeU5Y1d1VM+EsLBPZAifm9Ia7X9WCf 3/zJ9YAJe2V7ujo1a+uL4tyR80SwAfu/KO7aFCkaSyxqUVaqqhvov8iJ94fwNUjBhn4e4KYZX nakCaqZ8yB7fsqu/J/aEiurZSf2uO3s6icY23rDGopHJv0+VC79L60FSBigLAT1NBheQhBddZ V545Xd323JT3Ca0KCoUEIn3Qk9LzAcHs+Pnn+Q1FfM8KPYe84fbI5zMD8y4wGqljKqfZIgs75 WnzmrhA7Jbes88KwiKgil9o670pUhCcfFCCQee9zSAyB2CuDSdfZ8w0kuXsezmZ5ygM+zA7OT TmKfZyguI7IDezNZzOCy135mup6lIFkLdMiQqWFH6z45+aGGHoZ1bBXdZOiOnpVeShMiLGzRC lRri/DYHjbw+4seb6u1qxxDB9wI1VSaHKKsblXld4fNKYqrZ3wUlw2rRutaRQu5NGUPP/cx+t PAsPQwHRJhdIw0DJARW49g6Iv+QZh/ZiAiAlItgxfSIiC0oN0vqHETBYR1k5zEobA/Hgp+ZT3 MS2vMJiiWL2kOHEiUAcFtwTWzVzIngcX75LTawf7aef7V92bkRU8EPbSQVqLVxWoaoqL0uROi tCTgIzS0g2mv1M8NU/wTq/ELBvGVTnMEhsyCBe3GPGFbdK8JCsI1dLICypIm9Fgb/tMQiM7ZH 20fpkcFMt7mcS0OCqLGSMswtCmt7UrW5gBmiH4J7RcMrD08ZN4W8EnKKAlDI+7dGdiXwbpD+O xvxZTQ== Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: >> > How is this different from binding file-name-handler-alist to nil? >> > Tramp is nowadays the main consumer of this feature, and AFAIU your >> > suggestion above boils down to disabling Tramp. If so, what is left? >> >> jka-compr-handler, epa-file-handler and file-name-non-special are >> left. All of them have their reason. > > I know, but when I wrote that disabling file-handlers is > inconceivable, I meant remote files, not those other users of this > facility. > > Let me rephrase: running Emacs commands with disabled support for > remote files is inconceivable. Agreed. My proposal was to provide a convenience macro to disable Tramp when it is appropriate. Like (unless (file-remote-p file) (without-remote-files ...)) Instead of binding file-name-handler-alist to nil, as it is the current practice. And the command inhibit-remote-files shall be applied only by users who aren't interested in remote files at all. Again and again: these are ~50% of our users. > IMO, if tests against file-name-handler-alist are a significant > performance problem, we should look for ways of solving it without > disabling remote files. Sure. If there are proposals ... Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:04:52 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:04:51 2023 Received: from localhost ([127.0.0.1]:60896 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpoR-0000ed-IJ for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:04:51 -0400 Received: from mout.gmx.net ([212.227.17.22]:38723) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMpoP-0000eE-MH for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:04:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689944678; x=1690549478; i=michael.albinus@HIDDEN; bh=mEojxwmWoCYnd2KjUMuFxY1QKg/SmUVZDnPH7nwHO0Y=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=l2AztjWJ+psNo7jPo/NSd3YCStiN8j9nAZeCUaqn0nACaK4ab2UMu0833vCtkb9DMmKs0Ua z9e5lEUXzbMMWqOPOTY+dnHMmRmgI9j7ejatIO6f64sEIn6N5zuQXcXdcxgl0gLNcI0Czm/98 bPYp8NwGwJ8cahplao1OyA9dnMU7F8EoMOPm+GklxOTE15X65MmYqM4avfstaehsfVAVhX1XC Om15luApAo/556EvesrTAqRTvxhdrFj8/Koi6j+gnRZ8IIU/CIGli92WhFTZhINEMCW219Vou 4wnsMKYDy9D8Ofj6OJSsC0cN1pj/DmrPXOqikRQ0iW7oLeIfDWVg== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MQ5vc-1qa7TJ3eBU-00M7Xm; Fri, 21 Jul 2023 15:04:37 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87a5vpmopa.fsf@localhost> (Ihor Radchenko's message of "Fri, 21 Jul 2023 12:30:41 +0000") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87zg3pcuvv.fsf@HIDDEN> <87a5vpmopa.fsf@localhost> Date: Fri, 21 Jul 2023 15:04:36 +0200 Message-ID: <87mszpct5n.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:Hx1FSL6gvCifCKe6+4TZ+XsBqTdBx8tfAb7gpqbv/KPrffphCbz E8q4O7ymKkbLDJCQD1i6lViof1TLrNbMjs14PqYqhZCGW4U6MhGrCL6lvHmrz6vanpiUZV5 /MG20FnpwJorwhXWk8/EpoW6b1KpWaNe8s29/C/sKnR5Vc29YrdnP+eko6zg0N1skWibDLG llozUYPPOkOpQZfg04Vhg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:wwPfCQh3k1o=;WT7CsVauPABU9ltsvaM0cZhzGm+ uJDmH/zOJ7mhxCeCXAVu+GmZz+L4sk6NnI8miUOk4Xa7ji2WQDYRi41RMPd/mzKAWUFrdXKtb ZE8sAYrOgDPzu474eJgwiD49izz6PFY3FWAD3jtHBJkBtPXaQhxKGtZ3/GHk80/6QtC4fk1+G o2O6bsVwOXX1uSKQnszUurGXroDJIKsaHV86fbNbyRa1t7+eQqx2VTDltiIYY3IeSsXw5Tt+3 MzknyzMeQzBGvv/jDco2+R70TNopbT6PuiSubTrTs+NsDaN2DaKbs/ztElK1l2WmFP5nRJjCG +3oH4ZtIck14UaM37kYqadVn45ZH2W0k/ZKhapHLSVmHBSTquZJkn1GWapSz3GSMECBeLn2SO san55fCptVWZlxBgawjZRUPLsEaWPZ1KXbCmyo4cNZOTDbOfFtCrU5whNRrn1AaeEF8ytzC0J CY3ubH4ajYLT8HdG9gbv2dGRqr2fkUR+2DegSrLNn5hdKO4FmPFU4s3KsxAqEe+sy6NawStdT SuC3qt8z795lK7V/82ldGYjGTtjE+2FGY/ufwjiBtnV9SEwQOrmTVpxx4SPbsS9D80XB8prM0 1WklpXJFI67w3krnFY46SOcm55M97uzcOJn6WQZw/IdJUpiXS4Lra7vnI9jsxB9OxMkZu63rb L5lATEdST9wjgRqToi+JdZCc8DOYwn0v4Dc02hsg0fuc4Lz/zGpm3GcNpuwAKG2mfOZgN7vlU WmFWP8/tCDJ93qsinkz065NJ+WlES5la2PzhuAHGNT052TdkIsaOYmXupdSdOXyLADms0yx78 Qg1jgvoaZ7CFwVom7G6J8rGbT8F9tuxDcLEoQZA3wAcvAm90seKlU7KAshAYwkTZl3WKYKk0J lWU14LOLb8yaOwKGj0rmZxhudILuq7/qprxTICGClkBBNPbdJr4ACq7se7sm5+r0fZTZMW4uz rdrQBeSN27YcKKaTK2LwoSKgnIU= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Ihor Radchenko <yantar92@HIDDEN> writes: >>> I am not sure if I understand completely, but it does not appear that >>> this is used often during ordinary file operations that do not involve >>> tramp. >> >> Don't know, but it is a documented feature for many decades. We >> shouldn't destroy it intentionally. If we have an alternative, as I have >> proposed with without-remote-file-names. What's wrong with this? You >> could use it everywhere, where you let-bind file-name-handler-alist to >> nil these days. > > The idea was to make things work faster without modifying third-party > code. People use already (let (file-name-handler-alist) ...). As I have said, this could have unexpected side effects. I propose to replace this by (without-remote-files ...) > And what do you mean by destroy? "Destroy the feature". Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:02:21 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:02:21 2023
Received: from localhost ([127.0.0.1]:60885 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qMpm0-0000Xr-QO
for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:02:21 -0400
Received: from mout.gmx.net ([212.227.15.18]:47201)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <michael.albinus@HIDDEN>) id 1qMplr-0000XA-QZ
for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:02:19 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de;
s=s31663417; t=1689944518; x=1690549318; i=michael.albinus@HIDDEN;
bh=CrA1BJulU/5ibdeCNbGQ+hDEjrXAKL3OmN0+uEZwIe0=;
h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date;
b=miYFMKG5gh3Grf4LpJtNHCO3jBfhkcvxsWWzOx6a0XLJD1aUUeByXifYyfJuZoVKfTSPh2g
8jCFIibS1F4oLRelJbHaosuBME7RwB5IbibFyJB5FUiWlXyfG6pbJvG37VVDFiYQxDbglsVBw
j207VPOyycm3Eyng0ZG/obwS8ahFt0ed6I4Wibgozxg5g/1TMA2cEXN+qmfu8Uj9tws/gNWKN
9CwJ1kuPZx86Ns6W3mX9ABsFo32Kd39q6k8RVPXvgtzfD+svSlg6bv3jmPV7v56xKkEUHJN0J
R3yZm3406LWSx7Q0nyYukdifWS7cSQpr1n7LJGjn7xmbMKDFd0sQ==
X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a
Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx004
[212.227.17.190]) with ESMTPSA (Nemesis) id 1MoO2E-1pY9xG0gXI-00ojoE; Fri, 21
Jul 2023 15:01:58 +0200
From: Michael Albinus <michael.albinus@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <83v8edzb31.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 21 Jul
2023 15:46:26 +0300")
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN>
<87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN>
<87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN>
<87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN>
<87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN>
<874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost>
<87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost>
<87cz0lmoxy.fsf@localhost> <83v8edzb31.fsf@HIDDEN>
Date: Fri, 21 Jul 2023 15:01:56 +0200
Message-ID: <87r0p1cta3.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K1:rrfDIcNewcxn2AyGKRlVyVU0RKUnxdVBaiInX+Nual2vbMMsMy9
KwyoEZ5P0+Rc+fKFwiSKGw9MuSQMvTnga79OEVqCqo9JsG8NRRDFnynpl7j1RpRb2UN2Pfe
s3pzhPf2HHCYbH7P0sLNpCS9e96U1PrQzS4DSK5cHsO1bNca6BRt3Gx7LiSJ8TdpSRV8X2e
r+ktqDmzhBpYFHJsCmGBQ==
X-Spam-Flag: NO
UI-OutboundReport: notjunk:1;M01:P0:kO388ASvuTQ=;3l80Xhr7SG1qDsAAiN8b0MJNhV8
ghOcG6USDkQCO9AJo1aQ+i50QDZt1ngb06t2mpPldWFhio8bsGzCfV2GTa/05v3oHGwjXrQFd
IRh5IKjga8xTS5CDwkOQ+xks3cYCOrBrnvpwHotqB2Qy1Y0J9U+Fd35jheOwC1Ny1At9cw7HI
YLNKezDEb7JTkVPm6XlEsfKxT7cT2tcnxasiA9KhMD8KL4ZcIg5KVweK/iNZKBVSmAGOVytH8
ucmb1tOoZVQGQIr5TdNaa2cERPfeZLjyTJyikXD2RJOWiBhoaaJfvbYpPVgiLgT8hMYqTTllA
1s7xb/8fN1Tk5kc4jOSiVFWoFdSqc/85LKQvH9/nOaXGMpwGc60a3Tt5XSkcIg627sgyFmRSl
PJt4d7/FaiXqouhakZMW8RTX9dgyNgnKZ3gVZfm8XseH+IowgwvQYBVCFjT7uiDrswv54QoTZ
KCC2gsejjHAdFCS9TjNkU2Yi/K/nMfrwbbhI8whzUnQKV+gatMnBrmP02gHz1TJ51V2dEqAUn
ZcuoWC78r0ZAZcm3HlB71CD3x9814SQN6AvdRzndObJtjnB35wqtRwfcUNpglSnZRK3cnC8nq
IOV6M45IOEwkzU8vnWTJRwNEv2JuG2FP+Q4x7PF/AcmBRM4TEXuQwzUSPjFXdQfG+bhQJ1H1A
dlWau/ClJU1MvY2dAeeDQHmjw1niQIAcdTnBJG098eCEHexv8TvASKvsCAYI1eRI+FJrwwqc4
8EMkbELVhTj8tvt2X6epOQeJ74TtSbQVg0yGvum10djAt2qdfGnP3mm8d8spkVOtpOlCEyazl
qTXBau2/YSA/Mkcf1n4nZzXoxKPJT8CtVHv+jUiiHnvfXr6P6LJ2KdWlzGF9ACx/gNFpjXpIO
U8i+KEjbBdMl4/bZdZYIpRJX1b6fQLISARPml7lrPwpwCSlRuonCMBtvDy1UgWihiGN3rydJT
/2Yvp5N/BMuYuMbePZ+MfSEQZs8=
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, Ihor Radchenko <yantar92@HIDDEN>,
64735 <at> debbugs.gnu.org, sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)
Eli Zaretskii <eliz@HIDDEN> writes:
>> From: Ihor Radchenko <yantar92@HIDDEN>
>> Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735@HIDDEN=
rg,
>> sbaugh@HIDDEN
>> Date: Fri, 21 Jul 2023 12:25:29 +0000
>>
>> Ihor Radchenko <yantar92@HIDDEN> writes:
>>
>> > The question is: what is more costly
>> > (a) matching complex regexp && call function or
>> > (b) call function (lambda (fn) (when (and foo (match-string- ... fn))=
...))
>>
>> (benchmark-run-compiled 10000000 (string-match-p (caar file-name-handle=
r-alist) "/path/to/very/deep/file"))
>> ;; =3D> (1.495432981 0 0.0)
>> (benchmark-run-compiled 10000000 (funcall (lambda (fn) (and nil (string=
-match-p (caar file-name-handler-alist) fn))) "/path/to/very/deep/file"))
>> ;; =3D> (0.42053276500000003 0 0.0)
>>
>> Looks like even funcall overheads are not as bad as invoking regexp sea=
rch.
>
> But "nil" is not a faithful emulation of the real test which will have
> to be put there, is it?
Here are some other numbers. The definition of inhibit-remote-files and
without-remote-files is below.
=2D-8<---------------cut here---------------start------------->8---
(length (directory-files-recursively "~/src" ""))
146121
=2D-8<---------------cut here---------------end--------------->8---
A sufficient large directory.
=2D-8<---------------cut here---------------start------------->8---
(benchmark-run-compiled 1 (directory-files-recursively "~/src" ""))
(38.133906724000006 13 0.5019186470000001)
(benchmark-run-compiled 1 (let (file-name-handler-alist) (directory-files-=
recursively "~/src" "")))
(32.944982886 13 0.5274874450000002)
=2D-8<---------------cut here---------------end--------------->8---
There are indeed 5 sec overhead just for file name handler regexp checks.
=2D-8<---------------cut here---------------start------------->8---
(benchmark-run-compiled 1 (without-remote-files (directory-files-recursive=
ly "~/src" "")))
(33.261659676 13 0.5338916200000003)
=2D-8<---------------cut here---------------end--------------->8---
Removing just the Tramp file name handlers comes near to let-binding
file-name-handler-alist.
=2D-8<---------------cut here---------------start------------->8---
(inhibit-remote-files)
nil
(benchmark-run-compiled 1 (directory-files-recursively "~/src" ""))
(34.344226758000005 13 0.5421030509999998)
=2D-8<---------------cut here---------------end--------------->8---
And that's for the innocents, which aren't aware of Tramp overhead, and
which don't need it. As said, ~50% of Emacs users. Just adding
(inhibit-remote-files) to .emacs gives them a performance boost. W/o
touching any other code.
=2D-8<---------------cut here---------------start------------->8---
;;;###autoload
(progn (defun inhibit-remote-files ()
"Deactivate remote file names."
(interactive)
(when (fboundp 'tramp-cleanup-all-connections)
(funcall 'tramp-cleanup-all-connections))
(tramp-unload-file-name-handlers)
(setq tramp-mode nil)))
;;;###autoload
(progn (defmacro without-remote-files (&rest body)
"Deactivate remote file names temporarily.
Run BODY."
(declare (indent 0) (debug ((form body) body)))
`(let ((file-name-handler-alist (copy-tree file-name-handler-alist))
tramp-mode)
(tramp-unload-file-name-handlers)
,@body)))
=2D-8<---------------cut here---------------end--------------->8---
Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 13:01:00 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 09:01:00 2023 Received: from localhost ([127.0.0.1]:60881 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpki-0000S3-Cl for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:01:00 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:52801) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMpkf-0000RZ-CB for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 09:00:58 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 85C5F3200925; Fri, 21 Jul 2023 09:00:50 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 21 Jul 2023 09:00:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1689944450; x=1690030850; bh=sY0D68lTqr/ynw2XUE4suwbwfPW+oxwP+QB VJArAoV4=; b=eh8tNW4KF08VnaC78zNhHFQwXlscbHE+E8FfufG3crEk97q1n0I 8BqVvYVhE2O22U8oo4vw0KZTKdzjC4EdjdI8k44VQbuK2EGUEc+Y5d7YsGtpUokn lSgtdMyxVrbCSI/yEi0sTV0f8ke6C5ePdPRaF9EL7RyFEb4EgL2R5MKUUFaBjbQP mNGu6lr3cQyPN9HB2f9kjZVgew8rx5mIsRFZhYLVWSVHdYAVlATRUhKll+HKKd+A 3VixBFKr/uYRkWHLbvJwwbgRgwYlhCOjKK3LttQEizDICvOZK1IJoi3ls1eyUbm4 zxt0zFnTUXBpxnS80ksV0iyR/9XSYxA/O8w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689944450; x=1690030850; bh=sY0D68lTqr/ynw2XUE4suwbwfPW+oxwP+QB VJArAoV4=; b=gHuwnthOfVBS3u9JoCmwQXrZDnhKoUaG/ZMCTxKUyQL/9HHKXk4 m9rypFbeyjY9EuG1MYhy62673pW0EBZtM6l046fnRedDBfAL2yNSkkl9e66y7/rI y/upjk9VYJ7nUVrjvePdd7ofniuByTgOOrFHlrrBFi1VNhU8DTwL+KG172/b5W0L 3RM1IuOXzrcLj6D27f6pCye56QHE1VlK+OOr2d9VRVptXPNbQ8S4B3LMc4aNnDi4 tpkDzFwsZaz6fEcDE+QPsFvHCWN5oZFTLiMZhAuF4Q/9Cjw6QnTikhWqvjXq8dtP 3LXta0XDebQPDSqPVbV3mtNs0sk91ZfVMoA== X-ME-Sender: <xms:gYG6ZM1ufL-31Vd4TLcKGRv8PaJ6JkZMu-DzfLs8h21yxYl5ciQoDw> <xme:gYG6ZHF3eYZAexW-zpAZpkMg0pUdSh5jMO6CTZhd_39e2NGcVnhmgAcwCxS-3erVb tqGzL-PAhqOyuXnx9g> X-ME-Received: <xmr:gYG6ZE65k3RvhT2DQ9sGufRllCVju45yI2B-MwQQTUHfkb2HeT5glWpt2KRVvXQ> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgdehiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:gYG6ZF1mLRhvGvJrco8ic9_JrcHD7_3pA57w9hIUmw2FA5orokvITA> <xmx:gYG6ZPHnS1WJ_XNocUmLmt-asbzoQV-JCb91kvDtXB0xkNz9BbVmcQ> <xmx:gYG6ZO_jxtTmxnMGT_na8NS_8dWlfKn5yNIf57NtEK6CSfHCE2hAXA> <xmx:goG6ZESidxXWGlm-6SvWm6b7UA4DUSfMah9DD5PTIuB6EuiEWp0qRA> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 21 Jul 2023 09:00:47 -0400 (EDT) Message-ID: <0c6f4daf-ea8a-0636-1b51-f4640ff39834@HIDDEN> Date: Fri, 21 Jul 2023 16:00:46 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> <87y1j9l8u2.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87y1j9l8u2.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 21/07/2023 15:58, Ihor Radchenko wrote: > Dmitry Gutov<dmitry@HIDDEN> writes: > >>>> Disabling file-handlers is inconceivable in Emacs. >>> Indeed. But we are talking about Emacs find vs. GNU find here. >>> In the scenarios where GNU find can be used, it is also safe to disable >>> file handlers, AFAIU. >> GNU find can be used on a remote machine. In all the same cases as when >> it can be used on the local one. > But GNU find does not take into account Emacs' file-handlers for each > directory when traversing directories. Indeed. Such usage always assumes the initial invocation and each visited directory belong to the same remote host. Which is usually a correct assumption.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:58:42 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:58:42 2023 Received: from localhost ([127.0.0.1]:60876 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpiT-0000Kc-OX for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:58:42 -0400 Received: from mout01.posteo.de ([185.67.36.65]:44675) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMpiR-0000KF-Dz for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:58:40 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id D7F6824002A for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 14:58:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689944313; bh=EKe2XiBJlS9VdwpiD7r5P2c95uMxpRUKrmKcF23M0KM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=CeZPDWXFBBwJdA9/YvL2lLP7g3MP6xl6unkTDdPZf5RlyTq0RPNl/TwbQ1Za48RAM KydziKyIrDxCBwSJoZIGwqQW2D/NSssWZnG2EHZzwmSVvwZNm+TOxSoZITkoCht6QQ AMjuc4h6dTMo/JHJEEPyuPKqqKlljoe19VIvDpPHttoVeoN7OzZzcQg1RpH6iG95IQ P7DKKacS0pisS0KGXJgALzHM2g1p7fpvJcagmVF4SbzF941zuESCTXCACbuAmmg7W7 0RlkGP8HjkiyhdnOZtJ7aSDX1Q6dsENunAQZlbuc0TtbEZd3mzP9NH5hbwL6kLh+5K cIEpHzczJo27Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6qP51dtHz6tw3; Fri, 21 Jul 2023 14:58:32 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> Date: Fri, 21 Jul 2023 12:58:45 +0000 Message-ID: <87y1j9l8u2.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Dmitry Gutov <dmitry@HIDDEN> writes: >>> Disabling file-handlers is inconceivable in Emacs. >> >> Indeed. But we are talking about Emacs find vs. GNU find here. >> In the scenarios where GNU find can be used, it is also safe to disable >> file handlers, AFAIU. > > GNU find can be used on a remote machine. In all the same cases as when > it can be used on the local one. But GNU find does not take into account Emacs' file-handlers for each directory when traversing directories. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:56:31 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:56:31 2023 Received: from localhost ([127.0.0.1]:60870 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpgG-0000Ee-Vh for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:56:31 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47372) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMpgC-0000E4-VB for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:56:23 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMpWI-0005pr-KM; Fri, 21 Jul 2023 08:46:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=pA2D+kQCW2pjhu8iqOuIoNirmB1G0z71J5unhP3zcBw=; b=rUVHr4cIgeSQ VDx1gBiDvX5dR8hhZQwW8gVdrtAhDpNzIcHpMM1u9gpLUHPFo+tagGM0/8A/SicaO5AS7hvgVmEuM 45jfL6u7ezqo68KPFZS7qK3yW81DspJvXE/XDVCe9YYUaCEqd+WFWQZmKdq3pjns4qgAWsz4LMmhT Qet7uLX+ZE0Fjm69+7mffRAO8cKNx0JbuV5bV0k10vyc19Bl3hIENiQaVhFSqr13wNLDxom8mGGO2 QtZpvzfddUUF/qmHYuuGp4ywR9dRuywmdMZDHyjl3xxyEKVBOHQfrLCbNUM1ew17y2UYaYtaSPH6K vfMHiJObUosgRxSLXxBjJw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMpW1-0006Ea-R3; Fri, 21 Jul 2023 08:46:05 -0400 Date: Fri, 21 Jul 2023 15:46:26 +0300 Message-Id: <83v8edzb31.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87cz0lmoxy.fsf@localhost> (message from Ihor Radchenko on Fri, 21 Jul 2023 12:25:29 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, > sbaugh@HIDDEN > Date: Fri, 21 Jul 2023 12:25:29 +0000 > > Ihor Radchenko <yantar92@HIDDEN> writes: > > > The question is: what is more costly > > (a) matching complex regexp && call function or > > (b) call function (lambda (fn) (when (and foo (match-string- ... fn)) ...)) > > (benchmark-run-compiled 10000000 (string-match-p (caar file-name-handler-alist) "/path/to/very/deep/file")) > ;; => (1.495432981 0 0.0) > (benchmark-run-compiled 10000000 (funcall (lambda (fn) (and nil (string-match-p (caar file-name-handler-alist) fn))) "/path/to/very/deep/file")) > ;; => (0.42053276500000003 0 0.0) > > Looks like even funcall overheads are not as bad as invoking regexp search. But "nil" is not a faithful emulation of the real test which will have to be put there, is it?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:46:17 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:46:17 2023 Received: from localhost ([127.0.0.1]:60851 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpWT-0008Iy-Bd for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:46:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35306) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMpWR-0008Io-OP for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:46:16 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMpWI-0005pr-KM; Fri, 21 Jul 2023 08:46:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=pA2D+kQCW2pjhu8iqOuIoNirmB1G0z71J5unhP3zcBw=; b=rUVHr4cIgeSQ VDx1gBiDvX5dR8hhZQwW8gVdrtAhDpNzIcHpMM1u9gpLUHPFo+tagGM0/8A/SicaO5AS7hvgVmEuM 45jfL6u7ezqo68KPFZS7qK3yW81DspJvXE/XDVCe9YYUaCEqd+WFWQZmKdq3pjns4qgAWsz4LMmhT Qet7uLX+ZE0Fjm69+7mffRAO8cKNx0JbuV5bV0k10vyc19Bl3hIENiQaVhFSqr13wNLDxom8mGGO2 QtZpvzfddUUF/qmHYuuGp4ywR9dRuywmdMZDHyjl3xxyEKVBOHQfrLCbNUM1ew17y2UYaYtaSPH6K vfMHiJObUosgRxSLXxBjJw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMpW1-0006Ea-R3; Fri, 21 Jul 2023 08:46:05 -0400 Date: Fri, 21 Jul 2023 15:46:26 +0300 Message-Id: <83v8edzb31.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87cz0lmoxy.fsf@localhost> (message from Ihor Radchenko on Fri, 21 Jul 2023 12:25:29 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87cz0lmoxy.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, michael.albinus@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, > sbaugh@HIDDEN > Date: Fri, 21 Jul 2023 12:25:29 +0000 > > Ihor Radchenko <yantar92@HIDDEN> writes: > > > The question is: what is more costly > > (a) matching complex regexp && call function or > > (b) call function (lambda (fn) (when (and foo (match-string- ... fn)) ...)) > > (benchmark-run-compiled 10000000 (string-match-p (caar file-name-handler-alist) "/path/to/very/deep/file")) > ;; => (1.495432981 0 0.0) > (benchmark-run-compiled 10000000 (funcall (lambda (fn) (and nil (string-match-p (caar file-name-handler-alist) fn))) "/path/to/very/deep/file")) > ;; => (0.42053276500000003 0 0.0) > > Looks like even funcall overheads are not as bad as invoking regexp search. But "nil" is not a faithful emulation of the real test which will have to be put there, is it?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:38:49 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:38:49 2023 Received: from localhost ([127.0.0.1]:60830 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpPE-000855-Uf for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:38:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47620) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMpPA-00084b-IB for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:38:46 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMpP3-0003He-7M; Fri, 21 Jul 2023 08:38:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=hyEisAq6Nd4xnGbML/VwTNxtJuSZgWcVONyZWancEI0=; b=SwtOBz6qqWfE jG6koNJ6kI+gASSUBQnFyIsow9Ij5+872PstKmeYt/wi9ACp3/NHtG+RDH3aQeOfRCtjSdcXbF8d8 WtFZ8AkGUajuAGrqK8xdfI0jmGMyJFgd6qvPvMeGRUTn+xT3NX8b21U4up8Y6Yf3qPxGSMnkhA5i0 yGyRL4LYGH9dx4/dn5JraIwpH9CjI4T+sScGOjzM81rCu9ZRwt3CQSr5i06220MFm+eYcQC8qmLdB lauv3L9/IL6ZiX57jjSVjx7iyK3LXr0HxlLXbG+xEX4/S9kgZMOd3dVXJzGCE8EUAr55Ao0Y9rJtP Xta30zchkiDx1XDChAclVg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMpP2-0005Hq-NH; Fri, 21 Jul 2023 08:38:37 -0400 Date: Fri, 21 Jul 2023 15:39:11 +0300 Message-Id: <83y1j9zbf4.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> In-Reply-To: <874jlxebz5.fsf@HIDDEN> (message from Michael Albinus on Fri, 21 Jul 2023 13:32:46 +0200) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Michael Albinus <michael.albinus@HIDDEN> > Cc: yantar92@HIDDEN, dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, > sbaugh@HIDDEN > Date: Fri, 21 Jul 2023 13:32:46 +0200 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> Users can call the command, if they know for sure they don't use remote > >> files ever. Authors could use the macro in case they know for sure they > >> are working over local files only. > >> > >> WDYT? > > > > How is this different from binding file-name-handler-alist to nil? > > Tramp is nowadays the main consumer of this feature, and AFAIU your > > suggestion above boils down to disabling Tramp. If so, what is left? > > jka-compr-handler, epa-file-handler and file-name-non-special are > left. All of them have their reason. I know, but when I wrote that disabling file-handlers is inconceivable, I meant remote files, not those other users of this facility. Let me rephrase: running Emacs commands with disabled support for remote files is inconceivable. IMO, if tests against file-name-handler-alist are a significant performance problem, we should look for ways of solving it without disabling remote files. In general, disabling general-purpose Emacs features because they cause slow-down should be the last resort, after we tried and failed to use smarter solutions.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:38:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:38:39 2023 Received: from localhost ([127.0.0.1]:60826 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpP5-00084c-MM for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:38:39 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:44387) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMpP4-00084M-4j for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:38:38 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 3C2A23200942; Fri, 21 Jul 2023 08:38:32 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 21 Jul 2023 08:38:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1689943111; x=1690029511; bh=413mKa+MZEDBWIGIfkiXPnltNP9SNs7dKSL kOe/d5GU=; b=IUQR+J9qQ6YZyh9uARi/dIRO6ECVrVL7aw+0nXlOo7r3uYJ294n EsShLuntm3t+vU0S9Lv6CiJLyqidJbKPNlGXRYbbnyIusuoXG483zz1DZ4SV2vJI a/glqphBvg+8df0BXCAmGc/gMQxIalg3n4p0ItcuCUPhjN3sYZxVIKfTdWlG7Gv8 DqChX8wAiSzKeT7irQFwoO7+fy22GasrPRDzB9d8gnSn3tmfsRD4UAYyZIkRfaus aveQglvbD11nCQ2dHfcHQIRvmtEBn4vTzsRxUC8DLPbTjpm7dNAOXW+rKOpCno2P fKy8WIR7oiCeujSg0X0gEMd/uAPVliYYeQg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689943111; x=1690029511; bh=413mKa+MZEDBWIGIfkiXPnltNP9SNs7dKSL kOe/d5GU=; b=Vdsw+Pcl4kc9TgFdC8gaxFEUS/a/QBum3XtL2/XhlqxHLtmtG5I 7ilL4tmNnvIRMYjO8DK2vG+HWw6hrFtuCEGeq1toyEGYBergkP3kne8whATF/yDm 5+uoWmGEfKCuJS5BXKwNk/mc2NI/h6sDgKRCyKkdIBhmj5HiEhyaNbW3YaO98G/a aMHZynrV+pDKp4mh6XcnErrZBpsoqHYkvEXlJb5c4C0mb3JKh/5Z1E23dGz0GHJl vuBac+nheGE7ZSkapyodLtF7UU5A+jwjBvTBa2v8nxJ0RENkoIz82H+olVDAficw 6sTpzLObMEEczt21ySQFiW1ZhA0fyDS814Q== X-ME-Sender: <xms:R3y6ZNFlSvLnoRYHTbTZ0VR7pGDlEazMtJz9Hd4x1CoIW-C9qmzmlQ> <xme:R3y6ZCU8yWN_E95BB6UZ5AeSiwGwucLEMBjdGauuog75mThEBggFqZv6DSEy0hT3U ZRCzFtNGS8r2vxn-FI> X-ME-Received: <xmr:R3y6ZPIxNMjDiuziaQbv3NSLTBNG1fZL4-H-fSzv19Px0jm4xA2qV_qSj_80ROw> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgdehudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:R3y6ZDEcFE8DK0vRGB07Km0ORhdxc34DNOHilEsO4wTSnXB-5rtLfw> <xmx:R3y6ZDXLU3QOJqfzmuG80eb4rlHuJEvaMz6pCcXNvkG5QXcJn0Aesw> <xmx:R3y6ZOO_AZwMvMprkKDGWfk2Eis1Y3YzevuEa3StKWwWR0yEfCTZ6g> <xmx:R3y6ZOQe39DpOIA70qvuj1WgKFwr7X87IqepwCUxXI1ZA32297hH-Q> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 21 Jul 2023 08:38:30 -0400 (EDT) Message-ID: <a4e7c1b6-483f-7cde-96df-03012c241fb2@HIDDEN> Date: Fri, 21 Jul 2023 15:38:29 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Eli Zaretskii <eliz@HIDDEN>, Michael Albinus <michael.albinus@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <83edl11qzn.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 21/07/2023 13:46, Eli Zaretskii wrote: > How is this different from binding file-name-handler-alist to nil? > Tramp is nowadays the main consumer of this feature, and AFAIU your > suggestion above boils down to disabling Tramp. If so, what is left? I don't understand this either.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:38:17 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:38:17 2023 Received: from localhost ([127.0.0.1]:60822 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpOg-00083v-2L for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:38:17 -0400 Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:58505) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMpOa-00083c-AR for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:38:12 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 8019D32001BB; Fri, 21 Jul 2023 08:38:02 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Fri, 21 Jul 2023 08:38:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1689943082; x=1690029482; bh=RsEw8kz80yGAbUVUnq8QI+goFsNzdHjT5B8 YJkWevZE=; b=oTQR3kiI2Vbd6ncYOpu3qjR/3+jiUDGpY3mQFEN41AEuxCZmKQo 6coAjkxkc+J5peKuWlIaxBj0WEn9vPlZcnFxl81CQZ4Ik+l5V2i7MiC0yA7FWFbJ wxZOa0Xiig/ydmO+uX1ais0YJWMuzFzNLhf2L8hNvx5F5RjZcU/0W9fjGIqkdp1v FizpKqIFZaLviaq6S2S0ebRMowabGC/OeuRLc+JlaISnlVF10hL69lTLNB7hlmRF xuQaCVdh2Ht/fAUDzUIZ0AhfpR+bwqUBgZuU6ACZn5xZlVcSEzMl4oG1jEv/3Z9F 36GZnonWogiM/Cxkit5ahEosrAk1mA6Nn+g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689943082; x=1690029482; bh=RsEw8kz80yGAbUVUnq8QI+goFsNzdHjT5B8 YJkWevZE=; b=Xs8eaVGFCgU5yocLqYGSxKm9La5FFnpZgj2fJRiXFaXnjkGKbu6 fpJu8tRfaBQ8WhOtSlUgsJjEZKbj6Rp9fxAj4taO/lrWEwGLn5ktSeyCSBI/YHPy uCMoI2UPByWiIHjeWAdBbS9H6Gay4gEL0odHzWSK5ZyBvBc5G6U1fSDko/TzZQf0 seuGMgeNgE2HnZfCytisBikCYf8RXhBcMgx3oEx9UBKGSCZGFs8wFwx1nWP7EV2f 29qGwVGC0EYcZ28PnBC5UXxCuAND2huL2gGQzqYTC+oY2pgV0Slz5/OygoHwKfWk Jp6stgxTLQlTbwTTntGsYqmnbkhZbQzR9ow== X-ME-Sender: <xms:KXy6ZNl67cGjVYnKSfk-tWa80-u0Wgi9H6ut5GesMi234dSAnqMprQ> <xme:KXy6ZI1gfqQ8m473Gmlr_qL8k6x02XI07vKbetTn1RW9GxQXFbW-7XjwZFaNGuSvN x9WR9VSej1z3TplNko> X-ME-Received: <xmr:KXy6ZDoi6tOfT6UfaUKsUxJa7HCNfS-1ns5-f17CX2Hzjkzbj4ZvfslMcWTgGuM> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgdehudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:KXy6ZNkazLM-cpyDV_737Bi7DxG64uJKcOrqbXJfgrxadnyxtO8_5w> <xmx:KXy6ZL0oVhgm8l5kbrV1fK9LCJ2G3XelWx2gG4RFQEOR4DuKVchEbA> <xmx:KXy6ZMvCa7pv2yZHN9vNJUcMXtmmMyxVaqIo1Ool9bjXnKjP25xCLQ> <xmx:Kny6ZJCwhKlIuwpg3fk7OKRGYkQt7SvEFJeb7HGx5dPe6_l4i5R24g> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 21 Jul 2023 08:37:59 -0400 (EDT) Message-ID: <6f55b301-7610-ee4a-996e-fc6a8db33c33@HIDDEN> Date: Fri, 21 Jul 2023 15:37:56 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN>, Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87ttty8l73.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87ttty8l73.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 20/07/2023 21:57, Ihor Radchenko wrote: > Eli Zaretskii <eliz@HIDDEN> writes: > >>> Disabling `file-name-handler-alist' cuts the time more than 2x. >> >> Disabling file-handlers is inconceivable in Emacs. > > Indeed. But we are talking about Emacs find vs. GNU find here. > In the scenarios where GNU find can be used, it is also safe to disable > file handlers, AFAIU. GNU find can be used on a remote machine. In all the same cases as when it can be used on the local one.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:30:39 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:30:39 2023 Received: from localhost ([127.0.0.1]:60817 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpHL-0007tF-30 for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:30:39 -0400 Received: from mout01.posteo.de ([185.67.36.65]:59723) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMpHI-0007t1-DU for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:30:37 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 814BC240029 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 14:30:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689942630; bh=3REpDgzWakRA6nIlYtKgvy2DHPEtlvUW/pYE1dukDIQ=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=RiEyIuy0/kDr4SDvJISs3xurw1bdyspFuq9lto7FTVU/H8pcio208p+4FkaOth+D6 k7s+XRutRy32BvuxizmO/66KmsCUnKnPAUYW9Sc7eQ4tbN/ljf79wa3ykl7e0lvru8 AdaT/pZAd/aL7gUnwJ+JyaF4rHGaTDcYm076JuQdWnCymXPxsQfmXoPg3ZNjjTpElA Dk0G/y0qkG0FczrEcMFwUQjoBlEnAuLqr9BMqUfJIwqZmsKYzsQGqMFOxD/3UwAaR+ T+RIgiOAkafX8pd2wT/0EJ0HKCF2fIJuq4l0vQ9e5KGA0ZDBrgKZGsnHu0WqZqhzVa lww5g3hJMuHYg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6pmj5gjQz9rxW; Fri, 21 Jul 2023 14:30:29 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87zg3pcuvv.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> <87zg3pcuvv.fsf@HIDDEN> Date: Fri, 21 Jul 2023 12:30:41 +0000 Message-ID: <87a5vpmopa.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Michael Albinus <michael.albinus@HIDDEN> writes: >> I am not sure if I understand completely, but it does not appear that >> this is used often during ordinary file operations that do not involve >> tramp. > > Don't know, but it is a documented feature for many decades. We > shouldn't destroy it intentionally. If we have an alternative, as I have > proposed with without-remote-file-names. What's wrong with this? You > could use it everywhere, where you let-bind file-name-handler-alist to > nil these days. The idea was to make things work faster without modifying third-party code. And what do you mean by destroy? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:27:32 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:27:32 2023 Received: from localhost ([127.0.0.1]:60800 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpEJ-0007mA-UI for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:27:32 -0400 Received: from mout.gmx.net ([212.227.17.21]:41949) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMpEH-0007lv-F1 for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:27:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689942437; x=1690547237; i=michael.albinus@HIDDEN; bh=PdWE4sqdgCU6TXGxrtApxXqAb+IvIMteqiGJLLSrV5U=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=bh7srkFGGXpPFLR0dnlpbW8zFjzSxTffEeSNZVsmRcqf/bll1NsPOKHAbuWEYc3P3LGZyq0 QH/ZXWYrtj8Mjfe9vQj/DBgPh9GWV+uFGYoA2L12IlvWRVSms1X/jlVO9Ab5jyChh3DlMx2VP x/QhQoSaTG7hcNDBIhJ8X9ib0Nc7E6mgG+j4gev3dcZxzoolV/O/6kAkmbR/QhdMC7k8/PD39 RxoTuKRasB+MLody17K3SL6rBKenNZpM+5H2D/yzQopuLV8++mboy5YbOYUcLMX+jw7GN9pOp w1IvVRnw74lLx/aeVLTY57dAGFmtKMan3xCwRuF0JJpuHlTgTc0g== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MlNtF-1pe6H21y3B-00lkG7; Fri, 21 Jul 2023 14:27:17 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87fs5hmp6i.fsf@localhost> (Ihor Radchenko's message of "Fri, 21 Jul 2023 12:20:21 +0000") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> Date: Fri, 21 Jul 2023 14:27:16 +0200 Message-ID: <87zg3pcuvv.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:MbCHBgWcexySIXXbWFTXnfLWfJ/TBafSqIfayEd4z2Glr4vihPt eGo8f7MOKhl0Kqc2ppJ5wuGsKM7sKWXeoZXsfhBVrZ3M1DNH//F2hMjbujaFOwSkRumrVh2 TTlz8fKGBeVJNsk9ab5l01ZW3cbk3BzTy00r8RVblwnNyXp2aYoWR7cjP3r82fI8lyh3rDJ 7MfOLlHL0VqQuQXqcB4Fg== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:0wKlWK00M0k=;y6VHWtQ5zMY1BVZe2ieGbXnbkq+ ytaptkaSvkt5SKZrQ57OCTfTsX8AL7DyEHE4/IKk4KPoCidsSQTnBFSJWjuF3DnoZZUM7JbSt njCOps7B8frYV9hrfUgmH+UIWOAYqm1M4NVwtdpFyXGs58RrlI4TtVLzfsMzzwxwVFN920Suc dTj/FpUhX0CY+LX1MN459hFK2jdZSB3giKzUdZKo04ktlntK4gMxABObSb88+iCEJZKM72iO0 2p1Ki9IXOJsaixXUFng0ec0rhN15v6mNobJo6QR0+PRYioJZkSUsbzy16fF36fyyXcuiVTohu 9aSaCWeh2k50NDrCu9wPOzSFdKz6MXuuz5H95VsYLtUovKQOh334V1EX5Ef4/f+tFbyafIlgs KDkRACG+/Ed6aOgqe+kegj8UCQajoe60TKrsPrmqxartg4KE/eZQnhoIErFUixbPPqNVT5ptv gMwQT+0E9wV/OMDGh2nN6iP/7VJIA2V03Pz/xQdoEMdts3Ebls2pvpKBjOHsf5m2VF6CiTmor mZNt1tgg/Do3Ls3hItRwORVJj3Aqs0B+iUcwdeRJwUP1tP6rLPLmLoAZ7T3vzNAyub0AEbc/e tE+gvOw7t5+XrO3/f2qPCEdHo52cSlmQU+b+Eb1y7WSRooHq5xqRCw/Ei6yRvpsvxtoinLwPZ hTxbIlvlQxEI0+5RVwBVkbJEV3NVLxlue78ket2i2yMRjnkzTpJro1g3zuoooejfSPSe7cDnB 0XBctmSnPVoLTAYEqs4jHxk5n3kGR3Kv9ZDxOpWU+qjBLCqf9l9rRIc1mOkzdmU8V6Vu46gva 3n7PnlqB7zTwoZcGP4xJH0vWy9+1rzK0WdIVe+fQm4Zu6jXKSlJ81kQ4iUFByrqNSb7z+iI6y GBQnY3s5WyCeMiOSWK4vkkIL2uE4PZmtQZ+rDSpI70yFR2s4LXeEjw1qZHMnG4Sz4749twFY7 ithQ1aEGmbTKGbh5f/x+/nKpEeI= X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Ihor Radchenko <yantar92@HIDDEN> writes: >> And there is also the case, that due to inhibit-file-name-handlers and >> inhibit-file-name-operation we can allow a remote file name operation >> for a given function, and disable it for another function. Tramp uses >> this mechanism. The general flag tramp-mode is not sufficient for this >> scenario. > > I am not sure if I understand completely, but it does not appear that > this is used often during ordinary file operations that do not involve > tramp. Don't know, but it is a documented feature for many decades. We shouldn't destroy it intentionally. If we have an alternative, as I have proposed with without-remote-file-names. What's wrong with this? You could use it everywhere, where you let-bind file-name-handler-alist to nil these days. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:25:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:25:26 2023 Received: from localhost ([127.0.0.1]:60791 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMpCI-0007if-3n for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:25:26 -0400 Received: from mout01.posteo.de ([185.67.36.65]:42283) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMpCF-0007iK-Ue for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:25:24 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 23A81240027 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 14:25:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689942318; bh=trV0crLwf1UqcRt+UAMceMZcJX4t5oopw9xIAI/9cww=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=fQbbywY9hNHWg4mLFUwEfFPdTtUIgYcv7faneB9q0b8HKVoZYiiA1VyKaIwrZX95u UHWpC/Hw8LbLFDdL9WhB+zfW1VjRDI5Nw6zmuh/WswSH5viG0S/9FuVIPKtT9VpL27 UFdrxelBVRBYvHekACdanV1QrHgTr0SLz/jWxbh3L+lzBAfy1JS7wkLuBwUiDpq/v2 x83WmLLEReXMbVgG7KPT4vATJYrSjiC0HQHwHsdU4Y5oS7izLmx9EVYwT9Q5etNnsv mKZRHLtHPhjspjumi9+F4nqXjpMSR5nVInTpOYl4tnct7BvEFfdUVvG8+cDbBjuIxS w5X7rDR3/m7sQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6pfj1hN8z9rxB; Fri, 21 Jul 2023 14:25:16 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87fs5hmp6i.fsf@localhost> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> <87fs5hmp6i.fsf@localhost> Date: Fri, 21 Jul 2023 12:25:29 +0000 Message-ID: <87cz0lmoxy.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Ihor Radchenko <yantar92@HIDDEN> writes: >> Hmm, this would mean to extend the file-name-handler-alist spec. Instead >> of a regexp to check, we would need to allow a function call or >> alike. Don't know whether this pays for optimization. > > The question is: what is more costly > (a) matching complex regexp && call function or > (b) call function (lambda (fn) (when (and foo (match-string- ... fn)) ...)) (benchmark-run-compiled 10000000 (string-match-p (caar file-name-handler-alist) "/path/to/very/deep/file")) ;; => (1.495432981 0 0.0) (benchmark-run-compiled 10000000 (funcall (lambda (fn) (and nil (string-match-p (caar file-name-handler-alist) fn))) "/path/to/very/deep/file")) ;; => (0.42053276500000003 0 0.0) Looks like even funcall overheads are not as bad as invoking regexp search. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:20:18 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:20:18 2023 Received: from localhost ([127.0.0.1]:60779 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMp7K-0007Zs-0l for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:20:18 -0400 Received: from mout02.posteo.de ([185.67.36.66]:46591) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMp7I-0007ZZ-5b for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:20:17 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 6FB43240104 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 14:20:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689942010; bh=bj6Pbz+tDJVbigiJXKSn27FKQVj2T2ypIk4qjwHBemg=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=Tl118zM9dAHlbHJ0BbDzoHuNpHaAkkr0QNcRrGsOt7cnArfror8mydG6wGgmK1801 QHJLOSE76EnHiTjjN1uqEolGAzpH4gbYkTR3lme1tRSLIj3YiHXfvfHImsPEdN5UU5 YFpVmq97Ul+nkydjL4J13ePpasH8ao9GL/RJuE2m1qChkRPQ8dMIC7R+kZfJr8Omjz b9ArkNcuKjijNAuasT6tmTqy4SqfmBHJ3pU+9a8AQQqrfhz7vDW6jCYq9FYBcmuEzu p5IlFuhyQkUnsgPThP9hTdCxqcDVY6gfLX3/u0FozTCPGtq+k6UxOYQI51h1bZN4+A fGsaAWJskFlwQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6pXn32Qlz9rxM; Fri, 21 Jul 2023 14:20:09 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87edl1scbw.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> <87edl1scbw.fsf@HIDDEN> Date: Fri, 21 Jul 2023 12:20:21 +0000 Message-ID: <87fs5hmp6i.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Michael Albinus <michael.albinus@HIDDEN> writes: >> Or, alternatively, may we fence the regexp matches in >> `file-name-handler-alist' behind boolean switches? >> I examined what the actual handlers do, and I can see >> `jka-compr-inhibit', `epa-inhibit', `tramp-archive-enabled', >> and `tramp-mode' are used to force-execute the original handler. If we >> could make Emacs perform these checks earlier, the whole expensive >> regexp matching phase could be bypassed. > > Hmm, this would mean to extend the file-name-handler-alist spec. Instead > of a regexp to check, we would need to allow a function call or > alike. Don't know whether this pays for optimization. The question is: what is more costly (a) matching complex regexp && call function or (b) call function (lambda (fn) (when (and foo (match-string- ... fn)) ...)) > And there is also the case, that due to inhibit-file-name-handlers and > inhibit-file-name-operation we can allow a remote file name operation > for a given function, and disable it for another function. Tramp uses > this mechanism. The general flag tramp-mode is not sufficient for this > scenario. I am not sure if I understand completely, but it does not appear that this is used often during ordinary file operations that do not involve tramp. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 12:01:44 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 08:01:44 2023 Received: from localhost ([127.0.0.1]:60721 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMopM-0006zx-7M for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:01:44 -0400 Received: from mout.gmx.net ([212.227.17.22]:42975) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMopH-0006z1-38 for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 08:01:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689940884; x=1690545684; i=michael.albinus@HIDDEN; bh=QcLiGo/W3UV7LcMJBeDurtVoqkz/s2rAoMIjLHKgiA4=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=q07kKQXwbTdynr2O+e45L3D/JdBh3VFHx51UYtkudsGqntl9TIPSv2ZLw63/rf0h4d4ldn6 +AY6ms8f5S5f4zd0QAhZ7R4g4QxRJxXuXZS3/06KTdrU5l/sJCKc7gKTjlOSFKdVAwXUQbY3U 6iLchaIHBUO/LstYtm7ow/iLOCmNbKIV8FyVtS9Qq85F2c2Xvsa7NCAucd0MeThQeeZSHpJrl Oe32Qr+wCsRC2c4dQ0k8tNpMawXBB11SKuO+nDqCYoE9MWvWCp46kUXIZggrr3BA7mjf4hhue Oa0rfNJmAkJHoNBkHXuV/oDgtp6HgAbg5yB7JPvQBAMcSqDl/PoA== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MfYLQ-1ph9LQ0VCq-00fxI9; Fri, 21 Jul 2023 14:01:24 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87lef9mqio.fsf@localhost> (Ihor Radchenko's message of "Fri, 21 Jul 2023 11:51:27 +0000") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> <87lef9mqio.fsf@localhost> Date: Fri, 21 Jul 2023 14:01:23 +0200 Message-ID: <87edl1scbw.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:f1YXj636mHpw+6Y3TVPGruGdz/kNXGxBZDGfKcTcGg7LHenqwbr uTHXPkMoPumPqg/pPe8NbZE1a/R4ucxYMpYdgOq36vLn+WEJED3LiYE46DOHtBfFKIbwFl+ AdRij82V7EMAb3PtykF6xKElGP3Asqhkde4iNgkufrT5c2Q3aRKqsjxT6CSHSXhtIAXlAsL nrCdVP4hDNo4MwezePtiA== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:zFm1Vi+vbhc=;V1XnBEtLh6oAsdcRPABYMcfUwTQ PONtNRjxhtTYLecq2k+Z/wmzkD3Slr9YM1ZtK6UESYP/rZEMlc/I+/9R6/AS+N1msOD/YWojk 0qEJmTa/usUDsEIbmml3toEijdELvZJZuXm9d59FS7M7K0x08cZ66sQTZUKPqNCQ5jDV9jrUd 33PFmoc9rIeFiEV8G96CQOBBpJHAb5ell3RJN6XchOqG4LYw3tOj1y6zMpDjOSUCoJLvX2UHs kY95m64Zlq10/J80m+gkRH37p+Tl3piMd6HI3xQiG3FHja1YRyIbXbzauXx4o9ovYuoGQCy3Z apie3ZIDYe9qBF9NBK/U4es8dgoE3MGfdaKPF4ff/CK2mgAFj6W9N4WRUIK+82DGFcmji7d1h MO/6RNJS4tRl+/Ufm3dvl3UMl/2nUZPWRbBL5sFodTQUwtO54kO1rGjkCPAx6m3esarKTkj3h +0GtCPOnRhRin9YoYB5MDw0ZtGy04o8sTX3SNhkNZ4RB+mx/D1+Ia07KPP6g9vAOUAch5gsuM s+ol0tiXReR+34ud2CYoKgn2p/ffeucP7VEKJ18ZElLvBV9y29rQntoi6wIk3vDo4ZFXRlok2 Ewpm3LTZjxYjEDpEltYA+7OOJtAUv5vRKNeAaTi7uCPLP0jagCyjxBe5/eiAG9oBaUXuguc9N 0Qi2iWhDprx7gUqtXVQ14Rgn01k8X+De6wkHJo6+LQmFQElP2w/jnBT4aFvrOQm/fOUM6Yg/N frTtLTs2Tj4R+zcKmQ+afimyj/tYJYqvZHEDLnfvXQywaz6P3Ay5lgIS/YkBi0S+3vLY8gXia RqRPhNQnNEhm3YmnE0dawzRNJYohp6OsOCA1O85DYrSHDKbuVmkviY+f2FakcjzKvZ4gMNkte PPoAr28oqoKC0YG9ji3FZkl0iHNqwr9+RA+2VKwneLB5AeY6En7IdOmobnRnvOIYQVlHw/0KN rYa/gg== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Ihor Radchenko <yantar92@HIDDEN> writes: Hi Ihor, > May tramp only set file-name-handler-alist when a tramp command is > actually invoked? It's the other direction: Tramp is only invoked after a check in file-name-handler-alist. > Or, alternatively, may we fence the regexp matches in > `file-name-handler-alist' behind boolean switches? > I examined what the actual handlers do, and I can see > `jka-compr-inhibit', `epa-inhibit', `tramp-archive-enabled', > and `tramp-mode' are used to force-execute the original handler. If we > could make Emacs perform these checks earlier, the whole expensive > regexp matching phase could be bypassed. Hmm, this would mean to extend the file-name-handler-alist spec. Instead of a regexp to check, we would need to allow a function call or alike. Don't know whether this pays for optimization. And there is also the case, that due to inhibit-file-name-handlers and inhibit-file-name-operation we can allow a remote file name operation for a given function, and disable it for another function. Tramp uses this mechanism. The general flag tramp-mode is not sufficient for this scenario. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 11:51:28 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 07:51:28 2023 Received: from localhost ([127.0.0.1]:60717 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMofP-0006iv-W0 for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 07:51:28 -0400 Received: from mout01.posteo.de ([185.67.36.65]:54241) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMofL-0006if-4Z for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 07:51:26 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 228CF240027 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 13:51:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689940277; bh=QC/THAI+BgEmnqDXoXb5znOC5j0bqRw3NkxABy+pxh8=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=EE5HJTVIOknipwh0UI7mtTKZxZkORNVUkS/VftoMudXbXvtAO/QhLjBH16ejh3a5r jxgK13sOeJ06fJ3wic+pxUtCYRVQkwOvdz2wHgZTKncn2h2vwRDh0nOp+6KfVP7KRI C50anp3HwWo+OnLpxHOgHhaSf0TMBdkJpZ0TSvcUhLCJ7uZ+Z+ifq6AInpCTiNhDQh i3BGgbxjZU4V2xjc83wBthhv8CqommcApqz07IArpkz2MP0DL9ariBN0KuYGfPpnh3 AnJsl2q3TYcEAnj0ncbseLIWdcGULA8+wBCGL44t26Cg4xK+NAAsPvnEvGbCWV9XFu bJeqUyWATykqg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6nvR6WYPz6tsg; Fri, 21 Jul 2023 13:51:15 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <874jlxebz5.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> <874jlxebz5.fsf@HIDDEN> Date: Fri, 21 Jul 2023 11:51:27 +0000 Message-ID: <87lef9mqio.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Michael Albinus <michael.albinus@HIDDEN> writes: > And this is just the case to handle it in Lisp code, with > without-remote-files. According to the last Emacs Survey, more than 50% > of Emacs users don't use Tramp, never ever. But they must live with the > useless checks in file-name-handler-alist for Tramp. All of them would > profit, if they add (inhibit-remote-files) in their .emacs file. May tramp only set file-name-handler-alist when a tramp command is actually invoked? Or, alternatively, may we fence the regexp matches in `file-name-handler-alist' behind boolean switches? I examined what the actual handlers do, and I can see `jka-compr-inhibit', `epa-inhibit', `tramp-archive-enabled', and `tramp-mode' are used to force-execute the original handler. If we could make Emacs perform these checks earlier, the whole expensive regexp matching phase could be bypassed. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 11:33:05 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 07:33:05 2023 Received: from localhost ([127.0.0.1]:60705 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMoNc-0006Ds-KX for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 07:33:04 -0400 Received: from mout.gmx.net ([212.227.17.20]:47715) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMoNY-0006DM-U1 for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 07:33:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689939168; x=1690543968; i=michael.albinus@HIDDEN; bh=4AYvM/GxjOD43VQAQNyrJyHFc1yV12mOYsZPQDtWwxA=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=ai3V0ZrQIx0/SnVyFvU59KGnk4eUNSvAijrWzSHtxH6CxLkMpB854/rXytVqiVlfuhWXkJy kPy3cCC5MwtgS3TlUcnZCjue4JHvNpoOgC4KHbPU//GKEI14+M7p/Ph0P/NhCvdVDB42anu9H dnShr08TT7Tnj4YrtBGsXsbI5lIXeQ2Pz6tBxwX3zVXG/MJdRRwNe1sU/BDXLANN++90pSk0i zhCT8LUeudahD2vdzbU1dYp4GJdPMQVsGA+o6NTiojnDsKKHgDYa/QLhWeqxEBBhZiUlg+r2N AQ5zwEQI0UhPmV2y49srsqKu/mpKmYgZ3G4VtQVCxWpigfaXh3ig== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1M5wPb-1qFwtP1GBv-007Rl4; Fri, 21 Jul 2023 13:32:48 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83edl11qzn.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 21 Jul 2023 13:46:52 +0300") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> <83edl11qzn.fsf@HIDDEN> Date: Fri, 21 Jul 2023 13:32:46 +0200 Message-ID: <874jlxebz5.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:jujrNfCX3m1C2xpqCl2zIk1mHl8y0EIBWeUWbAbvgQZhtR5RRZh P/RDPfFxus6W2HDer1okrZLfHb0JvVJ2IaE9WLbAHb5rbqoUiRVgMQ1G9OeTuyHxeC4k4Ba yBiLJRqILSTL0/z0UuxwisdFZQpCT2Bs1GBoiJLxtBxnTijTjLOzl2jtGiorCuRtbIlr+dg ljtnuCmOn1AMUYQSQM7AA== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:GdV5zH+8PQo=;8nOyLUXpG/pkB5OvDgIGTFKi7lw A/PnKfomnUQoVeZAL8kXwjrgI1jqr0diErC9R3dmshIfuXq4g/StKewKuc0TZz1pvVByZtloD qwZ+njcweXF2DWI0AJNagZu26htz44t3/CGZ05meswMHR3K3JyhDEG5jOgF7A+xEIVVzEPAS5 64828y3hEjROAh8T1gjdOSVdUKdcRERCt/jGTM2GFEOvi3rktRN98Y9cZTd1fC7qA1lbCYYsV YUWydDKNwqbK0AGlBd0GSDB8rQoVNkK5pe/Gv/gPjjiKy/vQNdRBAOlt2zjLR+wRmwxNXPbAH Vkr27MvPIchhULtooXKeoZS8/CL2+gMxFrYM9REUqyAhb5GPjUtYXm+SQM5/Hn3gQDBGsATW5 S90ZYG0MZK1yz8pZ8FBbQULie2XDw5lYC06ph44sk0lY4NDwLpyzyOLyxSJdEdwtTK9+pEr8s w9Utj3FUNs/Aq64TP/A6xScJuofN6yiSig4mzhbqf+SvwMlHvDfTx2e4xPHbVkbLaVNxjVYCJ 8CXykBa8bpyygAOzmYa3q9Ayiu5Y7LyjX9jkI0Xt3aTehhBhaZVxL98TRvOHqEszqWQTdc+8h E4fCfXX8x1p8nu+mFCD5DAfiAgRBbB+c61izShCYbw9KsMODsfaEts1KKu3wJ2XA78Bc/jjFg +rbziAlSf1zAV2g/JI4HOzYTxsfh4qWOnWCxveC407VXX9f1XJgzMTaceC9PdSA37Vxi7A67H LM6T9pZU32CgsV+GIIO4AF3WlCCFR6qXgsLkBL+eY320sW+MNVLm8S+nRPSfSZaFV3K/17GyG 3WlamMPWRXuUyqv8MTFLX+e4p/QAhc5OLfnR6Xz2icPSMw9qeP3xho496FnRsK44ZKmmciByk 4PkOKRYqgoge5Z+ZtzDF6UlAxfJNCHcm/KHZwhO7puwR1lYIh6A+jmiSIpsMU9/W8BWbJAD1P YpiPz3Zha2MeGDFK3zKwlxNnZsg= Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: Hi Eli, >> However, the fattest regexps in file-name-handler-alist are those for >> tramp-archive-file-name-handler, tramp-completion-file-name-handler and >> tramp-file-name-handler. Somewhere else I have proposed to write a macr= o >> without-remote-files and a command inhibit-remote-files, which disable >> Tramp and remove its file name handlers from file-name-handler-alist. >> Either temporarily, or permanent. >> >> Users can call the command, if they know for sure they don't use remote >> files ever. Authors could use the macro in case they know for sure they >> are working over local files only. >> >> WDYT? > > How is this different from binding file-name-handler-alist to nil? > Tramp is nowadays the main consumer of this feature, and AFAIU your > suggestion above boils down to disabling Tramp. If so, what is left? jka-compr-handler, epa-file-handler and file-name-non-special are left. All of them have their reason. And there are packages out in the wild, which add other handlers. Like jarchive--file-name-handler and sweeprolog-file-name-handler, I've checked only (Non)GNU ELPA. All of them would suffer from the bind-file-name-handler-alist-to-nil trick. There's a reason we haven't documented it in the manuals. And this is just the case to handle it in Lisp code, with without-remote-files. According to the last Emacs Survey, more than 50% of Emacs users don't use Tramp, never ever. But they must live with the useless checks in file-name-handler-alist for Tramp. All of them would profit, if they add (inhibit-remote-files) in their .emacs file. Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 10:46:38 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 06:46:38 2023 Received: from localhost ([127.0.0.1]:60642 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMnef-0002NQ-N5 for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 06:46:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36474) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMneb-0002N8-Dk for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 06:46:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMneV-0003zn-O5; Fri, 21 Jul 2023 06:46:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=mA/4X9NeX9Pf0zYNgafEeJTngSKAXdHVQV28bgEHkFU=; b=guEr5RE5j6aT JG2kuUwi/1TKwMsnCjHxvryAsdY/cd9v1UprMtJ+zr5HawezFipSYvFx6kdE56VlSyQ/Mg0gO6UmD wNJmsf/8X97fDsmOKM6C9TdlXaGPP8vFAfqQm0lR8z7PmrdNF+HG8vBgr9Ho38JP4PjAjtKj8ofqC O7zJd1szsTIvKt4rTozMf/6/diWkYM7hBsifP+57zgmFOgjG4jgmeR7/8G/400mwXjfyb5oG3au4h 4ryjg9NqcwtfE3SJWi3iKp7vuvg1RpzzUiF8ZrIOTIBs8RJAdW8F2fKLg2Y+jMjGJHx3Q+j4B/0+1 Js52vLroPFIGJpuxirLZuA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMneN-00084R-3C; Fri, 21 Jul 2023 06:46:27 -0400 Date: Fri, 21 Jul 2023 13:46:52 +0300 Message-Id: <83edl11qzn.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Michael Albinus <michael.albinus@HIDDEN> In-Reply-To: <87fs5hemi1.fsf@HIDDEN> (message from Michael Albinus on Fri, 21 Jul 2023 09:45:26 +0200) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> <87fs5hemi1.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Michael Albinus <michael.albinus@HIDDEN> > Cc: Ihor Radchenko <yantar92@HIDDEN>, dmitry@HIDDEN, > 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN > Date: Fri, 21 Jul 2023 09:45:26 +0200 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > Disabling file-handlers is inconceivable in Emacs. > > Agreed. > > However, the fattest regexps in file-name-handler-alist are those for > tramp-archive-file-name-handler, tramp-completion-file-name-handler and > tramp-file-name-handler. Somewhere else I have proposed to write a macro > without-remote-files and a command inhibit-remote-files, which disable > Tramp and remove its file name handlers from file-name-handler-alist. > Either temporarily, or permanent. > > Users can call the command, if they know for sure they don't use remote > files ever. Authors could use the macro in case they know for sure they > are working over local files only. > > WDYT? How is this different from binding file-name-handler-alist to nil? Tramp is nowadays the main consumer of this feature, and AFAIU your suggestion above boils down to disabling Tramp. If so, what is left?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 09:14:39 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 05:14:39 2023 Received: from localhost ([127.0.0.1]:60554 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMmDf-0008Cz-8o for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 05:14:39 -0400 Received: from mout01.posteo.de ([185.67.36.65]:43201) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMmDc-0008Cl-MN for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 05:14:38 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id B768C240028 for <64735 <at> debbugs.gnu.org>; Fri, 21 Jul 2023 11:14:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689930870; bh=Jv/l9yzMWUP+gQ7eH2Q9XgWsHG+p53GrjjKRe8nGoJM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=BKXVeIQIVULAFObnXo7+g0vFgVMtvVo6E991rs6AVqIh/9GZvmhw6xRBc+FFjMtp8 1HyHp90CTfyazVM5wdtuWm+pGIzCkgaCXg1VhMX2GuDwqylWXuOfiyBJoLlt//6is5 WQVTWRpyiYxDZJIlyKT3oTdvDpJMi+9sEfvX0/L31tujIuTH+1TVQwZboivwZANh7n nInpYhHTPW8xpy+CypkTjtFTBjspYjuCgIfsWRsYXCuMMsZA8/gsHr/lSzAzV7ssU6 obOd3zrcS51K5xMULkgrf85Ub1I6BgglHZyzrnc0nDvgbE885WkEYkwg7CzntI/Y+r Kjmkd8XadqGHw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6kQY5VV9z9s06; Fri, 21 Jul 2023 11:14:29 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <615e8997-049b-0685-07db-c87dbbc0ac85@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <aba1400a-3d00-44f0-90f1-e5b549de8ae7@HIDDEN> <87r0p2po29.fsf@localhost> <615e8997-049b-0685-07db-c87dbbc0ac85@HIDDEN> Date: Fri, 21 Jul 2023 09:14:41 +0000 Message-ID: <87wmyt62ym.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Dmitry Gutov <dmitry@HIDDEN> writes: >> You are likely right. >> Then, what about applying regexps manually, on the full file list >> returned by find? > > It will almost certainly be slower in cases where several (few) ignore > entries help drop whole big directories from traversal. Right. Then, what about limiting find to -depth 1, filtering the output, and re-running find on matching entries? It gets complicated though, and the extra overheads associated with invoking a new process may not be worth it. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 07:45:43 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jul 21 03:45:43 2023 Received: from localhost ([127.0.0.1]:60466 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMkpb-0005kh-9B for submit <at> debbugs.gnu.org; Fri, 21 Jul 2023 03:45:43 -0400 Received: from mout.gmx.net ([212.227.17.20]:42393) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <michael.albinus@HIDDEN>) id 1qMkpY-0005kS-2m for 64735 <at> debbugs.gnu.org; Fri, 21 Jul 2023 03:45:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1689925528; x=1690530328; i=michael.albinus@HIDDEN; bh=LaAANdTiOwyvYVLhZ/U4cpjZeiP13y74Zb8Gr3kAvQk=; h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date; b=jjwiVmxbbF+qNxyEoocizZ6zXga58d6u7hr97HrZmWeV/uaMRK9bhC4B2UKwhG8LHW4ajUc ftCYlNiOkIR35fl4Cg5EUa1CzM8IZwgGnEbJXj/Yo2QFR24M/BanCBKnV+q+drz3s07cR/0/6 3wgdVeo1E1k7TmzifJVgQFXrl9p6uYZIbUxMpHTtNZJ26e+UbNNv0QmiXjGDnDrPbbeHcUAYk rwJ07niWHY15Dw1HS7EEs7VP5HK2CEq5vgK+bZaTo6Oeu8xCigncmMKi568Iq1CW7L1oco8ZC O23I3TqvWlnEJ2FDm8L9DZZy5Gw+BeU4X83Lw3zh86eMK1+B077w== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from gandalf.gmx.de ([185.89.39.27]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1N5mGB-1ppdop24Wx-017Fm0; Fri, 21 Jul 2023 09:45:28 +0200 From: Michael Albinus <michael.albinus@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83wmyu1l1k.fsf@HIDDEN> (Eli Zaretskii's message of "Thu, 20 Jul 2023 21:43:03 +0300") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> Date: Fri, 21 Jul 2023 09:45:26 +0200 Message-ID: <87fs5hemi1.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Provags-ID: V03:K1:b/SmY+B/DfjMtgpc7w5QOzZlx0NVKW5zW8c/dmgIkZBy4x4+fsq BdTJHxb7trcu6HjJrQXDtTHkbzetjnKeS+0Ppa5/D76IHljQH09zzeZ90umJe5QXIxC6rZF ppkglRAMejmX2XiMtA/nCv9G9ntNgtIlNxKsxjjwPSGKfBSYbtcJwl9Rd1IsRwUj7eLaxk6 Do7hOnudHPLkhifHOzZzA== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:4306727PI8M=;TirqMR+X4Crd1a37T6jhMaRjden 4HgVoO3lrTFgVOzGSMVw5SmCc00vwzy1CkXDYrWW7bS4LVmSmRzGHp7SP/+EJFvDHb+6BsxiU YsiMDgyt7+4t7mFQucZhlQdNXmXvIXnZIbFdTgnyER9tLSVC4jqglWn+a8ZDjD8Ztq7n+/mBb 893eS+P8KAgO6fCo9ZvYMaOu9q4/0j/zvvfND/B3Vvb/6MV3smYhO/p059WzSNUS2sf99f0kR tm375msFOmFm+ulZfuCSTrraDfT+3QnDSOnMNxWY1zjfEKgSIhE1BffeMfrjKBWsWh+9MS5u+ jo15TiebbCBqQiegRbtBSO5BjQWT8sL0yq7RKqaA4Ag72FIevG6eHONs68Qqp9ovDCg4oYULV J0+tsH/uPJlDFaOleiCup+ZYO/2+cDdKVX+1PRqnhyPdhWeDUTjERTOxMx89cje1VJukeg2/h /XY23KoUGG2EcrxpQMs98+hyw5e55d8bhdX8HyVUh3C1QllM2rGCSo2qsyUYPSkpCm0MQwhW6 hjhpGmUFty50vGcG32KzyelOj/6eVoqc/yR0BAKRVRcO3wMzKoRByQBvQxB5CdppuBAJjecrA Tc+gzXEgoUDZZQpMOtV4RZcU2NJIOELbxusAE1jP6+I+qnzPGEeFXnLPKEvmo/L+HQwmLGJq5 cf0n0hv+vSC1BLBC6Ke0RjNnIF8pMDF6u6sITmeWD0B76QjwSLTqcWdUgbCcSpFLAOnGO57sw 6RKLdX+0Jx24G80WRltG35DWdpZXqP49x4faxpaM16dI2m/2fMy+ScOZ4Z4QNL2p1FIZQT487 m0RRDQKfwdJB/WfDNf1IJe7Xi1xsPv6NUFSlzHRxKTtDu5e5PuIh/zM805pJdm560hcZ3AlOw glTwHw6oszNNVtO2s82avdCQpcmIYomqKzRfS8gl20J5eB+Wc4PuOV7kn2Tco88yL8ZHWtdh3 kREWyw== X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, Ihor Radchenko <yantar92@HIDDEN>, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Eli Zaretskii <eliz@HIDDEN> writes: Hi Eli, >> Disabling `file-name-handler-alist' cuts the time more than 2x. > > Disabling file-handlers is inconceivable in Emacs. Agreed. However, the fattest regexps in file-name-handler-alist are those for tramp-archive-file-name-handler, tramp-completion-file-name-handler and tramp-file-name-handler. Somewhere else I have proposed to write a macro without-remote-files and a command inhibit-remote-files, which disable Tramp and remove its file name handlers from file-name-handler-alist. Either temporarily, or permanent. Users can call the command, if they know for sure they don't use remote files ever. Authors could use the macro in case they know for sure they are working over local files only. WDYT? Best regards, Michael.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 21 Jul 2023 02:42:54 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 22:42:54 2023 Received: from localhost ([127.0.0.1]:60280 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMg6X-0004wG-P7 for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 22:42:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48852) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <rms@HIDDEN>) id 1qMg6V-0004vy-0J for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 22:42:52 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <rms@HIDDEN>) id 1qMg6P-0000i3-4d; Thu, 20 Jul 2023 22:42:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=Date:References:Subject:In-Reply-To:To:From: mime-version; bh=uRns8MqLFonCUBub+9Oj8dRSdji8Hd54ty2rfUcthX4=; b=CWrYhKcLOmHD mBdqCBeKUKOMy+CF4SdjFNzAJyAGNZ59jG1npS60kRNMLQVPIfLRhPWRZr+1zmusyjxBPe4cEQxru /tb5+7uKNlD1sYaVjhF9htonSMa+SNBuggYuROYo9b3bWbDFHAzDGUnSgCVRoGoQAd6ys/HsL7m++ IboE8JBIrZqXiLMlKQNoK5T9ZHc/svCk6TdgF+29sBXL7Q9tFZYHKVvkoD6oeqf27lm5/EUMgLs+s a5l7jGTA2KZPpB2PnhhnBLSItiXurE2qJN39es74acX/1/V1kMnMbnBC8Q9L4m9UBVU1/Ni8TrpNT PjuiCtG8912tPUmzjzB6TA==; Received: from rms by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from <rms@HIDDEN>) id 1qMg6O-0003aa-SZ; Thu, 20 Jul 2023 22:42:44 -0400 Content-Type: text/plain; charset=Utf-8 From: Richard Stallman <rms@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> In-Reply-To: <iermszrwqj4.fsf@HIDDEN> (message from Spencer Baugh on Wed, 19 Jul 2023 17:16:31 -0400) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> Message-Id: <E1qMg6O-0003aa-SZ@HIDDEN> Date: Thu, 20 Jul 2023 22:42:44 -0400 X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Reply-To: rms@HIDDEN Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] I will take a look at this. In case they are reluctant because of being busy, would anyone like to help out by writing the code to do the optimization? -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org)
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 18:57:51 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 14:57:51 2023 Received: from localhost ([127.0.0.1]:59905 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMYqV-0005Xk-Ev for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:57:51 -0400 Received: from mout01.posteo.de ([185.67.36.65]:47081) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMYqR-0005XS-4x for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:57:49 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 359F1240028 for <64735 <at> debbugs.gnu.org>; Thu, 20 Jul 2023 20:57:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689879461; bh=lwc38aSJm0wtB/Mnc62i8GtLe7/JMZwJTG3YpzRVRdM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=AlX61wIzULoJDplBECgtlDqpoggQcvaLD7nOihCTX713WjzOSxFB8nbama740p+Lr XZ+JzbjeshmZEySiEE9Miudzi8oAXsKVpkxKbBfpCOvMfomKACwTE3ayo5bMEWSUSj s4BEit/i2/Tcih4DKkavwws91Sx0kNouwOrIc2WSbVaCf0ruW26v3myn6RfA6gYqDo UycodDkfT7KONyxic/1n6J+Yn4HhciWJCgPSE0Ie+IUJahyiRTLhkpnvanH3Wmp1Oz 04MHsl3O6ewpDjjb4wOb/h/Dzg3sy5FT2ZwSSLQP22anPbamaI+QL0cdzeZVPC6tU/ dFRNb1EYJNPDA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6MPw2shfz6ty3; Thu, 20 Jul 2023 20:57:40 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <83wmyu1l1k.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> <83wmyu1l1k.fsf@HIDDEN> Date: Thu, 20 Jul 2023 18:57:52 +0000 Message-ID: <87ttty8l73.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> Disabling `file-name-handler-alist' cuts the time more than 2x. > > Disabling file-handlers is inconceivable in Emacs. Indeed. But we are talking about Emacs find vs. GNU find here. In the scenarios where GNU find can be used, it is also safe to disable file handlers, AFAIU. > ... And I suspect that > find-file-name-handler is mostly called not from directory-files, but > from expand-file-name -- another call that cannot possibly be bypassed > in Emacs, since Emacs behaves as if CWD were different for each > buffer. And expand-file-name also conses file names. And then we > have encoding and decoding file names, something that with Find we do > much less. Etc. etc. expand-file-name indeed calls Ffind_file_name_handler multiple times. And what is worse: (1) `find-lisp-find-files-internal' calls `expand-file-name' on every file in lisp, even when it is already expanded (which it is, for every sub-directory); (2) `directory-files' calls Fexpand_file_name again, on already expanded directory name; (3) `directory-files' calls Ffind_file_name_handler yet again on top of what was already done by Fexpand_file_name; (4) `directory-files' calls `directory_files_internal' that calls `Fdirectory_file_name' that searches `Ffind_file_name_handler' yet one more time. There is a huge amount of repetitive calls to Ffind_file_name_handler going on. They could at least be cached or re-used. I do not see much of encoding and consing present in perf stats. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 18:56:21 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 14:56:21 2023 Received: from localhost ([127.0.0.1]:59899 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMYp3-0005VY-0O for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:56:21 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:36627) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMYp0-0005V6-82 for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:56:18 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id 7A8A33200BFF; Thu, 20 Jul 2023 14:56:12 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Thu, 20 Jul 2023 14:56:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1689879372; x=1689965772; bh=sUqPPrSfAAds49KCIzaIu4SWGxKzBxO6mq7 kYqVW7HM=; b=M+NQ7Ot8YD1xDWcoiSZRB/mLdj2xA3fTx9j78kUiAyT+cc/9V7+ rvmj4dmiQKCFFb0aNI5deUkf0/oCI83YsjGOawUpoElPTECbzPiJx+MEi01O4k5K pamOia56mkmt9X2RWe1WUEu4fecXKt24HkywiWAHZFij6YhLicaNICG+JfYSgIhh ATVkozciPlbDw+sLL+sM8H56DYIkGU+xB1kCyexGv8zK2C7B7WCKzhTZaDA31rhp vtr9FV/aVPGfvQvGTrYDhaH9gM6LM0H3ucMgJ7n8pm1NZ5yPG3tN/a2jmEKBDCUW ITPNUCQ+eDuQKVylRCA5ZRqOsB9buL/WPLQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689879372; x=1689965772; bh=sUqPPrSfAAds49KCIzaIu4SWGxKzBxO6mq7 kYqVW7HM=; b=WJdQfUeVHFLbrqKqos9mIkaP68l1gRsH/9OKWLcGOfzalOhejIP Ts9hXwi+6t8phq2Bz3MgTs2W0BtMufBchH/5gf8rFPoFAMFXvUXP34jnp7hfB829 jU3QH73I9Y8blvdHupCxZFjGPZypK3EbyUyznRvTI3RQq5oKeXhEvgkOle4yKvet fVjR3E2hb/a2/ur6ZC3Edcf00c7wMWr9MqwjrrbYeB8v0rknMetl/MJW/SSxB7Ev 6z37QeqyVYT/2VYPtfyRRX6JmDbEOq3zceSUlARd6QYjuqb+UVicYCTdCNVP4qU2 As7LrzNYcHJAnFwZthadJuAiOaDSSrzpqDg== X-ME-Sender: <xms:S4O5ZPNckgOcBmNU8QeKBFuWNKVhO0EviIIfHL2tlde59Uvmamg5EA> <xme:S4O5ZJ8BzraAjBNbMBtyJF230BphmZVdT89SfmvAwY9rQUbMlTmGkoQLDyFhR0vsS q1afmobwHqXJPzRFYM> X-ME-Received: <xmr:S4O5ZOTiH0NEtJRaGD6oJWkKYY509SPGFRtsks21UXdBzSOaEKLEPkGiyHRkrzw> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedtgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:S4O5ZDvff2GtIwpOMfViplhaEfxyKyowp-TqErsLl0KbdMKvdOPmRQ> <xmx:S4O5ZHfSqJZhgzXy4JKPMln8Am9DfuxsnPYB6B3g2EBWxwUu7dxQmQ> <xmx:S4O5ZP0ik4tNpwSPOMPl2ad6llIpQGRuiX0DS_PxViWdfz0ye-fMCg> <xmx:TIO5ZPF2_wADtSI1wVtR0BnKXAraTzt2_kGaLg5S_E2RNOTDmnBYNw> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 20 Jul 2023 14:56:10 -0400 (EDT) Message-ID: <615e8997-049b-0685-07db-c87dbbc0ac85@HIDDEN> Date: Thu, 20 Jul 2023 21:56:09 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <aba1400a-3d00-44f0-90f1-e5b549de8ae7@HIDDEN> <87r0p2po29.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87r0p2po29.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 20/07/2023 19:03, Ihor Radchenko wrote: > Dmitry Gutov<dmitry@HIDDEN> writes: > >>> And things are not as horrible as 15x slowdown in find. >> We haven't compared to the "optimized regexps" solution in find, though. > Fair point. > >> Still like 2.5x slower, then? That's significant. > It is, but it is workable if we try to optimize Emacs' > `directory-files'/`file-name-all-completions' internals. > >>> I am pretty sure that Emacs's native file routines can be optimized to >>> the level of find. >> I don't know, the GNU tools are often ridiculously optimized. At least >> certain file paths. Sorry, I meant "code paths" here. > You are likely right. > Then, what about applying regexps manually, on the full file list > returned by find? It will almost certainly be slower in cases where several (few) ignore entries help drop whole big directories from traversal.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 18:54:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 14:54:47 2023 Received: from localhost ([127.0.0.1]:59891 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMYnX-0005Se-7L for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:54:47 -0400 Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:42967) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMYnT-0005SO-CO for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:54:46 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id C871C3202AEB; Thu, 20 Jul 2023 14:54:36 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 20 Jul 2023 14:54:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1689879276; x=1689965676; bh=jmu7/QfDNUt5SDBOBPFWjQRSk/qub2wO593 n2iePy0E=; b=cf6XbX662hXkX442wPgZ1cx4N7CgTaV42f2E0C7uDrEODKm0svp CHvlI28ED+wftitd+JFue49DQyIuCwxStee4PFkgU1N9QxP/Hn5iLkSNURelPhzJ /U3tvBm+CSf2b908wmdr0J3j9Dacf54OikLjyVX257n+ZeNPBfasIrKVJSMn/ih1 wsYjd6WEfcM8JYc7wWdGZT5xqnGxKEH3e/51XBUEK5LgLLRBYoAN4igJSmrlvsfP 320qz368nhNcvJlG7y2EL9G6YjfofcwcLMeuHct4MZs1jRXjrzuW+veXesr5wpyi pL6Gj8E1qlK7OSRGIYX9bU+qRUTaBskTbEA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689879276; x=1689965676; bh=jmu7/QfDNUt5SDBOBPFWjQRSk/qub2wO593 n2iePy0E=; b=NXyEf2CKwbKnGIHmKhlqQxy+v2O4CqWBK1LfjAXNWJW36uMhGHf 6KTIStziOoBq37qK2wYLVpkj3/IDZIS7SyEIKs0uIo6WWn8I487O3jXqsuq9QVbi qY2Zca8kTlwiVa8/s/kHSClP0Ijnxsxj2IBTCf1p34Gk55UAP6NxLKlh6SFrDLoE sUNnGKuAwyiTG7ZZR5XSMYiAPVACg7gMAumRqCSXR3j01adUt5wKaNq1WT0Hscuk TeDJc0prnb9w1ZM/pSryjYa9c2B/LUba323FEeATvF6MD/h2pUk8scue+UJpmWKf 1fko45n6+Z7KCeRtmRVPVspZ9gTVmudwqKA== X-ME-Sender: <xms:64K5ZIMysOxa_qPrZnSMWBL24ZMHDGqwtzykfZC_n0rAJkfYkwlEmg> <xme:64K5ZO9PIYcipKAcql7FsBbTYxjAxXDa4cDxTs-xBRy7XpDDfFvm8tChv1_ms7uT0 5A4-JwUJOHijOXKNac> X-ME-Received: <xmr:64K5ZPQatGDnkCzM4QOEfrzf2iw_UQYNlMmTN8952FA4d8GI-1UnJMLlJAZx6vw> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedtgddufedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:64K5ZAseurLoQoanAvpcSCmKIhj1aq21qiEVw3G0WdnYOTVrH4yhvw> <xmx:64K5ZAdxpMhIZ5ccQpJsGXrJEMl8fSehbkPIqT6wpSpGegqH1EHx_A> <xmx:64K5ZE0G6_DxtohTAvwoQO61wJLPyfVmoTv2sPov3fCR2Vdqq8gvXQ> <xmx:7IK5ZBr4M2v1PKA9tyX_Oa4oD3AZBjy1krY3h-q4__TOXTBZ_W1u6A> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 20 Jul 2023 14:54:34 -0400 (EDT) Message-ID: <3a33eb87-4f1c-efee-3d2d-d3ae2c3e7e4b@HIDDEN> Date: Thu, 20 Jul 2023 21:54:32 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Spencer Baugh <sbaugh@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <837cqv41ob.fsf@HIDDEN> <87mszqixhh.fsf@HIDDEN> <4d4f029f-f32f-13df-ffc3-3952d62d8bb3@HIDDEN> <ierjzuuwvds.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <ierjzuuwvds.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 20/07/2023 16:43, Spencer Baugh wrote: >> That's only a problem when the default file listing logic is used (and >> we usually delegate to something like 'git ls-files' instead, when the >> vc-aware backend is used). > > Hm, yes, but things like C-u project-find-regexp will use the default > find-based file listing logic instead of git ls-files, as do a few other > things. Right. > I wonder, could we just go ahead and make a vc function which is > list-files(GLOBS) and returns a list of files? Both git and hg support > this. Then we could have C-u project-find-regexp use that instead of > find, by taking the cross product of dirs-to-search and > file-name-patterns-to-search. (And this would let me delete a big chunk > of my own project backend, so I'd be happy to implement it.) I started out on this inside the branch scratch/project-regen. Didn't have time to dedicate to it recently, but the basics are there, take a look (the method is called project-files-filtered). The difficulty with making such changes, is the project protocol grows in size, it becomes difficult for a user to understand what is mandatory, what's obsolete, and how to use it, especially in the face of backward compatibility requirements. Take a look, feedback is welcome, it should help move this forward. We should also transition to returning relative file names when possible, for performance (optionally or always). > Fundamentally it seems a little silly for project-ignores to ever be > used for a vc project; if the vcs gives us ignores, we can probably just > ask the vcs to list the files too, and it will have an efficient > implementation of that. Possibly, yes. But there will likely remain cases when the project-files could stay useful for callers, to construct some bigger command line for some new feature. Though perhaps we'll be able to drop that need by extracting the theoretically best performance from project-files (using a process object or some abstraction), to facilitate low-overhead piping. > If we do that uniformly, then this find slowness would only affect > transient projects, and transient projects pull their ignores from > grep-find-ignored-files just like rgrep, so improvements will more > easily be applied to both. (And maybe we could even get rid of > project-ignores entirely, then?) Regarding removing it, see above. And it'll take a number of years anyway ;-(
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 18:42:39 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 14:42:39 2023 Received: from localhost ([127.0.0.1]:59878 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMYbm-0002YM-QG for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:42:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58592) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMYbk-0002Y6-9K for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:42:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMYbe-0001xF-4f; Thu, 20 Jul 2023 14:42:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=UAOLTQFKlvjSNejI7YfyNhtOOVwoCwqhzecvWxmLM9E=; b=pEBFyNdjek6+ BEev7CPSPrwq4Yrw6K51pmI6LHBdHMGh0FfSCDiCGB2Gc1UEgDAILNbxohA6AkCh6ohIzuM4PirJN ggBNEuguQ1zMeRWwFyG+kH0Q/sA6yvjbFwtJ2akMUg7DHKKYfuONS89VM0Q8qqQmNVem7YU7ptJzg OAyQPrQ1NPAR+EeRensTgMV5EDh837vvzlekn99ZtC56NcQqZ1YaNJCsrAIOQ4WcBwxXMZgtQzMXv BOvBBTivQqQwb90c6x043IArI7wNk81Dxg82pnhJXmJe0fCWuhM/3+z6aQeWXv3OSHsTtIjE9PHSY DqkGPYeZKBqj3G84jIBZXA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMYbd-0000pN-H6; Thu, 20 Jul 2023 14:42:29 -0400 Date: Thu, 20 Jul 2023 21:43:03 +0300 Message-Id: <83wmyu1l1k.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87wmyu8mi0.fsf@localhost> (message from Ihor Radchenko on Thu, 20 Jul 2023 18:29:43 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN> <87wmyu8mi0.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: dmitry@HIDDEN, sbaugh@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Thu, 20 Jul 2023 18:29:43 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> No. It uses less excessive regexp matching Emacs is trying to do in > >> file-name-handler-alist. > > > > Where do you see regexp matching in the profiles you provided? > > I did the analysis earlier for `directory-files-recursively'. See > https://yhetil.org/emacs-devel/87cz0p2xlc.fsf@localhost/ > > Just to be sure, here is perf data for > (ignore (let ((gc-cons-threshold most-positive-fixnum)) (benchmark-progn (find-lisp-find-files "/home/yantar92/.data" "")))) > > 54.89% emacs emacs [.] re_match_2_internal > 10.19% emacs emacs [.] re_search_2 > 3.35% emacs emacs [.] unbind_to > 3.02% emacs emacs [.] compile_pattern > 3.02% emacs emacs [.] execute_charset > 3.00% emacs emacs [.] process_mark_stack > 1.59% emacs emacs [.] plist_get > 1.26% emacs emacs [.] RE_SETUP_SYNTAX_TABLE_FOR_OBJECT > 1.17% emacs emacs [.] update_syntax_table > 1.02% emacs emacs [.] Fexpand_file_name > > Disabling `file-name-handler-alist' cuts the time more than 2x. Disabling file-handlers is inconceivable in Emacs. And I suspect that find-file-name-handler is mostly called not from directory-files, but from expand-file-name -- another call that cannot possibly be bypassed in Emacs, since Emacs behaves as if CWD were different for each buffer. And expand-file-name also conses file names. And then we have encoding and decoding file names, something that with Find we do much less. Etc. etc.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 18:29:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 14:29:46 2023
Received: from localhost ([127.0.0.1]:59853 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qMYPK-00028w-0q
for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:29:46 -0400
Received: from mout01.posteo.de ([185.67.36.65]:52861)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qMYPH-00028i-Cu
for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:29:44 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout01.posteo.de (Postfix) with ESMTPS id 6AB81240028
for <64735 <at> debbugs.gnu.org>; Thu, 20 Jul 2023 20:29:37 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1689877777; bh=mQLb4duNHTj+Rup1cO0lQWX+la/b6pu2r50GOUbAdBM=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=Sx7269XC6tXWA+ScQeZbDM+4jzju/qgszq4pw0rLAnCnzukirO8OZxquayNKZxD9p
Py378yn2SmkziXaFmj3eqVZNZsqS+DonwDdwBQdx/uKLXW3ECU/7DjE8M/EzeJc3MM
rMSJeH/nPP96lY+z2wEQrXs2xTLqujF+1IiUBt+BfDQ0Xbymyd+8EO3+/H7qdTXNIr
aUPyBMFF1Ad/sHaTmLIWdeP9aTwFclp4duYztU1iSIIQRxLsL7GWY3M1QvSsS6GLEw
gWGHSYMZ8mPGaxA2kydg6bH3Sw7tyg65Oy+E6Mv52OplJ1isenPqtX+AZ0d4paxZ6T
6pCjzrx9Q6log==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R6LnX3wTTz9rxg;
Thu, 20 Jul 2023 20:29:36 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <831qh230h5.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN>
<87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN>
<87lefapkdx.fsf@localhost> <831qh230h5.fsf@HIDDEN>
Date: Thu, 20 Jul 2023 18:29:43 +0000
Message-ID: <87wmyu8mi0.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Eli Zaretskii <eliz@HIDDEN> writes:
>> No. It uses less excessive regexp matching Emacs is trying to do in
>> file-name-handler-alist.
>
> Where do you see regexp matching in the profiles you provided?
I did the analysis earlier for `directory-files-recursively'. See
https://yhetil.org/emacs-devel/87cz0p2xlc.fsf@localhost/
Just to be sure, here is perf data for
(ignore (let ((gc-cons-threshold most-positive-fixnum)) (benchmark-progn (find-lisp-find-files "/home/yantar92/.data" ""))))
54.89% emacs emacs [.] re_match_2_internal
10.19% emacs emacs [.] re_search_2
3.35% emacs emacs [.] unbind_to
3.02% emacs emacs [.] compile_pattern
3.02% emacs emacs [.] execute_charset
3.00% emacs emacs [.] process_mark_stack
1.59% emacs emacs [.] plist_get
1.26% emacs emacs [.] RE_SETUP_SYNTAX_TABLE_FOR_OBJECT
1.17% emacs emacs [.] update_syntax_table
1.02% emacs emacs [.] Fexpand_file_name
Disabling `file-name-handler-alist' cuts the time more than 2x.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 18:24:00 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 14:24:00 2023 Received: from localhost ([127.0.0.1]:59848 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMYJk-00020Z-4W for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:24:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48944) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMYJg-00020I-OJ for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 14:23:58 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMYJa-0005os-BD; Thu, 20 Jul 2023 14:23:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=GGMHbvHUoxTIXZqRWzh1c6/mWU+i8T1NZX8DHE1s/fU=; b=ic7upQgdezJi YVFFf4CihdqzvZqnxZ+I4KKb0zMU/KX9Vbj+O4QxxmofUfU6YtUB/kAUH3kiskrbzZbnAqTISHlk5 u0FR4I4ncSCfUKLldUpf//IsNK8dmhLLCGRTptFOLdkjNik/6lqSlK0vdbF7uVUMhSDNvAzN+BFhb //h/5RQqWdRrMmL793JUcJVTCm4DlhTyC9Um59naiD2cN0uQ64T1RXj8jiEUPfrshvWCfIQMR1FaF cYvHFzUV2NOw4QWO1FgyQ8K7/hAMvb91pepzm/wx4zup5cWGjNq/fbQZUP3y6xeD65QT9UzEZZc7O iWAjN/zmYHNvVIaVLt+Vgg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMYJZ-0003H6-Rf; Thu, 20 Jul 2023 14:23:50 -0400 Date: Thu, 20 Jul 2023 21:24:22 +0300 Message-Id: <831qh230h5.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87lefapkdx.fsf@localhost> (message from Ihor Radchenko on Thu, 20 Jul 2023 17:23:22 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN> <87lefapkdx.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: dmitry@HIDDEN, sbaugh@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Thu, 20 Jul 2023 17:23:22 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > >> I am not sure if this specific issue is important. > >> If we want to use find from Emacs, we would need to create Emacs > >> string/strings when reading the output of find anyway. > > > > So how do you explain that using Find is faster than using > > find-lisp.el? > > > > I think the answer is that using Find as a subprocess conses less. > > No. It uses less excessive regexp matching Emacs is trying to do in > file-name-handler-alist. Where do you see regexp matching in the profiles you provided?
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 17:25:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 13:25:24 2023 Received: from localhost ([127.0.0.1]:59809 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMXP0-0000H9-Kl for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 13:25:24 -0400 Received: from mout01.posteo.de ([185.67.36.65]:54703) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMXOj-0000Fh-Sn for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 13:25:20 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 5AF1E240027 for <64735 <at> debbugs.gnu.org>; Thu, 20 Jul 2023 19:25:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689873900; bh=gRuq5oCQCBhnUAel41GxugDO7R/A9YScDamizVYkx8U=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=OhtPruGy3NSD25fD1PWJzX1ndDTmTEVEgi22E25FwF6i9+wgUAz0Fyrv2Y+lUcwcb VoY5xvN5fB25nr7xzBjtBtrc+FR25dJ5j/8GACWbJraX/S1Eq1KecLiZLwC8dCD4UV E4VLq1vJ19vgO75ZmuBixeP401IoeqbIHQ4SN8WmxG8a+GCc1pnQHdoo2YiOxDg3IX Pi1D2kNAuh/RDUFB1SsbqDYPK/lXsh5qZYr6ijKEjwe91e0XLT5ZhliTo0VjQskCRK AwPR4sambX8F6wx7CHa96OlhU1AcozSudG9rerlJyBzDdrjlwWcHk640/gSj5FP/yc xqk4qtozzgfgA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6KLz4DQyz9rxR; Thu, 20 Jul 2023 19:24:59 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <ierfs5iwlx3.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> Date: Thu, 20 Jul 2023 17:25:11 +0000 Message-ID: <87ilaepkaw.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Spencer Baugh <sbaugh@HIDDEN> writes: >> Sure. It might also be optimized. Without trying to convince find devs >> to do something about regexp handling. > > Not to derail too much, but find as a subprocess has one substantial > advantage over find in Lisp: It can run in parallel with Emacs, so that > we actually use multiple CPU cores. Does find use multiple CPU cores? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 17:24:18 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 13:24:18 2023 Received: from localhost ([127.0.0.1]:59800 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMXNx-0000Ed-Nb for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 13:24:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54902) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMXNu-0000EO-KP for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 13:24:17 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMXNo-0005cY-D5; Thu, 20 Jul 2023 13:24:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=WY48oObOttS85iuWJ3C8cMAvETfvKkT0g1fbrE+ePxM=; b=GvzfFssuXimY CsLrPg79A6BXb+q6uBxbs10mptW8j5nsJl89FbpB5obvgNuPqhuv6BzeCZxw99uNko7JjTOQzh5aA pypsY/UkuaLPkSB1/2iikOKFihjJN+18JZx3hpwpvuznJAVmjuJ2MouuEA4jk13m7swvXo8xTpeK9 n9bORHxIQXX9CFm1zR7P1wDBJivBhuq2T5eYUbpfpljJGUr8R/60pFyC6euKOR+UpgtB31KXyPBqN GiRybavqSGDNvh/qpE5qhvMOQM1O6ubzhOFvPfg1R6r4tqagIkdu0L/PlyZKgBWAgXEjduisTXSq3 vtHKII78XMYNnmGOMAHhCA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMXNn-0005hb-Rd; Thu, 20 Jul 2023 13:24:08 -0400 Date: Thu, 20 Jul 2023 20:24:41 +0300 Message-Id: <83351i338m.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> In-Reply-To: <ierfs5iwlx3.fsf@HIDDEN> (message from Spencer Baugh on Thu, 20 Jul 2023 13:08:24 -0400) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <ierfs5iwlx3.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, yantar92@HIDDEN, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Cc: Dmitry Gutov <dmitry@HIDDEN>, 64735 <at> debbugs.gnu.org > From: Spencer Baugh <sbaugh@HIDDEN> > Date: Thu, 20 Jul 2023 13:08:24 -0400 > > (Really it's entirely plausible that Emacs could be improved by > *removing* directory-files-recursively, in favor of invoking find as a > subprocess: faster, parallelized execution, and better remote support.) No, there's no reason to remove anything that useful from Emacs. If this or that API is not the optimal choice for some job, it is easy enough not to use it.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 17:23:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 13:23:37 2023
Received: from localhost ([127.0.0.1]:59795 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1qMXNJ-0000DH-6V
for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 13:23:37 -0400
Received: from mout01.posteo.de ([185.67.36.65]:33023)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <yantar92@HIDDEN>) id 1qMXND-0000D0-LQ
for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 13:23:35 -0400
Received: from submission (posteo.de [185.67.36.169])
by mout01.posteo.de (Postfix) with ESMTPS id C8DBA240028
for <64735 <at> debbugs.gnu.org>; Thu, 20 Jul 2023 19:23:25 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017;
t=1689873805; bh=4wHMnJ3CJE3RL+TFR2BXOtnSoIN8OM7SMTBeej3VEb4=;
h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From;
b=XF8cL+KuwyqHDNkhK32ovQY5Pd5FDsLDKahNh6Natg6Djpi9lY899cCKqyFkPKLsr
KCAeRIb8gIJNOQYcaF8GwHEjb8Gx7RyA55ztgbGDs0qHptz9cMeUuaC9cfcU7ppjRy
FQVZJLOJnT8Mxi9bBAiCqcPjU4r+lFchMuTxxX2sozXJBbpzaktQZDclUDLeNi0TlV
eV2f5fZ6RZxCNLYB+Mgzz/LxbM5kUoKM3RpBqQymmPLogkkPXntQjqbdFcUARfNPrS
1ElfiljnXHib7zo7JrjiJj5hCcm1kvagge/tWR2rw3+RYNFFcZXMH0rJC3kWLYOCat
Yoq6ZEzSE7msg==
Received: from customer (localhost [127.0.0.1])
by submission (posteo.de) with ESMTPSA id 4R6KJz04Nrz6txN;
Thu, 20 Jul 2023 19:23:14 +0200 (CEST)
From: Ihor Radchenko <yantar92@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of
ignores
In-Reply-To: <834jly351p.fsf@HIDDEN>
References: <iermszrwqj4.fsf@HIDDEN>
<1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN>
<87wmyupvlw.fsf@localhost>
<5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN>
<87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN>
<87o7k6pmk3.fsf@localhost> <834jly351p.fsf@HIDDEN>
Date: Thu, 20 Jul 2023 17:23:22 +0000
Message-ID: <87lefapkdx.fsf@localhost>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 64735
Cc: dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Eli Zaretskii <eliz@HIDDEN> writes:
>> I am not sure if this specific issue is important.
>> If we want to use find from Emacs, we would need to create Emacs
>> string/strings when reading the output of find anyway.
>
> So how do you explain that using Find is faster than using
> find-lisp.el?
>
> I think the answer is that using Find as a subprocess conses less.
No. It uses less excessive regexp matching Emacs is trying to do in
file-name-handler-alist.
(ignore (let ((gc-cons-threshold most-positive-fixnum)) (benchmark-progn (find-lisp-find-files "/home/yantar92/.data" ""))))
;; Elapsed time: 2.982393s
(ignore (let ((gc-cons-threshold most-positive-fixnum) file-name-handler-alist) (benchmark-progn (find-lisp-find-files "/home/yantar92/.data" ""))))
;; Elapsed time: 0.784461s
22.83% emacs emacs [.] Fnconc
10.01% emacs emacs [.] Fexpand_file_name
9.22% emacs emacs [.] eval_sub
3.47% emacs emacs [.] assq_no_quit
2.68% emacs emacs [.] getenv_internal_1
2.50% emacs emacs [.] mem_insert.isra.0
2.24% emacs emacs [.] Fassq
2.02% emacs emacs [.] set_buffer_internal_2
(ignore (let ((gc-cons-threshold most-positive-fixnum) file-name-handler-alist) (benchmark-progn (find-lisp-find-files "/home/yantar92/.data" ""))))
;; Elapsed time: 0.624987s
12.39% emacs emacs [.] eval_sub
12.07% emacs emacs [.] Fexpand_file_name
4.97% emacs emacs [.] assq_no_quit
4.11% emacs emacs [.] getenv_internal_1
2.77% emacs emacs [.] set_buffer_internal_2
2.61% emacs emacs [.] mem_insert.isra.0
2.47% emacs emacs [.] make_clear_multibyte_string.part.0
Non-recursive version of `find-lisp-find-files-internal' is below,
though it provides limited improvement.
(defun find-lisp-find-files-internal (directory file-predicate
directory-predicate)
"Find files under DIRECTORY which satisfy FILE-PREDICATE.
FILE-PREDICATE is a function which takes two arguments: the file and its
directory.
DIRECTORY-PREDICATE is used to decide whether to descend into directories.
It is a function which takes two arguments, the directory and its parent."
(setq directory (file-name-as-directory directory))
(let (results fullname (dirs (list (expand-file-name directory))))
(while dirs
(setq directory (pop dirs))
(dolist (file (directory-files directory nil nil t))
(setq fullname (concat directory file))
(when (file-readable-p fullname)
;; If a directory, check it we should descend into it
(and (file-directory-p fullname)
(setq fullname (concat fullname "/"))
(funcall directory-predicate file directory)
(push fullname dirs))
;; For all files and directories, call the file predicate
(and (funcall file-predicate file directory)
(push fullname results)))))
results))
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 17:08:35 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 13:08:35 2023 Received: from localhost ([127.0.0.1]:59756 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMX8l-0008Gh-AE for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 13:08:35 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:32891) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <sbaugh@HIDDEN>) id 1qMX8g-0008GR-RD for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 13:08:33 -0400 From: Spencer Baugh <sbaugh@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <87tttypp2e.fsf@localhost> (Ihor Radchenko's message of "Thu, 20 Jul 2023 15:42:17 +0000") References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> Date: Thu, 20 Jul 2023 13:08:24 -0400 Message-ID: <ierfs5iwlx3.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: Dmitry Gutov <dmitry@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Ihor Radchenko <yantar92@HIDDEN> writes: > Dmitry Gutov <dmitry@HIDDEN> writes: > >>>> ... Last I checked, Lisp-native file >>>> listing was simply slower than 'find'. >>> >>> Could it be changed? >>> In my tests, I was able to improve performance of the built-in >>> `directory-files-recursively' simply by disabling >>> `file-name-handler-alist' around its call. >> >> Then it won't work with Tramp, right? I think it's pretty nifty that >> project-find-regexp and dired-do-find-regexp work over Tramp. > > Sure. It might also be optimized. Without trying to convince find devs > to do something about regexp handling. Not to derail too much, but find as a subprocess has one substantial advantage over find in Lisp: It can run in parallel with Emacs, so that we actually use multiple CPU cores. Between that, and the remote support part, I personally much prefer find to be a subprocess rather than in Lisp. I don't think optimizing directory-files-recursively is a great solution. (Really it's entirely plausible that Emacs could be improved by *removing* directory-files-recursively, in favor of invoking find as a subprocess: faster, parallelized execution, and better remote support.)
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 16:45:26 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 12:45:26 2023 Received: from localhost ([127.0.0.1]:59689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMWmK-0007b3-8y for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 12:45:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45838) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMWm9-0007aZ-RC for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 12:45:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMWm3-0001k2-UN; Thu, 20 Jul 2023 12:45:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=dro429C2Y35t3PhtT4ea935VNb6hkQBswpgY2OSZIL4=; b=R8RIevVRlVD4 ortqVTqLw/iZPZ8j0+sDHTh1q3trfZH72bT4i5ZkmAt/6kGk1AjXs1TfJRFW1+mmgRo8ywjiwVwzz gTE8TYW1w+CTmUkF77SF40MjONUcCMSX0/1l0BR1WGKbg+wfzSPmmUkDXkNY4xhvkBEdiicI6BU7f 5NHQkHDpdxYnrz+W3jXkAkURG7MIE+HHvZzMuzEAy2Wq/ummIm4clnlC/hJnyktlvHQASw9RXS4oq o/PnPuBGAKjALkOBslwoUdIo2WHeNxnzccDIZp2QtXqTME7461nMNnl6a4zqAQZ6N6oX4NwvKTGtY LyZ+PVLYyEa4JUia2dMOvg==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMWm0-000512-It; Thu, 20 Jul 2023 12:45:06 -0400 Date: Thu, 20 Jul 2023 19:45:38 +0300 Message-Id: <834jly351p.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87o7k6pmk3.fsf@localhost> (message from Ihor Radchenko on Thu, 20 Jul 2023 16:36:28 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> <87o7k6pmk3.fsf@localhost> X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ihor Radchenko <yantar92@HIDDEN> > Cc: dmitry@HIDDEN, sbaugh@HIDDEN, 64735 <at> debbugs.gnu.org > Date: Thu, 20 Jul 2023 16:36:28 +0000 > > Eli Zaretskii <eliz@HIDDEN> writes: > > > Maybe it can be improved, but not to the same level as Find, because > > consing Lisp strings, something that Find doesn't do, does have its > > overhead. > > I am not sure if this specific issue is important. > If we want to use find from Emacs, we would need to create Emacs > string/strings when reading the output of find anyway. So how do you explain that using Find is faster than using find-lisp.el? I think the answer is that using Find as a subprocess conses less.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 16:36:36 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 12:36:36 2023 Received: from localhost ([127.0.0.1]:59651 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMWdm-0007Ma-Lv for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 12:36:36 -0400 Received: from mout02.posteo.de ([185.67.36.66]:46625) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMWdb-0007MC-82 for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 12:36:33 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 28F9F240103 for <64735 <at> debbugs.gnu.org>; Thu, 20 Jul 2023 18:36:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689870977; bh=lZoY/NFHCgKwbBO/IpY2d82D3y7ZM+sPptco6FsjCNg=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=JZ7cqGF0H86x69sakAy3zrW1o47aGJcsUrfMr2ARcpCGboBcVvUkqzf2ByZfFrQU4 TTSc6FXYvlCDnrpg9JmVD51I7keSTYUelI+rtMn++WG8+FqVebEGaRBq30xYzZF/h0 awcUcNhUnUg4q02H9QQAP2weXl2G2QOb+6SelUsFvNrIqSlNkNyEQ4TQ2k+kDdtrzR Sbvuf3wyy2zO6P4oY7IrLikzYemZsFKR0bjkHDMy79NmsuA92VyFpoKWhGziofQXN3 a+PkpLKTvpdUO9kuxZSLFISf0lB+alnKZ+YS4odjChP2IDjglUbzm4iedp3/TBCE0z fYsWFrsM8ePgQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6JGm0lyMz9rxQ; Thu, 20 Jul 2023 18:36:15 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Eli Zaretskii <eliz@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <837cqu35lw.fsf@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <837cqu35lw.fsf@HIDDEN> Date: Thu, 20 Jul 2023 16:36:28 +0000 Message-ID: <87o7k6pmk3.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Eli Zaretskii <eliz@HIDDEN> writes: >> I am pretty sure that Emacs's native file routines can be optimized to >> the level of find. > > Maybe it can be improved, but not to the same level as Find, because > consing Lisp strings, something that Find doesn't do, does have its > overhead. I am not sure if this specific issue is important. If we want to use find from Emacs, we would need to create Emacs string/strings when reading the output of find anyway. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 16:33:06 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 12:33:06 2023 Received: from localhost ([127.0.0.1]:59629 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMWaP-0007GN-Sz for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 12:33:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59722) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMWaO-0007Ft-MB for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 12:33:05 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMWaI-0006VR-2b; Thu, 20 Jul 2023 12:32:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=P7AW6H5hFcHS6VDU4iRlOkQQe7XoZo8sJ8ELI94JyNM=; b=LPAoDG4AZ1sy YVdwj6D+797N/ep48CxRP7PwqBm2VfLn9ddwa729JoM+3uXFlONGq0CYhIjRXxHeIn7yaC5uu3lVy 6+HtZozWLpNKbnCTTXP+vwAtNkWiUzwcHyrN9Iy0rukX91J6ZoxRUtgpLLUfumYukabwFG0IqQz9c hy8TT0fcd2DuRRXQOKTDb8T2MorUo7OW3SAIX3HwiPyPzN3lEEjoMhEPewMr2qO/w2fw9QpnbgPu4 HpSXb6IW9YmeFHBBxstMWZGH1fFtGQ6wnoXWDpti/+6y0DxARskO8TE6HUKmaqH56V9QghKsCOGVM mLtEixKSmAurPx524yy4QA==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMWaH-0004ia-HE; Thu, 20 Jul 2023 12:32:57 -0400 Date: Thu, 20 Jul 2023 19:33:31 +0300 Message-Id: <837cqu35lw.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ihor Radchenko <yantar92@HIDDEN> In-Reply-To: <87tttypp2e.fsf@localhost> (message from Ihor Radchenko on Thu, 20 Jul 2023 15:42:17 +0000) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: dmitry@HIDDEN, 64735 <at> debbugs.gnu.org, sbaugh@HIDDEN X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org > From: Ihor Radchenko <yantar92@HIDDEN> > Date: Thu, 20 Jul 2023 15:42:17 +0000 > > I am pretty sure that Emacs's native file routines can be optimized to > the level of find. Maybe it can be improved, but not to the same level as Find, because consing Lisp strings, something that Find doesn't do, does have its overhead.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 16:04:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 12:04:01 2023 Received: from localhost ([127.0.0.1]:59554 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMW8H-0006Lw-Cz for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 12:04:01 -0400 Received: from mout01.posteo.de ([185.67.36.65]:55665) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMW8D-0006LS-4O for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 12:04:00 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id D4494240027 for <64735 <at> debbugs.gnu.org>; Thu, 20 Jul 2023 18:03:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689869030; bh=nRBVd84U/Ajo6awG4TzA4W75Q4Ju8nNFrs/XjjrGsVY=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=iz4LZy3L1GnR72eU3zvT8VUKR6WNviCxANe1kcH48v4KoAz3d+C0MZx7k2gOSxJE6 3dRa/9r68jMRQcMyoHy+ARkH1AOF8e4bVyaMOvBfUvFNS+D5Ov2lMgF8jQln0192Ah yUCQzOoW4BIG1VDTn1kuY98eYlhC+8M4Tl1W62HJp9s01LxZYdikh0WNZask4YMeOU pyMFTtKJPsUxoB+NwJFamEvhLg7/xhqgp0npvyfRrIUgI/wwLFaidUZuHTHQxrGmU1 /s7BDbKwDqTYYe6zQ58oAqQh4F3xpyelXqE1Ul1fIv2NsmrsoUgl50wAYgh/KIHPLw /Y0VQ28/FqB+A== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6HYL0lvWz9rxM; Thu, 20 Jul 2023 18:03:50 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <aba1400a-3d00-44f0-90f1-e5b549de8ae7@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> <aba1400a-3d00-44f0-90f1-e5b549de8ae7@HIDDEN> Date: Thu, 20 Jul 2023 16:03:58 +0000 Message-ID: <87r0p2po29.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Dmitry Gutov <dmitry@HIDDEN> writes: >> And things are not as horrible as 15x slowdown in find. > > We haven't compared to the "optimized regexps" solution in find, though. Fair point. > Still like 2.5x slower, then? That's significant. It is, but it is workable if we try to optimize Emacs' `directory-files'/`file-name-all-completions' internals. >> I am pretty sure that Emacs's native file routines can be optimized to >> the level of find. > > I don't know, the GNU tools are often ridiculously optimized. At least > certain file paths. You are likely right. Then, what about applying regexps manually, on the full file list returned by find? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 15:57:12 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 11:57:12 2023 Received: from localhost ([127.0.0.1]:59535 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMW1f-00066i-II for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 11:57:11 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:43917) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMW1c-00066O-Vu for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 11:57:10 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 753315C006A; Thu, 20 Jul 2023 11:57:03 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 20 Jul 2023 11:57:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1689868623; x=1689955023; bh=z1IsZ/XMTPa5iEDsDduIuDCFm7BcSEM4i1N RFXergfo=; b=ZNJvCK4k2/XFMedHCEzSJAOeRx2vUWk7HvekHvDxtD40sYafr7E 7jkr0HOgwxWCNhyWyD50osd+HUufJPyvVUXrK3oA0v2SIZcbYl1fmq/++ZXgnIZZ W8Jescl4j20XI0W6gQnS63MlMCc6uEZe1NMNMebkN1WWJn6G7CwA/CB3cREnNji7 v3dqV6aS8iMaVcNEsx2Q6RWFQ1XGPsCtiQ1SiLnKDmqvmcJjij4HbMvHuRR9LqkP MYYv5pHMznERxEmCvLOJw0TILcBY83joI1MfzJOWzm1Lo0k9VxqFXhCGrfZdfIV4 Sm+i6Bj8yRSXVrPuy0YOYbREZDd1eH/oU+A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689868623; x=1689955023; bh=z1IsZ/XMTPa5iEDsDduIuDCFm7BcSEM4i1N RFXergfo=; b=Ykajmm7JQDfj6pOc5RNaSu6v4Pg9MUdWnIqA4qiZkkNyJDyFo5X VWVgRh7DP0mpl9PrXOBCMoYSmkQL5DwRfUZ54YjjsUb+RyAmymhbzGujOdcAIVZi hxW9slhl/6i/60Na2akU7u/jpTSuAm+pVBYbjLxmlguMo7HAIrhsf1qmbRjuEFwg llgARL2YpaLOs4cCf6Ti7tTSLA542UJlZVk0JlcvVRgVUQsemAievN/gxOcSpN+/ jLVM0L8Tx/3XTYOQ3pfpn/EZ5TkxCj4Lnigt5FohkH8PEz0V8wLz7+fXrijO55Y/ 0BbBY0qoCV1iccULY+69WraFyMwSambgezw== X-ME-Sender: <xms:T1m5ZMhS-tBhtfMtPLw2rkCFyyg_16HEPXFII8sPPnHGbmkc-d_qBA> <xme:T1m5ZFALS-kse30EhHOtpHyyCWX4nY7Dl9UVxXtZd6IXdNoK5wr4AxVNcQzLprJVb IUMF14BJLUB4NOc42A> X-ME-Received: <xmr:T1m5ZEEJPoZ09XT-BYONdOzDRdaEMCyezHxwfnJReKUO67PXGA5pe2qPY-vFxto> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedtgdelgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeffhefgkeevheevvdeutedtkeeijeduueejheethedtgfdutdetveffvdevteef ueenucffohhmrghinhephihhvghtihhlrdhorhhgpdihrghnthgrrhelvddruggrthgrne cuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepughmihht rhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:T1m5ZNTCiQ8HAWWX4_7zpUFx-P5lS1oqQ1KcJRVrK6QSYOwRn9IChg> <xmx:T1m5ZJwwZhLWLyH_SNWjXM_YHXouuw2foW1AFcBRQIYiFnXitNkCcQ> <xmx:T1m5ZL525T1YvxwAO7MBHGZ8ohlC9wQzv4Z86_D7p4Pdra1s6cWt_w> <xmx:T1m5ZKrUvoDFx0LWrZvZfJCnrBdXf0O-au6G1RpNHvJEYZUt2SK5jQ> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 20 Jul 2023 11:57:02 -0400 (EDT) Message-ID: <aba1400a-3d00-44f0-90f1-e5b549de8ae7@HIDDEN> Date: Thu, 20 Jul 2023 18:57:00 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> <87tttypp2e.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87tttypp2e.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 20/07/2023 18:42, Ihor Radchenko wrote: > Dmitry Gutov <dmitry@HIDDEN> writes: > >>>> ... Last I checked, Lisp-native file >>>> listing was simply slower than 'find'. >>> >>> Could it be changed? >>> In my tests, I was able to improve performance of the built-in >>> `directory-files-recursively' simply by disabling >>> `file-name-handler-alist' around its call. >> >> Then it won't work with Tramp, right? I think it's pretty nifty that >> project-find-regexp and dired-do-find-regexp work over Tramp. > > Sure. It might also be optimized. Without trying to convince find devs > to do something about regexp handling. > > And things are not as horrible as 15x slowdown in find. We haven't compared to the "optimized regexps" solution in find, though. >>> See https://yhetil.org/emacs-devel/87cz0p2xlc.fsf@localhost/ >>> (the thread also continues off-list, and it looks like there is a lot of >>> room for improvement in this area) >> >> Does it get close enough to the performance of 'find' this way? > > Comparable: > > (ignore (let ((gc-cons-threshold most-positive-fixnum)) (benchmark-progn (directory-files-recursively "/home/yantar92/.data" "")))) > ;; Elapsed time: 0.633713s > (ignore (let ((gc-cons-threshold most-positive-fixnum)) (benchmark-progn (let ((file-name-handler-alist)) (directory-files-recursively "/home/yantar92/.data" ""))))) > ;; Elapsed time: 0.324341s > ;; time find /home/yantar92/.data >/dev/null > ;; real 0m0.129s > ;; user 0m0.017s > ;; sys 0m0.111s Still like 2.5x slower, then? That's significant. >> Also note that processing all matches in Lisp, with many ignores >> entries, will incur the proportional overhead in Lisp. Which might be >> relatively slow as well. > > Not significant. > I tried to unwrap recursion in `directory-files-recursively' and tried > to play around with regexp matching of the file list itself - no > significant impact compared to `file-name-handler-alist'. I suppose that can make sense, if find's slowdown is due to it issuing repeated 'stat' calls for every match. > I am pretty sure that Emacs's native file routines can be optimized to > the level of find. I don't know, the GNU tools are often ridiculously optimized. At least certain file paths.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 15:42:24 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 11:42:24 2023 Received: from localhost ([127.0.0.1]:59487 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMVnL-0005dq-Uh for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 11:42:24 -0400 Received: from mout02.posteo.de ([185.67.36.66]:51251) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMVnI-0005db-8R for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 11:42:21 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id D68B3240101 for <64735 <at> debbugs.gnu.org>; Thu, 20 Jul 2023 17:42:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689867733; bh=YZc7W2lILHnFsPNcNeE1M++pUIcYef2prBgPrynZ/3c=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=YfE3crK8tyMvo+wbAFsbjnq9B0Jtmzm+RDsFyfwQQ4XxpWBZBzSU9ApPYaN5X0ph6 F0dXtEa3zS0cyQv7F2tTI3gRYFKCrhR6hnPhjfK8jAgkLWrmMMVLNsZbEMjIfhE2XI gygfT557H3TALI9F5tQDn3xcXP2kjHW80g/+ZnWoL/xjaHTHEEK8JzNWtuLQobKcdG 8AmFLy2xMS6jmY36Q420L9YnG9NrMlxjKuM+2kXgvNXHZwoRkR39G6sgiKfxkOtZzB pKL3xkMqOwYms7K2ZvRxHBC4H/LxNsJJjtEwPulXje2eFb5q5OYbXhq0/lwLBO7Gri BbLvUUbfsHtBw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6H4K2HtCz9rxW; Thu, 20 Jul 2023 17:42:09 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> Date: Thu, 20 Jul 2023 15:42:17 +0000 Message-ID: <87tttypp2e.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Dmitry Gutov <dmitry@HIDDEN> writes: >>> ... Last I checked, Lisp-native file >>> listing was simply slower than 'find'. >> >> Could it be changed? >> In my tests, I was able to improve performance of the built-in >> `directory-files-recursively' simply by disabling >> `file-name-handler-alist' around its call. > > Then it won't work with Tramp, right? I think it's pretty nifty that > project-find-regexp and dired-do-find-regexp work over Tramp. Sure. It might also be optimized. Without trying to convince find devs to do something about regexp handling. And things are not as horrible as 15x slowdown in find. >> See https://yhetil.org/emacs-devel/87cz0p2xlc.fsf@localhost/ >> (the thread also continues off-list, and it looks like there is a lot of >> room for improvement in this area) > > Does it get close enough to the performance of 'find' this way? Comparable: (ignore (let ((gc-cons-threshold most-positive-fixnum)) (benchmark-progn (directory-files-recursively "/home/yantar92/.data" "")))) ;; Elapsed time: 0.633713s (ignore (let ((gc-cons-threshold most-positive-fixnum)) (benchmark-progn (let ((file-name-handler-alist)) (directory-files-recursively "/home/yantar92/.data" ""))))) ;; Elapsed time: 0.324341s ;; time find /home/yantar92/.data >/dev/null ;; real 0m0.129s ;; user 0m0.017s ;; sys 0m0.111s > Also note that processing all matches in Lisp, with many ignores > entries, will incur the proportional overhead in Lisp. Which might be > relatively slow as well. Not significant. I tried to unwrap recursion in `directory-files-recursively' and tried to play around with regexp matching of the file list itself - no significant impact compared to `file-name-handler-alist'. I am pretty sure that Emacs's native file routines can be optimized to the level of find. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 15:19:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 11:19:53 2023 Received: from localhost ([127.0.0.1]:59463 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMVRZ-00055q-9s for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 11:19:53 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:49533) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMVRW-00055b-Cb for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 11:19:51 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id C21C55C01A8; Thu, 20 Jul 2023 11:19:44 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 20 Jul 2023 11:19:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1689866384; x=1689952784; bh=kBQ8rrezP2nvOULyLtlijS6nHHWfTOxYnG5 cb4dMJb8=; b=MWPD22kYmau0paO2O9wEwuBqZgC06njXTyZFq4GQ52efIwPQ0fY cYnpMnpjztmUbpc8NFhWKG0XC8/WXMeE7vRJoCGd0ufY82EgAPoK7dggMU5CSY52 RXDV4lRaImipI2RVu0a/URwbS3gizVyKQyfoH3kavbYordGOYVm6RSlHzVt2hace MGCWezf6mymqV2TTPadHUa6NKSUR9rRxRJ70mRstTOAt/tFWUAf47POyQLCu2vhs lZLahPnq7RWim0HG2I8F4Ohp03cxp76CDZRErrnAObQGg29X3QwvGU4V/U23SptK TxZdOctFcrfng+kIxvjSXnIawuMTK2JhAXg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689866384; x=1689952784; bh=kBQ8rrezP2nvOULyLtlijS6nHHWfTOxYnG5 cb4dMJb8=; b=Hxfxu480T/y+kzV5o3kVfWzrkXsUY8u1Smds3M9HGsi82BvWFj9 gj2v3uLRH7Je64iq7S51xyB/GSHuA5TcfOFCpe9BEgNUzYfTvcL9R14ow55LODvn j4GrTopwS1Oj7KoivipbdW/j3JI7X3HWWp+qZCFNMdzybs/a4dUaB7ibff+cu33E m+9PA4yKwtw8dJ2K3UOPp/1mJHB43qjSVzH5d016qFkcsy9DhxavE4tce7hP58Q1 kjwTZL2SxvZ51qg/1Vj/IxFbTPRG5W7R/lJC5o1GYQ/ET3MlolDD2VdT7vGSKs6g Nz4433pFlIYZuBshiLrRKaUNQbUGw+a4HJw== X-ME-Sender: <xms:kFC5ZOOaHf14ylpi52WBtVrGZoIaPrkmEvP2Kn6hsORU0tqzc0FpSA> <xme:kFC5ZM8A_Umm1_8ZEOv2jg2vw3MfQJ47s4GyZiE3WTrDmMsjfAXlDzKUGtCN6hI04 -naFGayenwfTa1sSdk> X-ME-Received: <xmr:kFC5ZFSqpweNih1powzsUkjHoR5kuggp8PFV7ydrriVHkgWF_ofI9xvk0GnvN8c> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedtgdekiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeehleeflefhheeujeduteejveeggeejjefhleeljeettdfhjedthfejgfeivdeu leenucffohhmrghinhephihhvghtihhlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomhepughmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:kFC5ZOtpvgdAUMeU1lpjJXkwiqvBdTqUNyH20QDdiW4_8Vu1tT8zBA> <xmx:kFC5ZGf_IwrscIXa6sKf-xQ8l64RTwL7HD97Z_eJL3mK1no8mtrgzg> <xmx:kFC5ZC3lpO7bNZ0rMs1-9GpYFk2t0l1uklxKrApaK0Pp25PrEMOuRg> <xmx:kFC5ZKGnoqL6w9ZjeaAVsZTfGMYaUgfotDwrc0kuguM1N_PNBURURA> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 20 Jul 2023 11:19:43 -0400 (EDT) Message-ID: <5c4d9bea-3eb9-b262-138a-4ea0cb203436@HIDDEN> Date: Thu, 20 Jul 2023 18:19:41 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Ihor Radchenko <yantar92@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> <87wmyupvlw.fsf@localhost> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87wmyupvlw.fsf@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 20/07/2023 16:20, Ihor Radchenko wrote: > Dmitry Gutov <dmitry@HIDDEN> writes: > >> ... Last I checked, Lisp-native file >> listing was simply slower than 'find'. > > Could it be changed? > In my tests, I was able to improve performance of the built-in > `directory-files-recursively' simply by disabling > `file-name-handler-alist' around its call. Then it won't work with Tramp, right? I think it's pretty nifty that project-find-regexp and dired-do-find-regexp work over Tramp. > See https://yhetil.org/emacs-devel/87cz0p2xlc.fsf@localhost/ > (the thread also continues off-list, and it looks like there is a lot of > room for improvement in this area) Does it get close enough to the performance of 'find' this way? Also note that processing all matches in Lisp, with many ignores entries, will incur the proportional overhead in Lisp. Which might be relatively slow as well.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 13:44:06 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 09:44:06 2023 Received: from localhost ([127.0.0.1]:58037 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMTws-000261-EF for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 09:44:06 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]:34341) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <sbaugh@HIDDEN>) id 1qMTwq-00025Q-Ez for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 09:44:05 -0400 From: Spencer Baugh <sbaugh@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <4d4f029f-f32f-13df-ffc3-3952d62d8bb3@HIDDEN> (Dmitry Gutov's message of "Thu, 20 Jul 2023 15:42:59 +0300") References: <iermszrwqj4.fsf@HIDDEN> <837cqv41ob.fsf@HIDDEN> <87mszqixhh.fsf@HIDDEN> <4d4f029f-f32f-13df-ffc3-3952d62d8bb3@HIDDEN> Date: Thu, 20 Jul 2023 09:43:59 -0400 Message-ID: <ierjzuuwvds.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 64735 Cc: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Dmitry Gutov <dmitry@HIDDEN> writes: > On 20/07/2023 15:22, sbaugh@HIDDEN wrote: >>> I'm not sure we should bother more than these two simple measures. >> Unfortunately those two simple measures help rgrep but they don't help >> project-find-regexp (and others project.el commands using >> project--files-in-directory such as project-find-file), since those >> project commands pull their ignores from the version control system >> through vc (not grep-find-ignored-files), and then pass them to find. > > That's only a problem when the default file listing logic is used (and > we usually delegate to something like 'git ls-files' instead, when the > vc-aware backend is used). Hm, yes, but things like C-u project-find-regexp will use the default find-based file listing logic instead of git ls-files, as do a few other things. I wonder, could we just go ahead and make a vc function which is list-files(GLOBS) and returns a list of files? Both git and hg support this. Then we could have C-u project-find-regexp use that instead of find, by taking the cross product of dirs-to-search and file-name-patterns-to-search. (And this would let me delete a big chunk of my own project backend, so I'd be happy to implement it.) Fundamentally it seems a little silly for project-ignores to ever be used for a vc project; if the vcs gives us ignores, we can probably just ask the vcs to list the files too, and it will have an efficient implementation of that. If we do that uniformly, then this find slowness would only affect transient projects, and transient projects pull their ignores from grep-find-ignored-files just like rgrep, so improvements will more easily be applied to both. (And maybe we could even get rid of project-ignores entirely, then?) > Anyway, some optimization could be useful there too. The extra > difficulty, though, is that the entries in IGNORES already can come as > wildcards. Can we merge several wildcards? Though I suppose if we use > a regexp, we could construct an alternation anyway. > > Another question it would be helpful to check, is whether the > different versions of 'find' out there work fine with -regex instead > of -name, and don't get slowed down simply because of that > feature. The old built-in 'find' on macOS, for example.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 13:21:01 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 09:21:01 2023 Received: from localhost ([127.0.0.1]:58014 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMTaW-0001UF-SL for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 09:21:01 -0400 Received: from mout01.posteo.de ([185.67.36.65]:33423) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <yantar92@HIDDEN>) id 1qMTaR-0001Tp-H2 for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 09:20:59 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 6C452240029 for <64735 <at> debbugs.gnu.org>; Thu, 20 Jul 2023 15:20:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689859249; bh=g0MjRjEdI72WIvAo0hppkgv+5rtrhexKT12P0v7grYM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=TGWNbr+2TTYBmW18ZUHY5fSKmksDiLo4gcrryCAQYCLRnrhpha5yo4oP5lWgZpFgd V7ZLHV0p+A0WHjESuCq9EcsILyOFcG7e8uONYFuR5Ba4LwBACN8NxwRiHtbgIEx1zW ND28bk2nxbFXBBBfcwqf0rtiLm9l+d1k/X/1MjHgsx1iGllDQXad2RreIyRvXnJdLZ 7AlFHGv/JYRDmO+Pl2gEa14XPdnutrprrLQxEtmuhtw0zC+eqB8eJDiF/HkYyozXB6 nZrCmu8o/XShKZrlvVlU/ihNftYKCQFcmsTpNkUDfET+zuXeoM1nV7DnZSEDyvlNlF z27B55ohMNO5A== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R6CxD145sz9rxX; Thu, 20 Jul 2023 15:20:47 +0200 (CEST) From: Ihor Radchenko <yantar92@HIDDEN> To: Dmitry Gutov <dmitry@HIDDEN> Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> Date: Thu, 20 Jul 2023 13:20:59 +0000 Message-ID: <87wmyupvlw.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Dmitry Gutov <dmitry@HIDDEN> writes: > ... Last I checked, Lisp-native file > listing was simply slower than 'find'. Could it be changed? In my tests, I was able to improve performance of the built-in `directory-files-recursively' simply by disabling `file-name-handler-alist' around its call. See https://yhetil.org/emacs-devel/87cz0p2xlc.fsf@localhost/ (the thread also continues off-list, and it looks like there is a lot of room for improvement in this area) -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92>
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 12:43:09 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 08:43:09 2023 Received: from localhost ([127.0.0.1]:57941 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMSzt-0008WL-Go for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 08:43:09 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:49359) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMSzr-0008W4-0A for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 08:43:07 -0400 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id E9E9C5C0089; Thu, 20 Jul 2023 08:43:01 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Thu, 20 Jul 2023 08:43:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1689856981; x=1689943381; bh=7buN++tSPFVB6cJ+5C11cPD2buTnj18XMf+ 6YvmsJHU=; b=Y1CHSHW+cRmEMYugxT1OjEbR9FXLunZN7xPngGXI1IwRL4v/cl/ ytvU0KrlsmOg7GP7v/H0uQNbRlo43FHcyNkDDEsjFZuv6FmDeQBOijo4JMRd9G59 4RLQ6Zd/oIaVEKYcmWFYfO7zUDd/oP6qrKz0AwZpf1JfeaVUyuKUDcGM+6DTF+Zk 2rXG4QbU8TEPe/c8xETSzck/dkogZz9c7+N1K4hJcZqwnbBvp1NqHNVxZSq33LtV uZ+KmFuC/NNS3QF8MZKET1gQMUpLQXqgDDOBPCi822Tllyta5GSDI5rJ75nUdZ2/ g1sSrq+AzxB/jqH3A3yZaPBtZOrtAIyV1gQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1689856981; x=1689943381; bh=7buN++tSPFVB6cJ+5C11cPD2buTnj18XMf+ 6YvmsJHU=; b=MGntHloOtIKzkmlzfogInZkqLqeYo5IJLyetuQj+H26XqflBUxh 5GRzL7thcaMAWWlFxsF4vo2FUIET0hxTtXbMyuzjUm2SsPvLLtzOlDRfi1go+a5U tiIJi394QswMJz/kN7psPHXt3eDt3QJJySO5GCEUYdznna7oXj9L0LXF3A73spwN bh7TO05SWVuqbPAnQSP1IieVlEfWwjKcAnkGB2RqwAngTRRj2RsAypu1tRJaHqES DolTFMrkZSkc6B1IsgWCas/hlOaormJL4C7x1I2b7C9cwzo5gwQu50pBy9cDcwl6 t9lez3oglwQ0G3An1bELR659aShACKqh3uQ== X-ME-Sender: <xms:1Su5ZKCLBQ4oD8sunxuRErJdaGoOBIBcqFsMK0xHo3eEvGdytDQQLA> <xme:1Su5ZEhPRLMh1UR9m0x47FId56lSCJ9QTVyzD2PYHwP1QWtO8f-HKFszRxQ2eK6to 2P1CsECVDxgaKHM9GA> X-ME-Received: <xmr:1Su5ZNnJEDEKUka4JKuFhBnSIqpWiCM_knFYsDjd1h_m6icmEzHScPhJt09L4dY> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedtgdehhecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfevfhfhjggtgfesthejredttdefjeenucfhrhhomhepffhmihht rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth gvrhhnpeeigfetveehveevffehledtueekieeikeeufeegudfgfeeghfdulefgfeevledv veenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh hithhrhiesghhuthhovhdruggvvh X-ME-Proxy: <xmx:1Su5ZIxTboA1G5kMeLOioJClBjlYOC5blFjm3MKTP7YW-2g9b7JTVA> <xmx:1Su5ZPQeXXcXl5vsf70RyElFO8edRKoHEOtctGapF35EQYJLdXTTIw> <xmx:1Su5ZDYCTB71e-H5otQbAf8s__qVc7i0ewNFfj8c_tD-rL7TRw_R-g> <xmx:1Su5ZKeWSfvXVzZrOM0uZMQlVO6f3eXQSw2ncCBDJWuckGsSR_bSpg> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 20 Jul 2023 08:43:00 -0400 (EDT) Message-ID: <4d4f029f-f32f-13df-ffc3-3952d62d8bb3@HIDDEN> Date: Thu, 20 Jul 2023 15:42:59 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: sbaugh@HIDDEN, Eli Zaretskii <eliz@HIDDEN> References: <iermszrwqj4.fsf@HIDDEN> <837cqv41ob.fsf@HIDDEN> <87mszqixhh.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <87mszqixhh.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.8 (/) X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 20/07/2023 15:22, sbaugh@HIDDEN wrote: >> I'm not sure we should bother more than these two simple measures. > Unfortunately those two simple measures help rgrep but they don't help > project-find-regexp (and others project.el commands using > project--files-in-directory such as project-find-file), since those > project commands pull their ignores from the version control system > through vc (not grep-find-ignored-files), and then pass them to find. That's only a problem when the default file listing logic is used (and we usually delegate to something like 'git ls-files' instead, when the vc-aware backend is used). Anyway, some optimization could be useful there too. The extra difficulty, though, is that the entries in IGNORES already can come as wildcards. Can we merge several wildcards? Though I suppose if we use a regexp, we could construct an alternation anyway. Another question it would be helpful to check, is whether the different versions of 'find' out there work fine with -regex instead of -name, and don't get slowed down simply because of that feature. The old built-in 'find' on macOS, for example.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 12:38:27 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 08:38:27 2023 Received: from localhost ([127.0.0.1]:57936 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMSvJ-0008PF-TR for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 08:38:27 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:45303) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dmitry@HIDDEN>) id 1qMSv9-0008Ot-NQ for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 08:38:24 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 7A7CC5C0159; Thu, 20 Jul 2023 08:38:08 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 20 Jul 2023 08:38:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm1; t= 1689856688; x=1689943088; bh=GbBbzLKPBKtUO8BYZ1LApyV+j398P3T87SF co2FbRdQ=; b=dAt2IVxYL9WcotUybjgTGMPWWC6O09AHEZhLR6aOcv/Dr+qva7f pW/0retKKGFK0JbLj1/FadESjLG7VyYhAU8H9RXvfJPGrzpsKsg7eQfi24uGwfzr NKSLSYjS4vlHmmq2CwW6/8opH5yjXD5a+BbLMv7Y9RKSOBdCLHTjnJBypg6x1I48 nisHenuubAEHB03N2F1l8ugrZefhY6tNSfjglsdRRdPiHnvaoqmgpVm2q2RQJQE0 1F+izvBLkAJi8q+qoggbcQS3MBixIEz/EwiI05hOzZdNbeze5aRWS3bPg7rBw+E9 7PrQH81jjHpJpI1WhKmmlNk+4jBR89c4cmg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1689856688; x= 1689943088; bh=GbBbzLKPBKtUO8BYZ1LApyV+j398P3T87SFco2FbRdQ=; b=c jbpX4JTp2f8VGcgJDGaNg4DaYFJYs5ulvWQgiJw8jg2jankMJWL2tPolB12NJVla mGy+OfUwFEEhLIOStyji78w01uNS/j6y33YFEqC07V0F4O4CvaLAK7pRcN7z6xnM Yd9vvy/6QSuEorY09sVoaIZi6Sp03eqOOuO/jZ1mkGYBzb1jpDTFiSMoVbu0TCJY zzAX9bwWXkq+Ncekni72Olg7Nh8n1wh3sR4zGNWVgHsoKruBBsieLNktrBt3bQy3 fBaqe+cirFzqbHWEfyjXP+095P5nsdrIIMMY0ntyFNmPCaXI/A9Jt5bZZoV61Rfq Icjpcycy/4GC74IsIEUsA== X-ME-Sender: <xms:ryq5ZOIg1uaziGx2U17YbyarpLsIzwtPW6z9Otv5EoPgNsV8PLa3Qw> <xme:ryq5ZGIoBT_gFu9xijve9C9-eK7DUfSd1U6hwFXN4eKu2TTxUz_0Db1UQUL4WeXWv EHd4tO-3_k9L5idhSM> X-ME-Received: <xmr:ryq5ZOuVMqXuW3W2z9rvss2-FCYOPXh5ULchoCgF64QKQR77qgsVXaUah6j-BtA> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedtgdehgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkffggfgfuvfhfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhithhr hicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrthhtvg hrnhepgeehtdehudejieetuedttdejkeehjeetueekheffuddtjeeuvdefveeuveeiuddu necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepughmih htrhihsehguhhtohhvrdguvghv X-ME-Proxy: <xmx:ryq5ZDZjwgy55FwHBcqvrdNHskelja29lMAngqc2Y9VOk8MWI4SpJw> <xmx:ryq5ZFaG8pmBahNlovbgUlsbLjOsIbtBJzt75e8pUQhqwi6na42aJg> <xmx:ryq5ZPASI10-9PmfdJ8VHW4qi6hzCPHMM2kUEKpFjrRE2EomVDhaFA> <xmx:sCq5ZGBl5n5a4FNXIsbsInITSiywmmfFDLbGz7MzDoPetaknpM_UuA> Feedback-ID: i0e71465a:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 20 Jul 2023 08:38:07 -0400 (EDT) Message-ID: <1fd5e3ed-e1c3-5d6e-897f-1d5d55e379fa@HIDDEN> Date: Thu, 20 Jul 2023 15:38:04 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores Content-Language: en-US To: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org References: <iermszrwqj4.fsf@HIDDEN> From: Dmitry Gutov <dmitry@HIDDEN> In-Reply-To: <iermszrwqj4.fsf@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -0.1 (/) X-Debbugs-Envelope-To: 64735 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.8 (-) On 20/07/2023 00:16, Spencer Baugh wrote: > In Emacs alone, there are a few things we could do: > - we could mitigate the find bug by optimizing the regexp before we pass > it to find; this should basically remove all the overhead but makes the > find command uglier and harder to edit > - we could remove rare and likely irrelevant things from > completion-ignored-extensions and vc-ignore-dir-regexp (which are used > to build these lists of ignores) I like these two approaches. > - we could use our own recursive directory-tree walking implementation > (directory-files-recursively), if we found a nice way to pipe its output > directly to grep etc without going through Lisp. (This could be nice > for project-files, at least) This will probably not work as well. Last I checked, Lisp-native file listing was simply slower than 'find'.
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 12:22:29 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 08:22:29 2023 Received: from localhost ([127.0.0.1]:57916 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMSft-0007u9-7F for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 08:22:29 -0400 Received: from s.wrqvtzvf.outbound-mail.sendgrid.net ([149.72.126.143]:9974) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <bounces+21787432-b76c-64735=debbugs.gnu.org@HIDDEN>) id 1qMSfp-0007tl-SU for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 08:22:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=catern.com; h=from:subject:in-reply-to:references:mime-version:to:cc:content-type: content-transfer-encoding:cc:content-type:from:subject:to; s=s1; bh=UiqMYTQYxvgL3MBaew3BvGTzw7g3XSXWfNVimM1Y7Fw=; b=UdPXIw/bRReu3aWEAmOgXEZBgGAorj39Fsv0pMIcwt0hmuHeY3fQWrzUgwXdlifySkLx /bxomw+FaN8iWU1k47PlokNHrDdFoUrB+TcpmBSANgvN1ncAGqhBKVx5BDYgk7JIrCwpIR nyCFCnPgbwm0LHwnj727nquP8LNoWQEJUPJmnvLfgvvTfpACy28xEZ7sX8YH9ZSpVPzyTW 1Wq4yhbRSwVH9omXQmwYMM2QElI9el4WMl+nELvkuKz72ioZBZe31oMBKY0XEQBkadk4VK kH1/S1eRUATeaPMs6YYutEriDxegyOBPpzvT4e3jFzBvku6KIosO1nuSdRGLWI7w== Received: by filterdrecv-84b96456cb-ggcm8 with SMTP id filterdrecv-84b96456cb-ggcm8-1-64B926FB-3 2023-07-20 12:22:19.133355491 +0000 UTC m=+6093829.264883699 Received: from earth.catern.com (unknown) by geopod-ismtpd-21 (SG) with ESMTP id 1cEVyLHCSEWk33BgfeXgcg Thu, 20 Jul 2023 12:22:18.966 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=::1; helo=localhost; envelope-from=sbaugh@HIDDEN; receiver=gnu.org Received: from localhost (localhost [IPv6:::1]) by earth.catern.com (Postfix) with ESMTPSA id 6EF5760166; Thu, 20 Jul 2023 08:22:18 -0400 (EDT) From: sbaugh@HIDDEN Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores In-Reply-To: <837cqv41ob.fsf@HIDDEN> (Eli Zaretskii's message of "Thu, 20 Jul 2023 08:00:52 +0300") References: <iermszrwqj4.fsf@HIDDEN> <837cqv41ob.fsf@HIDDEN> Date: Thu, 20 Jul 2023 12:22:19 +0000 (UTC) Message-ID: <87mszqixhh.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 X-SG-EID: =?us-ascii?Q?ZgbRq7gjGrt0q=2FPjvxk7wM0yQFRdOkTJAtEbkjCkHbLj6wGsjDHg7xvKHIL=2Fey?= =?us-ascii?Q?25KEMZmf2YDGJ0YTmF1VpAa1ZeMkuuTv+lYnC+Y?= =?us-ascii?Q?I63BCfDIXY+O6E1TrpH9Pg4wFH0v+AYij4tmvxX?= =?us-ascii?Q?z4JBa0G7wrGZp3lO9vZhPjQ7T6UNEiK8iLJ05nS?= =?us-ascii?Q?MrWgk10RolcDY=2FIlZpGEWN2GDpF8d6jJiRQ=3D=3D?= To: Eli Zaretskii <eliz@HIDDEN> X-Entity-ID: d/0VcHixlS0t7iB1YKCv4Q== Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Eli Zaretskii <eliz@HIDDEN> writes: >> From: Spencer Baugh <sbaugh@HIDDEN> >> Date: Wed, 19 Jul 2023 17:16:31 -0400 >> >> >> Several important commands and functions invoke find; for example [...] Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.2 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net [Blocked - see <https://www.spamcop.net/bl.shtml?149.72.126.143>] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [149.72.126.143 listed in wl.mailspike.net] 0.0 UNPARSEABLE_RELAY Informational: message has unparseable relay lines 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders -0.0 T_SCC_BODY_TEXT_LINE No description available. X-Debbugs-Envelope-To: 64735 Cc: Spencer Baugh <sbaugh@HIDDEN>, 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: 0.2 (/) Eli Zaretskii <eliz@HIDDEN> writes: >> From: Spencer Baugh <sbaugh@HIDDEN> >> Date: Wed, 19 Jul 2023 17:16:31 -0400 >> >> >> Several important commands and functions invoke find; for example rgrep >> and project-find-regexp. >> >> Most of these add some set of ignores to the find command, pulling from >> grep-find-ignored-files in the former case. So the find command looks >> like: >> >> find -H . \( -path \*/SCCS/\* -o -path \*/RCS/\* [...more ignores...] \) >> -prune -o -type f -print0 >> >> Alas, on my system, using GNU find, these ignores slow down find by >> about 15x on a large directory tree, taking it from around .5 seconds to >> 7.8 seconds. >> >> This is very noticeable overhead; removing the ignores makes rgrep and >> other find-invoking commands substantially faster for me. > > grep-find-ignored-files is a customizable user option, so if this > slowdown bothers you, just customize it to avoid that. I think the fact that the default behavior is very slow, is bad. > And if there are patterns there that are no longer pertinent or rare, > we could remove them from the default value. Sure! So the thing to narrow down would be completion-ignored-extensions, which is what populates grep-find-ignored-files. Most things in that list are irrelevant to most users, but all of them are relevant to some users. Most of these are language-specific things - e.g. there's a bunch of Common Lisp compiled object (or something) extensions. Perhaps we could modularize this, so that individual packages add things to completion-ignored-extensions at load time. Then completion-ignored-extensions would only include things which are relevant to a given user, as determined by what packages they load. > I'm not sure we should bother more than these two simple measures. Unfortunately those two simple measures help rgrep but they don't help project-find-regexp (and others project.el commands using project--files-in-directory such as project-find-file), since those project commands pull their ignores from the version control system through vc (not grep-find-ignored-files), and then pass them to find. >> The overhead is linear in the number of ignores - that is, each >> additional ignore adds a small fixed cost. This suggests that find is >> linearly scanning the list of ignores and checking each one, rather than >> optimizing them to a single regexp and checking that regexp. > > If it uses fnmatch, it cannot do it any other way, I think
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at 64735) by debbugs.gnu.org; 20 Jul 2023 05:00:35 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jul 20 01:00:35 2023 Received: from localhost ([127.0.0.1]:57476 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMLmF-0000dP-Bd for submit <at> debbugs.gnu.org; Thu, 20 Jul 2023 01:00:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38924) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1qMLmD-0000dD-OT for 64735 <at> debbugs.gnu.org; Thu, 20 Jul 2023 01:00:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMLm5-0008Tb-BP; Thu, 20 Jul 2023 01:00:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=Qajid7jJKHLDlWyI8UooSVse/iLLIIrDfCgNsoUijCc=; b=mRBs5bgjjFVS ZzzXHzH0TkiBLN3rKMY95qLleqgw82LrS1lYfXqAtG0RXv1x4eZuoZMeX+egx38EJI8gHOsapm12h hPKg1JyObjcAOQwZrqRmrqQOE6a6PpGiYfEN/gwLFiNeub8SZ2MhmVPj13Sgl1xlOj6ON31DVdQQu G/b/jDi5omqsJGE9fvwmhtk1OzHng+MY8RbGn4b4/S4MRg78LlqYOV/mJpFiUtUqs2Lhcz+w0G6AR XGVue1tIE5jH98GyELqtW5XbqdNPQskgczRXdYqnaJ28521TSCjrgWbd+h5257p72/lXeCb/yM3vv Hft2E/bOMN9zxsGcUuJ/lw==; Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1qMLm1-0002Ui-EF; Thu, 20 Jul 2023 01:00:24 -0400 Date: Thu, 20 Jul 2023 08:00:52 +0300 Message-Id: <837cqv41ob.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Spencer Baugh <sbaugh@HIDDEN> In-Reply-To: <iermszrwqj4.fsf@HIDDEN> (message from Spencer Baugh on Wed, 19 Jul 2023 17:16:31 -0400) Subject: Re: bug#64735: 29.0.92; find invocations are ~15x slower because of ignores References: <iermszrwqj4.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 64735 Cc: 64735 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Spencer Baugh <sbaugh@HIDDEN> > Date: Wed, 19 Jul 2023 17:16:31 -0400 > > > Several important commands and functions invoke find; for example rgrep > and project-find-regexp. > > Most of these add some set of ignores to the find command, pulling from > grep-find-ignored-files in the former case. So the find command looks > like: > > find -H . \( -path \*/SCCS/\* -o -path \*/RCS/\* [...more ignores...] \) > -prune -o -type f -print0 > > Alas, on my system, using GNU find, these ignores slow down find by > about 15x on a large directory tree, taking it from around .5 seconds to > 7.8 seconds. > > This is very noticeable overhead; removing the ignores makes rgrep and > other find-invoking commands substantially faster for me. grep-find-ignored-files is a customizable user option, so if this slowdown bothers you, just customize it to avoid that. And if there are patterns there that are no longer pertinent or rare, we could remove them from the default value. I'm not sure we should bother more than these two simple measures. > The overhead is linear in the number of ignores - that is, each > additional ignore adds a small fixed cost. This suggests that find is > linearly scanning the list of ignores and checking each one, rather than > optimizing them to a single regexp and checking that regexp. If it uses fnmatch, it cannot do it any other way, I think
bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.Received: (at submit) by debbugs.gnu.org; 19 Jul 2023 21:16:47 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 19 17:16:46 2023 Received: from localhost ([127.0.0.1]:57195 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1qMEXO-00048U-DV for submit <at> debbugs.gnu.org; Wed, 19 Jul 2023 17:16:46 -0400 Received: from lists.gnu.org ([2001:470:142::17]:46920) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <sbaugh@HIDDEN>) id 1qMEXK-00048F-3o for submit <at> debbugs.gnu.org; Wed, 19 Jul 2023 17:16:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <sbaugh@HIDDEN>) id 1qMEXE-0006kz-T1 for bug-gnu-emacs@HIDDEN; Wed, 19 Jul 2023 17:16:36 -0400 Received: from mxout5.mail.janestreet.com ([64.215.233.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <sbaugh@HIDDEN>) id 1qMEXB-0007dQ-V9 for bug-gnu-emacs@HIDDEN; Wed, 19 Jul 2023 17:16:36 -0400 From: Spencer Baugh <sbaugh@HIDDEN> To: bug-gnu-emacs@HIDDEN Subject: 29.0.92; find invocations are ~15x slower because of ignores Date: Wed, 19 Jul 2023 17:16:31 -0400 Message-ID: <iermszrwqj4.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=64.215.233.18; envelope-from=sbaugh@HIDDEN; helo=mxout5.mail.janestreet.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.1 (/) Several important commands and functions invoke find; for example rgrep and project-find-regexp. Most of these add some set of ignores to the find command, pulling from grep-find-ignored-files in the former case. So the find command looks like: find -H . \( -path \*/SCCS/\* -o -path \*/RCS/\* [...more ignores...] \) -prune -o -type f -print0 Alas, on my system, using GNU find, these ignores slow down find by about 15x on a large directory tree, taking it from around .5 seconds to 7.8 seconds. This is very noticeable overhead; removing the ignores makes rgrep and other find-invoking commands substantially faster for me. The overhead is linear in the number of ignores - that is, each additional ignore adds a small fixed cost. This suggests that find is linearly scanning the list of ignores and checking each one, rather than optimizing them to a single regexp and checking that regexp. Obviously, GNU find should be optimizing this. However they have previously said they will not optimize this; I commented on this bug https://savannah.gnu.org/bugs/index.php?58197 to request they rethink that. Hopefully as a fellow GNU project they will be interested in helping us... In Emacs alone, there are a few things we could do: - we could mitigate the find bug by optimizing the regexp before we pass it to find; this should basically remove all the overhead but makes the find command uglier and harder to edit - we could remove rare and likely irrelevant things from completion-ignored-extensions and vc-ignore-dir-regexp (which are used to build these lists of ignores) - we could use our own recursive directory-tree walking implementation (directory-files-recursively), if we found a nice way to pipe its output directly to grep etc without going through Lisp. (This could be nice for project-files, at least) Incidentally, I tried a find alternative, "bfs", https://github.com/tavianator/bfs and it doesn't optimize this either, sadly, so it also has the 15x slowdown. In GNU Emacs 29.0.92 (build 5, x86_64-pc-linux-gnu, X toolkit, cairo version 1.15.12, Xaw scroll bars) of 2023-07-10 built on Repository revision: dd15432ffacbeff0291381c0109f5b1245060b1d Repository branch: emacs-29 Windowing system distributor 'The X.Org Foundation', version 11.0.12011000 System Description: Rocky Linux 8.8 (Green Obsidian) Configured using: 'configure --config-cache --with-x-toolkit=lucid --with-gif=ifavailable' Configured features: CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM LUCID ZLIB Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: Shell Memory information: ((conses 16 1939322 193013) (symbols 48 76940 49) (strings 32 337371 45355) (string-bytes 1 12322013) (vectors 16 148305) (vector-slots 8 3180429 187121) (floats 8 889 751) (intervals 56 152845 1238) (buffers 976 235) (heap 1024 978725 465480))
Spencer Baugh <sbaugh@HIDDEN>:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#64735; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.