GNU bug report logs - #26710
project-find-regexp blocks the UI

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Severity: wishlist; Reported by: Hariharan Rangasamy <hariharanrangasamy@HIDDEN>; dated Sat, 29 Apr 2017 16:55:03 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 3 May 2017 02:35:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 02 22:35:22 2017
Received: from localhost ([127.0.0.1]:51736 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d5k8Q-0005Dz-Gn
	for submit <at> debbugs.gnu.org; Tue, 02 May 2017 22:35:22 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55468)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1d5k8O-0005Dm-Op
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 22:35:21 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1d5k8F-0000G6-Kc
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 22:35:15 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:40165)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1d5k8F-0000G0-Gz; Tue, 02 May 2017 22:35:11 -0400
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3725
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1d5k8E-0003aV-OS; Tue, 02 May 2017 22:35:11 -0400
Date: Wed, 03 May 2017 05:34:47 +0300
Message-Id: <83wp9yhdmg.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dgutov@HIDDEN>
In-reply-to: <5d72a20b-30b5-51fd-fbc0-ca5fa95573f7@HIDDEN> (message from
 Dmitry Gutov on Wed, 3 May 2017 03:14:52 +0300)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> <83ziexka0s.fsf@HIDDEN>
 <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN> <83inlljb5r.fsf@HIDDEN>
 <b283056e-7b17-86c2-7d59-1f9015146130@HIDDEN> <83lgqfivb0.fsf@HIDDEN>
 <9474c678-b092-81f0-f5b3-f26d4467ac86@HIDDEN> <838tmfi30k.fsf@HIDDEN>
 <5d72a20b-30b5-51fd-fbc0-ca5fa95573f7@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
> From: Dmitry Gutov <dgutov@HIDDEN>
> Date: Wed, 3 May 2017 03:14:52 +0300
> 
> On 02.05.2017 20:26, Eli Zaretskii wrote:
> 
> > The sentinel/filter won't be called at all if keyboard/mouse input is
> > available.  Once they are called, if each call takes a long processing
> > time, the UI could feel sluggish, yes.
> 
> Hmm, and if we're in "many calls, each of them fairly fast" situation?

Then there should be no problem with UI responsiveness.

> Sounds like the UI might be quite usable (but doing anything with it 
> would slow down the processing of search results).

Correct.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 3 May 2017 00:15:05 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 02 20:15:05 2017
Received: from localhost ([127.0.0.1]:51697 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d5hwf-00028a-JD
	for submit <at> debbugs.gnu.org; Tue, 02 May 2017 20:15:05 -0400
Received: from mail-wm0-f54.google.com ([74.125.82.54]:38664)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raaahh@HIDDEN>) id 1d5hwd-00027q-Id
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 20:15:03 -0400
Received: by mail-wm0-f54.google.com with SMTP id r190so39068502wme.1
 for <26710 <at> debbugs.gnu.org>; Tue, 02 May 2017 17:15:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:subject:to:cc:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=c0Xr6c3HL28txLafUATtAEomNlU7LW59Aro9+1pdMX4=;
 b=uNerrVNT4LTbC2P1TLXf+6hmeVkCzhEY06vJkgo5iFwObueVCCDhwZbzb5U/1sdv0S
 RbgrPDTRU95f6ce5Tdi/a3gAJudv2RU3i6djbdoaxHh8TlAxPAiAAnzPIMA4HRrs0G+4
 v52k6QMu2VRkCXKyXpV/rg/opnGkUGUQ+RKBmq8obyo7vlgfv9WlXOZR0TWesU2QGqfm
 dCAPFBRtBEvt1GbGKBJLq+ZUz2D+0hvGbbm9b6m0wNLAp+zItI/OEzNv9Hjz7WHCM8np
 VFLI6/rrQkK7EhIN2A/C2jceyIQGnMaEhZOzgnG3EDnq7KOUGMJ5UNaqPvx2SP12mZ+e
 FxHw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:subject:to:cc:references:from:message-id
 :date:user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=c0Xr6c3HL28txLafUATtAEomNlU7LW59Aro9+1pdMX4=;
 b=A/LU38i33JU0u3k5O3BWLMVYuvXJqblQp7bBCBRwZyv2mUkT7lYHj7M3nnSFI2shY+
 y4JpUm0Aad/k92DaaHa1iYxgQlZdXO9JPDXi7100G2rfPw50ZncDjSfyTWL9aI+AKo/C
 ZhMAJBITympwfiMrmX2gFdaA9vv7fHr4d86Cxli+tbt1LQgBusufSWWtc8+WenMftNh6
 /+D/nGCRsEA6rgAYkhjKUgNUSdHRkUYKMxe7cNVVu8rQFW2dKVuEbNDjjL2FZKb+WQEL
 ylHWNJqM5qgzM3n1shfCJlYiREHmGgGRs+aZAGtpbN9811v9ZaE5cFBbBy2m7gN6aEfm
 oa6w==
X-Gm-Message-State: AN3rC/6XqX/TGF7POV22zzRwl0i/frys2mojhEP8+B3c3iOayq4J492O
 HisNzeDQcHxHgQ==
X-Received: by 10.28.214.212 with SMTP id n203mr4392104wmg.61.1493770497934;
 Tue, 02 May 2017 17:14:57 -0700 (PDT)
Received: from [192.168.1.3] ([185.105.173.156])
 by smtp.googlemail.com with ESMTPSA id p17sm2653662wmd.11.2017.05.02.17.14.55
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 02 May 2017 17:14:57 -0700 (PDT)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
To: Eli Zaretskii <eliz@HIDDEN>
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> <83ziexka0s.fsf@HIDDEN>
 <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN> <83inlljb5r.fsf@HIDDEN>
 <b283056e-7b17-86c2-7d59-1f9015146130@HIDDEN> <83lgqfivb0.fsf@HIDDEN>
 <9474c678-b092-81f0-f5b3-f26d4467ac86@HIDDEN> <838tmfi30k.fsf@HIDDEN>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <5d72a20b-30b5-51fd-fbc0-ca5fa95573f7@HIDDEN>
Date: Wed, 3 May 2017 03:14:52 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101
 Thunderbird/53.0
MIME-Version: 1.0
In-Reply-To: <838tmfi30k.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.7 (/)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.7 (/)

On 02.05.2017 20:26, Eli Zaretskii wrote:

> The sentinel/filter won't be called at all if keyboard/mouse input is
> available.  Once they are called, if each call takes a long processing
> time, the UI could feel sluggish, yes.

Hmm, and if we're in "many calls, each of them fairly fast" situation?

Sounds like the UI might be quite usable (but doing anything with it 
would slow down the processing of search results).

> But I don't quite see how
> using threads will avoid the same problem, since the mechanism for
> thread switch is basically the same as for multiplexing UI with
> subprocess output.

Right, threads would server only to make the code more readable. With 
filters, we'll have callbacks.

Threads can make this code look sequential, like iterating over a sequence.

> IMO, we should first explore the async subprocess road.

OK.

>> We'll probably be saved by filters having to wait until the current
>> command finishes executing, though.
> 
> Not sure I follow you: a filter function is called whenever some
> output arrives from the subprocess.  So they don't need to wait for
> the subprocess to finish.

The current command, as in "Emacs command loop" command. Anyway, you've 
addressed this issue in the next email.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 2 May 2017 17:42:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 02 13:42:47 2017
Received: from localhost ([127.0.0.1]:51422 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d5bp0-0006UH-P8
	for submit <at> debbugs.gnu.org; Tue, 02 May 2017 13:42:46 -0400
Received: from eggs.gnu.org ([208.118.235.92]:32970)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1d5boz-0006U5-SV
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 13:42:46 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1d5bor-0001On-Ku
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 13:42:40 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33805)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1d5bor-0001Ob-H1; Tue, 02 May 2017 13:42:37 -0400
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3134
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1d5boo-00037L-Mg; Tue, 02 May 2017 13:42:37 -0400
Date: Tue, 02 May 2017 20:41:57 +0300
Message-Id: <837f1zi2ai.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: dgutov@HIDDEN
In-reply-to: <838tmfi30k.fsf@HIDDEN> (message from Eli Zaretskii on Tue, 02
 May 2017 20:26:19 +0300)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN>
 <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN>
 <83ziexka0s.fsf@HIDDEN>
 <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN>
 <83inlljb5r.fsf@HIDDEN>
 <b283056e-7b17-86c2-7d59-1f9015146130@HIDDEN>
 <83lgqfivb0.fsf@HIDDEN>
 <9474c678-b092-81f0-f5b3-f26d4467ac86@HIDDEN> <838tmfi30k.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> Date: Tue, 02 May 2017 20:26:19 +0300
> From: Eli Zaretskii <eliz@HIDDEN>
> Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
> 
> > It seems a bit brittle, though: if the process filter is supposed to be 
> > calling the callback for each item, the callback has to be in place 
> > right away. And the process will be started before that happens.
> 
> You can countermand that by using make-process with the :stop
> attribute, then use 'continue-process' when everything is set up.

Darn, this won't work on systems without SIGCONT support, like
MS-Windows.

But I don't think this is a real problem anyway: Emacs will not try
sensing for subprocess output until it becomes idle, so as long as the
code which sets up the process's filter and sentinel functions and
their respective callbacks runs, Emacs will not try to call the
filter/sentinel functions.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 2 May 2017 17:27:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 02 13:27:08 2017
Received: from localhost ([127.0.0.1]:51408 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d5bZs-00067c-ME
	for submit <at> debbugs.gnu.org; Tue, 02 May 2017 13:27:08 -0400
Received: from eggs.gnu.org ([208.118.235.92]:57094)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1d5bZr-00067P-Ax
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 13:27:07 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1d5bZj-0003EI-2i
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 13:27:02 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33591)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1d5bZi-0003E8-VH; Tue, 02 May 2017 13:26:58 -0400
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3121
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1d5bZg-0003bz-6k; Tue, 02 May 2017 13:26:58 -0400
Date: Tue, 02 May 2017 20:26:19 +0300
Message-Id: <838tmfi30k.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dgutov@HIDDEN>
In-reply-to: <9474c678-b092-81f0-f5b3-f26d4467ac86@HIDDEN> (message from
 Dmitry Gutov on Tue, 2 May 2017 13:00:06 +0300)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> <83ziexka0s.fsf@HIDDEN>
 <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN> <83inlljb5r.fsf@HIDDEN>
 <b283056e-7b17-86c2-7d59-1f9015146130@HIDDEN> <83lgqfivb0.fsf@HIDDEN>
 <9474c678-b092-81f0-f5b3-f26d4467ac86@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
> From: Dmitry Gutov <dgutov@HIDDEN>
> Date: Tue, 2 May 2017 13:00:06 +0300
> 
> On 02.05.2017 10:15, Eli Zaretskii wrote:
> 
> > Can you explain the significance of xref--regexp-syntax-dependent-p's
> > tests?  I don't know enough about xref to grasp that just by looking
> > at the changes.
> 
> When it returns nil (the regexp is not affected by syntax-table):
> 
> If the file containing the hit is not open, we now skip inserting the 
> first few lines of that file into the temporary buffer, and calling 
> set-auto-mode.
> 
> And, whether it's open or not, we skip the syntax-propertize call.

OK, I will look at that function with this in mind.

> Still, if the filter function and sentinel functions take a lot of time 
> (and/or get called a lot), like it will be in this example, the UI can't 
> as responsive as usual, can it?

The sentinel/filter won't be called at all if keyboard/mouse input is
available.  Once they are called, if each call takes a long processing
time, the UI could feel sluggish, yes.  But I don't quite see how
using threads will avoid the same problem, since the mechanism for
thread switch is basically the same as for multiplexing UI with
subprocess output.

> I'd like a more general advice first. E.g. do we want to go this road? 

IMO, we should first explore the async subprocess road.

> It seems a bit brittle, though: if the process filter is supposed to be 
> calling the callback for each item, the callback has to be in place 
> right away. And the process will be started before that happens.

You can countermand that by using make-process with the :stop
attribute, then use 'continue-process' when everything is set up.

> We'll probably be saved by filters having to wait until the current 
> command finishes executing, though.

Not sure I follow you: a filter function is called whenever some
output arrives from the subprocess.  So they don't need to wait for
the subprocess to finish.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 2 May 2017 10:00:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 02 06:00:18 2017
Received: from localhost ([127.0.0.1]:50337 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d5UbS-000229-LV
	for submit <at> debbugs.gnu.org; Tue, 02 May 2017 06:00:18 -0400
Received: from mail-wm0-f50.google.com ([74.125.82.50]:38211)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raaahh@HIDDEN>) id 1d5UbP-00021u-5p
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 06:00:16 -0400
Received: by mail-wm0-f50.google.com with SMTP id r190so13211895wme.1
 for <26710 <at> debbugs.gnu.org>; Tue, 02 May 2017 03:00:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:subject:to:cc:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=g0DYidR6e5Hn4EJ+tVKGYAUL5J1PZEUs9T0O8S4tjTo=;
 b=ieZRlNVRP7P4S56c/Mk3ObhMsnuZlBPTLcZ3a2uKINELrgljgJXeY9oD/l6+7CYCTE
 B+TFnrk5fDTYONc4wiq5ysrL47Sj/R7iJ7tLK7rnQFQ1lilVtYD7L50vEJZIvPH3/2Ku
 7xKIpf09rtDOwJfRG/s/2lkzC+ZIT9pyZe4vRoKj9vrd+xy/7Db6BBKQMeoqrZBHZTpV
 a+dGdEYFXgPqTK6GyCbexRYi0tcs9Y9cHOYbuzYMZjGF6g10XcJsbVKx+ZcV/Qa3nPL2
 Xl0TgR94/e7+gpkiCnGwX4sSj/LbWer7fQVTjwiya8eueCjYGfGAVMkIiC1XWpqBKwb+
 uTZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:subject:to:cc:references:from:message-id
 :date:user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=g0DYidR6e5Hn4EJ+tVKGYAUL5J1PZEUs9T0O8S4tjTo=;
 b=EikTFxjoljC/60fPBS//m8staGGhSuX7n2QZUoA2dQV+ERJompuYkFWRxHiiGzxgBs
 Bk8IJ8zTOKTgVNL9rzzvOSi+Nktl4erbjr1px7VYcyYO9n2smH6ytBFRXpPq3UdCWMXS
 rY+W+tKHg8Y2sr0ZE2TGZCMrm3SA7pU4S5u0YkBKZ3BJG57/1h+FEIYV0Ioj6ZSNLJB6
 4LCzJrxk4OFVLe8vSAkp6ZOr5Sj3Xt6T7lLK2lD7oPdLwHtTToGzon0FxGRGiYQq55Hz
 QmhhVPMiQQCTdxkM8lSD2vAkF66e7pgYMtcUMszzw5QIz2l6XvXpe3ktIPF79iI2dtfZ
 2DgA==
X-Gm-Message-State: AN3rC/4gALTRgSaK8f4JVxXXpZZmJKIhrmOj//bEaUUTzjqNnl2Gbed8
 B5ciIj+TYvUgSw==
X-Received: by 10.28.0.201 with SMTP id 192mr1613001wma.126.1493719209376;
 Tue, 02 May 2017 03:00:09 -0700 (PDT)
Received: from [192.168.1.3] ([185.105.173.156])
 by smtp.googlemail.com with ESMTPSA id l82sm378164wmf.17.2017.05.02.03.00.07
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 02 May 2017 03:00:08 -0700 (PDT)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
To: Eli Zaretskii <eliz@HIDDEN>
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> <83ziexka0s.fsf@HIDDEN>
 <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN> <83inlljb5r.fsf@HIDDEN>
 <b283056e-7b17-86c2-7d59-1f9015146130@HIDDEN> <83lgqfivb0.fsf@HIDDEN>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <9474c678-b092-81f0-f5b3-f26d4467ac86@HIDDEN>
Date: Tue, 2 May 2017 13:00:06 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101
 Thunderbird/53.0
MIME-Version: 1.0
In-Reply-To: <83lgqfivb0.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.2 (/)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.2 (/)

On 02.05.2017 10:15, Eli Zaretskii wrote:

> Can you explain the significance of xref--regexp-syntax-dependent-p's
> tests?  I don't know enough about xref to grasp that just by looking
> at the changes.

When it returns nil (the regexp is not affected by syntax-table):

If the file containing the hit is not open, we now skip inserting the 
first few lines of that file into the temporary buffer, and calling 
set-auto-mode.

And, whether it's open or not, we skip the syntax-propertize call.

>> With this, project-find-regexp for 'emacs' finally completes in ~10
>> seconds on my machine.
> 
> It takes about 15 here (and 45 in an unoptimized build).  I guess this
> slowdown is expected, since this is a 32-bit build --with-wide-int, so
> should be 30% slower than with native ints.

Thanks for testing. To be more accurate, it's about 10 seconds in my 
normal session, and about 6 seconds starting with 'emacs -Q'. My laptop 
is most likely faster.

> If the processing is in filter and sentinel functions, I'm not sure we
> will need any further speedups, because the UI will remain responsive.

The filter and sentinel functions are not allowed to have direct access 
to the final output buffer, hence the need for abstraction.

I guess you favor the "one callback per hit" approach, then.

Still, if the filter function and sentinel functions take a lot of time 
(and/or get called a lot), like it will be in this example, the UI can't 
as responsive as usual, can it?

>>> But that doesn't need
>>> to involve threads, and is being done in many packages/features out
>>> there, so I'm not sure what did you ask me to do with this.
>>
>> I imagined that the xref API that allows this kind of asynchronous
>> results might look better and more readable if it's implemented with
>> threads underneath.
> 
> If you need advice for how to implement something like that, I can try
> helping with threads.

I'd like a more general advice first. E.g. do we want to go this road? 
The dir-status-files like scheme should work without threads, too.

It seems a bit brittle, though: if the process filter is supposed to be 
calling the callback for each item, the callback has to be in place 
right away. And the process will be started before that happens.

We'll probably be saved by filters having to wait until the current 
command finishes executing, though.

>> The main thing to understand is the xref API, not the internals of the
>> package.
> 
> Well, I lack that understanding as well.

I'm hoping it's not too hard to obtain even just by reading the 
Commentary section in xref.el. But hey, you don't have to.

The callbacks approach seems viable, too.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 2 May 2017 07:15:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 02 03:15:56 2017
Received: from localhost ([127.0.0.1]:50212 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d5S2M-00032t-Ip
	for submit <at> debbugs.gnu.org; Tue, 02 May 2017 03:15:56 -0400
Received: from eggs.gnu.org ([208.118.235.92]:46372)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1d5S2K-00032g-Uf
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 03:15:53 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1d5S2B-00048T-3b
 for 26710 <at> debbugs.gnu.org; Tue, 02 May 2017 03:15:47 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:53709)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1d5S2A-000480-Ml; Tue, 02 May 2017 03:15:43 -0400
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1928
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1d5S28-00086g-AJ; Tue, 02 May 2017 03:15:41 -0400
Date: Tue, 02 May 2017 10:15:15 +0300
Message-Id: <83lgqfivb0.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dgutov@HIDDEN>
In-reply-to: <b283056e-7b17-86c2-7d59-1f9015146130@HIDDEN> (message from
 Dmitry Gutov on Tue, 2 May 2017 00:46:25 +0300)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> <83ziexka0s.fsf@HIDDEN>
 <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN> <83inlljb5r.fsf@HIDDEN>
 <b283056e-7b17-86c2-7d59-1f9015146130@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
> From: Dmitry Gutov <dgutov@HIDDEN>
> Date: Tue, 2 May 2017 00:46:25 +0300
> 
> See commit c99a3b9. Please take a look at 
> xref--regexp-syntax-dependent-p specifically, and see if any significant 
> false negatives come to mind.

Can you explain the significance of xref--regexp-syntax-dependent-p's
tests?  I don't know enough about xref to grasp that just by looking
at the changes.

> With this, project-find-regexp for 'emacs' finally completes in ~10 
> seconds on my machine.

It takes about 15 here (and 45 in an unoptimized build).  I guess this
slowdown is expected, since this is a 32-bit build --with-wide-int, so
should be 30% slower than with native ints.

I don't remember the original timings, but this looks like a good
improvement, thanks.

> >> What we _can_ manage to run in parallel, in the find-grep process in the
> >> background, and the post-processing of the results in Elisp.
> > 
> > Yes, you can -- if you invoke find-grep asynchronously and move the
> > processing of the hits to the filter function.
> 
> Yes, these parts are necessary either way. What I was describing would 
> go on top of them, as an abstraction.

If the processing is in filter and sentinel functions, I'm not sure we
will need any further speedups, because the UI will remain responsive.

> > But that doesn't need
> > to involve threads, and is being done in many packages/features out
> > there, so I'm not sure what did you ask me to do with this.
> 
> I imagined that the xref API that allows this kind of asynchronous 
> results might look better and more readable if it's implemented with 
> threads underneath.

If you need advice for how to implement something like that, I can try
helping with threads.

> The main thing to understand is the xref API, not the internals of the 
> package.

Well, I lack that understanding as well.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 1 May 2017 21:46:38 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon May 01 17:46:38 2017
Received: from localhost ([127.0.0.1]:50031 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d5J9R-0004h4-SP
	for submit <at> debbugs.gnu.org; Mon, 01 May 2017 17:46:38 -0400
Received: from mail-wr0-f172.google.com ([209.85.128.172]:34114)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raaahh@HIDDEN>) id 1d5J9O-0004gq-Iq
 for 26710 <at> debbugs.gnu.org; Mon, 01 May 2017 17:46:35 -0400
Received: by mail-wr0-f172.google.com with SMTP id l9so68997326wre.1
 for <26710 <at> debbugs.gnu.org>; Mon, 01 May 2017 14:46:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:subject:to:cc:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=+rPPyCKncW6BCXmgbCfFLmAeyN0ot7U4Y0VEi5LWIl8=;
 b=cTETh+RfGfF/tYHw314GOtGic1wlzNjxbVY59sjf6LpWXiWGshjXhzl+XLTvyERKAy
 UHMZQmuC7VYFDtbzV56ko/uQnUHbQ/8kXvLJuDcG5stX3rkBL5TSbaxANAwP4UPbI103
 I4rba2Fsqbpq2SAfAqUvXPUEtLK+q30lT9CMgAjHjrqwoj2lTSeHIhvJ1lgD9+cBYXh+
 ZsJgNJ75i9op3jb9e8L96EIAuKJ5zC98KhMmHDNc55jGEJRR5TX/2shF4UgnCeBD1cZc
 aegTWVcnTX5p2V5S9TDObA74L+H6p7GL8Hq0uOhvmsm6NmEMsx5HH4b+T1YU1y9KAx34
 qUtQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:subject:to:cc:references:from:message-id
 :date:user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=+rPPyCKncW6BCXmgbCfFLmAeyN0ot7U4Y0VEi5LWIl8=;
 b=D7uY+NCZAOu1urxIeaNW4/AqKyzaPQ1jKZW+JXh1nHLhsFg21o57j6CvPPdort7A4c
 Hy5ij/zJ1jIbMEAR06sBnE4ItDGwTypTCFQZToqYkFLL01nIMLdT924Cmf4lB3HRnuTj
 izdau5RE/nj5D4/BaKC/OWZzsGi0peoRplyqY/ViqdZf/SoXqSloJUC/QtxaAcGZhubf
 WqgKV3Z0JOvYW588FoFbXCo2g/q3ZDrESHkRCaZznm1RFYx5ApOXopwzV2JvKHSRLtig
 CUPg4vYD3HLvSSTFY3yH/wZWsv4PQZvFWfpv2UOG2RnaEvcE77KUFQvKLAlDUg2dU4JF
 egyg==
X-Gm-Message-State: AN3rC/668v8MyyQkAA0D77ZhkIQ2115laMlwGNjOxVhYJIM1FwM7XOwD
 bp0oLeR3xUVv5Q==
X-Received: by 10.223.134.238 with SMTP id 43mr16558298wry.80.1493675188674;
 Mon, 01 May 2017 14:46:28 -0700 (PDT)
Received: from [192.168.1.3] ([185.105.173.156])
 by smtp.googlemail.com with ESMTPSA id o18sm22250822wrb.47.2017.05.01.14.46.26
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Mon, 01 May 2017 14:46:27 -0700 (PDT)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
To: Eli Zaretskii <eliz@HIDDEN>
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> <83ziexka0s.fsf@HIDDEN>
 <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN> <83inlljb5r.fsf@HIDDEN>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <b283056e-7b17-86c2-7d59-1f9015146130@HIDDEN>
Date: Tue, 2 May 2017 00:46:25 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101
 Thunderbird/53.0
MIME-Version: 1.0
In-Reply-To: <83inlljb5r.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.6 (--)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

On 01.05.2017 10:20, Eli Zaretskii wrote:

> In my testing, find-grep finishes almost instantaneously.  The
> exception is when you have a cold cache, but even then it takes about
> 10% of the total run time, for the Emacs source tree (which yields
> about 100,000 hits in the test case).

This particular example, uses a very frequent term. I get 61000 hits or 
so, and it's still a lot, the search never finishes here (probably 
because I have more minor modes and customizations enabled).

I don't think this is the common case, but let's try to remove some 
unnecessary work in Elisp first.

See commit c99a3b9. Please take a look at 
xref--regexp-syntax-dependent-p specifically, and see if any significant 
false negatives come to mind.

With this, project-find-regexp for 'emacs' finally completes in ~10 
seconds on my machine. That's still more than 10 times longer than the 
external process takes, but I'm out of big optimization ideas at this point.

> I thought the request was to allow the user do something in the
> foreground, while this processing runs in the background.  If that's
> not what was requested, then I guess I no longer understand the
> request.

If the project is huge, and there are only a few hits, parallelizing the 
search and processing will allow the user to do whatever they want in 
the foreground. Because processing in Elisp, while slow, will still take 
a small fraction of the time.

If the search term returns a lot of hits (compared to the size of the 
project), processing might indeed take a lot of time, and the UI might 
appear sluggish (not sure how sluggish, though, that should depend on 
the scheduling of the main and background threads).

Even if it's sluggish, at least the user will see that the search has 
started, and there is some progress. We could even allow them to stop 
the search midway, and still do something with the first results.

These are some of the advantages 'M-x rgrep' has over project-find-regexp.

>> What we _can_ manage to run in parallel, in the find-grep process in the
>> background, and the post-processing of the results in Elisp.
> 
> Yes, you can -- if you invoke find-grep asynchronously and move the
> processing of the hits to the filter function.

Yes, these parts are necessary either way. What I was describing would 
go on top of them, as an abstraction.

> But that doesn't need
> to involve threads, and is being done in many packages/features out
> there, so I'm not sure what did you ask me to do with this.

I imagined that the xref API that allows this kind of asynchronous 
results might look better and more readable if it's implemented with 
threads underneath.

> IOW, it
> should be "trivial", at least in principle, to make this command work
> in the background, just like, say, "M-x grep".

In Compilation buffers (of which Grep is one example), the sentinel code 
has access to the buffer where the results are displayed. And the 
process outputs to that buffer as well. And 'M-x rgrep' doesn't have to 
abstract over possible way to obtain search results.

None of those are the case with the xref API, or the results rendering 
code, which has to work with the values returned by an arbitrary xref 
backend, as documented.

Right now, an xref backend implements several methods that are allowed 
to return the same type of value: "a list of xref items".

Our task, as I see it, is to generalize that return value type for 
asynchronous work, and to do that as sanely as possible.

Threads are not strictly necessary for this (see the last paragraph of 
my previous email), but this case seems like it could be a good, limited 
in scope, showcase for the threading functionality.

> I'm not sure I understand the need for this complexity, given that
> async subprocesses are available.  I'm probably missing something
> because I know too little about the internals of the involved code.

The main thing to understand is the xref API, not the internals of the 
package.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 1 May 2017 07:21:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon May 01 03:21:11 2017
Received: from localhost ([127.0.0.1]:48453 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d55du-0002Oq-Of
	for submit <at> debbugs.gnu.org; Mon, 01 May 2017 03:21:10 -0400
Received: from eggs.gnu.org ([208.118.235.92]:36638)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1d55dt-0002Oe-Bm
 for 26710 <at> debbugs.gnu.org; Mon, 01 May 2017 03:21:09 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1d55dj-0005p2-G3
 for 26710 <at> debbugs.gnu.org; Mon, 01 May 2017 03:21:04 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:34759)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1d55dj-0005or-Cc; Mon, 01 May 2017 03:20:59 -0400
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:3077
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1d55di-0005RA-8O; Mon, 01 May 2017 03:20:58 -0400
Date: Mon, 01 May 2017 10:20:32 +0300
Message-Id: <83inlljb5r.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dgutov@HIDDEN>
In-reply-to: <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN> (message from
 Dmitry Gutov on Mon, 1 May 2017 05:42:21 +0300)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> <83ziexka0s.fsf@HIDDEN>
 <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> Cc: hariharanrangasamy@HIDDEN, control <at> debbugs.gnu.org,
>  26710 <at> debbugs.gnu.org
> From: Dmitry Gutov <dgutov@HIDDEN>
> Date: Mon, 1 May 2017 05:42:21 +0300
> 
> On 30.04.2017 21:47, Eli Zaretskii wrote:
> 
> > I'll try to look at this.  According to my profiling, the lion's share
> > of time is taken by xref--collect-matches, so that's the place to try
> > concurrency.
> 
> I think that's too late. By the time xref--collect-matches is called 
> (and it's called for each hit), we've already spent time synchronously 
> waiting for the find-grep invocation to finish.

In my testing, find-grep finishes almost instantaneously.  The
exception is when you have a cold cache, but even then it takes about
10% of the total run time, for the Emacs source tree (which yields
about 100,000 hits in the test case).

> When there are a lot of matches, xref--collect-matches can take some 
> significant time, with opening the buffers and everything. That can be 
> optimized, however, as a separate issue, and I don't think there's 
> anything to parallelize there, since it all happens in Elisp.

I thought the request was to allow the user do something in the
foreground, while this processing runs in the background.  If that's
not what was requested, then I guess I no longer understand the
request.

> What we _can_ manage to run in parallel, in the find-grep process in the 
> background, and the post-processing of the results in Elisp.

Yes, you can -- if you invoke find-grep asynchronously and move the
processing of the hits to the filter function.  But that doesn't need
to involve threads, and is being done in many packages/features out
there, so I'm not sure what did you ask me to do with this.  IOW, it
should be "trivial", at least in principle, to make this command work
in the background, just like, say, "M-x grep".

> Here's how I imagine it:

I'm not sure I understand the need for this complexity, given that
async subprocesses are available.  I'm probably missing something
because I know too little about the internals of the involved code.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 1 May 2017 02:42:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 30 22:42:32 2017
Received: from localhost ([127.0.0.1]:48347 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d51IG-0002Ts-E8
	for submit <at> debbugs.gnu.org; Sun, 30 Apr 2017 22:42:32 -0400
Received: from mail-wr0-f171.google.com ([209.85.128.171]:34448)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raaahh@HIDDEN>)
 id 1d51ID-0002Ta-19; Sun, 30 Apr 2017 22:42:29 -0400
Received: by mail-wr0-f171.google.com with SMTP id l9so58795651wre.1;
 Sun, 30 Apr 2017 19:42:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:subject:to:cc:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=ZqYeSoWgePUpClRvkL+wpfRpEhcNlYhG1g2/k9wMXvY=;
 b=rLVOQda88NaYXL50Z0NUAKw5YUb55pDziuEUhpRh7FbBthGx3bhRi5yqpTliGV+Llu
 cKhqbSViTn/FftArQPNb/byxC+0InGEgq/OKycnX/BeEUlA9C8zXqcrlFJDG+3587ejr
 8U3LBmjXwbimwZdw4Dr+PClRDPPuQ6bnRs80zjie3mj0AFRN/0eqe49aAWZ6If5RUmEq
 b2x2GcHI/zwmgxLQyp3mUPm1iGVrfOY0N6F+tDv/TxgHJ3MNUyc14oIoVDDur1l3aBXQ
 HK+ydsuy3IobRd4ga/WepBU6JeE2C1bIIJiLbgwTzNTlSjJUbQscRVAc4TI+1myHXAlH
 tZTw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:subject:to:cc:references:from:message-id
 :date:user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=ZqYeSoWgePUpClRvkL+wpfRpEhcNlYhG1g2/k9wMXvY=;
 b=UNNIIiHPWzUICU/gvj1rX96XaCCKuoDONOLjjaZJm7Qj+jjvsTS/DoCivF74vOW11I
 iDHViqcrkBRmF/1ESPgMPMvkH/sisaVwdur1onHTu0WAyy89aukoMUdaVGrGpFO6wqIt
 sr6uKnY4lCtp1gfWsfUqBQ7egF3p8/M+u+LHSYE1iwbSMg9quuUr0wWgIaGesDKG6VfJ
 qjvbTBZovz0F47pjkMbOb/qFmLlwmAKCFMuI6lHxYAoZutyCbKoTy73y/JzLjhtUFWoX
 F2Itw/OqU6rvV3idXrKaxDDdyQ+HTuwfXx9eh4frfaFcXPV1H2ovsmLzSCOcvrf0dO8b
 z01w==
X-Gm-Message-State: AN3rC/5ai8HbB712VzADoGV/cMRyjrfU5auonak4GQsHxB7wZg4yrqJW
 3TxvjvMz+Z0cyA==
X-Received: by 10.223.165.71 with SMTP id j7mr16434604wrb.35.1493606543286;
 Sun, 30 Apr 2017 19:42:23 -0700 (PDT)
Received: from [192.168.1.3] ([185.105.173.156])
 by smtp.googlemail.com with ESMTPSA id o77sm14099490wrc.38.2017.04.30.19.42.22
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Sun, 30 Apr 2017 19:42:22 -0700 (PDT)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
To: Eli Zaretskii <eliz@HIDDEN>
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> <83ziexka0s.fsf@HIDDEN>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <77b3a404-adac-fd1c-bd99-ad10e2450338@HIDDEN>
Date: Mon, 1 May 2017 05:42:21 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101
 Thunderbird/53.0
MIME-Version: 1.0
In-Reply-To: <83ziexka0s.fsf@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.6 (--)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, control <at> debbugs.gnu.org,
 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

On 30.04.2017 21:47, Eli Zaretskii wrote:

> I'll try to look at this.  According to my profiling, the lion's share
> of time is taken by xref--collect-matches, so that's the place to try
> concurrency.

I think that's too late. By the time xref--collect-matches is called 
(and it's called for each hit), we've already spent time synchronously 
waiting for the find-grep invocation to finish.

When there are a lot of matches, xref--collect-matches can take some 
significant time, with opening the buffers and everything. That can be 
optimized, however, as a separate issue, and I don't think there's 
anything to parallelize there, since it all happens in Elisp.

What we _can_ manage to run in parallel, in the find-grep process in the 
background, and the post-processing of the results in Elisp. Depending 
on how matches there are in total, compared to the project size (which 
affects how long find-grep will run), the second part will still affect 
the responsiveness of the UI to a smaller or larger degree, but 
ultimately there's no way around this AFAIK, as long as Elisp threads do 
not provide parallelism.

>> - Being able to hook up in an asynchronous fashion to that sequence in a
>> (second?) background thread, to render the search results in the xref
>> buffer as soon as they appear.
> 
> For the other thread to be background, it will need to yield from time
> to time, otherwise the user experience will be identical to what we
> have today, because an un-yielding thread will hold the execution unit
> until it does its job completely, and no other thread gets to run
> until it does.

Here's how I imagine it:

- Main thread creates a thread P which invoked the find-grep, somehow 
creates a "generator" object, yields to the main thread.

- The main thread creates the "other thread" which creates a buffer for 
displaying the xref items and marks it as still loading (e.g. with a 
spinner in the mode-line). And then repeatedly calls the generator for 
the next element. There are three choices:

1. An element is returned. Render it into the buffer.
2. No next element available. That should automatically yield from the 
current thread until it becomes available. That kind of logic should 
reside somewhere inside the generator, along with storing the reference 
to the current thread, to resume it later.
3. No more items, the process in P is finished. Mark the xref buffer as 
completed.

The things I'm not clear on are:

- How to best "convert" the process buffer into a generator object.

- Which generator type to use. Not sure if any of the ones we already 
have (generator.el, or iterator.el and stream.el in ELPA) will help.

- If the thread P is needed at all, or we could just one the main one 
instead of it.

- Whether we should forget all that concurrency nonsense (at least for 
this problem), and instead of a generator go with callbacks, similar to 
the API of the dir-status-files VC command. This way, each callback will 
render a new line, and the last one will mark the buffer as completed.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 30 Apr 2017 18:48:26 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 30 14:48:26 2017
Received: from localhost ([127.0.0.1]:48038 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d4ttS-0006Yi-BU
	for submit <at> debbugs.gnu.org; Sun, 30 Apr 2017 14:48:26 -0400
Received: from eggs.gnu.org ([208.118.235.92]:59924)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1d4ttO-0006YV-Nm
 for 26710 <at> debbugs.gnu.org; Sun, 30 Apr 2017 14:48:24 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1d4ttE-0003lV-VG
 for 26710 <at> debbugs.gnu.org; Sun, 30 Apr 2017 14:48:17 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:49055)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1d4tt4-0003jH-IP; Sun, 30 Apr 2017 14:48:02 -0400
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2516
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1d4tt3-00029g-ER; Sun, 30 Apr 2017 14:48:02 -0400
Date: Sun, 30 Apr 2017 21:47:31 +0300
Message-Id: <83ziexka0s.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dgutov@HIDDEN>
In-reply-to: <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN> (message from
 Dmitry Gutov on Sun, 30 Apr 2017 13:35:53 +0300)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
 <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, control <at> debbugs.gnu.org,
 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> Cc: 26710 <at> debbugs.gnu.org, control <at> debbugs.gnu.org
> From: Dmitry Gutov <dgutov@HIDDEN>
> Date: Sun, 30 Apr 2017 13:35:53 +0300
> 
> > When the search is in progress, I'm unable to use emacs.
> > Emacs doesn't respond to any action until the search is over.
> 
> OK, that's a reasonable complaint, especially when the search term is a 
> rarely occurring one (so most of the time is spent in the external process).
> 
> It is non-trivial to fix, however, while keeping the xref backend API 
> and associated code sane.
> 
> I'm hoping that the newly-added concurrency support can help us in that. 
> I have not looked into actually using it, though.
> 
> If someone (Eli?) would like to try their hand at it, or to even outline 
> the basic direction such solution would use, that would be welcome.

I'll try to look at this.  According to my profiling, the lion's share
of time is taken by xref--collect-matches, so that's the place to try
concurrency.

> - Being able to hook up in an asynchronous fashion to that sequence in a 
> (second?) background thread, to render the search results in the xref 
> buffer as soon as they appear.

For the other thread to be background, it will need to yield from time
to time, otherwise the user experience will be identical to what we
have today, because an un-yielding thread will hold the execution unit
until it does its job completely, and no other thread gets to run
until it does.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.
Changed bug title to 'project-find-regexp blocks the UI' from 'Fwd: 25.2; project-find-regexp makes emacs use 100% cpu' Request was from Dmitry Gutov <dgutov@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 30 Apr 2017 10:36:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 30 06:36:04 2017
Received: from localhost ([127.0.0.1]:46949 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d4mCy-0008Sv-D0
	for submit <at> debbugs.gnu.org; Sun, 30 Apr 2017 06:36:04 -0400
Received: from mail-wm0-f49.google.com ([74.125.82.49]:35702)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raaahh@HIDDEN>)
 id 1d4mCw-0008SM-0B; Sun, 30 Apr 2017 06:36:02 -0400
Received: by mail-wm0-f49.google.com with SMTP id w64so71913800wma.0;
 Sun, 30 Apr 2017 03:36:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:subject:to:cc:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=zvOdF25UmChNkWstCyIVectyghwR5uVoFlTExjLCmyU=;
 b=BMDkWFnD1nCCfq5eiwjy0TFKRSKKSzR4PII04hYfeGF/PI+XLWSrXr7chCUVZPlyBY
 eNPeI63UWCXGxktgkvrCXg3R6h09AiXmpx4JzKNNxXr5zK2GbcVdurebS4Dyx02uQ1KS
 csx4SznBAMIEiZDiMnLiDUObzWariadeQ+OSov3I7EbMDHXmwC5M2SRIWTIqAQI8asGQ
 CVF1YY2WjHdU/GFaTC/bfvAo+Pd5sypB/vnFLE68EEXaueXiTbY1w3G/RN2LWLr3Y61S
 TLlYYjEicz8hnMDB526mBcCBFnV+kLojcqfwusx3FyhyEvHKabPGbfLfVNkTvHX+7a7r
 8Ntw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:subject:to:cc:references:from:message-id
 :date:user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=zvOdF25UmChNkWstCyIVectyghwR5uVoFlTExjLCmyU=;
 b=RrhXoGcQ75aSeqE9DYHGPxIcuEtS3BeDsSYq++WHKqMym7xc/C6q0B5m03s+seCAMN
 TMUeDuteLWfZy+CsV5MdyFIfxu2Z7x81fcjp4SEpXgqfUQVbQav6sT2xBiq7m4qUkh1I
 ZKZJDR+SWIrWG05IWHuJaL8LoEoQ72oEmlAp/9zTScgWlkD0g34enINIfzGZTM2TWb9Z
 IAvdf5UZTBGupj5MfSKeyoP2//QMYs2a/GNNEizh5bJb7cCxBVky73wDwN5b9+YadzN/
 ZmBocJryP37gjXQIJ+yXDa+ySfiRulKwKzkKaTBKs3wPCbVPlKTUyvQ0sW+P4rDSwGIG
 uX/A==
X-Gm-Message-State: AN3rC/7MpRnzwVCQtiE28EGJtQT1H1rKo7rXbnY7cVfC4tfrcNwjCRyV
 ZG7kdlLAR1RaqA==
X-Received: by 10.28.12.78 with SMTP id 75mr3715721wmm.13.1493548556295;
 Sun, 30 Apr 2017 03:35:56 -0700 (PDT)
Received: from [192.168.1.3] ([185.105.173.156])
 by smtp.googlemail.com with ESMTPSA id l81sm6534605wmi.22.2017.04.30.03.35.54
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Sun, 30 Apr 2017 03:35:55 -0700 (PDT)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
To: Hariharan Rangasamy <hariharanrangasamy@HIDDEN>,
 Eli Zaretskii <eliz@HIDDEN>
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
 <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <3d76a3ac-32ad-412d-349d-5904fc964a2b@HIDDEN>
Date: Sun, 30 Apr 2017 13:35:53 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101
 Thunderbird/53.0
MIME-Version: 1.0
In-Reply-To: <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.2 (/)
X-Debbugs-Envelope-To: 26710
Cc: control <at> debbugs.gnu.org, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.2 (/)

retitle 26710 project-find-regexp blocks the UI
stop

On 30.04.2017 7:13, Hariharan Rangasamy wrote:
> When the search is in progress, I'm unable to use emacs.
> Emacs doesn't respond to any action until the search is over.

OK, that's a reasonable complaint, especially when the search term is a 
rarely occurring one (so most of the time is spent in the external process).

It is non-trivial to fix, however, while keeping the xref backend API 
and associated code sane.

I'm hoping that the newly-added concurrency support can help us in that. 
I have not looked into actually using it, though.

If someone (Eli?) would like to try their hand at it, or to even outline 
the basic direction such solution would use, that would be welcome.

It seems we need two parts:

- Generating some sort of lazy sequence from the external process' 
output, so that the return value of an xref backend call is still a 
sequence.

- Being able to hook up in an asynchronous fashion to that sequence in a 
(second?) background thread, to render the search results in the xref 
buffer as soon as they appear.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 30 Apr 2017 04:13:30 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Apr 30 00:13:30 2017
Received: from localhost ([127.0.0.1]:46828 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d4gEk-0007mX-FJ
	for submit <at> debbugs.gnu.org; Sun, 30 Apr 2017 00:13:30 -0400
Received: from mail-ua0-f182.google.com ([209.85.217.182]:33411)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hariharanrangasamy@HIDDEN>) id 1d4gEi-0007mJ-PH
 for 26710 <at> debbugs.gnu.org; Sun, 30 Apr 2017 00:13:29 -0400
Received: by mail-ua0-f182.google.com with SMTP id j59so55152923uad.0
 for <26710 <at> debbugs.gnu.org>; Sat, 29 Apr 2017 21:13:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:in-reply-to:references:from:date:message-id:subject:to
 :cc; bh=kkYNURiMYCpz8BU0PmUju5iIXcMCR9Wwjth16+KzRXo=;
 b=J/tBF8v//9P6s6LpC0ebkX+jzmsJTJl1V5aiYB5QK3Rm9qCND+TDf5IA2U4U6hlYYt
 gHFfPMzYvhzHZko/Pp4iIYV7+VpWAqQ5cDaulu3Kr3FaX16XJA3gwx03qlzqVxd0kl+c
 MlEiYCsMZtIWh+koG6FBLF6Gu/hLX2mEvy5n7Le16UWn/xMAcS5SHDJ7eTidHxcDc5w5
 KFXs8tP/AjSfUTeriC6hXvuxGkxY7TxR9O4USzukna6nW0v9hk+sZNEZ0Hbw0SwpTFC4
 Pn1yalwDvEXqHMkENg5xSoLOGE0ykbk0btvu/N7l4LA7uPhg9zFppru7PSbz73uPEbhj
 zI3Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:in-reply-to:references:from:date
 :message-id:subject:to:cc;
 bh=kkYNURiMYCpz8BU0PmUju5iIXcMCR9Wwjth16+KzRXo=;
 b=SGnObaluel23mfA+IWHCPIRlSWbbgPNUcvSyfJqWnSHPBpZmb3dpCQYQ7braMEy8AG
 G0E/F9L5rfh0dKDb1d3mLixoER+DorkE1LwTFWFaUxITFy+gsuLCKiuUC3t6zYOQmmqo
 q7NPNjvm+SySUl3xtMF0Djj8CQeZKdBc4BhLDc5+y0LoNGHF4Z7jH2gz0bb4BzSzDFs1
 3q0lBGfXJbQCxY/CcJCsIIWe2dWMDY/rmHzoGCCATvUBnSKmGTpyDOgmG4mD1teIq37C
 uweizY1e7kdVPNyLIyrm9hK6tf6YwYl7uPs7jznvL+O1h0wpUmswEgRkCQMAp+iGwbtb
 TODQ==
X-Gm-Message-State: AN3rC/7TsqOdUKwOcRlcQyHNfjkScnfSvbMuKs9DhTvEnl5xqWny+lFj
 HZOqggoH7WzlYw7zdgqXJeMhP3YlUA==
X-Received: by 10.159.36.242 with SMTP id 105mr10639870uar.56.1493525603263;
 Sat, 29 Apr 2017 21:13:23 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.83.2 with HTTP; Sat, 29 Apr 2017 21:13:22 -0700 (PDT)
In-Reply-To: <83vapnktcn.fsf@HIDDEN>
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> <83vapnktcn.fsf@HIDDEN>
From: Hariharan Rangasamy <hariharanrangasamy@HIDDEN>
Date: Sun, 30 Apr 2017 09:43:22 +0530
Message-ID: <CAGkYVKmbi4gSqZkssJgFFkJffcEaKhKSULMkYu5=SFWASrDByQ@HIDDEN>
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
To: Eli Zaretskii <eliz@HIDDEN>
Content-Type: text/plain; charset=UTF-8
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 26710
Cc: 26710 <at> debbugs.gnu.org, Dmitry Gutov <dgutov@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

When the search is in progress, I'm unable to use emacs.
Emacs doesn't respond to any action until the search is over.

On Sat, Apr 29, 2017 at 11:07 PM, Eli Zaretskii <eliz@HIDDEN> wrote:
>> From: Dmitry Gutov <dgutov@HIDDEN>
>> Date: Sat, 29 Apr 2017 20:00:35 +0300
>>
>> On 29.04.2017 11:55, Hariharan Rangasamy wrote:
>> > using project-find-regexp to find text in a folder makes emacs use 100%
>> > CPU
>>
>> Why is that a problem?
>
> Indeed: CPU-intensive processing will always make one execution unit
> 100% busy, as long as the processing goes on.  This is normal.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 29 Apr 2017 17:38:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Apr 29 13:38:35 2017
Received: from localhost ([127.0.0.1]:46576 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d4WKJ-00082U-C9
	for submit <at> debbugs.gnu.org; Sat, 29 Apr 2017 13:38:35 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55892)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1d4WKH-00082D-6P
 for 26710 <at> debbugs.gnu.org; Sat, 29 Apr 2017 13:38:33 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1d4WK8-0007GM-Qj
 for 26710 <at> debbugs.gnu.org; Sat, 29 Apr 2017 13:38:27 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_40,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:33731)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1d4WK8-0007GI-Na; Sat, 29 Apr 2017 13:38:24 -0400
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1423
 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1d4WK6-0006Cd-Qr; Sat, 29 Apr 2017 13:38:24 -0400
Date: Sat, 29 Apr 2017 20:37:44 +0300
Message-Id: <83vapnktcn.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dgutov@HIDDEN>
In-reply-to: <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN> (message from
 Dmitry Gutov on Sat, 29 Apr 2017 20:00:35 +0300)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
 <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 26710
Cc: hariharanrangasamy@HIDDEN, 26710 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Dmitry Gutov <dgutov@HIDDEN>
> Date: Sat, 29 Apr 2017 20:00:35 +0300
> 
> On 29.04.2017 11:55, Hariharan Rangasamy wrote:
> > using project-find-regexp to find text in a folder makes emacs use 100%
> > CPU
> 
> Why is that a problem?

Indeed: CPU-intensive processing will always make one execution unit
100% busy, as long as the processing goes on.  This is normal.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at 26710 <at> debbugs.gnu.org:


Received: (at 26710) by debbugs.gnu.org; 29 Apr 2017 17:00:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Apr 29 13:00:45 2017
Received: from localhost ([127.0.0.1]:46551 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d4Vjh-0007Bi-0i
	for submit <at> debbugs.gnu.org; Sat, 29 Apr 2017 13:00:45 -0400
Received: from mail-wr0-f177.google.com ([209.85.128.177]:34722)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raaahh@HIDDEN>) id 1d4Vjf-0007BV-Ha
 for 26710 <at> debbugs.gnu.org; Sat, 29 Apr 2017 13:00:43 -0400
Received: by mail-wr0-f177.google.com with SMTP id l9so47226745wre.1
 for <26710 <at> debbugs.gnu.org>; Sat, 29 Apr 2017 10:00:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:subject:to:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=17jxy4yOqQmRA7UQdRXJTrca6fWVaaqQdVzWdePvTfE=;
 b=VsH9Lzau1BhRhD7KGTEu98p27+8Ds51/lJ5J26Wad/8EkrZh6lQu0Jq4LRrrZJSdwU
 M0NabUrJdsqXhHjRW67dx0qLVCFcjszvM5itII0+nSRXCKEjpRy2/Y1Qni6KELQ9MAvq
 ONkWVUQ5fBJFQ3Je0OznXOX39KRiJ9A2PjKEdDgxpIQjXIvmImyDZEteuVG/JNNhZumD
 GDgAHmjU2RSThxx22GRQXjZIKirLss58Kujo5+SNo8sIM+F6jd3o1i48dsxiK0CtQTHm
 EWBvaYFvGQMKTEkhaqip4k0EB0BZQ3uxypGU+q0HiLQMsz/XyIJlJ6fubsQ4MWKlZ4YW
 37bw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:subject:to:references:from:message-id
 :date:user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=17jxy4yOqQmRA7UQdRXJTrca6fWVaaqQdVzWdePvTfE=;
 b=p6WdxfY/2wdgLxtxNBGf49zuK9oEy+LjYyvyLocFAzuGwYKCodBwug31HHV04pKklx
 oQOjghoHtgBorZrwJg9ZYhaPa3C/9w0RXO/GTPjD4v4O3wN6/u1X9fsKufXBPOdrQL8C
 371MV3u5EeiVJDVD7xbaH4CY/fe+FXsoe2j2tHbJVXJUN8SsR6aX1OlwZqrgS98YBop7
 fQe+gi9axEodi7Bz7+urjLtiDaF759LLABXPs/Sg39hWwKDjIL5fDV+uzgnajf/dRMUX
 qBCleDDeCB+bju0xB0C6gU87BC7NFSE584Qu89Q21ryrmG02sU/7vJWRDpp20zKabCmv
 F9Sw==
X-Gm-Message-State: AN3rC/7BgyFNSCeo5eIx3hK4m9vM2UgzfMRPayZ1hLb85eZBhHq5QTCd
 yzAYGUCBw3PCYg==
X-Received: by 10.223.154.100 with SMTP id z91mr10929492wrb.76.1493485237780; 
 Sat, 29 Apr 2017 10:00:37 -0700 (PDT)
Received: from [192.168.1.3] ([185.105.173.156])
 by smtp.googlemail.com with ESMTPSA id 13sm12386212wml.25.2017.04.29.10.00.36
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Sat, 29 Apr 2017 10:00:36 -0700 (PDT)
Subject: Re: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
To: Hariharan Rangasamy <hariharanrangasamy@HIDDEN>, 26710 <at> debbugs.gnu.org
References: <87a86zu3gf.fsf@HIDDEN>
 <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <cbe6ee00-51e6-9458-a00f-1cf6b0ac30b8@HIDDEN>
Date: Sat, 29 Apr 2017 20:00:35 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101
 Thunderbird/53.0
MIME-Version: 1.0
In-Reply-To: <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.6 (--)
X-Debbugs-Envelope-To: 26710
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

Hi,

On 29.04.2017 11:55, Hariharan Rangasamy wrote:
> using project-find-regexp to find text in a folder makes emacs use 100%
> CPU

Why is that a problem? Should it use 10% CPU and take 10 times as long?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 29 Apr 2017 16:55:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Apr 29 12:55:00 2017
Received: from localhost ([127.0.0.1]:46535 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1d4Ve7-00071G-B0
	for submit <at> debbugs.gnu.org; Sat, 29 Apr 2017 12:54:59 -0400
Received: from eggs.gnu.org ([208.118.235.92]:44925)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <hariharanrangasamy@HIDDEN>) id 1d4O9q-0000QZ-CC
 for submit <at> debbugs.gnu.org; Sat, 29 Apr 2017 04:55:14 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <hariharanrangasamy@HIDDEN>) id 1d4O9j-0005dE-L4
 for submit <at> debbugs.gnu.org; Sat, 29 Apr 2017 04:55:09 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM,
 T_DKIM_INVALID autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:37673)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <hariharanrangasamy@HIDDEN>)
 id 1d4O9j-0005d5-HS
 for submit <at> debbugs.gnu.org; Sat, 29 Apr 2017 04:55:07 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35894)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <hariharanrangasamy@HIDDEN>) id 1d4O9i-0001dQ-13
 for bug-gnu-emacs@HIDDEN; Sat, 29 Apr 2017 04:55:07 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <hariharanrangasamy@HIDDEN>) id 1d4O9g-0005br-SJ
 for bug-gnu-emacs@HIDDEN; Sat, 29 Apr 2017 04:55:06 -0400
Received: from mail-ua0-x241.google.com ([2607:f8b0:400c:c08::241]:34342)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <hariharanrangasamy@HIDDEN>)
 id 1d4O9g-0005bf-Lk
 for bug-gnu-emacs@HIDDEN; Sat, 29 Apr 2017 04:55:04 -0400
Received: by mail-ua0-x241.google.com with SMTP id 110so6530492uas.1
 for <bug-gnu-emacs@HIDDEN>; Sat, 29 Apr 2017 01:55:04 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:in-reply-to:references:from:date:message-id:subject:to
 :content-transfer-encoding;
 bh=mHBcijus1Et07yPFWGXCJiR53gFTgd86suIBDovWa1M=;
 b=APOGOg/tLu+m8IBwNIrhSl9SC8wx5gULQE3OqD5+8gzqfbi/1/vSk6AM0EiIfFhGed
 WMQraRzj72jNP6kY6AHikJQFZ5DF5TfI1ruigIK+GZK/BJf/ryF0c1XzUWE3s/L4bvd/
 N9W2QLkvzYgNGfGYY7F1ReWDol6Z2poRZ2Z0FOs2MMhwOeIS1W68Lq53YZJFXanaqWmF
 y0IvFe845Fo34f6ddofhZcYMbG3GL4KNbvae/QPDSlu8QJIdRrGs7JJCPSfOW4C3Efyt
 tQZmMH/Bl5fFEbyYfabYaMIwNG2+PI7UYltrs4I2w9s0xwQRM/mR/FEUtBdjngCR08ry
 rTKw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:in-reply-to:references:from:date
 :message-id:subject:to:content-transfer-encoding;
 bh=mHBcijus1Et07yPFWGXCJiR53gFTgd86suIBDovWa1M=;
 b=LbzLYSf3+FzrWgmz3ToCF/NTa70DVUjhuawUMt05mfNlonz/XsCWFlQHEGG9JT5dhm
 s0icVWYJw7vQZJ8ktDSMzdWCNzIg8LHt2Q5Bli8nskSFizEe1BS0jgoy9g0270s8g9b8
 2mQMxW14FuB4WMtvkLmzt2s4kDCummPX3+Pl0SGYY7CYmTo95f5rne7L/gMHRDsVqvzW
 oVwVt5duRqnt/uEBBTsWt5bNi3FnjvAjNNT9Le2YX6l+DRsXAE627uyBFZIdlBY5AgkJ
 iWVRGX46akLAjo3gmiVrhVJATYZdtgNg3xkGMhhEg1uy8RQIeS2GSF/SEMFJl9we21N1
 R1gg==
X-Gm-Message-State: AN3rC/6S1CG+Fv0PrCr1CLE9ycrVULAiAtWOjD1my8REXBfy1Sl/1xWK
 SAY3DYBEfyQ9iRbyEbGLK0htpfkVJwti
X-Received: by 10.159.35.53 with SMTP id 50mr7949589uae.91.1493456103502; Sat,
 29 Apr 2017 01:55:03 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.176.83.2 with HTTP; Sat, 29 Apr 2017 01:55:03 -0700 (PDT)
In-Reply-To: <87a86zu3gf.fsf@HIDDEN>
References: <87a86zu3gf.fsf@HIDDEN>
From: Hariharan Rangasamy <hariharanrangasamy@HIDDEN>
Date: Sat, 29 Apr 2017 14:25:03 +0530
Message-ID: <CAGkYVKkbULDUOm90j1fUUpkmqcLHYhjzrL-1aZ7NxYyeijpm3g@HIDDEN>
Subject: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
To: bug-gnu-emacs@HIDDEN
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.0 (----)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Sat, 29 Apr 2017 12:54:58 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.0 (----)

using project-find-regexp to find text in a folder makes emacs use 100%
CPU

Steps to reproduce:
1. emacs -nw -Q
2. M-x project-find-regexp
3. choose search item as emacs
4. Choose the project as downloaded emacs-25.2/ folder
5. Give top command and check the cpu usage of emacs-25.2 process
6. emacs utilises 100% CPU till the search is over.






In GNU Emacs 25.2.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.18.9)
 of 2017-04-29 built on hari-laptop
System Description:     Ubuntu 16.04.2 LTS

Configured features:
XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS NOTIFY GNUTLS FREETYPE XFT
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LANG: en_IN
  value of $XMODIFIERS: @im=3Dibus
  locale-coding-system: iso-latin-1-unix

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
  diff-auto-refine-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
user-error: No matches for: helloworld
Using =E2=80=99~/Downloads/emacs-25.2/=E2=80=99 as a transient project root
Enriched: decoding document...
Indenting...
Using =E2=80=99~/Downloads/emacs-25.2/=E2=80=99 as a transient project root
Enriched: decoding document...
Indenting...
Making completion list...
delete-backward-char: Text is read-only
user-error: Beginning of history; no preceding item [2 times]
Quit [2 times]

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rfc822 mml mml-sec epg
epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils
autoconf autoconf-mode m4-mode gud nroff-mode texinfo sgml-mode python
tramp-sh tramp tramp-compat auth-source mm-util mail-prsvr
password-cache tramp-loaddefs trampver ucs-normalize advice json map seq
semantic/bovine/grammar semantic/wisent/grammar semantic/bovine
semantic/grammar help-fns semantic/idle semantic/grammar-wy bat-mode
cc-awk enriched cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs nxml-uchnm rng-xsd xsd-regexp
rng-cmpct rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse
rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode nxml-outln
nxml-rap nxml-util nxml-glyph nxml-enc xmltok ruby-mode perl-mode
verilog-mode diff conf-mode make-mode tex-mode shell latexenc
org-element org-rmail org-mhe org-irc org-info org-gnus gnus-util
org-docview org-bibtex bibtex org-bbdb org-w3m org org-macro
org-footnote org-pcomplete pcomplete org-list org-faces org-entities
noutline outline org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob
ob-table ob-exp org-src ob-keys ob-comint ob-core ob-eval org-compat
org-macs org-loaddefs format-spec cal-menu calendar cal-loaddefs
srecode/srt-mode semantic/analyze semantic/sort semantic/scope
semantic/analyze/fcn semantic/db semantic/format ezimage
srecode/template srecode/srt-wy semantic/wisent semantic/wisent/wisent
semantic/ctxt srecode/ctxt semantic/tag-ls semantic/find srecode/compile
srecode/dictionary srecode/table srecode eieio-base semantic/util-modes
semantic/util semantic semantic/tag semantic/lex cedet doc-view subr-x
jka-compr image-mode ps-mode add-log sh-script smie executable
find-dired dired semantic/fw mode-local find-func grep compile comint
ansi-color vc-mtn vc-hg vc-git diff-mode easy-mmode vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs vc vc-dispatcher thingatpt etags xref
cl-seq ring eieio byte-opt bytecomp byte-compile cl-extra help-mode
easymenu cconv eieio-core cl-macs gv cl-loaddefs pcase cl-lib project
term/xterm xterm time-date disp-table mule-util tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 4729592 517895)
 (symbols 48 40455 0)
 (miscs 40 112 310)
 (strings 32 292026 313931)
 (string-bytes 1 15565717)
 (vectors 16 275283)
 (vector-slots 8 2028489 240208)
 (floats 8 530 633)
 (intervals 56 977354 6401)
 (buffers 976 23)
 (heap 1024 235047 29774))




Acknowledgement sent to Hariharan Rangasamy <hariharanrangasamy@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#26710; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.