GNU bug report logs - #895
slow processing of process output

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Dan Nicolaescu <dann@HIDDEN>; Done: Noam Postavsky <npostavs@HIDDEN>; Maintainer for emacs is bug-gnu-emacs@HIDDEN.
bug archived. Request was from Debbugs Internal Request <help-debbugs@HIDDEN> to internal_control <at> debbugs.gnu.org. Full text available.

Message received at 895-done <at> debbugs.gnu.org:


Received: (at 895-done) by debbugs.gnu.org; 5 Jun 2016 01:51:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 04 21:51:34 2016
Received: from localhost ([127.0.0.1]:54791 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1b9NDy-0006bd-1B
	for submit <at> debbugs.gnu.org; Sat, 04 Jun 2016 21:51:34 -0400
Received: from mail-oi0-f46.google.com ([209.85.218.46]:33137)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1b9NDw-0006bQ-Fd
 for 895-done <at> debbugs.gnu.org; Sat, 04 Jun 2016 21:51:33 -0400
Received: by mail-oi0-f46.google.com with SMTP id k23so180382257oih.0
 for <895-done <at> debbugs.gnu.org>; Sat, 04 Jun 2016 18:51:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:sender:from:date:message-id:subject:to;
 bh=Aqnd3B2fkBoS0cf0vLKa2wyxXQX4qSxKzggQjeYGAV4=;
 b=udP/JnBt2E5TCFTSAKQWU2c5R+cyr6hPypjmc/pve024po1mCCplO92jSmHSrreQbN
 bpZBUm1hb0me4BitJDhJLIHvYj5c2bPMMP3x7n2/iTOhlXIbZlT/O962mrFYD/+Xro9q
 EVSrCnFZi9BmLafgqACXxO92yZCFnvJZVFn7MeDTnvWybNbHSJu/Ujk0G53WO/81RIjk
 Loi+bPGcC1XBvU/UvxyaKlcwnHl2V1mOW0a8YCd89E+TS6kQ1eeJYlQZCzMMarWM1aus
 DzFIhke+RmindUcAL7TbpoFZnj1Nq/0jgNlq7DSY97A8v20KfgAF1PUDhhDcg9UPBbno
 Q8mg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:mime-version:sender:from:date:message-id:subject
 :to; bh=Aqnd3B2fkBoS0cf0vLKa2wyxXQX4qSxKzggQjeYGAV4=;
 b=dbdCCIKNP474vgia7CqZrflPF28ssTF+DF3L8APbvgkO5KG1fXELsVfgPO/Rno94eV
 rkD/XGsA8VMqzowxqvXzDFiRJp8kWMj6WpioJUeThuwx2u4aBqO3GXdOqd0jkeGI17Up
 erEtvEjLoGzufJjCd06Ml+HL0WrvpCjsLXL3sDc3AlrXgBPWLDp0OTY82qKYeiZCsVqi
 +rx6yUxrMXxwXapFj1KxojoGN9YjmkjPu5yNNU4twYZxY9KTBHqm0UHmnR35uP5VEe3q
 BdZghQ2fnIo0ruJ8M/aeJUBvqhb8zaj5ZmRHhUrUB7fKXJ1EOqRAKkJ7hB78rWlrmtqQ
 u74g==
X-Gm-Message-State: ALyK8tJwXMrpgt7AFBdwQJfuy1mhjEan2SxAkchd7ptRa8dvv0U6r2JwvF/mESkO5UWARZCorEquMe0PsSTyYw==
X-Received: by 10.202.245.196 with SMTP id t187mr4577851oih.178.1465091486661; 
 Sat, 04 Jun 2016 18:51:26 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.157.5.168 with HTTP; Sat, 4 Jun 2016 18:51:26 -0700 (PDT)
From: Noam Postavsky <npostavs@HIDDEN>
Date: Sat, 4 Jun 2016 21:51:26 -0400
X-Google-Sender-Auth: I4DOHtYfzWc_Fsl8YPsTp7iiGjw
Message-ID: <CAM-tV-_=RH9+RKrJ+PWcg85WxCy7ObxvX5uDiaV96R3zhY_aBQ@HIDDEN>
Subject: Re: bug#895: slow processing of process output
To: 895-done <at> debbugs.gnu.org
Content-Type: text/plain; charset=UTF-8
X-Spam-Score: -0.5 (/)
X-Debbugs-Envelope-To: 895-done
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.5 (/)

Dmitry Gutov <dgutov <at> yandex.ru> wrote:
>
> Looks fixed:
>
> With warm cache, the original commands takes 4 seconds in Emacs, also 4
> seconds in my shell (fish inside terminator), and produces the output of
> 17000 lines.
>
> It's still <1 second when redirected to a file, but that's to be
> expected, I think.

Closing.




Notification sent to Dan Nicolaescu <dann@HIDDEN>:
bug acknowledged by developer. Full text available.
Reply sent to Noam Postavsky <npostavs@HIDDEN>:
You have taken responsibility. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 8 Jan 2016 10:55:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 08 05:55:07 2016
Received: from localhost ([127.0.0.1]:42316 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aHUhG-0000WY-QG
	for submit <at> debbugs.gnu.org; Fri, 08 Jan 2016 05:55:06 -0500
Received: from eggs.gnu.org ([208.118.235.92]:34032)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <eliz@HIDDEN>) id 1aHUhF-0000W1-3o
 for 895 <at> debbugs.gnu.org; Fri, 08 Jan 2016 05:55:05 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1aHUh5-0005NS-J8
 for 895 <at> debbugs.gnu.org; Fri, 08 Jan 2016 05:54:59 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41890)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1aHUh5-0005NI-FL; Fri, 08 Jan 2016 05:54:55 -0500
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1267
 helo=HOME-C4E4A596F7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1aHUh4-0006YU-P3; Fri, 08 Jan 2016 05:54:55 -0500
Date: Fri, 08 Jan 2016 12:54:52 +0200
Message-Id: <83mvsgpchv.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dgutov@HIDDEN>
In-reply-to: <568F8A5A.8030101@HIDDEN> (message from Dmitry Gutov on Fri, 8
 Jan 2016 13:07:22 +0300)
Subject: Re: bug#895: slow processing of process output
References: <200809051543.m85FhWHV008307@HIDDEN>
 <CAPM58oi9QCxrdRJaSA5LV-G++fXE8rMoJuABT60b8_ihuPuR5Q@HIDDEN>
 <83ziwgpge7.fsf@HIDDEN> <568F8A5A.8030101@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 895
Cc: rcopley@HIDDEN, 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> Cc: 895 <at> debbugs.gnu.org
> From: Dmitry Gutov <dgutov@HIDDEN>
> Date: Fri, 8 Jan 2016 13:07:22 +0300
> 
> On 01/08/2016 12:30 PM, Eli Zaretskii wrote:
> 
> > Would someone please try reproducing this on GNU or Unix system?
> 
> Looks fixed:
> 
> With warm cache, the original commands takes 4 seconds in Emacs, also 4 
> seconds in my shell (fish inside terminator), and produces the output of 
> 17000 lines.
> 
> It's still <1 second when redirected to a file, but that's to be 
> expected, I think.

Thanks.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 8 Jan 2016 10:07:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 08 05:07:31 2016
Received: from localhost ([127.0.0.1]:42278 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aHTxD-0007oB-5r
	for submit <at> debbugs.gnu.org; Fri, 08 Jan 2016 05:07:31 -0500
Received: from mail-lb0-f176.google.com ([209.85.217.176]:34462)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <raaahh@HIDDEN>) id 1aHTxB-0007ny-Nx
 for 895 <at> debbugs.gnu.org; Fri, 08 Jan 2016 05:07:30 -0500
Received: by mail-lb0-f176.google.com with SMTP id cl12so4640643lbc.1
 for <895 <at> debbugs.gnu.org>; Fri, 08 Jan 2016 02:07:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=sender:subject:to:references:cc:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-type:content-transfer-encoding;
 bh=S+kI4z2nVtRJML0QGXjMaXul7uGv2JaibSJu5n+g6KA=;
 b=CCFY5vKg1CSiYeGktEefxmiyYvBM2jSdn1mIayfKtTPM5NFZWvhKzl2xqOU/VIL99v
 RLFM0B+TBmzYGP/Bz/3Sv+P3LQQ3MZ/5jcQIBcVjYJWklqchGWET9H3dRp22QzMlrhx6
 +ZURK00EI2RIPWdeI9bHM+ihBN3bJ3OuX6FaMk5RHCX0NfqVEghCvs6cXQUSyfmQ4yzg
 czny6rjorMtvLw6LMPZt/w0xfMNrs9ze0AZdOSULUokOouxtpW8etBoRgdEN2eip+Lee
 H8xY3Edp+gjlW1joAlJoiBeltbC35DyiTv8mX6pKnYqAx/qfTBi9ugcBTmGzoUi0XsTt
 tJLA==
X-Received: by 10.112.129.134 with SMTP id nw6mr40287592lbb.10.1452247643992; 
 Fri, 08 Jan 2016 02:07:23 -0800 (PST)
Received: from [192.168.1.190] ([178.252.127.222])
 by smtp.googlemail.com with ESMTPSA id
 h196sm18906443lfb.48.2016.01.08.02.07.22
 (version=TLSv1/SSLv3 cipher=OTHER);
 Fri, 08 Jan 2016 02:07:23 -0800 (PST)
Subject: Re: bug#895: slow processing of process output
To: Eli Zaretskii <eliz@HIDDEN>, Richard Copley <rcopley@HIDDEN>
References: <200809051543.m85FhWHV008307@HIDDEN>
 <CAPM58oi9QCxrdRJaSA5LV-G++fXE8rMoJuABT60b8_ihuPuR5Q@HIDDEN>
 <83ziwgpge7.fsf@HIDDEN>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <568F8A5A.8030101@HIDDEN>
Date: Fri, 8 Jan 2016 13:07:22 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:43.0) Gecko/20100101
 Thunderbird/43.0
MIME-Version: 1.0
In-Reply-To: <83ziwgpge7.fsf@HIDDEN>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.4 (/)
X-Debbugs-Envelope-To: 895
Cc: 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.4 (/)

On 01/08/2016 12:30 PM, Eli Zaretskii wrote:

> Would someone please try reproducing this on GNU or Unix system?

Looks fixed:

With warm cache, the original commands takes 4 seconds in Emacs, also 4 
seconds in my shell (fish inside terminator), and produces the output of 
17000 lines.

It's still <1 second when redirected to a file, but that's to be 
expected, I think.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 8 Jan 2016 10:00:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 08 05:00:20 2016
Received: from localhost ([127.0.0.1]:42270 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aHTqG-0007dS-6f
	for submit <at> debbugs.gnu.org; Fri, 08 Jan 2016 05:00:20 -0500
Received: from eggs.gnu.org ([208.118.235.92]:48174)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <eliz@HIDDEN>) id 1aHTqF-0007dG-Gz
 for 895 <at> debbugs.gnu.org; Fri, 08 Jan 2016 05:00:19 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1aHTq4-0005SW-Q3
 for 895 <at> debbugs.gnu.org; Fri, 08 Jan 2016 05:00:14 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:38322)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1aHTq4-0005SL-Nl; Fri, 08 Jan 2016 05:00:08 -0500
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1196
 helo=HOME-C4E4A596F7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1aHTq3-0002wS-Rd; Fri, 08 Jan 2016 05:00:08 -0500
Date: Fri, 08 Jan 2016 12:00:05 +0200
Message-Id: <83vb74pf16.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Alexis <flexibeast@HIDDEN>
In-reply-to: <87k2nkpfbc.fsf@HIDDEN> (message from Alexis on Fri, 08 Jan
 2016 20:53:59 +1100)
Subject: Re: bug#895: slow processing of process output
References: <200809051543.m85FhWHV008307@HIDDEN>
 <CAPM58oi9QCxrdRJaSA5LV-G++fXE8rMoJuABT60b8_ihuPuR5Q@HIDDEN>
 <83ziwgpge7.fsf@HIDDEN> <87k2nkpfbc.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 895
Cc: rcopley@HIDDEN, 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Alexis <flexibeast@HIDDEN>
> Cc: Richard Copley <rcopley@HIDDEN>, 895 <at> debbugs.gnu.org
> Date: Fri, 08 Jan 2016 20:53:59 +1100
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> > I think we need to try reproducing this on a Posix host before 
> > we can conclude the bug is fixed.  The original problem was 
> > reported on GNU/Linux, AFAICT, and the machinery used on Windows 
> > for communicating with subprocesses is very different from that 
> > used on Unix.
> >
> > Would someone please try reproducing this on GNU or Unix system?
> 
> Compiled from current master, running the rgrep command as per the 
> OP successfully returns a buffer of ~17000 lines almost instantly 
> on my Debian Jessie system.

Thanks, I guess this does mean we fixed it somehow.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 8 Jan 2016 09:54:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 08 04:54:11 2016
Received: from localhost ([127.0.0.1]:42266 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aHTkJ-0007TV-Hv
	for submit <at> debbugs.gnu.org; Fri, 08 Jan 2016 04:54:11 -0500
Received: from mail-pf0-f173.google.com ([209.85.192.173]:34511)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <flexibeast@HIDDEN>) id 1aHTkJ-0007TK-2a
 for 895 <at> debbugs.gnu.org; Fri, 08 Jan 2016 04:54:11 -0500
Received: by mail-pf0-f173.google.com with SMTP id q63so7916235pfb.1
 for <895 <at> debbugs.gnu.org>; Fri, 08 Jan 2016 01:54:10 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=references:from:to:cc:subject:in-reply-to:date:message-id
 :mime-version:content-type;
 bh=eni3KQkmkkcW7MLBC6ZXj+qz/UFBK1kotT9kWsUGDds=;
 b=J631HCQz/0ZpMldqaj3Xn0da1EglUBq20h8EjYk7xtMZvtNbHGNtcrYVgKc98EzZLj
 HSy5c2LtyHWhHW8xAmW7kR8+hrAf5y5RATp4quJBFbJBJD6bhMFmGUJG9FuntPa5DKWL
 LjuV8MMYIXnZ/xtMGGaNtMbghbT751XZlOGxbm87K+lNQq72gOe8NNymR+qB/6n7+dGt
 0uODfDzohXZ6yMF+JfmBAtSZfb2YakPxcGd7ngiFk/lRxTYJPqfleo4ArNGJLwq+JIfB
 eRkF/4Mgh2oHne/OHB25ZAmSzAP6tFKJTL8Q10erBPz2m/e7v9bmU84TqfylQ2H5Pybh
 LxCQ==
X-Received: by 10.98.75.22 with SMTP id y22mr3074932pfa.147.1452246845204;
 Fri, 08 Jan 2016 01:54:05 -0800 (PST)
Received: from localhost (114-198-94-242.dyn.iinet.net.au. [114.198.94.242])
 by smtp.gmail.com with ESMTPSA id g10sm3460920pfd.49.2016.01.08.01.54.02
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Fri, 08 Jan 2016 01:54:03 -0800 (PST)
References: <200809051543.m85FhWHV008307@HIDDEN>
 <CAPM58oi9QCxrdRJaSA5LV-G++fXE8rMoJuABT60b8_ihuPuR5Q@HIDDEN>
 <83ziwgpge7.fsf@HIDDEN>
From: Alexis <flexibeast@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#895: slow processing of process output
In-reply-to: <83ziwgpge7.fsf@HIDDEN>
Date: Fri, 08 Jan 2016 20:53:59 +1100
Message-ID: <87k2nkpfbc.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; format=flowed
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 895
Cc: Richard Copley <rcopley@HIDDEN>, 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)


Eli Zaretskii <eliz@HIDDEN> writes:

> I think we need to try reproducing this on a Posix host before 
> we can conclude the bug is fixed.  The original problem was 
> reported on GNU/Linux, AFAICT, and the machinery used on Windows 
> for communicating with subprocesses is very different from that 
> used on Unix.
>
> Would someone please try reproducing this on GNU or Unix system?

Compiled from current master, running the rgrep command as per the 
OP successfully returns a buffer of ~17000 lines almost instantly 
on my Debian Jessie system.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 8 Jan 2016 09:30:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jan 08 04:30:54 2016
Received: from localhost ([127.0.0.1]:42249 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aHTNl-0006vb-Un
	for submit <at> debbugs.gnu.org; Fri, 08 Jan 2016 04:30:54 -0500
Received: from eggs.gnu.org ([208.118.235.92]:40611)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <eliz@HIDDEN>) id 1aHTNk-0006vQ-Nq
 for 895 <at> debbugs.gnu.org; Fri, 08 Jan 2016 04:30:52 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1aHTNb-00052E-JV
 for 895 <at> debbugs.gnu.org; Fri, 08 Jan 2016 04:30:47 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:37453)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1aHTNb-00052A-GD; Fri, 08 Jan 2016 04:30:43 -0500
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:1172
 helo=HOME-C4E4A596F7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1aHTNa-0004n9-Oh; Fri, 08 Jan 2016 04:30:43 -0500
Date: Fri, 08 Jan 2016 11:30:40 +0200
Message-Id: <83ziwgpge7.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Richard Copley <rcopley@HIDDEN>
In-reply-to: <CAPM58oi9QCxrdRJaSA5LV-G++fXE8rMoJuABT60b8_ihuPuR5Q@HIDDEN>
 (message from Richard Copley on Fri, 8 Jan 2016 01:24:08 +0000)
Subject: Re: bug#895: slow processing of process output
References: <200809051543.m85FhWHV008307@HIDDEN>
 <CAPM58oi9QCxrdRJaSA5LV-G++fXE8rMoJuABT60b8_ihuPuR5Q@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 895
Cc: 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: Richard Copley <rcopley@HIDDEN>
> Date: Fri, 8 Jan 2016 01:24:08 +0000
> 
> I tested in current master with mingw-w64 on Windows. The rgrep
> command given in the OP takes 29 s in Emacs, but the corresponding
> shell command[1] takes over a minute in Windows cmd.exe (at a guess,
> that's because cmd.exe scrolls line-by-line so has to push a lot of
> pixels).
> 
> Likewise, building Emacs is faster with "M-x compile" than it is on
> the shell (from a clean checkout on my configuration it takes 7 min 43
> seconds in Emacs and 9 minutes 15 seconds in cmd.exe).
> 
> I'm tempted to conclude that this bug has been fixed.

Thanks.

I think we need to try reproducing this on a Posix host before we can
conclude the bug is fixed.  The original problem was reported on
GNU/Linux, AFAICT, and the machinery used on Windows for communicating
with subprocesses is very different from that used on Unix.

Would someone please try reproducing this on GNU or Unix system?

TIA




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 8 Jan 2016 01:27:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 07 20:27:10 2016
Received: from localhost ([127.0.0.1]:42096 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aHLpe-0000QT-Cg
	for submit <at> debbugs.gnu.org; Thu, 07 Jan 2016 20:27:10 -0500
Received: from mail-yk0-f175.google.com ([209.85.160.175]:36517)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <rcopley@HIDDEN>) id 1aHLpd-0000QF-Em
 for 895 <at> debbugs.gnu.org; Thu, 07 Jan 2016 20:27:09 -0500
Received: by mail-yk0-f175.google.com with SMTP id v14so254052773ykd.3
 for <895 <at> debbugs.gnu.org>; Thu, 07 Jan 2016 17:27:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:from:date:message-id:subject:to:content-type;
 bh=gXHWC8RNYhCMouu/pzHhtIhDw4chjGGfnaVS0zYZg3A=;
 b=Gsr+GtOq8lWTrv47H285wfwjfDb53X5R1Sxo3y0/vLFVf5HKlCGXpa1dv1reG4hWyb
 ckIVtNwrLtZZypD1X/6L+KZyHq2vXRz0IG992lM+xsxJmwY0eaDUPZ1xT74xCM19EoAm
 fx6R9SyRd9ExE02KxQR9yZIpsRCeV0v+KLNG/0QF1yZ75kPgQaMD1paOqdgYXbO0Vmzk
 N7pmyGuUlM8TyjbB/wAX//ITXNQ5E8tRS2KhogsMz25FB9ZAF0tcoBub/ZvGx0uCRVqK
 RkzJqGG/RbyoL+bfO2xkz+mvTFelC2mLsGDsp8p4p5HCea63p14i5Xq2qO+zAr8Kkj9Q
 9Khg==
X-Received: by 10.13.246.130 with SMTP id g124mr78620962ywf.29.1452216424222; 
 Thu, 07 Jan 2016 17:27:04 -0800 (PST)
MIME-Version: 1.0
Received: by 10.37.207.214 with HTTP; Thu, 7 Jan 2016 17:26:34 -0800 (PST)
From: Richard Copley <rcopley@HIDDEN>
Date: Fri, 8 Jan 2016 01:26:34 +0000
Message-ID: <CAPM58ojL3LdEfBy2uyaLh3euLGrhoh6s=DLYYLtz3rhXuwnrOQ@HIDDEN>
Subject: Re: bug#895: slow processing of process output
To: 895 <at> debbugs.gnu.org
Content-Type: text/plain; charset=UTF-8
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 895
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

tags 685872 unreproducible




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 8 Jan 2016 01:24:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 07 20:24:45 2016
Received: from localhost ([127.0.0.1]:42092 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aHLnI-0000MZ-TZ
	for submit <at> debbugs.gnu.org; Thu, 07 Jan 2016 20:24:45 -0500
Received: from mail-yk0-f175.google.com ([209.85.160.175]:35967)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <rcopley@HIDDEN>) id 1aHLnH-0000MJ-77
 for 895 <at> debbugs.gnu.org; Thu, 07 Jan 2016 20:24:43 -0500
Received: by mail-yk0-f175.google.com with SMTP id v14so254005791ykd.3
 for <895 <at> debbugs.gnu.org>; Thu, 07 Jan 2016 17:24:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:from:date:message-id:subject:to:content-type;
 bh=SrWmehT8P6pJPqmwSlutCaSmui+aEYYdGgAoSZmP4rM=;
 b=NHX0e0za2Eq3hwkC5ybMwfHCSCWgxC3LDdlgwo3y2oo5IGIAPwm09RttVaKrnxWFOF
 URWQKwDYu02rd4vXFMpIRBS6ahxxzQRdcdw4ZTYVofgr95HNmRjDGYbucMLSgoSQ1Dyr
 +YDZLR9NCubCr41RF91Xj4FcoiExqAIGDD+C0x7z/ExnQqGVGJf2KYhOTUff7VHyZ8gS
 q3VgcwpVWvCe3sqmJjWSIPAHe10t4ksCIhLN5Hg2LTyA3JwMKTDYi+GN8ukF1g3AiKE6
 X2NZhtJh0Ro8xl30JzC0GZQy9y8XRVC1sPbXKWeJ49WF4a2y9utL1HlD9ZfQ7am+HeDT
 /Djw==
X-Received: by 10.13.218.198 with SMTP id c189mr87179816ywe.165.1452216277667; 
 Thu, 07 Jan 2016 17:24:37 -0800 (PST)
MIME-Version: 1.0
Received: by 10.37.207.214 with HTTP; Thu, 7 Jan 2016 17:24:08 -0800 (PST)
From: Richard Copley <rcopley@HIDDEN>
Date: Fri, 8 Jan 2016 01:24:08 +0000
Message-ID: <CAPM58oi9QCxrdRJaSA5LV-G++fXE8rMoJuABT60b8_ihuPuR5Q@HIDDEN>
Subject: Re: bug#895: slow processing of process output
To: 895 <at> debbugs.gnu.org
Content-Type: text/plain; charset=UTF-8
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 895
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

I tested in current master with mingw-w64 on Windows. The rgrep
command given in the OP takes 29 s in Emacs, but the corresponding
shell command[1] takes over a minute in Windows cmd.exe (at a guess,
that's because cmd.exe scrolls line-by-line so has to push a lot of
pixels).

Likewise, building Emacs is faster with "M-x compile" than it is on
the shell (from a clean checkout on my configuration it takes 7 min 43
seconds in Emacs and 9 minutes 15 seconds in cmd.exe).

I'm tempted to conclude that this bug has been fixed.

[1] G:\usr\bin\find.exe . -type d "(" -path "*/SCCS" -o -path "*/RCS"
-o -path "*/CVS" -o -path "*/MCVS" -o -path "*/.src" -o -path "*/.svn"
-o -path "*/.git" -o -path "*/.hg" -o -path "*/.bzr" -o -path "*/_MTN"
-o -path "*/_darcs" -o -path "*/{arch}" ")" -prune -o ^"^!^" -type d
"(" -name ".#*" -o -name "*.o" -o -name "*~" -o -name "*.bin" -o -name
"*.bak" -o -name "*.obj" -o -name "*.map" -o -name "*.ico" -o -name
"*.pif" -o -name "*.lnk" -o -name "*.a" -o -name "*.ln" -o -name
"*.blg" -o -name "*.bbl" -o -name "*.dll" -o -name "*.drv" -o -name
"*.vxd" -o -name "*.386" -o -name "*.elc" -o -name "*.lof" -o -name
"*.glo" -o -name "*.idx" -o -name "*.lot" -o -name "*.fmt" -o -name
"*.tfm" -o -name "*.class" -o -name "*.fas" -o -name "*.lib" -o -name
"*.mem" -o -name "*.x86f" -o -name "*.sparcf" -o -name "*.dfsl" -o
-name "*.pfsl" -o -name "*.d64fsl" -o -name "*.p64fsl" -o -name
"*.lx64fsl" -o -name "*.lx32fsl" -o -name "*.dx64fsl" -o -name
"*.dx32fsl" -o -name "*.fx64fsl" -o -name "*.fx32fsl" -o -name
"*.sx64fsl" -o -name "*.sx32fsl" -o -name "*.wx64fsl" -o -name
"*.wx32fsl" -o -name "*.fasl" -o -name "*.ufsl" -o -name "*.fsl" -o
-name "*.dxl" -o -name "*.lo" -o -name "*.la" -o -name "*.gmo" -o
-name "*.mo" -o -name "*.toc" -o -name "*.aux" -o -name "*.cp" -o
-name "*.fn" -o -name "*.ky" -o -name "*.pg" -o -name "*.tp" -o -name
"*.vr" -o -name "*.cps" -o -name "*.fns" -o -name "*.kys" -o -name
"*.pgs" -o -name "*.tps" -o -name "*.vrs" -o -name "*.pyc" -o -name
"*.pyo" ")" -prune -o  -type f "(" -iname "*.el" ")" -exec
G:\usr\bin\grep.exe --color=always -i -nH -e "emacs" {} +


In GNU Emacs 25.0.50.1 (x86_64-w64-mingw32)
 of 2016-01-02 built on MACHINE
Repository revision: 372d00a981942a5da868a33372eb29d5806f2ab3
Windowing system distributor 'Microsoft Corp.', version 10.0.10586
Configured using:
 'configure --prefix /c/emacs/emacs-20160101-122333
 --without-imagemagick --disable-dependency-tracking
 --enable-locallisppath=%emacs_dir%/../site-lisp 'CFLAGS=-Og -g -ggdb''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS

Important settings:
  value of $LANG: ENG
  locale-coding-system: cp1252

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message dired format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns help-mode easymenu cl-loaddefs pcase cl-lib mail-prsvr
mail-utils time-date mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table
w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote w32notify dbusbind w32 multi-tty
make-network-process emacs)

Memory information:
((conses 16 83465 5112)
 (symbols 56 19008 0)
 (miscs 48 36 86)
 (strings 32 14081 5015)
 (string-bytes 1 386417)
 (vectors 16 11288)
 (vector-slots 8 408630 5027)
 (floats 8 139 4)
 (intervals 56 239 22)
 (buffers 976 11))




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 21 Feb 2012 22:18:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Feb 21 17:18:10 2012
Received: from localhost ([127.0.0.1]:49333 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1Rzy2C-000734-MP
	for submit <at> debbugs.gnu.org; Tue, 21 Feb 2012 17:18:10 -0500
Received: from mail-wi0-f196.google.com ([209.85.212.196]:59087)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <mh.origin@HIDDEN>) id 1Rzxsc-0006nz-67
	for 895 <at> debbugs.gnu.org; Tue, 21 Feb 2012 17:08:15 -0500
Received: by wibhn14 with SMTP id hn14so585019wib.3
	for <895 <at> debbugs.gnu.org>; Tue, 21 Feb 2012 14:05:50 -0800 (PST)
Received-SPF: pass (google.com: domain of mh.origin@HIDDEN designates
	10.180.107.68 as permitted sender) client-ip=10.180.107.68; 
Authentication-Results: mr.google.com;
	spf=pass (google.com: domain of mh.origin@HIDDEN
	designates 10.180.107.68 as permitted sender)
	smtp.mail=mh.origin@HIDDEN;
	dkim=pass header.i=mh.origin@HIDDEN
Received: from mr.google.com ([10.180.107.68])
	by 10.180.107.68 with SMTP id ha4mr15442729wib.9.1329861950021
	(num_hops = 1); Tue, 21 Feb 2012 14:05:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type;
	bh=kXbavkVzeUhB4tpdjCoskCQAJ2cAI2F/XcuoFUiEm6w=;
	b=UeVFuuto9WC6pzZXiEQ50t5dTgaxzSc7qJUCtqJV8rR0I971TdxYzmrtT/QRVX7d4v
	ct6AecgfBOVVmjm6cBzfQdVhcy1C6MroTFr0SWRwLzHgPWId+tFUAL58dA+v6QprMFZB
	ret7Y4scPMmYYT2kPUaNx+YnJFx3XXvjjOkTk=
MIME-Version: 1.0
Received: by 10.180.107.68 with SMTP id ha4mr12808624wib.9.1329861948676; Tue,
	21 Feb 2012 14:05:48 -0800 (PST)
Received: by 10.216.168.11 with HTTP; Tue, 21 Feb 2012 14:05:48 -0800 (PST)
Date: Tue, 21 Feb 2012 17:05:48 -0500
Message-ID: <CAHSQ_j8_gwThDyTsOo5brVNAwqSAwHvUoKx3iQE=08BMsduSiA@HIDDEN>
Subject: slow processing of process output on Mac
From: mh origin <mh.origin@HIDDEN>
To: 895 <at> debbugs.gnu.org
Content-Type: multipart/alternative; boundary=e89a8f13ec340d184404b9809db6
X-Spam-Score: -2.6 (--)
X-Debbugs-Envelope-To: 895
X-Mailman-Approved-At: Tue, 21 Feb 2012 17:18:07 -0500
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.6 (--)

--e89a8f13ec340d184404b9809db6
Content-Type: text/plain; charset=ISO-8859-1

I see this with matlab-shell, when trying to tab-complete, in Emacs 23 and
emacs 24 on the Mac, both on Lion and on Snow Leopard.  I fired this up in
the debugger and it looks like it gets stuck on a system call but I'm
afraid I couldn't figure out any more than that.

At first start, Matlab returns completions just fine, but after about an
hour it starts hanging when tab is pressed.

The function called on completion is here:
https://github.com/ruediger/matlab-emacs/blob/master/matlab.el#L4831

And the accept-process-output call that hangs is here:
https://github.com/ruediger/matlab-emacs/blob/master/matlab.el#L5304

I'm using Emacs 24.0.93 from here:
http://emacsformacosx.com/emacs-builds/Emacs-pretest-24.0.93-universal-10.6.8.dmg

Can I do anything else to help people track this down?

--e89a8f13ec340d184404b9809db6
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I see this with matlab-shell, when trying to tab-complete, in Emacs 23=20
and emacs 24 on the Mac, both on Lion and on Snow Leopard.=A0 I fired this
 up in the debugger and it looks like it gets stuck on a system call but
=20
I&#39;m afraid I couldn&#39;t figure out any more than that.<br>

<br>

At first start, Matlab returns completions just fine, but after about an ho=
ur it starts hanging when tab is pressed. <br>

<br>

The function called on completion is here:<br>

<a href=3D"https://github.com/ruediger/matlab-emacs/blob/master/matlab.el#L=
4831">https://github.com/ruediger/matlab-emacs/blob/master/matlab.el#L4831<=
/a><br>

<br>

And the accept-process-output call that hangs is here:<br>

<a href=3D"https://github.com/ruediger/matlab-emacs/blob/master/matlab.el#L=
5304">https://github.com/ruediger/matlab-emacs/blob/master/matlab.el#L5304<=
/a><br>

<br>
I&#39;m using Emacs 24.0.93 from here:<br>
<a href=3D"http://emacsformacosx.com/emacs-builds/Emacs-pretest-24.0.93-uni=
versal-10.6.8.dmg">http://emacsformacosx.com/emacs-builds/Emacs-pretest-24.=
0.93-universal-10.6.8.dmg</a><br>

<br>
Can I do anything else to help people track this down?

--e89a8f13ec340d184404b9809db6--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 6 Jul 2011 19:17:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 06 15:17:42 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QeXbS-00038z-Df
	for submit <at> debbugs.gnu.org; Wed, 06 Jul 2011 15:17:42 -0400
Received: from ironport2-out.teksavvy.com ([206.248.154.181]
	helo=ironport2-out.pppoe.ca)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <monnier@HIDDEN>) id 1QeXbQ-00038g-2h
	for 895 <at> debbugs.gnu.org; Wed, 06 Jul 2011 15:17:40 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Av0EAJKzFE5FxIV4/2dsb2JhbABTqAd4iHrDT4Y3BJ5rhC8
X-IronPort-AV: E=Sophos;i="4.65,488,1304308800"; d="scan'208";a="120708537"
Received: from 69-196-133-120.dsl.teksavvy.com (HELO ceviche.home)
	([69.196.133.120])
	by ironport2-out.pppoe.ca with ESMTP/TLS/ADH-AES256-SHA;
	06 Jul 2011 15:17:34 -0400
Received: by ceviche.home (Postfix, from userid 20848)
	id 39F8B660D9; Wed,  6 Jul 2011 15:17:34 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Dan Nicolaescu <dann@HIDDEN>
Subject: Re: bug#895: slow processing of process output
Message-ID: <jwv1uy3qkxx.fsf-monnier+emacs@HIDDEN>
References: <m11vthqpn9.fsf@HIDDEN> <m3liwk86q9.fsf@HIDDEN>
	<yxqaacrmnkm.fsf@HIDDEN>
Date: Wed, 06 Jul 2011 15:17:34 -0400
In-Reply-To: <yxqaacrmnkm.fsf@HIDDEN> (Dan Nicolaescu's message of
	"Wed, 06 Jul 2011 11:32:57 -0400")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.1 (--)
X-Debbugs-Envelope-To: 895
Cc: larsi@HIDDEN, 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.1 (--)

> For example compare the time to run make bootstrap when using M-x
> compile vs doing it in an xterm.

Agreed.  I encourage people to try and look into this problem.
First thing is to profile to figure out where the time is spent.


        Stefan




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 6 Jul 2011 19:16:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 06 15:16:07 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QeXZu-00035w-Qc
	for submit <at> debbugs.gnu.org; Wed, 06 Jul 2011 15:16:07 -0400
Received: from ironport2-out.teksavvy.com ([206.248.154.183]
	helo=ironport2-out.pppoe.ca)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <monnier@HIDDEN>) id 1QeXZs-00035K-0m
	for 895 <at> debbugs.gnu.org; Wed, 06 Jul 2011 15:16:05 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Av0EAJKzFE5FxIV4/2dsb2JhbABTqAd4iHrDT4Y3BJ5rhC8
X-IronPort-AV: E=Sophos;i="4.65,488,1304308800"; d="scan'208";a="120708357"
Received: from 69-196-133-120.dsl.teksavvy.com (HELO ceviche.home)
	([69.196.133.120])
	by ironport2-out.pppoe.ca with ESMTP/TLS/ADH-AES256-SHA;
	06 Jul 2011 15:15:58 -0400
Received: by ceviche.home (Postfix, from userid 20848)
	id CD158660D9; Wed,  6 Jul 2011 15:15:42 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Lars Magne Ingebrigtsen <larsi@HIDDEN>
Subject: Re: bug#895: slow processing of process output
Message-ID: <jwv7h7vql4q.fsf-monnier+emacs@HIDDEN>
References: <m11vthqpn9.fsf@HIDDEN> <m3liwk86q9.fsf@HIDDEN>
	<jwvbox7tpng.fsf-monnier+emacs@HIDDEN>
	<m3liwb766o.fsf@HIDDEN>
Date: Wed, 06 Jul 2011 15:15:42 -0400
In-Reply-To: <m3liwb766o.fsf@HIDDEN> (Lars Magne Ingebrigtsen's
	message of "Wed, 06 Jul 2011 17:57:35 +0200")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.1 (--)
X-Debbugs-Envelope-To: 895
Cc: 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.1 (--)

> I wonder whether (in the long term) it would make more sense to have the
> `M-x grep' command be less...  shell-script-ey.

There are other commands that do that that (e.g. rgrep).
I like grep specifically because the Emacs wrapper is shallow which lets
me do anything I fancy so long as the output has the expected format.

> That is, if we instead prompted for the regexp we wanted to match
> explicitly, then we could highlight it ourselves instead of relying on
> grep to do it.

Actually doing this reliably can be tricky because of slight divergence
in the syntax and features of regular expressions accepted by Emacs and
by the various greps out there.

> Ok, here's one other possibility off the top of my head: What about if
> Emacs parsed the command line?  :-)

You go boy!


        Stefan




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 6 Jul 2011 18:50:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 06 14:50:27 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QeXB5-0002P4-1B
	for submit <at> debbugs.gnu.org; Wed, 06 Jul 2011 14:50:27 -0400
Received: from hermes.netfonds.no ([80.91.224.195])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <larsi@HIDDEN>) id 1QeXB2-0002Or-Js
	for 895 <at> debbugs.gnu.org; Wed, 06 Jul 2011 14:50:25 -0400
Received: from cm-84.215.51.58.getinternet.no ([84.215.51.58]
	helo=quimbies.gnus.org)
	by hermes.netfonds.no with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.72) (envelope-from <larsi@HIDDEN>)
	id 1QeXAo-0000SM-1f; Wed, 06 Jul 2011 20:50:10 +0200
From: Lars Magne Ingebrigtsen <larsi@HIDDEN>
To: "Drew Adams" <drew.adams@HIDDEN>
Subject: Re: bug#895: slow processing of process output
In-Reply-To: <6D56BDCE22F94C77B10AB679E92860C6@HIDDEN> (Drew Adams's
	message of "Wed, 6 Jul 2011 10:58:35 -0700")
Date: Wed, 06 Jul 2011 20:50:00 +0200
Message-ID: <m3liwbw8fb.fsf@HIDDEN>
References: <m11vthqpn9.fsf@HIDDEN> <m3liwk86q9.fsf@HIDDEN>
	<jwvbox7tpng.fsf-monnier+emacs@HIDDEN>
	<m3liwb766o.fsf@HIDDEN>
	<6D56BDCE22F94C77B10AB679E92860C6@HIDDEN>
User-Agent: Gnus/5.110018 (No Gnus v0.18) Emacs/24.0.50 (gnu/linux)
X-Now-Playing: Demdike Stare's _Tryptych (2): Liberation Through Hearing_:
	"Matilda's Dream"
X-Hashcash: 1:23:110706:drew.adams@HIDDEN::Bk3elmnRM7ozp3tS:00000000000000000000000000000000000000008/YC
X-Hashcash: 1:23:110706:895 <at> debbugs.gnu.org::0nPrlRFanogbVn1r:000000000000000000000000000000000000000000jXFv
X-Hashcash: 1:23:110706:monnier@HIDDEN::8dnDg7sl/Ucw1177:00000000000000000000000000000000000019y8j
MIME-Version: 1.0
Content-Type: text/plain
X-MailScanner-ID: 1QeXAo-0000SM-1f
X-Netfonds-MailScanner: Found to be clean
X-Netfonds-MailScanner-From: larsi@HIDDEN
MailScanner-NULL-Check: 1310583010.57599@wkeZ+fuZLJ36arVmLEwKjA
X-Spam-Status: No
X-Spam-Score: -2.8 (--)
X-Debbugs-Envelope-To: 895
Cc: 'Stefan Monnier' <monnier@HIDDEN>, 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.8 (--)

"Drew Adams" <drew.adams@HIDDEN> writes:

> Absolutely.  But there is no reason to conflate "[having] the `M-x
> grep' command be less...shell-script-ey" with doing the highlighting
> in Lisp.  Why do you want less shell-script-ey?

For the very simple use cases, having "grep -nH -e " seems like a
distraction.  But I usually want it more shell-script-ey than less --
that `M-x grep' adds "/dev/null" at the end has often annoyed me
greatly, because you can't then say "grep foo *.el | grep -v bar".

> That is what I did, IIUYC.  The relevant bit of the compile+20.el code:
>
> ;; Remember `grep-pattern' for highlighting, if highlighting is possible.
> (cond (;; Quoted pattern (either "..." or '...')

Yeah, That's what I was thinking of.  :-)

> FWIW, I think the Emacs Dev approach of using external `grep'
> highlighting is generally not bad (but there have been a few bugs that
> needed to be fixed along the way).

Yes, it's a good idea, but there are these corner cases where it doesn't
work quite well.  Like when you have some (implicit) commands in the
pipeline that reacts badly to being run in a terminal, like the guy who
did a "git log" (or something?), and git then started "less", because it
thought it was in a terminal.

So to me it looks like the best long-term general solution is to bind
`process-connection-type' to nil in compile.el, and fix the highlighting
in Emacs Lisp, like your code does.

Perhaps.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 6 Jul 2011 17:58:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 06 13:58:55 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QeWND-0007jo-Dz
	for submit <at> debbugs.gnu.org; Wed, 06 Jul 2011 13:58:55 -0400
Received: from acsinet15.oracle.com ([141.146.126.227])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <drew.adams@HIDDEN>) id 1QeWNB-0007jW-8n
	for 895 <at> debbugs.gnu.org; Wed, 06 Jul 2011 13:58:54 -0400
Received: from acsinet21.oracle.com (acsinet21.oracle.com [141.146.126.237])
	by acsinet15.oracle.com (Switch-3.4.4/Switch-3.4.4) with ESMTP id
	p66Hwj2J026791
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK);
	Wed, 6 Jul 2011 17:58:47 GMT
Received: from acsmt358.oracle.com (acsmt358.oracle.com [141.146.40.158])
	by acsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id
	p66HwhD5013963
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Wed, 6 Jul 2011 17:58:44 GMT
Received: from abhmt110.oracle.com (abhmt110.oracle.com [141.146.116.62])
	by acsmt358.oracle.com (8.12.11.20060308/8.12.11) with ESMTP id
	p66HwbGg032697; Wed, 6 Jul 2011 12:58:38 -0500
Received: from dradamslap1 (/10.159.55.239)
	by default (Oracle Beehive Gateway v4.0)
	with ESMTP ; Wed, 06 Jul 2011 10:58:37 -0700
From: "Drew Adams" <drew.adams@HIDDEN>
To: "'Lars Magne Ingebrigtsen'" <larsi@HIDDEN>,
	"'Stefan Monnier'" <monnier@HIDDEN>
References: <m11vthqpn9.fsf@HIDDEN>
	<m3liwk86q9.fsf@HIDDEN><jwvbox7tpng.fsf-monnier+emacs@HIDDEN>
	<m3liwb766o.fsf@HIDDEN>
Subject: RE: bug#895: slow processing of process output
Date: Wed, 6 Jul 2011 10:58:35 -0700
Message-ID: <6D56BDCE22F94C77B10AB679E92860C6@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Mailer: Microsoft Office Outlook 11
In-Reply-To: <m3liwb766o.fsf@HIDDEN>
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6109
Thread-Index: Acw7+Dck5Q7YfjFtQh2hedHjZSK3NQAAsV8A
X-Source-IP: acsinet21.oracle.com [141.146.126.237]
X-Auth-Type: Internal IP
X-CT-RefId: str=0001.0A020208.4E14A257.00DD:SCFMA922111,ss=1,re=-4.000,fgs=0
X-Spam-Score: -4.4 (----)
X-Debbugs-Envelope-To: 895
Cc: 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -4.4 (----)

> That is, if we instead prompted for the regexp we wanted to match
> explicitly, then we could highlight it ourselves instead of relying on
> grep to do it.

I did that (without prompting), for Emacs 20 & 21.  See:
http://www.emacswiki.org/emacs/download/compile%2b20.el

It was I who proposed that vanilla Emacs itself highlight the matches.  That
idea was adopted, but Emacs Dev decided to do the highlighting using escape
chars provided by (some) external `grep' commands.

(So I adjusted my code too to use that approach:
http://www.emacswiki.org/emacs/download/grep%2b.el)

> However, `M-x grep' as it is is pretty neat, because you can 
> fiddle with all the options and do
> grep ... | grep -v | etc

Absolutely.  But there is no reason to conflate "[having] the `M-x grep' command
be less...shell-script-ey" with doing the highlighting in Lisp.  Why do you want
less shell-script-ey?

My code did the highlighting in Lisp, but it should not in any way interfere
with piping etc.  (However, for unrelated reasons piping with Emacs `grep' is
not available until Emacs 22, and it was Emacs 22 that introduced
external-`grep' highlighting.)

> So providing something that is as flexible as `M-x grep' is today, but
> allow Emacs greater control of the output sounds difficult.

Not at all (IIUYC).

> Ok, here's one other possibility off the top of my head: What about if
> Emacs parsed the command line?  :-) That is, if you're typing
> grep -nH -e foo.*bar *.el
> then the grep syntax isn't that impossible to parse.  I mean, finding
> the regexp, and highlighting the matching bits.

That is what I did, IIUYC.  The relevant bit of the compile+20.el code:

;; Remember `grep-pattern' for highlighting, if highlighting is possible.
(cond (;; Quoted pattern (either "..." or '...')
       (string-match
        (concat
         grep-program
         "[ \t]*\\(-[a-zA-Z]+\\s-+\\)*[ \t]*\\('[^']+'\\|\"[^\"]+\"\\)") ;"
        command-args)
       (setq grep-pattern
             (substring command-args
                        (1+ (match-beginning 2)) (1- (match-end 2)))))
      (;; Unquoted pattern.
       (string-match
        (concat grep-program
                "[ \t]*\\(-[a-zA-Z]+\\s-+\\)*[ \t]*\\([^ \n\t'\"]+\\)") ; "
        command-args)
       (setq grep-pattern
             (substring command-args (match-beginning 2) (match-end 2))))
      (t;; Bad pattern.
       (setq grep-pattern nil)))

;; Account for a case-insensitivity option.
(when (and (not (string= "" grep-case-insensitive-option))
           (string-match grep-case-insensitive-option command-args))
  (setq grep-pattern (mapconcat
                      (lambda (char)
                        (if (or (and (>= char ?a) (<= char ?z))
                                (and (>= char ?A) (<= char ?Z)))
                            (concat "["  (char-to-string (downcase char))
                                    (char-to-string (upcase char)) "]")
                          (char-to-string char)))
                      grep-pattern "")))

This is a quick hack that works well in general (for Emacs 20/21).  But note
that the `grep-pattern' highlighting is not anchored after the file name and
line number, so if the file name itself matches the `grep-pattern' then it
doesn't highlight the latter.

Someone could work out the code to anchor the pattern match properly.  I didn't
bother to try, since it is good enough as is, for my use.

> It isn't difficult to imagine more complex command lines that Emacs
> wouldn't be able to parse, though.  And you'd lose highlighting of the
> matches there...

Yes.

FWIW, I think the Emacs Dev approach of using external `grep' highlighting is
generally not bad (but there have been a few bugs that needed to be fixed along
the way).





Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 6 Jul 2011 15:57:56 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 06 11:57:56 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QeUU7-0002K6-Sm
	for submit <at> debbugs.gnu.org; Wed, 06 Jul 2011 11:57:56 -0400
Received: from hermes.netfonds.no ([80.91.224.195])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <larsi@HIDDEN>) id 1QeUU4-0002Ji-E5
	for 895 <at> debbugs.gnu.org; Wed, 06 Jul 2011 11:57:54 -0400
Received: from cm-84.215.51.58.getinternet.no ([84.215.51.58]
	helo=quimbies.gnus.org)
	by hermes.netfonds.no with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.72) (envelope-from <larsi@HIDDEN>)
	id 1QeUTt-00067I-Aj; Wed, 06 Jul 2011 17:57:41 +0200
From: Lars Magne Ingebrigtsen <larsi@HIDDEN>
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#895: slow processing of process output
In-Reply-To: <jwvbox7tpng.fsf-monnier+emacs@HIDDEN> (Stefan Monnier's message
	of "Wed, 06 Jul 2011 11:07:13 -0400")
Date: Wed, 06 Jul 2011 17:57:35 +0200
Message-ID: <m3liwb766o.fsf@HIDDEN>
References: <m11vthqpn9.fsf@HIDDEN> <m3liwk86q9.fsf@HIDDEN>
	<jwvbox7tpng.fsf-monnier+emacs@HIDDEN>
User-Agent: Gnus/5.110018 (No Gnus v0.18) Emacs/24.0.50 (gnu/linux)
X-Now-Playing: Various's _Delta Swamp Rock (2)_: "Dan Penn - If Love Was Money"
X-Hashcash: 1:23:110706:895 <at> debbugs.gnu.org::StG46zftbYi8qiXX:0000000000000000000000000000000000000000000FHr
X-Hashcash: 1:23:110706:monnier@HIDDEN::n9cF/ZAmnjNeAfRC:0000000000000000000000000000000000000FYDM
MIME-Version: 1.0
Content-Type: text/plain
X-MailScanner-ID: 1QeUTt-00067I-Aj
X-Netfonds-MailScanner: Found to be clean
X-Netfonds-MailScanner-From: larsi@HIDDEN
MailScanner-NULL-Check: 1310572661.43299@Bp/mkq7HdeQdCVNmXKDFxA
X-Spam-Status: No
X-Spam-Score: -2.8 (--)
X-Debbugs-Envelope-To: 895
Cc: 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.8 (--)

I wonder whether (in the long term) it would make more sense to have the
`M-x grep' command be less...  shell-script-ey.

That is, if we instead prompted for the regexp we wanted to match
explicitly, then we could highlight it ourselves instead of relying on
grep to do it.

However, `M-x grep' as it is is pretty neat, because you can fiddle with
all the options and do

grep ... | grep -v | etc

So providing something that is as flexible as `M-x grep' is today, but
allow Emacs greater control of the output sounds difficult.

Ok, here's one other possibility off the top of my head: What about if
Emacs parsed the command line?  :-)

That is, if you're typing

grep -nH -e foo.*bar *.el

then the grep syntax isn't that impossible to parse.  I mean, finding
the regexp, and highlighting the matching bits.

It isn't difficult to imagine more complex command lines that Emacs
wouldn't be able to parse, though.  And you'd lose highlighting of the
matches there...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 6 Jul 2011 15:57:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 06 11:57:54 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QeUU5-0002Jt-Rf
	for submit <at> debbugs.gnu.org; Wed, 06 Jul 2011 11:57:54 -0400
Received: from hermes.netfonds.no ([80.91.224.195])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <larsi@HIDDEN>) id 1QeUU3-0002Jg-TB
	for 895 <at> debbugs.gnu.org; Wed, 06 Jul 2011 11:57:52 -0400
Received: from cm-84.215.51.58.getinternet.no ([84.215.51.58]
	helo=quimbies.gnus.org)
	by hermes.netfonds.no with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.72) (envelope-from <larsi@HIDDEN>)
	id 1QeUTs-00067B-CB; Wed, 06 Jul 2011 17:57:40 +0200
From: Lars Magne Ingebrigtsen <larsi@HIDDEN>
To: Stefan Monnier <monnier@HIDDEN>
Subject: Re: bug#895: slow processing of process output
In-Reply-To: <jwvbox7tpng.fsf-monnier+emacs@HIDDEN> (Stefan Monnier's message
	of "Wed, 06 Jul 2011 11:07:13 -0400")
Date: Wed, 06 Jul 2011 17:51:07 +0200
Message-ID: <m3pqln76hg.fsf@HIDDEN>
References: <m11vthqpn9.fsf@HIDDEN> <m3liwk86q9.fsf@HIDDEN>
	<jwvbox7tpng.fsf-monnier+emacs@HIDDEN>
User-Agent: Gnus/5.110018 (No Gnus v0.18) Emacs/24.0.50 (gnu/linux)
X-Now-Playing: Various's _Delta Swamp Rock (2)_: "Dan Penn - If Love Was Money"
X-Hashcash: 1:23:110706:895 <at> debbugs.gnu.org::pibxMAGxl2BBw0gs:0000000000000000000000000000000000000000004DKy
X-Hashcash: 1:23:110706:monnier@HIDDEN::LQSGtZ+DbdD4tWGM:0000000000000000000000000000000000000nx5c
MIME-Version: 1.0
Content-Type: text/plain
X-MailScanner-ID: 1QeUTs-00067B-CB
X-Netfonds-MailScanner: Found to be clean
X-Netfonds-MailScanner-From: larsi@HIDDEN
MailScanner-NULL-Check: 1310572660.45649@wI+7ZjIT3clFIz2BuHE1EA
X-Spam-Status: No
X-Spam-Score: -2.8 (--)
X-Debbugs-Envelope-To: 895
Cc: 895 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.8 (--)

Stefan Monnier <monnier@HIDDEN> writes:

>> I've now bound `process-connection-type' to nil in `rgrep' for Emacs 24.
>
> What makes it faster?
> Could it be that with process-connection-type set to nil grep doesn't emit
> the color escape sequences?

Yes.

I've now reverted the change and reopened this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.
Did not alter fixed versions and reopened. Request was from Debbugs Internal Request <help-debbugs@HIDDEN> to internal_control <at> debbugs.gnu.org. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 6 Jul 2011 15:33:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 06 11:33:07 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QeU66-0001nJ-Rn
	for submit <at> debbugs.gnu.org; Wed, 06 Jul 2011 11:33:07 -0400
Received: from fencepost.gnu.org ([140.186.70.10])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <dann@HIDDEN>) id 1QeU64-0001mn-OI
	for 895 <at> debbugs.gnu.org; Wed, 06 Jul 2011 11:33:05 -0400
Received: from dann by fencepost.gnu.org with local (Exim 4.71)
	(envelope-from <dann@HIDDEN>)
	id 1QeU5x-0003tk-F7; Wed, 06 Jul 2011 11:32:57 -0400
From: Dan Nicolaescu <dann@HIDDEN>
To: 895 <at> debbugs.gnu.org
Subject: Re: bug#895: slow processing of process output
References: <m11vthqpn9.fsf@HIDDEN> <m3liwk86q9.fsf@HIDDEN>
X-Debbugs-No-Ack: yes
Date: Wed, 06 Jul 2011 11:32:57 -0400
In-Reply-To: <m3liwk86q9.fsf@HIDDEN> (Lars Magne Ingebrigtsen's
	message of "Thu, 30 Jun 2011 03:10:06 +0200")
Message-ID: <yxqaacrmnkm.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Spam-Score: -6.5 (------)
X-Debbugs-Envelope-To: 895
Cc: larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -6.5 (------)

Lars Magne Ingebrigtsen <larsi@HIDDEN> writes:

> Markus Triska <markus.triska@HIDDEN> writes:
>
>> I can reproduce the problem on OSX, and could significantly improve the
>> running time by setting process-connection-type to nil before doing the
>> rgrep. Let rgrep.el consist of:
>>
>>    (grep-compute-defaults)
>>    (setq process-connection-type nil)
>>    (rgrep "emacs" "*.el" "~/emacs/lisp/")
>>
>>    (while (process-status "grep")
>>      (sit-for 0.1))
>
> I've now bound `process-connection-type' to nil in `rgrep' for Emacs 24.

This might solve the specific `rgrep' case, but that was only a specific
easy to reproduce example.
Processing process output is slow for other things too.
For example compare the time to run
make bootstrap
when using M-x compile

vs doing it in an xterm.





Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895 <at> debbugs.gnu.org:


Received: (at 895) by debbugs.gnu.org; 6 Jul 2011 15:07:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jul 06 11:07:37 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QeThQ-0001Bc-RJ
	for submit <at> debbugs.gnu.org; Wed, 06 Jul 2011 11:07:37 -0400
Received: from ironport2-out.teksavvy.com ([206.248.154.183]
	helo=ironport2-out.pppoe.ca)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <monnier@HIDDEN>) id 1QeThO-0001BL-W9
	for 895 <at> debbugs.gnu.org; Wed, 06 Jul 2011 11:07:35 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AhAIAAF5FE5FxIV4/2dsb2JhbABTmRmOa3iIesMqhjcEnmuELw
X-IronPort-AV: E=Sophos;i="4.65,487,1304308800"; d="scan'208";a="120677377"
Received: from 69-196-133-120.dsl.teksavvy.com (HELO ceviche.home)
	([69.196.133.120])
	by ironport2-out.pppoe.ca with ESMTP/TLS/ADH-AES256-SHA;
	06 Jul 2011 11:07:28 -0400
Received: by ceviche.home (Postfix, from userid 20848)
	id 7B64C660D9; Wed,  6 Jul 2011 11:07:13 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: 895 <at> debbugs.gnu.org
Subject: Re: bug#895: slow processing of process output
Message-ID: <jwvbox7tpng.fsf-monnier+emacs@HIDDEN>
References: <m11vthqpn9.fsf@HIDDEN> <m3liwk86q9.fsf@HIDDEN>
Date: Wed, 06 Jul 2011 11:07:13 -0400
In-Reply-To: <m3liwk86q9.fsf@HIDDEN> (Lars Magne Ingebrigtsen's
	message of "Thu, 30 Jun 2011 03:10:06 +0200")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.1 (--)
X-Debbugs-Envelope-To: 895
Cc: larsi@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -2.1 (--)

>> I can reproduce the problem on OSX, and could significantly improve the
>> running time by setting process-connection-type to nil before doing the
>> rgrep. Let rgrep.el consist of:
>> 
>> (grep-compute-defaults)
>> (setq process-connection-type nil)
>> (rgrep "emacs" "*.el" "~/emacs/lisp/")
>> 
>> (while (process-status "grep")
>> (sit-for 0.1))

> I've now bound `process-connection-type' to nil in `rgrep' for Emacs 24.

What makes it faster?
Could it be that with process-connection-type set to nil grep doesn't emit
the color escape sequences?


        Stefan




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#895; Package emacs. Full text available.

Message received at 895-close <at> debbugs.gnu.org:


Received: (at 895-close) by debbugs.gnu.org; 30 Jun 2011 01:10:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jun 29 21:10:26 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1Qc5ly-0000x6-En
	for submit <at> debbugs.gnu.org; Wed, 29 Jun 2011 21:10:26 -0400
Received: from hermes.netfonds.no ([80.91.224.195])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <larsi@HIDDEN>) id 1Qc5lv-0000ws-Ns
	for 895-close <at> debbugs.gnu.org; Wed, 29 Jun 2011 21:10:24 -0400
Received: from cm-84.215.51.58.getinternet.no ([84.215.51.58]
	helo=quimbies.gnus.org)
	by hermes.netfonds.no with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
	(Exim 4.72) (envelope-from <larsi@HIDDEN>)
	id 1Qc5li-0007fT-SC; Thu, 30 Jun 2011 03:10:10 +0200
From: Lars Magne Ingebrigtsen <larsi@HIDDEN>
To: Markus Triska <markus.triska@HIDDEN>
Subject: Re: bug#895: slow processing of process output
In-Reply-To: <m11vthqpn9.fsf@HIDDEN> (Markus Triska's message of "Sun, 01 Mar
	2009 15:33:14 +0100")
Date: Thu, 30 Jun 2011 03:10:06 +0200
Message-ID: <m3liwk86q9.fsf@HIDDEN>
References: <m11vthqpn9.fsf@HIDDEN>
User-Agent: Gnus/5.110018 (No Gnus v0.18) Emacs/24.0.50 (gnu/linux)
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAHlBMVEWm6vtBXblmkfDA/v8H
	BAtUe+EKBxMPDiMMCRYfJ1pH944OAAACZElEQVQ4jXWSwWrjMBCGZ5sXWC2Y4lvxkkNvhT7BgrB9
	XBl86N0oyrEQOvK9SOqtLl2UedudkZM0h90JsTX/p/k1sgSweTzFZVACAP4HLjl8P48eHmFzBb69
	XwD7wMMF3H9c+W+uAZ2H4vIPADfNHT/vf16DB2i24zj+3jzC7fsXGESUGHgNnDelj2ZLS3XSx4aX
	R2SvG8ku8jhUBWxW/RJbM4wruNaNqol+rKA5i6IRKVWNwx2DefUWcVGndQSUVriIFOdNCdkIQHXy
	WFjZ8nsFA0DxXZQScJ8FyC63bFXn+U/TmC8gvZkRhifED86qLQPKJHplqhGaMxi/wCiAFOUVjM0T
	SW8CaoK2RaoYiDKsLbGVIrBa6zSrqi4bkY4U/wwDryetHVLGef3uxqjKZLDBupSssw59Qikzo6qe
	EFic2hSSDykl9HFe2IrIQwoupD70Kb3YlCLyn1T2CYK1Nk52si449vTBttYiIhTvJICtXCi6cwyc
	7XKmOacQkpRIJ9YmhF3Qtg08LfrkbKkTECF5JwMJu9YIcJEXD3ZKxYKXTmuIFU/iCSEwkJ1EUZN0
	xVNFYHPOZjrFomDGEiWt+Yg5RpOXGjjPkp/vm0Q97TPweykp3zSpqGtFoQ0IPLWjxTwTkrmtlFHL
	wWPkxblc61+1DvqtPj7X5PeH+Bl3M1TmqPWe+Bzb+dDRi37LUZoBdcsaIetI2GWtO+wi0Qw1HnWX
	D3LyLc6e3zvs/WsLVH/qNdqDPr7yBYjJ9cEC63wZNBbARkHrfkptC9n3Mx0LEN27oLtJ9xZ2HjGc
	argx+XDxVffxLxcXekNt/DaTAAAAAElFTkSuQmCC
X-Now-Playing: Epic 45's _In All The Empty Houses_: "Ghosts on Tape"
X-Hashcash: 1:23:110630:dann@HIDDEN::0qNOi9Mtu9eG16VD:00j9D
X-Hashcash: 1:23:110630:markus.triska@HIDDEN::VMD1H/77MgQ/L78N:00000000000000000000000000000000000000000K1Gi
X-Hashcash: 1:23:110630:895 <at> debbugs.gnu.org::Z4xCKGmPL+rAVSZO:000000000000000000000000000000000000000000dyqW
MIME-Version: 1.0
Content-Type: text/plain
X-MailScanner-ID: 1Qc5li-0007fT-SC
X-Netfonds-MailScanner: Found to be clean
X-Netfonds-MailScanner-From: larsi@HIDDEN
MailScanner-NULL-Check: 1310001011.05366@t0hkuzpgKubgisFd0QqAtg
X-Spam-Status: No
X-Spam-Score: -3.1 (---)
X-Debbugs-Envelope-To: 895-close
Cc: 895-close <at> debbugs.gnu.org, Dan Nicolaescu <dann@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -3.1 (---)

Markus Triska <markus.triska@HIDDEN> writes:

> I can reproduce the problem on OSX, and could significantly improve the
> running time by setting process-connection-type to nil before doing the
> rgrep. Let rgrep.el consist of:
>
>    (grep-compute-defaults)
>    (setq process-connection-type nil)
>    (rgrep "emacs" "*.el" "~/emacs/lisp/")
>
>    (while (process-status "grep")
>      (sit-for 0.1))

I've now bound `process-connection-type' to nil in `rgrep' for Emacs 24.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/




Notification sent to Dan Nicolaescu <dann@HIDDEN>:
bug acknowledged by developer. Full text available.
Reply sent to Lars Magne Ingebrigtsen <larsi@HIDDEN>:
You have taken responsibility. Full text available.

Message received at 895@HIDDEN:


Received: (at 895) by emacsbugs.donarmstrong.com; 1 Mar 2009 14:33:31 +0000
From markus.triska@HIDDEN Sun Mar  1 06:33:31 2009
X-Spam-Checker-Version: SpamAssassin 3.2.5-bugs.debian.org_2005_01_02
	(2008-06-10) on rzlab.ucr.edu
X-Spam-Level: 
X-Spam-Bayes: score:0.5 Bayes not run. spammytokens:Tokens not available.
	hammytokens:Tokens not available.
X-Spam-Status: No, score=0.0 required=4.0 tests=FVGT_m_MULTI_ODD autolearn=ham
	version=3.2.5-bugs.debian.org_2005_01_02
Received: from mail.gmx.net (mail.gmx.net [213.165.64.20])
	by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with SMTP id n21EXQLA004532
	for <895@HIDDEN>; Sun, 1 Mar 2009 06:33:28 -0800
Received: (qmail invoked by alias); 01 Mar 2009 14:33:20 -0000
Received: from chello062178240212.3.14.tuwien.teleweb.at (EHLO mt-computer.local) [62.178.240.212]
  by mail.gmx.net (mp069) with SMTP; 01 Mar 2009 15:33:20 +0100
X-Authenticated: #4064391
X-Provags-ID: V01U2FsdGVkX1+m4SEYTzCmidDawEt7CR4g81B0TPSYSGcLlI8908
	k4VUNHG03I/AYV
Received: by mt-computer.local (Postfix, from userid 502)
	id 92400C63B3C; Sun,  1 Mar 2009 15:33:14 +0100 (CET)
From: Markus Triska <markus.triska@HIDDEN>
To: 895 <at> debbugs.gnu.org
Cc: Dan Nicolaescu <dann@HIDDEN>
Subject: Re: slow processing of process output
Date: Sun, 01 Mar 2009 15:33:14 +0100
Message-ID: <m11vthqpn9.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Y-GMX-Trusted: 0
X-FuHaFi: 0.72


I can reproduce the problem on OSX, and could significantly improve the
running time by setting process-connection-type to nil before doing the
rgrep. Let rgrep.el consist of:

   (grep-compute-defaults)
   (setq process-connection-type nil)
   (rgrep "emacs" "*.el" "~/emacs/lisp/")

   (while (process-status "grep")
     (sit-for 0.1))

and do:

   $ time emacs -Q --script rgrep.el

Then contrast this with retaining t for process-connection-type.

I have seen that using a pty can be a major performance problem also
when communicating with other external programs, and I recommend to
let-bind it to nil before starting the process when a pipe suffices.




Acknowledgement sent to Markus Triska <markus.triska@HIDDEN>:
Extra info received and forwarded to list. Copy sent to Emacs Bugs <bug-gnu-emacs@HIDDEN>. Full text available.
Information forwarded to bug-submit-list@HIDDEN, Emacs Bugs <bug-gnu-emacs@HIDDEN>:
bug#895; Package emacs. Full text available.

Message received at (unknown):


Received: (at unknown) by unknown; unknown
X-Spam-Checker-Version: SpamAssassin 3.2.3-bugs.debian.org_2005_01_02
	(2007-08-08) on rzlab.ucr.edu
X-Spam-Level: 
X-Spam-Status: No, score=-6.4 required=4.0 tests=AWL,BAYES_00,FOURLA,
	HAS_BUG_NUMBER autolearn=ham version=3.2.3-bugs.debian.org_2005_01_02
Received: (at 895) by emacsbugs.donarmstrong.com; 2 Oct 2008 09:29:35 +0000
Received: from sallyv2.ics.uci.edu (sallyv2.ics.uci.edu [128.195.1.120])
	by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id m929TW3o024226
	for <895@HIDDEN>; Thu, 2 Oct 2008 02:29:33 -0700
Received: from mothra.ics.uci.edu (mothra.ics.uci.edu [128.195.6.93])
	by sallyv2.ics.uci.edu (8.13.7+Sun/8.13.7) with ESMTP id m929TDmJ029441
	for <895@HIDDEN>; Thu, 2 Oct 2008 02:29:13 -0700 (PDT)
Received: (from dann@localhost)
	by mothra.ics.uci.edu (8.13.8+Sun/8.13.6/Submit) id m929TCiH016508;
	Thu, 2 Oct 2008 02:29:12 -0700 (PDT)
Date: Thu, 2 Oct 2008 02:29:12 -0700 (PDT)
Message-Id: <200810020929.m929TCiH016508@HIDDEN>
From: Dan Nicolaescu <dann@HIDDEN>
To: 895 <at> debbugs.gnu.org
Subject: Re: bug#895: slow processing of process output
References: <200809051543.m85FhWHV008307@HIDDEN>
In-Reply-To: <200809051543.m85FhWHV008307@HIDDEN> (Dan
	Nicolaescu's message of "Fri, 05 Sep 2008 08:43:32 -0700")
Lines: 25
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-ICS-MailScanner-Information: Please contact the ISP for more information
X-ICS-MailScanner-ID: m929TDmJ029441
X-ICS-MailScanner: Found to be clean
X-ICS-MailScanner-SpamCheck: not spam, SpamAssassin (score=-1.44, required 5,
	autolearn=disabled, ALL_TRUSTED -1.44)
X-ICS-MailScanner-From: dann@HIDDEN

Dan Nicolaescu <dann@HIDDEN> writes:

  > emacs -Q -nw
  > M-x rgrep RET emacs RET *.el RET PATH_TO_EMACS_SOURCE_TREE/lisp RET
  > 
  > takes a few minutes.  The output is about 9000 lines.
  > 
  > Running the correspondind command:
  > 
  > find . \( -path \*/SCCS -o -path \*/RCS -o -path \*/CVS -o -path \*/MCVS -o -path \*/.svn -o -path \*/.git -o -path \*/.hg -o -path \*/.bzr -o -path \*/_MTN -o -path \*/_darcs -o -path \*/\{arch\} \) -prune -o -type f \( -name *.el \) -print0 | xargs -0 -e grep -i -nH -e emacs
  > 
  > from a shell (and redirecting the output to a file) takes less than one second.

One more data point:

emacs -Q -nw

M-x lgrep RET emacs RET *.el RET PATH_TO_EMACS_SOURCE_TREE/lisp RET

takes 33 seconds.  It runs the command:  grep -i -nH -e emacs *.el

M-x shell RET time grep -i -nH -e emacs *.el
0.087u 0.020s 0:01.60 6.2%      0+0k 0+0io 0pf+0w
              ^^^^^^^
              A lot less than 33 seconds.




Acknowledgement sent to Dan Nicolaescu <dann@HIDDEN>:
Extra info received and forwarded to list. Copy sent to Emacs Bugs <bug-gnu-emacs@HIDDEN>. Full text available.
Information forwarded to bug-submit-list@HIDDEN, Emacs Bugs <bug-gnu-emacs@HIDDEN>:
bug#895; Package emacs. Full text available.

Message received at (unknown):


Received: (at unknown) by unknown; unknown
X-Spam-Checker-Version: SpamAssassin 3.2.3-bugs.debian.org_2005_01_02
	(2007-08-08) on rzlab.ucr.edu
X-Spam-Level: 
X-Spam-Status: No, score=-4.6 required=4.0 tests=AWL,BAYES_00,FOURLA,
	RCVD_IN_DNSWL_LOW autolearn=ham version=3.2.3-bugs.debian.org_2005_01_02
Received: (at submit) by emacsbugs.donarmstrong.com; 5 Sep 2008 15:44:26 +0000
Received: from lists.gnu.org (lists.gnu.org [199.232.76.165])
	by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id m85FiMUd023438
	for <submit@HIDDEN>; Fri, 5 Sep 2008 08:44:23 -0700
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1KbdTy-0005yp-6x
	for bug-gnu-emacs@HIDDEN; Fri, 05 Sep 2008 11:44:22 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1KbdTw-0005yP-F7
	for bug-gnu-emacs@HIDDEN; Fri, 05 Sep 2008 11:44:21 -0400
Received: from [199.232.76.173] (port=35094 helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43)
	id 1KbdTw-0005yM-Ap
	for bug-gnu-emacs@HIDDEN; Fri, 05 Sep 2008 11:44:20 -0400
Received: from sallyv1.ics.uci.edu ([128.195.1.109]:33879)
	by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_3DES_EDE_CBC_SHA1:24)
	(Exim 4.60)
	(envelope-from <dann@HIDDEN>)
	id 1KbdTv-0001pk-8r
	for bug-gnu-emacs@HIDDEN; Fri, 05 Sep 2008 11:44:19 -0400
X-ICS-MailScanner-Watermark: 1221234215.93362@/IJjnuIL3w8yKubib+KrGw
Received: from mothra.ics.uci.edu (mothra.ics.uci.edu [128.195.6.93])
	by sallyv1.ics.uci.edu (8.13.7+Sun/8.13.7) with ESMTP id m85FhWHV008307
	for <bug-gnu-emacs@HIDDEN>; Fri, 5 Sep 2008 08:43:33 -0700 (PDT)
Message-Id: <200809051543.m85FhWHV008307@HIDDEN>
From: Dan Nicolaescu <dann@HIDDEN>
To: bug-gnu-emacs <bug-gnu-emacs@HIDDEN>
Subject: slow processing of process output
Lines: 11
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Date: Fri, 05 Sep 2008 08:43:32 -0700
Sender: dann@HIDDEN
X-ICS-MailScanner: Found to be clean
X-ICS-MailScanner-SpamCheck: not spam, SpamAssassin (score=-1.44, required 5,
	autolearn=disabled, ALL_TRUSTED -1.44)
X-ICS-MailScanner-From: dann@HIDDEN
X-detected-kernel: by monty-python.gnu.org: Solaris 10 (beta)


emacs -Q -nw
M-x rgrep RET emacs RET *.el RET PATH_TO_EMACS_SOURCE_TREE/lisp RET

takes a few minutes.  The output is about 9000 lines.

Running the correspondind command:

find . \( -path \*/SCCS -o -path \*/RCS -o -path \*/CVS -o -path \*/MCVS -o -path \*/.svn -o -path \*/.git -o -path \*/.hg -o -path \*/.bzr -o -path \*/_MTN -o -path \*/_darcs -o -path \*/\{arch\} \) -prune -o -type f \( -name *.el \) -print0 | xargs -0 -e grep -i -nH -e emacs

from a shell (and redirecting the output to a file) takes less than one second.





Acknowledgement sent to Dan Nicolaescu <dann@HIDDEN>:
New bug report received and forwarded. Copy sent to Emacs Bugs <bug-gnu-emacs@HIDDEN>. Full text available.
Report forwarded to bug-submit-list@HIDDEN, Emacs Bugs <bug-gnu-emacs@HIDDEN>:
bug#895; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.