Received: (at 60506) by debbugs.gnu.org; 3 Jan 2023 22:33:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 03 17:33:28 2023
Received: from localhost ([127.0.0.1]:47011 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pCpqa-0001yV-Ex
for submit <at> debbugs.gnu.org; Tue, 03 Jan 2023 17:33:28 -0500
Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:38735)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pj@HIDDEN>) id 1pCpqX-0001yI-PH
for 60506 <at> debbugs.gnu.org; Tue, 03 Jan 2023 17:33:26 -0500
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
by mailout.west.internal (Postfix) with ESMTP id 8A9833200945;
Tue, 3 Jan 2023 17:33:19 -0500 (EST)
Received: from imap51 ([10.202.2.101])
by compute6.internal (MEProxy); Tue, 03 Jan 2023 17:33:19 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:content-type:date:date:feedback-id
:feedback-id:from:from:in-reply-to:in-reply-to:message-id
:mime-version:references:reply-to:sender:subject:subject:to:to
:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
fm2; t=1672785199; x=1672871599; bh=qIhnPwU5g44s+774c2EkmlFgQDeb
eVuQ86+T0JVEh9k=; b=g7/YG8j/hyIz/xztkykoVeDytKTGMsCRg3RJXEqnDICC
MiTE7nuy9FjtfYDQSSqLmr+0xl4iUkrNVqRmGBVDu3ENaWJUibCrIj/XkzK7nY1I
pcvBWmRljTU115Yq2KC9VDsKsAh+lyHDHaGZrJIpSfQxt63GBBQrO+mRAXm7825+
kTrynCD2bb2w+pKJhLsuut2mKPg0jWzZTYWpwaJM5hMiZX7L3n8FoXXbl3rq7gPQ
quqMtDiEsHTYBleoCWnIRioRN4domgWKLslOMVw9aV805Xlxdt8XWK2p0pW7KAUS
+oeUNsUV8tYt9YGmzfmgZALPqmGH6TTvNTowVwK7mQ==
X-ME-Sender: <xms:Lq20Y43glVr3piRv4WVOUY47PSf1XAybzZfNwmbS6gQrlA8Na2eGrQ>
<xme:Lq20YzFmgclY_sYbxao_fBRQHzajMqdvjvIzfCj7DbgHpuVR29b8yJLUKuEFUVu5V
Bft-EKSZfVC>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrjeeggdduiedvucetufdoteggodetrfdotf
fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
cujfgurhepofgfggfkjghffffhvffutgesrgdtreerreertdenucfhrhhomhepfdfrrghu
lhculfgrtghkshhonhdfuceophhjsehushgrrdhnvghtqeenucggtffrrghtthgvrhhnpe
ejteetudfhffdvgeekieeiteefvdefteejteduvdejtefhueekgfeludeuleelieenucff
ohhmrghinhepthhhvghphihthhhonhhitggtohifrdhushenucevlhhushhtvghrufhiii
gvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpjhesuhhsrgdrnhgvth
X-ME-Proxy: <xmx:Lq20Yw7hdLjS_26u5eFKAoZ8kLtqyJYCUcWLI0kPBdIidVg_GPoMNQ>
<xmx:Lq20Yx3TwW1qyKYJvnUJhjy4lr2f-LLXen-pWwNZHWLNlJx7wPcRog>
<xmx:Lq20Y7G1MuCw-rXBs6OrYuGh1hbsNMYprClzxWL_WB2cVxnoW3uooA>
<xmx:L620Y-PRK8DD0CClum1ljrkYKE4bbZzjumi_0daN4ibF4jyR4oDalQ>
Feedback-ID: i047841af:Fastmail
Received: by mailuser.nyi.internal (Postfix, from userid 501)
id B81EEB60086; Tue, 3 Jan 2023 17:33:18 -0500 (EST)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.7.0-alpha0-1185-g841157300a-fm-20221208.002-g84115730
Mime-Version: 1.0
Message-Id: <eaa1bb2a-bb8b-430c-8971-0fae539ce5ac@HIDDEN>
In-Reply-To: <786328389.5641240.1672780518915@HIDDEN>
References: <CAPuuP=7KCvt2-zgJAdY8CSEAmuubnbpnmznqEfbArf-LaO2q8w@HIDDEN>
<04d86085-a044-4b9b-8451-b6e0c3586bb3@HIDDEN>
<31e8cb06-606f-ea05-7d99-08e0311920a5@HIDDEN>
<68afdacb-e8db-41c4-a3bd-1ce5ddd185ac@HIDDEN>
<786328389.5641240.1672780518915@HIDDEN>
Date: Tue, 03 Jan 2023 16:32:01 -0600
From: "Paul Jackson" <pj@HIDDEN>
To: "David G. Pickett" <dgpickett@HIDDEN>,
"eggert@HIDDEN" <eggert@HIDDEN>,
"60506 <at> debbugs.gnu.org" <60506 <at> debbugs.gnu.org>
Subject: Re: bug#60506: feature: parallel grep --recursive
Content-Type: multipart/alternative; boundary=3e9d5f89839d42a281a1fa27514752b9
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 60506
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--3e9d5f89839d42a281a1fa27514752b9
Content-Type: text/plain
David Pickett wrote:
<< I also wrote a simpler, line oriented, faster xargs, fxargs! >>
I've been quite pleased with an xargs wrapper I wrote that basically
converts newlines to nuls, and then invokes either "xargs" or, if asked
to run multiple threads, "parallel --xargs", passing all the "xargs" arguments
to "xargs --null".
I got all the exit status's and such just right, and preferred having all the
xargs options available, once this hack worked around the confused
space character handling of xargs without the --null option.
I call my wrapper "x", a short name since I use it a lot, having been a regular
xargs user since it was added to Version 7 Unix, inside Bell Labs, back around
1978.
You can find my wrapper at:
http://thepythoniccow.us/x.c
By the way, even the original author of xargs, Herb Gellis, agrees that its
interface is somewhat borked. See a note Gellis posted a decade after writing
xargs, which I include in the above "x.c" source. An amusing bit of history ...
--
Paul Jackson
pj@HIDDEN
--3e9d5f89839d42a281a1fa27514752b9
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}
p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div style=3D"f=
ont-family:Arial;">David Pickett wrote:</div><div style=3D"font-family:A=
rial;"><div><< I also wrote a simpler, line oriented, faster xargs=
, fxargs! >><br></div><div><br></div><div>I've been quite pl=
eased with an xargs wrapper I wrote that basically<br></div><div style=3D=
"font-family:Arial;">converts newlines to nuls, and then invokes either =
"xargs" or, if asked<br></div><div style=3D"font-family:Arial;">to run m=
ultiple threads, "parallel --xargs", passing all the "xargs" arguments<b=
r></div><div style=3D"font-family:Arial;">to "xargs --null".<br><br>I go=
t all the exit status's and such just right, and preferred having all th=
e</div><div style=3D"font-family:Arial;">xargs options available, once t=
his hack worked around the confused<br></div><div style=3D"font-family:A=
rial;">space character handling of xargs without the --null option.<br><=
/div><div style=3D"font-family:Arial;"><br>I call my wrapper "x", a shor=
t name since I use it a lot, having been a regular</div><div style=
=3D"font-family:Arial;">xargs user since it was added to Version 7 Unix,=
inside Bell Labs, back around<br></div><div style=3D"font-family:Arial;=
"><div>1978.<br><br>You can find my wrapper at:</div><div><br></div><div=
><a href=3D"http://thepythoniccow.us/x.c" style=3D"outline-width:0px !im=
portant;user-select:auto !important;">http://thepythoniccow.us/x.c</a><b=
r></div><div><br></div><div>By the way, even the original author of xarg=
s, Herb Gellis, agrees that its<br></div><div style=3D"font-family:=
Arial;">interface is somewhat borked. See a note Gellis posted a d=
ecade after writing<br></div><div style=3D"font-family:Arial;">xargs, wh=
ich I include in the above "x.c" source. An amusing bit of history=
...<br><br></div></div></div><div id=3D"sig78241"><pre class=3D"signatu=
re">--=20
Paul Jackson
pj@HIDDEN
<br></pre></div><div style=3D"font-family:Arial;"><br></div></body></htm=
l>
--3e9d5f89839d42a281a1fa27514752b9--
bug-grep@HIDDEN:bug#60506; Package grep.
Full text available.
Received: (at 60506) by debbugs.gnu.org; 3 Jan 2023 02:59:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 02 21:59:25 2023
Received: from localhost ([127.0.0.1]:44426 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pCXWP-0004TW-0i
for submit <at> debbugs.gnu.org; Mon, 02 Jan 2023 21:59:25 -0500
Received: from out4-smtp.messagingengine.com ([66.111.4.28]:58123)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pj@HIDDEN>) id 1pCXWN-0004TK-My
for 60506 <at> debbugs.gnu.org; Mon, 02 Jan 2023 21:59:24 -0500
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
by mailout.nyi.internal (Postfix) with ESMTP id 5237E5C0103;
Mon, 2 Jan 2023 21:59:18 -0500 (EST)
Received: from imap51 ([10.202.2.101])
by compute6.internal (MEProxy); Mon, 02 Jan 2023 21:59:18 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:content-type:date:date:feedback-id
:feedback-id:from:from:in-reply-to:in-reply-to:message-id
:mime-version:references:reply-to:sender:subject:subject:to:to
:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
fm2; t=1672714758; x=1672801158; bh=YreBbwxX1LGnJ9S7cKaJWSqM1fvm
jLb+hcQ9OMtwFM0=; b=t67jRLJbvuRycBG7myxtiFv7MU+LF/5YQ1TVRW4as6xg
NDsJpMTk9qBTJ3gREFZ8VTlFFA3oRqs47mmcJBbSGmbLt3UDokjx0VsaDOcLAk0K
u8K8qkF7+oCaYSy9I+waEwc41JIag4/tMjr1oXQZJ/w+zs6pYrcQB032e+NtUFHi
nIRDuTx7qO/buqlY5/N4veRIGhLKFT/NqTu+N4orIM2oc+MN4lT5CELofj7sweeN
kn3oxXPk6s0y0rRyB4vYGsn6kFxiHG+7lT1o7jXO92fjNeMRx7oNEZfkWnVsjczw
jPqNhNmCUUhlvXcratV8TU1m9TvqjU9mODZWTAaE/w==
X-ME-Sender: <xms:BpqzYzLUuLNRkdlpVKNfwrWZfjP25LMWeCC-xmNq6AAYj2BGllNIFg>
<xme:BpqzY3JhZda02riB1v0y_mq3Of0qmI7mf8h89xncgNWRUU2p7rEpgXnChPaO2n173
h2RS7kjx6hA>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrjeefgdehhecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
fjughrpefofgggkfgjfhffhffvufgtsehttdertderredtnecuhfhrohhmpedfrfgruhhl
ucflrggtkhhsohhnfdcuoehpjhesuhhsrgdrnhgvtheqnecuggftrfgrthhtvghrnhephf
evtdehjeethfegjedtiefgffdtlefgvdekueefheejkeevveehudffgfduleejnecuvehl
uhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhjsehushgrrd
hnvght
X-ME-Proxy: <xmx:BpqzY7tBLwed7aSr11dhGTBWHaFKf5rWecw7N208dssnM9ytPNriOw>
<xmx:BpqzY8ZqdIsnAkvc5DV-rwTY8u95arDqi5-veYK5WyyQVn8-fg3psQ>
<xmx:BpqzY6YeS1fZR9XvRswc29VljJtyo4mu2haClGFy90snSTJ5P5VdKw>
<xmx:BpqzY42Q1e0SawA6qtuj3S99GoZzd-6Nsq3y3VnwukWM8u7LBerS_A>
Feedback-ID: i047841af:Fastmail
Received: by mailuser.nyi.internal (Postfix, from userid 501)
id 2932BB60086; Mon, 2 Jan 2023 21:59:18 -0500 (EST)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.7.0-alpha0-1185-g841157300a-fm-20221208.002-g84115730
Mime-Version: 1.0
Message-Id: <68afdacb-e8db-41c4-a3bd-1ce5ddd185ac@HIDDEN>
In-Reply-To: <31e8cb06-606f-ea05-7d99-08e0311920a5@HIDDEN>
References: <CAPuuP=7KCvt2-zgJAdY8CSEAmuubnbpnmznqEfbArf-LaO2q8w@HIDDEN>
<04d86085-a044-4b9b-8451-b6e0c3586bb3@HIDDEN>
<31e8cb06-606f-ea05-7d99-08e0311920a5@HIDDEN>
Date: Mon, 02 Jan 2023 20:56:23 -0600
From: "Paul Jackson" <pj@HIDDEN>
To: "Paul Eggert" <eggert@HIDDEN>, 60506 <at> debbugs.gnu.org
Subject: Re: bug#60506: feature: parallel grep --recursive
Content-Type: text/plain
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 60506
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
<< a parallel grep to search a single large file >>
I'm but one user, and a rather idiosyncratic user at that,
but for my usage patterns, the specialized logic that it
would take to run a parallelized grep on a large file
would likely not shrink the elapsed time enough to justify
the coding, documentation, and maintenance effort.
I would expect the time to read the large file in from disk to
dominate the total elapsed time in any case.
(or maybe I am just jealous that I didn't think of that parallel
grep use case myself <grin>.)
--
Paul Jackson
pj@HIDDEN
bug-grep@HIDDEN:bug#60506; Package grep.
Full text available.Received: (at 60506) by debbugs.gnu.org; 3 Jan 2023 02:49:04 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 02 21:49:04 2023 Received: from localhost ([127.0.0.1]:44407 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pCXMN-0004Dv-Pk for submit <at> debbugs.gnu.org; Mon, 02 Jan 2023 21:49:04 -0500 Received: from zimbra.cs.ucla.edu ([131.179.128.68]:57358) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eggert@HIDDEN>) id 1pCXMI-0004DK-5L for 60506 <at> debbugs.gnu.org; Mon, 02 Jan 2023 21:49:02 -0500 Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id DE8F2160044; Mon, 2 Jan 2023 18:48:50 -0800 (PST) Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id yQj8tRKMvcRo; Mon, 2 Jan 2023 18:48:50 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.cs.ucla.edu (Postfix) with ESMTP id 3776C160045; Mon, 2 Jan 2023 18:48:50 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.9.2 zimbra.cs.ucla.edu 3776C160045 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.ucla.edu; s=78364E5A-2AF3-11ED-87FA-8298ECA2D365; t=1672714130; bh=a3Anb5xxlFFoRHMTQAYDQvA8yY+tyHdXS9UeQM35mRY=; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type: Content-Transfer-Encoding; b=Ed4DuX8cR6YmF2ul/xclcOqyRW/trGujR68XcUD56lG9nh8CneFV8GTl+ez6SRwIG +Y3m12AxQNh4v0nR7IfCHkmqHnybmJHiQoMnpjmLpR9n5u46OECCqBEMCCAUDnttAq /Yb8j45JZJQ2cN0frIwg3W6CNPQn4mKASg1+9Ecw= X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu Received: from zimbra.cs.ucla.edu ([127.0.0.1]) by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id hjzqh-yIerpF; Mon, 2 Jan 2023 18:48:50 -0800 (PST) Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com [172.91.119.151]) by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 0C84C160044; Mon, 2 Jan 2023 18:48:50 -0800 (PST) Message-ID: <31e8cb06-606f-ea05-7d99-08e0311920a5@HIDDEN> Date: Mon, 2 Jan 2023 18:48:49 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: bug#60506: feature: parallel grep --recursive Content-Language: en-US To: Paul Jackson <pj@HIDDEN>, 60506 <at> debbugs.gnu.org References: <CAPuuP=7KCvt2-zgJAdY8CSEAmuubnbpnmznqEfbArf-LaO2q8w@HIDDEN> <04d86085-a044-4b9b-8451-b6e0c3586bb3@HIDDEN> From: Paul Eggert <eggert@HIDDEN> Organization: UCLA Computer Science Department In-Reply-To: <04d86085-a044-4b9b-8451-b6e0c3586bb3@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -3.4 (---) X-Debbugs-Envelope-To: 60506 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -4.4 (----) On 2023-01-02 18:34, Paul Jackson wrote: > There's no need for special logic in grep to run parallel grep's. There might be, if one wants to use a parallel grep to search a single large file.
bug-grep@HIDDEN:bug#60506; Package grep.
Full text available.
Received: (at submit) by debbugs.gnu.org; 3 Jan 2023 02:35:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 02 21:35:10 2023
Received: from localhost ([127.0.0.1]:44403 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1pCX8w-0003qC-BI
for submit <at> debbugs.gnu.org; Mon, 02 Jan 2023 21:35:10 -0500
Received: from lists.gnu.org ([209.51.188.17]:33340)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <pj@HIDDEN>) id 1pCX8u-0003q4-3E
for submit <at> debbugs.gnu.org; Mon, 02 Jan 2023 21:35:08 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <pj@HIDDEN>) id 1pCX8q-00062n-AX
for bug-grep@HIDDEN; Mon, 02 Jan 2023 21:35:07 -0500
Received: from out1-smtp.messagingengine.com ([66.111.4.25])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <pj@HIDDEN>) id 1pCX8o-0007bm-8a
for bug-grep@HIDDEN; Mon, 02 Jan 2023 21:35:03 -0500
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47])
by mailout.nyi.internal (Postfix) with ESMTP id 43F4E5C00B3
for <bug-grep@HIDDEN>; Mon, 2 Jan 2023 21:35:00 -0500 (EST)
Received: from imap51 ([10.202.2.101])
by compute6.internal (MEProxy); Mon, 02 Jan 2023 21:35:00 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:content-type:date:date:feedback-id
:feedback-id:from:from:in-reply-to:in-reply-to:message-id
:mime-version:references:reply-to:sender:subject:subject:to:to
:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=
fm2; t=1672713300; x=1672799700; bh=oPNbA7dbSc8BH6V1Q+cCvZOMRR6T
B3tJE0hao5Hqpdw=; b=j05cZ5E8qtyZiGZTduqTsO/iwGdtjdoRmfzFczvRHptf
bbdQw2o4vQ0RCPEYLuM3b9Jmv62UpbIC/DQiufybJjpv9aYjaUqf4Ccf8kIg15B4
VV4sl12TK6gNr/G78wPCvAT5PBKWUaT6zH735GtRwSBPE7b4HDZ252qgaJQf7Y3u
JBdScSXwNSN7EKX5uKFgeqZw1jvfa36TioPkTUpwfAgOuOowBaISy2KSMDqyYHJR
tq1pyVPsj4hc29f6lLqB3WP1RikkVOCNvJj4Bcti7gbQKLX9NrC+YAube/chz2Er
y6d2XBefjf+8vw5aRehPihePVqoHMouEUZQjYpRe1Q==
X-ME-Sender: <xms:U5SzY2DPL4AUhfz8hzz0rmcerklhBTNFj3gu0tgDYm0Pk8aeGu9oag>
<xme:U5SzYwioIT3sjlfUI1zzRTToWlPntMTJkqFpgSaAeV-mT_hA3Oa8iWZ8lid214vCt
YUEpIX2loDe>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrjeefgdehtdcutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkjghffffhvffutgesthdtre
dtreertdenucfhrhhomhepfdfrrghulhculfgrtghkshhonhdfuceophhjsehushgrrdhn
vghtqeenucggtffrrghtthgvrhhnpefhvedtheejtefhgeejtdeigffftdelgfdvkeeufe
ehjeekveevhedufffgudeljeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhep
mhgrihhlfhhrohhmpehpjhesuhhsrgdrnhgvth
X-ME-Proxy: <xmx:U5SzY5kTDBMvvHHWisChbMmoTQshs_PLOS-ls1dhJukDI22o8FLWNg>
<xmx:U5SzY0w6r_nqbe359Cv3ktdWwDTm-9X-8bESU4MYlKy-SbMOn3Uutg>
<xmx:U5SzY7TeT6C0DQyg7Cwi70SMPC57FCxGsFjl2LQuMOsY8SC1z8894A>
<xmx:VJSzYyfGJl2pzUlT65WZ7PD5QwEdUiX2OGtJYyTqOL_MVOUKAUDfnw>
Feedback-ID: i047841af:Fastmail
Received: by mailuser.nyi.internal (Postfix, from userid 501)
id CBF30B60086; Mon, 2 Jan 2023 21:34:59 -0500 (EST)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.7.0-alpha0-1185-g841157300a-fm-20221208.002-g84115730
Mime-Version: 1.0
Message-Id: <04d86085-a044-4b9b-8451-b6e0c3586bb3@HIDDEN>
In-Reply-To: <CAPuuP=7KCvt2-zgJAdY8CSEAmuubnbpnmznqEfbArf-LaO2q8w@HIDDEN>
References: <CAPuuP=7KCvt2-zgJAdY8CSEAmuubnbpnmznqEfbArf-LaO2q8w@HIDDEN>
Date: Mon, 02 Jan 2023 20:34:39 -0600
From: "Paul Jackson" <pj@HIDDEN>
To: bug-grep@HIDDEN
Subject: Re: bug#60506: feature: parallel grep --recursive
Content-Type: text/plain
Received-SPF: neutral client-ip=66.111.4.25; envelope-from=pj@HIDDEN;
helo=out1-smtp.messagingengine.com
X-Spam_score_int: -17
X-Spam_score: -1.8
X-Spam_bar: -
X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001,
RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001,
SPF_NEUTRAL=0.779 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.6 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)
There's no need for special logic in grep to run parallel grep's.
The "parallel" command can handle that for you.
For example, on the 12 core, 24 thread Ryzen CPU that I am using:
find $HOME -xdev -type f -ctime -333 | wc -l ## counts 136126 files.
find $HOME -xdev -type f -ctime -333 |
parallel -m grep -l foobar | wc -l ## takes about 13 seconds
find $HOME -xdev -type f -ctime -333 |
xargs -d '\n' grep -l foobar | wc -l ## takes about 52 seconds
The above parallel invocation ran 24 grep commands in parallel, and took
about 1/4 the time, otherwise performing rather like xargs, which ran one grep
command at a time.
(Granted, reading either the 'parallel' or 'xargs' man pages is not easy <grin>.)
--
Paul Jackson
pj@HIDDEN
bug-grep@HIDDEN:bug#60506; Package grep.
Full text available.Received: (at submit) by debbugs.gnu.org; 3 Jan 2023 00:21:18 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jan 02 19:21:18 2023 Received: from localhost ([127.0.0.1]:44339 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1pCV3N-0000QW-Dq for submit <at> debbugs.gnu.org; Mon, 02 Jan 2023 19:21:18 -0500 Received: from lists.gnu.org ([209.51.188.17]:37968) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <foonlyboy@HIDDEN>) id 1pCRkV-0003O3-A5 for submit <at> debbugs.gnu.org; Mon, 02 Jan 2023 15:49:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <foonlyboy@HIDDEN>) id 1pCRkN-0001Dv-GY for bug-grep@HIDDEN; Mon, 02 Jan 2023 15:49:28 -0500 Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from <foonlyboy@HIDDEN>) id 1pCRkL-0006ss-Sl for bug-grep@HIDDEN; Mon, 02 Jan 2023 15:49:27 -0500 Received: by mail-pj1-x102d.google.com with SMTP id 60-20020a17090a0fc200b002264ebad204so7528461pjz.1 for <bug-grep@HIDDEN>; Mon, 02 Jan 2023 12:49:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=iGsFLjN3sBQM/cCbEVpMF+Yjthf1zgCYgzgj6dGJPsQ=; b=Ez7mx8a6G5hbD2qAxXdVC61TUHVZQBRly2s2qBZrZb6DTmh3YFzOTvj5JLWTtrTotX KgQbGn7bU10I6CzvRUSax4aAbtjFEe0SVvKTxjZU3j2MW46xhKJ3qbIFmr8OpDRm7L20 mZmyDpq45vRtCfZuYD8xZvHJMJ7ViMLWyTloOeEKsdePzCROEjIKAweOlcrYRKjW2ISE iVMiaYR0cbSsegaa8Zosd1EPqrU/KDTnEWha+qxoJoxm3UeGo0yRAdB/TwnIPuPS7t6s P/8U/VDhG7B/rgQQeaHoDqCt2hhwUmxL7jB0VTnp/EqHXU4bXDnjqU7Q0zHMeITNJmy9 5ffg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=iGsFLjN3sBQM/cCbEVpMF+Yjthf1zgCYgzgj6dGJPsQ=; b=Ck8FW46wvQdsDgastJeFEz+fhQvtwHOXPXjW6PT+48zRMauaooJKRn9MpZH14mIJKh 3rCrYVSp6XehNcCI5BKAF4BNwbWsXK4EL5BIWhhsQKGW9J7cVhFLSKMb5GYDtMTaF1v2 sXZXRejq6Dqt1Bq5Ab7zJJbVLeLU9RROi7m6zqzWb3+VzPZpwj1vaAvIicjopAUeRLRK /Bu1Od4th7ms3SYZIf0J1bQwM7y1YMH2Ag4FxCa86ALTaeM0UJz2/17gUr+Ag8TTtqVH FgRk9e4Bd/Y0T3TH1eGIiThrqJBwt9quyG64PduuEBnuEoNyGm/itodHHHFlM+6Fq6uN 74Aw== X-Gm-Message-State: AFqh2kroBDo3UCtPw2XITyLUtzpb2W3myL4ROIdt7Vk3PcQdhZrmei0q wrHgB+qDGe2s7zVtzU13ObyvJn6mI3DkkPDq0yfgZsoWJfo= X-Google-Smtp-Source: AMrXdXtsqiUck7P9LxMZO67tu/h6209D4R4WxwfprNkUStLTf3CmVVvnZPENCAY/dJpk2UTKZiUmKBx5LWqzh4LEZow= X-Received: by 2002:a17:90a:fe93:b0:223:4a7d:878d with SMTP id co19-20020a17090afe9300b002234a7d878dmr3898903pjb.29.1672692563881; Mon, 02 Jan 2023 12:49:23 -0800 (PST) MIME-Version: 1.0 From: Eike Dierks <foonlyboy@HIDDEN> Date: Mon, 2 Jan 2023 21:49:13 +0100 Message-ID: <CAPuuP=7KCvt2-zgJAdY8CSEAmuubnbpnmznqEfbArf-LaO2q8w@HIDDEN> Subject: feature: parallel grep --recursive To: bug-grep@HIDDEN Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::102d; envelope-from=foonlyboy@HIDDEN; helo=mail-pj1-x102d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Mon, 02 Jan 2023 19:21:14 -0500 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.3 (--) Hi at the gnu grep development team I'd like to suggest a new feature for: grep --recursive The grep --recursive should work in parallel. Rational: This could speed up the grep by the numbers of threads Currently, the --recursive option works on every file in sequence. Instead, I want to start some greps in parallel. If we want to be good, then we would parse the expression first (which might be expensive) and then fork on the files. The master grep process would then collect the results, so that the results would be serialized to be identical with the current implementation. I'd like to suggest a --fast option, where results show up, as soon as they are found. .... I am fed up with all that precomputed indexes. I want to grep it really fast now. I expect that the file access is fast now, but has latency. I want the grep to saturate the machine. // job card .
Eike Dierks <foonlyboy@HIDDEN>:bug-grep@HIDDEN.
Full text available.bug-grep@HIDDEN:bug#60506; Package grep.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.