GNU bug report logs - #46351
28.0.50; Add convenient way to bypass Eshell's own pipelining

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Sean Whitton <spwhitton@HIDDEN>; dated Sat, 6 Feb 2021 20:07:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 46351 <at> debbugs.gnu.org:


Received: (at 46351) by debbugs.gnu.org; 7 Feb 2021 19:01:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Feb 07 14:01:14 2021
Received: from localhost ([127.0.0.1]:48805 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l8pJ3-0006qa-V2
	for submit <at> debbugs.gnu.org; Sun, 07 Feb 2021 14:01:14 -0500
Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:33571)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <spwhitton@HIDDEN>) id 1l8pJ1-0006qM-7F
 for 46351 <at> debbugs.gnu.org; Sun, 07 Feb 2021 14:01:12 -0500
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailout.west.internal (Postfix) with ESMTP id F1B1BBEC;
 Sun,  7 Feb 2021 14:01:04 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute4.internal (MEProxy); Sun, 07 Feb 2021 14:01:05 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spwhitton.name;
 h=from:to:cc:subject:in-reply-to:references:date:message-id
 :mime-version:content-type:content-transfer-encoding; s=fm2; bh=
 l4TucaqnSeQL34nI/CzS5SIgXJyUcBefPZGO3Ierqes=; b=Hy/T9ZzxNCMBsAoE
 JYNZnjCW7VU3HXFqdwjAhkj/ctu4ePAPza0Iep9RwO14VGn5w68+oieStpH7RQXh
 VdfJ2yWzulSEE5fIN3FRARvF+Gj4WDQsNS1W9KW+ESZeAjawY58JYZOwc4klK7Ac
 NYggbYRvlxyIF3pqpMCgD2fAfBk/08dwdr5KyPCL59pMjCX8lENkzc2iJQnbrGUM
 K89oLRqDLT4uRTqXB7KxLXMHUqCc88mCNASaak5tuA0leik9HQwoV0tDRRORnBA/
 JlcHhTrY5Bwz/zaWRYJG/nLlgIXZcjp+Tucu51aOqJ9UsOP83S0PPnXIGdkeweyO
 Y0oCgQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:content-transfer-encoding:content-type
 :date:from:in-reply-to:message-id:mime-version:references
 :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
 :x-sasl-enc; s=fm2; bh=l4TucaqnSeQL34nI/CzS5SIgXJyUcBefPZGO3Ierq
 es=; b=hWNeVHeY3rlw1Ywy3lgIDxLxfOqZ1tNvT3hhj8hpFqzkH/e8XwGDfaDUM
 KBUR4MwtatJOggL7/Pet1sMHh9+v0/x5AbfiUBjF0mlWQ1hORSdP9oYmJhE+WAof
 lc3p9JOvP9SHRcbtFoBqG5yoS5XB5k1NZCFB69HFGT9J1ARSFifkYtG/TkqapeBp
 yDVudeTgqCyyZXM2EJFsbDzEcMdAbx+X1qHfy8Db7XBkeaotlucOlIExLMTE1CjP
 eOwB2wqxT74ihUA9CIa1Ca11wsWY7mYFs1zC3an7yNQ7IRlFIjN1fYRtkquY+f3D
 EW43lzYBhK8ZWcO7JaEvbc4R5Lz8A==
X-ME-Sender: <xms:8DggYMOxOsyjsXqXCXqgQt8aPuNiu39gfflBK_q8pjwlXdCJ3xVfWA>
 <xme:8DggYAzwloMqyiX1hqA4WpBm2urqmMPsB1QBfkX3OOtyGPsnc81iAqk9P1xJmy5bo
 7rLXOmlnTBHq6wXlw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrhedugdduvddtucetufdoteggodetrfdotf
 fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
 uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
 cujfgurhephffvufgjfhffkfggtgfgsehtqhertddttdejnecuhfhrohhmpefuvggrnhcu
 hghhihhtthhonhcuoehsphifhhhithhtohhnsehsphifhhhithhtohhnrdhnrghmvgeqne
 cuggftrfgrthhtvghrnhepvdfgkeevtdetvedvhffhgeevleelfeekveeuveehffduvdei
 udfhgeelvefghfehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilh
 hfrhhomhepshhpfihhihhtthhonhesshhpfihhihhtthhonhdrnhgrmhgv
X-ME-Proxy: <xmx:8DggYNsyQAMBmp1c6HrOkz9qfzFLBpbr0YaqHZ61p4VYIT9mEukN6w>
 <xmx:8DggYB63gCoYt-asBcf4EWbmTMs-xkcHfqqB1ZEIAtJsLI-KD7ybng>
 <xmx:8DggYET_5RhFPLGgb8RmTARVjIWK4tiNvBsVG75XHREf-kPIs4HOKA>
 <xmx:8DggYPNlTjc0IxSw1iDYu5i55z2accEYa9b5v7_qU2SkCbgZ_TYZDw>
From: Sean Whitton <spwhitton@HIDDEN>
To: Michael Albinus <michael.albinus@HIDDEN>
Subject: Re: bug#46351: 28.0.50; Add convenient way to bypass Eshell's own
 pipelining
In-Reply-To: <87eehsz170.fsf@HIDDEN>
References: <878s812c6a.fsf@HIDDEN> <87eehsz170.fsf@HIDDEN>
Date: Sun, 07 Feb 2021 12:01:58 -0700
Message-ID: <874kin1z2x.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46351
Cc: 46351 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

On Sun 07 Feb 2021 at 10:17AM +01, Michael Albinus wrote:

> Applying your patch, I get the compiler warning
>
> --8<---------------cut here---------------start------------->8---
>   ELC      eshell/esh-mode.elc
>
> In end of data:
> eshell/esh-mode.el:1114:1: Warning: the function =E2=80=98(setf buffer-su=
bstring)=E2=80=99 is
>     not known to be defined.
> --8<---------------cut here---------------end--------------->8---

Hrm, I can't reproduce this, but looking at the docs, I think that the
problem is a missing (require 'cl-lib).  Would you mind seeing whether
that eliminates the warning at your end?

> And using it in eshell, there is
>
> --8<---------------cut here---------------start------------->8---
> ~/src/emacs $ !! cat ~/.emacs | grep albinus
> !!: command not found
> --8<---------------cut here---------------end--------------->8---

You need to add eshell-expand-to-eshell-shell-command to
eshell-expand-input-functions and (optionally)
eshell-restore-unexpanded-input to eshell-input-filter-functions.  Or
maybe (the first of) these should be added by default?  What do you think?

For testing you can type these forms into an Eshell buffer:

(add-hook 'eshell-expand-input-functions #'eshell-expand-to-eshell-shell-co=
mmand nil t)
(add-hook 'eshell-input-filter-functions #'eshell-restore-unexpanded-input =
nil t)

> I wanted to see, whether this works also for remote directories. Have
> you tested this?

Yes, I have, and it works (I did cat largefile >file and confirmed it's
much faster when prefixed with !!, of course because the data doesn't
get copied to the local machine).

--=20
Sean Whitton




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46351; Package emacs. Full text available.

Message received at 46351 <at> debbugs.gnu.org:


Received: (at 46351) by debbugs.gnu.org; 7 Feb 2021 09:17:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Feb 07 04:17:53 2021
Received: from localhost ([127.0.0.1]:46872 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l8gCX-0002GW-6J
	for submit <at> debbugs.gnu.org; Sun, 07 Feb 2021 04:17:53 -0500
Received: from mout.gmx.net ([212.227.17.21]:46649)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael.albinus@HIDDEN>) id 1l8gCU-0002GI-MJ
 for 46351 <at> debbugs.gnu.org; Sun, 07 Feb 2021 04:17:51 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net;
 s=badeba3b8450; t=1612689461;
 bh=kBYyXybfJTGujIUP+XabvU5wXuS8zAEj+aZOi8HbzjE=;
 h=X-UI-Sender-Class:From:To:Cc:Subject:References:Date:In-Reply-To;
 b=aYg0nScMWrDxfGF1omlTGwEzZpeV4OMnKY1tC3tRF+ztY0DrdGhkKTKNMZM/hn9F6
 ejYFHpiCxbE6757lWLuncgqpDvzP9wXGYM3bz+sSy8jHqtKejaYwwRWi8Z/m3Ffm/e
 seSnTw3pKz0Z9JyH24DkPAv1P02yZx+gm4tlaql0=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from gandalf.gmx.de ([178.20.94.105]) by mail.gmx.net (mrgmx104
 [212.227.17.168]) with ESMTPSA (Nemesis) id 1Ma20q-1lTbFZ0kiK-00VvEz; Sun, 07
 Feb 2021 10:17:41 +0100
From: Michael Albinus <michael.albinus@HIDDEN>
To: Sean Whitton <spwhitton@HIDDEN>
Subject: Re: bug#46351: 28.0.50; Add convenient way to bypass Eshell's own
 pipelining
References: <878s812c6a.fsf@HIDDEN>
Date: Sun, 07 Feb 2021 10:17:39 +0100
In-Reply-To: <878s812c6a.fsf@HIDDEN> (Sean Whitton's message
 of "Sat, 06 Feb 2021 13:06:53 -0700")
Message-ID: <87eehsz170.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K1:3D9BpDFhXMiKRCUyou6dlXWDmN/cx+UzIGwhHHN/1Mgu/coJW6q
 Qb60jvGbf4QS64+H9XZlBjPIhQ8vx5qNHQZYPKJnoYdn8KS8M4Iu3SQSwsYBfyrdQSNUWB8
 pMIHvipTO9aP1R4L41GaxIEuPfXNVzvWkDu+alP3/D3IIanbcgFPDPBCGrrbZ8V+YHFrLgC
 D0YA3wJkLT9NQErH+ROPA==
X-Spam-Flag: NO
X-UI-Out-Filterresults: notjunk:1;V03:K0:6mVKEQ8ht3k=:4QkjRb2CmWuTZ/Gb7y8snJ
 SEy8GxDcqrkt8vTMvmbyj6vZo/c6XbFBSxUJe3k6HkehEBcKJOLrSMkL4ECUoZiBnQ3Wv0vzV
 +m0Hq7LXgOrLVbUyNLYP8noSWcm2C+SaqEIn3Sf4pHff/wyeC6ePIb49bnFz8Jy0h8KC7ACx5
 RSNANzPUD5r+nEZ8rgNxnsQuDPvfQ7CiT1aR5DE9VxjOmWy6lbUG9PgS/MlFev8zyGAmBHyb8
 jZFb4HWApG1VOe7hqIyDgp3tlyf8/SI3SLSSuPL/+TNpeL9eG6AwlFjor704ZWBiF4qSOeLM5
 qpxrIERN1UOI6WcMqzHCdC6r1hlhslEvIcuEFn6ozMgfNtUd1AuebcdtpE8muAC233BQftA6p
 CxMN0MmdeVYsUodRm2vzSFiYLibJ13Pn3IxPcBbNHDu+tTyKUW+ZCgR7dIMsSQspzDTxDx9RF
 ySXyh6NTJbtog5I0sMkB8B6hLaYP6C2qYtdKMuyyWCNLqhhYtSyBdpBEgKmm5RMhHIVqkky1f
 0QsXjFtn4I/pSKJIfQ0FOs5BmXaSx70QZwXxZe752RzdUV9syjKA0J824yZYW4lRhYZuLtjgQ
 iOhGtHTM4kszI9gtZzvylQJr1njDjOcTq/e2hXyxppNHlFWOhbmmuMHJ9a85D9n00E5nVr8sX
 /khj1TPfDwn5z8a1fLRS0+Rp/7UdkwEOLt9AIw/wKhDMEG3UkbtS3kOUTnk1JJKtEonR/737d
 brXCKsU1A8Jab/SLaAixiInUOUM56PS3cvltuEWqnXpqmPNBvo1y1IMxlt6TaM3b/I+rMnKT0
 oF0inHathM5HNcjp5ZhuPlxb8cqdS/GuZAtfyjre57WAFFkeSQXMzPn1R4CFtrP53OPNBJ9u2
 OHpIC2bdHhGQi+G5ewuQ==
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 46351
Cc: 46351 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Sean Whitton <spwhitton@HIDDEN> writes:

Hi Sean,

> I think that it would be good to have a way to easily toggle Eshell's
> own pipelining support on and off for particular commands.  I have come
> up with the attached patches.  When the new functions I've defined are
> enabled, you can type
>
>     !! foo | bar 'arg' >baz >>#<buffer *scratch*>
>
> and it will be executed by Eshell as if you had typed
>
>     bash -c 'foo | bar '"'"'arg'"'"' >baz' >>#<buffer *scratch*>
>
> The idea is that you can easily toggle Eshell's pipelining on and off as
> appropriate to your needs just by adding and removing the "!!" prefix.

Nice idea.

> I think that the patches I've prepared are a clean implementation of
> this feature that would be good to include in Emacs.

Applying your patch, I get the compiler warning

--8<---------------cut here---------------start------------->8---
  ELC      eshell/esh-mode.elc

In end of data:
eshell/esh-mode.el:1114:1: Warning: the function =E2=80=98(setf buffer-subs=
tring)=E2=80=99 is
    not known to be defined.
--8<---------------cut here---------------end--------------->8---

And using it in eshell, there is

--8<---------------cut here---------------start------------->8---
~/src/emacs $ !! cat ~/.emacs | grep albinus
!!: command not found
--8<---------------cut here---------------end--------------->8---

I wanted to see, whether this works also for remote directories. Have
you tested this?

> Sean Whitton

Best regards, Michael.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#46351; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 6 Feb 2021 20:06:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Feb 06 15:06:14 2021
Received: from localhost ([127.0.0.1]:46507 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1l8TqQ-0006Ew-3Q
	for submit <at> debbugs.gnu.org; Sat, 06 Feb 2021 15:06:14 -0500
Received: from lists.gnu.org ([209.51.188.17]:41388)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <spwhitton@HIDDEN>) id 1l8TqK-0006Eg-7c
 for submit <at> debbugs.gnu.org; Sat, 06 Feb 2021 15:06:12 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:37002)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <spwhitton@HIDDEN>)
 id 1l8TqJ-0001CF-Ug
 for bug-gnu-emacs@HIDDEN; Sat, 06 Feb 2021 15:06:08 -0500
Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:39175)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <spwhitton@HIDDEN>)
 id 1l8TqE-0003WB-Kn
 for bug-gnu-emacs@HIDDEN; Sat, 06 Feb 2021 15:06:07 -0500
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
 by mailout.west.internal (Postfix) with ESMTP id F182FD43
 for <bug-gnu-emacs@HIDDEN>; Sat,  6 Feb 2021 15:06:00 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute4.internal (MEProxy); Sat, 06 Feb 2021 15:06:01 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spwhitton.name;
 h=from:to:subject:date:message-id:mime-version:content-type; s=
 fm2; bh=xFVppfuaA/xSv23xhAoBA9UZEx2Wo1lsi1ufwdwgpMM=; b=QGSrkhHY
 Yv10jcwaOND6I5REHP2wQGwsv19RbMGPE03KhZKXYENFgZUZCui4rqW0Tkha8TXK
 OkItXe9Fk3tXD5Bt6zkZ2HA3IexRLHb31M1F6QVaUET+oJmoJGoEZ7pRjmJrq2hv
 XxLKn7AwGtB7yb84bmyPqdDHaQIuAwEkpygVsOBw04zdsNsNHCfald2Q1au2uGhJ
 Mk18KuW/7M9lkwK9JgeUmwFD4nMqJKE+/b7nx9Mk/RaqLH+4uWOi/Ntz9A/nNKx/
 zfAyHn+pyzORBLNpNIrCjy+WtqmQSgv1tV2rmzrifrWK4xo8KnaxnwWISTpyYOM1
 6T+lFnVfQgBl7A==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=content-type:date:from:message-id
 :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender
 :x-me-sender:x-sasl-enc; s=fm2; bh=xFVppfuaA/xSv23xhAoBA9UZEx2Wo
 1lsi1ufwdwgpMM=; b=AEOEE9LgoSEYjTxB6ZtSbJJplGhJaspOoSUPiyWZetDwL
 MsQ0+bwtlARVBi9ezNrmMz0ICPkhdhlrDEFIo+mIb+/ivDCWzy03Y5zFM463K18g
 2lM7h7b/Lyyep6GhtRfBXFp6R9e1Q5v4s+PPfh9QnvqsEcReDqJg8MkjpCWtidXZ
 0zn8LkcA39bghTSE8AkAmfZ3BQdMM6Z4EkPQVE+9VJ1B0hVt/gN8tKJoopcgIDKe
 fAwOb2Rb2NWykHli0k/lr2JTWBAj+CJ6CEz3+cKk9eLw7K2lVjSzUJJ0v59aVBcX
 pd8MoyyUlBOfOdAMAXBbcWZokWPPXRJKg3frvzL8A==
X-ME-Sender: <xms:qPYeYBevWkUpkI852GXIKFXPqb0l7NoihWcznCvtJ5x4J2d3spEcBg>
 <xme:qPYeYPNSFaRdFtbEKKxjqu776AIKI0xapA9t_1kAhNOrxsXPln4U2RExT2tQEQMCN
 5MnO0plShXkR_kK3A>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrgeekgddufeefucetufdoteggodetrfdotf
 fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
 uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkgggtsehmtderredttd
 dtnecuhfhrohhmpefuvggrnhcuhghhihhtthhonhcuoehsphifhhhithhtohhnsehsphif
 hhhithhtohhnrdhnrghmvgeqnecuggftrfgrthhtvghrnhepteegtdejheejvdehgeettd
 dvjeffudetuedutdegkefhtdetteeuueetkeefkeegnecuvehluhhsthgvrhfuihiivgep
 tdenucfrrghrrghmpehmrghilhhfrhhomhepshhpfihhihhtthhonhesshhpfihhihhtth
 honhdrnhgrmhgv
X-ME-Proxy: <xmx:qPYeYKj7_i4HU40aL2VBcayWYOPLJMxk42pnnK1iso0Ai2I5fhJkqQ>
 <xmx:qPYeYK-CbQwPmNvZHP3z1p_EkVZeYgzzJcHkH0rMsIcRnSgc7bv_wg>
 <xmx:qPYeYNuioSUdlyd2e0g51nwy_WUpkLFEmQ7-ywRReTFuBJC2lzfE5A>
 <xmx:qPYeYL5FE2cL9kYMxErcQhtGXSXuhGsgg_JimkioHYzuSQCRSCHVZg>
From: Sean Whitton <spwhitton@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 28.0.50; Add convenient way to bypass Eshell's own pipelining
Date: Sat, 06 Feb 2021 13:06:53 -0700
Message-ID: <878s812c6a.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
Received-SPF: pass client-ip=64.147.123.24;
 envelope-from=spwhitton@HIDDEN; helo=wout1-smtp.messagingengine.com
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001,
 SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

--=-=-=
Content-Type: text/plain

Most people who use Eshell a lot need to be aware of the limitations of
its support for shell pipelines: primarily, that all data in pipelines
has to go through Emacs buffers, and that can be slow.  If you're
running pipelines which will move a lot of data, you should use a system
shell which will be more performant.

Currently, if Eshell is your primary shell, it's not particularly
convenient to switch to using an operating system shell when you're
preparing a pipeline that you know is going to move a lot of data.  You
would need either to switch to a shell-mode buffer, or quote and escape
your Eshell input and put something like `bash -c' in front of it.

I think that it would be good to have a way to easily toggle Eshell's
own pipelining support on and off for particular commands.  I have come
up with the attached patches.  When the new functions I've defined are
enabled, you can type

    !! foo | bar 'arg' >baz >>#<buffer *scratch*>

and it will be executed by Eshell as if you had typed

    bash -c 'foo | bar '"'"'arg'"'"' >baz' >>#<buffer *scratch*>

The idea is that you can easily toggle Eshell's pipelining on and off as
appropriate to your needs just by adding and removing the "!!" prefix.

I think that the patches I've prepared are a clean implementation of
this feature that would be good to include in Emacs.

-- 
Sean Whitton

--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=0001-Add-eshell-restore-unexpanded-input.patch

From ccc4427ff06bf6dc63840517aefbb7fb899d4b8c Mon Sep 17 00:00:00 2001
From: Sean Whitton <spwhitton@HIDDEN>
Date: Sat, 6 Feb 2021 00:01:50 -0700
Subject: [PATCH 1/2] Add eshell-restore-unexpanded-input

* lisp/eshell/esh-mode.el (eshell-send-input): Store the original
input before running eshell-expand-input-functions.
* lisp/eshell/esh-mode.el (eshell-restore-unexpanded-input): Define
new function and register as a customization option for
eshell-input-filter-functions.
---
 lisp/eshell/esh-mode.el | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/lisp/eshell/esh-mode.el b/lisp/eshell/esh-mode.el
index d29b010ea0..6036829941 100644
--- a/lisp/eshell/esh-mode.el
+++ b/lisp/eshell/esh-mode.el
@@ -197,6 +197,7 @@ This is used by `eshell-watch-for-password-prompt'."
 ;; byte-compiler, when compiling other files which `require' this one
 (defvar eshell-mode nil)
 (defvar eshell-command-running-string "--")
+(defvar eshell-last-unexpanded-input nil)
 (defvar eshell-last-input-start nil)
 (defvar eshell-last-input-end nil)
 (defvar eshell-last-output-start nil)
@@ -631,6 +632,7 @@ newline."
 		(progn
 		  (setq input (buffer-substring-no-properties
 			       eshell-last-output-end (1- (point))))
+                  (setq eshell-last-unexpanded-input input)
 		  (run-hook-with-args 'eshell-expand-input-functions
 				      eshell-last-output-end (1- (point)))
 		  (let ((cmd (eshell-parse-command-input
@@ -664,6 +666,21 @@ newline."
 
 (custom-add-option 'eshell-input-filter-functions 'eshell-kill-new)
 
+(defun eshell-restore-unexpanded-input ()
+  "Restore the input text before `eshell-expand-input-functions' ran.
+Useful when you want to see the unexpanded input rather than the
+expanded input in the Eshell buffer, and when you want later
+entries in `eshell-input-filter-functions' to use the unexpanded
+input.  For example, you might want the unexpanded input to be
+what gets stored in the history list."
+  (setf (buffer-substring eshell-last-input-start
+                          (1- eshell-last-input-end))
+        eshell-last-unexpanded-input)
+  (eshell-update-markers eshell-last-output-end))
+
+(custom-add-option 'eshell-input-filter-functions
+                   'eshell-restore-unexpanded-input)
+
 (defun eshell-output-filter (process string)
   "Send the output from PROCESS (STRING) to the interactive display.
 This is done after all necessary filtering has been done."
-- 
2.29.2


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=0002-Add-eshell-shell-command-and-eshell-expand-to-eshell.patch

From d5d48427c75555cd5cf2f6eb5b89e0c832cf7edf Mon Sep 17 00:00:00 2001
From: Sean Whitton <spwhitton@HIDDEN>
Date: Sat, 6 Feb 2021 00:48:32 -0700
Subject: [PATCH 2/2] Add eshell-shell-command and
 eshell-expand-to-eshell-shell-command

* lisp/eshell/esh-mode.el (eshell-shell-command,
eshell-expand-to-eshell-shell-command): Define new functions.
Register eshell-expand-to-eshell-shell-command as a customization
option for eshell-expand-input-functions.
---
 lisp/eshell/esh-mode.el | 77 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/lisp/eshell/esh-mode.el b/lisp/eshell/esh-mode.el
index 6036829941..b3132393a5 100644
--- a/lisp/eshell/esh-mode.el
+++ b/lisp/eshell/esh-mode.el
@@ -62,6 +62,7 @@
 (require 'esh-module)
 (require 'esh-cmd)
 (require 'esh-arg)                      ;For eshell-parse-arguments
+(require 'subr-x)
 
 (defgroup eshell-mode nil
   "This module contains code for handling input from the user."
@@ -660,6 +661,82 @@ newline."
 	       (run-hooks 'eshell-post-command-hook)
 	       (insert-and-inherit input)))))))))
 
+(defun eshell-shell-command (command)
+  "Execute COMMAND using the operating system shell."
+  (throw 'eshell-replace-command
+	 (eshell-parse-command shell-file-name
+			       (list shell-command-switch command))))
+
+(defun eshell-expand-to-eshell-shell-command (beg end)
+  "Expand Eshell input starting with '!!' to use `eshell-shell-command'.
+
+This is intended to provide a convenient way to bypass Eshell's
+own pipelining which can be slow when executing shell pipelines
+which move a lot of data.  It is also useful to avoid
+roundtripping data when running pipelines on remote hosts.
+
+To enable, add this function to `eshell-expand-input-functions'.
+
+For example, this function expands the input
+
+    !! cat *.ogg | my-cool-decoder >file
+
+to
+
+    eshell-shell-command \"cat *.ogg | my-cool-decoder >file\"
+
+Executing the latter command will not copy all the data in the
+*.ogg files into Emacs buffers, as would normally happen with
+Eshell's own pipelining.
+
+This function tries to extract Eshell-specific redirects and
+avoids passing these to the operating system shell.  For example,
+
+    !! foo | bar >>#<buffer scratch>
+
+will be expanded to
+
+    eshell-shell-command \"foo | bar\" >>#<buffer scratch>
+
+This function works well in combination with adding
+`eshell-restore-unexpanded-input' to `eshell-input-filter-functions'."
+  (save-excursion
+    (goto-char beg)
+    (when (looking-at "!!\\s-*")
+      (let ((end (copy-marker end)) ; needs to be a marker
+            redirects)
+        ;; drop the !!
+        (delete-region beg (match-end 0))
+        ;; extract Eshell-specific redirects
+        (while (search-forward-regexp "[0-9]?>+&?[0-9]?\\s-*\\S-" nil t)
+          (let ((beg (match-beginning 0)))
+            (forward-char -1) ; start from the redirect target
+            (when-let ((end (cond
+                             ;; this is a redirect to a process or a
+                             ;; buffer
+                             ((looking-at "#<")
+                              (forward-char 1)
+                              (1+ (eshell-find-delimiter ?\< ?\>)))
+                             ;; this is a redirect to a virtual target
+                             ((and (looking-at "/\\S-+")
+                                   (assoc (match-string 0)
+                                          eshell-virtual-targets))
+                              (match-end 0)))))
+              (push (buffer-substring-no-properties beg end) redirects)
+              (delete-region beg end)
+              (just-one-space))))
+        ;; wrap the remaining text and reinstate Eshell redirects
+        (let ((pipeline (string-trim
+                         (buffer-substring-no-properties beg end))))
+          (delete-region beg end)
+          (insert "eshell-shell-command ")
+          (prin1 pipeline (current-buffer))
+          (when redirects
+            (insert " " (string-join (nreverse redirects) " "))))))))
+
+(custom-add-option 'eshell-expand-input-functions
+                   'eshell-expand-to-eshell-shell-command)
+
 (defsubst eshell-kill-new ()
   "Add the last input text to the kill ring."
   (kill-ring-save eshell-last-input-start eshell-last-input-end))
-- 
2.29.2


--=-=-=--




Acknowledgement sent to Sean Whitton <spwhitton@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#46351; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 7 Feb 2021 19:15:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.