GNU bug report logs -
#56025
29.0.50; em-extpipe-test-2 times out on EMBA and Cygwin
Previous Next
Reported by: Ken Brown <kbrown <at> cornell.edu>
Date: Thu, 16 Jun 2022 18:36:02 UTC
Severity: normal
Found in version 29.0.50
Fixed in version 29.1
Done: Lars Ingebrigtsen <larsi <at> gnus.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 56025 in the body.
You can then email your comments to 56025 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
spwhitton <at> spwhitton.name, bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Thu, 16 Jun 2022 18:36:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Ken Brown <kbrown <at> cornell.edu>:
New bug report received and forwarded. Copy sent to
spwhitton <at> spwhitton.name, bug-gnu-emacs <at> gnu.org.
(Thu, 16 Jun 2022 18:36:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
In commit 231a1ba3, Lars disabled the extpipe tests on EMBA "because they
apparently time out". The EMBA log he cited in the commit message shows that
only em-extpipe-test-2 times out. And this same test (but no others) also times
out on Cygwin:
$ make -C test em-extpipe-tests
[...]
Test em-extpipe-test-2 backtrace:
signal(error ("timed out waiting for subprocess(es)"))
error("timed out waiting for subprocess(es)")
eshell-wait-for-subprocess()
eshell-command-result-p("echo \"bar\" | rev *>/tmp/emacs-test-Rw5ILv
(let ((input (replace-regexp-in-string "temp\\([^>]\\|\\'\\)" temp (
(unwind-protect (let ((input (replace-regexp-in-string "temp\\([^>]\
(let ((temp-buffer (generate-new-buffer " *temp*" t))) (unwind-prote
(progn (let ((temp-buffer (generate-new-buffer " *temp*" t))) (unwin
(unwind-protect (progn (let ((temp-buffer (generate-new-buffer " *te
(let* ((coding-system-for-write nil) (temp-file (identity (make-temp
(save-current-buffer (set-buffer eshell-buffer) (let* ((fn-38 #'exec
(unwind-protect (save-current-buffer (set-buffer eshell-buffer) (let
(let* ((process-environment (cons "HISTFILE" process-environment)) (
(progn (let* ((process-environment (cons "HISTFILE" process-environm
(unwind-protect (progn (let* ((process-environment (cons "HISTFILE"
(let* ((coding-system-for-write nil) (temp-file (file-name-as-direct
(save-current-buffer (let* ((coding-system-for-write nil) (temp-file
(let ((input "echo \"bar\" | rev *>temp")) (save-current-buffer (let
(progn (let ((value-24 (gensym "ert-form-evaluation-aborted-"))) (le
(closure (t) nil (progn (let ((value-24 (gensym "ert-form-evaluation
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name em-extpipe-test-2 :documentation nil
ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
ert-run-tests((not (or (tag :unstable) (tag :nativecomp))) #f(compil
ert-run-tests-batch((not (or (tag :unstable) (tag :nativecomp))))
ert-run-tests-batch-and-exit((not (or (tag :unstable) (tag :nativeco
eval((ert-run-tests-batch-and-exit '(not (or (tag :unstable) (tag :n
command-line-1(("-L" ":../../master/test" "-l" "ert" "-l" "lisp/eshe
command-line()
normal-top-level()
Test em-extpipe-test-2 condition:
(error "timed out waiting for subprocess(es)")
FAILED 10/17 em-extpipe-test-2 (5.237846 sec) at
../../master/test/lisp/eshell/em-extpipe-tests.el:87
I don't see what's special about this test that would cause it to time out on
some systems. Sean, do you have any ideas? If not, I suggest just skipping it
on Cygwin. And maybe we should skip just this extpipe test (rather than all of
them) on EMBA.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Thu, 16 Jun 2022 19:34:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Hello Ken,
On Thu 16 Jun 2022 at 02:30pm -04, Ken Brown wrote:
> In commit 231a1ba3, Lars disabled the extpipe tests on EMBA "because they
> apparently time out". The EMBA log he cited in the commit message shows that
> only em-extpipe-test-2 times out. And this same test (but no others) also times
> out on Cygwin:
That test invokes `sh -c "rev >temp"` as its only subprocess, so that's
probably what is timing out. Is there something different about
Cygwin's sh? Something about EOFs?
--
Sean Whitton
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Thu, 16 Jun 2022 22:02:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/16/2022 3:30 PM, Sean Whitton wrote:
> Hello Ken,
>
> On Thu 16 Jun 2022 at 02:30pm -04, Ken Brown wrote:
>
>> In commit 231a1ba3, Lars disabled the extpipe tests on EMBA "because they
>> apparently time out". The EMBA log he cited in the commit message shows that
>> only em-extpipe-test-2 times out. And this same test (but no others) also times
>> out on Cygwin:
>
> That test invokes `sh -c "rev >temp"` as its only subprocess, so that's
> probably what is timing out. Is there something different about
> Cygwin's sh? Something about EOFs?
I'm not aware of anything different. Here's what happens when I run the test
interactively:
$ time echo \"bar\" | sh -c "rev >temp"
real 0m0.100s
user 0m0.030s
sys 0m0.030s
$ cat temp
"rab"
And keep in mind that the test also times out on EMBA.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 17 Jun 2022 13:40:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/16/2022 6:01 PM, Ken Brown wrote:
> On 6/16/2022 3:30 PM, Sean Whitton wrote:
>> Hello Ken,
>>
>> On Thu 16 Jun 2022 at 02:30pm -04, Ken Brown wrote:
>>
>>> In commit 231a1ba3, Lars disabled the extpipe tests on EMBA "because they
>>> apparently time out". The EMBA log he cited in the commit message shows that
>>> only em-extpipe-test-2 times out. And this same test (but no others) also times
>>> out on Cygwin:
>>
>> That test invokes `sh -c "rev >temp"` as its only subprocess, so that's
>> probably what is timing out. Is there something different about
>> Cygwin's sh? Something about EOFs?
>
> I'm not aware of anything different. Here's what happens when I run the test
> interactively:
>
> $ time echo \"bar\" | sh -c "rev >temp"
>
> real 0m0.100s
> user 0m0.030s
> sys 0m0.030s
>
> $ cat temp
> "rab"
>
> And keep in mind that the test also times out on EMBA.
I just tried a different experiment: In an interactive emacs-29 session, I
started eshell and typed
echo \"bar\" | rev *>temp
Nothing visible happens until I type 'C-c C-c'. Then a prompt appears again,
and 'ls -l' shows that temp exists and is empty.
Prior to typing 'C-c C-c', 'M-x list-processes' (or 'C-c C-s') shows a bash
process running but it doesn't show 'rev'. But running 'ps' outside of emacs
shows both 'rev' and its parent 'bash' process.
It does seem that there's an actual bug here, not just a test that should be
skipped because it times out. It could be a Cygwin bug, of course, but that
doesn't explain the EMBA failure.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 00:58:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Hello,
On Fri 17 Jun 2022 at 09:39am -04, Ken Brown wrote:
> On 6/16/2022 6:01 PM, Ken Brown wrote:
>> On 6/16/2022 3:30 PM, Sean Whitton wrote:
>>> Hello Ken,
>>>
>>> On Thu 16 Jun 2022 at 02:30pm -04, Ken Brown wrote:
>>>
>>>> In commit 231a1ba3, Lars disabled the extpipe tests on EMBA "because they
>>>> apparently time out". The EMBA log he cited in the commit message shows that
>>>> only em-extpipe-test-2 times out. And this same test (but no others) also times
>>>> out on Cygwin:
>>>
>>> That test invokes `sh -c "rev >temp"` as its only subprocess, so that's
>>> probably what is timing out. Is there something different about
>>> Cygwin's sh? Something about EOFs?
>>
>> I'm not aware of anything different. Here's what happens when I run the test
>> interactively:
>>
>> $ time echo \"bar\" | sh -c "rev >temp"
>>
>> real 0m0.100s
>> user 0m0.030s
>> sys 0m0.030s
>>
>> $ cat temp
>> "rab"
>>
>> And keep in mind that the test also times out on EMBA.
>
> I just tried a different experiment: In an interactive emacs-29 session, I
> started eshell and typed
>
> echo \"bar\" | rev *>temp
>
> Nothing visible happens until I type 'C-c C-c'. Then a prompt appears again,
> and 'ls -l' shows that temp exists and is empty.
>
> Prior to typing 'C-c C-c', 'M-x list-processes' (or 'C-c C-s') shows a bash
> process running but it doesn't show 'rev'. But running 'ps' outside of emacs
> shows both 'rev' and its parent 'bash' process.
>
> It does seem that there's an actual bug here, not just a test that should be
> skipped because it times out. It could be a Cygwin bug, of course, but that
> doesn't explain the EMBA failure.
Could you see if the same thing happens if you type
echo "bar" | sh -c "rev >temp"
into an interactive session, please?
If it's the same then extpipe has uncovered a general Eshell bug.
--
Sean Whitton
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 02:08:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/17/2022 8:57 PM, Sean Whitton wrote:
> On Fri 17 Jun 2022 at 09:39am -04, Ken Brown wrote:
>> I just tried a different experiment: In an interactive emacs-29 session, I
>> started eshell and typed
>>
>> echo \"bar\" | rev *>temp
>>
>> Nothing visible happens until I type 'C-c C-c'. Then a prompt appears again,
>> and 'ls -l' shows that temp exists and is empty.
>>
>> Prior to typing 'C-c C-c', 'M-x list-processes' (or 'C-c C-s') shows a bash
>> process running but it doesn't show 'rev'. But running 'ps' outside of emacs
>> shows both 'rev' and its parent 'bash' process.
>>
>> It does seem that there's an actual bug here, not just a test that should be
>> skipped because it times out. It could be a Cygwin bug, of course, but that
>> doesn't explain the EMBA failure.
>
> Could you see if the same thing happens if you type
>
> echo "bar" | sh -c "rev >temp"
>
> into an interactive session, please?
>
> If it's the same then extpipe has uncovered a general Eshell bug.
Yes, it's the same. And it's even the same if I remove the quotation marks
around "rev >temp".
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 02:36:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/17/2022 10:07 PM, Ken Brown wrote:
> On 6/17/2022 8:57 PM, Sean Whitton wrote:
>> On Fri 17 Jun 2022 at 09:39am -04, Ken Brown wrote:
>>> I just tried a different experiment: In an interactive emacs-29 session, I
>>> started eshell and typed
>>>
>>> echo \"bar\" | rev *>temp
>>>
>>> Nothing visible happens until I type 'C-c C-c'. Then a prompt appears again,
>>> and 'ls -l' shows that temp exists and is empty.
>>>
>>> Prior to typing 'C-c C-c', 'M-x list-processes' (or 'C-c C-s') shows a bash
>>> process running but it doesn't show 'rev'. But running 'ps' outside of emacs
>>> shows both 'rev' and its parent 'bash' process.
>>>
>>> It does seem that there's an actual bug here, not just a test that should be
>>> skipped because it times out. It could be a Cygwin bug, of course, but that
>>> doesn't explain the EMBA failure.
>>
>> Could you see if the same thing happens if you type
>>
>> echo "bar" | sh -c "rev >temp"
>>
>> into an interactive session, please?
>>
>> If it's the same then extpipe has uncovered a general Eshell bug.
>
> Yes, it's the same. And it's even the same if I remove the quotation marks
> around "rev >temp".
And it's also the same if remove the output redirection, i.e., if I type 'echo
bar | sh -c rev'. On the other hand, if I type 'echo bar | rev', then I see the
output 'rab'; but rev doesn't exit, and I don't get the eshell prompt, until I
type 'C-c C-c'. On the third hand, 'echo bar | cat' almost works; I see the
output 'bar' followed immediately by the eshell prompt (with no newline after
'bar'). Is eshell stripping newlines in some circumstances? If so, I wonder if
this accounts for the failure of rev to exit in some of the earlier examples,
since it operates on lines?
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 03:51:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/17/2022 7:07 PM, Ken Brown wrote:
> On 6/17/2022 8:57 PM, Sean Whitton wrote:
>> Could you see if the same thing happens if you type
>>
>> echo "bar" | sh -c "rev >temp"
>>
>> into an interactive session, please?
>>
>> If it's the same then extpipe has uncovered a general Eshell bug.
>
> Yes, it's the same. And it's even the same if I remove the quotation
> marks around "rev >temp".
Does the above command also fail on Emacs 28? I changed some aspects of
process management for Eshell in Emacs 29, so it's possible this is a
regression. If it works correctly under Emacs 28, I'd be very interested
to see the results of bisecting to find the breaking commit.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 17:53:01 GMT)
Full text and
rfc822 format available.
Message #29 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/17/2022 11:50 PM, Jim Porter wrote:
> On 6/17/2022 7:07 PM, Ken Brown wrote:
>> On 6/17/2022 8:57 PM, Sean Whitton wrote:
>>> Could you see if the same thing happens if you type
>>>
>>> echo "bar" | sh -c "rev >temp"
>>>
>>> into an interactive session, please?
>>>
>>> If it's the same then extpipe has uncovered a general Eshell bug.
>>
>> Yes, it's the same. And it's even the same if I remove the quotation marks
>> around "rev >temp".
>
> Does the above command also fail on Emacs 28? I changed some aspects of process
> management for Eshell in Emacs 29, so it's possible this is a regression. If it
> works correctly under Emacs 28, I'd be very interested to see the results of
> bisecting to find the breaking commit.
No, I'm seeing the same results on Emacs 28. On both Emacs 28 and Emacs 29, rev
is apparently not seeing EOF unless echo outputs a newline, so rev keeps waiting
for input.
[Side note: It took me a while to sort this out because (a) Eshell's echo does
not output a newline by default, in contrast to Bash's builtin echo; (b) in
Eshell in Emacs 28, you use '-n' to add a newline, while in Bash '-n' suppresses
the newline; and (c) in Eshell in Emacs 29, you use '-N' to add a newline.]
Here's my simplest reproduction recipe for the bug: Type 'echo bar | rev' into
Eshell. In both Emacs 28 and Emacs 29, 'rab' is output but rev keeps running.
But if I make echo output a newline (by '-n' in Emacs 28 and '-N' in Emacs 29),
then rev exits after outputting 'rab' (followed by newline).
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 19:03:01 GMT)
Full text and
rfc822 format available.
Message #32 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/18/2022 10:52 AM, Ken Brown wrote:
> No, I'm seeing the same results on Emacs 28. On both Emacs 28 and Emacs
> 29, rev is apparently not seeing EOF unless echo outputs a newline, so
> rev keeps waiting for input.
Ah ha! Thanks for debugging this. The minimal fix then would be to
change the command in em-extpipe-test-2 to either of these:
echo -N "bar" | rev *>temp
*echo "bar" | rev *>temp
One last[1] question: if you ran "echo -n bar | rev" in Cygwin Bash,
does it hang there too? Maybe this is just a Cygwin limitation, or maybe
Eshell is doing something wrong with its built-in pipelines in this
situation.
> [Side note: It took me a while to sort this out because (a) Eshell's
> echo does not output a newline by default, in contrast to Bash's builtin
> echo; (b) in Eshell in Emacs 28, you use '-n' to add a newline, while in
> Bash '-n' suppresses the newline; and (c) in Eshell in Emacs 29, you use
> '-N' to add a newline.]
This is one of the parts of Eshell that's always bothered me a bit.
Eshell's echo is different enough from other echo implementations that
it's easy to get tripped up. There's some further discussion of this in
bug#12689 as well. I'm hesitant to change Eshell's echo too much, since
it could break user scripts, but it would be nice if we could find a
reasonably-compatible way of making it work more like /bin/echo.
[1] Well, probably last.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 20:52:01 GMT)
Full text and
rfc822 format available.
Message #35 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/18/2022 3:02 PM, Jim Porter wrote:
> On 6/18/2022 10:52 AM, Ken Brown wrote:
>> No, I'm seeing the same results on Emacs 28. On both Emacs 28 and Emacs 29,
>> rev is apparently not seeing EOF unless echo outputs a newline, so rev keeps
>> waiting for input.
>
> Ah ha! Thanks for debugging this. The minimal fix then would be to change the
> command in em-extpipe-test-2 to either of these:
>
> echo -N "bar" | rev *>temp
This doesn't work. It still hangs when run interactively, as does the equivalent
echo -N bar | sh -c "rev >temp"
> *echo "bar" | rev *>temp
This works interactively, but I don't know the appropriate syntax for modifying
the test. Naively replacing each 'echo' by '*echo' caused the 'should-parse' to
fail.
> One last[1] question: if you ran "echo -n bar | rev" in Cygwin Bash, does it
> hang there too?
No.
> Maybe this is just a Cygwin limitation, or maybe Eshell is doing
> something wrong with its built-in pipelines in this situation.
My guess is that it's the latter, but I don't know if it's worth pursuing this
if Cygwin and EMBA are the only platforms on which there's a problem. Of
course, there might be other platforms and no one has reported it.
Once the test is fixed to succeed on Cygwin, we should probably revert the
change that caused the extpipe tests to be skipped on EMBA, just to make sure
that the same fix works there.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 22:01:01 GMT)
Full text and
rfc822 format available.
Message #38 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/18/2022 1:51 PM, Ken Brown wrote:
> On 6/18/2022 3:02 PM, Jim Porter wrote:
>> On 6/18/2022 10:52 AM, Ken Brown wrote:
>>> No, I'm seeing the same results on Emacs 28. On both Emacs 28 and
>>> Emacs 29, rev is apparently not seeing EOF unless echo outputs a
>>> newline, so rev keeps waiting for input.
>>
>> Ah ha! Thanks for debugging this. The minimal fix then would be to
>> change the command in em-extpipe-test-2 to either of these:
>>
>> echo -N "bar" | rev *>temp
>
> This doesn't work. It still hangs when run interactively...
Just to confirm, the above command hangs, but the following works, correct?
echo -N "bar" | rev
>> *echo "bar" | rev *>temp
>
> This works interactively...
All this makes me think that we could be dealing with a race condition
in how Eshell pipes I/O around. Maybe there's a timing issue in
`eshell-close-target' where we end up not sending EOF to the "rev" (or
"sh") process?
I'd be interested to see the results if you ran `M-x trace-function' for
`eshell-close-target' and `process-status' before trying these commands.
`process-status' should return `run' when called from inside
`eshell-close-target'. If it doesn't, then we'd neglect to send EOF to
"rev" (or "sh"), which would cause a hang like what you're seeing.
If that's not the issue, then I'm not sure what the issue would be
exactly, but poking around in `eshell-close-target',
`eshell-insertion-filter', and `eshell-sentinel' might yield some useful
info.
> My guess is that it's the latter, but I don't know if it's worth
> pursuing this if Cygwin and EMBA are the only platforms on which there's
> a problem. Of course, there might be other platforms and no one has
> reported it.
I think if we could figure out the real issue, it would be great to fix
it. Though if we can't, it would probably be ok to just fix the test by
avoiding the issue.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 18 Jun 2022 23:47:01 GMT)
Full text and
rfc822 format available.
Message #41 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Hello,
On Sat 18 Jun 2022 at 03:00PM -07, Jim Porter wrote:
> I think if we could figure out the real issue, it would be great to fix
> it. Though if we can't, it would probably be ok to just fix the test by
> avoiding the issue.
We might first want to add another general Eshell test which shows up
the problem, though?
--
Sean Whitton
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 19 Jun 2022 16:03:01 GMT)
Full text and
rfc822 format available.
Message #44 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/18/2022 6:00 PM, Jim Porter wrote:
> On 6/18/2022 1:51 PM, Ken Brown wrote:
>> On 6/18/2022 3:02 PM, Jim Porter wrote:
>>> On 6/18/2022 10:52 AM, Ken Brown wrote:
>>>> No, I'm seeing the same results on Emacs 28. On both Emacs 28 and Emacs 29,
>>>> rev is apparently not seeing EOF unless echo outputs a newline, so rev keeps
>>>> waiting for input.
>>>
>>> Ah ha! Thanks for debugging this. The minimal fix then would be to change the
>>> command in em-extpipe-test-2 to either of these:
>>>
>>> echo -N "bar" | rev *>temp
>>
>> This doesn't work. It still hangs when run interactively...
>
> Just to confirm, the above command hangs, but the following works, correct?
>
> echo -N "bar" | rev
Correct.
>>> *echo "bar" | rev *>temp
>>
>> This works interactively...
>
> All this makes me think that we could be dealing with a race condition in how
> Eshell pipes I/O around. Maybe there's a timing issue in `eshell-close-target'
> where we end up not sending EOF to the "rev" (or "sh") process?
I think I've just discovered an anomaly in "rev" on Cygwin that could partially
explain what I'm seeing. I'll investigate that before proceeding further.
> I'd be interested to see the results if you ran `M-x trace-function' for
> `eshell-close-target' and `process-status' before trying these commands.
> `process-status' should return `run' when called from inside
> `eshell-close-target'. If it doesn't, then we'd neglect to send EOF to "rev" (or
> "sh"), which would cause a hang like what you're seeing.
>
> If that's not the issue, then I'm not sure what the issue would be exactly, but
> poking around in `eshell-close-target', `eshell-insertion-filter', and
> `eshell-sentinel' might yield some useful info.
>
>> My guess is that it's the latter, but I don't know if it's worth pursuing this
>> if Cygwin and EMBA are the only platforms on which there's a problem. Of
>> course, there might be other platforms and no one has reported it.
>
> I think if we could figure out the real issue, it would be great to fix it.
> Though if we can't, it would probably be ok to just fix the test by avoiding the
> issue.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 24 Jun 2022 01:19:01 GMT)
Full text and
rfc822 format available.
Message #47 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/19/2022 12:02 PM, Ken Brown wrote:
> On 6/18/2022 6:00 PM, Jim Porter wrote:
>> On 6/18/2022 1:51 PM, Ken Brown wrote:
>>> On 6/18/2022 3:02 PM, Jim Porter wrote:
>>>> On 6/18/2022 10:52 AM, Ken Brown wrote:
>>>>> No, I'm seeing the same results on Emacs 28. On both Emacs 28 and Emacs
>>>>> 29, rev is apparently not seeing EOF unless echo outputs a newline, so rev
>>>>> keeps waiting for input.
>>>>
>>>> Ah ha! Thanks for debugging this. The minimal fix then would be to change
>>>> the command in em-extpipe-test-2 to either of these:
>>>>
>>>> echo -N "bar" | rev *>temp
>>>
>>> This doesn't work. It still hangs when run interactively...
>>
>> Just to confirm, the above command hangs, but the following works, correct?
>>
>> echo -N "bar" | rev
>
> Correct.
>
>>>> *echo "bar" | rev *>temp
>>>
>>> This works interactively...
>>
>> All this makes me think that we could be dealing with a race condition in how
>> Eshell pipes I/O around. Maybe there's a timing issue in `eshell-close-target'
>> where we end up not sending EOF to the "rev" (or "sh") process?
>
> I think I've just discovered an anomaly in "rev" on Cygwin that could partially
> explain what I'm seeing. I'll investigate that before proceeding further.
OK, I think I've got it sorted out now. The anomaly I referred to above is
actually an anomaly in the stdio routines, not in "rev". It's discussed in item
2 below. There are two issues.
1. I think there's a bug in eshell-close-target, in which it's assumed that
sending C-d indicates end-of-file. This is only true if there's no input waiting
to be read. [In an interactive situation, this means we're at the beginning of
a line.] Otherwise, it takes a second C-d to indicate EOF. So one C-d should
suffice in the "echo -N bar" situation, but two are needed after "echo bar".
This bug probably went unnoticed because eshell-close-target was called twice in
the case we were discussing, so process-send-eof was called twice.
2. On Cygwin and some other platforms, including Solaris 11.4 I think, it
actually takes a third C-d, for reasons explained in the email thread starting
at https://cygwin.com/pipermail/cygwin/2022-June/251672.html. We're probably
going to change this on Cygwin, but that still leaves other platforms.
The following patch resolves both issues:
diff --git a/lisp/eshell/esh-io.el b/lisp/eshell/esh-io.el
index 3644c1a18b..1c4131cb07 100644
--- a/lisp/eshell/esh-io.el
+++ b/lisp/eshell/esh-io.el
@@ -276,8 +276,8 @@ eshell-close-target
;; If we're redirecting to a process (via a pipe, or process
;; redirection), send it EOF so that it knows we're finished.
((eshell-processp target)
- (if (eq (process-status target) 'run)
- (process-send-eof target)))
+ (while (eq (process-status target) 'run)
+ (process-send-eof target)))
;; A plain function redirection needs no additional arguments
;; passed.
I'm about to go AFK for a few days. If the eshell people agree that something
like this patch should be installed, please go ahead. I think it would then be
worth re-enabling the extpipe tests on EMBA to see if the problem is fixed there
too.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 24 Jun 2022 04:41:01 GMT)
Full text and
rfc822 format available.
Message #50 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Hello,
On Thu 23 Jun 2022 at 09:18pm -04, Ken Brown wrote:
> diff --git a/lisp/eshell/esh-io.el b/lisp/eshell/esh-io.el
> index 3644c1a18b..1c4131cb07 100644
> --- a/lisp/eshell/esh-io.el
> +++ b/lisp/eshell/esh-io.el
> @@ -276,8 +276,8 @@ eshell-close-target
> ;; If we're redirecting to a process (via a pipe, or process
> ;; redirection), send it EOF so that it knows we're finished.
> ((eshell-processp target)
> - (if (eq (process-status target) 'run)
> - (process-send-eof target)))
> + (while (eq (process-status target) 'run)
> + (process-send-eof target)))
>
> ;; A plain function redirection needs no additional arguments
> ;; passed.
>
> I'm about to go AFK for a few days. If the eshell people agree that something
> like this patch should be installed, please go ahead. I think it would then be
> worth re-enabling the extpipe tests on EMBA to see if the problem is fixed there
> too.
I'm a bit queasy about an unbounded loop here. Why not just try three
times? Or, better, try twice, and a third time only if we're on a
platform where we know it's needed.
Many thanks for the investigative work.
--
Sean Whitton
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 24 Jun 2022 06:08:02 GMT)
Full text and
rfc822 format available.
Message #53 received at 56025 <at> debbugs.gnu.org (full text, mbox):
> Date: Thu, 23 Jun 2022 21:18:24 -0400
> From: Ken Brown <kbrown <at> cornell.edu>
>
> 2. On Cygwin and some other platforms, including Solaris 11.4 I think, it
> actually takes a third C-d, for reasons explained in the email thread starting
> at https://cygwin.com/pipermail/cygwin/2022-June/251672.html. We're probably
> going to change this on Cygwin, but that still leaves other platforms.
>
> The following patch resolves both issues:
>
> diff --git a/lisp/eshell/esh-io.el b/lisp/eshell/esh-io.el
> index 3644c1a18b..1c4131cb07 100644
> --- a/lisp/eshell/esh-io.el
> +++ b/lisp/eshell/esh-io.el
> @@ -276,8 +276,8 @@ eshell-close-target
> ;; If we're redirecting to a process (via a pipe, or process
> ;; redirection), send it EOF so that it knows we're finished.
> ((eshell-processp target)
> - (if (eq (process-status target) 'run)
> - (process-send-eof target)))
> + (while (eq (process-status target) 'run)
> + (process-send-eof target)))
Please add there comments explaining why this is done, or at least
point to relevant messages in this bug's discussion (NOT just to the
bug number, as the discussion is long and it will be hard to
understand what part of it is relevant). Such "tricky" code should
always have comments explaining it.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 24 Jun 2022 16:54:01 GMT)
Full text and
rfc822 format available.
Message #56 received at 56025 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 6/23/2022 9:40 PM, Sean Whitton wrote:
> I'm a bit queasy about an unbounded loop here. Why not just try three
> times? Or, better, try twice, and a third time only if we're on a
> platform where we know it's needed.
How about the attached patch? I didn't check for specific platforms to
enable the "third EOF" behavior, since a) it's hard to say for sure
which platforms might have this issue (especially since Cygwin will be
fixing it), and b) this lets us avoid worrying about Tramp compatibility.
> Many thanks for the investigative work.
Agreed, this turned out to be a much subtler problem than I had
initially suspected. Thanks!
On 6/23/2022 11:07 PM, Eli Zaretskii wrote:
> Please add there comments explaining why this is done, or at least
> point to relevant messages in this bug's discussion (NOT just to the
> bug number, as the discussion is long and it will be hard to
> understand what part of it is relevant). Such "tricky" code should
> always have comments explaining it.
I added a comment explaining this to the best of my knowledge. There's
one additional caveat I didn't mention there though, since it's only
somewhat related. I believe this was mentioned earlier in the thread,
but when Eshell creates a pipe, it routes both stdout and stderr to the
next process's stdin (there's no way to control this behavior yet). When
closing the handles from the initial process, it then calls
`eshell-close-target' twice: once for stdout and once for stderr. Thus,
with this patch, we'll call `process-send-eof' up to six times.
I'm not sure this is really a problem in practice today, but it might
come up if Eshell gains the ability to redirect stdout and stderr
separately.
[0001-When-closing-an-Eshell-process-target-send-EOF-three.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 24 Jun 2022 22:24:02 GMT)
Full text and
rfc822 format available.
Message #59 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Hello,
On Fri 24 Jun 2022 at 09:53AM -07, Jim Porter wrote:
> On 6/23/2022 9:40 PM, Sean Whitton wrote:
> > I'm a bit queasy about an unbounded loop here. Why not just try three
> > times? Or, better, try twice, and a third time only if we're on a
> > platform where we know it's needed.
>
> How about the attached patch? I didn't check for specific platforms to
> enable the "third EOF" behavior, since a) it's hard to say for sure
> which platforms might have this issue (especially since Cygwin will be
> fixing it), and b) this lets us avoid worrying about Tramp compatibility.
Avoiding the TRAMP issues makes sense, but could you explain why you
don't think there could be an issue with sending a process too many
EOFs? It's not immediately obvious to me.
--
Sean Whitton
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 24 Jun 2022 23:04:02 GMT)
Full text and
rfc822 format available.
Message #62 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/24/2022 3:23 PM, Sean Whitton wrote:
> On Fri 24 Jun 2022 at 09:53AM -07, Jim Porter wrote:
>
>> How about the attached patch? I didn't check for specific platforms to
>> enable the "third EOF" behavior, since a) it's hard to say for sure
>> which platforms might have this issue (especially since Cygwin will be
>> fixing it), and b) this lets us avoid worrying about Tramp compatibility.
>
> Avoiding the TRAMP issues makes sense, but could you explain why you
> don't think there could be an issue with sending a process too many
> EOFs? It's not immediately obvious to me.
Eshell was already sending too many EOFs in some cases, and we haven't
seen any issues with it (that I know of). For example, consider the command:
*echo hi | rev
In this case, we send the string "hi\n" over the pipe, followed by 2
EOFs (one from the stdout handle and one from the stderr handle). The
POSIX standard says[1] (thanks to Eliot Moss on the Cygwin mailing list
for citing this passage):
When [EOF is] received, all the bytes waiting to be read are
immediately passed to the process without waiting for a <newline>, and
the EOF is discarded. Thus, if there are no bytes waiting (that is,
the EOF occurred at the beginning of a line), a byte count of zero
shall be returned from the read(), representing an end-of-file
indication.
I interpret that to mean that the preferred way to indicate end-of-file
to `rev' in this case is to send it "hi [NL] [EOF]". The second EOF that
Eshell sends when closing the stderr output handle is superfluous, but
it works fine as far as I can tell.
[1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap11.html
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 25 Jun 2022 05:35:01 GMT)
Full text and
rfc822 format available.
Message #65 received at 56025 <at> debbugs.gnu.org (full text, mbox):
> Cc: larsi <at> gnus.org, 56025 <at> debbugs.gnu.org
> From: Jim Porter <jporterbugs <at> gmail.com>
> Date: Fri, 24 Jun 2022 16:03:13 -0700
>
> POSIX standard says[1] (thanks to Eliot Moss on the Cygwin mailing list
> for citing this passage):
>
> When [EOF is] received, all the bytes waiting to be read are
> immediately passed to the process without waiting for a <newline>, and
> the EOF is discarded. Thus, if there are no bytes waiting (that is,
> the EOF occurred at the beginning of a line), a byte count of zero
> shall be returned from the read(), representing an end-of-file
> indication.
Perhaps some reference to this should also be in the comments.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 25 Jun 2022 16:14:01 GMT)
Full text and
rfc822 format available.
Message #68 received at 56025 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 6/24/2022 10:34 PM, Eli Zaretskii wrote:
>> Cc: larsi <at> gnus.org, 56025 <at> debbugs.gnu.org
>> From: Jim Porter <jporterbugs <at> gmail.com>
>> Date: Fri, 24 Jun 2022 16:03:13 -0700
>>
>> POSIX standard says[1] (thanks to Eliot Moss on the Cygwin mailing list
>> for citing this passage):
>>
>> When [EOF is] received, all the bytes waiting to be read are
>> immediately passed to the process without waiting for a <newline>, and
>> the EOF is discarded. Thus, if there are no bytes waiting (that is,
>> the EOF occurred at the beginning of a line), a byte count of zero
>> shall be returned from the read(), representing an end-of-file
>> indication.
>
> Perhaps some reference to this should also be in the comments.
How about this patch? I added a reference to the specific section of the
POSIX specification so that it's (hopefully) easy for people to look up
if they need further details.
[0001-When-closing-an-Eshell-process-target-send-EOF-three.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 25 Jun 2022 16:54:01 GMT)
Full text and
rfc822 format available.
Message #71 received at 56025 <at> debbugs.gnu.org (full text, mbox):
> Cc: larsi <at> gnus.org, 56025 <at> debbugs.gnu.org, spwhitton <at> email.arizona.edu,
> kbrown <at> cornell.edu
> From: Jim Porter <jporterbugs <at> gmail.com>
> Date: Sat, 25 Jun 2022 09:13:46 -0700
>
> >> When [EOF is] received, all the bytes waiting to be read are
> >> immediately passed to the process without waiting for a <newline>, and
> >> the EOF is discarded. Thus, if there are no bytes waiting (that is,
> >> the EOF occurred at the beginning of a line), a byte count of zero
> >> shall be returned from the read(), representing an end-of-file
> >> indication.
> >
> > Perhaps some reference to this should also be in the comments.
>
> How about this patch? I added a reference to the specific section of the
> POSIX specification so that it's (hopefully) easy for people to look up
> if they need further details.
LGTM, thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 26 Jun 2022 16:28:02 GMT)
Full text and
rfc822 format available.
Message #74 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Eli Zaretskii <eliz <at> gnu.org> writes:
>> How about this patch? I added a reference to the specific section of the
>> POSIX specification so that it's (hopefully) easy for people to look up
>> if they need further details.
>
> LGTM, thanks.
Pushed to Emacs 29 now.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
bug marked as fixed in version 29.1, send any further explanations to
56025 <at> debbugs.gnu.org and Ken Brown <kbrown <at> cornell.edu>
Request was from
Lars Ingebrigtsen <larsi <at> gnus.org>
to
control <at> debbugs.gnu.org.
(Sun, 26 Jun 2022 16:28:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 26 Jun 2022 17:13:02 GMT)
Full text and
rfc822 format available.
Message #79 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Hello,
On Fri 24 Jun 2022 at 04:03pm -07, Jim Porter wrote:
> When [EOF is] received, all the bytes waiting to be read are
> immediately passed to the process without waiting for a <newline>, and
> the EOF is discarded. Thus, if there are no bytes waiting (that is,
> the EOF occurred at the beginning of a line), a byte count of zero
> shall be returned from the read(), representing an end-of-file
> indication.
>
> I interpret that to mean that the preferred way to indicate end-of-file
> to `rev' in this case is to send it "hi [NL] [EOF]". The second EOF that
> Eshell sends when closing the stderr output handle is superfluous, but
> it works fine as far as I can tell.
>
> [1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap11.html
The text states unconditionally that when an EOF is received it is
discarded by the OS. So we can infer that it's fine to send three,
according to the standard -- it's not just that it happens to work.
Thanks again for working on this.
--
Sean Whitton
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 26 Jun 2022 17:23:01 GMT)
Full text and
rfc822 format available.
Message #82 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/26/2022 10:12 AM, Sean Whitton wrote:
> On Fri 24 Jun 2022 at 04:03pm -07, Jim Porter wrote:
>
>> When [EOF is] received, all the bytes waiting to be read are
>> immediately passed to the process without waiting for a <newline>, and
>> the EOF is discarded. Thus, if there are no bytes waiting (that is,
>> the EOF occurred at the beginning of a line), a byte count of zero
>> shall be returned from the read(), representing an end-of-file
>> indication.
>>
>> I interpret that to mean that the preferred way to indicate end-of-file
>> to `rev' in this case is to send it "hi [NL] [EOF]". The second EOF that
>> Eshell sends when closing the stderr output handle is superfluous, but
>> it works fine as far as I can tell.
>>
>> [1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap11.html
>
> The text states unconditionally that when an EOF is received it is
> discarded by the OS. So we can infer that it's fine to send three,
> according to the standard -- it's not just that it happens to work.
>
> Thanks again for working on this.
Ah, good catch. I glossed over the last sentence in that paragraph in
the spec (hence why I didn't copy-paste it).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 26 Jun 2022 21:12:02 GMT)
Full text and
rfc822 format available.
Message #85 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Hello,
On Sun 26 Jun 2022 at 10:22AM -07, Jim Porter wrote:
> On 6/26/2022 10:12 AM, Sean Whitton wrote:
>> On Fri 24 Jun 2022 at 04:03pm -07, Jim Porter wrote:
>>
>>> When [EOF is] received, all the bytes waiting to be read are
>>> immediately passed to the process without waiting for a <newline>, and
>>> the EOF is discarded. Thus, if there are no bytes waiting (that is,
>>> the EOF occurred at the beginning of a line), a byte count of zero
>>> shall be returned from the read(), representing an end-of-file
>>> indication.
>>>
>>> I interpret that to mean that the preferred way to indicate end-of-file
>>> to `rev' in this case is to send it "hi [NL] [EOF]". The second EOF that
>>> Eshell sends when closing the stderr output handle is superfluous, but
>>> it works fine as far as I can tell.
>>>
>>> [1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap11.html
>>
>> The text states unconditionally that when an EOF is received it is
>> discarded by the OS. So we can infer that it's fine to send three,
>> according to the standard -- it's not just that it happens to work.
>>
>> Thanks again for working on this.
>
> Ah, good catch. I glossed over the last sentence in that paragraph in
> the spec (hence why I didn't copy-paste it).
I was actually thinking it was implied by the first sentence of what you
quoted.
--
Sean Whitton
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 27 Jun 2022 13:26:01 GMT)
Full text and
rfc822 format available.
Message #88 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Thanks to all of you for working on this while I was gone. Unfortunately, the
problem is still present on Cygwin. In my haste to get away, I neglected to
mention that there is apparently a timing issue in Eshell on Cygwin, so that
even three EOFs do not always suffice to kill the process.
My test case is to run
echo bar | sh -c rev
in Eshell. For reasons I don't understand, EOF almost always has to be sent
more than 3 times times before the "sh" process dies. The maximum I've observed
is 93. Inserting "(sit-for 0.01)" after each EOF eliminates the need for extra
EOFs; this is why I referred to the problem as a timing issue.
I propose the following workaround:
--- a/lisp/eshell/esh-io.el
+++ b/lisp/eshell/esh-io.el
@@ -284,10 +284,16 @@ eshell-close-target
;; end-of-file to the reading process. However, some platforms
;; (e.g. Solaris) actually require sending a *third* EOF. Since
;; sending extra EOFs while the process is running shouldn't break
- ;; anything, we'll just send the maximum we'd ever need. See
- ;; bug#56025 for further details.
- (let ((i 0))
- (while (and (<= (cl-incf i) 3)
+ ;; anything, we'll send up to three on all platforms.
+
+ ;; There's an extra wrinkle on Cygwin where, apparently due to an
+ ;; unknown timing issue, it sometimes takes more than three EOFs
+ ;; to kill the process. (This only happens in Eshell, not in an
+ ;; ordinary Cygwin shell.) We work around this problem by sending
+ ;; up to 1000 EOFs on Cygwin. See bug#56025 for further details.
+ (let ((i 0)
+ (n (if (eq system-type 'cygwin) 1000 3)))
+ (while (and (<= (cl-incf i) n)
(eq (process-status target) 'run))
(process-send-eof target))))
Ken
P.S. Has anyone checked to see if the extpipe tests are now passing on EMBA? If
not, maybe the workaround is needed there too. Alternatively, we could simply
use the workaround on all platforms. I don't see what harm it could do.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 27 Jun 2022 15:52:02 GMT)
Full text and
rfc822 format available.
Message #91 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Ken Brown <kbrown <at> cornell.edu> writes:
Hi Ken,
> P.S. Has anyone checked to see if the extpipe tests are now passing on
> EMBA?
Seems to work there. <https://emba.gnu.org/emacs/emacs/-/jobs/48285>
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 27 Jun 2022 16:23:01 GMT)
Full text and
rfc822 format available.
Message #94 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/27/2022 11:51 AM, Michael Albinus wrote:>> P.S. Has anyone checked to see
if the extpipe tests are now passing on
>> EMBA?
>
> Seems to work there. <https://emba.gnu.org/emacs/emacs/-/jobs/48285>
Thanks, Michael. In that case we should probably go ahead with the Cygwin fix,
but I'll wait to hear from Eli or Lars.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 27 Jun 2022 19:14:01 GMT)
Full text and
rfc822 format available.
Message #97 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Hello,
On Mon 27 Jun 2022 at 12:22pm -04, Ken Brown wrote:
> External Email
>
> On 6/27/2022 11:51 AM, Michael Albinus wrote:>> P.S. Has anyone checked to see
> if the extpipe tests are now passing on
>>> EMBA?
>>
>> Seems to work there. <https://emba.gnu.org/emacs/emacs/-/jobs/48285>
>
> Thanks, Michael. In that case we should probably go ahead with the Cygwin fix,
> but I'll wait to hear from Eli or Lars.
The three times fix or the thousand times fix, do you mean?
--
Sean Whitton
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 27 Jun 2022 19:19:01 GMT)
Full text and
rfc822 format available.
Message #100 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/27/2022 6:25 AM, Ken Brown wrote:
> Thanks to all of you for working on this while I was gone.
> Unfortunately, the problem is still present on Cygwin. In my haste to
> get away, I neglected to mention that there is apparently a timing issue
> in Eshell on Cygwin, so that even three EOFs do not always suffice to
> kill the process.
>
> My test case is to run
>
> echo bar | sh -c rev
>
> in Eshell. For reasons I don't understand, EOF almost always has to be
> sent more than 3 times times before the "sh" process dies. The maximum
> I've observed is 93. Inserting "(sit-for 0.01)" after each EOF
> eliminates the need for extra EOFs; this is why I referred to the
> problem as a timing issue.
>
> I propose the following workaround:
>
> --- a/lisp/eshell/esh-io.el
> +++ b/lisp/eshell/esh-io.el
> @@ -284,10 +284,16 @@ eshell-close-target
> ;; end-of-file to the reading process. However, some platforms
> ;; (e.g. Solaris) actually require sending a *third* EOF. Since
> ;; sending extra EOFs while the process is running shouldn't break
> - ;; anything, we'll just send the maximum we'd ever need. See
> - ;; bug#56025 for further details.
> - (let ((i 0))
> - (while (and (<= (cl-incf i) 3)
> + ;; anything, we'll send up to three on all platforms.
> +
> + ;; There's an extra wrinkle on Cygwin where, apparently due to an
> + ;; unknown timing issue, it sometimes takes more than three EOFs
> + ;; to kill the process. (This only happens in Eshell, not in an
> + ;; ordinary Cygwin shell.) We work around this problem by sending
> + ;; up to 1000 EOFs on Cygwin. See bug#56025 for further details.
> + (let ((i 0)
> + (n (if (eq system-type 'cygwin) 1000 3)))
> + (while (and (<= (cl-incf i) n)
> (eq (process-status target) 'run))
> (process-send-eof target))))
I'd be very hesitant to do this, since as you mention above, this seems
like a timing issue, and it's entirely possible that there are other,
more widespread issues on Cygwin here. We'd also want to check the
system that the process is actually running on; otherwise, remoting into
a Cygwin system (via Tramp) would still exhibit the problem. I'll see if
I can get a Cygwin environment up to test things out in the next week-ish.
If there's no other way that we can come up with here, I'd lean towards
a defcustom so that users can tweak this if needed.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 27 Jun 2022 21:18:01 GMT)
Full text and
rfc822 format available.
Message #103 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/27/2022 3:13 PM, Sean Whitton wrote:
> Hello,
>
> On Mon 27 Jun 2022 at 12:22pm -04, Ken Brown wrote:
>
>> External Email
>>
>> On 6/27/2022 11:51 AM, Michael Albinus wrote:>> P.S. Has anyone checked to see
>> if the extpipe tests are now passing on
>>>> EMBA?
>>>
>>> Seems to work there. <https://emba.gnu.org/emacs/emacs/-/jobs/48285>
>>
>> Thanks, Michael. In that case we should probably go ahead with the Cygwin fix,
>> but I'll wait to hear from Eli or Lars.
>
> The three times fix or the thousand times fix, do you mean?
I meant the fix that's 1000 on Cygwin and 3 elsewhere. But let's see if Jim
comes up with something better.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 27 Jun 2022 21:21:01 GMT)
Full text and
rfc822 format available.
Message #106 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/27/2022 3:18 PM, Jim Porter wrote:
> On 6/27/2022 6:25 AM, Ken Brown wrote:
>> Thanks to all of you for working on this while I was gone. Unfortunately, the
>> problem is still present on Cygwin. In my haste to get away, I neglected to
>> mention that there is apparently a timing issue in Eshell on Cygwin, so that
>> even three EOFs do not always suffice to kill the process.
>>
>> My test case is to run
>>
>> echo bar | sh -c rev
>>
>> in Eshell. For reasons I don't understand, EOF almost always has to be sent
>> more than 3 times times before the "sh" process dies. The maximum I've
>> observed is 93. Inserting "(sit-for 0.01)" after each EOF eliminates the need
>> for extra EOFs; this is why I referred to the problem as a timing issue.
>>
>> I propose the following workaround:
>>
>> --- a/lisp/eshell/esh-io.el
>> +++ b/lisp/eshell/esh-io.el
>> @@ -284,10 +284,16 @@ eshell-close-target
>> ;; end-of-file to the reading process. However, some platforms
>> ;; (e.g. Solaris) actually require sending a *third* EOF. Since
>> ;; sending extra EOFs while the process is running shouldn't break
>> - ;; anything, we'll just send the maximum we'd ever need. See
>> - ;; bug#56025 for further details.
>> - (let ((i 0))
>> - (while (and (<= (cl-incf i) 3)
>> + ;; anything, we'll send up to three on all platforms.
>> +
>> + ;; There's an extra wrinkle on Cygwin where, apparently due to an
>> + ;; unknown timing issue, it sometimes takes more than three EOFs
>> + ;; to kill the process. (This only happens in Eshell, not in an
>> + ;; ordinary Cygwin shell.) We work around this problem by sending
>> + ;; up to 1000 EOFs on Cygwin. See bug#56025 for further details.
>> + (let ((i 0)
>> + (n (if (eq system-type 'cygwin) 1000 3)))
>> + (while (and (<= (cl-incf i) n)
>> (eq (process-status target) 'run))
>> (process-send-eof target))))
>
> I'd be very hesitant to do this, since as you mention above, this seems like a
> timing issue, and it's entirely possible that there are other, more widespread
> issues on Cygwin here. We'd also want to check the system that the process is
> actually running on; otherwise, remoting into a Cygwin system (via Tramp) would
> still exhibit the problem. I'll see if I can get a Cygwin environment up to test
> things out in the next week-ish.
OK, thanks. Let me know if you need any help with that.
> If there's no other way that we can come up with here, I'd lean towards a
> defcustom so that users can tweak this if needed.
Sounds good.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 01 Jul 2022 03:54:02 GMT)
Full text and
rfc822 format available.
Message #109 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/27/2022 2:19 PM, Ken Brown wrote:
> On 6/27/2022 3:18 PM, Jim Porter wrote:
>> I'd be very hesitant to do this, since as you mention above, this
>> seems like a timing issue, and it's entirely possible that there are
>> other, more widespread issues on Cygwin here. We'd also want to check
>> the system that the process is actually running on; otherwise,
>> remoting into a Cygwin system (via Tramp) would still exhibit the
>> problem. I'll see if I can get a Cygwin environment up to test things
>> out in the next week-ish.
>
> OK, thanks. Let me know if you need any help with that.
Ok, I've got Cygwin set up (though I'm just using the prebuilt Cygwin
Emacs for now). I can confirm that the following hangs until I send
another EOF via `C-c C-d':
echo hi | rev
However, if I evaluate the following first, the above command works just
fine:
(add-to-list 'eshell-needs-pipe "rev")
Normally, Eshell starts each process using ptys to control them.
However, the above Elisp code tells Eshell to use a pipe for "rev"[1].
I'm not totally clear on all the subtleties here, but it seems to me
that it would make more sense for rev to use a pipe for its stdin and a
pty for its stdout. That's not possible with subprocesses in Emacs
though (as far as I know).
However, I don't think this fully answers things, since I also see
inconsistent results if I run "echo hi | rev" a bunch of times.
Sometimes it prints "ih" and then I need to hit `C-c C-d` once to stop
it. Other times it doesn't print anything and I need to hit `C-c C-d'
twice. There must be some kind of race condition, maybe in Emacs's
src/process.c?
Hopefully this helps get us closer to a proper answer here though...
[1] Only when rev is being piped *to* in an Eshell pipeline.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 01 Jul 2022 03:59:02 GMT)
Full text and
rfc822 format available.
Message #112 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/30/2022 8:52 PM, Jim Porter wrote:
> Normally, Eshell starts each process using ptys to control them.
> However, the above Elisp code tells Eshell to use a pipe for "rev"[1].
> I'm not totally clear on all the subtleties here, but it seems to me
> that it would make more sense for rev to use a pipe for its stdin and a
> pty for its stdout. That's not possible with subprocesses in Emacs
> though (as far as I know).
Oh, I forgot to mention that bug#56013 might be related to this in some
way, since it exhibits some problems with ptys when used in Eshell
pipelines too.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Wed, 06 Jul 2022 22:34:01 GMT)
Full text and
rfc822 format available.
Message #115 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 6/30/2022 11:52 PM, Jim Porter wrote:
> On 6/27/2022 2:19 PM, Ken Brown wrote:
>> On 6/27/2022 3:18 PM, Jim Porter wrote:
>>> I'd be very hesitant to do this, since as you mention above, this seems like
>>> a timing issue, and it's entirely possible that there are other, more
>>> widespread issues on Cygwin here. We'd also want to check the system that the
>>> process is actually running on; otherwise, remoting into a Cygwin system (via
>>> Tramp) would still exhibit the problem. I'll see if I can get a Cygwin
>>> environment up to test things out in the next week-ish.
>>
>> OK, thanks. Let me know if you need any help with that.
>
> Ok, I've got Cygwin set up (though I'm just using the prebuilt Cygwin Emacs for
> now). I can confirm that the following hangs until I send another EOF via `C-c
> C-d':
>
> echo hi | rev
Yes, but that's because of the behavior of certain platforms (e.g., Cygwin and
Solaris) with respect to EOF, as I said in an earlier message. We've changed
that for Cygwin, so that Cygwin now behaves the same as GNU/Linux, but the
change won't take effect until Cygwin 3.4.0 is released. In any case, that
issue has already been fixed on the master branch.
Aside from that issue, I never had an issue with
echo hi | rev
but only with
echo hi | sh -c rev
I have no idea why that should be different.
> However, if I evaluate the following first, the above command works just fine:
>
> (add-to-list 'eshell-needs-pipe "rev")
>
> Normally, Eshell starts each process using ptys to control them. However, the
> above Elisp code tells Eshell to use a pipe for "rev"[1].
That makes sense. You're no longer relying on Eshell sending EOF to rev, but
rather you're letting rev discover EOF because no process holds the pipe open
for writing, forcing any pending read to stop blocking.
And, for the same reason, evaluating
(add-to-list 'eshell-needs-pipe "sh")
solves the problem with "echo hi | sh -c rev".
> I'm not totally clear
> on all the subtleties here, but it seems to me that it would make more sense for
> rev to use a pipe for its stdin and a pty for its stdout. That's not possible
> with subprocesses in Emacs though (as far as I know).
>
> However, I don't think this fully answers things, since I also see inconsistent
> results if I run "echo hi | rev" a bunch of times. Sometimes it prints "ih" and
> then I need to hit `C-c C-d` once to stop it. Other times it doesn't print
> anything and I need to hit `C-c C-d' twice.
Interesting. I've never seen that. It's as though "rev" just didn't get one of
the EOFs.
> There must be some kind of race
> condition, maybe in Emacs's src/process.c?
I'll poke around, but I'm no expert on how this all works.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Thu, 07 Jul 2022 04:36:01 GMT)
Full text and
rfc822 format available.
Message #118 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/6/2022 3:33 PM, Ken Brown wrote:
> On 6/30/2022 11:52 PM, Jim Porter wrote:
>> Ok, I've got Cygwin set up (though I'm just using the prebuilt Cygwin
>> Emacs for now). I can confirm that the following hangs until I send
>> another EOF via `C-c C-d':
>>
>> echo hi | rev
>
> Yes, but that's because of the behavior of certain platforms (e.g.,
> Cygwin and Solaris) with respect to EOF, as I said in an earlier
> message.
Yeah, I think that's fine, and the change to send an extra EOF for
compatibility with (non-master) Cygwin and Solaris makes sense to me. I
just wanted to be sure to mention that I could see the issue too so that
I can (hopefully) verify that it's fixed if/when we come up with a
more-reliable fix.
>> However, if I evaluate the following first, the above command works
>> just fine:
>>
>> (add-to-list 'eshell-needs-pipe "rev")
>>
>> Normally, Eshell starts each process using ptys to control them.
>> However, the above Elisp code tells Eshell to use a pipe for "rev"[1].
>
> That makes sense. You're no longer relying on Eshell sending EOF to
> rev, but rather you're letting rev discover EOF because no process holds
> the pipe open for writing, forcing any pending read to stop blocking.
Maybe it would be good to do it this way in general though, since this
would let us completely avoid the behavioral differences of EOF on
various platforms. I believe using a pipe should work consistently
everywhere, right? (It would also probably fix some other issues with
Eshell pipelines, but I'll need to read up on ptys, since it's been a
long time since I've done anything with them.)
>> However, I don't think this fully answers things, since I also see
>> inconsistent results if I run "echo hi | rev" a bunch of times.
>> Sometimes it prints "ih" and then I need to hit `C-c C-d` once to stop
>> it. Other times it doesn't print anything and I need to hit `C-c C-d'
>> twice.
>
> Interesting. I've never seen that. It's as though "rev" just didn't
> get one of the EOFs.
Yeah, that's what it seems like to me too. I'm not able to reproduce
this on GNU/Linux (at least not yet; I'll try some more things out).
I'll keep poking at the Cygwin version too, and start experimenting with
Emacs's src/process.c to try and allow using a pty for only stdin *or*
stdout (instead of both). I think that would make Eshell's pipelines
behaves more like other shells, which would squash a lot of bugs in this
area.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Thu, 07 Jul 2022 04:43:01 GMT)
Full text and
rfc822 format available.
Message #121 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/6/2022 9:35 PM, Jim Porter wrote:
> Maybe it would be good to do it this way in general though, since this
> would let us completely avoid the behavioral differences of EOF on
> various platforms. I believe using a pipe should work consistently
> everywhere, right? (It would also probably fix some other issues with
> Eshell pipelines, but I'll need to read up on ptys, since it's been a
> long time since I've done anything with them.)
Just to clarify: by "do it this way in general", I only mean using a
pipe when connecting commands in Eshell via "|", not using
`eshell-needs-pipe'. This would necessitate enhancing `make-process' and
friends to support what I described elsewhere.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Thu, 07 Jul 2022 12:43:01 GMT)
Full text and
rfc822 format available.
Message #124 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/7/2022 12:42 AM, Jim Porter wrote:
> On 7/6/2022 9:35 PM, Jim Porter wrote:
>> Maybe it would be good to do it this way in general though, since this would
>> let us completely avoid the behavioral differences of EOF on various
>> platforms. I believe using a pipe should work consistently everywhere, right?
>> (It would also probably fix some other issues with Eshell pipelines, but I'll
>> need to read up on ptys, since it's been a long time since I've done anything
>> with them.)
>
> Just to clarify: by "do it this way in general", I only mean using a pipe when
> connecting commands in Eshell via "|", not using `eshell-needs-pipe'. This would
> necessitate enhancing `make-process' and friends to support what I described
> elsewhere.
That makes sense to me.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 17 Jul 2022 02:36:02 GMT)
Full text and
rfc822 format available.
Message #127 received at 56025 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 7/7/2022 5:42 AM, Ken Brown wrote:
> On 7/7/2022 12:42 AM, Jim Porter wrote:
>> On 7/6/2022 9:35 PM, Jim Porter wrote:
>>> Maybe it would be good to do it this way in general though, since
>>> this would let us completely avoid the behavioral differences of EOF
>>> on various platforms. I believe using a pipe should work consistently
>>> everywhere, right? (It would also probably fix some other issues with
>>> Eshell pipelines, but I'll need to read up on ptys, since it's been a
>>> long time since I've done anything with them.)
>>
>> Just to clarify: by "do it this way in general", I only mean using a
>> pipe when connecting commands in Eshell via "|", not using
>> `eshell-needs-pipe'. This would necessitate enhancing `make-process'
>> and friends to support what I described elsewhere.
>
> That makes sense to me.
Ok, attached is a WIP patch to do this. It seems to work for me under
Cygwin, although I've only lightly tested it in that environment. If
this works for you too, I'll finish cleaning this up and add
tests/documentation for it.
Note that in my patch, I temporarily undid my previous patch to send EOF
multiple times. This is just for testing purposes, but since we're using
a pipe for this connection now, a single call to `process-send-eof'
should be sufficient. (There are some obscure cases where we might want
to keep the current behavior, like redirecting to a process created some
other way, so I think it makes sense to keep that code. Probably...)
[0001-WIP-Allow-using-PTYs-for-just-stdin-or-stdout-in-mak.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 17 Jul 2022 06:04:01 GMT)
Full text and
rfc822 format available.
Message #130 received at 56025 <at> debbugs.gnu.org (full text, mbox):
> Cc: larsi <at> gnus.org, 56025 <at> debbugs.gnu.org
> From: Jim Porter <jporterbugs <at> gmail.com>
> Date: Sat, 16 Jul 2022 19:35:12 -0700
>
> Ok, attached is a WIP patch to do this. It seems to work for me under
> Cygwin, although I've only lightly tested it in that environment. If
> this works for you too, I'll finish cleaning this up and add
> tests/documentation for it.
>
> Note that in my patch, I temporarily undid my previous patch to send EOF
> multiple times. This is just for testing purposes, but since we're using
> a pipe for this connection now, a single call to `process-send-eof'
> should be sufficient. (There are some obscure cases where we might want
> to keep the current behavior, like redirecting to a process created some
> other way, so I think it makes sense to keep that code. Probably...)
Could you please describe the main ideas of the changeset? It is hard
to be sure I understand what you are trying to do by reading the
patch.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 17 Jul 2022 17:45:01 GMT)
Full text and
rfc822 format available.
Message #133 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/16/2022 11:03 PM, Eli Zaretskii wrote:
> Could you please describe the main ideas of the changeset? It is hard
> to be sure I understand what you are trying to do by reading the
> patch.
Sure. I mentioned it briefly earlier in the thread, but a more-complete
summary would probably help.
Normally, Eshell connects programs in a pipeline like "foo | bar" by
setting a process filter for "foo", and inside that filter, (eventually)
calling `process-send-string' for "bar". In most shells, you'd expect
that connection to be a pipe, but in Eshell, the processes are created
with a PTY connection by default.
My patch adds support for `make-process' to use a PTY only for the child
process's stdin or its stdout (in addition to the preexisting behaviors
of PTY for both or neither). This then lets Eshell request a pipe for
foo's stdout and bar's stdin, while using PTYs for foo's stdin and bar's
stdout:
Before:
[pty 1] -> foo -> [pty 1] -> Eshell -> [pty 2] -> bar -> [pty 2]
After:
[pty 1] -> foo -> [pipe] -> Eshell -> [pipe] -> bar -> [pty 2]
This should make Eshell behave quite a bit more similarly to other
shells, which will hopefully reduce the number of bugs like this one.
This change also allowed me to remove the workaround for bug#1388. In
that bug, there was an issue where this command didn't work[1]:
*echo 1+1 | bc
Before the fix for bug#1388, "bc" would have seen that its stdin was a
PTY, and then started an interactive session. Bug#1388 fixed this by
adding `eshell-needs-pipe-p' to identify specific programs that need a
pipe connection when being piped to like the above. With my patch here,
that workaround won't be necessary anymore, since programs in a pipeline
will be connected via pipes. (Note that technically, this pipe
connection is indirect, since there's one pipe from foo to Emacs, and
another pipe from Emacs to bar.)
This patch should hopefully fix the issues on Cygwin (as described in
this bug) because, when using pipes to connect programs, the behavior
should be more consistent across multiple platforms.
[1] I'm using "*echo" here to use /bin/echo so that it writes a newline.
Eshell's built-in echo doesn't write a newline by default.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 17 Jul 2022 18:28:02 GMT)
Full text and
rfc822 format available.
Message #136 received at 56025 <at> debbugs.gnu.org (full text, mbox):
> Cc: larsi <at> gnus.org, 56025 <at> debbugs.gnu.org, spwhitton <at> email.arizona.edu,
> kbrown <at> cornell.edu
> From: Jim Porter <jporterbugs <at> gmail.com>
> Date: Sun, 17 Jul 2022 10:44:26 -0700
>
> My patch adds support for `make-process' to use a PTY only for the child
> process's stdin or its stdout (in addition to the preexisting behaviors
> of PTY for both or neither). This then lets Eshell request a pipe for
> foo's stdout and bar's stdin, while using PTYs for foo's stdin and bar's
> stdout:
>
> Before:
> [pty 1] -> foo -> [pty 1] -> Eshell -> [pty 2] -> bar -> [pty 2]
>
> After:
> [pty 1] -> foo -> [pipe] -> Eshell -> [pipe] -> bar -> [pty 2]
This assumes that we never want foo to behave as it does when
displaying on a terminal device. Are we sure we will never want that?
E.g., what about the equivalent of "fgrep ... | less" -- don't we want
fgrep to produce colorized output as it does when it writes to a
terminal device?
Perhaps the use of pipes should be controllable?
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 17 Jul 2022 18:52:02 GMT)
Full text and
rfc822 format available.
Message #139 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/17/2022 11:26 AM, Eli Zaretskii wrote:
>> Cc: larsi <at> gnus.org, 56025 <at> debbugs.gnu.org, spwhitton <at> email.arizona.edu,
>> kbrown <at> cornell.edu
>> From: Jim Porter <jporterbugs <at> gmail.com>
>> Date: Sun, 17 Jul 2022 10:44:26 -0700
>>
>> My patch adds support for `make-process' to use a PTY only for the child
>> process's stdin or its stdout (in addition to the preexisting behaviors
>> of PTY for both or neither). This then lets Eshell request a pipe for
>> foo's stdout and bar's stdin, while using PTYs for foo's stdin and bar's
>> stdout:
>>
>> Before:
>> [pty 1] -> foo -> [pty 1] -> Eshell -> [pty 2] -> bar -> [pty 2]
>>
>> After:
>> [pty 1] -> foo -> [pipe] -> Eshell -> [pipe] -> bar -> [pty 2]
>
> This assumes that we never want foo to behave as it does when
> displaying on a terminal device. Are we sure we will never want that?
> E.g., what about the equivalent of "fgrep ... | less" -- don't we want
> fgrep to produce colorized output as it does when it writes to a
> terminal device?
Well, for something like fgrep, the usual way to do this in a regular
shell would be "fgrep --color=always ... | less", which should work the
same in Eshell. There are a few caveats to this though:
1. "fgrep" is actually a built-in Eshell command that opens a
compilation buffer and runs "grep -F ..." in it, so piping it to "less"
normally isn't necessary. Still, you could always use the external fgrep
program by specifying the full path or saying "*fgrep" though.
2. External commands see Eshell as a dumb terminal, and so they usually
won't colorize their output in the first place without the user forcing
it. Piping to "less" doesn't change the situation there.
3. Piping to "less" is probably going to have problems, even with this
change. Eshell considers less to be a "visual command", so it opens it
up in an M-x term buffer (and I don't think the Eshell->term code is
able to support pipelines like this yet). Even if that were fixed, I
think it would be tricky to get less working in Eshell. That said, I
have a plan to make a built-in version of less for Eshell written in
Elisp that should do pretty much what Eshell users would expect. This is
a complex project though (I started it in February!), and I have a few
more preliminary changes to make to Eshell to make this easier to do.
> Perhaps the use of pipes should be controllable?
However, with all the above said, I think we *do* want the use of pipes
to be controllable in at least some cases. For example, due to the
differences between Eshell and regular shells, commands like "xdg-open"
don't work properly (this is bug#56013). It would be nice if Eshell
could make those commands Just Work, but I'm not sure that's feasible
given how Eshell works. I think the most straightforward way to resolve
that would be to declare "xdg-open" (and similar commands) as *always*
using pipes, no matter what. Maybe there are commands that always want a
PTY too.
It wouldn't be too hard to have a mapping from command names to
connection-types that would handle this. It would be sort of like the
`eshell-needs-pipe-p' code that I removed in my WIP patch, but with
finer-grained control.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 17 Jul 2022 22:00:02 GMT)
Full text and
rfc822 format available.
Message #142 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/16/2022 10:35 PM, Jim Porter wrote:
> Ok, attached is a WIP patch to do this. It seems to work for me under Cygwin,
> although I've only lightly tested it in that environment. If this works for you
> too, I'll finish cleaning this up and add tests/documentation for it.
It does work for me too. Thanks!
> Note that in my patch, I temporarily undid my previous patch to send EOF
> multiple times. This is just for testing purposes, but since we're using a pipe
> for this connection now, a single call to `process-send-eof' should be
> sufficient.
There shouldn't be a need for any calls to process-send-eof. This is a noop
anyway when writing to a pipe, as it should be. A process reading from a pipe
automatically recognizes EOF when a read returns 0 bytes, which is supposed to
happen when no process has the pipe open for writing.
> (There are some obscure cases where we might want to keep the
> current behavior, like redirecting to a process created some other way, so I
> think it makes sense to keep that code. Probably...)
Ideally, Eshell should know whether it's writing to a pipe or a pty. It should
send up to 3 EOFs in the latter case and 0 in the former case. If it's too hard
to arrange that, then it's probably harmless to send up to 3 EOFs in the pipe
case too. But then maybe a comment in the code would be useful, so that readers
don't wonder why you're sending EOF to a pipe.
Thanks again for your work.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 18 Jul 2022 05:27:02 GMT)
Full text and
rfc822 format available.
Message #145 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/17/2022 2:59 PM, Ken Brown wrote:
> On 7/16/2022 10:35 PM, Jim Porter wrote:
>> Ok, attached is a WIP patch to do this. It seems to work for me under
>> Cygwin, although I've only lightly tested it in that environment. If
>> this works for you too, I'll finish cleaning this up and add
>> tests/documentation for it.
>
> It does work for me too. Thanks!
Great! This should make Eshell behave a bit more similarly to other
shells, so hopefully this will help prevent other issues in this area.
>> Note that in my patch, I temporarily undid my previous patch to send
>> EOF multiple times. This is just for testing purposes, but since we're
>> using a pipe for this connection now, a single call to
>> `process-send-eof' should be sufficient.
>
> There shouldn't be a need for any calls to process-send-eof. This is a
> noop anyway when writing to a pipe, as it should be.
Looking at the implementation of `process-send-eof', I think it's
(somewhat misleadingly) also responsible for closing the file descriptor
when writing to a pipe, so I believe we'll need at least one call to
that function.
As you say though, it shouldn't be too hard for Eshell to check whether
it's writing to a pipe and only try to call `process-send-eof' once in
that case.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Mon, 18 Jul 2022 08:10:02 GMT)
Full text and
rfc822 format available.
Message #148 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Jim Porter <jporterbugs <at> gmail.com> writes:
Hi Jim,
> Sure. I mentioned it briefly earlier in the thread, but a
> more-complete summary would probably help.
>
> Normally, Eshell connects programs in a pipeline like "foo | bar" by
> setting a process filter for "foo", and inside that filter,
> (eventually) calling `process-send-string' for "bar". In most shells,
> you'd expect that connection to be a pipe, but in Eshell, the
> processes are created with a PTY connection by default.
>
> My patch adds support for `make-process' to use a PTY only for the
> child process's stdin or its stdout (in addition to the preexisting
> behaviors of PTY for both or neither). This then lets Eshell request a
> pipe for foo's stdout and bar's stdin, while using PTYs for foo's
> stdin and bar's stdout:
>
> Before:
> [pty 1] -> foo -> [pty 1] -> Eshell -> [pty 2] -> bar -> [pty 2]
>
> After:
> [pty 1] -> foo -> [pipe] -> Eshell -> [pipe] -> bar -> [pty 2]
I haven't tested, but it looks like it won't work with remote processes
foo and bar. Something like
--8<---------------cut here---------------start------------->8---
~ $ cd /ssh:remotehost:
/ssh:remotehost:~ $ *ls | *grep a
--8<---------------cut here---------------end--------------->8---
I have no idea whether such a usage pattern makes sense, but it seems to
work ATM. The alternative, calling "*ls *| *grep a", works as well, but
there are different results.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Tue, 19 Jul 2022 01:59:02 GMT)
Full text and
rfc822 format available.
Message #151 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/18/2022 1:09 AM, Michael Albinus wrote:
> I haven't tested, but it looks like it won't work with remote processes
> foo and bar. Something like
>
> --8<---------------cut here---------------start------------->8---
> ~ $ cd /ssh:remotehost:
> /ssh:remotehost:~ $ *ls | *grep a
> --8<---------------cut here---------------end--------------->8---
Hmm, yeah. We'll need to do something with the Tramp case. I think the
minimal fix would be to update `tramp-sh-handle-make-process' so that it
doesn't signal an error when :connection-type is a cons cell. That's
pretty easy.
However, it seems that Tramp doesn't fully support :connection-type yet
(testing on both Emacs 28.1 and 29). I used the following Python script,
and tried a few varieties of `make-process' calls over Tramp. I'd expect
that when I set :connection-type to `pipe', it would print False for all
the streams, but it seems that the script always sees stdin and stdout
as TTYs no matter what options I use.
------------------------------
#!/usr/bin/env python3
import sys
print('stdin: {}\nstdout: {}\nstderr: {}\n'.format(
sys.stdin.isatty(),
sys.stdout.isatty(),
sys.stderr.isatty()
))
------------------------------
On the other hand, if I set :stderr to be a buffer, then the script
*does* see stderr as a pipe, but that makes sense looking at the code:
when using :stderr, Tramp redirects the process's stderr to a file (and
then cats it back out).
Maybe there's a relatively easy way to get Tramp to respect
:connection-type, but I tried a couple of things and they didn't work
quite right. Still, that could be a followup for later. So long as it
doesn't error out immediately when it's a cons cell, I think it would be ok.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Tue, 19 Jul 2022 08:01:02 GMT)
Full text and
rfc822 format available.
Message #154 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Jim Porter <jporterbugs <at> gmail.com> writes:
Hi Jim,
>> I haven't tested, but it looks like it won't work with remote processes
>> foo and bar. Something like
>> --8<---------------cut here---------------start------------->8---
>> ~ $ cd /ssh:remotehost:
>> /ssh:remotehost:~ $ *ls | *grep a
>> --8<---------------cut here---------------end--------------->8---
>
> Hmm, yeah. We'll need to do something with the Tramp case. I think the
> minimal fix would be to update `tramp-sh-handle-make-process' so that
> it doesn't signal an error when :connection-type is a cons
> cell. That's pretty easy.
FTR, there are three different implementations of `make-process' in
Tramp. A fourth one, in tramp-smb.el, waits for implementation.
> However, it seems that Tramp doesn't fully support :connection-type
> yet (testing on both Emacs 28.1 and 29).
Indeed. Tramp just checks that :connection-type is set to a proper
value, that's it. I was never urged to do more :-)
> Maybe there's a relatively easy way to get Tramp to respect
> :connection-type, but I tried a couple of things and they didn't work
> quite right. Still, that could be a followup for later. So long as it
> doesn't error out immediately when it's a cons cell, I think it would
> be ok.
Best would be you write a new bug report for Tramp, handling
:connection-type proper in make-process. At least it should accept the
cons cell (once the API has been stabilized, documented and pushed to
Emacs). And perhaps Tramp could do better in general wrt :connection-type.
Best regards, Michael.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 22 Jul 2022 04:17:02 GMT)
Full text and
rfc822 format available.
Message #157 received at 56025 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 7/17/2022 10:26 PM, Jim Porter wrote:
> On 7/17/2022 2:59 PM, Ken Brown wrote:
>> It does work for me too. Thanks!
>
> Great! This should make Eshell behave a bit more similarly to other
> shells, so hopefully this will help prevent other issues in this area.
Ok, I *think* this is done. The patches have docs/tests that should
hopefully explain everything in detail, but here's a high-level overview:
Patch 1:
--------
Add the ability to pass a cons cell for `:connection-type' to
`make-process'. This lets you specify whether to use a pipe or pty
independently for the input and output of the subprocess. This also
removes the restriction that specifying `:stderr' forces
`:connection-type' to be `pipe'. Now, it only makes stderr use a pipe.
This should be enough to fix the test failures mentioned in this bug,
and should also make Eshell pipelines work more like in other shells:
normally, when executing something like `foo | bar', foo's stdout and
bar's stdin are pipes.[1]
I also removed the `eshell-needs-pipe-p' function since it's not
necessary in its current form anymore. However, a new function along
these lines might help to resolve bug#56013. I looked into this briefly
and it's not terribly complicated, but it would take a bit of work to
get right, so I think it'd be best to do it separately.
Patch 2:
--------
Add the ability to check whether each of a subprocess's `stdin',
`stdout', or `stderr' are TTYs or pipes by passing one of those symbols
as the second argument to `process-tty-name'. This lets us avoid the
"send 3 EOFs" behavior most of the time in Eshell. (Note that if a user
created a subprocess some other way and connected it via Eshell, they
might need the 3 EOFs behavior, hence why I kept that code around.)
I debated whether `process-tty-name' was the right place to do this or
if a new `process-connection-type' function would be better, but I went
with this way in the end. I don't really have a strong preference though.
--------
I added tests for this, and they all pass for me, though admittedly I
didn't run the entire Emacs test suite against these patches yet...
[1] Note that currently, Eshell always pipes both stdout and stderr (see
bug#21605). I'm tinkering with a patch for this too.
[0001-Allow-creating-processes-where-only-one-of-stdin-or-.patch (text/plain, attachment)]
[0002-Add-STREAM-argument-to-process-tty-name.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Fri, 22 Jul 2022 19:01:02 GMT)
Full text and
rfc822 format available.
Message #160 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/22/2022 12:16 AM, Jim Porter wrote:
> Ok, I *think* this is done.
I can confirm that the em-extpipe tests now all pass on Cygwin, as do the
process tests.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 04:06:02 GMT)
Full text and
rfc822 format available.
Message #163 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/22/2022 12:00 PM, Ken Brown wrote:
> On 7/22/2022 12:16 AM, Jim Porter wrote:
>> Ok, I *think* this is done.
>
> I can confirm that the em-extpipe tests now all pass on Cygwin, as do
> the process tests.
Thanks for testing. I'm glad everything seems to be working for you too.
I did a bit more testing on my end to check out performance, since I
figured we'd see at least some improvement from switching to pipes for
passing data between processes. I wasn't prepared for just how much of
an improvement though. On my system (GNU/Linux), this change makes
piping in Eshell faster by a factor of 35x![1]
I'll repeat that since I'm pretty shocked myself: Eshell pipes are
*thirty-five* times faster now!
For some details: I tested this by running "time *cat config.log | wc"
in Eshell (n=20), and it went from an average of 4.80s to an average of
0.134s. (Note that `time' in Eshell only times the first command, not
the whole pipeline.) I chose this to test since config.log is reasonably
big (1.13MiB on my system), the external cat program is pretty simple
and mostly just does I/O, and wc's output is short so we don't have to
worry about it writing a bunch of output to its (slow) PTY.
However, to put just a bit of a damper on things, this is still 5-10x
slower than doing it in Bash, or with Eshell extpipes (which is really
the same as just doing it in Bash, ultimately).
[1] YMMV, of course.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 05:20:02 GMT)
Full text and
rfc822 format available.
Message #166 received at 56025 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
I thought about it some more, and just to be sure the Eshell bits don't
regress at some point in the future, I added some new unit tests in
test/lisp/eshell/esh-proc-tests.el to make sure that Eshell sets the
`:connection-type' properly. (They're pretty similar to the tests in
test/src/process-tests.el, really.)
[0001-Allow-creating-processes-where-only-one-of-stdin-or-.patch (text/plain, attachment)]
[0002-Add-STREAM-argument-to-process-tty-name.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 05:30:01 GMT)
Full text and
rfc822 format available.
Message #169 received at 56025 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 7/23/2022 10:19 PM, Jim Porter wrote:
> I thought about it some more, and just to be sure the Eshell bits don't
> regress at some point in the future, I added some new unit tests in
> test/lisp/eshell/esh-proc-tests.el...
Oops. I forgot to add some `(skip-unless ...)' forms for these tests,
so... here they are. Hopefully this will be the last message from me for
a bit. :C
[0001-Allow-creating-processes-where-only-one-of-stdin-or-.patch (text/plain, attachment)]
[0002-Add-STREAM-argument-to-process-tty-name.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 09:09:02 GMT)
Full text and
rfc822 format available.
Message #172 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Jim Porter <jporterbugs <at> gmail.com> writes:
> Oops. I forgot to add some `(skip-unless ...)' forms for these tests,
> so... here they are. Hopefully this will be the last message from me
> for a bit. :C
:-)
Since this (mainly) affects Cygwin builds, could someone who uses
Windows give the patch a look-over and apply it?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 09:48:01 GMT)
Full text and
rfc822 format available.
Message #175 received at 56025 <at> debbugs.gnu.org (full text, mbox):
> From: Jim Porter <jporterbugs <at> gmail.com>
> Cc: larsi <at> gnus.org, 56025 <at> debbugs.gnu.org
> Date: Sat, 23 Jul 2022 22:29:28 -0700
>
> -@defun process-tty-name process
> +@defun process-tty-name process &optional stream
> This function returns the terminal name that @var{process} is using for
> its communication with Emacs---or @code{nil} if it is using pipes
> instead of a pty (see @code{process-connection-type} in
> -@ref{Asynchronous Processes}). If @var{process} represents a program
> -running on a remote host, the terminal name used by that program on
> -the remote host is provided as process property @code{remote-tty}. If
> -@var{process} represents a network, serial, or pipe connection, the
> -value is @code{nil}.
> +@ref{Asynchronous Processes}). If @var{stream} is one of @code{stdin},
> +@code{stdout}, or @code{stderr}, this function returns the terminal
> +name (or @code{nil}, as above) that @var{process} uses for that stream
> +specifically. You can use this to determine whether a particular
> +stream uses a pipe or a pty.
This text doesn't tell what happens if STREAM is nil or omitted.
> +If @var{process} represents a program running on a remote host, the
> +terminal name used by that program on the remote host is provided as
> +process property @code{remote-tty}. If @var{process} represents a
> +network, serial, or pipe connection, the value is @code{nil}.
If the previous paragraph is only for local subprocesses, the text
there should say so.
> @end defun
>
> @defun process-coding-system process
> diff --git a/etc/NEWS b/etc/NEWS
> index dc79f0826a..23777d349e 100644
> --- a/etc/NEWS
> +++ b/etc/NEWS
> @@ -3198,7 +3198,10 @@ invocation. Such shells are POSIX conformant by default.
> ** 'make-process' can set connection type independently for input and output.
> When calling 'make-process', communication via pty can be enabled
> selectively for just input or output by passing a cons cell for
> -':connection-type', e.g. '(pipe . pty)'.
> +':connection-type', e.g. '(pipe . pty)'. When examining a process
> +later, you can determine whether a particular stream for a process
> +uses a pty by passing one of 'stdin', 'stdout', or 'stderr' as the
> +second argument to 'process-tty-name'.
>
> +++
> ** 'signal-process' now consults the list 'signal-process-functions'.
> diff --git a/lisp/eshell/esh-io.el b/lisp/eshell/esh-io.el
> index c035890ddf..68e52a2c9c 100644
> --- a/lisp/eshell/esh-io.el
> +++ b/lisp/eshell/esh-io.el
> @@ -276,18 +276,21 @@ eshell-close-target
> ;; If we're redirecting to a process (via a pipe, or process
> ;; redirection), send it EOF so that it knows we're finished.
> ((eshell-processp target)
> - ;; According to POSIX.1-2017, section 11.1.9, sending EOF causes
> - ;; all bytes waiting to be read to be sent to the process
> - ;; immediately. Thus, if there are any bytes waiting, we need to
> - ;; send EOF twice: once to flush the buffer, and a second time to
> - ;; cause the next read() to return a size of 0, indicating
> - ;; end-of-file to the reading process. However, some platforms
> - ;; (e.g. Solaris) actually require sending a *third* EOF. Since
> - ;; sending extra EOFs while the process is running shouldn't break
> - ;; anything, we'll just send the maximum we'd ever need. See
> - ;; bug#56025 for further details.
> - (let ((i 0))
> - (while (and (<= (cl-incf i) 3)
> + ;; According to POSIX.1-2017, section 11.1.9, when communicating
> + ;; via terminal, sending EOF causes all bytes waiting to be read
> + ;; to be sent to the process immediately. Thus, if there are any
> + ;; bytes waiting, we need to send EOF twice: once to flush the
> + ;; buffer, and a second time to cause the next read() to return a
> + ;; size of 0, indicating end-of-file to the reading process.
> + ;; However, some platforms (e.g. Solaris) actually require sending
> + ;; a *third* EOF. Since sending extra EOFs while the process is
> + ;; running are a no-op, we'll just send the maximum we'd ever
> + ;; need. See bug#56025 for further details.
> + (let ((i 0)
> + ;; Only call `process-send-eof' once if communicating via a
> + ;; pipe (in truth, this just closes the pipe).
> + (max-attempts (if (process-tty-name target 'stdin) 3 1)))
> + (while (and (<= (cl-incf i) max-attempts)
> (eq (process-status target) 'run))
> (process-send-eof target))))
>
> diff --git a/src/process.c b/src/process.c
> index da5e9cb182..adc508156f 100644
> --- a/src/process.c
> +++ b/src/process.c
> @@ -1243,14 +1243,29 @@ DEFUN ("process-command", Fprocess_command, Sprocess_command, 1, 1, 0,
> return XPROCESS (process)->command;
> }
>
> -DEFUN ("process-tty-name", Fprocess_tty_name, Sprocess_tty_name, 1, 1, 0,
> +DEFUN ("process-tty-name", Fprocess_tty_name, Sprocess_tty_name, 1, 2, 0,
> doc: /* Return the name of the terminal PROCESS uses, or nil if none.
> This is the terminal that the process itself reads and writes on,
> -not the name of the pty that Emacs uses to talk with that terminal. */)
> - (register Lisp_Object process)
> +not the name of the pty that Emacs uses to talk with that terminal.
> +
> +If STREAM is one of `stdin', `stdout', or `stderr', return the name of
> +the terminal PROCESS uses for that stream. This can be used to detect
> +whether a particular stream is connected via a pipe or a pty. */)
> + (register Lisp_Object process, Lisp_Object stream)
Same here: the call without the optional argument returns something
whose relation to the value when STREAM is non-nil is not clear from
the doc string.
Thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 09:49:01 GMT)
Full text and
rfc822 format available.
Message #178 received at 56025 <at> debbugs.gnu.org (full text, mbox):
> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: Ken Brown <kbrown <at> cornell.edu>, Sean Whitton
> <spwhitton <at> email.arizona.edu>, Eli Zaretskii <eliz <at> gnu.org>,
> 56025 <at> debbugs.gnu.org
> Date: Sun, 24 Jul 2022 11:08:25 +0200
>
> Jim Porter <jporterbugs <at> gmail.com> writes:
>
> > Oops. I forgot to add some `(skip-unless ...)' forms for these tests,
> > so... here they are. Hopefully this will be the last message from me
> > for a bit. :C
>
> :-)
>
> Since this (mainly) affects Cygwin builds, could someone who uses
> Windows give the patch a look-over and apply it?
I did the review and tested on native MS-Windows, but I think we
should wait for Ken to try this on Cygwin.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 17:37:01 GMT)
Full text and
rfc822 format available.
Message #181 received at 56025 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 7/24/2022 2:47 AM, Eli Zaretskii wrote:
>> From: Jim Porter <jporterbugs <at> gmail.com>
>> Cc: larsi <at> gnus.org, 56025 <at> debbugs.gnu.org
>> Date: Sat, 23 Jul 2022 22:29:28 -0700
>>
>> -@defun process-tty-name process
>> +@defun process-tty-name process &optional stream
>> This function returns the terminal name that @var{process} is using for
>> its communication with Emacs---or @code{nil} if it is using pipes
>> instead of a pty (see @code{process-connection-type} in
>> -@ref{Asynchronous Processes}). If @var{process} represents a program
>> -running on a remote host, the terminal name used by that program on
>> -the remote host is provided as process property @code{remote-tty}. If
>> -@var{process} represents a network, serial, or pipe connection, the
>> -value is @code{nil}.
>> +@ref{Asynchronous Processes}). If @var{stream} is one of @code{stdin},
>> +@code{stdout}, or @code{stderr}, this function returns the terminal
>> +name (or @code{nil}, as above) that @var{process} uses for that stream
>> +specifically. You can use this to determine whether a particular
>> +stream uses a pipe or a pty.
>
> This text doesn't tell what happens if STREAM is nil or omitted.
Ok, I expanded this to clarify things. (Same for the docstring.)
Hopefully that provides enough detail. I tried to explain the behavior
without going overly in-depth and explaining all the implementation
details of how PTYs get set up. Let me know if it needs any further tweaks.
>> +If @var{process} represents a program running on a remote host, the
>> +terminal name used by that program on the remote host is provided as
>> +process property @code{remote-tty}. If @var{process} represents a
>> +network, serial, or pipe connection, the value is @code{nil}.
>
> If the previous paragraph is only for local subprocesses, the text
> there should say so.
I've added an explanation of what (I think) this means for remote
processes: `process-tty-name' returns the name of the local TTY (so, the
TTY used by ssh, for example), whereas the `remote-tty' property returns
the name of, well... the remote TTY. I'm pretty sure that's what the
behavior is at least, based on my reading of the code.
On 7/24/2022 2:48 AM, Eli Zaretskii wrote:
>> From: Lars Ingebrigtsen <larsi <at> gnus.org>
>>
>> Since this (mainly) affects Cygwin builds, could someone who uses
>> Windows give the patch a look-over and apply it?
>
> I did the review and tested on native MS-Windows, but I think we
> should wait for Ken to try this on Cygwin.
I tested the v4 patch on Cygwin (and GNU/Linux) and all the new tests I
added passed. Ken also tested patch v2 and things worked.
[0001-Allow-creating-processes-where-only-one-of-stdin-or-.patch (text/plain, attachment)]
[0002-Add-STREAM-argument-to-process-tty-name.patch (text/plain, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 20:31:02 GMT)
Full text and
rfc822 format available.
Message #184 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/24/2022 1:36 PM, Jim Porter wrote:
> On 7/24/2022 2:47 AM, Eli Zaretskii wrote:
> > I did the review and tested on native MS-Windows, but I think we
> > should wait for Ken to try this on Cygwin.
>
> I tested the v4 patch on Cygwin (and GNU/Linux) and all the new tests I added
> passed. Ken also tested patch v2 and things worked.
And now I've tested the latest version, and it still looks good.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 24 Jul 2022 21:05:01 GMT)
Full text and
rfc822 format available.
Message #187 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/24/2022 5:08 AM, Lars Ingebrigtsen wrote:
> Jim Porter <jporterbugs <at> gmail.com> writes:
>
>> Oops. I forgot to add some `(skip-unless ...)' forms for these tests,
>> so... here they are. Hopefully this will be the last message from me
>> for a bit. :C
>
> :-)
>
> Since this (mainly) affects Cygwin builds, could someone who uses
> Windows give the patch a look-over and apply it?
It actually affects all builds, even though the motivation for it came from
problems on Cygwin. But it's now been tested on GNU/Linux, native MS-Windows,
and Cygwin. So I think it should be OK.
Ken
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sun, 31 Jul 2022 01:02:01 GMT)
Full text and
rfc822 format available.
Message #190 received at 56025 <at> debbugs.gnu.org (full text, mbox):
On 7/24/2022 1:30 PM, Ken Brown wrote:
> On 7/24/2022 1:36 PM, Jim Porter wrote:
>> On 7/24/2022 2:47 AM, Eli Zaretskii wrote:
>> > I did the review and tested on native MS-Windows, but I think we
>> > should wait for Ken to try this on Cygwin.
>>
>> I tested the v4 patch on Cygwin (and GNU/Linux) and all the new tests
>> I added passed. Ken also tested patch v2 and things worked.
>
> And now I've tested the latest version, and it still looks good.
Thanks for checking. Unless anyone has any objections, I'll merge this
in a couple days then.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 06 Aug 2022 01:11:01 GMT)
Full text and
rfc822 format available.
Message #193 received at 56025-done <at> debbugs.gnu.org (full text, mbox):
On 7/30/2022 6:01 PM, Jim Porter wrote:
> Thanks for checking. Unless anyone has any objections, I'll merge this
> in a couple days then.
Ok, I've merged these changes in 4e59830bc0ab17cdbd85748b133c97837bed99e3.
Hopefully I've done everything correctly, since this is the first time
I've done the merge myself (I'm plenty familiar with git, but may have
missed some Emacs-specific procedure). If I did miss something, just let
me know so I can avoid issues in the future.
Thanks for all the reviews/testing.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#56025; Package
emacs.
(Sat, 06 Aug 2022 12:19:01 GMT)
Full text and
rfc822 format available.
Message #196 received at 56025 <at> debbugs.gnu.org (full text, mbox):
Jim Porter <jporterbugs <at> gmail.com> writes:
> Hopefully I've done everything correctly, since this is the first time
> I've done the merge myself (I'm plenty familiar with git, but may have
> missed some Emacs-specific procedure). If I did miss something, just
> let me know so I can avoid issues in the future.
I had a quick look at the commit, and I don't see any problems, so I
think it worked fine. (Note: I didn't look at the actual semantic
changes, since Eli has already done that in the code review, but only
the general commit.)
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Sun, 04 Sep 2022 11:24:07 GMT)
Full text and
rfc822 format available.
This bug report was last modified 2 years and 301 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.