GNU bug report logs -
#18745
24.3; MS Windows, `call-process-shell-command' fails on `shell-quote-argument'ed bat file with quoted args
Previous Next
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 18745 in the body.
You can then email your comments to 18745 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 04:35:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Noam Postavsky <npostavs <at> users.sourceforge.net>:
New bug report received and forwarded. Copy sent to
bug-gnu-emacs <at> gnu.org.
(Thu, 16 Oct 2014 04:35:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From emacs -Q, evaluating
(call-process-shell-command (shell-quote-argument "c:/path with
space/foo-bar.bat") nil t t (shell-quote-argument "x y"))
or the equivalent
(call-process-shell-command "\"c:/path with space/foo-bar.bat\"" nil
t t "\"x y\"")
gives
'c:/path' is not recognized as an internal or external command,
operable program or batch file.
It also fails for paths without a space:
(call-process-shell-command "\"c:/path-without-space/foo-bar.bat\""
nil t t "\"x y\"")
gives
'c:/path-without-space/foo-bar.bat" "x' is not recognized as an
internal or external command,
operable program or batch file.
Doing the same with an .exe file works correctly. Quoting the spaces
with ^ works for the .bat file:
(call-process-shell-command "c:/path^ with^ space/foo-bar.bat" nil t
t "\"x y\"")
gives
this is foo-bar, arg1 = "x y", arg2 =
But this doesn't work for .exe files.
foo-bar.bat has contents
@echo this is foo-bar, arg1 = %1, arg2 = %2
In GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
of 2013-03-17 on MARVIN
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
`configure --with-gcc (4.7) --cflags
-ID:/devel/emacs/libs/libXpm-3.5.8/include
-ID:/devel/emacs/libs/libXpm-3.5.8/src
-ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include
-ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include
-ID:/devel/emacs/libs/giflib-4.1.4-1/include
-ID:/devel/emacs/libs/jpeg-6b-4/include
-ID:/devel/emacs/libs/tiff-3.8.2-1/include
-ID:/devel/emacs/libs/gnutls-3.0.9/include
-ID:/devel/emacs/libs/libiconv-1.13.1-1-dev/include
-ID:/devel/emacs/libs/libxml2-2.7.8/include/libxml2'
Important settings:
value of $LANG: ENC
locale-coding-system: cp1252
default enable-multibyte-characters: t
Major mode: Lisp Interaction
Minor modes in effect:
tooltip-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
line-number-mode: t
transient-mark-mode: t
Recent input:
<help-echo> ( c a l l - s h e l C-M-i SPC C-a C-y C-k
C-x C-e C-p C-p C-p C-M-f C-o C-o C-p RET C-y C-x C-e
C-p C-p C-p C-f C-SPC C-M-f M-w C-x C-x M-w M-x r e
p o r t SPC e m <tab> <return>
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark set
1
Mark set
1
Load-path shadows:
None found.
Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns disp-table w32-win
w32-vars tool-bar dnd fontset image regexp-opt fringe tabulated-list
newcomment lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32 multi-tty emacs)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 06:51:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> Date: Thu, 16 Oct 2014 00:33:55 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
>
> >From emacs -Q, evaluating
>
> (call-process-shell-command (shell-quote-argument "c:/path with
> space/foo-bar.bat") nil t t (shell-quote-argument "x y"))
>
> or the equivalent
>
> (call-process-shell-command "\"c:/path with space/foo-bar.bat\"" nil
> t t "\"x y\"")
>
> gives
>
> 'c:/path' is not recognized as an internal or external command,
> operable program or batch file.
>
> It also fails for paths without a space:
>
> (call-process-shell-command "\"c:/path-without-space/foo-bar.bat\""
> nil t t "\"x y\"")
>
> gives
>
> 'c:/path-without-space/foo-bar.bat" "x' is not recognized as an
> internal or external command,
> operable program or batch file.
This is a Windows misfeature, not an Emacs bug: the CreateProcess
primitive fails like that when passed a command that is a batch file.
To work around, try using cmd.exe as the command, and the batch file
with arguments as its arguments, with "/c" prepended.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 13:48:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 18745 <at> debbugs.gnu.org (full text, mbox):
>> (call-process-shell-command "\"c:/path-without-space/foo-bar.bat\""
>> nil t t "\"x y\"")
[...]
> This is a Windows misfeature, not an Emacs bug: the CreateProcess
> primitive fails like that when passed a command that is a batch file.
> To work around, try using cmd.exe as the command, and the batch file
> with arguments as its arguments, with "/c" prepended.
But call-process-shell-command (contrary to call-process) should already
be using cmd.exe, no?
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 13:58:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Noam Postavsky <npostavs <at> users.sourceforge.net>, 18745 <at> debbugs.gnu.org
> Date: Thu, 16 Oct 2014 09:47:20 -0400
>
> >> (call-process-shell-command "\"c:/path-without-space/foo-bar.bat\""
> >> nil t t "\"x y\"")
> [...]
> > This is a Windows misfeature, not an Emacs bug: the CreateProcess
> > primitive fails like that when passed a command that is a batch file.
> > To work around, try using cmd.exe as the command, and the batch file
> > with arguments as its arguments, with "/c" prepended.
>
> But call-process-shell-command (contrary to call-process) should already
> be using cmd.exe, no?
No, shell-file-name on Windows is set to invoke cmdproxy.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 16:30:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 18745 <at> debbugs.gnu.org (full text, mbox):
On Thu, Oct 16, 2014 at 2:50 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> This is a Windows misfeature, not an Emacs bug: the CreateProcess
> primitive fails like that when passed a command that is a batch file.
So there is no way for Emacs to paper over the Windows ugliness?
> To work around, try using cmd.exe as the command, and the batch file
> with arguments as its arguments, with "/c" prepended.
Doesn't work :( I tried
(call-process-shell-command
"cmd.exe" nil t t
"/c" "\"c:/path with space/foo-bar.bat\"" "\"x y\"")
(call-process-shell-command
"cmd.exe" nil t t
"/c \"c:/path with space/foo-bar.bat\" \"x y\"")
(call-process-shell-command
"cmd.exe" nil t t
"/c" "c:/path with space/foo-bar.bat" "x y")
All of them give
'c:/path' is not recognized as an internal or external command,
operable program or batch file.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 17:07:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> Date: Thu, 16 Oct 2014 12:28:58 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 18745 <at> debbugs.gnu.org
>
> On Thu, Oct 16, 2014 at 2:50 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > This is a Windows misfeature, not an Emacs bug: the CreateProcess
> > primitive fails like that when passed a command that is a batch file.
>
> So there is no way for Emacs to paper over the Windows ugliness?
I don't know. In fact, I don't even know the exact nature of the
"ugliness", as these intimate details of the CreateProcess API are not
documented anywhere, AFAIK, at least not officially. I just know
about this because I bumped into it in the past.
This is exacerbated in Emacs by the fact that the shell command is
processed twice: once by Emacs, and then again by cmdproxy.
GNU Make overcomes this by detecting these cases, and invoking
CreateProcess in a special way (NULL as the first argument), see the
function process_begin there, around line 710 of sub_proc.c in the GNU
Make sources. If you can come up with a way to do the same in Emacs,
by some suitable patch to cmdproxy.c, such a patch will be welcome
(assuming either the patch is small, or you will agree to sign legal
papers necessary for submitting substantial patches to FSF projects).
Failing that, I can suggest a workaround: use the short 8+3 alias of
the file name with whitespace. You can obtain the short alias of any
existing file's name by calling w32-short-filename. Then you won't
need to quote the batch file name.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 20:16:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> (call-process-shell-command
> "cmd.exe" nil t t
> "/c \"c:/path with space/foo-bar.bat\" \"x y\"")
A few things to note:
- The "rest" arguments to call-process-shell-command are deprecated.
I.e. you should use
(call-process-shell-command
"cmd.exe /c \"c:/path with space/foo-bar.bat\" \"x y\"" nil t t)
instead (which is equivalent anyway, and I think it's more honest
since it doesn't make it seem like those rest args are correctly
separated).
- Why not use call-process instead (since you don't actually use the
"shell-command" part, really)?
Something like either
(call-process "cmd.exe" nil t t
"/c" "\"c:/path with space/foo-bar.bat\" \"x y\"")
or
(call-process "c:/path with space/foo-bar.bat" nil t t "x" "y")
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 21:31:01 GMT)
Full text and
rfc822 format available.
Message #26 received at 18745 <at> debbugs.gnu.org (full text, mbox):
On Thu, Oct 16, 2014 at 1:06 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> GNU Make overcomes this by detecting these cases, and invoking
> CreateProcess in a special way (NULL as the first argument), see the
> function process_begin there, around line 710 of sub_proc.c in the GNU
> Make sources. If you can come up with a way to do the same in Emacs,
> by some suitable patch to cmdproxy.c, such a patch will be welcome
> (assuming either the patch is small, or you will agree to sign legal
> papers necessary for submitting substantial patches to FSF projects).
I'll take a look (btw I have already signed for Emacs).
> Failing that, I can suggest a workaround: use the short 8+3 alias of
> the file name with whitespace. You can obtain the short alias of any
> existing file's name by calling w32-short-filename. Then you won't
> need to quote the batch file name.
s/w32-short-filename/w32-short-file-name/
That works regardless if the program is a bat or exe which will make
things simpler, thanks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 16 Oct 2014 21:51:02 GMT)
Full text and
rfc822 format available.
Message #29 received at 18745 <at> debbugs.gnu.org (full text, mbox):
On Thu, Oct 16, 2014 at 4:15 PM, Stefan Monnier
<monnier <at> iro.umontreal.ca> wrote:
> - The "rest" arguments to call-process-shell-command are deprecated.
I was unaware of that, is it written anywhere?
> - Why not use call-process instead (since you don't actually use the
> "shell-command" part, really)?
That would probably make more sense, but it turns out not to help anyway:
> (call-process "cmd.exe" nil t t
> "/c" "\"c:/path with space/foo-bar.bat\" \"x y\"")
'\"c:/path with space/foo-bar.bat\"' is not recognized as an internal
or external command,
operable program or batch file.
> (call-process "c:/path with space/foo-bar.bat" nil t t "x" "y")
'c:\path' is not recognized as an internal or external command,
operable program or batch file.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Fri, 17 Oct 2014 00:39:01 GMT)
Full text and
rfc822 format available.
Message #32 received at 18745 <at> debbugs.gnu.org (full text, mbox):
>> - The "rest" arguments to call-process-shell-command are deprecated.
> I was unaware of that, is it written anywhere?
It's new in 24.4 (it was actually an oversight: this calling convention
was deprecated for start-process-shell-command in 24.1 IIRC, but for
some reason I didn't notice that it was also used in
call-process-shell-command).
>> - Why not use call-process instead (since you don't actually use the
>> "shell-command" part, really)?
> That would probably make more sense, but it turns out not to help anyway:
I kind of guessed it. I remember reports that starting commands with
a space in its name is pretty much impossible. In the case that the
space is in the directory name, I heard you might work around that by
adding the directory to exec-path:
(let ((exec-path (cons "c:/path with space/" exec-path)))
(call-process "foo-bar.bat" nil t t "x" "y")
This is all hearsay, tho, because luckily I never need to use
a Windows system.
Stefan
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Fri, 17 Oct 2014 06:06:01 GMT)
Full text and
rfc822 format available.
Message #35 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> Date: Thu, 16 Oct 2014 17:50:32 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 18745 <at> debbugs.gnu.org
>
> > - Why not use call-process instead (since you don't actually use the
> > "shell-command" part, really)?
>
> That would probably make more sense, but it turns out not to help anyway:
Since call-process-shell-command simply calls call-process, that's
expected.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Fri, 17 Oct 2014 06:11:01 GMT)
Full text and
rfc822 format available.
Message #38 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz <at> gnu.org>, 18745 <at> debbugs.gnu.org
> Date: Thu, 16 Oct 2014 20:38:38 -0400
>
> I remember reports that starting commands with a space in its name
> is pretty much impossible.
That's an entirely different problem, the reason being how Windows
normalizes file names. It has nothing to do with CreateProcess
quirks.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Wed, 22 Oct 2014 01:13:02 GMT)
Full text and
rfc822 format available.
Message #41 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Thu, Oct 16, 2014 at 5:30 PM, Noam Postavsky
<npostavs <at> users.sourceforge.net> wrote:
> On Thu, Oct 16, 2014 at 1:06 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> GNU Make overcomes this by detecting these cases, and invoking
>> CreateProcess in a special way (NULL as the first argument), see the
>> function process_begin there, around line 710 of sub_proc.c in the GNU
>> Make sources. If you can come up with a way to do the same in Emacs,
>> by some suitable patch to cmdproxy.c, such a patch will be welcome
>> (assuming either the patch is small, or you will agree to sign legal
>> papers necessary for submitting substantial patches to FSF projects).
>
> I'll take a look (btw I have already signed for Emacs).
Patching cmdproxy.c fixes the call-process-shell-command case:
(call-process-shell-command
"\"c:/path with space/foo-bar.bat\" \"x &y\"" nil '(t t) t)
To fix the call-process case:
(call-process
"c:/path with space/foo-bar.bat" nil '(t t) t "x &y")
required a patch to w32proc.c. This is my first patch to Emacs proper;
hopefully I got everything in the right format.
[bat-quote.ChangeLog (application/octet-stream, attachment)]
[bat-quote.patch (application/octet-stream, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Wed, 22 Oct 2014 17:13:02 GMT)
Full text and
rfc822 format available.
Message #44 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> Date: Tue, 21 Oct 2014 21:12:27 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 18745 <at> debbugs.gnu.org
>
> Patching cmdproxy.c fixes the call-process-shell-command case:
>
> (call-process-shell-command
> "\"c:/path with space/foo-bar.bat\" \"x &y\"" nil '(t t) t)
>
> To fix the call-process case:
>
> (call-process
> "c:/path with space/foo-bar.bat" nil '(t t) t "x &y")
>
> required a patch to w32proc.c. This is my first patch to Emacs proper;
> hopefully I got everything in the right format.
Thanks, the changes look reasonable. If you didn't already, please
run the test suite and make sure there are no regressions due to these
changes. Bonus points for adding specialized tests to test this
specific issue.
> * nt/cmdproxy.c (batch_file_p): new function.
> (spawn): if calling a quoted batch file pass NULL for progname.
> * src/w32proc.c (create_child): if calling a quoted batch file
> pass NULL for exe.
In the future, please begin each ChangeLog entry with a capital
letter.
> + /* CreateProcess handles batch files as progname specially. This
> + special handling fails when both the batch file and arguments are
> + quoted. We pass NULL as progname to avoid the special
> + handling. */
Please in the future leave 2 spaces between sentences, per the US
English convention we use in Emacs.
> + if (progname != NULL && cmdline[0] == '"' && batch_file_p(progname))
> + progname = NULL;
Our coding style is to leave one space between the function name and
the opening parenthesis that follows it.
If there are no more comments or objections, I will commit in a few
days.
Thanks again for working on this.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 23 Oct 2014 01:35:01 GMT)
Full text and
rfc822 format available.
Message #47 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Wed, Oct 22, 2014 at 1:12 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> Thanks, the changes look reasonable. If you didn't already, please
> run the test suite and make sure there are no regressions due to these
> changes. Bonus points for adding specialized tests to test this
> specific issue.
The tests in test/automated? No regressions.
Here is a new patch with added test and formatting issues fixed.
[bat-quote-v2.ChangeLog (application/octet-stream, attachment)]
[bat-quote-v2.patch (application/octet-stream, attachment)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 23 Oct 2014 15:54:01 GMT)
Full text and
rfc822 format available.
Message #50 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> Date: Wed, 22 Oct 2014 21:33:59 -0400
> From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> Cc: 18745 <at> debbugs.gnu.org
>
> The tests in test/automated? No regressions.
Great, thanks.
> Here is a new patch with added test and formatting issues fixed.
Looks good, thanks. And you do get the bonus points.
As I said, will commit in a couple of days.
Reply sent
to
Eli Zaretskii <eliz <at> gnu.org>:
You have taken responsibility.
(Sat, 25 Oct 2014 09:17:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Noam Postavsky <npostavs <at> users.sourceforge.net>:
bug acknowledged by developer.
(Sat, 25 Oct 2014 09:17:04 GMT)
Full text and
rfc822 format available.
Message #55 received at 18745-done <at> debbugs.gnu.org (full text, mbox):
> Date: Thu, 23 Oct 2014 18:52:58 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 18745 <at> debbugs.gnu.org
>
> > Date: Wed, 22 Oct 2014 21:33:59 -0400
> > From: Noam Postavsky <npostavs <at> users.sourceforge.net>
> > Cc: 18745 <at> debbugs.gnu.org
> >
> > The tests in test/automated? No regressions.
>
> Great, thanks.
>
> > Here is a new patch with added test and formatting issues fixed.
>
> Looks good, thanks. And you do get the bonus points.
>
> As I said, will commit in a couple of days.
Done, as trunk revision 118196. I'm therefore closing this bug.
Thanks a lot for working on this.
Btw, the email address mentioned in your copyright assignment is
different from the one you used in this thread. Is that address still
valid, or should we always use the one you used here?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Sat, 25 Oct 2014 14:05:02 GMT)
Full text and
rfc822 format available.
Message #58 received at 18745-done <at> debbugs.gnu.org (full text, mbox):
On Sat, Oct 25, 2014 at 5:16 AM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> Btw, the email address mentioned in your copyright assignment is
> different from the one you used in this thread. Is that address still
> valid, or should we always use the one you used here?
Oh, this one is just an alias for the other. I generally try to use
this one for public mailing lists.
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Sun, 23 Nov 2014 12:24:04 GMT)
Full text and
rfc822 format available.
bug unarchived.
Request was from
Juanma Barranquero <lekktu <at> gmail.com>
to
control <at> debbugs.gnu.org.
(Tue, 27 Oct 2015 11:44:01 GMT)
Full text and
rfc822 format available.
bug Marked as found in versions 25.0.50 and reopened.
Request was from
Juanma Barranquero <lekktu <at> gmail.com>
to
control <at> debbugs.gnu.org.
(Tue, 27 Oct 2015 11:44:01 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Tue, 27 Oct 2015 11:50:03 GMT)
Full text and
rfc822 format available.
Message #67 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
process-test-quoted-batfile fails on my Windows 10 Home
F process-test-quoted-batfile
Check that Emacs hides CreateProcess deficiency (bug#18745).
(ert-test-failed
((should
(string=
(buffer-string)
"arg1 = \"x &y\", arg2 =
"))
:form
(string= "arg1 = \"x &y\", arg2 =
" "arg1 = \"x &y\", arg2 =
")
:value nil))
The result lacks the trailing space before the \n.
"arg1 = \"x &y\", arg2 =\n"
instead of
"arg1 = \"x &y\", arg2 = \n"
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Tue, 27 Oct 2015 19:14:01 GMT)
Full text and
rfc822 format available.
Message #70 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> From: Juanma Barranquero <lekktu <at> gmail.com>
> Date: Tue, 27 Oct 2015 12:49:01 +0100
>
> process-test-quoted-batfile fails on my Windows 10 Home
>
> F process-test-quoted-batfile
> Check that Emacs hides CreateProcess deficiency (bug#18745).
> (ert-test-failed
> ((should
> (string=
> (buffer-string)
> "arg1 = \"x &y\", arg2 =
> "))
> :form
> (string= "arg1 = \"x &y\", arg2 =
> " "arg1 = \"x &y\", arg2 =
> ")
> :value nil))
>
> The result lacks the trailing space before the \n.
>
> "arg1 = \"x &y\", arg2 =\n"
>
> instead of
>
> "arg1 = \"x &y\", arg2 = \n"
Do you see the same result if you invoke 'echo' from the cmd prompt?
IOW, does this mean Windows 10 changed the behavior of the built-in
'echo', and it now removes trailing whitespace from its argument? If
that's the reason, how about modifying the test so that it expects the
correct result depending on the value returned by x-server-version?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Tue, 27 Oct 2015 23:26:02 GMT)
Full text and
rfc822 format available.
Message #73 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Tue, Oct 27, 2015 at 8:14 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> Do you see the same result if you invoke 'echo' from the cmd prompt?
> IOW, does this mean Windows 10 changed the behavior of the built-in
> 'echo', and it now removes trailing whitespace from its argument?
IIUC your question, no
C:\Users\Juanma>echo A>A.txt
C:\Users\Juanma>echo B >B.txt
C:\Users\Juanma>dir *.txt
28/10/2015 00:23 3 A.txt
28/10/2015 00:23 7 B.txt
2 archivos 10 bytes
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Wed, 28 Oct 2015 16:02:02 GMT)
Full text and
rfc822 format available.
Message #76 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> From: Juanma Barranquero <lekktu <at> gmail.com>
> Date: Wed, 28 Oct 2015 00:24:29 +0100
> Cc: 18745 <at> debbugs.gnu.org
>
> > Do you see the same result if you invoke 'echo' from the cmd prompt?
> > IOW, does this mean Windows 10 changed the behavior of the built-in
> > 'echo', and it now removes trailing whitespace from its argument?
>
> IIUC your question, no
>
> C:\Users\Juanma>echo A>A.txt
>
> C:\Users\Juanma>echo B >B.txt
>
> C:\Users\Juanma>dir *.txt
>
> 28/10/2015 00:23 3 A.txt
> 28/10/2015 00:23 7 B.txt
> 2 archivos 10 bytes
The test actually uses a batch file, so I think it's worthwhile to try
that, and see if the behavior of 'echo' when called from a batch file
changed.
If not, then please try digging deeper, because the unmodified test
work OK for me (not on Windows 10).
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 29 Oct 2015 03:40:03 GMT)
Full text and
rfc822 format available.
Message #79 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Wed, Oct 28, 2015 at 5:01 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> If not, then please try digging deeper, because the unmodified test
> work OK for me (not on Windows 10).
Well, mystery solved. It's not a bug or change in Windows 10, it's an
incompatibility between CMD.EXE and TCC.EXE (the TakeCommand shell). I
usually run everything from inside TakeCommand, so COMSPEC points to
TCC.EXE, not CMD.EXE.
Not sure how to fix it. I suppose "nothing to do, just run the tests with
the right COMSPEC" is an option. OTOH, the test is supposed to check for a
CreateProcess "deficiency". And certainly 4DOS/4NT/TCC is a quite popular
shell.
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 29 Oct 2015 16:20:03 GMT)
Full text and
rfc822 format available.
Message #82 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> From: Juanma Barranquero <lekktu <at> gmail.com>
> Date: Thu, 29 Oct 2015 04:38:47 +0100
> Cc: 18745 <at> debbugs.gnu.org
>
> Well, mystery solved. It's not a bug or change in Windows 10, it's an incompatibility between CMD.EXE and TCC.EXE (the TakeCommand shell). I usually run everything from inside TakeCommand, so COMSPEC points to TCC.EXE, not CMD.EXE.
>
> Not sure how to fix it. I suppose "nothing to do, just run the tests with the right COMSPEC" is an option. OTOH, the test is supposed to check for a CreateProcess "deficiency". And certainly 4DOS/4NT/TCC is a quite popular shell.
The CreateProcess deficiency we test there has nothing to do with
trailing whitespace, btw.
Anyway, does it work to put "ComSpec=%windir%\system32\cmd.exe" into
process-environment before running that code?
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Thu, 29 Oct 2015 17:24:01 GMT)
Full text and
rfc822 format available.
Message #85 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Thu, Oct 29, 2015 at 5:17 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> The CreateProcess deficiency we test there has nothing to do with
> trailing whitespace, btw.
That's what I meant: that changing the test to accommodate for a TCC
incompatibility still would make sense because the trailing space is not
what the test is about. Only I said it with less words and zero clarity.
> Anyway, does it work to put "ComSpec=%windir%\system32\cmd.exe" into
> process-environment before running that code?
No. I had already tried that, and now I've done some more checking. (In all
cases, I run the test from the shell with emacs -batch -l ert -l
process-test [etc]).
- Testing from CMD.EXE works (as expected)
- Setting COMSPEC to point to CMD and testing from TCC.EXE also works.
- let-binding process-environment to (cons
"ComSpec=C:\\windows\\system32\\cmd.exe" process-environment) around the
test does not work.
- Using (setenv "ComSpec" "C:\\Windows\\system32\\cmd.exe") or (setenv
"ComSpec" "%windir%\\system32\\cmd.exe" t) in the test function before
calling call-process does not work
- Same for variants with forward slashes, %windir% vs. explicit path, etc.
I would be very surprised that the process-environment does not affect
call-process, so I *must* be missing something obvious. I'll take a closer
look as soon as I have a little more time.
J
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Fri, 30 Oct 2015 14:27:02 GMT)
Full text and
rfc822 format available.
Message #88 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
> I would be very surprised that the process-environment does not affect
call-process, so I *must* be missing something obvious. I'll take a closer
look as soon as I have a little more time.
OK, now this is freak.
REM ---- test.bat ---
@echo off
echo TEST=%TEST>test.log
echo COMSPEC=%COMSPEC>>test.log
REM -- end of test.bat ---
C:\...\test> emacs -Q --batch --eval "(call-process
\"c:/devel/emacs/repo/trunk/test.bat\")"
C:\...\test> type test.log
TEST=AAA
COMSPEC=C:\bin64\JPSoft\TCMD\TCC.EXE
C:\...\test> emacs -Q --batch --eval "(let ((process-environment
'(\"TEST=BBB\" \"COMSPEC=C:\\Windows\\system32\\cmd.exe\"))) (call-process
\"c:/devel/emacs/repo/trunk/test.bat\"))"
C:\...\test> type test.log
TEST=BBB
COMSPEC=C:\bin64\JPSoft\TCMD\TCC.EXE
Is that expected?
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Fri, 30 Oct 2015 14:56:02 GMT)
Full text and
rfc822 format available.
Message #91 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> From: Juanma Barranquero <lekktu <at> gmail.com>
> Date: Fri, 30 Oct 2015 15:25:37 +0100
> Cc: 18745 <at> debbugs.gnu.org
>
> C:\...\test> emacs -Q --batch --eval "(let ((process-environment '(\"TEST=BBB\"
> \"COMSPEC=C:\\Windows\\system32\\cmd.exe\"))) (call-process
> \"c:/devel/emacs/repo/trunk/test.bat\"))"
>
> C:\...\test> type test.log
> TEST=BBB
> COMSPEC=C:\bin64\JPSoft\TCMD\TCC.EXE
>
> Is that expected?
No, but it could be some TCC magic, aimed at intercepting any explicit
references to cmd.exe.
What happens if you replace test.bat with a compiled C program that
calls getenv to look at these two variables -- do you see the same
there? (You could use Emacs with a suitable --eval argument to
emulate such a "C program".)
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Fri, 30 Oct 2015 17:11:02 GMT)
Full text and
rfc822 format available.
Message #94 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Fri, Oct 30, 2015 at 3:55 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> No, but it could be some TCC magic, aimed at intercepting any explicit
> references to cmd.exe.
I think you're right. Running the let-binding variant under CMD.EXE works
as expected (the called process receives the right COMSPEC value).
> What happens if you replace test.bat with a compiled C program that
> calls getenv to look at these two variables -- do you see the same
> there?
Both in TCC.EXE and CMD.EXE the let-binding works correctly.
I don't think there's anything more to do here, other than perhaps adding a
note somewhere that the test doesn't work under TCC.
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Fri, 30 Oct 2015 20:11:02 GMT)
Full text and
rfc822 format available.
Message #97 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> From: Juanma Barranquero <lekktu <at> gmail.com>
> Date: Fri, 30 Oct 2015 18:09:38 +0100
> Cc: 18745 <at> debbugs.gnu.org
>
> I don't think there's anything more to do here, other than perhaps adding a
> note somewhere that the test doesn't work under TCC.
If the batch says
cmd /c echo arg1 = %1, arg2 = %2
i.e. invokes cmd.exe from the batch file, does the test still fail?
If it still fails, I think making the test more tolerant of the
trailing whitespace is IMO a better alternative than letting it fail.
After all, that trailing space is not what we are testing there.
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Sat, 31 Oct 2015 23:22:01 GMT)
Full text and
rfc822 format available.
Message #100 received at 18745 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Fri, Oct 30, 2015 at 9:10 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> If the batch says
>
> cmd /c echo arg1 = %1, arg2 = %2
>
> i.e. invokes cmd.exe from the batch file, does the test still fail?
With
(with-temp-file batfile
(insert "@cmd /c echo arg1 = %1, arg2 = %2\n"))
it still works on cmd.exe, fails on tcc.exe
> If it still fails, I think making the test more tolerant of the
> trailing whitespace is IMO a better alternative than letting it fail.
> After all, that trailing space is not what we are testing there.
Something like this, you mean?
diff --git a/test/automated/process-tests.el
b/test/automated/process-tests.el
index 1dab615..58a2de7 100644
--- a/test/automated/process-tests.el
+++ b/test/automated/process-tests.el
@@ -61,15 +61,15 @@ process-test-sentinel-wait-function-working-p
;; to force quoting.
(setq batfile (make-temp-file "echo args" nil ".bat"))
(with-temp-file batfile
- (insert "@echo arg1 = %1, arg2 = %2\n"))
+ (insert "@echo arg1=%1, arg2=%2\n"))
(with-temp-buffer
(call-process batfile nil '(t t) t "x &y")
- (should (string= (buffer-string) "arg1 = \"x &y\", arg2 =
\n")))
+ (should (string= (buffer-string) "arg1=\"x &y\", arg2=\n")))
(with-temp-buffer
(call-process-shell-command
(mapconcat #'shell-quote-argument (list batfile "x &y") " ")
nil '(t t) t)
- (should (string= (buffer-string) "arg1 = \"x &y\", arg2 =
\n"))))
+ (should (string= (buffer-string) "arg1=\"x &y\", arg2=\n"))))
(when batfile (delete-file batfile))))))
(ert-deftest process-test-stderr-buffer ()
warning: LF will be replaced by CRLF in test/automated/process-tests.el.
The file will have its original line endings in your working directory.
BTW, curiously, the stderr-related tests in the same test file
(process-test-stderr-buffer and process-test-stderr-filter) fail sometimes.
Much more frequently on TCC than CMD, but they also fail in CMD, as in
these examples:
C:\...\automated> ..\..\src\emacs.exe -Q -batch -l ert -l process-tests.elc
-f ert-run-tests-batch-and-exit
Running 5 tests (2015-10-31 23:35:46+0100)
passed 1/5 process-test-quoted-batfile
passed 2/5 process-test-sentinel-accept-process-output
passed 3/5 process-test-sentinel-sit-for
Test process-test-stderr-buffer backtrace:
#[nil "\306\307C \310 \311 \3121 \313\216\314 \"\211 )0\202 \210
ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
ert-run-test([cl-struct-ert-test process-test-stderr-buffer nil #[ni
ert-run-or-rerun-test([cl-struct-ert--stats t [[cl-struct-ert-test p
ert-run-tests(t #[385 "\306 \307\"\203G \211\211G\310U\203 \211@\20
ert-run-tests-batch(nil)
ert-run-tests-batch-and-exit()
command-line-1(("-l" "ert" "-l" "process-tests.elc" "-f" "ert-run-te
command-line()
normal-top-level()
Test process-test-stderr-buffer condition:
(ert-test-failed
((should
(with-current-buffer stderr-buffer
(goto-char ...)
(looking-at "hello stderr!")))
:form
(save-current-buffer
(set-buffer stderr-buffer)
(goto-char
(point-min))
(looking-at "hello stderr!"))
:value nil))
FAILED 4/5 process-test-stderr-buffer
passed 5/5 process-test-stderr-filter
Ran 5 tests, 4 results as expected, 1 unexpected (2015-10-31 23:35:47+0100)
1 unexpected results:
FAILED process-test-stderr-buffer
C:\...\automated> ..\..\src\emacs.exe -Q -batch -l ert -l process-tests.elc
-f ert-run-tests-batch-and-exit
Running 5 tests (2015-10-31 23:34:42+0100)
passed 1/5 process-test-quoted-batfile
passed 2/5 process-test-sentinel-accept-process-output
passed 3/5 process-test-sentinel-sit-for
passed 4/5 process-test-stderr-buffer
Test process-test-stderr-filter backtrace:
#[nil "\306\307C \310 \311 \3121 \313\216\314 \"\211 )0\202 \210
ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
ert-run-test([cl-struct-ert-test process-test-stderr-filter nil #[ni
ert-run-or-rerun-test([cl-struct-ert--stats t [[cl-struct-ert-test p
ert-run-tests(t #[385 "\306 \307\"\203G \211\211G\310U\203 \211@\20
ert-run-tests-batch(nil)
ert-run-tests-batch-and-exit()
command-line-1(("-l" "ert" "-l" "process-tests.elc" "-f" "ert-run-te
command-line()
normal-top-level()
Test process-test-stderr-filter condition:
(ert-test-failed
((should stderr-sentinel-called)
:form stderr-sentinel-called :value nil))
FAILED 5/5 process-test-stderr-filter
Ran 5 tests, 4 results as expected, 1 unexpected (2015-10-31 23:34:43+0100)
1 unexpected results:
FAILED process-test-stderr-filter
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-gnu-emacs <at> gnu.org:
bug#18745; Package
emacs.
(Sun, 01 Nov 2015 17:50:03 GMT)
Full text and
rfc822 format available.
Message #103 received at 18745 <at> debbugs.gnu.org (full text, mbox):
> From: Juanma Barranquero <lekktu <at> gmail.com>
> Date: Sun, 1 Nov 2015 00:20:34 +0100
> Cc: 18745 <at> debbugs.gnu.org
>
> > If it still fails, I think making the test more tolerant of the
> > trailing whitespace is IMO a better alternative than letting it fail.
> > After all, that trailing space is not what we are testing there.
>
> Something like this, you mean?
Yes, thanks.
> BTW, curiously, the stderr-related tests in the same test file
> (process-test-stderr-buffer and process-test-stderr-filter) fail sometimes.
> Much more frequently on TCC than CMD, but they also fail in CMD, as in these
> examples:
It doesn't fail for me, even if I run the test many times.
Reply sent
to
Juanma Barranquero <lekktu <at> gmail.com>:
You have taken responsibility.
(Sun, 01 Nov 2015 18:31:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Noam Postavsky <npostavs <at> users.sourceforge.net>:
bug acknowledged by developer.
(Sun, 01 Nov 2015 18:31:02 GMT)
Full text and
rfc822 format available.
Message #108 received at 18745-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Sun, Nov 1, 2015 at 6:49 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> Yes, thanks.
OK, thanks. Installed. I'm re-closing this thread.
> It doesn't fail for me, even if I run the test many times.
That's interesting. I'll look into it.
[Message part 2 (text/html, inline)]
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org.
(Mon, 30 Nov 2015 12:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 8 years and 170 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.