Stefan Kangas <stefankangas@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 71472) by debbugs.gnu.org; 11 Jun 2024 20:41:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 11 16:41:31 2024
Received: from localhost ([127.0.0.1]:36755 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1sH8J8-0006IE-FG
for submit <at> debbugs.gnu.org; Tue, 11 Jun 2024 16:41:31 -0400
Received: from eggs.gnu.org ([209.51.188.92]:47376)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <eliz@HIDDEN>) id 1sH82P-0002i3-6G
for 71472 <at> debbugs.gnu.org; Tue, 11 Jun 2024 16:24:13 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1sGvuZ-0008LC-Th; Tue, 11 Jun 2024 03:27:20 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
mime-version; bh=goOOg6NLSqFV8eDSUSgUcR3eYtiD7VIk3VnIF1bqpOo=; b=ozdVzstE4cUB
9GSnDQBzQxcKc7P1t6Hm5gUVaud1lMHoaq7EA+Vghi/EwJPBZCh8lJlSLlMboqIlTayOXGAlJAmhW
lhXUhzKq1/6vNWafygwmBzMHLWGyZXUU5f7BFE2j1VXpzi7QbcF4Mzln4YZTXHMxaTg6FzSDvsua+
8H3mUQssI6fGVg7mpdgTQQ06GPigCzXZUjsGGpvmxTk8/0Gc2j/rIN9mruMtdIIKVALpHINdKFqyq
5WKVq+kwnlpckSjTmpjYXG3UYQLVsoWM8xWVll5/Q2y3D9qb+LBQwdCenOmU+IoZY5Nn0+BRN8fLv
ongcbnYsTCoEqWXh6+ADSw==;
Date: Tue, 11 Jun 2024 10:27:04 +0300
Message-Id: <86ed946it3.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Ke Wu <ellpih@HIDDEN>
In-Reply-To: <190055cd3c0.5289e49215028.2058921479589116968@HIDDEN>
(message from Ke Wu on Tue, 11 Jun 2024 12:34:48 +0900)
Subject: Re: bug#71472: [PATCH] Add pty support by using ConPTY on Windows
References: <874ja1m6u1.fsf@HIDDEN> <86jziw956n.fsf@HIDDEN>
<190055cd3c0.5289e49215028.2058921479589116968@HIDDEN>
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 71472
Cc: 71472 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
[Please use Reply All to reply, to keep the bug tracker CC'ed.]
> Date: Tue, 11 Jun 2024 12:34:48 +0900
> From: Ke Wu <ellpih@HIDDEN>
>
> > If we must use UTF-8 as the only encoding to talk to sub-processes via
> > ConPTY, that makes the number of applications that can be used this
> > way very small, since most programs we are used to run as
> > subprocesses, in particularly ports of GNU software like GCC, GDB,
> > Grep, Find, and many others, cannot reliably talk to Emacs in UTF-8
> > encoding on MS-Windows.
>
> The statement is not so accurate. On Emacs side, UTF-8 is assumed due
> to the limitation of ConPTY (it would communicate with the console only in
> UTF-8). However, on the subprocesses side, ConPTY would respect its
> codepage and translate it into UTF-8 when sending to the console. So
> we can make these subprocesses run in the codepage other than
> 65001(UTF-8).
This is inaccurate: ConPTY always assumes the process running on the
other side of the connection uses the system codepage. If the
subprocess expects some other encoding, ConPTY will not know that, and
Emacs has no way of telling ConPTY to use a different encoding. This
is the essence of the issue I filed with them, and they basically told
me that what ConPTY does is "by design".
This is not an academic issue: some very important programs we invoke
from Emacs need us to talk to them in encoding different from the
system codepage. A notable example is Git, which wants UTF-8 (it can
support other encodings, but that is not recommended, and Emacs
doesn't really support that well on Windows).
> I am not very familiar with these GNU software ports :(
> Please let me know if there will be problems with ConPTY translating from
> UTF-8 to other codepages.
See above. There's no way for Emacs to set that up, except when the
"other codepage" is the system codepage.
> > https://github.com/microsoft/terminal/issues/9174
>
> I think a possible solution to this issue is to use a wrapper program to
> set the codepage for the applications that do not call `SetConsoleOutputCP`.
> As a proof of concept, the following code snippet uses cmdproxy.exe to
> change the codepage to 1255. Please replace the cmdproxy.exe path in the
> snippet.
>
> (progn
> (set-buffer
> (apply #'make-term
> "terminal"
> "C:/Users/oracl/Documents/Programs/emacs-master/nt/cmdproxy.exe"
> nil
> '("-c" "chcp 1255 && call cmd")))
> (term-char-mode)
> (pop-to-buffer-same-window "*terminal*"))
>
> The codepage can be verified by either using `chcp` in the newly created cmd process.
> Also, the following hack can be applied to make the created conhost.exe visible.
> Therefore, the codepage can be directly verified by viewing the properties of the
> conhost.exe window.
>
> --- a/src/w32.c
> +++ b/src/w32.c
> @@ -11208,7 +11208,7 @@ make_console_with_pipe (ptrdiff_t nargs, Lisp_Object * args, const int * fds)
>
> command_new = CALLN (Flist,
> build_string ("conhost.exe"),
> - build_string ("--headless"),
> + /* build_string ("--headless"), */
> build_string ("--feature"),
> build_string ("pty"));
> if (!NILP (width))
>
> Therefore, we can have subprocesses run in codepage other than 65001 or the OEM default
> codepage. And as a console program, Emacs talks in UTF-8. It may be feasible if we add a
> `:coding` to function `term`, which builds up a wrapper to change the code page before the
> real program starts.
cmdproxy is only used when invoking programs via the shell. But Emacs
also invokes programs directly (call-process etc.), in which case
cmdproxy (or any other kind of wrapper) will be very problematic at
best, if not impossible. See below about the complications this
causes wrt quoting of command-line arguments, for example.
Please keep in mind how Emacs arranges to use correct encoding when
invoking other programs: we have data structures
(process-coding-system-alist etc.) which define the correct encoding
by program name, and we also have variables (coding-system-for-read
etc.) that can be bound to override those defaults temporarily. The
encoding is applied separately to the program's command-line arguments
and to the stuff we write and read to and from the process. How can
all this work reliably with ConPTY, even if the wrapper trick could
sometimes work? Specifically:
. how do we control encoding of command-line arguments? most
programs running on Windows cannot handle UTF-8 encoded command
lines
. what if the encoding we need doesn't have a corresponding Windows
codepage (which means chcp will not work)?
. how can we handle the eol-conversion part of the encoding (some
programs _must_ be fed with Unix EOLs)?
Also please note that using a wrapper adds another layer of
interpreting command-line arguments, which might break some
complicated cases that use fancy quoting of special characters. Any
wrapper we provide will be compiled with MinGW, so it will use the
MinGW startup code to process quoting. But the program the wrapper
runs might not be a MinGW program, so it could use different ways of
processing quotes. The simplest example of such a combination is
cmd.exe itself: its quoting rules are very different from what MinGW
uses. This will definitely break some cases. For example, Git uses
the '^' character for special purposes, and some Windows styles of
quoting interpret '^' as a quote character -- this could easily break
Emacs commands that invoke Git.
If someone can figure out how to do all this stuff with ConPTY, then
okay, we could use it. But it is not a trivial problem, not at all.
The way ConPTY was designed is the way Windows works everywhere else:
it doesn't allow applications to communicate with raw bytestreams
without interpreting; instead, Windows _interprets_ the bytestreams as
characters encoded in the encoding it assumes for the source, and then
converts those characters to the encoding of the destination. This
basic design principle is built into every part of Windows APIs. For
example, a program whose 'main' function is declared as accepting
wchar_t (i.e. UTF-16) command-line arguments will magically have the
command-line arguments converted to UTF-16, even if the calling
process uses plain ASCII. ConPTY uses the same design principles, so
it is inherently unable to pass through raw bytes without interpreting
them. And without that, we cannot easily implement the way Emacs
expects this stuff to work, because Emacs assumes the encoding to be a
private contract between Emacs and the program it calls, with nothing
in-between interfering.
I hope I explained some of the issues with ConPTY, and why we cannot
install its support without some reasonably reliable solutions for
those problematic aspects.
Thanks.
bug-gnu-emacs@HIDDEN:bug#71472; Package emacs.
Full text available.Received: (at 71472) by debbugs.gnu.org; 11 Jun 2024 20:24:14 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 11 16:24:14 2024 Received: from localhost ([127.0.0.1]:36381 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1sH82Q-0002k3-Ct for submit <at> debbugs.gnu.org; Tue, 11 Jun 2024 16:24:14 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47376) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1sH82M-0002i3-Tw for 71472 <at> debbugs.gnu.org; Tue, 11 Jun 2024 16:24:11 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1sGx4x-0006Xq-BF; Tue, 11 Jun 2024 04:42:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=h0YDBv8Kx2X3XzAJA2Z7bCp/znk2X+VxfKq+CXFzoX4=; b=Mo7dEIWZ3MAT kYPIVd+oAjsH7htugP1P9lSkt3T2fcMUr/79SdDg2G8apqKC8q3A8mTTfV45hFigXHAhndnwHpPWc kApLj+XG9jlX0UZOs6/Xep0nlHHW3Y3fFjjqbi+1dVMLPvDG5t06LJo5/BmzLI1MJNW4nfte6rP1q l01OXHVBdmAnGdtgkuykJEqA+VBlUw/IGCEKu1X5cjEgoCyItGnzKZTwqhuyRXxR3Z2ENahr5FgwD v24u2w5AcoYSH3wtXZKTkjeAqMfpcO5rcFhUNf51nAelDOS2aXhGyhSNMgtDFpgLFjN2I2sWfm2HQ HGEn2PzyE2QpC9GERvfUFg==; Date: Tue, 11 Jun 2024 11:42:02 +0300 Message-Id: <865xuf7twl.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ke Wu <ellpih@HIDDEN> In-Reply-To: <19006667d08.6141650b15233.4287906273312116966@HIDDEN> (message from Ke Wu on Tue, 11 Jun 2024 17:24:58 +0900) Subject: Re: bug#71472: [PATCH] Add pty support by using ConPTY on Windows References: <874ja1m6u1.fsf@HIDDEN> <86jziw956n.fsf@HIDDEN> <190055cd3c0.5289e49215028.2058921479589116968@HIDDEN> <86ed946it3.fsf@HIDDEN> <19006667d08.6141650b15233.4287906273312116966@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 71472 Cc: 71472 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > Date: Tue, 11 Jun 2024 17:24:58 +0900 > From: Ke Wu <ellpih@HIDDEN> > Cc: "71472" <71472 <at> debbugs.gnu.org> > > Thank you for your detailed explanation. The patch itself cannot provide > a satisfying pty support on Windows. Its original purpose is to provide > support to term.el and other packages which setup a terminal emulator > inside Emacs. It is not meant to be a replacement of `make-process` on > Windows, but is used as a supplementary method to patch the broken > term.el, which has long been an issue for Windows users. OK, that narrows the applicability and thus the extent of possible problems. But still some problems remain: does that work with running in "M-x term" programs that display fancy/unusual characters? Did you try that with Git, for example, showing changes whose authors or text of the diffs include Unicode characters? Or what happens if you run the DIR command of cmd.exe in a directory which has files whose names aren't supported by the system codepage? Or what happens if you run a program whose command-line arguments include characters not supported by the current system codepage? IOW, we must understand the limitations of this before we decide whether this feature is justified. Installing features with too many limitations will annoy the users and cause complaints and bug reports. Thanks.
bug-gnu-emacs@HIDDEN:bug#71472; Package emacs.
Full text available.Received: (at 71472) by debbugs.gnu.org; 11 Jun 2024 16:48:07 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 11 12:48:06 2024 Received: from localhost ([127.0.0.1]:36139 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1sH4fG-0005H0-00 for submit <at> debbugs.gnu.org; Tue, 11 Jun 2024 12:48:06 -0400 Received: from sender2-pp-o93.zoho.jp ([165.173.185.27]:58820) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ellpih@HIDDEN>) id 1sH458-0001HV-Ik for 71472 <at> debbugs.gnu.org; Tue, 11 Jun 2024 12:10:47 -0400 Received: from mail.zoho.jp by mx.zoho.jp with SMTP id 1718094298386426.0997017958897; Tue, 11 Jun 2024 17:24:58 +0900 (JST) Received: from [202.81.228.38] by mail.zoho.jp with HTTP;Tue, 11 Jun 2024 17:24:58 +0900 (JST) Date: Tue, 11 Jun 2024 17:24:58 +0900 From: Ke Wu <ellpih@HIDDEN> To: "Eli Zaretskii" <eliz@HIDDEN> Message-ID: <19006667d08.6141650b15233.4287906273312116966@HIDDEN> In-Reply-To: <86ed946it3.fsf@HIDDEN> References: <874ja1m6u1.fsf@HIDDEN> <86jziw956n.fsf@HIDDEN> <190055cd3c0.5289e49215028.2058921479589116968@HIDDEN> <86ed946it3.fsf@HIDDEN> Subject: Re: bug#71472: [PATCH] Add pty support by using ConPTY on Windows MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 71472 X-Mailman-Approved-At: Tue, 11 Jun 2024 12:48:05 -0400 Cc: 71472 <71472 <at> debbugs.gnu.org> X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Thank you for your detailed explanation. The patch itself cannot provide a satisfying pty support on Windows. Its original purpose is to provide support to term.el and other packages which setup a terminal emulator inside Emacs. It is not meant to be a replacement of `make-process` on Windows, but is used as a supplementary method to patch the broken term.el, which has long been an issue for Windows users. Thanks.
bug-gnu-emacs@HIDDEN:bug#71472; Package emacs.
Full text available.Received: (at 71472) by debbugs.gnu.org; 10 Jun 2024 15:41:19 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 10 11:41:19 2024 Received: from localhost ([127.0.0.1]:42754 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1sGh94-0007yM-Ql for submit <at> debbugs.gnu.org; Mon, 10 Jun 2024 11:41:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:42104) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1sGh92-0007xx-W2 for 71472 <at> debbugs.gnu.org; Mon, 10 Jun 2024 11:41:17 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <eliz@HIDDEN>) id 1sGh8e-0000zI-0m; Mon, 10 Jun 2024 11:40:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=g23sbqiFSTU0ofHwdn6N6B+VxOE2YcAfTYuft+5IeHk=; b=puu7fktm6slu mdszOjbgQ4omgfRqYxwjhgTu9JBiCp/E2E3wmbutSJYwp92+stv37Hd9T9b03Z7NZEguyAihC5hW5 skYBQxYe024vvSnkaTiv3+1IRQYPEXiehKESs0DsFVoPQl1PlILP/KJd6Wk4jafC9AVNbOIKYu36V HxRvJ0XzTk5lwDlvTLkKVqYaAiF+dJd2w7AfNYOaHcAw6e1dVwkUj5/eo/M9ZJDzLa//LtanmB2kA 0Zxu/3LRs+z4weUW51NzbyFImuomZvppt9d318yFLdceCgAKNLIzU1G5SiYwCx5kDgs5XeD0grJN8 uFIBmwNwz/bn9jtxdsZLUA==; Date: Mon, 10 Jun 2024 18:40:48 +0300 Message-Id: <86jziw956n.fsf@HIDDEN> From: Eli Zaretskii <eliz@HIDDEN> To: Ke Wu <ellpih@HIDDEN> In-Reply-To: <874ja1m6u1.fsf@HIDDEN> (message from Ke Wu on Mon, 10 Jun 2024 18:26:46 +0800) Subject: Re: bug#71472: [PATCH] Add pty support by using ConPTY on Windows References: <874ja1m6u1.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 71472 Cc: 71472 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) > From: Ke Wu <ellpih@HIDDEN> > Date: Mon, 10 Jun 2024 18:26:46 +0800 > > This patch adds pty support by using ConPTY on Windows. The conhost.exe > runs in pty mode and provides pty interface. The package term is also > patched to make it usable on Windows. Thanks. It's a pity you didn't discuss this before working on the code, because ConPTY is not very useful on MS-Windows due to the limitations of its encoding. See this issue I opened 3 years ago, and which they meanwhile closed (I guess they don't plan on fixing this anytime soon). See https://github.com/microsoft/terminal/issues/9174 If we must use UTF-8 as the only encoding to talk to sub-processes via ConPTY, that makes the number of applications that can be used this way very small, since most programs we are used to run as subprocesses, in particularly ports of GNU software like GCC, GDB, Grep, Find, and many others, cannot reliably talk to Emacs in UTF-8 encoding on MS-Windows. And without that, what would we use this feature for? So unfortunately, I don't think we should install these patches.
bug-gnu-emacs@HIDDEN:bug#71472; Package emacs.
Full text available.
Received: (at submit) by debbugs.gnu.org; 10 Jun 2024 14:40:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 10 10:40:17 2024
Received: from localhost ([127.0.0.1]:38647 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1sGgC0-00020U-1j
for submit <at> debbugs.gnu.org; Mon, 10 Jun 2024 10:40:17 -0400
Received: from lists.gnu.org ([209.51.188.17]:41086)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ellpih@HIDDEN>) id 1sGeCX-00045G-7O
for submit <at> debbugs.gnu.org; Mon, 10 Jun 2024 08:32:42 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ellpih@HIDDEN>)
id 1sGcEx-0006aT-KU
for bug-gnu-emacs@HIDDEN; Mon, 10 Jun 2024 06:27:03 -0400
Received: from sender-pp-o93.zoho.jp ([103.163.153.27])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ellpih@HIDDEN>)
id 1sGcEv-0006ZN-3p
for bug-gnu-emacs@HIDDEN; Mon, 10 Jun 2024 06:27:03 -0400
Received: by mx.zoho.jp with SMTPS id 17180152099851001.886543643876;
Mon, 10 Jun 2024 19:26:49 +0900 (JST)
From: Ke Wu <ellpih@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: [PATCH] Add pty support by using ConPTY on Windows
Date: Mon, 10 Jun 2024 18:26:46 +0800
Message-ID: <874ja1m6u1.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Zoho-Virus-Status: 1
X-Zoho-AV-Stamp: zmail-av-1.3.1/217.927.51
X-ZohoMailClient: External
Received-SPF: pass client-ip=103.163.153.27; envelope-from=ellpih@HIDDEN;
helo=sender-pp-o93.zoho.jp
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
X-Mailman-Approved-At: Mon, 10 Jun 2024 09:30:33 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)
--=-=-=
Content-Type: text/plain
Tags: patch
Tags: patch
This patch adds pty support by using ConPTY on Windows. The conhost.exe
runs in pty mode and provides pty interface. The package term is also
patched to make it usable on Windows.
In GNU Emacs 30.0.50 (build 4, x86_64-w64-mingw32) of 2024-06-10 built
on WIN1729
Repository revision: ed122417b98d711bacf5ed24778886bf21d86956
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 10.0.22631
System Description: Microsoft Windows 10 Pro (v10.0.2009.22631.3672)
Configured using:
'configure --prefix=/c/Users/oracl/Documents/Programs/emacs-dist
--without-dbus'
--=-=-=
Content-Type: text/patch
Content-Disposition: attachment;
filename=0001-Add-pty-support-by-using-ConPTY-on-Windows.patch
From 99bdacc3d6b9e6ea1ddcb0ae31b44afffe0ff697 Mon Sep 17 00:00:00 2001
From: Ke Wu <ellpih@HIDDEN>
Date: Mon, 10 Jun 2024 18:09:45 +0800
Subject: [PATCH] Add pty support by using ConPTY on Windows
* src/w32proc.c (w32-make-console-process): Add a function to create
process with pty interface using conhost.exe in pty mode.
* src/w32.c (make_console_with_pipe): Make a process using
conhost.exe. Attach a pipe to conhost.exe for resize. This function is
used by 'w32-make-console-process'.
* src/w32.h (make_console_with_pipe): Make above function definition
an extern function.
* src/sysdep.c (set_window_size): Implement window resize for
conhost.exe.
* src/process.c (set-process-window-size): Implement window resize.
* src/process.c (deactivate_process): Close pipe fd when process
exits. The pipe is attached to conhost.exe in 'make_console_with_pipe'.
* lisp/term.el (term-exec-1): Use 'w32-make-console-process' to create a
process with pty interface.
* lisp/term.el (term-coding-system): Set coding system to UTF-8 for
newly created buffer with terminal process attached since conhost.exe
assumes UTF-8 encoding in ConPTY mode.
---
lisp/term.el | 39 +++++++++++++++--------
src/process.c | 28 +++++++++++++++++
src/sysdep.c | 12 +++++++
src/w32.c | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++
src/w32.h | 2 ++
src/w32proc.c | 35 +++++++++++++++++++++
6 files changed, 190 insertions(+), 13 deletions(-)
diff --git a/lisp/term.el b/lisp/term.el
index c15f6cf2e9f..24509037805 100644
--- a/lisp/term.el
+++ b/lisp/term.el
@@ -1746,18 +1746,24 @@ term-exec-1
(when (term--bash-needs-EMACSp)
(push (format "EMACS=%s (term:%s)" emacs-version term-protocol-version)
process-environment))
- (apply #'start-process name buffer
- ;; On Android, /bin doesn't exist, and the default shell is
- ;; found as /system/bin/sh.
- (if (eq system-type 'android)
- "/system/bin/sh"
- "/bin/sh")
- "-c"
- (format "stty -nl echo rows %d columns %d sane 2>%s;\
+ (if (eq system-type 'windows-nt)
+ ;; Use `w32-make-console-process' to make use of the Windows ConPTY.
+ (apply #'w32-make-console-process
+ (append (list :name name :buffer buffer)
+ (list :command (cons command switches))
+ (list :width term-width :height term-height)))
+ (apply #'start-process name buffer
+ ;; On Android, /bin doesn't exist, and the default shell is
+ ;; found as /system/bin/sh.
+ (if (eq system-type 'android)
+ "/system/bin/sh"
+ "/bin/sh")
+ "-c"
+ (format "stty -nl echo rows %d columns %d sane 2>%s;\
if [ $1 = .. ]; then shift; fi; exec \"$@\""
- term-height term-width null-device)
- ".."
- command switches)))
+ term-height term-width null-device)
+ ".."
+ command switches))))
;;; Input history processing in a buffer
@@ -3007,6 +3013,13 @@ term-control-seq-regexp
(defconst term-control-seq-prefix-regexp
"[\032\e]")
+(defconst term-coding-system
+ (if (eq system-type 'windows-nt)
+ ;; The conhost.exe assumes UTF-8 for ConPTY.
+ 'utf-8-dos
+ locale-coding-system)
+ "Coding system for terminal in term.el.")
+
(defun term-emulate-terminal (proc str)
(when (buffer-live-p (process-buffer proc))
(with-current-buffer (process-buffer proc)
@@ -3074,7 +3087,7 @@ term-emulate-terminal
(setq decoded-substring
(decode-coding-string
(substring str i funny)
- locale-coding-system t))
+ term-coding-system t))
;; Check for multibyte characters that ends
;; before end of string, and save it for
;; next time.
@@ -3173,7 +3186,7 @@ term-emulate-terminal
(- ctl-end
(if (eq (aref str (- ctl-end 2)) ?\r)
2 1)))
- locale-coding-system t)))
+ term-coding-system t)))
(?\e
(pcase (aref str (1+ i))
(?\[
diff --git a/src/process.c b/src/process.c
index 9670be64279..b3e56bc8d9c 100644
--- a/src/process.c
+++ b/src/process.c
@@ -1497,12 +1497,32 @@ DEFUN ("set-process-window-size", Fset_process_window_size,
unsigned short h = check_uinteger_max (height, USHRT_MAX);
unsigned short w = check_uinteger_max (width, USHRT_MAX);
+#ifdef WINDOWSNT
+
+ Lisp_Object sigfd_obj;
+ int sigfd;
+ if (NETCONN_P (process))
+ return Qnil;
+
+ sigfd_obj = plist_get (XPROCESS (process) -> plist, QCfile_handler);
+ if (!FIXNUMP (sigfd_obj))
+ return Qnil;
+ sigfd = XFIXNUM (sigfd_obj);
+ if(sigfd < 0 || set_window_size (sigfd, h, w) < 0)
+ return Qnil;
+ else
+ return Qt;
+
+#else
+
if (NETCONN_P (process)
|| XPROCESS (process)->infd < 0
|| set_window_size (XPROCESS (process)->infd, h, w) < 0)
return Qnil;
else
return Qt;
+
+#endif
}
DEFUN ("set-process-inherit-coding-system-flag",
@@ -4783,6 +4803,14 @@ deactivate_process (Lisp_Object proc)
/* Delete GnuTLS structures in PROC, if any. */
emacs_gnutls_deinit (proc);
#endif /* HAVE_GNUTLS */
+#ifdef WINDOWSNT
+ /* Close write side of the pipe in console process of Windows */
+ Lisp_Object infd = plist_get(p -> plist, QCfile_handler);
+ if (!NILP (infd) && FIXNUMP(infd)) {
+ i = XFIXNUM(infd);
+ emacs_close(i);
+ }
+#endif /* WINDOWSNT */
if (p->read_output_delay > 0)
{
diff --git a/src/sysdep.c b/src/sysdep.c
index 07237885cb9..144ea4b413b 100644
--- a/src/sysdep.c
+++ b/src/sysdep.c
@@ -1481,8 +1481,20 @@ set_window_size (int fd, int height, int width)
size.ts_cols = width;
return ioctl (fd, TIOCGSIZE, &size);
+
+#else
+#ifdef WINDOWSNT
+
+ /* Windows console process */
+ unsigned short signal_packet[3];
+ signal_packet[0] = 8u;
+ signal_packet[1] = width;
+ signal_packet[2] = height;
+ return write(fd, signal_packet, sizeof(signal_packet));
+
#else
return -1;
+#endif /* not Windows NT */
#endif /* not SunOS-style */
#endif /* not BSD-style */
}
diff --git a/src/w32.c b/src/w32.c
index 6d0b178e978..b5708458a8c 100644
--- a/src/w32.c
+++ b/src/w32.c
@@ -11166,6 +11166,93 @@ register_aux_fd (int infd)
fd_info[ infd ].flags |= FILE_DONT_CLOSE;
}
+/* Start a console process by wraping the command in conhost.exe. The process is
+ started by calling make-process. The variable fds stores the infd and outfd of
+ the pipe serving as the signal pipe of conhost.exe. */
+Lisp_Object
+make_console_with_pipe (ptrdiff_t nargs, Lisp_Object * args, const int * fds)
+{
+
+ Lisp_Object command, contact;
+ Lisp_Object command_new, contact_new;
+ Lisp_Object width, height;
+ Lisp_Object process;
+ unsigned long pipe_outhnd;
+
+ HANDLE parent, newoutfd;
+
+ parent = GetCurrentProcess ();
+
+ /* Make inheritable copies of the fds[0]. */
+ if (!DuplicateHandle (parent,
+ (HANDLE) _get_osfhandle (fds[0]),
+ parent,
+ &newoutfd,
+ 0,
+ TRUE,
+ DUPLICATE_SAME_ACCESS))
+ report_file_error ("Duplicating input handle for child", Qnil);
+
+ emacs_close( fds[0] );
+ pipe_outhnd = (unsigned long) newoutfd;
+
+ /* Compose new command based on given parameters. */
+ contact = Flist (nargs, args);
+ command = plist_get (contact, QCcommand);
+ if (NILP (command))
+ return Qnil;
+ if (!CONSP (command))
+ command = list1 (command);
+ width = plist_get (contact, QCwidth);
+ height = plist_get (contact, QCheight);
+
+ command_new = CALLN (Flist,
+ build_string ("conhost.exe"),
+ build_string ("--headless"),
+ build_string ("--feature"),
+ build_string ("pty"));
+ if (!NILP (width))
+ command_new = CALLN (Fappend,
+ command_new,
+ CALLN (Flist,
+ build_string ("--width"),
+ CALLN (Fformat, build_string ("%d"), width)));
+
+ if (!NILP (height))
+ command_new = CALLN (Fappend,
+ command_new,
+ CALLN (Flist,
+ build_string ("--height"),
+ CALLN (Fformat, build_string ("%d"), height)));
+
+ command_new = CALLN (Fappend,
+ command_new,
+ CALLN (Flist, build_string ("--signal"),
+ CALLN (Fformat,
+ build_string ("0x%x"),
+ make_uint (pipe_outhnd))));
+
+ command_new = CALLN (Fappend,
+ command_new,
+ command);
+
+ contact_new = plist_put (contact, QCcommand, command_new);
+ process = CALLN (Fapply, Qmake_process, contact_new);
+
+ CloseHandle (newoutfd);
+ if (NILP (process))
+ emacs_close( fds[1] );
+ else {
+ fd_info[ fds[1] ].cp = fd_info[XPROCESS (process) -> infd].cp;
+ /* Store the signal pipe's out fd in process plist. Not sure if
+ QCfile_handler is a proper key. */
+ XPROCESS (process) -> plist =
+ plist_put (XPROCESS (process) -> plist,
+ QCfile_handler, make_uint(fds[1]));
+ }
+ return process;
+}
+
#ifdef HAVE_GNUTLS
ssize_t
diff --git a/src/w32.h b/src/w32.h
index cf470ae9901..7393c5299f7 100644
--- a/src/w32.h
+++ b/src/w32.h
@@ -230,6 +230,8 @@ #define FILE_DONT_CLOSE 0x1000
extern int lchmod (char const *, mode_t);
extern bool symlinks_supported (const char *);
+/* Create console process with signal pipe */
+extern Lisp_Object make_console_with_pipe (ptrdiff_t, Lisp_Object *, const int *);
/* Return total and free memory info. */
extern int w32_memory_info (unsigned long long *, unsigned long long *,
diff --git a/src/w32proc.c b/src/w32proc.c
index 55ead13647b..efb32c3ed7e 100644
--- a/src/w32proc.c
+++ b/src/w32proc.c
@@ -3726,6 +3726,39 @@ DEFUN ("w32-set-keyboard-layout", Fw32_set_keyboard_layout,
return Fw32_get_keyboard_layout ();
}
+DEFUN ("w32-make-console-process", Fw32_make_console_process,
+ Sw32_make_console_process, 0, MANY, 0,
+ doc: /* Start a process wrapped in conhost.exe.
+
+This is similar to `make-process', which following extra arguments:
+
+:width WIDTH -- WIDTH is the initial width of the conhost.exe process.
+
+:width HEIGHT -- HEIGHT is the initial height of the conhost.exe process.
+
+The conhost.exe runs in pty mode, which acts like a pty devices in *NIX. With
+following differences:
+
+1. The text going in and out from the stdin and the stdout of conhost.exe are
+always encoded in UTF-8.
+
+2. The conhost.exe requires an extra pipe to send signals which causes the
+console resize. This is implemented in this function. */)
+ (ptrdiff_t nargs, Lisp_Object *args)
+{
+
+ Lisp_Object process;
+ int fds[2];
+
+ /* Create signal pipe for this process */
+ if (emacs_pipe (fds) < 0)
+ report_file_error ("Creating signal pipe for console process", Qnil);
+ fd_info[ fds[1] ].hnd = (HANDLE) _get_osfhandle (fds[1]);
+
+ process = make_console_with_pipe(nargs, args, fds);
+ return process;
+}
+
/* Two variables to interface between get_lcid and the EnumLocales
callback function below. */
#ifndef LOCALE_NAME_MAX_LENGTH
@@ -3949,6 +3982,8 @@ syms_of_ntproc (void)
defsubr (&Sw32_get_keyboard_layout);
defsubr (&Sw32_set_keyboard_layout);
+ defsubr (&Sw32_make_console_process);
+
DEFVAR_LISP ("w32-quote-process-args", Vw32_quote_process_args,
doc: /* Non-nil enables quoting of process arguments to ensure correct parsing.
Because Windows does not directly pass argv arrays to child processes,
--
2.45.1
--=-=-=--
Ke Wu <ellpih@HIDDEN>:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#71472; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.