GNU bug report logs - #47861
Starting `jsonrpc-process-connection' over Tramp fails if the process writes to stderr

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 47861 in the body.
You can then email your comments to 47861 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Starting `jsonrpc-process-connection' over Tramp fails if the process
 writes to stderr
Date: Sat, 17 Apr 2021 20:53:37 -0700

When starting a `jsonrpc-process-connection' over Tramp, if the
process writes to stderr, Emacs (usually) fails with `Wrong type
argument: "inserted-chars 204"'. I'm able to reproduce this pretty
consistently when connecting from an MS Windows client to a Linux
server; however, this may apply to other platform combinations as
well. Mysteriously, it works fine about 1 in 10 times, though I
haven't been able to decipher why.

This is a reduced test case from an eglot issue that prevents users
from using eglot with clangd over Tramp (or likely any LSP server that
writes to stderr): <https://github.com/joaotavora/eglot/issues/662>.
Michael Albinus has a workaround for this for the next version of
Tramp (2.5.0.4, I believe), but suggested I file an issue here to help
find a more-permanent solution.

The following recipe reproduces the issue for me on both Emacs 27.2
and the prebuilt Emacs 28 snapshot from 2021-01-15. `jsonrpc-minimal'
is a cut-down version of `eglot--connect'. Note that this test tries
to run `clangd', but any executable that writes to stderr on startup
should reproduce this issue (including a nonexistent executable).
After running "emacs -Q":

----------[ Eval the following ]----------
;; Optional: disable ControlMaster if using ssh on Win32; it doesn't work there.
(require 'tramp)
(setq tramp-use-ssh-controlmaster-options nil)

(require 'jsonrpc)
(defun jsonrpc-minimal ()
  (interactive)
  (make-instance
   'jsonrpc-process-connection
   :name "jsonrpc-minimal"
   :process (lambda ()
              (make-process
               :name "jsonrpc-minimal"
               :command '("sh" "-c" "stty raw > /dev/null; clangd")
               :connection-type 'pipe
               :coding 'utf-8-emacs-unix
               :noquery t
               :stderr (get-buffer-create "*jsonrpc-minimal stderr*")
               :file-handler t))))
----------[ End ]----------

Then run `C-x C-f /sshx:server:~/path/to/file.txt RET' followed by
`M-x jsonrpc-minimal'. (Other Tramp methods such as plink should also
reproduce this issue.) If `debug-on-error' is set, I get the following
backtrace:

----------[ Begin backtrace ]----------
  signal(wrong-type-argument ("inserted-chars 204"))
  tramp-signal-hook-function(wrong-type-argument ("inserted-chars 204"))
  signal(wrong-type-argument ("inserted-chars 204"))
  tramp-handle-insert-file-contents("/sshx:server:/tmp/tramp.lLZT4H"
nil nil nil nil)
  apply(tramp-handle-insert-file-contents
("/sshx:server:/tmp/tramp.lLZT4H" nil nil nil nil))
  tramp-sh-file-name-handler(insert-file-contents
"/sshx:server:/tmp/tramp.lLZT4H" nil nil nil nil)
  apply(tramp-sh-file-name-handler insert-file-contents
("/sshx:server:/tmp/tramp.lLZT4H" nil nil nil nil))
  tramp-file-name-handler(insert-file-contents
"/sshx:server:/tmp/tramp.lLZT4H" nil nil nil nil)
  insert-file-contents("/sshx:server:/tmp/tramp.lLZT4H" nil nil nil nil)
  insert-file-contents-literally("/sshx:server:/tmp/tramp.lLZT4H")
  tramp-sh-handle-make-process(:name "jsonrpc-minimal" :command ("sh"
"-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding
utf-8-emacs-unix :noquery t :stderr #<buffer *jsonrpc-minimal stderr*>
:file-handler t)
  apply(tramp-sh-handle-make-process (:name "jsonrpc-minimal" :command
("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe
:coding utf-8-emacs-unix :noquery t :stderr #<buffer *jsonrpc-minimal
stderr*> :file-handler t))
  tramp-sh-file-name-handler(make-process :name "jsonrpc-minimal"
:command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type
pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer
*jsonrpc-minimal stderr*> :file-handler t)
  apply(tramp-sh-file-name-handler make-process (:name
"jsonrpc-minimal" :command ("sh" "-c" "stty raw > /dev/null; clangd")
:connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr
#<buffer *jsonrpc-minimal stderr*> :file-handler t))
  tramp-file-name-handler(make-process :name "jsonrpc-minimal"
:command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type
pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer
*jsonrpc-minimal stderr*> :file-handler t)
  make-process(:name "jsonrpc-minimal" :command ("sh" "-c" "stty raw >
/dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix
:noquery t :stderr #<buffer *jsonrpc-minimal stderr*> :file-handler t)
  (closure (t) nil (make-process :name "jsonrpc-minimal" :command
'("sh" "-c" "stty raw > /dev/null; clangd") :connection-type 'pipe
:coding 'utf-8-emacs-unix :noquery t :stderr (get-buffer-create
"*jsonrpc-minimal stderr*") :file-handler t))()
  #f(compiled-function (cl--cnm conn slots) #<bytecode
0x18017ea42140ed74>)(#f(compiled-function (&rest cnm-args) #<bytecode
0x69ccf178fb9f2f9>) #<jsonrpc-process-connection
jsonrpc-process-connection-192d52c> (:name "jsonrpc-minimal" :process
(closure (t) nil (make-process :name "jsonrpc-minimal" :command '("sh"
"-c" "stty raw > /dev/null; clangd") :connection-type 'pipe :coding
'utf-8-emacs-unix :noquery t :stderr (get-buffer-create
"*jsonrpc-minimal stderr*") :file-handler t))))
  apply(#f(compiled-function (cl--cnm conn slots) #<bytecode
0x18017ea42140ed74>) #f(compiled-function (&rest cnm-args) #<bytecode
0x69ccf178fb9f2f9>) (#<jsonrpc-process-connection
jsonrpc-process-connection-192d52c> (:name "jsonrpc-minimal" :process
(closure (t) nil (make-process :name "jsonrpc-minimal" :command '("sh"
"-c" "stty raw > /dev/null; clangd") :connection-type 'pipe :coding
'utf-8-emacs-unix :noquery t :stderr (get-buffer-create
"*jsonrpc-minimal stderr*") :file-handler t)))))
  #f(compiled-function (&rest args) #<bytecode
0x101ee737e4a4203a>)(#<jsonrpc-process-connection
jsonrpc-process-connection-192d52c> (:name "jsonrpc-minimal" :process
(closure (t) nil (make-process :name "jsonrpc-minimal" :command '("sh"
"-c" "stty raw > /dev/null; clangd") :connection-type 'pipe :coding
'utf-8-emacs-unix :noquery t :stderr (get-buffer-create
"*jsonrpc-minimal stderr*") :file-handler t))))
  apply(#f(compiled-function (&rest args) #<bytecode
0x101ee737e4a4203a>) #<jsonrpc-process-connection
jsonrpc-process-connection-192d52c> (:name "jsonrpc-minimal" :process
(closure (t) nil (make-process :name "jsonrpc-minimal" :command '("sh"
"-c" "stty raw > /dev/null; clangd") :connection-type 'pipe :coding
'utf-8-emacs-unix :noquery t :stderr (get-buffer-create
"*jsonrpc-minimal stderr*") :file-handler t))))
  initialize-instance(#<jsonrpc-process-connection
jsonrpc-process-connection-192d52c> (:name "jsonrpc-minimal" :process
(closure (t) nil (make-process :name "jsonrpc-minimal" :command '("sh"
"-c" "stty raw > /dev/null; clangd") :connection-type 'pipe :coding
'utf-8-emacs-unix :noquery t :stderr (get-buffer-create
"*jsonrpc-minimal stderr*") :file-handler t))))
  #f(compiled-function (class &rest slots) "Default constructor for
CLASS `eieio-default-superclass'.\nSLOTS are the initialization slots
used by `initialize-instance'.\nThis static method is called when an
object is constructed.\nIt allocates the vector used to represent an
EIEIO object, and then\ncalls `initialize-instance' on that object."
#<bytecode -0x1cebd9ffedfcea53>)(jsonrpc-process-connection :name
"jsonrpc-minimal" :process (closure (t) nil (make-process :name
"jsonrpc-minimal" :command '("sh" "-c" "stty raw > /dev/null; clangd")
:connection-type 'pipe :coding 'utf-8-emacs-unix :noquery t :stderr
(get-buffer-create "*jsonrpc-minimal stderr*") :file-handler t)))
  apply(#f(compiled-function (class &rest slots) "Default constructor
for CLASS `eieio-default-superclass'.\nSLOTS are the initialization
slots used by `initialize-instance'.\nThis static method is called
when an object is constructed.\nIt allocates the vector used to
represent an EIEIO object, and then\ncalls `initialize-instance' on
that object." #<bytecode -0x1cebd9ffedfcea53>)
jsonrpc-process-connection (:name "jsonrpc-minimal" :process (closure
(t) nil (make-process :name "jsonrpc-minimal" :command '("sh" "-c"
"stty raw > /dev/null; clangd") :connection-type 'pipe :coding
'utf-8-emacs-unix :noquery t :stderr (get-buffer-create
"*jsonrpc-minimal stderr*") :file-handler t))))
  make-instance(jsonrpc-process-connection :name "jsonrpc-minimal"
:process (closure (t) nil (make-process :name "jsonrpc-minimal"
:command '("sh" "-c" "stty raw > /dev/null; clangd") :connection-type
'pipe :coding 'utf-8-emacs-unix :noquery t :stderr (get-buffer-create
"*jsonrpc-minimal stderr*") :file-handler t)))
  jsonrpc-minimal()
  funcall-interactively(jsonrpc-minimal)
  call-interactively(jsonrpc-minimal record nil)
  command-execute(jsonrpc-minimal record)
  execute-extended-command(nil "jsonrpc-minimal" "jsonrpc-minimal")
  funcall-interactively(execute-extended-command nil "jsonrpc-minimal"
"jsonrpc-minimal")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)
----------[ End backtrace ]----------

----------[ Begin Emacs debug info ]----------
In GNU Emacs 28.0.50 (build 1, x86_64-w64-mingw32)
 of 2021-01-15 built on CIRROCUMULUS
Repository revision: f45be48ddbde00610e1e08fca6590dcf24a4e1b5
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 10.0.19041
System Description: Microsoft Windows 10 Pro (v10.0.2004.19041.928)

Configured using:
 'configure --without-dbus --without-compress-install -C 'CFLAGS=-O2
 -static -g3' PKG_CONFIG_PATH=/mingw64/lib/pkgconfig'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NOTIFY
W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS XPM
ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils mule-util help-fns
radix-tree cl-print cus-start cus-load vc-hg vc-git diff-mode easy-mmode
vc-bzr cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align
cc-engine cc-vars cc-defs tramp-cache tramp-sh jsonrpc ert pp ewoc debug
backtrace help-mode easymenu find-func pcase warnings tramp
tramp-loaddefs trampver tramp-integration files-x tramp-compat shell
pcomplete comint ansi-color ring parse-time iso8601 format-spec
auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map seq byte-opt gv bytecomp byte-compile cconv
time-date subr-x cl-loaddefs cl-lib iso-transl tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp
disp-table term/w32-win w32-win w32-vars term/common-win tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads w32notify
w32 lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 148954 7276)
 (symbols 48 13698 1)
 (strings 32 40410 1316)
 (string-bytes 1 1461011)
 (vectors 16 21458)
 (vector-slots 8 300360 13838)
 (floats 8 55 160)
 (intervals 56 2822 0)
 (buffers 984 18))
----------[ End Emacs debug info ]----------

Message #8 received at 47861 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: 47861 <at> debbugs.gnu.org
Subject: Re: bug#47861: Starting `jsonrpc-process-connection' over Tramp
 fails if the process writes to stderr
Date: Sat, 01 May 2021 11:43:42 +0200

Jim Porter <jporterbugs <at> gmail.com> writes:

> When starting a `jsonrpc-process-connection' over Tramp, if the
> process writes to stderr, Emacs (usually) fails with `Wrong type
> argument: "inserted-chars 204"'. I'm able to reproduce this pretty
> consistently when connecting from an MS Windows client to a Linux
> server; however, this may apply to other platform combinations as
> well. Mysteriously, it works fine about 1 in 10 times, though I
> haven't been able to decipher why.

In current Tramp 2.5.0.4, this has been mitigated by wrapping the
insert-file-contents-literally call in tramp-sh-handle-make-process with
ignore-errors.

However, handling stderr buffers in Tramp's make-process is still a
mess. So I've started to reimplement this, using a named pipe on the
remote machine. make-pipe-process, the natural choice, does not work for
remote processes.

Best regards, Michael.

Message #11 received at 47861 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: 47861 <at> debbugs.gnu.org
Subject: Re: bug#47861: Starting `jsonrpc-process-connection' over Tramp
 fails if the process writes to stderr
Date: Sat, 15 May 2021 19:32:59 +0200

[Message part 1 (text/plain, inline)]

Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Jim,

> However, handling stderr buffers in Tramp's make-process is still a
> mess. So I've started to reimplement this, using a named pipe on the
> remote machine. make-pipe-process, the natural choice, does not work for
> remote processes.

I've finished a first shot on this, see the appended patch. Would you
mind to test this in your environment, for example with eglot?

Thanks, and best regards, Michael.

[Message part 2 (text/x-patch, attachment)]

Message #14 received at 47861 <at> debbugs.gnu.org (full text, mbox):

From: Jim Porter <jporterbugs <at> gmail.com>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 47861 <at> debbugs.gnu.org
Subject: Re: bug#47861: Starting `jsonrpc-process-connection' over Tramp fails
 if the process writes to stderr
Date: Fri, 21 May 2021 21:53:10 -0700

On Sat, May 15, 2021 at 10:33 AM Michael Albinus <michael.albinus <at> gmx.de> wrote:
> Michael Albinus <michael.albinus <at> gmx.de> writes:
>
> > However, handling stderr buffers in Tramp's make-process is still a
> > mess. So I've started to reimplement this, using a named pipe on the
> > remote machine. make-pipe-process, the natural choice, does not work for
> > remote processes.
>
> I've finished a first shot on this, see the appended patch. Would you
> mind to test this in your environment, for example with eglot?

I've tested this patch (using commit
e89a2304a152d6ee9e5d75efe0d22cabab771388 from the Tramp repository,
which includes your above patch), and everything works correctly as
far as I can tell.

I also wanted to be sure I could still reproduce the original issue,
so I confirmed that if I roll Tramp back to a previous revision before
your workaround (I used commit
f9301e5d4424bd567dc212f579350c5e04397ba1), I see the bug again.

Thanks for the fix!

Message #19 received at 47861-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Jim Porter <jporterbugs <at> gmail.com>
Cc: 47861-done <at> debbugs.gnu.org
Subject: Re: bug#47861: Starting `jsonrpc-process-connection' over Tramp
 fails if the process writes to stderr
Date: Sat, 22 May 2021 09:27:57 +0200

Version: 28.1

Jim Porter <jporterbugs <at> gmail.com> writes:

Hi Jim,

>> I've finished a first shot on this, see the appended patch. Would you
>> mind to test this in your environment, for example with eglot?
>
> I've tested this patch (using commit
> e89a2304a152d6ee9e5d75efe0d22cabab771388 from the Tramp repository,
> which includes your above patch), and everything works correctly as
> far as I can tell.
>
> I also wanted to be sure I could still reproduce the original issue,
> so I confirmed that if I roll Tramp back to a previous revision before
> your workaround (I used commit
> f9301e5d4424bd567dc212f579350c5e04397ba1), I see the bug again.

Thanks for the feedback! I'm closing the bug, the patch will be included
in the next Tramp 2.5.0.5 version.

> Thanks for the fix!

Best regards, Michael.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.