GNU bug report logs - #12440
24.2; process-send-string only sends 4K for long strings

Previous Next

To reply to this bug, email your comments to 12440 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Hendrik Tews <tews <at> os.inf.tu-dresden.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.2; process-send-string only sends 4K for long strings
Date: Fri, 14 Sep 2012 11:08:00 +0200

Hi,

in Emacs 24.2, the following code

  (let* ((process-connection-type t)
         (process (start-process "cat" (current-buffer) "cat")))
    (process-send-string process (format "%s\n" (make-string 6000 ?a))))

inserts only 4095 characters in the current buffer. Setting
process-connection-type to nil will insert all characters. In Emacs 23
the code works always as expected.

This is probably related to Emacs bug #7078. I cannot reproduce the
problem described there (with 257 characters), probably because the
buffer size was increased to 4K.

Bye,

Hendrik Tews



In GNU Emacs 24.2.1 (i686-pc-linux-gnu, GTK+ Version 2.24.10)
 of 2012-09-13 on blau
Windowing system distributor `The XFree86 Project, Inc', version 11.0.40300000
Configured using:
 `configure '--prefix' '/usr/local/stow/emacs-24.2' '--with-xpm=no'
 '--with-jpeg=no' '--with-gif=no' '--with-tiff=no''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<return> C-x s ! ! <backspace> C-x s ! <f9> C-x b T 
O <tab> <return> C-x C-v <return> <down> <down> <down> 
<down> <down> <down> <down> C-SPC <down> <down> M-w 
C-x b <return> <backspace> C-y <left> C-j M-x e m a 
c s - v e r <tab> <return> <up> <up> <down> <down> 
<down> C-k C-k <up> <up> <up> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <backspace> <backspace> <backspace> 6 0 0 
0 C-e C-j <down> <down> <down> C-SPC C-e M-= M-< A-f 
<down> <down> <down> <C-down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <up> <M-right> * <right> <right> C-o ( p r o 
c e s s - c o n n M-/ <left> <left> C-h v <return> 
C-e SPC n i l ) <down> <tab> C-e <backspace> ) <down> 
<down> <down> <down> <down> <down> C-a C-k <up> <up> 
<up> <up> <up> C-e <backspace> ) C-j C-x 1 <down> <down> 
<down> <down> <down> <down> C-SPC C-e M-= C-a C-k <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> C-e <left> 
<M-backspace> t <down> <down> C-e C-j <down> <down> 
<down> <down> <down> <down> <down> <down> <down> C-SPC 
C-e M-= C-a C-SPC <right> M-= C-a C-SPC C-e M-= C-a 
M-f C-a M-1 C-f C-a M-4 M-0 M-9 M-6 C-f C-a <up> C-a 
M-4 M-0 M-9 M-5 C-f C-a <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <down> C-k 
C-k C-k C-k C-k C-x b T O <tab> <return> <up> <up> 
C-y <return> C-k C-k C-k C-k C-o C-x C-s <up> <up> 
<up> <up> <down> M-x e m a c s - v e r <tab> <return> 
M-x r e p o r t - e m <tab> <return>

Recent messages:
Mark set
Region has 1 line, 1 word, and 4095 characters.
Mark activated
Region has 1 line, 1 word, and 1 character.
Mark activated
Region has 1 line, 1 word, and 4095 characters.
Mark set
Saving file /home/tews/src/pg/TODO...
Wrote /home/tews/src/pg/TODO
GNU Emacs 24.2.1 (i686-pc-linux-gnu, GTK+ Version 2.24.10) of 2012-09-13 on blau

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils jka-compr find-func etags debug dabbrev parse-time
vc-cvs tabulated-list pp help-mode view which-func imenu coq edmacro
kmacro coq-smie-lexer smie coq-indent coq-abbrev coq-local-vars
local-vars-list coq-syntax regexp-opt coq-db holes proof proof-shell
pg-user completion pg-goals pg-response proof-toolbar pg-assoc
proof-tree proof-script proof-menu cus-edit cus-start cus-load wid-edit
span proof-auxmodes proof-utils scomint proof-syntax advice help-fns
advice-preload bufhist easy-mmode ring proof-splash derived pg-custom
proof-config proof-faces proof-useropts pg-pamacs proof-compat easymenu
proof-site proof-autoloads cl pg-vars misearch multi-isearch time-date
tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar
dnd fontset image fringe lisp-mode register page menu-bar rfn-eshadow
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

Message #10 received at 12440 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Hendrik Tews <tews <at> os.inf.tu-dresden.de>
Cc: 12440 <at> debbugs.gnu.org
Subject: Re: bug#12440: 24.2; process-send-string only sends 4K for long
 strings
Date: Mon, 07 Oct 2019 16:53:24 +0200

Hendrik Tews <tews <at> os.inf.tu-dresden.de> writes:

> in Emacs 24.2, the following code
>
>   (let* ((process-connection-type t)
>          (process (start-process "cat" (current-buffer) "cat")))
>     (process-send-string process (format "%s\n" (make-string 6000 ?a))))
>
> inserts only 4095 characters in the current buffer. Setting
> process-connection-type to nil will insert all characters.

I can confirm that this bug is still present in Emacs 27.

It's odd that this doesn't cause more problems than it does in practice,
but I guess it's rare that we talk to processes this way.

The bug was apparently introduced by the patch below, but I have to
admit when reading it, I don't quite understand what's going on...

commit 2b0a91e78f83fb446cc38efb99399e83ad2cded3
Author: Stefan Monnier <monnier <at> iro.umontreal.ca>
Date:   Mon Apr 12 22:07:48 2010 -0400

    Try to solve the problem of spurious EOF chars in long lines of text
    sent to interactive subprocesses.
    * sysdep.c (child_setup_tty): Do not enable ICANON any more.
    (system_process_attributes): Remove unused var `ttotal'.
    * process.c (send_process): Don't bother breaking long line with EOF
    chars when talking to ttys any more.
    (wait_reading_process_output): Output a warning when called in such
    a way that it could block without being interruptible.

diff --git a/src/ChangeLog b/src/ChangeLog
index ad88dc8311..f9567b1308 100644
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,5 +1,14 @@
 2010-04-13  Stefan Monnier  <monnier <at> iro.umontreal.ca>
 
+	Try to solve the problem of spurious EOF chars in long lines of text
+	sent to interactive subprocesses.
+	* sysdep.c (child_setup_tty): Do not enable ICANON any more.
+	(system_process_attributes): Remove unused var `ttotal'.
+	* process.c (send_process): Don't bother breaking long line with EOF
+	chars when talking to ttys any more.
+	(wait_reading_process_output): Output a warning when called in such
+	a way that it could block without being interruptible.
+
 	Try to detect file modification within the same second.
 	* buffer.h (struct buffer): New field modtime_size.
 	* buffer.c (reset_buffer): Initialize it.
diff --git a/src/process.c b/src/process.c
index 34aa2c4fcf..7e8f4cc57b 100644
--- a/src/process.c
+++ b/src/process.c
@@ -4643,6 +4643,10 @@ wait_reading_process_output (time_limit, microsecs, read_kbd, do_display,
   FD_ZERO (&Connecting);
 #endif
 
+  if (time_limit == 0 && wait_proc && !NILP (Vinhibit_quit)
+      && !(CONSP (wait_proc->status) && EQ (XCAR (wait_proc->status), Qexit)))
+    message ("Blocking call to accept-process-output with quit inhibited!!");
+
   /* If wait_proc is a process to watch, set wait_channel accordingly.  */
   if (wait_proc != NULL)
     wait_channel = wait_proc->infd;
@@ -5768,34 +5772,6 @@ send_process (proc, buf, len, object)
 	{
 	  int this = len;
 
-	  /* Decide how much data we can send in one batch.
-	     Long lines need to be split into multiple batches.  */
-	  if (p->pty_flag)
-	    {
-	      /* Starting this at zero is always correct when not the first
-		 iteration because the previous iteration ended by sending C-d.
-		 It may not be correct for the first iteration
-		 if a partial line was sent in a separate send_process call.
-		 If that proves worth handling, we need to save linepos
-		 in the process object.  */
-	      int linepos = 0;
-	      unsigned char *ptr = (unsigned char *) buf;
-	      unsigned char *end = (unsigned char *) buf + len;
-
-	      /* Scan through this text for a line that is too long.  */
-	      while (ptr != end && linepos < pty_max_bytes)
-		{
-		  if (*ptr == '\n')
-		    linepos = 0;
-		  else
-		    linepos++;
-		  ptr++;
-		}
-	      /* If we found one, break the line there
-		 and put in a C-d to force the buffer through.  */
-	      this = ptr - buf;
-	    }
-
 	  /* Send this batch, using one or more write calls.  */
 	  while (this > 0)
 	    {
@@ -5899,11 +5875,6 @@ send_process (proc, buf, len, object)
 	      len -= rv;
 	      this -= rv;
 	    }
-
-	  /* If we sent just part of the string, put in an EOF (C-d)
-	     to force it through, before we send the rest.  */
-	  if (len > 0)
-	    Fprocess_send_eof (proc);
 	}
     }
   else
diff --git a/src/sysdep.c b/src/sysdep.c
index 37e7dfbaf9..506af23ef3 100644
--- a/src/sysdep.c
+++ b/src/sysdep.c
@@ -529,8 +529,6 @@ child_setup_tty (out)
 #endif
   s.main.c_oflag &= ~TAB3;	/* Disable tab expansion */
   s.main.c_cflag = (s.main.c_cflag & ~CSIZE) | CS8; /* Don't strip 8th bit */
-  s.main.c_lflag |= ICANON;	/* Enable erase/kill and eof processing */
-  s.main.c_cc[VEOF] = 04;	/* insure that EOF is Control-D */
   s.main.c_cc[VERASE] = CDISABLE;	/* disable erase processing */
   s.main.c_cc[VKILL] = CDISABLE;	/* disable kill processing */
 
@@ -560,7 +558,6 @@ child_setup_tty (out)
   /* rms: Formerly it set s.main.c_cc[VINTR] to 0377 here
      unconditionally.  Then a SIGNALS_VIA_CHARACTERS conditional
      would force it to 0377.  That looks like duplicated code.  */
-  s.main.c_cc[VEOL] = CDISABLE;
   s.main.c_cflag = (s.main.c_cflag & ~CBAUD) | B9600; /* baud rate sanity */
 #endif /* AIX */
 
@@ -573,6 +570,18 @@ child_setup_tty (out)
   s.main.sg_kill = 0377;
   s.lmode = LLITOUT | s.lmode;        /* Don't strip 8th bit */
 
+  /* We used to enable ICANON (and set VEOF to 04), but this leads to
+     problems where process.c wants to send EOFs every once in a while
+     to force the output, which leads to weird effects when the
+     subprocess has disabled ICANON and ends up seeing those spurious
+     extra EOFs.  So we don't send EOFs any more in
+     process.c:send_process, and instead we disable ICANON by default,
+     so if a subsprocess sets up ICANON, it's his problem (or the Elisp
+     package that talks to it) to deal with lines that are too long.  */
+  s.main.c_lflag &= ~ICANON;	/* Disable line editing and eof processing */
+  s.main.c_cc[VMIN] = 1;
+  s.main.c_cc[VTIME] = 0;
+
 #endif /* not HAVE_TERMIO */
 
   EMACS_SET_TTY (out, &s, 0);
@@ -3344,7 +3353,7 @@ system_process_attributes (Lisp_Object pid)
   unsigned long minflt, majflt, cminflt, cmajflt, vsize;
   time_t sec;
   unsigned usec;
-  EMACS_TIME tnow, tstart, tboot, telapsed,ttotal;
+  EMACS_TIME tnow, tstart, tboot, telapsed;
   double pcpu, pmem;
   Lisp_Object attrs = Qnil;
   Lisp_Object cmd_str, decoded_cmd, tem;


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #15 received at 12440 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 12440 <at> debbugs.gnu.org, tews <at> os.inf.tu-dresden.de
Subject: Re: bug#12440: 24.2;
 process-send-string only sends 4K for long strings
Date: Mon, 07 Oct 2019 19:45:11 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Date: Mon, 07 Oct 2019 16:53:24 +0200
> Cc: 12440 <at> debbugs.gnu.org
> 
> Hendrik Tews <tews <at> os.inf.tu-dresden.de> writes:
> 
> > in Emacs 24.2, the following code
> >
> >   (let* ((process-connection-type t)
> >          (process (start-process "cat" (current-buffer) "cat")))
> >     (process-send-string process (format "%s\n" (make-string 6000 ?a))))
> >
> > inserts only 4095 characters in the current buffer. Setting
> > process-connection-type to nil will insert all characters.
> 
> I can confirm that this bug is still present in Emacs 27.
> 
> It's odd that this doesn't cause more problems than it does in practice,
> but I guess it's rare that we talk to processes this way.

IMO, when a Lisp program intends to communicate with a subprocess this
way, it should indeed set process-connect-type to nil.  It makes no
sense to me to use PTYs to communicate massive amounts of data in
batch-like manner.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.