GNU logs - #33747, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#33747: 26.1; process-send-string exceeds max-specpdl-size
Resent-From: Markus Triska <triska@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 14 Dec 2018 17:51:02 +0000
Resent-Message-ID: <handler.33747.B.154480985519923 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 33747 <at> debbugs.gnu.org
From: Markus Triska <triska@HIDDEN>
Date: Fri, 14 Dec 2018 18:49:09 +0100
Message-ID: <87bm5o2e3u.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

To reproduce this issue, please install the PostScript viewer
Ghostscript ("gs"), and put the following forms in "pipe.el":

    (setq target (start-process "gs" nil "gs" "-g100x100")
          source (start-process "bash" nil "bash"
                                "-c" "while true; do echo \
      rand 2 31 exp div setgray 0 0 100 100 rectfill; done"))

    (set-process-filter source 'send-to-target)

    (defun send-to-target (proc str)
      (process-send-string target str))

When you then invoke Emacs via:

    $ emacs -Q pipe.el --eval '(eval-buffer)'

you get almost instantly:

    error in process filter: Variable binding depth exceeds max-specpdl-size

The reason for this seems to be that process-send-string also builds up
frames for unwind-protect when it waits for input if the queue is full.

However, in this concrete case, I would prefer to simply send (or queue)
the string and *not* wait for any input within the process filter.

Is there a way to accomplish this? Alternatively, would you please
consider to provide such a mechanism?

For example, when sending output to a process and the output queue of
that process is full, would it be admissible or even preferable to wait
only for input of that specific process, instead of also waiting for and
handling input from other processes (as seems to be the case now)?

Thank you and all the best!
Markus

In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars)
 of 2018-11-18 built on debian
Repository revision: 07f8f9bc5a51f5aa94eb099f3e15fbe0c20ea1ea
Windowing system distributor 'The X.Org Foundation', version 11.0.11902000
System Description:	Debian GNU/Linux 9.6 (stretch)

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Markus Triska <triska@HIDDEN>
Subject: bug#33747: Acknowledgement (26.1; process-send-string exceeds
 max-specpdl-size)
Message-ID: <handler.33747.B.154480985519923.ack <at> debbugs.gnu.org>
References: <87bm5o2e3u.fsf@HIDDEN>
X-Gnu-PR-Message: ack 33747
X-Gnu-PR-Package: emacs
Reply-To: 33747 <at> debbugs.gnu.org
Date: Fri, 14 Dec 2018 17:51:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 33747 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
33747: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D33747
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#33747: 26.1; process-send-string exceeds max-specpdl-size
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 14 Dec 2018 18:41:02 +0000
Resent-Message-ID: <handler.33747.B33747.154481282224844 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Markus Triska <triska@HIDDEN>
Cc: 33747 <at> debbugs.gnu.org
Date: Fri, 14 Dec 2018 20:40:04 +0200
Message-Id: <83o99oeyuz.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-reply-to: <87bm5o2e3u.fsf@HIDDEN> (message from Markus Triska on Fri, 
 14 Dec 2018 18:49:09 +0100)
References: <87bm5o2e3u.fsf@HIDDEN>
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

> From: Markus Triska <triska@HIDDEN>
> Date: Fri, 14 Dec 2018 18:49:09 +0100
> 
> The reason for this seems to be that process-send-string also builds up
> frames for unwind-protect when it waits for input if the queue is full.
> 
> However, in this concrete case, I would prefer to simply send (or queue)
> the string and *not* wait for any input within the process filter.
> 
> Is there a way to accomplish this?

AFAIK, process-send-string is a blocking function: it cannot return
before the entire string was sent.

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#33747: 26.1; process-send-string exceeds max-specpdl-size
In-Reply-To: <87bm5o2e3u.fsf@HIDDEN>
Resent-From: Markus Triska <triska@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 14 Dec 2018 18:51:02 +0000
Resent-Message-ID: <handler.33747.B33747.154481344525834 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 33747 <at> debbugs.gnu.org
From: Markus Triska <triska@HIDDEN>
References: <87bm5o2e3u.fsf@HIDDEN> <83o99oeyuz.fsf@HIDDEN>
Date: Fri, 14 Dec 2018 19:50:39 +0100
Message-ID: <874lbgeydc.fsf@HIDDEN>
User-Agent: Emacs/24.5
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Eli Zaretskii <eliz@HIDDEN> writes:

> AFAIK, process-send-string is a blocking function: it cannot return
> before the entire string was sent.

Yes indeed. However, the C function send_process internally calls
wait_reading_process_output, and this may again invoke the filter!  What
I find surprising is that this call of wait_reading_process_output is
not limited to the process whose output queue is full, i.e., the one for
which process-send-string was actually invoked. Would it work to limit
it to that process, or could there be an alternative means to prevent
reading from other processes in that case? It would be very useful in
such situations where I simply want to send, and not read anything.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#33747: 26.1; process-send-string exceeds max-specpdl-size
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 14 Dec 2018 19:56:02 +0000
Resent-Message-ID: <handler.33747.B33747.154481730232429 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 33747
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Markus Triska <triska@HIDDEN>
Cc: 33747 <at> debbugs.gnu.org
Received: via spool by 33747-submit <at> debbugs.gnu.org id=B33747.154481730232429
          (code B ref 33747); Fri, 14 Dec 2018 19:56:02 +0000
Received: (at 33747) by debbugs.gnu.org; 14 Dec 2018 19:55:02 +0000
Received: from localhost ([127.0.0.1]:48577 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gXtY6-0008Qz-4b
	for submit <at> debbugs.gnu.org; Fri, 14 Dec 2018 14:55:02 -0500
Received: from eggs.gnu.org ([208.118.235.92]:37873)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1gXtY4-0008QN-Qm
 for 33747 <at> debbugs.gnu.org; Fri, 14 Dec 2018 14:55:01 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1gXtXv-0004dC-NB
 for 33747 <at> debbugs.gnu.org; Fri, 14 Dec 2018 14:54:55 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20 autolearn=disabled
 version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:53904)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1gXtXv-0004d5-9U; Fri, 14 Dec 2018 14:54:51 -0500
Received: from [176.228.60.248] (port=4986 helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1gXtXv-0004u4-1J; Fri, 14 Dec 2018 14:54:51 -0500
Date: Fri, 14 Dec 2018 21:54:48 +0200
Message-Id: <83mup7g9yv.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-reply-to: <874lbgeydc.fsf@HIDDEN> (message from Markus Triska on Fri, 
 14 Dec 2018 19:50:39 +0100)
References: <87bm5o2e3u.fsf@HIDDEN> <83o99oeyuz.fsf@HIDDEN>
 <874lbgeydc.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -6.0 (------)

> From: Markus Triska <triska@HIDDEN>
> Cc: 33747 <at> debbugs.gnu.org
> Date: Fri, 14 Dec 2018 19:50:39 +0100
> 
> Eli Zaretskii <eliz@HIDDEN> writes:
> 
> > AFAIK, process-send-string is a blocking function: it cannot return
> > before the entire string was sent.
> 
> Yes indeed. However, the C function send_process internally calls
> wait_reading_process_output

That's so as not to wedge the entire session.  Communications with
async subprocesses aren't supposed to stop everything in its tracks.

> and this may again invoke the filter!

In general, calling APIs that could take a long time from a process
filter is not a very good idea, btw.  Remember that process filters
are run when Emacs is idle, so when you call an API that could take a
long time, you make Emacs less responsive.  And then there's the
infinite recursion case that you hit.

> What I find surprising is that this call of
> wait_reading_process_output is not limited to the process whose
> output queue is full, i.e., the one for which process-send-string
> was actually invoked.

I don't see any reason why it should only wait for output from the
same process.  Once again, this is Emacs's way to wait without
stopping everything else, so waiting for input from any source makes
perfect sense to me.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#33747: 26.1; process-send-string exceeds max-specpdl-size
Resent-From: Markus Triska <triska@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 14 Dec 2018 21:24:01 +0000
Resent-Message-ID: <handler.33747.B33747.15448225998840 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 33747
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 33747 <at> debbugs.gnu.org
Received: via spool by 33747-submit <at> debbugs.gnu.org id=B33747.15448225998840
          (code B ref 33747); Fri, 14 Dec 2018 21:24:01 +0000
Received: (at 33747) by debbugs.gnu.org; 14 Dec 2018 21:23:19 +0000
Received: from localhost ([127.0.0.1]:48627 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gXuvX-0002IW-Cj
	for submit <at> debbugs.gnu.org; Fri, 14 Dec 2018 16:23:19 -0500
Received: from metalevel.at ([78.46.218.83]:47014)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <triska@HIDDEN>) id 1gXuvV-0002IN-KL
 for 33747 <at> debbugs.gnu.org; Fri, 14 Dec 2018 16:23:18 -0500
Received: by metalevel.at (Postfix, from userid 1000)
 id 32F89A122B; Fri, 14 Dec 2018 22:23:16 +0100 (CET)
From: Markus Triska <triska@HIDDEN>
References: <87bm5o2e3u.fsf@HIDDEN> <83o99oeyuz.fsf@HIDDEN>
 <874lbgeydc.fsf@HIDDEN> <83mup7g9yv.fsf@HIDDEN>
Date: Fri, 14 Dec 2018 22:23:16 +0100
In-Reply-To: <83mup7g9yv.fsf@HIDDEN> (Eli Zaretskii's message of "Fri, 14 Dec
 2018 21:54:48 +0200")
Message-ID: <87mup7by63.fsf@HIDDEN>
User-Agent: Emacs/24.5
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Eli Zaretskii <eliz@HIDDEN> writes:

> stopping everything else, so waiting for input from any source makes
> perfect sense to me.

In this concrete case, I am looking for a way to make Emacs not wait
for, or at least not process, input from a specific process while
sending output to a different process.

For the case I posted, I can alleviate the issue a bit if I set:

    (setq max-specpdl-size 5000)

However, if I then use for example the following definition of
send-to-target instead of the one I posted:

   (defun send-to-target (proc str)
     (if (process-live-p target)
         (process-send-string target str)
       (kill-process source)))

then I get, for the receipe I posted:

    Lisp nesting exceeds =E2=80=98max-lisp-eval-depth=E2=80=99

This is unfortunate, because increasing this limit will eventually crash
Emacs. The root cause is that process-send-string triggers the filter of
the same process that caused the filter to be invoked. I find this
unexpected, because the target process is different, and if possible, I
would like a way do prevent this. Could you please consider adding a
feature that prevents triggering the filter in such cases?

Thank you and all the best!
Markus

Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.