GNU bug report logs - #39075
28.0.50; Emacs hangs on 100% CPU and grows beyond bounds in shell-mode

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 39075 in the body.
You can then email your comments to 39075 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter <at> vanoostrum.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; Emacs hangs on 100% CPU and grows beyond bounds in shell-mode
Date: Fri, 10 Jan 2020 22:16:58 +0100

1) Emacs -Q
2) M-x shell
3) type some command, and use some filename completions on the way
(using TAB).
4) Type a semicolon (;)

Now Emacs hangs, the ; does not appear, and it doesn't react to a C-g typed.
It uses 100% CPU and memory grows beyond bounds, eventually just making
my whole computer unresponsive.

Typing a whole series of C-g's might stop the loop (with the ; appearing
then) or it might crash Emacs:
Fatal error 11: Segmentation fault
Abort trap: 6

This report is typed in a session where the sequence of C-g's did stop
the loop.
The C-g caused this message:
Error in post-command-hook (completion-in-region--postch): (quit)
Although the option Enter debugger on Quit/C-g was set, no backtrace was
generated.

This emacs was compiled from master today, but it also happens in earlier versions.


In GNU Emacs 28.0.50 (build 1, i686-apple-darwin10.0.0, NS appkit-1561.61 Version 10.13.6 (Build 17G10021))
 of 2020-01-10 built on cochabamba.vanoostrum.org
Repository revision: 17cfd708575c351d030f8b05c5921d1867028d79
Repository branch: fix
Windowing system distributor 'Apple', version 10.3.1561
System Description:  Mac OS X 10.13.6

Recent messages:
Quit
No match [2 times]
~ 
Error in post-command-hook (completion-in-region--postch): (quit)
Quit [2 times]
Complete, but not unique
Making completion list...
Complete, but not unique
Error in post-command-hook (completion-in-region--postch): (quit)
Quit
Quit
Configured using:
 'configure --build i686-apple-darwin10.0.0 --without-dbus --with-ns
 build_alias=i686-apple-darwin10.0.0 'CFLAGS=-pipe -march=nocona'
 PKG_CONFIG_PATH=/opt/local/lib/pkgconfig/:/usr/X11R6/pkgconfig/:/usr/local/lib/pkgconfig/:/usr/lib/pkgconfig/'

Configured features:
RSVG GLIB NOTIFY KQUEUE ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS XIM
NS MODULES THREADS PDUMPER LCMS2

Important settings:
  value of $LC_CTYPE: UTF-8
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Shell

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
shell pcomplete comint ansi-color ring tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads kqueue cocoa ns
lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 55230 57689)
 (symbols 48 7130 11)
 (strings 32 18286 5181)
 (string-bytes 1 600859)
 (vectors 16 11939)
 (vector-slots 8 157593 69152)
 (floats 8 19 126)
 (intervals 56 1161 97)
 (buffers 1000 14))

-- 
Pieter van Oostrum <pieter <at> vanoostrum.org>
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]


-- 
Pieter van Oostrum <pieter <at> vanoostrum.org>
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #8 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Pieter van Oostrum <pieter <at> vanoostrum.org>
Cc: 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50;
 Emacs hangs on 100% CPU and grows beyond bounds in shell-mode
Date: Sat, 11 Jan 2020 11:59:37 +0200

> Date: Fri, 10 Jan 2020 22:16:58 +0100
> From: Pieter van Oostrum <pieter <at> vanoostrum.org>
> 
> 1) Emacs -Q
> 2) M-x shell
> 3) type some command, and use some filename completions on the way
> (using TAB).
> 4) Type a semicolon (;)
> 
> Now Emacs hangs, the ; does not appear, and it doesn't react to a C-g typed.
> It uses 100% CPU and memory grows beyond bounds, eventually just making
> my whole computer unresponsive.

I cannot reproduce this on GNU/Linux, so it's probably macOS-specific.

Message #11 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Sun, 12 Jan 2020 18:11:22 +0100

[Message part 1 (text/plain, inline)]

Eli Zaretskii <eliz <at> gnu.org> writes:

>> Date: Fri, 10 Jan 2020 22:16:58 +0100
>> From: Pieter van Oostrum <pieter <at> vanoostrum.org>
>> 
>> 1) Emacs -Q
>> 2) M-x shell
>> 3) type some command, and use some filename completions on the way
>> (using TAB).
>> 4) Type a semicolon (;)
>> 
>> Now Emacs hangs, the ; does not appear, and it doesn't react to a C-g typed.
>> It uses 100% CPU and memory grows beyond bounds, eventually just making
>> my whole computer unresponsive.
>
> I cannot reproduce this on GNU/Linux, so it's probably macOS-specific.

It's not so clear what could be MacOS-specific.

I ran it under gdb, and interrupted it several times with C-z in gdb. Most of the stack traces were in the garbage collector, suggesting that it is collecting like crazy. This doesn't surprise me, as it is constantly allocating new memory. The rest of this stack trace doesn't have useful information.

I managed to get a stack trace where it is processing. I haven't analysed these yet, but I include both here.

[semicolon-emacs-loop (application/octet-stream, attachment)]

[Message part 3 (text/plain, inline)]

-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #14 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
Cc: 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Sun, 12 Jan 2020 19:43:18 +0200

> From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
> Cc: 39075 <at> debbugs.gnu.org
> Date: Sun, 12 Jan 2020 18:11:22 +0100
> 
> I ran it under gdb, and interrupted it several times with C-z in gdb. Most of the stack traces were in the garbage collector, suggesting that it is collecting like crazy. This doesn't surprise me, as it is constantly allocating new memory. The rest of this stack trace doesn't have useful information.
> 
> I managed to get a stack trace where it is processing. I haven't analysed these yet, but I include both here.

Thanks, this was very useful.  It turns out to reproduce one must do
this at the shell's prompt, after "M-x shell":

  $ cd /some/directory/;

The /some/directory/ part should be a real directory.  Once one types
the semi-colon, Emacs hangs.  Here's the Lisp backtrace:

  "Automatic GC" (0x0)
  "looking-at" (0x766f5fc8)
  "shell--parse-pcomplete-arguments" (0x766f64f8)
  "pcomplete-parse-arguments" (0x766f6a90)
  "pcomplete-completions" (0x766f6f60)
  "pcomplete-completions-at-point" (0x766f7698)
  "run-hook-with-args-until-success" (0x766f7690)
  "comint-completion-at-point" (0x766f7b10)
  0x317f7b0 PVEC_COMPILED
  "completion-in-region--postch" (0x766f8450)

So I think shell--parse-pcomplete-arguments infloops in this case.

Message #17 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Mon, 13 Jan 2020 14:58:05 +0100

Eli Zaretskii <eliz <at> gnu.org> writes:

> Thanks, this was very useful.  It turns out to reproduce one must do
> this at the shell's prompt, after "M-x shell":
>
>   $ cd /some/directory/;
>
> The /some/directory/ part should be a real directory.  Once one types
> the semi-colon, Emacs hangs.  Here's the Lisp backtrace:
>
>   "Automatic GC" (0x0)
>   "looking-at" (0x766f5fc8)
>   "shell--parse-pcomplete-arguments" (0x766f64f8)
>   "pcomplete-parse-arguments" (0x766f6a90)
>   "pcomplete-completions" (0x766f6f60)
>   "pcomplete-completions-at-point" (0x766f7698)
>   "run-hook-with-args-until-success" (0x766f7690)
>   "comint-completion-at-point" (0x766f7b10)
>   0x317f7b0 PVEC_COMPILED
>   "completion-in-region--postch" (0x766f8450)
>
> So I think shell--parse-pcomplete-arguments infloops in this case.

Yes, I have traced it.

shell--parse-pcomplete-arguments splits the line into chunk, each chunk being either
1) a sequence of chars not containing white space, \ " ' ;
2) a sequence of chars between apostrophes (') not containing ', maybe final one missing
3) a sequence of chars between quotes (") not containing unescaped ", maybe final one missing
4) a backslash (\) possibly followed by a char

It uses a regexp for that.
It collects a list of these chunks.
It skips over white space between the chunks.

The problem is that a semicolon ; is not covered by this regexp. In the case that caused the error the end position was after the semicolon but the match loop stopped just before the semicolon, and would never advance. It kept going in an infinite loop, continually pushing empty strings on the result list, thereby exhausting memory, and using 100% CPU time.
I don't know why it wouldn't react to C-g, but I guess because after some time it would be mostly in the garbage collector.

Originally case 1) did not have the semicolon.
It was introduced in commit eaeeece92da51b517097667f13d580aa92ad5d59 on Dec 4, 2018 18:39:47 +0100.
There was a test case added in test/lisp/shell-tests.el, but it cheated by positioning point before the semicolon.

I think the simplest solution is to add a semicolon to the part where it skips over white space, i.e. treat the semicolon like white space. But I am not wholly sure that the caller doesn't want to see the semicolon. Otherwise the semicolon should be pushed to the result.

diff -u /Users/pieter/TEMP/shell.el.\~1\~ /Users/pieter/TEMP/shell.el
--- /Users/pieter/TEMP/shell.el.~1~	2020-01-13 14:37:40.000000000 +0100
+++ /Users/pieter/TEMP/shell.el	2020-01-13 14:38:07.000000000 +0100
@@ -428,7 +428,7 @@
     (save-excursion
       (goto-char begin)
       (while (< (point) end)
-	(skip-chars-forward " \t\n")
+	(skip-chars-forward " \t\n;")
 	(push (point) begins)
         (let ((arg ()))
           (while (looking-at

Diff finished.  Mon Jan 13 14:38:22 2020

-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #20 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Michael Welsh Duggan <mwd <at> md5i.com>
To: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Tue, 14 Jan 2020 22:17:28 -0500

This seems very similar to the bug I reported at bug#38549.  Could you
see if this also fixes that recipe and, if so, merge the bugs?

-- 
Michael Welsh Duggan
(md5i <at> md5i.com)

Message #25 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
To: Michael Welsh Duggan <mwd <at> md5i.com>
Cc: 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Wed, 15 Jan 2020 20:15:03 +0100

Michael Welsh Duggan <mwd <at> md5i.com> writes:

> This seems very similar to the bug I reported at bug#38549.  Could you
> see if this also fixes that recipe and, if so, merge the bugs?
>
Yes, the same fix solves that bug also. It is the same bug.
-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #28 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
To: Michael Welsh Duggan <mwd <at> md5i.com>
Cc: 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Thu, 16 Jan 2020 20:21:37 +0100

[Message part 1 (text/plain, inline)]

Pieter van Oostrum <pieter-l <at> vanoostrum.org> writes:

> Michael Welsh Duggan <mwd <at> md5i.com> writes:
>
>> This seems very similar to the bug I reported at bug#38549.  Could you
>> see if this also fixes that recipe and, if so, merge the bugs?
>>
> Yes, the same fix solves that bug also. It is the same bug.

I propose to amend the test also, so that it would have caught this error (by hanging). I add both patches here now. I have tested that all other things worked the same and found no problems.

Is there an official procedure for requesting the change, or is posting it here sufficient?

[shell.patch (text/x-patch, attachment)]

[Message part 3 (text/plain, inline)]

-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #31 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Fri, 17 Jan 2020 11:57:51 +0100

[Message part 1 (text/plain, inline)]

Pieter van Oostrum <pieter-l <at> vanoostrum.org> writes:

> Eli Zaretskii <eliz <at> gnu.org> writes:
>
>> Thanks, this was very useful.  It turns out to reproduce one must do
>> this at the shell's prompt, after "M-x shell":
>>
>>   $ cd /some/directory/;
>>
>> The /some/directory/ part should be a real directory.  Once one types
>> the semi-colon, Emacs hangs.  Here's the Lisp backtrace:
>>
>>   "Automatic GC" (0x0)
>>   "looking-at" (0x766f5fc8)
>>   "shell--parse-pcomplete-arguments" (0x766f64f8)
>>   "pcomplete-parse-arguments" (0x766f6a90)
>>   "pcomplete-completions" (0x766f6f60)
>>   "pcomplete-completions-at-point" (0x766f7698)
>>   "run-hook-with-args-until-success" (0x766f7690)
>>   "comint-completion-at-point" (0x766f7b10)
>>   0x317f7b0 PVEC_COMPILED
>>   "completion-in-region--postch" (0x766f8450)
>>
>> So I think shell--parse-pcomplete-arguments infloops in this case.
>
> Yes, I have traced it.
>
> shell--parse-pcomplete-arguments splits the line into chunk, each chunk being either
> 1) a sequence of chars not containing white space, \ " ' ;
> 2) a sequence of chars between apostrophes (') not containing ', maybe final one missing
> 3) a sequence of chars between quotes (") not containing unescaped ", maybe final one missing
> 4) a backslash (\) possibly followed by a char
>
> It uses a regexp for that.
> It collects a list of these chunks.
> It skips over white space between the chunks.
>
> The problem is that a semicolon ; is not covered by this regexp. In the case that caused the error the end position was after the semicolon but the match loop stopped just before the semicolon, and would never advance. It kept going in an infinite loop, continually pushing empty strings on the result list, thereby exhausting memory, and using 100% CPU time.
> I don't know why it wouldn't react to C-g, but I guess because after some time it would be mostly in the garbage collector.
>
> Originally case 1) did not have the semicolon.
> It was introduced in commit eaeeece92da51b517097667f13d580aa92ad5d59 on Dec 4, 2018 18:39:47 +0100.
> There was a test case added in test/lisp/shell-tests.el, but it cheated by positioning point before the semicolon.
>
> I think the simplest solution is to add a semicolon to the part where it skips over white space, i.e. treat the semicolon like white space. But I am not wholly sure that the caller doesn't want to see the semicolon. Otherwise the semicolon should be pushed to the result.
>
> diff -u /Users/pieter/TEMP/shell.el.\~1\~ /Users/pieter/TEMP/shell.el
> --- /Users/pieter/TEMP/shell.el.~1~	2020-01-13 14:37:40.000000000 +0100
> +++ /Users/pieter/TEMP/shell.el	2020-01-13 14:38:07.000000000 +0100
> @@ -428,7 +428,7 @@
>      (save-excursion
>        (goto-char begin)
>        (while (< (point) end)
> -	(skip-chars-forward " \t\n")
> +	(skip-chars-forward " \t\n;")
>  	(push (point) begins)
>          (let ((arg ()))
>            (while (looking-at
>
> Diff finished.  Mon Jan 13 14:38:22 2020


I propose to amend the test also, so that it would have caught this
error (by hanging). I add both patches here now. I have tested that all
other things worked the same and found no problems.

Is there an official procedure for requesting the change, or is posting it here sufficient?

[shell.patch2 (text/plain, attachment)]

[Message part 3 (text/plain, inline)]

-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #36 received at 39075-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
Cc: mwd <at> md5i.com, 39075-done <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50;
 Emacs hangs on 100% CPU and grows beyond bounds in shell-mode
Date: Sat, 18 Jan 2020 11:57:44 +0200

> From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
> Date: Thu, 16 Jan 2020 20:21:37 +0100
> Cc: 39075 <at> debbugs.gnu.org
> 
> Pieter van Oostrum <pieter-l <at> vanoostrum.org> writes:
> 
> > Michael Welsh Duggan <mwd <at> md5i.com> writes:
> >
> >> This seems very similar to the bug I reported at bug#38549.  Could you
> >> see if this also fixes that recipe and, if so, merge the bugs?
> >>
> > Yes, the same fix solves that bug also. It is the same bug.
> 
> I propose to amend the test also, so that it would have caught this error (by hanging). I add both patches here now. I have tested that all other things worked the same and found no problems.

Thanks, pushed to the release branch.

> Is there an official procedure for requesting the change, or is posting it here sufficient?

It is sufficient to post here, but:

 . please in the future provide a ChangeLog-style commit log message
   (see CONTRIBUTE for more about this);
 . it looks like the disclaimer of your employer has expired several
   years ago, so if you want to continue contributing to Emacs, I
   suggest to start/renew your legal paperwork.

Message #44 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: 39075 <at> debbugs.gnu.org
Cc: eliz <at> gnu.org, pieter <at> vanoostrum.org
Subject: Re: bug#39075: 28.0.50;
 Emacs hangs on 100% CPU and grows beyond bounds in shell-mode
Date: Sun, 19 Jan 2020 13:08:43 -0500

This causes a test failure for me on CentOS 8.1.
(BTW, the bug# in the commit log has a typo, but obviosuly nothing can
be done about that.)


Test shell-tests-completion-before-semi backtrace:
  signal(ert-test-failed (((should (equal (shell--parse-pcomplete-argu
  ert-fail(((should (equal (shell--parse-pcomplete-arguments) '(("cd" 
  #f(compiled-function () #<bytecode 0x45cfa9>)()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name shell-tests-completion-before-semi :d
  ert-run-or-rerun-test(#s(ert--stats :selector (not (or (tag :expensi
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/shell-tests" "--eval
  command-line()
  normal-top-level()
Test shell-tests-completion-before-semi condition:
    (ert-test-failed
     ((should
       (equal
	(shell--parse-pcomplete-arguments)
	'...))
      :form
      (equal
       (("cd" "ba" "")
	1 4 7)
       (("cd" "ba" "")
	1 4))
      :value nil :explanation
      (proper-lists-of-different-length 4 3
					(("cd" "ba" "")
					 1 4 7)
					(("cd" "ba" "")
					 1 4)
					first-mismatch-at 3)))
   FAILED  1/2  shell-tests-completion-before-semi (0.000774 sec)

Message #47 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50;
 Emacs hangs on 100% CPU and grows beyond bounds in shell-mode
Date: Sun, 19 Jan 2020 13:11:53 -0500

Pieter van Oostrum wrote:

> I propose to amend the test also, so that it would have caught this
> error (by hanging).

I haven't read anything else, so take this with a grain of salt, but
tests that hang (rather than just fail) are a PITA for automated testing.

Message #50 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Sun, 19 Jan 2020 22:41:22 +0100

Glenn Morris <rgm <at> gnu.org> writes:

> Pieter van Oostrum wrote:
>
>> I propose to amend the test also, so that it would have caught this
>> error (by hanging).
>
> I haven't read anything else, so take this with a grain of salt, but
> tests that hang (rather than just fail) are a PITA for automated testing.

That is true, but the test is there to test the fix, which makes it no longer hang. And as these are in the same commit, the test should never hang, unless somebody breaks the fix later. But then hanging could occur with any bug.
-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #53 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Mon, 20 Jan 2020 01:29:28 +0100

Glenn Morris <rgm <at> gnu.org> writes:

> This causes a test failure for me on CentOS 8.1.
> (BTW, the bug# in the commit log has a typo, but obviosuly nothing can
> be done about that.)
>
>
> Test shell-tests-completion-before-semi backtrace:
>   signal(ert-test-failed (((should (equal (shell--parse-pcomplete-argu
>   ert-fail(((should (equal (shell--parse-pcomplete-arguments) '(("cd" 
>   #f(compiled-function () #<bytecode 0x45cfa9>)()
>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>   ert-run-test(#s(ert-test :name shell-tests-completion-before-semi :d
>   ert-run-or-rerun-test(#s(ert--stats :selector (not (or (tag :expensi
>   ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
>   ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
>   ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
>   eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
>   command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/shell-tests" "--eval
>   command-line()
>   normal-top-level()
> Test shell-tests-completion-before-semi condition:
>     (ert-test-failed
>      ((should
>        (equal
> 	(shell--parse-pcomplete-arguments)
> 	'...))
>       :form
>       (equal
>        (("cd" "ba" "")
> 	1 4 7)
>        (("cd" "ba" "")
> 	1 4))
>       :value nil :explanation
>       (proper-lists-of-different-length 4 3
> 					(("cd" "ba" "")
> 					 1 4 7)
> 					(("cd" "ba" "")
> 					 1 4)
> 					first-mismatch-at 3)))
>    FAILED  1/2  shell-tests-completion-before-semi (0.000774 sec)

Aahh! My bad. It should have been 1 4 7. I made a copying error.
-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #56 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
Cc: Glenn Morris <rgm <at> gnu.org>, 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Mon, 20 Jan 2020 08:47:11 +0100

Pieter van Oostrum <pieter-l <at> vanoostrum.org> writes:

>>> I propose to amend the test also, so that it would have caught this
>>> error (by hanging).
>>
>> I haven't read anything else, so take this with a grain of salt, but
>> tests that hang (rather than just fail) are a PITA for automated testing.
>
> That is true, but the test is there to test the fix, which makes it no
> longer hang. And as these are in the same commit, the test should
> never hang, unless somebody breaks the fix later. But then hanging
> could occur with any bug.

I haven't checked your test case, but in Tramp I try to avoid hanging by
wrapping process related tests with a timer.

Experience has taught me, that I'm able to break the tests all the time
by new code :-(

Best regards, Michael.

Message #59 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Mon, 20 Jan 2020 14:12:20 +0100

Michael Albinus <michael.albinus <at> gmx.de> writes:

> Pieter van Oostrum <pieter-l <at> vanoostrum.org> writes:
>
>>>> I propose to amend the test also, so that it would have caught this
>>>> error (by hanging).
>>>
>>> I haven't read anything else, so take this with a grain of salt, but
>>> tests that hang (rather than just fail) are a PITA for automated testing.
>>
>> That is true, but the test is there to test the fix, which makes it no
>> longer hang. And as these are in the same commit, the test should
>> never hang, unless somebody breaks the fix later. But then hanging
>> could occur with any bug.
>
> I haven't checked your test case, but in Tramp I try to avoid hanging by
> wrapping process related tests with a timer.
>
> Experience has taught me, that I'm able to break the tests all the time
> by new code :-(
>
If you know a better test to check for this particular fix, I would be happy if you could let me know. I don't know if some timer code would help.
-- 
Pieter van Oostrum
www: http://pieter.vanoostrum.org/
PGP key: [8DAE142BE17999C4]

Message #62 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Pieter van Oostrum <pieter-l <at> vanoostrum.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 39075 <at> debbugs.gnu.org
Subject: Re: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond
 bounds in shell-mode
Date: Mon, 20 Jan 2020 14:33:35 +0100

Pieter van Oostrum <pieter-l <at> vanoostrum.org> writes:

>>> That is true, but the test is there to test the fix, which makes it no
>>> longer hang. And as these are in the same commit, the test should
>>> never hang, unless somebody breaks the fix later. But then hanging
>>> could occur with any bug.
>>
>> I haven't checked your test case, but in Tramp I try to avoid hanging by
>> wrapping process related tests with a timer.
>>
>> Experience has taught me, that I'm able to break the tests all the time
>> by new code :-(
>>
> If you know a better test to check for this particular fix, I would be
> happy if you could let me know. I don't know if some timer code would
> help.

Which test do you mean? There is shell-tests-completion-before-semi, but
this doesn't use a process.

Best regards, Michael.

Message #65 received at 39075 <at> debbugs.gnu.org (full text, mbox):

From: Mattias Engdegård <mattiase <at> acm.org>
To: Pieter van Oostrum <pieter-l <at> vanoostrum.org>,
 Michael Albinus <michael.albinus <at> gmx.de>, Eli Zaretskii <eliz <at> gnu.org>
Cc: 39075 <at> debbugs.gnu.org
Subject: bug#39075: 28.0.50; Emacs hangs on 100% CPU and grows beyond  bounds
 in shell-mode 
Date: Mon, 20 Jan 2020 15:57:33 +0100

At the very least, a test named 'shell-tests-completion-before-semi' should not be changed into testing completion after a semicolon. I took the liberty to fixing that, and the erroneous test value that made the changed test fail.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.