Package: emacs;
Reported by: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com>
Date: Fri, 14 Feb 2020 10:50:01 UTC
Severity: normal
Tags: fixed
Found in version 26.3
Fixed in version 28.1
Done: Lars Ingebrigtsen <larsi <at> gnus.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 39598 in the body.
You can then email your comments to 39598 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Fri, 14 Feb 2020 10:50:02 GMT) Full text and rfc822 format available.Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com>:bug-gnu-emacs <at> gnu.org.
(Fri, 14 Feb 2020 10:50:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> To: bug-gnu-emacs <at> gnu.org Subject: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Fri, 14 Feb 2020 11:48:43 +0100
[Message part 1 (text/plain, inline)]
Emacs becomes extremely sluggish for me even on the most trivial python
files sometime. The most recent example I of this behavior that I can
reliably reproduce is the following:
1. Open an empty python buffer.
2. Insert a string "this is a test string" (including quotes) on the
first line of buffer.
3. Copy it and insert it 10 times.
4. Copy all the lines in the buffer and insert them 10 times more. This
should give you 100 lines of python strings.
5. By the fifth paste command you can see 1 or 2 second delays before the
buffer is updated.
6. After that navigating to the beginning or the end of buffer will
trigger a couple of seconds of hangup. Page scrolls can randomly
hangup. Iserting a blank line somewhere in the middle of the buffer can
cause a random 1 or 2 second delay as well.
Those are some excerpts from the profiler report:
- command-execute 48141 87%
- call-interactively 48137 87%
- funcall-interactively 48064 87%
- end-of-buffer 46719 84%
- recenter 46713 84%
- jit-lock-function 46713 84%
- jit-lock-fontify-now 46713 84%
- jit-lock--run-functions 46713 84%
- run-hook-wrapped 46713 84%
+ #<compiled 0xf8aa91> 46713 84%
...
- redisplay_internal (C function) 4935 8%
- jit-lock-function 4899 8%
- jit-lock-fontify-now 4897 8%
- jit-lock--run-functions 4894 8%
- run-hook-wrapped 4894 8%
+ #<compiled 0xf8aa91> 4894 8%
In GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
of 2019-12-03 built on lgw01-amd64-029
Repository revision: ea9cfdb4812c1140416c523bac34a6ec72db7fce
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.3 LTS
Recent messages:
Mark set
Quit
Mark set
next-line: End of buffer
Mark set [3 times]
Quit [2 times]
Mark set [4 times]
Quit
Mark set [2 times]
Quit
Configured using:
'configure --prefix= --prefix=/snap/emacs/current/usr 'CFLAGS=
-I/build/emacs/parts/emacs/install/usr/include' 'CPPFLAGS=
-I/build/emacs/parts/emacs/install/usr/include' 'LDFLAGS=
-L/build/emacs/parts/emacs/install/lib
-L/build/emacs/parts/emacs/install/usr/lib
-L/build/emacs/parts/emacs/install/lib/x86_64-linux-gnu
-L/build/emacs/parts/emacs/install/usr/lib/x86_64-linux-gnu''
Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS LIBSYSTEMD LCMS2
Important settings:
value of $LC_MONETARY: en_GB.UTF-8
value of $LC_NUMERIC: en_GB.UTF-8
value of $LC_TIME: en_GB.UTF-8
value of $LANG: en_GB.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix
Major mode: Python
Minor modes in effect:
shell-dirtrack-mode: t
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
line-number-mode: t
transient-mark-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils profiler python easymenu tramp-sh tramp
tramp-compat tramp-loaddefs trampver ucs-normalize shell pcomplete
parse-time format-spec advice auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache json map seq byte-opt gv bytecomp
byte-compile cconv comint ring cl-loaddefs cl-lib ansi-color elec-pair
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)
Memory information:
((conses 16 233944 16451)
(symbols 48 22819 2)
(miscs 40 68 448)
(strings 32 37667 2664)
(string-bytes 1 1114745)
(vectors 16 60243)
(vector-slots 8 1252726 14364)
(floats 8 70 421)
(intervals 56 875 0)
(buffers 992 14))
[Message part 2 (text/html, inline)]
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Fri, 14 Feb 2020 13:37:02 GMT) Full text and rfc822 format available.Message #8 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Fri, 14 Feb 2020 15:35:58 +0200
> From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> > Date: Fri, 14 Feb 2020 11:48:43 +0100 > > 1. Open an empty python buffer. > 2. Insert a string "this is a test string" (including quotes) on the > first line of buffer. > 3. Copy it and insert it 10 times. > 4. Copy all the lines in the buffer and insert them 10 times more. This > should give you 100 lines of python strings. > 5. By the fifth paste command you can see 1 or 2 second delays before the > buffer is updated. > 6. After that navigating to the beginning or the end of buffer will > trigger a couple of seconds of hangup. Page scrolls can randomly > hangup. Iserting a blank line somewhere in the middle of the buffer can > cause a random 1 or 2 second delay as well. > > Those are some excerpts from the profiler report: > > - command-execute 48141 87% > - call-interactively 48137 87% > - funcall-interactively 48064 87% > - end-of-buffer 46719 84% > - recenter 46713 84% > - jit-lock-function 46713 84% > - jit-lock-fontify-now 46713 84% > - jit-lock--run-functions 46713 84% > - run-hook-wrapped 46713 84% > + #<compiled 0xf8aa91> 46713 84% > > ... > > - redisplay_internal (C function) 4935 8% > - jit-lock-function 4899 8% > - jit-lock-fontify-now 4897 8% > - jit-lock--run-functions 4894 8% > - run-hook-wrapped 4894 8% > + #<compiled 0xf8aa91> 4894 8% Please show the entire expanded profile. Thanks.
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Fri, 14 Feb 2020 13:51:01 GMT) Full text and rfc822 format available.Message #11 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: oreshnikov.ivan <at> gmail.com Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Fri, 14 Feb 2020 15:50:33 +0200
> Date: Fri, 14 Feb 2020 15:35:58 +0200
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: 39598 <at> debbugs.gnu.org
>
> Please show the entire expanded profile.
My profile is below. Is yours similar? If so, I thing
python-nav-beginning-of-statement, python-nav-end-of-statement, and
python-nav--forward-sexp are the main culprits.
- redisplay_internal (C function) 1600 46%
- jit-lock-function 1600 46%
- jit-lock-fontify-now 1600 46%
- jit-lock--run-functions 1600 46%
- run-hook-wrapped 1600 46%
- #<compiled -0x1ffffffff8c1b778> 1600 46%
- font-lock-fontify-region 1600 46%
- font-lock-default-fontify-region 1600 46%
- font-lock-fontify-syntactically-region 1600 46%
- python-font-lock-syntactic-face-function 1600 46%
- python-info-docstring-p 1599 46%
- python-nav-backward-sexp 1583 46%
- python-nav-forward-sexp 1583 46%
- python-nav--forward-sexp 1580 46%
- python-info-end-of-block-p 893 26%
- python-info-end-of-statement-p 893 26%
- python-nav-end-of-statement 892 26%
syntax-ppss 591 17%
- python-info-line-ends-backslash-p 292 8%
syntax-ppss 289 8%
- python-info-beginning-of-statement-p 533 15%
- python-nav-beginning-of-statement 531 15%
syntax-ppss 275 8%
- python-info-line-ends-backslash-p 251 7%
- syntax-ppss 244 7%
#<compiled -0x1ffffffff8326490> 1 0%
back-to-indentation 4 0%
- python-nav-beginning-of-statement 79 2%
syntax-ppss 70 2%
back-to-indentation 3 0%
python-info-line-ends-backslash-p 2 0%
- python-syntax-context-type 52 1%
syntax-ppss 52 1%
- python-nav--lisp-forward-sexp 7 0%
forward-sexp 7 0%
- python-info-beginning-of-block-p 7 0%
python-info-statement-starts-block-p 3 0%
python-info-beginning-of-statement-p 2 0%
- python-info-current-line-empty-p 5 0%
match-string-no-properties 1 0%
syntax-class 1 0%
- python-nav-beginning-of-statement 8 0%
syntax-ppss 7 0%
- python-info-line-ends-backslash-p 1 0%
syntax-ppss 1 0%
- python-info-assignment-statement-p 4 0%
- python-nav-beginning-of-statement 4 0%
- python-info-line-ends-backslash-p 3 0%
syntax-ppss 3 0%
syntax-ppss 1 0%
- command-execute 1414 41%
- call-interactively 1414 41%
- funcall-interactively 1413 41%
- scroll-up-command 1412 41%
- scroll-up 1412 41%
- jit-lock-function 1412 41%
- jit-lock-fontify-now 1412 41%
- jit-lock--run-functions 1412 41%
- run-hook-wrapped 1412 41%
- #<compiled -0x1ffffffff8c1b778> 1412 41%
- font-lock-fontify-region 1412 41%
- font-lock-default-fontify-region 1412 41%
- font-lock-fontify-syntactically-region 1412 41%
- python-font-lock-syntactic-face-function 1412 41%
- python-info-docstring-p 1411 41%
- python-nav-backward-sexp 1402 40%
- python-nav-forward-sexp 1402 40%
- python-nav--forward-sexp 1400 40%
- python-info-beginning-of-block-p 1137 33%
- python-info-beginning-of-statement-p 585 17%
- python-nav-beginning-of-statement 585 17%
- python-info-line-ends-backslash-p 327 9%
syntax-ppss 322 9%
syntax-ppss 248 7%
back-to-indentation 7 0%
- python-info-statement-starts-block-p 550 16%
- python-nav-beginning-of-statement 546 15%
syntax-ppss 270 7%
- python-info-line-ends-backslash-p 262 7%
syntax-ppss 256 7%
back-to-indentation 6 0%
- python-nav-beginning-of-statement 190 5%
- python-info-line-ends-backslash-p 113 3%
syntax-ppss 110 3%
syntax-ppss 74 2%
back-to-indentation 1 0%
- python-syntax-context-type 61 1%
syntax-ppss 61 1%
- python-info-beginning-of-statement-p 4 0%
python-nav-beginning-of-statement 3 0%
- python-nav--lisp-forward-sexp 3 0%
forward-sexp 2 0%
python-info-current-line-empty-p 2 0%
- python-info-end-of-block-p 1 0%
python-info-end-of-statement-p 1 0%
- python-nav-beginning-of-statement 4 0%
- python-info-line-ends-backslash-p 3 0%
syntax-ppss 3 0%
syntax-ppss 1 0%
- python-info-assignment-statement-p 4 0%
- python-nav-beginning-of-statement 4 0%
syntax-ppss 2 0%
- python-info-line-ends-backslash-p 2 0%
syntax-ppss 2 0%
- execute-extended-command 1 0%
- sit-for 1 0%
- redisplay 1 0%
- redisplay_internal (C function) 1 0%
- tool-bar-make-keymap 1 0%
- tool-bar-make-keymap-1 1 0%
- mapcar 1 0%
- #<compiled -0x1ffffffffa0b8700> 1 0%
- eval 1 0%
- find-image 1 0%
image-type-available-p 1 0%
- byte-code 1 0%
- read-extended-command 1 0%
- completing-read 1 0%
completing-read-default 1 0%
- ... 406 11%
Automatic GC 406 11%
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Fri, 14 Feb 2020 16:09:02 GMT) Full text and rfc822 format available.Message #14 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> To: Eli Zaretskii <eliz <at> gnu.org> Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Fri, 14 Feb 2020 17:08:19 +0100
[Message part 1 (text/plain, inline)]
Sorry, didn't notice your first message (didn't expect such a quick
response :).
Yes, my profile is very similar to yours:
- redisplay_internal (C function) 18208 92%
- jit-lock-function 18198 92%
- jit-lock-fontify-now 18197 92%
- jit-lock--run-functions 18196 92%
- run-hook-wrapped 18196 92%
- #<compiled 0xf5b8a9> 18196 92%
- font-lock-fontify-region 18195 92%
- font-lock-default-fontify-region 18193 92%
- font-lock-fontify-syntactically-region 18163 92%
- python-font-lock-syntactic-face-function 18066 91%
- python-info-docstring-p 18054 91%
- python-nav-backward-sexp 17878 90%
- python-nav-forward-sexp 17872 90%
- python-nav--forward-sexp 17841 90%
- python-info-beginning-of-block-p 7363 37%
- python-info-beginning-of-statement-p 4163
21%
- python-nav-beginning-of-statement 4132 20%
- python-info-line-ends-backslash-p 2566 13%
- syntax-ppss 2510 12%
#<compiled 0xc88ab5> 1 0%
- syntax-ppss 1446 7%
#<compiled 0xec6b99> 3 0%
back-to-indentation 61 0%
- python-info-statement-starts-block-p 3180
16%
- python-nav-beginning-of-statement 3133 15%
- syntax-ppss 1500 7%
#<compiled 0xf43645> 5 0%
- python-info-line-ends-backslash-p 1492 7%
- syntax-ppss 1450 7%
#<compiled 0xc934cd> 1 0%
back-to-indentation 68 0%
- python-info-end-of-block-p 4882 24%
- python-info-end-of-statement-p 4869 24%
- python-nav-end-of-statement 4852 24%
- syntax-ppss 3048 15%
#<compiled 0xf62cf9> 4 0%
- python-info-line-ends-backslash-p 1687 8%
- syntax-ppss 1649 8%
#<compiled 0xf58689> 3 0%
- python-info-beginning-of-statement-p 3078 15%
- python-nav-beginning-of-statement 3051 15%
- python-info-line-ends-backslash-p 1520 7%
- syntax-ppss 1454 7%
#<compiled 0xc8c561> 1 0%
syntax-ppss 1408 7%
back-to-indentation 52 0%
- python-nav-beginning-of-statement 1473 7%
syntax-ppss 707 3%
- python-info-line-ends-backslash-p 705 3%
- syntax-ppss 681 3%
#<compiled 0xc84f2d> 1 0%
back-to-indentation 28 0%
- python-syntax-context-type 779 3%
- syntax-ppss 765 3%
#<compiled 0xedb06d> 3 0%
- python-nav--lisp-forward-sexp 149 0%
forward-sexp 139 0%
- python-info-current-line-empty-p 52 0%
match-string-no-properties 1 0%
syntax-after 5 0%
syntax-class 1 0%
- python-nav-beginning-of-statement 73 0%
syntax-ppss 42 0%
- python-info-line-ends-backslash-p 29 0%
syntax-ppss 29 0%
- python-info-assignment-statement-p 61 0%
- python-nav-beginning-of-statement 58 0%
syntax-ppss 31 0%
- python-info-line-ends-backslash-p 25 0%
syntax-ppss 25 0%
back-to-indentation 1 0%
python-util-forward-comment 1 0%
- syntax-propertize 86 0%
- #<compiled 0xe7bfcd> 85 0%
- python-syntax-stringify 76 0%
syntax-ppss 75 0%
match-string-no-properties 1 0%
syntax-ppss 8 0%
+ replace-regexp-in-string 2 0%
+ font-lock-fontify-keywords-region 26 0%
...
On Fri, 14 Feb 2020 at 14:50, Eli Zaretskii <eliz <at> gnu.org> wrote:
> > Date: Fri, 14 Feb 2020 15:35:58 +0200
> > From: Eli Zaretskii <eliz <at> gnu.org>
> > Cc: 39598 <at> debbugs.gnu.org
> >
> > Please show the entire expanded profile.
>
> My profile is below. Is yours similar? If so, I thing
> python-nav-beginning-of-statement, python-nav-end-of-statement, and
> python-nav--forward-sexp are the main culprits.
>
> - redisplay_internal (C function) 1600
> 46%
> - jit-lock-function 1600
> 46%
> - jit-lock-fontify-now 1600
> 46%
> - jit-lock--run-functions 1600
> 46%
> - run-hook-wrapped 1600
> 46%
> - #<compiled -0x1ffffffff8c1b778> 1600
> 46%
> - font-lock-fontify-region 1600
> 46%
> - font-lock-default-fontify-region 1600
> 46%
> - font-lock-fontify-syntactically-region 1600
> 46%
> - python-font-lock-syntactic-face-function 1600
> 46%
> - python-info-docstring-p 1599
> 46%
> - python-nav-backward-sexp 1583
> 46%
> - python-nav-forward-sexp 1583
> 46%
> - python-nav--forward-sexp 1580
> 46%
> - python-info-end-of-block-p 893
> 26%
> - python-info-end-of-statement-p 893
> 26%
> - python-nav-end-of-statement 892
> 26%
> syntax-ppss 591
> 17%
> - python-info-line-ends-backslash-p 292
> 8%
> syntax-ppss 289
> 8%
> - python-info-beginning-of-statement-p 533
> 15%
> - python-nav-beginning-of-statement 531
> 15%
> syntax-ppss 275
> 8%
> - python-info-line-ends-backslash-p 251
> 7%
> - syntax-ppss 244
> 7%
> #<compiled -0x1ffffffff8326490> 1
> 0%
> back-to-indentation 4
> 0%
> - python-nav-beginning-of-statement 79
> 2%
> syntax-ppss 70
> 2%
> back-to-indentation 3
> 0%
> python-info-line-ends-backslash-p 2
> 0%
> - python-syntax-context-type 52
> 1%
> syntax-ppss 52
> 1%
> - python-nav--lisp-forward-sexp 7
> 0%
> forward-sexp 7
> 0%
> - python-info-beginning-of-block-p 7
> 0%
> python-info-statement-starts-block-p
> 3 0%
> python-info-beginning-of-statement-p
> 2 0%
> - python-info-current-line-empty-p 5
> 0%
> match-string-no-properties 1
> 0%
> syntax-class 1
> 0%
> - python-nav-beginning-of-statement 8
> 0%
> syntax-ppss 7
> 0%
> - python-info-line-ends-backslash-p 1
> 0%
> syntax-ppss 1
> 0%
> - python-info-assignment-statement-p 4
> 0%
> - python-nav-beginning-of-statement 4
> 0%
> - python-info-line-ends-backslash-p 3
> 0%
> syntax-ppss 3
> 0%
> syntax-ppss 1
> 0%
> - command-execute 1414
> 41%
> - call-interactively 1414
> 41%
> - funcall-interactively 1413
> 41%
> - scroll-up-command 1412
> 41%
> - scroll-up 1412
> 41%
> - jit-lock-function 1412
> 41%
> - jit-lock-fontify-now 1412
> 41%
> - jit-lock--run-functions 1412
> 41%
> - run-hook-wrapped 1412
> 41%
> - #<compiled -0x1ffffffff8c1b778> 1412
> 41%
> - font-lock-fontify-region 1412
> 41%
> - font-lock-default-fontify-region 1412
> 41%
> - font-lock-fontify-syntactically-region 1412
> 41%
> - python-font-lock-syntactic-face-function
> 1412 41%
> - python-info-docstring-p 1411
> 41%
> - python-nav-backward-sexp 1402
> 40%
> - python-nav-forward-sexp 1402
> 40%
> - python-nav--forward-sexp 1400
> 40%
> - python-info-beginning-of-block-p 1137
> 33%
> - python-info-beginning-of-statement-p
> 585 17%
> - python-nav-beginning-of-statement
> 585 17%
> - python-info-line-ends-backslash-p
> 327 9%
> syntax-ppss 322
> 9%
> syntax-ppss 248
> 7%
> back-to-indentation 7
> 0%
> - python-info-statement-starts-block-p
> 550 16%
> - python-nav-beginning-of-statement
> 546 15%
> syntax-ppss 270
> 7%
> - python-info-line-ends-backslash-p
> 262 7%
> syntax-ppss 256
> 7%
> back-to-indentation 6
> 0%
> - python-nav-beginning-of-statement
> 190 5%
> - python-info-line-ends-backslash-p
> 113 3%
> syntax-ppss 110
> 3%
> syntax-ppss 74
> 2%
> back-to-indentation 1
> 0%
> - python-syntax-context-type 61
> 1%
> syntax-ppss 61
> 1%
> - python-info-beginning-of-statement-p
> 4 0%
> python-nav-beginning-of-statement
> 3 0%
> - python-nav--lisp-forward-sexp 3
> 0%
> forward-sexp 2
> 0%
> python-info-current-line-empty-p 2
> 0%
> - python-info-end-of-block-p 1
> 0%
> python-info-end-of-statement-p 1
> 0%
> - python-nav-beginning-of-statement 4
> 0%
> - python-info-line-ends-backslash-p 3
> 0%
> syntax-ppss 3
> 0%
> syntax-ppss 1
> 0%
> - python-info-assignment-statement-p 4
> 0%
> - python-nav-beginning-of-statement 4
> 0%
> syntax-ppss 2
> 0%
> - python-info-line-ends-backslash-p 2
> 0%
> syntax-ppss 2
> 0%
> - execute-extended-command 1
> 0%
> - sit-for 1
> 0%
> - redisplay 1
> 0%
> - redisplay_internal (C function) 1
> 0%
> - tool-bar-make-keymap 1
> 0%
> - tool-bar-make-keymap-1 1
> 0%
> - mapcar 1
> 0%
> - #<compiled -0x1ffffffffa0b8700> 1
> 0%
> - eval 1
> 0%
> - find-image 1
> 0%
> image-type-available-p 1
> 0%
> - byte-code 1
> 0%
> - read-extended-command 1
> 0%
> - completing-read 1
> 0%
> completing-read-default 1
> 0%
> - ... 406
> 11%
> Automatic GC 406
> 11%
>
[Message part 2 (text/html, inline)]
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Fri, 14 Feb 2020 16:18:02 GMT) Full text and rfc822 format available.Message #17 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Fri, 14 Feb 2020 18:17:32 +0200
> From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> > Date: Fri, 14 Feb 2020 17:08:19 +0100 > Cc: 39598 <at> debbugs.gnu.org > > Sorry, didn't notice your first message (didn't expect such a quick response :). > > Yes, my profile is very similar to yours: It is. Looks like python-nav--forward-sexp is indeed the main culprit. To narrow this down even more, I suggest to load python.el (not .elc!) manually into a new Emacs session, then repeat the experiment, and show the profile below python-nav--forward-sexp. This might point out the part of that function that takes the lion's share of the run time. Thanks.
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Fri, 14 Feb 2020 16:33:02 GMT) Full text and rfc822 format available.Message #20 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> To: Eli Zaretskii <eliz <at> gnu.org> Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Fri, 14 Feb 2020 17:31:48 +0100
[Message part 1 (text/plain, inline)]
Ok, here's the full expansion of the relevant part of the profiler report:
- redisplay_internal (C function) 22195 91%
- jit-lock-function
22181 91%
- jit-lock-fontify-now
22181 91%
- jit-lock--run-functions
22179 91%
- run-hook-wrapped
22179 91%
- #<compiled 0x1727cc5>
22178 91%
- font-lock-fontify-region
22178 91%
- font-lock-default-fontify-region
22177 91%
- font-lock-fontify-syntactically-region
22142 90%
- python-font-lock-syntactic-face-function
22044 90%
- if
22043 90%
- if
22043 90%
- python-info-docstring-p
22041 90%
- save-excursion
22038 90%
- let
21959 90%
- if
21957 90%
- and
21957 90%
- >=
21881 89%
- let
21881 89%
- while
21879 89%
- save-excursion
21864 89%
- python-nav-backward-sexp
21772 89%
- python-nav-forward-sexp
21759 89%
- while
21732 89%
- python-nav--forward-sexp
21711 89%
- if
21706 89%
- let*
21700 89%
- cond
20732 85%
- let*
20686 84%
- cond
10735 44%
- python-info-beginning-of-block-p
4750 19%
- and
4748 19%
- python-info-beginning-of-statement-p
3683 15%
- =
3676 15%
- save-excursion
3671 15%
- python-nav-beginning-of-statement
3667 15%
- let*
3631 14%
- cond
2691 11%
- save-excursion
2679 11%
- python-info-line-ends-backslash-p
2673 10%
- save-excursion
2665 10%
- while
2661 10%
- and
2658 10%
- nth
2640 10%
- or
2638 10%
- syntax-ppss
2607 10%
#<compiled 0x163a7bd>
2 0%
goto-char
15 0%
not
1 0%
- if
3 0%
equal
2 0%
- syntax-ppss
916 3%
#<compiled 0x15ecea1>
2 0%
- or
10 0%
- let
4 0%
and
1 0%
nth
1 0%
back-to-indentation
21 0%
- python-info-statement-starts-block-p
1061 4%
- save-excursion
1057 4%
- python-nav-beginning-of-statement
1045 4%
- let*
1006 4%
- syntax-ppss
898 3%
#<compiled 0x57b3bd>
1 0%
- cond
77 0%
- save-excursion
74 0%
- python-info-line-ends-backslash-p
61 0%
- save-excursion
54 0%
- while
47 0%
- and
44 0%
- nth
21 0%
or
16 0%
goto-char
16 0%
- if
5 0%
equal
3 0%
- or
10 0%
let
2 0%
nth
1 0%
back-to-indentation
26 0%
- python-info-end-of-block-p
4015 16%
- and
4012 16%
- python-info-end-of-statement-p
4009 16%
- =
4006 16%
- save-excursion
3996 16%
- python-nav-end-of-statement
3988 16%
- let
3977 16%
- while
3975 16%
- and
3971 16%
- cond
3945 16%
- setq
2982 12%
-
python-info-line-ends-backslash-p
1994 8%
- save-excursion
1990 8%
- while
1979 8%
- and
1977 8%
- nth
1954 8%
- or
1949 8%
- syntax-ppss
1931 7%
#<compiled 0x17508a9>
1 0%
goto-char
10 0%
not
1 0%
- if
7 0%
equal
6 0%
- let
975 4%
- or
971 3%
- syntax-ppss
953 3%
#<compiled 0x163ff59>
2 0%
and
1 0%
- nth
958 3%
- or
955 3%
syntax-ppss
934 3%
- or
19 0%
goto-char
14 0%
not
1 0%
- python-info-beginning-of-statement-p
1963 8%
- =
1960 8%
- save-excursion
1954 8%
- python-nav-beginning-of-statement
1949 8%
- let*
1910 7%
syntax-ppss
945 3%
- cond
931 3%
- save-excursion
922 3%
- python-info-line-ends-backslash-p
914 3%
- save-excursion
909 3%
- while
906 3%
- and
905 3%
- nth
884 3%
- or
881 3%
- syntax-ppss
864 3%
#<compiled 0x1687431>
2 0%
goto-char
13 0%
if
1 0%
- or
16 0%
- let
5 0%
and
1 0%
nth
2 0%
back-to-indentation
30 0%
- save-excursion
7969 32%
- cond
7763 31%
- python-info-beginning-of-block-p
3887 15%
- and
3887 15%
- python-info-statement-starts-block-p
2870 11%
- save-excursion
2864 11%
- python-nav-beginning-of-statement
2847 11%
- let*
2799 11%
- cond
1911 7%
- save-excursion
1904 7%
- python-info-line-ends-backslash-p
1896 7%
- save-excursion
1889 7%
- while
1878 7%
- and
1876 7%
- nth
1851 7%
- or
1846 7%
- syntax-ppss
1826 7%
#<compiled 0x16c50a9>
1 0%
goto-char
21 0%
not
2 0%
- if
8 0%
equal
4 0%
syntax-ppss
864 3%
- or
5 0%
let
4 0%
back-to-indentation
31 0%
- python-info-beginning-of-statement-p
1012 4%
- =
1011 4%
- save-excursion
1008 4%
- python-nav-beginning-of-statement
1001 4%
- let*
950 3%
- syntax-ppss
873 3%
#<compiled 0x1771691>
1 0%
- cond
43 0%
- save-excursion
36 0%
- python-info-line-ends-backslash-p
31 0%
- save-excursion
25 0%
- while
17 0%
- and
14 0%
goto-char
8 0%
not
1 0%
- if
3 0%
equal
2 0%
- or
11 0%
- let
4 0%
and
1 0%
nth
1 0%
back-to-indentation
38 0%
- python-info-beginning-of-statement-p
1959 8%
- =
1959 8%
- save-excursion
1954 8%
- python-nav-beginning-of-statement
1948 8%
- let*
1912 7%
- cond
1014 4%
- save-excursion
1009 4%
- python-info-line-ends-backslash-p
999 4%
- save-excursion
990 4%
- while
978 4%
- and
976 4%
- nth
957 3%
- or
955 3%
syntax-ppss
937 3%
goto-char
11 0%
not
3 0%
- if
7 0%
equal
5 0%
- syntax-ppss
867 3%
#<compiled 0x16c5751>
1 0%
- or
9 0%
- let
3 0%
and
1 0%
back-to-indentation
26 0%
- python-info-end-of-block-p
1905 7%
- and
1904 7%
- python-info-end-of-statement-p
1901 7%
- =
1900 7%
- save-excursion
1891 7%
- python-nav-end-of-statement
1887 7%
- let
1881 7%
- while
1879 7%
- and
1879 7%
- cond
1844 7%
- setq
968 3%
- let
946 3%
- or
944 3%
syntax-ppss
923 3%
and
1 0%
-
python-info-line-ends-backslash-p
14 0%
- save-excursion
11 0%
while
5 0%
- nth
870 3%
- or
869 3%
- syntax-ppss
857 3%
#<compiled 0x16bc029>
1 0%
- or
26 0%
goto-char
23 0%
not
1 0%
- if
194 0%
- python-nav--lisp-forward-sexp
189 0%
- let
171 0%
forward-sexp
150 0%
- if
7 0%
or
3 0%
- if
1969 8%
- cond
1965 8%
- python-nav-beginning-of-statement
1878 7%
- let*
1842 7%
- cond
947 3%
- save-excursion
942 3%
- python-info-line-ends-backslash-p
934 3%
- save-excursion
930 3%
- while
925 3%
- and
925 3%
- nth
910 3%
- or
907 3%
- syntax-ppss
889 3%
#<compiled 0x156fe99>
1 0%
goto-char
9 0%
not
1 0%
if
1 0%
- syntax-ppss
870 3%
#<compiled 0x163e3e9>
1 0%
- or
12 0%
- let
4 0%
and
1 0%
nth
3 0%
back-to-indentation
26 0%
- and
66 0%
- python-info-current-line-empty-p
54 0%
- save-excursion
49 0%
- string-equal
3 0%
match-string-no-properties
1 0%
eq
5 0%
- and
33 0%
- or
32 0%
- if
30 0%
- eq
26 0%
- syntax-class
16 0%
- syntax-after
7 0%
1-
2 0%
car
4 0%
memq
2 0%
- python-syntax-context-type
953 3%
- let
944 3%
- or
931 3%
- syntax-ppss
909 3%
#<compiled 0x172d4c5>
2 0%
cond
5 0%
- if
8 0%
and
1 0%
setq
1 0%
setq
2 0%
- or
20 0%
- setq
20 0%
- memq
11 0%
list
4 0%
or
2 0%
- and
81 0%
looking-at-p
39 0%
=
23 0%
- prog1
6 0%
not
4 0%
setq
1 0%
- not
76 0%
- python-info-assignment-statement-p
75 0%
- save-excursion
75 0%
- let
74 0%
- if
64 0%
- python-nav-beginning-of-statement
64 0%
- let*
61 0%
- cond
34 0%
- save-excursion
34 0%
- python-info-line-ends-backslash-p
34 0%
- save-excursion
34 0%
- while
34 0%
- and
34 0%
- nth
34 0%
- or
34 0%
syntax-ppss
33 0%
syntax-ppss
25 0%
back-to-indentation
2 0%
- while
9 0%
and
9 0%
- python-nav-beginning-of-statement
78 0%
- let*
77 0%
syntax-ppss
38 0%
- cond
37 0%
- save-excursion
37 0%
- python-info-line-ends-backslash-p
37 0%
- save-excursion
36 0%
- while
36 0%
- and
36 0%
- nth
36 0%
- or
36 0%
syntax-ppss
36 0%
- or
1 0%
let
1 0%
- syntax-propertize
85 0%
- #<lambda 0x317b158c9689>
84 0%
- while
84 0%
- cond
81 0%
- python-syntax-stringify
81 0%
- let*
79 0%
- prog2
76 0%
syntax-ppss
76 0%
- and
1 0%
- python-syntax-count-quotes
1 0%
- let
1 0%
- while
1 0%
and
1 0%
cond
1 0%
and
3 0%
syntax-ppss
7 0%
- replace-regexp-in-string
1 0%
#<compiled 0x25de21>
1 0%
- font-lock-fontify-keywords-region
32 0%
- #<lambda 0x317b2112709>
9 0%
- let
9 0%
- while
9 0%
- and
9 0%
setq
9 0%
- #<lambda 0x317b2112709>
4 0%
- let
4 0%
- while
4 0%
- and
4 0%
setq
4 0%
+ #<compiled 0x486c29>
6 0%
+ eval
5 0%
+ tool-bar-make-keymap
2 0%
kill-this-buffer-enabled-p
1 0%
[Message part 2 (text/html, inline)]
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Sat, 15 Feb 2020 08:06:02 GMT) Full text and rfc822 format available.Message #23 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Sat, 15 Feb 2020 10:05:27 +0200
> From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com>
> Date: Fri, 14 Feb 2020 17:31:48 +0100
> Cc: 39598 <at> debbugs.gnu.org
>
> Ok, here's the full expansion of the relevant part of the profiler report:
Looks like this part of python-nav--forward-sexp needs some
optimizations:
(t
;; This part handles the lispy feel of
;; `python-nav-forward-sexp'. Knowing everything about the
;; current context and the context of the next sexp tries to
;; follow the lisp sexp motion commands in a symmetric manner.
(let* ((context
(cond
((python-info-beginning-of-block-p) 'block-start)
((python-info-end-of-block-p) 'block-end)
((python-info-beginning-of-statement-p) 'statement-start)
((python-info-end-of-statement-p) 'statement-end)))
(next-sexp-pos
(save-excursion
(if safe
(python-nav--lisp-forward-sexp-safe dir)
(python-nav--lisp-forward-sexp dir))
(point)))
(next-sexp-context
(save-excursion
(goto-char next-sexp-pos)
(cond
((python-info-beginning-of-block-p) 'block-start)
((python-info-end-of-block-p) 'block-end)
((python-info-beginning-of-statement-p) 'statement-start)
((python-info-end-of-statement-p) 'statement-end)
((python-info-statement-starts-block-p) 'starts-block)
((python-info-statement-ends-block-p) 'ends-block)))))
For starters, it does a lot of processing to compute next-sexp-context
whose result is needed only in a couple of cases thereafter. So maybe
avoiding some of that when not needed would help.
What is/are the real-life use case(s) where such long series of
strings causes slowdown?
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Sat, 15 Feb 2020 18:50:01 GMT) Full text and rfc822 format available.Message #26 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> To: Eli Zaretskii <eliz <at> gnu.org> Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Sat, 15 Feb 2020 19:49:37 +0100
[Message part 1 (text/plain, inline)]
> What is/are the real-life use case(s) where such long series of strings causes slowdown? This is an intermediate state of python buffer when I am trying to build a large-ish dictionary with lists of strings as a value. The most recent time this happened to me is when I was writing a django migration that would organize a list of entities into groups. I did a query to the database, copied the column of string ids, added a double quote to the beginning and to the end and when I tried to group and rearrange the lines together before I wrap them into lists, but emacs became unresponsive. What I find a bit weird is that it only happens to a long series of bare strings. If I construct an equally long proper python list of strings I don't see this behavior.
[Message part 2 (text/html, inline)]
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Sat, 15 Feb 2020 19:23:01 GMT) Full text and rfc822 format available.Message #29 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Eli Zaretskii <eliz <at> gnu.org> To: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Sat, 15 Feb 2020 21:22:53 +0200
> From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> > Date: Sat, 15 Feb 2020 19:49:37 +0100 > Cc: 39598 <at> debbugs.gnu.org > > > What is/are the real-life use case(s) where such long series of strings causes slowdown? > > This is an intermediate state of python buffer when I am trying to build a large-ish dictionary with lists of strings > as a value. Why do you need that intermediate buffer to be in Python mode? > What I find a bit weird is that it only happens to a long series of bare strings. If I construct an equally long > proper python list of strings I don't see this behavior. I guess because the code is tuned to support valid Python code, not something that shouldn't happen in Python sources?
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Sat, 15 Feb 2020 19:33:01 GMT) Full text and rfc822 format available.Message #32 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com> To: Eli Zaretskii <eliz <at> gnu.org> Cc: 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Sat, 15 Feb 2020 20:31:45 +0100
[Message part 1 (text/plain, inline)]
> Why do you need that intermediate buffer to be in Python mode? That's an intermediate state of the buffer where the rest of the migration code is. I was not trying to hit a corner case, this is just the way I happen to write code -- sometimes a paste text into the the buffer where the rest of the code is and then I edit it until it becomes a valid data structure. I am pretty sure I am not the only one who does this. > I guess because the code is tuned to support valid Python code, not something that shouldn't happen in Python sources? That example *is* a valid python code. And the behavior I see *is* a bug. And honestly, I don't think that this was a design choice along the lines "we're going to write an optimal parser for this subset of the language but we will occasionally hang up on this one".
[Message part 2 (text/html, inline)]
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Fri, 13 Mar 2020 03:01:01 GMT) Full text and rfc822 format available.Message #35 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Noam Postavsky <npostavs <at> gmail.com> To: Eli Zaretskii <eliz <at> gnu.org> Cc: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com>, 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Thu, 12 Mar 2020 23:00:09 -0400
Eli Zaretskii <eliz <at> gnu.org> writes:
>> From: Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com>
>> Date: Fri, 14 Feb 2020 17:31:48 +0100
>> Cc: 39598 <at> debbugs.gnu.org
>>
>> Ok, here's the full expansion of the relevant part of the profiler report:
>
> Looks like this part of python-nav--forward-sexp needs some
> optimizations:
Actually, it seems the main problem is a bit higher up.
python-info-docstring-p calls python-nav-backward-sexp repeatedly, until
it hits a non-string sexp. Even though it's only checking for two sexp
strings.
(let ((counter 1)
...
;; Allow up to two consecutive docstrings only.
(>=
2
(let (last-backward-sexp-point)
(while (save-excursion
(python-nav-backward-sexp)
(setq backward-sexp-point (point))
(and (= indentation (current-indentation))
...
(looking-at-p
(concat "[uU]?[rR]?"
(python-rx string-delimiter)))))
;; Previous sexp was a string, restore point.
(goto-char backward-sexp-point)
(cl-incf counter))
counter)))
So any repetitions of the while loop after the second one are useless.
The patch (generated with --ignore-all-space) below fixes it:
--- c/lisp/progmodes/python.el
+++ i/lisp/progmodes/python.el
@@ -5135,7 +5135,8 @@ python-info-docstring-p
(>=
2
(let (last-backward-sexp-point)
- (while (save-excursion
+ (while (and (<= counter 2)
+ (save-excursion
(python-nav-backward-sexp)
(setq backward-sexp-point (point))
(and (= indentation (current-indentation))
@@ -5149,7 +5150,7 @@ python-info-docstring-p
backward-sexp-point))
(looking-at-p
(concat "[uU]?[rR]?"
- (python-rx string-delimiter)))))
+ (python-rx string-delimiter))))))
;; Previous sexp was a string, restore point.
(goto-char backward-sexp-point)
(cl-incf counter))
bug-gnu-emacs <at> gnu.org:bug#39598; Package emacs.
(Sun, 20 Sep 2020 08:48:01 GMT) Full text and rfc822 format available.Message #38 received at 39598 <at> debbugs.gnu.org (full text, mbox):
From: Lars Ingebrigtsen <larsi <at> gnus.org> To: Noam Postavsky <npostavs <at> gmail.com> Cc: Eli Zaretskii <eliz <at> gnu.org>, Ivan Oreshnikov <oreshnikov.ivan <at> gmail.com>, 39598 <at> debbugs.gnu.org Subject: Re: bug#39598: 26.3; Emacs is extremely unresponsive on a trivial python file Date: Sun, 20 Sep 2020 10:47:23 +0200
Noam Postavsky <npostavs <at> gmail.com> writes: > So any repetitions of the while loop after the second one are useless. > The patch (generated with --ignore-all-space) below fixes it: > > --- c/lisp/progmodes/python.el > +++ i/lisp/progmodes/python.el > @@ -5135,7 +5135,8 @@ python-info-docstring-p > (>= > 2 > (let (last-backward-sexp-point) > - (while (save-excursion > + (while (and (<= counter 2) > + (save-excursion There wasn't any discussion of this patch, but it seems to fix the problem and doesn't introduce any obvious regressions that I can see, so I've applied it to Emacs 28. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no
Lars Ingebrigtsen <larsi <at> gnus.org>
to control <at> debbugs.gnu.org.
(Sun, 20 Sep 2020 08:48:02 GMT) Full text and rfc822 format available.Lars Ingebrigtsen <larsi <at> gnus.org>
to control <at> debbugs.gnu.org.
(Sun, 20 Sep 2020 08:48:02 GMT) Full text and rfc822 format available.Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Sun, 18 Oct 2020 11:24:12 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.