GNU bug report logs - #41220
26.3; Repeated yielding in worker thread insufficient to process input events in the main thread

Previous Next

To reply to this bug, email your comments to 41220 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alexander Miller <alexanderm <at> web.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 26.3; Repeated yielding in worker thread insufficient to process
 input events in the main thread
Date: Tue, 12 May 2020 23:56:52 +0200

This report is based on the discussion in
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg01566.html.

In this thread we were thinking up ways how to make Elisp
asynchronous with the help of threads - to do heavy lifting in a worker
thread, but yield for user interactions. Part of that was a benchmark
for how well persistently yielding in the worker thread after every
finished work-unit would perform.

My conclusion was that it doesn't, input was ignored most of the
time. Stefan thought that it *should* work, and asked me to make a bug
report, so here it is.

I have modified my benchmark to work with emacs -q. It will set up a
worker thread, a queue, a timer to check the queue and wake up the
worker, as well as a long CPU consuming task that emulates the kind of
work I would want to use threads for in a real-world scenario.

The task needs close to half a minute on my system. While it runs
navigation with C-n/p is barely possible.

Here is the full code:

(defconst worker-mutex (make-mutex "*WORKER MUTEX*"))
(defconst worker-cond-var (make-condition-variable worker-mutex))
(defvar worker-queue (list))

(setf
 worker
 (make-thread
  (lambda ()
    (while t
      (while worker-queue
        (let* ((work-unit (pop worker-queue))
               (fn (car work-unit))
               (args (cdr work-unit)))
          (apply fn args)
          (thread-yield)))
      (message "Worker Goes To Sleep")
      (with-mutex worker-mutex
        (condition-wait worker-cond-var))))
  "*WORKER*"))

(setf worker-timer
      (run-with-idle-timer
       1 t (lambda ()
             (when worker-queue
               (with-mutex worker-mutex
                 (condition-notify worker-cond-var))))))

(defun get-the-things ()
  (let ((run-start (float-time))
        (things))
    (dotimes (_ 100000)
      (let* ((status (prog1 (random 4) (dotimes (_ 1000) (random))))
             (thing (pcase status
                      (0 'a)
                      (1 'b)
                      (2 'c)
                      (3 'd))))
        (push thing things))
      (thread-yield))
    (message "Run Time %ss" (- (float-time) run-start))))

(push (list #'get-the-things) worker-queue)

In GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.10)
of 2019-08-29 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Manjaro Linux

Recent messages:
worker-cond-var
worker-queue
#<thread *WORKER*>
Worker Goes To Sleep
[nil 0 1 0 t (closure (t) nil (if worker-queue (progn (let ((mutex
worker-mutex)) (mutex-lock mutex) (unwind-protect (progn
(condition-notify worker-cond-var)) (mutex-unlock mutex)))))) nil idle 0]
get-the-things
((get-the-things))
Auto-saving...
Run Time 90.80384421348572s
Worker Goes To Sleep

Configured using:
'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
--localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules
'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt'
CPPFLAGS=-D_FORTIFY_SOURCE=2
LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD LCMS2

Important settings:
value of $LC_MONETARY: de_DE.UTF-8
value of $LC_NUMERIC: de_DE.UTF-8
value of $LC_TIME: de_DE.UTF-8
value of $LANG: en_GB.utf8
locale-coding-system: utf-8

Major mode: Emacs-Lisp

Minor modes in effect:
ivy-posframe-mode: t
diff-auto-refine-mode: t
rainbow-delimiters-mode: t
rainbow-mode: t
display-line-numbers-mode: t
helm-mode: t
async-bytecomp-package-mode: t
helm--remap-mouse-mode: t
projectile-mode: t
global-company-mode: t
company-mode: t
global-evil-vimish-fold-mode: t
evil-vimish-fold-mode: t
vimish-fold-mode: t
show-smartparens-global-mode: t
show-smartparens-mode: t
smartparens-global-mode: t
smartparens-mode: t
org-super-agenda-mode: t
treemacs-icons-dired-mode: t
treemacs-filewatch-mode: t
treemacs-follow-mode: t
treemacs-git-mode: deferred
treemacs-fringe-indicator-mode: t
gcmh-mode: t
framey-mode: t
purpose-mode: t
shackle-mode: t
winum-mode: t
eyebrowse-mode: t
global-subword-mode: t
subword-mode: t
global-evil-surround-mode: t
evil-surround-mode: t
evil-lion-mode: t
evil-goggles-mode: t
global-undo-tree-mode: t
undo-tree-mode: t
shell-dirtrack-mode: t
evil-mode: t
evil-local-mode: t
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
prettify-symbols-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
line-number-mode: t
transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/26.3/lisp/emacs-lisp/let-alist hides
/home/am/.emacs.d/straight/build/let-alist/let-alist
/usr/share/emacs/site-lisp/mu4e/mu4e-headers hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-headers
/usr/share/emacs/site-lisp/mu4e/mu4e-draft hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-draft
/usr/share/emacs/site-lisp/mu4e/mu4e-lists hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-lists
/usr/share/emacs/site-lisp/mu4e/mu4e-proc hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-proc
/usr/share/emacs/site-lisp/mu4e/org-mu4e hides
/home/am/.emacs.d/straight/build/mu4e/org-mu4e
/usr/share/emacs/site-lisp/mu4e/mu4e-message hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-message
/usr/share/emacs/site-lisp/mu4e/mu4e-compose hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-compose
/usr/share/emacs/site-lisp/mu4e/mu4e-speedbar hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-speedbar
/usr/share/emacs/site-lisp/mu4e/mu4e-vars hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-vars
/usr/share/emacs/site-lisp/mu4e/mu4e-org hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-org
/usr/share/emacs/site-lisp/mu4e/mu4e-mark hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-mark
/usr/share/emacs/site-lisp/mu4e/mu4e-context hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-context
/usr/share/emacs/site-lisp/mu4e/mu4e-view hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-view
/usr/share/emacs/site-lisp/mu4e/mu4e-utils hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-utils
/usr/share/emacs/site-lisp/mu4e/mu4e-contrib hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-contrib
/usr/share/emacs/site-lisp/mu4e/mu4e-actions hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-actions
/usr/share/emacs/site-lisp/mu4e/mu4e hides
/home/am/.emacs.d/straight/build/mu4e/mu4e
/usr/share/emacs/site-lisp/mu4e/mu4e-main hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-main
/usr/share/emacs/site-lisp/mu4e/mu4e-icalendar hides
/home/am/.emacs.d/straight/build/mu4e/mu4e-icalendar
~/Documents/git/treemacs/src/elisp/treemacs-interface hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-interface
~/Documents/git/treemacs/src/elisp/treemacs-rendering hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-rendering
~/Documents/git/treemacs/src/elisp/treemacs-dom hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-dom
~/Documents/git/treemacs/src/elisp/treemacs-tags hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-tags
~/Documents/git/treemacs/src/elisp/treemacs hides
/home/am/.emacs.d/straight/build/treemacs/treemacs
~/Documents/git/treemacs/src/elisp/treemacs-workspaces hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-workspaces
~/Documents/git/treemacs/src/elisp/treemacs-customization hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-customization
~/Documents/git/treemacs/src/elisp/treemacs-faces hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-faces
~/Documents/git/treemacs/src/elisp/treemacs-themes hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-themes
~/Documents/git/treemacs/src/elisp/treemacs-mouse-interface hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-mouse-interface
~/Documents/git/treemacs/src/elisp/treemacs-mode hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-mode
~/Documents/git/treemacs/src/elisp/treemacs-icons hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-icons
~/Documents/git/treemacs/src/elisp/treemacs-compatibility hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-compatibility
~/Documents/git/treemacs/src/elisp/treemacs-follow-mode hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-follow-mode
~/Documents/git/treemacs/src/elisp/treemacs-visuals hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-visuals
~/Documents/git/treemacs/src/elisp/treemacs-core-utils hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-core-utils
~/Documents/git/treemacs/src/elisp/treemacs-extensions hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-extensions
~/Documents/git/treemacs/src/elisp/treemacs-filewatch-mode hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-filewatch-mode
~/Documents/git/treemacs/src/elisp/treemacs-persistence hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-persistence
~/Documents/git/treemacs/src/elisp/treemacs-async hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-async
~/Documents/git/treemacs/src/elisp/treemacs-bookmarks hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-bookmarks
~/Documents/git/treemacs/src/elisp/treemacs-tag-follow-mode hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-tag-follow-mode
~/Documents/git/treemacs/src/elisp/treemacs-logging hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-logging
~/Documents/git/treemacs/src/elisp/treemacs-fringe-indicator hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-fringe-indicator
~/Documents/git/treemacs/src/elisp/treemacs-diagnostics hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-diagnostics
~/Documents/git/treemacs/src/elisp/treemacs-macros hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-macros
~/Documents/git/treemacs/src/elisp/treemacs-scope hides
/home/am/.emacs.d/straight/build/treemacs/treemacs-scope

Features:
(shadow sort mail-extr emacsbug cl-print debug benchmark eros
company-quickhelp pos-tip ivy-posframe posframe swiper ivy delsel
ivy-overlay colir help-fns radix-tree org-clock holidays hol-loaddefs
diary-lib diary-loaddefs cal-iso smartparens-javascript js
smartparens-html sgml-mode cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs smartparens-python python
tramp-sh tramp tramp-compat tramp-loaddefs trampver ucs-normalize json
map fish-mode conf-mode i3wm-config-mode writeroom-mode
visual-fill-column autorevert org-indent disp-table ol-eww eww mm-url
ol-rmail ol-mhe ol-irc ol-info ol-gnus nnir gnus-sum gnus-group
gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source tls gnutls
utf7 netrc nnoo parse-time gnus-spec gnus-int gnus-range gnus-win gnus
nnheader ol-docview ol-bibtex bibtex ol-bbdb ol-w3m elfeed-org
elfeed-show elfeed-search wid-edit treemacs-bookmarks bookmark
elfeed-csv elfeed elfeed-curl url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf elfeed-log elfeed-db
elfeed-lib url-queue xml-query mailcap vc-mtn vc-hg vc-git diff-mode
vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs vc vc-dispatcher
rainbow-delimiters rainbow-mode xterm-color display-line-numbers
elec-pair winner helm-command helm-elisp helm-eval edebug helm-info info
shr-color color url-util url-parse url-vars shr svg dom face-remap pp
helm-mode helm-projectile helm-files helm-tags helm-buffers helm-occur
helm-grep helm-regexp helm-utils helm-locate helm-help helm-types
framey-helm helm-config async-bytecomp helm helm-source helm-multi-match
helm-lib async view treemacs-projectile projectile grep compile ibuf-ext
ibuffer ibuffer-loaddefs mu4e-alert time alert log4e notifications dbus
company-keywords company-dabbrev-code company-dabbrev company-yasnippet
company-files company-capf company server evil-vimish-fold vimish-fold
smartparens-config smartparens-org smartparens-text paren smartparens
xml gntp org-mu4e mu4e desktop frameset mu4e-org german-holidays
org-super-agenda ts org-habit org-element avl-tree generator org-agenda
org-refile org-bullets org ob ob-tangle ob-ref ob-lob ob-table ob-exp
org-macro org-footnote org-src ob-comint org-pcomplete org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp
ob-core ob-eval org-table ol org-keys org-compat org-macs org-loaddefs
find-func mu4e-main mu4e-view cal-menu calendar cal-loaddefs browse-url
mu4e-headers mu4e-compose mu4e-context mu4e-draft mu4e-actions ido
rfc2368 smtpmail auth-source sendmail mu4e-mark mu4e-message flow-fill
mu4e-proc mu4e-utils doc-view jka-compr mu4e-lists mu4e-vars message rmc
puny seq treemacs-icons-dired treemacs-evil treemacs
treemacs-header-line treemacs-compatibility treemacs-mode
treemacs-interface treemacs-extensions treemacs-persistence
treemacs-mouse-interface treemacs-tag-follow-mode
treemacs-filewatch-mode treemacs-tags imenu xref project filenotify
treemacs-follow-mode treemacs-rendering treemacs-async
treemacs-workspaces treemacs-dom treemacs-visuals
treemacs-fringe-indicator treemacs-scope treemacs-faces treemacs-icons
treemacs-themes treemacs-core-utils pfuture ace-window avy hl-line
treemacs-macros treemacs-logging treemacs-customization easy-mmode
dired+ image-dired image-mode image-file dired-x dired-aux dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader mu4e-meta gcmh hydra lv pcase
doom-modeline doom-modeline-segments doom-modeline-env
doom-modeline-core shrink-path rx f all-the-icons all-the-icons-faces
data-material data-weathericons data-octicons data-fileicons
data-faicons data-alltheicons memoize subr-x framey inline ht s
window-purpose window-purpose-fixes window-purpose-prefix-overload
window-purpose-switch let-alist window-purpose-layout
window-purpose-core window-purpose-configuration eieio-compat eieio
byte-opt bytecomp byte-compile cconv window-purpose-utils shackle trace
cl-extra help-mode winum eyebrowse format-spec morning-star-theme
cap-words superword subword evil-surround evil-lion evil-goggles pulse
cl-seq evil evil-keybindings evil-integration undo-tree diff evil-maps
evil-commands reveal flyspell ispell evil-jumps evil-command-window
evil-types evil-search evil-ex shell pcomplete comint ansi-color
evil-macros evil-repeat evil-states evil-core advice evil-common
windmove thingatpt rect evil-digraphs evil-vars ring edmacro kmacro dash
yasnippet-autoloads yaml-mode-autoloads xterm-color-autoloads
wttrin-autoloads writeroom-mode-autoloads with-editor-autoloads
winum-autoloads window-purpose-autoloads eieio-core cl-macs gv
eieio-loaddefs cl-loaddefs cl-lib vterm-autoloads
visual-fill-column-autoloads vimish-fold-autoloads undo-tree-autoloads
ts-autoloads tridactylrc-mode-autoloads treepy-autoloads
treemacs-autoloads transient-autoloads swiper-autoloads
straight-autoloads spinner-autoloads smartparens-autoloads
shrink-path-autoloads shackle-autoloads s-autoloads
rainbow-mode-autoloads rainbow-delimiters-autoloads projectile-autoloads
pretty-hydra-autoloads posframe-autoloads pos-tip-autoloads
popup-autoloads pkg-info-autoloads pfuture-autoloads
persp-mode-autoloads org-super-agenda-autoloads org-bullets-autoloads
org-autoloads multi-compile-autoloads mu4e-alert-autoloads
mu4e-autoloads morning-star-autoloads memoize-autoloads
markdown-mode-autoloads magit-autoloads macrostep-autoloads lv-autoloads
lsp-ui-autoloads lsp-treemacs-autoloads lsp-mode-autoloads
loop-autoloads log4e-autoloads let-alist-autoloads ledger-mode-autoloads
ivy-posframe-autoloads ivy-autoloads imenu-list-autoloads
i3wm-config-mode-autoloads hydra-autoloads ht-autoloads
helpful-autoloads helm-projectile-autoloads helm-org-autoloads
helm-easymenu easymenu helm-core-autoloads helm-ag-autoloads
helm-autoloads goto-chg-autoloads gntp-autoloads git-commit-autoloads
ghub-autoloads german-holidays-autoloads gcmh-autoloads framey-autoloads
forge-autoloads flycheck-pos-tip-autoloads flycheck-autoloads
fish-mode-autoloads fill-column-indicator-autoloads f-autoloads
eyebrowse-autoloads expand-region-autoloads evil-vimish-fold-autoloads
evil-surround-autoloads evil-nerd-commenter-autoloads
evil-magit-autoloads evil-lion-autoloads evil-ledger-autoloads
evil-goggles-autoloads evil-exchange-autoloads evil-collection-autoloads
evil-autoloads eros-autoloads epl-autoloads emacsql-sqlite-autoloads
emacsql-autoloads elisp-refs-autoloads elfeed-org-autoloads
elfeed-autoloads doom-modeline-autoloads doct-autoloads dired+-autoloads
dash-functional-autoloads dash-autoloads company-shell-autoloads
company-quickhelp-autoloads company-makefile-autoloads
company-box-autoloads company-autoloads closql-autoloads
buttercup-autoloads avy-autoloads async-autoloads anzu-autoloads
annalist-autoloads all-the-icons-autoloads alert-autoloads
ace-window-autoloads time-date mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 1134735 336269)
(symbols 48 62609 1)
(miscs 40 669 2944)
(strings 32 246316 89817)
(string-bytes 1 7685472)
(vectors 16 113372)
(vector-slots 8 2276151 279318)
(floats 8 6416 2552)
(intervals 56 38016 2345)
(buffers 992 44))

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.