GNU bug report logs - #36877
25.3; uniquify renaway uniquify-managed list

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36877 in the body.
You can then email your comments to 36877 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: David Biesack <David.Biesack <at> Apiture.com>
To: "bug-gnu-emacs <at> gnu.org" <bug-gnu-emacs <at> gnu.org>
Subject: 25.3; uniquify renaway uniquify-managed list
Date: Wed, 31 Jul 2019 16:04:56 +0000

The list uniquified-managed sometimes starts to grow in an unmanaged manner (I suspect exponentially) after running
Emacs for a few days. My work involves heavily editing many files named "openapi.yaml" in about 20 different
directories, often switching git branches so I need to revisit (M-x revert-buffer) these files a lot.

After a few days, I noticed opening new openapi.yaml files started to slow down dramatically - there would be several
seconds of delay, then tens of seconds. I narrowed this to uniquify. In my buffers, the local variable
uniquified-managed is more than 100,000 items long, with many duplicate items.

Here is some data (I grabbed the local variable uniquify-managed from my openapi.yaml buffer)

(length uniquify-managed)
100004

(cl-subseq uniquify-managed 0 19)
(
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/cards-api-def/src/openapi" #<buffer openapi.yaml<3>> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/acct-verifications-api-def" #<buffer openapi.yaml<acct-verifications-api-def>> "openapi.yaml<acct-verifications-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/organizations-api-def/src/openapi" #<buffer openapi.yaml> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/contacts-api-def/src/openapi" #<buffer openapi.yaml<2>> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/contacts-api-def" #<buffer openapi.yaml<contacts-api-def>> "openapi.yaml<contacts-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/transactions-api-def" #<buffer openapi.yaml<transactions-api-def>> "openapi.yaml<transactions-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/configurations-api-def" #<buffer openapi.yaml<configurations-api-def>> "openapi.yaml<configurations-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/users-api-def" #<buffer openapi.yaml<users-api-def>> "openapi.yaml</Users/david.biesack/dev/openapi/users-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/accounts-api-def/gen/openapi" #<buffer openapi.yaml<gen/openapi>> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/accounts-api-def/gen/openapi" #<buffer openapi.yaml<gen/openapi>> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/accounts-api-def/gen/openapi" #<buffer openapi.yaml<gen/openapi>> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/configurations-api-def" #<buffer openapi.yaml<configurations-api-def>> "openapi.yaml<configurations-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/users-api-def" #<buffer openapi.yaml<users-api-def>> "openapi.yaml</Users/david.biesack/dev/openapi/users-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/accounts-api-def/gen/openapi" #<buffer openapi.yaml<gen/openapi>> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/accounts-api-def/gen/openapi" #<buffer openapi.yaml<gen/openapi>> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/accounts-api-def/gen/openapi" #<buffer openapi.yaml<gen/openapi>> "openapi.yaml<openapi>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/configurations-api-def" #<buffer openapi.yaml<configurations-api-def>> "openapi.yaml<configurations-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/users-api-def" #<buffer openapi.yaml<users-api-def>> "openapi.yaml</Users/david.biesack/dev/openapi/users-api-def>"]
[cl-struct-uniquify-item "openapi.yaml" "/Users/david.biesack/dev/openapi/accounts-api-def/gen/openapi" #<buffer openapi.yaml<gen/openapi>> "openapi.yaml<openapi>"])

(mapcar (lambda (buf)
          (set-buffer buf)
          (format "uniquify-managed length is %d for buffer %s" (length uniquify-managed) (buffer-name)))
 (buffer-list))
"uniquify-managed length is 0 for buffer *scratch*"
"uniquify-managed length is 0 for buffer  *Minibuf-1*"
"uniquify-managed length is 0 for buffer *Ibuffer*"
"uniquify-managed length is 1 for buffer transient.el"
"uniquify-managed length is 1 for buffer uniquify.el.gz"
"uniquify-managed length is 100004 for buffer openapi.yaml<3>"
"uniquify-managed length is 1 for buffer emacs.el"
"uniquify-managed length is 3 for buffer schemas<dev>"
"uniquify-managed length is 1 for buffer browse-url.el.gz"
"uniquify-managed length is 0 for buffer *shell*"
"uniquify-managed length is 0 for buffer *compile-shell*"
"uniquify-managed length is 0 for buffer  SPEEDBAR"
"uniquify-managed length is 100004 for buffer openapi.yaml<acct-verifications-api-def>"
"uniquify-managed length is 5 for buffer package.json<localhost>"
"uniquify-managed length is 100004 for buffer openapi.yaml"
"uniquify-managed length is 5 for buffer package.json<organizations-api-def>"
"uniquify-managed length is 0 for buffer %1"
"uniquify-managed length is 1 for buffer .functions"
"uniquify-managed length is 1 for buffer .alias"
"uniquify-managed length is 5 for buffer package.json<starter-api-def>"
"uniquify-managed length is 5 for buffer package.json<contacts-api-def>"
"uniquify-managed length is 100004 for buffer openapi.yaml<2>"
"uniquify-managed length is 1 for buffer openapi"
"uniquify-managed length is 10 for buffer model.yaml<simpleContact>"
"uniquify-managed length is 1 for buffer README.md"
"uniquify-managed length is 1 for buffer contacts-api-def"
"uniquify-managed length is 100004 for buffer openapi.yaml<configurations-api-def>"
"uniquify-managed length is 1 for buffer accounts"
"uniquify-managed length is 100004 for buffer openapi.yaml<gen/openapi>"
"uniquify-managed length is 1 for buffer common"
"uniquify-managed length is 10 for buffer model.yaml<dev/schemas/configurations/configurationGroupSummary>"
"uniquify-managed length is 10 for buffer model.yaml<configurationGroup>"
"uniquify-managed length is 0 for buffer *Occur*"
"uniquify-managed length is 10 for buffer model.yaml<localhost:8080/schemas/configurations/configurationGroupSummary>"
"uniquify-managed length is 1 for buffer v1.0.0"
"uniquify-managed length is 1 for buffer configurationGroupSummary"
"uniquify-managed length is 2 for buffer configurations<localhost:8080>"
"uniquify-managed length is 5 for buffer package.json<cards-api-def>"
"uniquify-managed length is 1 for buffer labels.yaml"
"uniquify-managed length is 100004 for buffer openapi.yaml<transactions-api-def>"
"uniquify-managed length is 2 for buffer configurations<dev>"
"uniquify-managed length is 2 for buffer copy-schemas<david.biesack>"
"uniquify-managed length is 100004 for buffer openapi.yaml<users-api-def>"
"uniquify-managed length is 2 for buffer .gitignore<openapi-model-gen>"
"uniquify-managed length is 1 for buffer prettier.config.js"
"uniquify-managed length is 0 for buffer  *Minibuf-0*"
"uniquify-managed length is 0 for buffer *Messages*"
"uniquify-managed length is 0 for buffer  *code-conversion-work*"
"uniquify-managed length is 0 for buffer  *Echo Area 0*"
"uniquify-managed length is 0 for buffer  *Echo Area 1*"
"uniquify-managed length is 0 for buffer  *server*"
"uniquify-managed length is 0 for buffer *Diff*"
"uniquify-managed length is 0 for buffer  *code-converting-work*"
"uniquify-managed length is 0 for buffer *Compile-Log*"
"uniquify-managed length is 0 for buffer *Ediff Registry*"
"uniquify-managed length is 0 for buffer *vc*"
"uniquify-managed length is 100004 for buffer openapi.yaml<contacts-api-def>"
"uniquify-managed length is 0 for buffer *Completions*"
"uniquify-managed length is 1 for buffer filemenu"
"uniquify-managed length is 0 for buffer *Backtrace*"
"uniquify-managed length is 0 for buffer *Help*"

In GNU Emacs 25.3.1 (x86_64-apple-darwin13.4.0, NS appkit-1265.21 Version 10.9.5 (Build 13F1911))
 of 2017-09-12 built on builder10-9.local
Windowing system distributor 'Apple', version 10.3.1671
Configured using:
 'configure --with-ns '--enable-locallisppath=/Library/Application Support/Emacs/${version}/site-lisp:/Library/Application Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  iswitchb-mode: t
  diff-auto-refine-mode: t
  nxhtml-menu-mode: t
  nxhtml-tag-do-also: t
  popcmp-group-alternatives: t
  popcmp-short-help-beside-alts: t
  mlinks-active-links: t
  rngalt-minimal-validation-header: t
  rngalt-display-validation-header: t
  shell-dirtrack-mode: t
  show-paren-mode: t
  midnight-mode: t
  display-time-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent messages:
Error during redisplay: (jit-lock-function 5001) signaled (wrong-type-argument arrayp nil)
Loading mailabbrev...done
mail-signature-file-content: Opening input file: No such file or directory, /Users/david.biesack/.signature
(New file)
Mark set [2 times]
Wrote /Users/david.biesack/.signature
Saving file /Users/david.biesack/.signature...
Mark set [2 times]
Wrote /Users/david.biesack/.signature
Kill buffer *unsent mail to bug-gnu-emacs <at> gnu.org*? (y or n) y

Load-path shadows:
/Users/david.biesack/emacs/nxhtml/web-vcs hides /Users/david.biesack/emacs/web-vcs
/Users/david.biesack/emacs/nxhtml/nxhtml-web-vcs hides /Users/david.biesack/emacs/nxhtml-web-vcs
/Users/david.biesack/emacs/bs hides /Applications/Emacs.app/Contents/Resources/lisp/bs
/Users/david.biesack/emacs/wdired hides /Applications/Emacs.app/Contents/Resources/lisp/wdired
/Users/david.biesack/emacs/autoinsert hides /Applications/Emacs.app/Contents/Resources/lisp/autoinsert
/Users/david.biesack/emacs/align hides /Applications/Emacs.app/Contents/Resources/lisp/align
/Users/david.biesack/emacs/cmuscheme hides /Applications/Emacs.app/Contents/Resources/lisp/cmuscheme
/Users/david.biesack/emacs/indent hides /Applications/Emacs.app/Contents/Resources/lisp/indent
/Users/david.biesack/emacs/whitespace hides /Applications/Emacs.app/Contents/Resources/lisp/whitespace
/Users/david.biesack/emacs/midnight hides /Applications/Emacs.app/Contents/Resources/lisp/midnight
/Users/david.biesack/emacs/xml hides /Applications/Emacs.app/Contents/Resources/lisp/xml
/Users/david.biesack/emacs/url/url-methods hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-methods
/Users/david.biesack/emacs/url/url-dav hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-dav
/Users/david.biesack/emacs/url/url-file hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-file
/Users/david.biesack/emacs/url/url-cookie hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-cookie
/Users/david.biesack/emacs/url/url-util hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-util
/Users/david.biesack/emacs/url/url-http hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-http
/Users/david.biesack/emacs/url/url-cid hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-cid
/Users/david.biesack/emacs/url/url-handlers hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-handlers
/Users/david.biesack/emacs/url/url-gw hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-gw
/Users/david.biesack/emacs/url/url-auth hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-auth
/Users/david.biesack/emacs/url/url-expand hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-expand
/Users/david.biesack/emacs/url/url-vars hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-vars
/Users/david.biesack/emacs/url/url-irc hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-irc
/Users/david.biesack/emacs/url/url-mailto hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-mailto
/Users/david.biesack/emacs/url/url-proxy hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-proxy
/Users/david.biesack/emacs/url/url-news hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-news
/Users/david.biesack/emacs/url/url-history hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-history
/Users/david.biesack/emacs/url/url-privacy hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-privacy
/Users/david.biesack/emacs/url/url-nfs hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-nfs
/Users/david.biesack/emacs/url/url-about hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-about
/Users/david.biesack/emacs/url/url-dired hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-dired
/Users/david.biesack/emacs/url/url hides /Applications/Emacs.app/Contents/Resources/lisp/url/url
/Users/david.biesack/emacs/url/url-imap hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-imap
/Users/david.biesack/emacs/url/url-cache hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-cache
/Users/david.biesack/emacs/url/url-ldap hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-ldap
/Users/david.biesack/emacs/url/url-ns hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-ns
/Users/david.biesack/emacs/url/url-ftp hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-ftp
/Users/david.biesack/emacs/url/url-misc hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-misc
/Users/david.biesack/emacs/url/url-parse hides /Applications/Emacs.app/Contents/Resources/lisp/url/url-parse
/Users/david.biesack/emacs/table hides /Applications/Emacs.app/Contents/Resources/lisp/textmodes/table
/Users/david.biesack/emacs/glasses hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/glasses
/Users/david.biesack/emacs/icon hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/icon
/Users/david.biesack/emacs/antlr-mode hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/antlr-mode
/Users/david.biesack/emacs/cpp hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/cpp
/Users/david.biesack/emacs/xscheme hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/xscheme
/Users/david.biesack/emacs/bat-mode hides /Applications/Emacs.app/Contents/Resources/lisp/progmodes/bat-mode
/Users/david.biesack/emacs/zone hides /Applications/Emacs.app/Contents/Resources/lisp/play/zone
/Users/david.biesack/emacs/solitaire hides /Applications/Emacs.app/Contents/Resources/lisp/play/solitaire
/Users/david.biesack/emacs/metamail hides /Applications/Emacs.app/Contents/Resources/lisp/mail/metamail
/Users/david.biesack/emacs/registry hides /Applications/Emacs.app/Contents/Resources/lisp/gnus/registry
/Users/david.biesack/emacs/re-builder hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/re-builder
/Users/david.biesack/emacs/elp hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/elp
/Users/david.biesack/emacs/trace hides /Applications/Emacs.app/Contents/Resources/lisp/emacs-lisp/trace

Features:
(shadow message-mode-hooks mime-open quoted-printable qp rmail smtpmail auth-source eieio eieio-core mailto ph mail-extr emacsbug message rfc822 mml mml-sec password-cache epg mailabbrev gmm-utils mailheader debug edebug ibuf-ext ibuffer jka-compr ef vc vc-dispatcher smerge-mode rect align sh-script executable dabbrev two-column iso-transl sort filemenu json-mode json-reformat json-snatcher js cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align pcmpl-unix diff misearch multi-isearch flycheck json map subr-x dash add-hook davids-macbook-pro.local server resizing color-themes color-theme sendmail reporter color-theme-autoloads my-emacs my-keys thing gnus-dired goto-addr password-keeper-mode sensitive-mode password-generator ps-ccrypt dired-efap dired-sort-map dired-a dired-aux iswitchb msb javadecomp autobrowse last-mod highlight-indentation my-yaml speedbar sb-image ezimage dframe yaml-mode disp-table sgml-ext flymake-files flymakemsg nxhtml-autostart nxhtml-autoload majmodpri vc-git diff-mode nxhtml-menu udev-rinari udev-ecb udev flymake css-color nxhtml-mode html-quote tidy-xhtml ediff-merg ediff-wind ediff-diff ediff-mult ediff-help ediff-init ediff-util ediff html-imenu imenu loadhist popcmp xhtml-help mlinks html-toc xml fupd html-pagetoc appmenu-fold appmenu mumamo sgml-mode edmacro kmacro ruby-mode smie rngalt rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc xmltok mumamo-cmirr cc-engine cc-vars cc-defs flyspell ispell html-upl html-site ourcomments-util cl-seq recentf tree-widget mm-url gnus gnus-ems nnheader gnus-util mail-utils org org-macro org-footnote org-pcomplete org-list org-faces org-entities org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint ob-core ob-eval org-compat org-macs org-loaddefs format-spec find-func cal-menu calendar cal-loaddefs ido bookmark apropos grep ffip gimpedit foldit noutline outline hideshow dired web-vcs derived cl-macs rx url-http tls gnutls url url-proxy url-privacy url-expand url-methods url-history mailcap url-auth url-cookie timezone url-util url-parse url-gw url-vars mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mm-util help-fns mail-prsvr cus-edit wid-edit compile cl web-autoload nxhtml-base easy-mmode ascii-conversion bg shell pcomplete comint ansi-color ring quickurl browse-url pp thingatpt remote-shell warnings advice paren midnight time desktop frameset cus-start cus-load finder-inf package epg-config seq byte-opt gv bytecomp byte-compile cl-extra help-mode easymenu cconv cl-loaddefs pcase cl-lib time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel ns-win ucs-normalize term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 1028590 151817)
 (symbols 48 48777 26)
 (miscs 40 98718 2882)
 (strings 32 117740 17792)
 (string-bytes 1 3962735)
 (vectors 16 66514)
 (vector-slots 8 1265641 28139)
 (floats 8 533 616)
 (intervals 56 25142 771)
 (buffers 976 87))

David Biesack | Vice President, API Platforms
David.Biesack <at> apiture.com | @davidbiesack <https://twitter.com/davidbiesack>




Disclaimer: The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify Apiture immediately by replying to this message and deleting it from your computer.

Message #8 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: David Biesack <David.Biesack <at> Apiture.com>
Cc: 36877 <at> debbugs.gnu.org
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Thu, 08 Aug 2019 20:47:31 -0400

David Biesack <David.Biesack <at> Apiture.com> writes:

> The list uniquified-managed sometimes starts to grow in an unmanaged manner (I suspect exponentially) after running
> Emacs for a few days. My work involves heavily editing many files named "openapi.yaml" in about 20 different
> directories, often switching git branches so I need to revisit (M-x revert-buffer) these files a lot.
>
> After a few days, I noticed opening new openapi.yaml files started to slow down dramatically - there would be several
> seconds of delay, then tens of seconds. I narrowed this to uniquify. In my buffers, the local variable
> uniquified-managed is more than 100,000 items long, with many duplicate items.

Looks like the only place where uniquified-managed is set to non-nil is
by uniquify-rationalize, so could you run with the advice below and post
the backtrace you get with it?

    (defconst bug-36877-dup-table (make-hash-table :test 'eq))
    (defun bug-36877-debug-fix-list-dups (fix-list)
      (clrhash bug-36877-dup-table)
      (dolist (item fix-list)
        (let ((buf (uniquify-item-buffer item)))
          (if (gethash buf bug-36877-dup-table)
              (progn (debug nil "Adding duplicate to uniquify list!" item)
                     ;; Just once.
                     (advice-remove 'uniquify-rationalize
                                    #'bug-36877-debug-fix-list-dups))
            (puthash buf item bug-36877-dup-table)))))
    (advice-add 'uniquify-rationalize :before #'bug-36877-debug-fix-list-dups)

Message #11 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: David Biesack <David.Biesack <at> Apiture.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Fri, 9 Aug 2019 15:58:22 +0000

Debugger entered: ("Adding duplicate to uniquify list! (bug-36877)" [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"])
  (progn (debug nil "Adding duplicate to uniquify list! (bug-36877)" item) (advice-remove (quote uniquify-rationalize) (function bug-36877-debug-fix-list-dups)))
  (if (gethash buf bug-36877-dup-table) (progn (debug nil "Adding duplicate to uniquify list! (bug-36877)" item) (advice-remove (quote uniquify-rationalize) (function bug-36877-debug-fix-list-dups))) (puthash buf item bug-36877-dup-table))
  (let ((buf (progn nil (or (and (memq (aref item 0) cl-struct-uniquify-item-tags)) (signal (quote wrong-type-argument) (list (quote uniquify-item) item))) (aref item 3)))) (if (gethash buf bug-36877-dup-table) (progn (debug nil "Adding duplicate to uniquify list! (bug-36877)" item) (advice-remove (quote uniquify-rationalize) (function bug-36877-debug-fix-list-dups))) (puthash buf item bug-36877-dup-table)))
  (while --dolist-tail-- (setq item (car --dolist-tail--)) (let ((buf (progn nil (or (and (memq ... cl-struct-uniquify-item-tags)) (signal (quote wrong-type-argument) (list ... item))) (aref item 3)))) (if (gethash buf bug-36877-dup-table) (progn (debug nil "Adding duplicate to uniquify list! (bug-36877)" item) (advice-remove (quote uniquify-rationalize) (function bug-36877-debug-fix-list-dups))) (puthash buf item bug-36877-dup-table))) (setq --dolist-tail-- (cdr --dolist-tail--)))
  (let ((--dolist-tail-- fix-list) item) (while --dolist-tail-- (setq item (car --dolist-tail--)) (let ((buf (progn nil (or (and ...) (signal ... ...)) (aref item 3)))) (if (gethash buf bug-36877-dup-table) (progn (debug nil "Adding duplicate to uniquify list! (bug-36877)" item) (advice-remove (quote uniquify-rationalize) (function bug-36877-debug-fix-list-dups))) (puthash buf item bug-36877-dup-table))) (setq --dolist-tail-- (cdr --dolist-tail--))))
  bug-36877-debug-fix-list-dups(([cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0" #<buffer index.html> nil] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"]))
  apply(bug-36877-debug-fix-list-dups ([cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0" #<buffer index.html> nil] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"]))
  uniquify-rationalize(([cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0" #<buffer index.html> nil] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"] [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"]))
  uniquify-rationalize-file-buffer-names("index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0/" #<buffer index.html>)
  uniquify--create-file-buffer-advice(#[257 "\301
!\211\302\230\203\f

David Biesack | Vice President, API Platforms
David.Biesack <at> apiture.com | @davidbiesack <https://twitter.com/davidbiesack>





On 8/8/19, 8:47 PM, "Noam Postavsky" <npostavs <at> gmail.com> wrote:

    David Biesack <David.Biesack <at> Apiture.com> writes:

    > The list uniquified-managed sometimes starts to grow in an unmanaged manner (I suspect exponentially) after running
    > Emacs for a few days. My work involves heavily editing many files named "openapi.yaml" in about 20 different
    > directories, often switching git branches so I need to revisit (M-x revert-buffer) these files a lot.
    >
    > After a few days, I noticed opening new openapi.yaml files started to slow down dramatically - there would be several
    > seconds of delay, then tens of seconds. I narrowed this to uniquify. In my buffers, the local variable
    > uniquified-managed is more than 100,000 items long, with many duplicate items.

    Looks like the only place where uniquified-managed is set to non-nil is
    by uniquify-rationalize, so could you run with the advice below and post
    the backtrace you get with it?

        (defconst bug-36877-dup-table (make-hash-table :test 'eq))
        (defun bug-36877-debug-fix-list-dups (fix-list)
          (clrhash bug-36877-dup-table)
          (dolist (item fix-list)
            (let ((buf (uniquify-item-buffer item)))
              (if (gethash buf bug-36877-dup-table)
                  (progn (debug nil "Adding duplicate to uniquify list!" item)
                         ;; Just once.
                         (advice-remove 'uniquify-rationalize
                                        #'bug-36877-debug-fix-list-dups))
                (puthash buf item bug-36877-dup-table)))))
        (advice-add 'uniquify-rationalize :before #'bug-36877-debug-fix-list-dups)



Disclaimer: The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify Apiture immediately by replying to this message and deleting it from your computer.

Message #14 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: David Biesack <David.Biesack <at> Apiture.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Thu, 15 Aug 2019 13:20:13 -0400

David Biesack <David.Biesack <at> Apiture.com> writes:

>   uniquify-rationalize-file-buffer-names("index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0/" #<buffer index.html>)
>   uniquify--create-file-buffer-advice(#[257 "\301!\211\302\230\203\f

Ah right, Emacs 25 still has that thing where it cuts off nthe pasted
text after NUL bytes.  For next time, saving the backtrace to a file and
attaching it should work better.

Looking at the fix-list in the backtrace, we have:

  [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0" #<buffer index.html> nil]
  { ;; repeated 5 times...
  [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"]
  [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"]
  [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"]
  [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"]
  [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"]
  }

I think that means the (dolist (buffer (buffer-list))...) somehow went
over #<buffer index.html<docs/apis/sample>> 5 times, or else that buffer
already had duplicates in its uniquify-managed list (i.e., this
backtrace triggerred "too late").

Did you add the advice in an ongoing session?  If yes, then it you
should clear out existing duplicates first, so we can catch a "fresh"
instance.  Evaluating the below should do:

    (let ((htable (make-hash-table :test 'equal)))
      (save-current-buffer
        (dolist (buf (buffer-list))
          (set-buffer buf)
          (when (cdr uniquify-managed)
            (clrhash htable)
            (dolist (item (cdr uniquify-managed))
              (puthash item t htable))
            (let ((managed nil))
              (maphash (lambda (k _v)
                         (push k managed))
                       htable)
              (push (car uniquify-managed) managed)
              (setq uniquify-managed managed))))))

If that backtrace was actually from a new session (that is, the advice
was added at the beginning of the session), then it means the problem is
coming from some other unexpected spot.

Message #17 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: David Biesack <David.Biesack <at> Apiture.com>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Fri, 16 Aug 2019 14:36:25 +0000

I'll restart Emacs and when I get a backtrace again I'll attach it as file.

David Biesack | Vice President, API Platforms
David.Biesack <at> apiture.com | @davidbiesack <https://twitter.com/davidbiesack>





On 8/15/19, 1:20 PM, "Noam Postavsky" <npostavs <at> gmail.com> wrote:

    David Biesack <David.Biesack <at> Apiture.com> writes:

    >   uniquify-rationalize-file-buffer-names("index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0/" #<buffer index.html>)
    >   uniquify--create-file-buffer-advice(#[257 "\301!\211\302\230\203\f

    Ah right, Emacs 25 still has that thing where it cuts off nthe pasted
    text after NUL bytes.  For next time, saving the backtrace to a file and
    attaching it should work better.

    Looking at the fix-list in the backtrace, we have:

      [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0" #<buffer index.html> nil]
      { ;; repeated 5 times...
      [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"]
      [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"]
      [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"]
      [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"]
      [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"]
      }

    I think that means the (dolist (buffer (buffer-list))...) somehow went
    over #<buffer index.html<docs/apis/sample>> 5 times, or else that buffer
    already had duplicates in its uniquify-managed list (i.e., this
    backtrace triggerred "too late").

    Did you add the advice in an ongoing session?  If yes, then it you
    should clear out existing duplicates first, so we can catch a "fresh"
    instance.  Evaluating the below should do:

        (let ((htable (make-hash-table :test 'equal)))
          (save-current-buffer
            (dolist (buf (buffer-list))
              (set-buffer buf)
              (when (cdr uniquify-managed)
                (clrhash htable)
                (dolist (item (cdr uniquify-managed))
                  (puthash item t htable))
                (let ((managed nil))
                  (maphash (lambda (k _v)
                             (push k managed))
                           htable)
                  (push (car uniquify-managed) managed)
                  (setq uniquify-managed managed))))))

    If that backtrace was actually from a new session (that is, the advice
    was added at the beginning of the session), then it means the problem is
    coming from some other unexpected spot.



Disclaimer: The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify Apiture immediately by replying to this message and deleting it from your computer.

Message #20 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: David Biesack <David.Biesack <at> Apiture.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Sun, 18 Aug 2019 13:51:08 -0400

[Message part 1 (text/plain, inline)]

Noam Postavsky <npostavs <at> gmail.com> writes:

> Looking at the fix-list in the backtrace, we have:
>
>   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0" #<buffer index.html> nil]
>   { ;; repeated 5 times...
>   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"]
>   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"]
>   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"]
>   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"]
>   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"]
>   }

I managed to hit this myself.  I'm not exactly sure what triggered it.
My problematic fix-list looks like this:

    (#s(uniquify-item :base "term-tests.el" :dirname "/home/npostavs/src/emacs/master/test/lisp" :buffer #<buffer term-tests.el> :proposed nil)
    #3=#s(uniquify-item :base #1="term-tests.el" :dirname "/home/npostavs/src/emacs/emacs-master/test/lisp" :buffer #<buffer emacs-master/term-tests.el> :proposed #1#)
    #4=#s(uniquify-item :base #2="term-tests.el" :dirname "/home/npostavs/src/emacs/emacs-26/test/lisp" :buffer #<buffer emacs-26/term-tests.el> :proposed #2#)
    #3# #4#)

I suspect the problem is happening earlier, so we'll need some other
test to catch it.

[backtrace.txt (text/plain, attachment)]

Message #23 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: David Biesack <David.Biesack <at> Apiture.com>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Tue, 20 Aug 2019 15:07:49 +0000

Attached is a backtrace.
Let me know if there is other advice I can add to help track this down.

David Biesack | Vice President, API Platforms
David.Biesack <at> apiture.com | @davidbiesack <https://twitter.com/davidbiesack>





On 8/18/19, 1:51 PM, "Noam Postavsky" <npostavs <at> gmail.com> wrote:

    Noam Postavsky <npostavs <at> gmail.com> writes:

    > Looking at the fix-list in the backtrace, we have:
    >
    >   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/workflow/v0.13.0" #<buffer index.html> nil]
    >   { ;; repeated 5 times...
    >   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample" #<buffer index.html<docs/apis/sample>> "index.html<docs/apis/sample>"]
    >   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/doc/apis/sample" #<buffer index.html<doc/apis/sample>> "index.html<doc/apis/sample>"]
    >   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/apis/sample/sample" #<buffer index.html<sample/sample>> "index.html<sample/sample>"]
    >   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com" #<buffer index.html<developer.apiture.com>> "index.html<developer.apiture.com>"]
    >   [cl-struct-uniquify-item "index.html" "/Users/david.biesack/dev/developer.apiture.com/_site/docs/concepts" #<buffer index.html<concepts>> "index.html<concepts>"]
    >   }

    I managed to hit this myself.  I'm not exactly sure what triggered it.
    My problematic fix-list looks like this:

        (#s(uniquify-item :base "term-tests.el" :dirname "/home/npostavs/src/emacs/master/test/lisp" :buffer #<buffer term-tests.el> :proposed nil)
        #3=#s(uniquify-item :base #1="term-tests.el" :dirname "/home/npostavs/src/emacs/emacs-master/test/lisp" :buffer #<buffer emacs-master/term-tests.el> :proposed #1#)
        #4=#s(uniquify-item :base #2="term-tests.el" :dirname "/home/npostavs/src/emacs/emacs-26/test/lisp" :buffer #<buffer emacs-26/term-tests.el> :proposed #2#)
        #3# #4#)

    I suspect the problem is happening earlier, so we'll need some other
    test to catch it.




Disclaimer: The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify Apiture immediately by replying to this message and deleting it from your computer.

Message #26 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: David Biesack <David.Biesack <at> Apiture.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Tue, 20 Aug 2019 22:47:42 -0400

[Message part 1 (text/plain, inline)]

David Biesack <David.Biesack <at> Apiture.com> writes:

> Attached is a backtrace.
> Let me know if there is other advice I can add to help track this down.

The attachment seems to be missing.  But anyway, I have a simple
reproducer now, so we don't need any more backtraces.  I found setting
uniquify-after-kill-buffer-p to nil was needed to reproduce this, do you
have that setting in your init?

[repro.el (text/plain, attachment)]

Message #29 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: David Biesack <David.Biesack <at> Apiture.com>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Wed, 21 Aug 2019 12:47:51 +0000

No, uniquify-after-kill-buffer-p is set to t in my session; no customization.

David Biesack | Vice President, API Platforms
David.Biesack <at> apiture.com | @davidbiesack <https://twitter.com/davidbiesack>





On 8/20/19, 10:47 PM, "Noam Postavsky" <npostavs <at> gmail.com> wrote:

    David Biesack <David.Biesack <at> Apiture.com> writes:

    > Attached is a backtrace.
    > Let me know if there is other advice I can add to help track this down.

    The attachment seems to be missing.  But anyway, I have a simple
    reproducer now, so we don't need any more backtraces.  I found setting
    uniquify-after-kill-buffer-p to nil was needed to reproduce this, do you
    have that setting in your init?




Disclaimer: The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify Apiture immediately by replying to this message and deleting it from your computer.

Message #32 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: David Biesack <David.Biesack <at> Apiture.com>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Wed, 25 Nov 2020 17:19:41 +0000

Note that this problem is still persisting for me, both in Emacs 26 and also in Emacs 27.1
which I just upgraded to.

I have implemented a workaround to the uniquify-managed list growing
Exponentially after using revert-buffer. See this patch

  *** /dev/fd/632020-11-25 12:18:39.000000000 -0500
--- /Users/david.biesack/emacs/uniqify.el2020-11-25 11:24:19.000000000 -0500
***************
*** 69,74 ****
--- 69,77 ----
  ;;   Stefan Monnier <monnier <at> cs.yale.edu> 17 Nov 2000
  ;; Algorithm and data structure changed to reduce consing with lots of buffers
  ;;   Francesco Potortì <pot <at> gnu.org> (ideas by rms and monnier) 2001-07-18
+ ;; Added workaround to uniqify-managed list growing exponentially
+ ;;   (underlying problem still exists tho)
+ ;;   DavidBiesack <at> gmail.com 25Nov20

  ;; Valuable feedback was provided by
  ;; Paul Smith <psmith <at> baynetworks.com>,
***************
*** 270,275 ****
--- 273,285 ----
  (defun uniquify-rationalize (fix-list)
    ;; Set up uniquify to re-rationalize after killing/renaming
    ;; if there is a conflict.
+
+   ;; When using revert-buffer, the uniquify-managed list grows exponentially;
+   ;; and there are many duplicates items in the list. So sort and remove dups first.
+   ;; (The real fix would be to find/repair whatever allows dups in the list,
+   ;; or use a hashtable instead.)
+   (setq fix-list (delete-dups (sort fix-list 'uniquify-item-greaterp)))
+
    (dolist (item fix-list)
      (with-current-buffer (uniquify-item-buffer item)
        ;; Refresh the dirnames and proposed names.

David Biesack | Vice President, API Platforms
David.Biesack <at> apiture.com | @davidbiesack <https://twitter.com/davidbiesack>




On 8/21/19, 8:47 AM, "David Biesack" <David.Biesack <at> Apiture.com> wrote:

    No, uniquify-after-kill-buffer-p is set to t in my session; no customization.

    David Biesack | Vice President, API Platforms
    David.Biesack <at> apiture.com | @davidbiesack <https://twitter.com/davidbiesack>





    On 8/20/19, 10:47 PM, "Noam Postavsky" <npostavs <at> gmail.com> wrote:

        David Biesack <David.Biesack <at> Apiture.com> writes:

        > Attached is a backtrace.
        > Let me know if there is other advice I can add to help track this down.

        The attachment seems to be missing.  But anyway, I have a simple
        reproducer now, so we don't need any more backtraces.  I found setting
        uniquify-after-kill-buffer-p to nil was needed to reproduce this, do you
        have that setting in your init?





Disclaimer: The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify Apiture immediately by replying to this message and deleting it from your computer.

Message #37 received at 36877 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: David Biesack <David.Biesack <at> Apiture.com>
Cc: "36877 <at> debbugs.gnu.org" <36877 <at> debbugs.gnu.org>,
 Noam Postavsky <npostavs <at> gmail.com>
Subject: Re: bug#36877: 25.3; uniquify renaway uniquify-managed list
Date: Wed, 12 May 2021 17:37:05 +0200

David Biesack <David.Biesack <at> Apiture.com> writes:

> +   ;; When using revert-buffer, the uniquify-managed list grows exponentially;
> +   ;; and there are many duplicates items in the list. So sort and remove dups first.
> +   ;; (The real fix would be to find/repair whatever allows dups in the list

Thanks; this should now be fixed in Emacs 28 by doing just that -- not
allowing duplicates onto the list in the first place.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.