Tim Landscheidt <tim@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.Tim Landscheidt <tim@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.Received: (at submit) by debbugs.gnu.org; 2 Jun 2017 07:05:40 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Fri Jun 02 03:05:40 2017 Received: from localhost ([127.0.0.1]:50358 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1dGgeS-0002HT-2P for submit <at> debbugs.gnu.org; Fri, 02 Jun 2017 03:05:40 -0400 Received: from eggs.gnu.org ([208.118.235.92]:42077) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <Justus-dev@HIDDEN>) id 1dGgES-0001eE-M8 for submit <at> debbugs.gnu.org; Fri, 02 Jun 2017 02:38:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <Justus-dev@HIDDEN>) id 1dGgEL-0006Ox-Uj for submit <at> debbugs.gnu.org; Fri, 02 Jun 2017 02:38:43 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:56730) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <Justus-dev@HIDDEN>) id 1dGgEL-0006Oa-S0 for submit <at> debbugs.gnu.org; Fri, 02 Jun 2017 02:38:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33032) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <Justus-dev@HIDDEN>) id 1dGgEJ-0004Aw-R3 for bug-gnu-emacs@HIDDEN; Fri, 02 Jun 2017 02:38:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <Justus-dev@HIDDEN>) id 1dGgEG-0006J2-LP for bug-gnu-emacs@HIDDEN; Fri, 02 Jun 2017 02:38:39 -0400 Received: from smtp.uibk.ac.at ([138.232.1.140]:58670) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <Justus-dev@HIDDEN>) id 1dGgEG-0006GJ-9m for bug-gnu-emacs@HIDDEN; Fri, 02 Jun 2017 02:38:36 -0400 Received: from jp (213-47-186-175.cable.dynamic.surfer.at [213.47.186.175]) (authenticated bits=0) by smtp.uibk.ac.at (8.14.4/8.14.4/F1) with ESMTP id v526cWcl016643 (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256 verify=NO) for <bug-gnu-emacs@HIDDEN>; Fri, 2 Jun 2017 08:38:33 +0200 From: Justus-dev@HIDDEN To: bug-gnu-emacs@HIDDEN Subject: 25.2; gnus: nnmaildir: article numbers keep growing -> nnmaildir--new-number slow Date: Fri, 02 Jun 2017 08:38:32 +0200 Message-ID: <87lgparj13.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 2.75 at uibk.ac.at X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Fri, 02 Jun 2017 03:05:39 -0400 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -5.0 (-----) In Gnus' nnmaildir backend, article numbers grow without bounds. Getting new news and moving articles into and out of those of my inbox groups (that see a lot of churn) took really long lately (up to several minutes). Profiling revealed that 99% of this time was spent inside nnmaildir--new-number, and indeed, the affected .nnmaildir/num/ directories contained thousands of hardlinks to article numbers. For each affected group, I got rid of these exessive article number hardlinks as follows: 1. move all messages from group to group.new 2. edit the parameters of group.new to match those of group 3. delete group 4. rename group.new to group (I'm not sure this procedure is guaranteed to be free of unintended side effects, but I have not observed any.) Since then, Gnus is snappy again. This is apparently a poorly-designed data structure / algorithm. If nnmaildir requires gapless article number sequences, why doesn't it just store the highest allocated number? Why does it waste time on sequential search (as I suspect)? If it does not rely on gapless article numbers, why doesn't it remove hardlinks corresponding to articles that no longer exist in this group? (After moving all messages out of an affected group, all 15000+ hardlinks were still present in .nnmaildir/num/.) In GNU Emacs 25.2.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.22.10) of 2017-04-22 built on juergen Windowing system distributor 'The X.Org Foundation', version 11.0.11903000 System Description: Arch Linux Configured using: 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong' CPPFLAGS=-D_FORTIFY_SOURCE=2 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro' Configured features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 MODULES Important settings: value of $LC_COLLATE: de_AT.UTF-8 value of $LC_MONETARY: de_AT.UTF-8 value of $LC_TIME: de_AT.UTF-8 value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: Org Minor modes in effect: diff-auto-refine-mode: t magit-auto-revert-mode: t global-git-commit-mode: t async-bytecomp-package-mode: t org-indent-mode: t show-paren-mode: t shell-dirtrack-mode: t pdf-occur-global-minor-mode: t tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Recent messages: Expiring articles...done No more unseen articles No more unread articles Expiring articles...done Saving /home/piater/.newsrc.eld... Saving file /home/piater/.newsrc.eld... Wrote /home/piater/.newsrc.eld Saving /home/piater/.newsrc.eld...done (No files need saving) Scanning for dabbrevs...done Load-path shadows: None found. Features: (shadow emacsbug latexenc org-colview pcmpl-unix pcmpl-gnu gnus-eform gnus-fun kmacro gnus-uu yenc profiler eimp ffap locate pdf-sync pdf-annot pdf-outline pdf-links pdf-history tabify image-file org-capture term ehelp ispell gnus-draft grep sh-script smie tramp-cache view woman man vc-git magit-blame magit-stash magit-bisect magit-remote magit-commit magit-sequence magit magit-apply magit-wip magit-log magit-diff smerge-mode diff-mode magit-core magit-autorevert autorevert filenotify magit-process magit-mode magit-popup magit-git crm magit-section magit-utils git-commit log-edit pcvs-util add-log with-editor async-bytecomp async tramp-sh dash cal-move diary-lib diary-loaddefs cal-iso org-indent org-rmail org-mhe org-irc org-info org-gnus org-docview doc-view org-bibtex bibtex org-bbdb org-w3m org-agenda mm-extern mm-url nxml-uchnm rng-xsd xsd-regexp rng-cmpct rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc xmltok dabbrev mailalias smtpmail sendmail dired-aux flow-fill gnus-dup misearch multi-isearch gnus-cite smiley shr-color color url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util url-parse url-vars qp shr dom subr-x mm-archive mail-extr gnus-async gnus-bcklg sort gnus-ml disp-table gnus-topic utf-7 network-stream nsm starttls imap epa-file epa derived nndraft nnmh nnmaildir gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig mailcap nntp gnus-cache nnir gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source tls gnutls utf7 netrc nnoo parse-time gnus-spec gnus-int gnus-range message idna rfc822 mml mml-sec epg mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils mailheader gnus-win mmm-mode mmm-univ mmm-class mmm-region mmm-auto mmm-vars mmm-utils mmm-compat paren gnus gnus-ems nnheader mail-utils gnus-dired ox-latex ox-icalendar ox-html ox-ascii ox-publish ox org-element org-id org org-macro org-footnote org-pcomplete org-list org-faces org-entities org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint ob-core ob-eval org-compat org-macs org-loaddefs cal-menu calendar cal-loaddefs ido ess-toolbar ess-mouse mouseme thingatpt browse-url ess-menu ess-swv ess-noweb ess-noweb-font-lock-mode ess-bugs-l essd-els ess-sas-d ess-sas-l ess-sas-a ess-sta-d ess-sta-l cc-vars cc-defs make-regexp ess-sp6-d ess-dde ess-sp3-d ess-julia julia-mode ess-r-d ess-r-syntax ess-r-completion ess-roxy essddr noutline outline easy-mmode hideshow ess-help ess-r-package ess-s-l ess ess-inf ess-tracebug tramp tramp-compat auth-source cl-seq gnus-util mm-util help-fns mail-prsvr password-cache tramp-loaddefs trampver ucs-normalize shell pcomplete ess-mode ess-noweb-mode ess-utils ess-generics cl ess-custom executable ess-compat ess-site pdf-occur ibuf-ext ibuffer tablist tablist-filter semantic/wisent/comp semantic/wisent semantic/wisent/wisent semantic/util-modes semantic/util semantic semantic/tag semantic/lex semantic/fw eieio eieio-core cl-macs mode-local find-func cedet dired pdf-isearch let-alist pdf-misc imenu pdf-tools compile comint ansi-color ring cus-edit cus-start cus-load wid-edit pdf-view bookmark pp jka-compr pdf-cache pdf-info tq pdf-util advice format-spec image-mode mozmail server finder-inf info package epg-config seq byte-opt gv bytecomp byte-compile cl-extra help-mode easymenu cconv cl-loaddefs pcase cl-lib time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote dbusbind inotify dynamic-setting system-font-setting font-render-setting move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 4492676 448493) (symbols 48 279807 1477) (miscs 40 5257 10111) (strings 32 755088 73077) (string-bytes 1 46054567) (vectors 16 304729) (vector-slots 8 4348424 117230) (floats 8 9603 2491) (intervals 56 295380 4749) (buffers 976 313))
Justus-dev@HIDDEN:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#27194; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.