GNU bug report logs - #14776
24.3.50; [PATCH] parse-time-string performance

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 14776 in the body.
You can then email your comments to 14776 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3.50; [PATCH] parse-time-string performance
Date: Wed, 03 Jul 2013 03:21:09 +0200

[Message part 1 (text/plain, inline)]

I think parse-time-string's performance can be increased significantly
by the attached patch, which replaces `parse-time-string-chars' with
something more sensible.  This function gets called at least once for
every character of the date being parsed, e.g. ~100,000 times in a 3000
article gnus buffer.

I made a comparisons with 3000 dates extracted from mail articles like
this, which seems to suggest some improvement.  I hope that it is also
correct.

(progn
  (garbage-collect)
  (benchmark 100
             '(dolist (d dates)
                (parse-time-string d))))

status quo: 41.547726s (5.333477s in 28 GCs)
patched   : 10.132069s (0.389833s in 2 GCs)

-ap

[parse-date.diff.gz (text/x-diff, attachment)]

[Message part 3 (text/plain, inline)]

In GNU Emacs 24.3.50.2 (x86_64-unknown-linux-gnu, GTK+ Version 2.20.1)
 of 2013-05-23 on luca
Bzr revision: 112687 monnier <at> iro.umontreal.ca-20130523132345-1mk6eghoinse7o4k
Windowing system distributor `The X.Org Foundation', version 11.0.10707000
System Description:	Debian GNU/Linux 6.0.7 (squeeze)

Important settings:
  value of $LC_COLLATE: C
  value of $LC_MESSAGES: C
  value of $LANG: de_DE.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  diff-auto-refine-mode: t
  desktop-save-mode: t
  workgroups-mode: t
  ispell-track-input-method: t
  recentf-mode: t
  show-paren-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  window-numbering-mode: t
  shell-dirtrack-mode: t
  scroll-other-window-mode: t
  savehist-mode: t
  TeX-PDF-mode: t
  ekey-mode: t
  winner-mode: t
  eldoc-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-/ C-/ C-/ M-m C-y C-M-k C-u C-c C-c <switch-frame> 
C-M-a C-M-f C-x C-e <switch-frame> C-u C-c C-c <switch-frame> 
C-x 0 C-x C-e C-x o C-x b n e w - g <return> <switch-frame> 
<switch-frame> <switch-frame> C-, w d i f f <return> 
M-x d i f f - b u f f <return> <return> C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-x C-s C-g C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ 
C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-/ C-g 
C-/ C-/ C-/ C-/ C-/ C-/ M-x M-x <return> <return> C-x 
o C-v C-l C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p <switch-frame> <switch-frame> 
C-x o C-x 1 C-x m C-x k <return> M-x r e p o r t - 
e m a <tab> b u g <return>

Recent messages:
Redo! [2 times]
Undo! [2 times]
No further undo information [44 times]
Quit
Redo! [6 times]
View mode: type C-h for help, h for commands, q to quit.
End of buffer [3 times]
Beginning of buffer [10 times]
Auto-saving...done
Source file `/home/politza/src/emacs/trunk/lisp/gnus/nnir.el' newer than byte-compiled file

Load-path shadows:
/home/politza/.emacs.d/elpa/yasnippet-0.8.0/dropdown-list hides /home/politza/.emacs.d/plugins/yasnippet-0.6.1c/dropdown-list
/home/politza/.emacs.d/elpa/yasnippet-0.8.0/yasnippet hides /home/politza/.emacs.d/plugins/yasnippet-0.6.1c/yasnippet
/home/politza/.emacs.d/plugins/tblc hides /home/politza/.emacs.d/plugins/tblc/tblc
/home/politza/.emacs.d/plugins/haskell-mode/haskell-cabal hides /home/politza/.emacs.d/plugins/haskell/haskell-cabal
/home/politza/.emacs.d/plugins/haskell-mode/haskell-decl-scan hides /home/politza/.emacs.d/plugins/haskell/haskell-decl-scan
/home/politza/.emacs.d/plugins/haskell-mode/haskell-doc hides /home/politza/.emacs.d/plugins/haskell/haskell-doc
/home/politza/.emacs.d/plugins/haskell-mode/ghc-core hides /home/politza/.emacs.d/plugins/haskell/ghc-core
/home/politza/.emacs.d/plugins/haskell-mode/haskell-mode hides /home/politza/.emacs.d/plugins/haskell/haskell-mode
/home/politza/.emacs.d/plugins/haskell-mode/haskell-c hides /home/politza/.emacs.d/plugins/haskell/haskell-c
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indentation hides /home/politza/.emacs.d/plugins/haskell/haskell-indentation
/home/politza/.emacs.d/plugins/haskell-mode/haskell-site-file hides /home/politza/.emacs.d/plugins/haskell/haskell-site-file
/home/politza/.emacs.d/plugins/haskell-mode/haskell-ghci hides /home/politza/.emacs.d/plugins/haskell/haskell-ghci
/home/politza/.emacs.d/plugins/haskell-mode/inf-haskell hides /home/politza/.emacs.d/plugins/haskell/inf-haskell
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indent hides /home/politza/.emacs.d/plugins/haskell/haskell-indent
/home/politza/.emacs.d/plugins/haskell-mode/haskell-hugs hides /home/politza/.emacs.d/plugins/haskell/haskell-hugs
/home/politza/.emacs.d/plugins/haskell-mode/haskell-font-lock hides /home/politza/.emacs.d/plugins/haskell/haskell-font-lock
/home/politza/.emacs.d/plugins/haskell-mode/haskell-simple-indent hides /home/politza/.emacs.d/plugins/haskell/haskell-simple-indent
/home/politza/.emacs.d/plugins/jedi/scratch hides /home/politza/.emacs.d/plugins/ewm/scratch
/home/politza/.emacs.d/elpa/company-0.6.8/.dir-locals hides /home/politza/.emacs.d/plugins/el-get/.dir-locals
/home/politza/.emacs.d/plugins/imenu hides /home/politza/src/emacs/trunk/lisp/imenu
/home/politza/.emacs.d/plugins/term hides /home/politza/src/emacs/trunk/lisp/term
/home/politza/.emacs.d/plugins/python/python/python hides /home/politza/src/emacs/trunk/lisp/progmodes/python
/home/politza/src/emacs/trunk/lisp/net/shr hides /home/politza/src/emacs/trunk/lisp/gnus/shr
/home/politza/.emacs.d/elpa/company-0.6.8/.dir-locals hides /home/politza/src/emacs/trunk/lisp/gnus/.dir-locals
/home/politza/src/emacs/trunk/lisp/net/shr-color hides /home/politza/src/emacs/trunk/lisp/gnus/shr-color
/home/politza/src/emacs/trunk/lisp/net/xesam hides /home/politza/src/emacs/trunk/lisp/obsolete/xesam
/home/politza/.emacs.d/plugins/python/python/sym-comp hides /home/politza/src/emacs/trunk/lisp/obsolete/sym-comp
/home/politza/.emacs.d/plugins/matlab/matlab hides /usr/share/emacs-snapshot/site-lisp/emacs-goodies-el/matlab
/home/politza/.emacs.d/plugins/boxquote hides /usr/share/emacs-snapshot/site-lisp/emacs-goodies-el/boxquote
/home/politza/.emacs.d/plugins/bm hides /usr/share/emacs-snapshot/site-lisp/emacs-goodies-el/bm
/home/politza/.emacs.d/plugins/haskell-mode/haskell-decl-scan hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-decl-scan
/home/politza/.emacs.d/plugins/haskell-mode/haskell-c hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-c
/home/politza/.emacs.d/plugins/haskell-mode/haskell-ghci hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-ghci
/home/politza/.emacs.d/plugins/haskell-mode/haskell-doc hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-doc
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indent hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-indent
/home/politza/.emacs.d/plugins/haskell-mode/haskell-mode hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-mode
/home/politza/.emacs.d/plugins/haskell-mode/haskell-hugs hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-hugs
/home/politza/.emacs.d/plugins/haskell-mode/haskell-site-file hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-site-file
/home/politza/.emacs.d/plugins/haskell-mode/haskell-cabal hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-cabal
/home/politza/.emacs.d/plugins/haskell-mode/inf-haskell hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/inf-haskell
/home/politza/.emacs.d/plugins/haskell-mode/haskell-font-lock hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-font-lock
/home/politza/.emacs.d/plugins/haskell-mode/haskell-simple-indent hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-simple-indent
/home/politza/.emacs.d/plugins/haskell-mode/haskell-indentation hides /usr/share/emacs-snapshot/site-lisp/haskell-mode/haskell-indentation
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-links hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-links
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-render hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-render
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-occur hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-occur
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-history hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-history
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-tools hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-tools
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-outline hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-outline
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-misc hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-misc
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-info hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-info
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-util hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-util
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-annot hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-annot
/home/politza/.emacs.d/elpa/pdf-tools-0.2alpha1/pdf-isearch hides /home/politza/.emacs.d/working/pdf-tools/lisp/pdf-isearch

Features:
(shadow emacsbug bbdb-message sendmail nnir diff dired-aux whitespace
diff-mode vc ibuf-ext imenu-outline ispell quail cc-langs cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs find-dired debug shr-color color mm-archive misearch
multi-isearch reposition apropos calc-arith calc-misc calccomp calc-aent
calc-alg calc-units calc-ext calc-menu calc calc-loaddefs calc-macs
benchmark sort smiley gnus-cite mail-extr gnus-bcklg shr gnus-async qp
gnus-ml disp-table elp nndraft nnmh utf-7 gnutls network-stream starttls
nnfolder bbdb-gnus nnnil gnus-agent gnus-srvr gnus-score score-mode
nnvirtual gnus-msg gnus-art mm-uu mml2015 epg-config mm-view mml-smime
smime dig mailcap nntp gnus-cache hi-lock tex-buf reftex-dcr reftex-auc
font-latex vc-git pdftk-outline vc-bzr dired-eshell vc-dispatcher vc-svn
emacs-customizations nogroup-customizations wp-customizations
view-customizations tex-customizations reftex-customizations
reftex-miscellaneous-configurations-customizations
reftex-label-support-customizations
reftex-referencing-labels-customizations
reftex-defining-label-environments-customizations AUCTeX-customizations
preview-customizations preview-latex-customizations
preview-appearance-customizations TeX-parse-customizations
TeX-file-customizations TeX-command-customizations
TeX-view-customizations LaTeX-customizations LaTeX-macro-customizations
LaTeX-math-customizations LaTeX-indentation-customizations
table-customizations table-hooks-customizations outlines-customizations
programming-customizations tools-customizations vc-customizations
log-edit-customizations semantic-customizations makefile-customizations
etags-customizations ediff-customizations diff-mode-customizations
languages-customizations elpy-customizations matlab-customizations
sh-customizations python-customizations rx haskell-customizations
c-customizations asm-customizations multimedia-customizations
image-customizations help-customizations ekey-customizations
info-lookup-customizations info-customizations customize-customizations
custom-buffer-customizations apropos-customizations files-customizations
uniquify-customizations uniquify sunrise-customizations
recentf-customizations find-file-customizations backup-customizations
faces-customizations highlight-symbol-customizations
font-lock-customizations hi-lock-customizations facemenu-customizations
external-customizations server-customizations processes-customizations
shell-customizations proced-customizations gud-customizations
tooltip-customizations grep-customizations compilation-customizations
next-error-customizations comint-customizations SQL-customizations
man-customizations environment-customizations xterm-customizations
windows-customizations winner-customizations minibuffer-customizations
savehist-customizations completion-spelling lib-string
menu-customizations keyboard-customizations chistory-customizations
initialization-customizations frames-customizations
ediff-window-customizations desktop-customizations desktop
dired-customizations dired-x-customizations dired-x
dired-details-customizations editing-customizations
yasnippet-customizations paragraphs-customizations
matching-customizations paren-matching-customizations
paren-showing-customizations isearch-customizations
bookmark-customizations killing-customizations indent-customizations
fill-customizations emulations-customizations
editing-basics-customizations development-customizations
lisp-customizations re-builder-customizations
inferior-lisp-customizations ielm-customizations ert-customizations
edebug-customizations bytecomp-customizations advice-customizations
internal-customizations alloc-customizations extensions-customizations
eldoc-customizations cust-print-customizations data-customizations
save-place-customizations convenience-customizations
diminish-customizations diminish iedit-customizations
imenu-tree-customizations tags-tree-customizations
company-customizations workgroups-customizations workgroups bookmark pp
window-numbering-customizations pabbrev-customizations
kmacro-customizations imenu-customizations ibuffer-customizations
ibuf-macs hl-line-customizations hippie-expand-customizations
file-cache-customizations ffap-customizations completion-customizations
iswitchb-customizations browse-kill-ring-customizations
auto-revert-customizations auto-insert-customizations
Buffer-menu-customizations comm-customizations tramp-customizations
browse-url-customizations applications-customizations
mediawiki-customizations w3m-customizations package-customizations
mail-customizations bbdb-customizations bbdb-sendmail-customizations
bbdb-mua-customizations bbdb-mua bbdb-com crm bbdb
smtpmail-customizations sendmail-customizations gnus-customizations
nnmail-customizations gnus-summary-customizations
gnus-thread-customizations gnus-summary-various-customizations
gnus-summary-sort-customizations gnus-summary-marks-customizations
gnus-summary-maneuvering-customizations
gnus-summary-format-customizations gnus-summary-exit-customizations
gnus-sum gnus-group gnus-undo gnus-start gnus-spec gnus-win
gnus-start-customizations gnus-server-customizations
gnus-message-customizations message-customizations
message-various-customizations message-buffers-customizations
gnus-group-customizations gnus-group-visual-customizations
gnus-nnimap-format nnimap nnmail gnus-int mail-source message rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader parse-time tls utf7 netrc
nnoo gnus gnus-ems nnheader mail-utils gnus-files-customizations
gnus-newsrc-customizations gnus-exit-customizations
gnus-article-customizations gnus-article-hiding-customizations
ispell-customizations eshell-customizations eshell-module-customizations
eshell-smart-customizations eshell-hist-customizations
eshell-mode-customizations edebug doc-view-customizations
pdf-tools-customizations pdf-annot-customizations
pdf-links-customizations pdf-isearch-customizations pdf-annot pdf-occur
pdf-history pdf-outline pdf-links pdf-isearch pdf-misc imenu pdf-attach
pdf-info tq pdf-render pdf-tools pdf-util gnus-range tablist
tablist-filter pcase semantic/wisent/comp semantic/wisent
semantic/wisent/wisent semantic/util-modes semantic/util semantic
semantic/tag semantic/lex semantic/fw mode-local cedet doc-view
jka-compr image-mode calendar-customizations org-customizations
org-structure-customizations org-plain-lists-customizations
org-edit-structure-customizations org-startup-customizations
org-link-customizations org-latex-customizations
org-appearance-customizations holidays-customizations
calculator-customizations calc-customizations server recentf tree-widget
.autoload paren yasnippet dropdown-list help-mode window-numbering w3m
browse-url timezone w3m-hist w3m-e23 w3m-ccl ccl w3m-fsf w3m-favicon
w3m-image w3m-proc w3m-util view tramp tramp-compat tramp-loaddefs
trampver shell track-last-window scroll-other-window saveplace savehist
reftex reftex-vars pabbrev org ob-tangle ob-ref ob-lob ob-table
org-footnote org-src ob-comint ob-keys org-pcomplete org-list org-faces
org-entities noutline outline org-version ob-emacs-lisp ob org-compat
org-macs ob-eval org-loaddefs format-spec find-func cal-menu calendar
cal-loaddefs lib-edit lib-window lib-isearch lib-buffer reveal iswitchb
lib-basic lib-lispext latex easy-mmode tex-style tex dbus xml tex-site
auto-loads info-look info ibuffer hippie-exp grep compile filecache
edit-minibuffer eldoc-eval pcomplete esh-var esh-io esh-cmd esh-opt
esh-ext esh-proc esh-arg esh-groups eshell esh-module esh-mode esh-util
ekey assoc dired-details+ warnings dired dired-details
cool-prefix-bindings winner lib-kbd comint-history comint ansi-color
ring browse-kill-ring advice anticus edmacro kmacro derived cl-macs gv
ffap thingatpt url-parse auth-source eieio byte-opt bytecomp
byte-compile cconv gnus-util mm-util mail-prsvr password-cache url-vars
eldoc help-fns cus-edit easymenu cus-start cus-load wid-edit cl nadvice
cl-lib bbdb-loaddefs cl-format-autoloads company-autoloads
eldoc-eval-autoloads pdf-tools-autoloads yasnippet-autoloads package
time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind inotify
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: 24.3.50; [PATCH] parse-time-string performance
Date: Wed, 03 Jul 2013 03:27:31 +0200

[Message part 1 (text/plain, inline)]

Wrong patch, sorry.  This it is.

[parse-date-1.diff.gz (text/x-diff, attachment)]

Message #11 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Tue, 02 Jul 2013 23:32:02 -0400

Andreas Politz wrote:

> This function gets called at least once for every character of the
> date being parsed, e.g. ~100,000 times in a 3000 article gnus buffer.

Why is it called 30+ times per article? Naively, seems like a chance for
a ~ 30x speed-up, compared to the ~ 4x one you seem to achieve.

Message #14 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Tue, 02 Jul 2013 23:34:11 -0400

Glenn Morris wrote:

> Why is it called 30+ times per article? Naively, seems like a chance for
> a ~ 30x speed-up, compared to the ~ 4x one you seem to achieve.

Oh, parse-time-string-chars, not parse-time-string. Ignore me.

Message #17 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Thu, 04 Jul 2013 17:41:29 +0200

Glenn Morris <rgm <at> gnu.org> writes:

> Glenn Morris wrote:
>
>> Why is it called 30+ times per article? Naively, seems like a chance for
>> a ~ 30x speed-up, compared to the ~ 4x one you seem to achieve.
>
> Oh, parse-time-string-chars, not parse-time-string. Ignore me.

Yes, but I just noticed something different.

In my benchmark the unpatched function runs for 41.5s on 100*3000 calls,
which comes down to an average of 0.415s per 3000 dates (or articles in
a summary buffer displaying parsed dates).  This should actually not be
very noticeable.

But it was. It took gnus ~40s to display this amount of articles and
most of it was spend in parse-time-string.  After tweaking it (i.e. the
patch) it came down to ~10s.  Now, looking more closely at the above,
non-gnus numbers, this makes no sense. (Though it's appealing to jump to
a conclusion, because it appears to be the correct factor.)

I should have tested this with -Q.  The reason is a customized
gc-cons-percentage (0.2) and gc-cons-threshold (67108864).  Using the
default values shows almost no difference anymore in gnus, i.e. both
versions (stock and patched) need around ~10s.

I suppose there are various lessons to learn from this endeavour.

-ap

Message #20 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Thu, 04 Jul 2013 20:41:33 +0200

Andreas Politz <politza <at> hochschule-trier.de> writes:

> I should have tested this with -Q.  The reason is a customized
> gc-cons-percentage (0.2) and gc-cons-threshold (67108864).  Using the
> default values shows almost no difference anymore in gnus, i.e. both
> versions (stock and patched) need around ~10s.
>
> I suppose there are various lessons to learn from this endeavour.

:-)

But this reminds me: Why are we parsing data strings in Emacs Lisp at
all?  Doesn't libc (or gnulib) have date parsing functions?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #23 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Thu, 04 Jul 2013 21:47:58 +0200

Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> But this reminds me: Why are we parsing data strings in Emacs Lisp at
> all?  Doesn't libc (or gnulib) have date parsing functions?

Like strptime ?

Probably not worth it, since it doesn't seem to be a bottleneck,
contrary to what I was suggesting. Or are there other reasons ?

-ap

Message #26 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Thu, 04 Jul 2013 22:31:35 +0200

Andreas Politz <politza <at> hochschule-trier.de> writes:

> Like strptime ?

Or parse-datetime:

http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=blob_plain;f=lib/parse-datetime.y

> Probably not worth it, since it doesn't seem to be a bottleneck,
> contrary to what I was suggesting. Or are there other reasons ?

It's not much of a bottleneck, no:

(benchmark-elapse (dotimes (i 10000) (parse-time-string "Thu, 04 Jul 2013 20:06:00 +0200")))
=> 1.120856647

But sorting a summary buffer of 5K messages on Date (which some people
do) might get a performance boost.  But I was thinking that it might be
more likely that parse-datetime parses more date strings correctly than
the version in parse-time.el.  

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #29 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Thu, 04 Jul 2013 23:08:33 +0200

[Message part 1 (text/plain, inline)]

Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> It's not much of a bottleneck, no:
>
> (benchmark-elapse (dotimes (i 10000) (parse-time-string "Thu, 04 Jul 2013 20:06:00 +0200")))
> => 1.120856647

=> 0.215108395
;-O

But only with this:

[parse-time-rfc2822.el (application/emacs-lisp, inline)]

[Message part 3 (text/plain, inline)]

> But sorting a summary buffer of 5K messages on Date (which some people
> do) might get a performance boost.  But I was thinking that it might be
> more likely that parse-datetime parses more date strings correctly than
> the version in parse-time.el.  

It looks that way, i.e. parse-time-string is pretty simple compared to
that.  But most Date header I've seen popping up in my mail seem to
adhere to a strict rfc2822 format anyway, except for the occasional
non-strict timezone.

-ap

Message #32 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Thu, 04 Jul 2013 23:15:11 +0200

Andreas Politz <politza <at> hochschule-trier.de> writes:

>> (benchmark-elapse (dotimes (i 10000) (parse-time-string "Thu, 04 Jul
>> 2013 20:06:00 +0200")))
>> => 1.120856647
>
> => 0.215108395
> ;-O

Wow, that's a pretty impressive speed-up.  I do see one issue, though --
it doesn't parse 2-digit years?  That's a requirement.

Anyway, with this large speed-up, I think we should definitely consider
swapping out the current parse-time code with your code.  Could you
collect a bunch of Date headers from the wild and see whether the old
and new versions agree on them all?

> It looks that way, i.e. parse-time-string is pretty simple compared to
> that.  But most Date header I've seen popping up in my mail seem to
> adhere to a strict rfc2822 format anyway, except for the occasional
> non-strict timezone.

Yes, mailers have gotten a lot better about adhering to the RFC2822 date
format the last decade.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #35 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Fri, 05 Jul 2013 00:21:20 +0200

[Message part 1 (text/plain, inline)]

Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

> Wow, that's a pretty impressive speed-up.  I do see one issue, though --
> it doesn't parse 2-digit years?  That's a requirement.

There is also a missing downcase and the zone is incorrect...

I only considered the strict version.  Of course 2-digit years could be
incorporated, but I don't see this as a replacement anyway, rather an
addition or alternative: Try the rfc2822 format and fallback on the
current code:

[parse-time-string-rfc2822.diff (text/x-diff, inline)]

diff -wc --label /home/politza/src/emacs/trunk/lisp/calendar/parse-time.el --label \#\<buffer\ parse-time.el\<calendar\>\> /home/politza/src/emacs/trunk/lisp/calendar/parse-time.el /tmp/buffer-content-2968c1C
*** /home/politza/src/emacs/trunk/lisp/calendar/parse-time.el
--- #<buffer parse-time.el<calendar>>
***************
*** 176,186 ****
--- 176,240 ----
    "(slots predicate extractor...)")
  ;;;###autoload(put 'parse-time-rules 'risky-local-variable t)
  
+ (defconst parse-time-rfc2822-regex
+   (let* ((fws "[ \t\r\n]+")
+          (ofws "[ \t\r\n]*")
+          (day-of-week
+           (regexp-opt
+            (mapcar 'car parse-time-weekdays)))
+          (day "[0-9]\\{1,2\\}")
+          (month
+           (regexp-opt
+            (mapcar 'car parse-time-months)))
+          (year "[0-9]\\{4,\\}")
+          (hour "[0-9]\\{2\\}")
+          (minute hour)
+          (second hour)
+          (zone-hour (concat "[+-]" hour))
+          (zone-min hour)
+          ;; A rather non strict comment.
+          (cfws "(\\(?:.\\|\n\\)*)"))
+     (concat
+      (format "\\`%s\\(?:\\(%s\\),%s\\)?" ofws day-of-week ofws)
+      (format "\\(%s\\)%s\\(%s\\)%s\\(%s\\)%s"
+              day fws month fws year fws)
+      (format "\\(%s\\):\\(%s\\)\\(?::\\(%s\\)\\)?%s"
+              hour minute second fws)
+      (format "\\(%s\\)\\(%s\\)" zone-hour zone-min)
+      (format "%s\\(?:%s\\)?%s\\'" ofws cfws ofws)))
+   "A regex matching a strict rfc2822 date.")
+ 
+ (defun parse-time-string-rfc2822 (string)
+   "Parse the strict rfc2822 time-string STRING
+ 
+ Return either a list (SEC MIN HOUR DAY MON YEAR DOW DST TZ) or
+ nil, if STRING is not valid."
+   (setq string (downcase string))
+   (when (string-match parse-time-rfc2822-regex string)
+     (let ((dow (cdr (assoc (match-string 1 string)
+                            parse-time-weekdays)))
+           (day (string-to-number (match-string 2 string)))
+           (month (cdr (assoc (match-string 3 string)
+                              parse-time-months)))
+           (year (string-to-number (match-string 4 string)))
+           (hour (string-to-number (match-string 5 string)))
+           (min (string-to-number (match-string 6 string)))
+           (sec (string-to-number (or (match-string 7 string) "0")))
+           (zhour (string-to-number (match-string 8 string)))
+           (zmin (string-to-number (match-string 9 string))))
+       (let ((zone (+ (* (abs zhour) 3600)
+                      (* 60 zmin))))
+         (list sec min hour day month year dow
+               nil (if (>= zhour 0)
+                       zone
+                     (- zone)))))))
+ 
  ;;;###autoload
  (defun parse-time-string (string)
    "Parse the time-string STRING into (SEC MIN HOUR DAY MON YEAR DOW DST TZ).
  The values are identical to those of `decode-time', but any values that are
  unknown are returned as nil."
+   (or (parse-time-string-rfc2822 string)
        (let ((time (list nil nil nil nil nil nil nil nil nil))
              (temp (parse-time-tokenize (downcase string))))
          (while temp
***************
*** 216,222 ****
  				       (funcall this)))
  				 parse-time-val)))
  		  (rplaca (nthcdr (pop slots) time) new-val))))))))
!     time))
  
  (provide 'parse-time)
  
--- 270,276 ----
                                             (funcall this)))
                                       parse-time-val)))
                        (rplaca (nthcdr (pop slots) time) new-val))))))))
!         time)))
  
  (provide 'parse-time)
  

Diff finished.  Fri Jul  5 00:00:38 2013

[Message part 3 (text/plain, inline)]

> Could you collect a bunch of Date headers from the wild and see
> whether the old and new versions agree on them all?

These are 3000 dates from my inbox, all pretty standard:

[parse-time.ert (application/octet-stream, attachment)]

[Message part 5 (text/plain, inline)]

-ap

Message #38 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Sat, 06 Jul 2013 17:34:09 +0200

Andreas Politz <politza <at> hochschule-trier.de> writes:

> I only considered the strict version.  Of course 2-digit years could be
> incorporated, but I don't see this as a replacement anyway, rather an
> addition or alternative: Try the rfc2822 format and fallback on the
> current code:

I see no reason to keep both the old 10x slower version and the new fast
version.  The fast version could surely be made to parse all valid(ish)
RFC822 date headers?

I think there's a corpus of date headers to test with out there
somewhere, but I can't find it at the moment..

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #41 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Sat, 06 Jul 2013 17:50:14 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> I see no reason to keep both the old 10x slower version and the new fast
> version.  

Except for non-mail related packages/users.

> The fast version could surely be made to parse all valid(ish) RFC822
> date headers?  I think there's a corpus of date headers to test with
> out there somewhere, but I can't find it at the moment..

I'll see if I can find it.

-ap

Message #44 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Sat, 06 Jul 2013 17:56:08 +0200

Andreas Politz <politza <at> hochschule-trier.de> writes:

>> I see no reason to keep both the old 10x slower version and the new fast
>> version.  
>
> Except for non-mail related packages/users.

Well, the old function is designed for parsing RFC822 Date headers...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #47 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Sat, 06 Jul 2013 18:07:37 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Andreas Politz <politza <at> hochschule-trier.de> writes:
>
>>> I see no reason to keep both the old 10x slower version and the new fast
>>> version.  
>>
>> Except for non-mail related packages/users.
>
> Well, the old function is designed for parsing RFC822 Date headers...

That may be true, though there is no mentioning of it in the file.  But
that may not the only context in which it is used.  It accepts all kinds
of date formats, even without touching parse-time-rules.

-ap

Message #50 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Politz <politza <at> hochschule-trier.de>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Sat, 06 Jul 2013 19:47:30 +0200

[Message part 1 (text/plain, inline)]

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Well, the old function is designed for parsing RFC822 Date headers...

Also, people may be keeping ancient mail around, containing dates which
may be parsed by the current function.

Anyway here is an updated version, incorporating 2-digit years and
textual timezones.

[parse-time-rfc2822.el (application/emacs-lisp, inline)]

[Message part 3 (text/plain, inline)]

-ap

Message #53 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Lars Magne Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Mon, 08 Jul 2013 16:51:11 +0200

Andreas Politz <politza <at> hochschule-trier.de> writes:

> Also, people may be keeping ancient mail around, containing dates which
> may be parsed by the current function.

Yes, but perhaps your regexp-based parser could be extended to parse
those, too?  Ancient date formats looked quite RFC2822-ey, too.

> Anyway here is an updated version, incorporating 2-digit years and
> textual timezones.

Looks good.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Message #56 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Tue, 01 Mar 2016 12:05:34 +1100

Lars Magne Ingebrigtsen <larsi <at> gnus.org> writes:

>> Anyway here is an updated version, incorporating 2-digit years and
>> textual timezones.
>
> Looks good.

I did some further tests (after the version of parse-time-string on the
Emacs trunk had been ... fixed), and the regexp version is still 3x
faster (down from 10x).

It is, though, a stricter RFC2822 parser than the version we currently
have, so it fails on some of the test cases (dates without times, for
instance).  So I'm still not sure whether this is a suitable replacement
for parse-time-string.  And it seems odd to have two completely
different functions that almost parse the same thing...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #59 received at 14776 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Andreas Politz <politza <at> hochschule-trier.de>
Cc: 14776 <at> debbugs.gnu.org
Subject: Re: bug#14776: 24.3.50; [PATCH] parse-time-string performance
Date: Thu, 27 Jun 2019 13:12:19 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> It is, though, a stricter RFC2822 parser than the version we currently
> have, so it fails on some of the test cases (dates without times, for
> instance).  So I'm still not sure whether this is a suitable replacement
> for parse-time-string.  And it seems odd to have two completely
> different functions that almost parse the same thing...

So I don't think this is the right solution, and I'm closing this bug
report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.