GNU bug report logs - #13860
24.2; dir-locals-directory-cache unreliable in one rare case

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 3 Mar 2013 11:01:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 03 06:01:41 2013
Received: from localhost ([127.0.0.1]:56559 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1UC6fj-00042U-Od
	for submit <at> debbugs.gnu.org; Sun, 03 Mar 2013 06:01:41 -0500
Received: from eggs.gnu.org ([208.118.235.92]:53255)
	by debbugs.gnu.org with esmtp (Exim 4.72)
	(envelope-from <johanclaesson@HIDDEN>) id 1UC6ff-00042F-OW
	for submit <at> debbugs.gnu.org; Sun, 03 Mar 2013 06:01:38 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <johanclaesson@HIDDEN>) id 1UC6fQ-0004cF-53
	for submit <at> debbugs.gnu.org; Sun, 03 Mar 2013 06:01:22 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-101.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE,
	USER_IN_WHITELIST autolearn=unavailable version=3.3.2
Received: from lists.gnu.org ([208.118.235.17]:46516)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <johanclaesson@HIDDEN>) id 1UC6fQ-0004cB-28
	for submit <at> debbugs.gnu.org; Sun, 03 Mar 2013 06:01:20 -0500
Received: from eggs.gnu.org ([208.118.235.92]:36377)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <johanclaesson@HIDDEN>) id 1UC6fN-0000x3-PK
	for bug-gnu-emacs@HIDDEN; Sun, 03 Mar 2013 06:01:20 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <johanclaesson@HIDDEN>) id 1UC6fK-0004bN-W3
	for bug-gnu-emacs@HIDDEN; Sun, 03 Mar 2013 06:01:17 -0500
Received: from smtprelay-b31.telenor.se ([213.150.131.20]:56489)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <johanclaesson@HIDDEN>) id 1UC6fK-0004b1-Db
	for bug-gnu-emacs@HIDDEN; Sun, 03 Mar 2013 06:01:14 -0500
Received: from ipb4.telenor.se (ipb4.telenor.se [195.54.127.167])
	by smtprelay-b31.telenor.se (Postfix) with ESMTP id 707DCEACB1
	for <bug-gnu-emacs@HIDDEN>; Sun,  3 Mar 2013 12:01:11 +0100 (CET)
X-SMTPAUTH-B2: [b157288]
X-SENDER-IP: [85.224.213.5]
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AtdHAEUsM1FV4NUFPGdsb2JhbABEhWe1boZzexcDAQEBATg0gnwTASckDyUBBCUKS4d5oFagERWNR4EbRYMoA5dkkleBcQ
X-IronPort-AV: E=Sophos;i="4.84,774,1355094000"; d="scan'208";a="211807810"
Received: from c-05d5e055.1542-1-64736c20.cust.bredbandsbolaget.se (HELO
	goblin) ([85.224.213.5])
	by ipb4.telenor.se with ESMTP; 03 Mar 2013 12:01:10 +0100
From: Johan Claesson <johanclaesson@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 24.2; dir-locals-directory-cache unreliable in one rare case
Date: Sun, 03 Mar 2013 12:01:14 +0100
Message-ID: <87lia4d9xh.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
	recognized.
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 208.118.235.17
X-Spam-Score: -6.1 (------)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -6.1 (------)

--=-=-=
Content-Type: text/plain



Hi,

Maybe i have misunderstood something but i find the cache
mechanism for directory local variables unreliable in one rare
case.  If first .dir-locals.el is read and a binding say foo = 1 is
inserted in the dir-locals-directory-cache.  And then foo = 2 is
written in the same second.  Now next time the dir locals the old
binding foo = 1 will be returned.  The cache entry is considered
valid when it should not.  

Of course it is unlikely that a user is typing so fast that they
trigger this :).  But it can be triggered from Lisp.  I found out
when playing with a ert test case that was tampering with a dir
local.  

Here is a recipe for this:

(defvar foo nil)
(put 'foo 'safe-local-variable 'numberp)
(let ((dir (make-temp-file "cache-test" t)))
  ;; Create .dir-locals.el with foo = 1.
  (with-temp-buffer
    (cd dir)
    (add-dir-local-variable nil 'foo 1)
    (save-buffer)
    (kill-buffer))
  ;; Read it into dir-locals-directory-cache
  ;; and save .dir-locals.el with foo = 2.
  (with-temp-buffer
    (cd dir)
    (make-local-variable 'foo)
    (hack-dir-local-variables-non-file-buffer)
    (add-dir-local-variable nil 'foo 2)
    (save-buffer)
    (kill-buffer))
  ;; Read it back again.
  ;; It should be 2 at this point but is 1 most of the times. 
  ;; (At the rare occasion that seconds increase
  ;; between the two add-dir-local-variable it returns 1).
  (with-temp-buffer
    (cd dir)
    (make-local-variable 'foo)
    (hack-dir-local-variables-non-file-buffer)
    (delete-file dir-locals-file)
    (delete-directory dir) 
    foo))



The current mechanism: The bindings found when reading a
.dir-locals.el file is put into dir-locals-directory-cache.  The
mtime of the file is stored in the cache entry.  Later a cache
entry is considered valid if it's mtime equals that of the
file-attribute mtime of corresponding .dir-locals.el file.

Suggestion: The mtime stored in the cache entry should be the
time when .dir-local.el was read into memory.  A cache entry
should be considered valid if mtime of cache entry is greater
than the file-attribute mtime of corresponding .dir-locals.el
file.  This will invalidate a couple of entries that should
ideally have been considered valid but it will not let through
any invalid ones.

Or it should not let through any invalid entries, but in fact it
seem to do so sometimes.  If the above test is looped it will
return the incorrect value one time in about 500 on my standard
Ubuntu GNU/Linux system.  I don't know why, maybe reading the mtime
from a file is not that reliable.  Anyway, to be on the safe side i
subtract one second from the mtime when storing it to the cache.
Now the test ran 50000 times with no trouble.

Attached is a patch for files.el with this modification.  If this
makes sense and is accepted i will update the patch with the
appropriate documentation changes as well.

Regards,

/Johan


--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline; filename=files.patch

--- a/files.el	2013-03-03 10:15:44.498835000 +0000
+++ b/files.el	2013-03-03 10:28:02.662835000 +0000
@@ -3610,14 +3610,14 @@ (defun dir-locals-find-file (file)
 		(let ((cached-file (expand-file-name dir-locals-file-name
 						     (car dir-elt))))
 		  (and (file-readable-p cached-file)
-		       (equal (nth 2 dir-elt)
-			      (nth 5 (file-attributes cached-file))))))
+		       (time-less-p (nth 5 (file-attributes cached-file))
+				    (nth 2 dir-elt)))))
 	    ;; This cache entry is OK.
-	    dir-elt
+	      dir-elt
 	  ;; This cache entry is invalid; clear it.
 	  (setq dir-locals-directory-cache
 		(delq dir-elt dir-locals-directory-cache))
-	  ;; Return the first existing dir-locals file.  Might be the same
+	  ;; Return the first existing dir-locals file.	 Might be the same
 	  ;; as dir-elt's, might not (eg latter might have been deleted).
 	  locals-file)
       ;; No cache entry.
@@ -3638,10 +3638,13 @@ (defun dir-locals-read-from-file (file)
 	    (let* ((dir-name (file-name-directory file))
 		   (class-name (intern dir-name))
 		   (variables (let ((read-circle nil))
-				(read (current-buffer)))))
+				(read (current-buffer))))
+		   (just-now (time-subtract (current-time)
+					    (list 0 1 0)))
+		   (mtime (with-decoded-time-value ((high low micro just-now))
+			    (encode-time-value high low micro 1))))
 	      (dir-locals-set-class-variables class-name variables)
-	      (dir-locals-set-directory-class dir-name class-name
-					      (nth 5 (file-attributes file)))
+	      (dir-locals-set-directory-class dir-name class-name mtime)
 	      class-name))
 	(error (message "Error reading dir-locals: %S" err) nil)))))
 

--=-=-=
Content-Type: text/plain




In GNU Emacs 24.2.2 (i686-pc-linux-gnu, GTK+ Version 2.24.10)
 of 2013-01-12 on goblin
Windowing system distributor `The X.Org Foundation', version 11.0.11203000
Configured using:
 `configure '--prefix=/home/jcl/usr' '--without-toolkit-scroll-bars'
 '-C' '--disable-maintainer-mode' '--without-compress-info''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Fundamental

Minor modes in effect:
  auto-fill-function: do-auto-fill
  diff-auto-refine-mode: t
  shell-dirtrack-mode: t
  global-cwarn-mode: t
  ido-everywhere: t
  display-time-mode: t
  icomplete-mode: t
  minibuffer-depth-indicate-mode: t
  which-function-mode: t
  electric-layout-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<up> <up> M-f M-f C-h f <return> <down> <down> <down> 
<up> <up> <up> <up> <up> <up> <up> <down> <tab> <down> 
<tab> <down> <tab> <down> <tab> <down> <tab> <down> 
<tab> <C-tab> C-q C-SPC <C-up> <C-up> <C-tab> C-q C-q 
SPC SPC <return> C-h k C-c C-s C-c C-s i a a <C-tab> 
C-x C-f C-f <return> SPC <return> C-h v w h i <tab> 
t e s p <tab> m o <tab> <return> <C-tab> <M-next> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <down> <down> <down> 
<down> <down> <down> <home> <end> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <end> <up> b r <return> C-x C-s <C-tab> M-x 
e m a c s C-g <switch-frame> q p p p p p <return> q 
p <return> M-g C-s d i r C-s C-s C-s C-s C-s C-s C-s 
<home> C-s l o c a l C-s C-s <home> C-s c a c h e <down> 
<down> v k v k v k v k v k q <switch-frame> C-x C-s 
M-x e m a c s - b u g <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> r e p o r t - e m <tab> b <tab> - q <tab> 
<return> d i r SPC l o c s <backspace> a l s <return> 
<tab> <return> M-x <up> C-e <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <return>

Recent messages:
Saving file /home/jcl/.newsrc...
Wrote /home/jcl/.newsrc
Saving /home/jcl/.newsrc.eld...
Saving file /home/jcl/.newsrc.eld...
Wrote /home/jcl/.newsrc.eld
Saving /home/jcl/.newsrc.eld...done
Error during redisplay: (wrong-type-argument arrayp nil) [50 times]
Contacting host: debbugs.gnu.org:80
Error during redisplay: (wrong-type-argument arrayp nil) [9 times]
Reporting new bug!

Load-path shadows:
/home/jcl/elisp/elpa/flymake-0.4.13/flymake hides /home/jcl/usr/share/emacs/24.2/lisp/progmodes/flymake

Features:
(shadow emacsbug zone lisp-mnt flymake-autoloads org-datetree org-table
org-lparse esh-var esh-io esh-cmd esh-ext esh-proc esh-arg esh-groups
eshell esh-module esh-mode esh-util org-mks org-id ob-octave pcase
tree-widget slime-banner slime-enclosing-context tar-mode autoload
org-element finder-inf mailalias info-look jcl-sl-init nrepl arc-mode
archive-mode wesnoth-mode wesnoth-wml-data wesnoth-update calc
calc-loaddefs calc-macs speedbar sb-image ezimage dframe woman tex-mode
longlines rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse
rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode nxml-outln
nxml-rap nxml-util nxml-glyph nxml-enc xmltok make-mode python
cperl-mode tcl prolog align inf-haskell clojure-mode inf-lisp scheme
gdb-mi json gud asm-mode glasses elp image-dired cal-dst ind-util
gnus-fun w3m-form sort gnus-cite flow-fill mail-extr gnus-bcklg
gnus-async rx chistory desktop apropos etags nnrss xml mm-url url-cache
url-http url-gw url-auth tabify man dabbrev misearch multi-isearch
cl-specs edebug hippie-exp haskell-mode smiley jcl-replace jcl-ruby
jcl-w3m-init w3m-search jcl-games-init jcl-text-translator-init
text-translator text-translator-window text-translator-vars
text-translator-sites jcl-muse-init htmlize-hack htmlize muse-latex
muse-html muse-xml-common muse-colors cus-edit cus-start cus-load
muse-publish muse-project muse-protocols muse-regexps derived muse
muse-nested-tags muse-mode jcl-dictem-init dictem jcl-yaoddmuse-init
yaoddmuse-extension w3m doc-view jka-compr image-mode timezone w3m-hist
w3m-fb bookmark-w3m w3m-ems w3m-ccl ccl w3m-favicon w3m-image w3m-proc
w3m-util yaoddmuse url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-util url-parse url-vars skeleton sgml-mode
add-log gnus-dired vc-bzr vc-sccs vc-svn vc-cvs vc-rcs vc-dir vc
vc-dispatcher find-lisp vc-git image-file org-indent org-wl org-w3m
org-vm org-rmail org-mhe org-mew org-irc org-jsinfo org-infojs org-html
org-info org-gnus org-docview org-bibtex bibtex org-bbdb
jcl-renegade-goblin jcl-home-boot jcl-status autorevert network-stream
tls jcl-stumpwm jcl-slime slime-fancy slime-fontifying-fu
slime-package-fu slime-references slime-scratch slime-presentations
slime-fuzzy slime-fancy-inspector slime-c-p-c slime-editing-commands
slime-autodoc slime-parse slime-repl slime hyperspec browse-url
mule-util flyspell qp parse-time gnus-ml nndraft nnmh nnfolder nnml
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art
mm-uu mml2015 epg-config mm-view mml-smime smime dig nntp gnus-cache
jcl-goblin fuzzy-match jcl-load jcl-torrent text-translator-load
jcl-calendar-init appt jcl-calendar holidays hol-loaddefs jcl-emms-init
emms-playing-time emms-mode-line emms-cache emms-info-ogginfo
emms-info-mp3info emms-info later-do emms-playlist-mode emms-player-vlc
emms-player-mplayer emms-player-simple emms-source-playlist
emms-source-file emms-setup emms emms-compat jcl-gnus-init
org-import-icalendar icalendar diary-lib diary-loaddefs jcl-gnus
gnus-sum nnoo gnus-group gnus-undo nnmail mail-source gnus-start
gnus-spec gnus-int gnus-range gnus-win mailcap starttls smtpmail
sendmail message rfc822 mml mml-sec mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils
mailheader gnus gnus-ems nnheader mail-utils wid-edit jcl-org-init
org-latex org-export-latex org-beamer org-exp ob-exp org-exp-blocks
org-agenda footnote org ob-tangle ob-ref ob-lob ob-table org-footnote
org-src ob-comint ob-keys org-pcomplete org-list org-faces org-entities
org-version ob-emacs-lisp ob org-compat org-macs ob-eval org-loaddefs
cal-menu calendar cal-loaddefs jcl-midnite-init midnight
jcl-picpocket-init picpocket eldoc files-x edmacro cl-lib jcl-erc-init
jcl-file-cache-init jcl-grep-sbg sbg jcl-ido-init jcl-dired-init
jcl-dired dired-x dired-details wdired dired-aux jcl-register-init
jcl-command-subset-init jcl-generic-init generic-x jcl-erlang-init
jcl-ssit jcl-sbg jcl-erlang-log bookmark pp jcl-eel bindat jcl-sbg-ssit
erlang-eunit jcl-erlang distel-ie edb patmatch erl-service derl erlext
epmd net-fsm erl distel erlang jcl-safe-init jcl-abbrev-init
jcl-swedish-postfix quail help-mode jcl-ediff-init ediff-merg ediff-diff
ediff-wind ediff-help ediff-util ediff-mult ediff-init ediff
jcl-term-init jcl-term time-stamp ange-ftp jcl-face-init hl-line
jcl-elisp-init hi-lock jcl-template-init jcl-template jcl-file-cache ert
find-func ewoc debug filecache jcl-register jcl-grep-init jcl-grep
jcl-duff jcl-motion jcl-windows jcl-keys-init diff-mode term disp-table
ehelp kmacro tramp tramp-compat auth-source eieio assoc gnus-util
mm-util mail-prsvr password-cache shell pcomplete format-spec
tramp-loaddefs windmove jcl-keys jcl-modes-init jcl-rfc rfcview view
goto-addr proced table picture noutline outline easy-mmode inf-ruby
ruby-mode sh-script cwarn grep compile jcl-command-subset ido
jcl-global-init time winner thingatpt paren mic-paren printing ps-print
ps-def lpr icomplete ispell uniquify mb-depth whitespace ffap byte-opt
warnings bytecomp byte-compile cconv macroexp saveplace
bitlbee-autoloads cl-lib-autoloads dired-details-autoloads
eimp-autoloads el-mock-autoloads emms-autoloads erlang-autoloads
fuzzy-match-autoloads haskell-mode-autoloads htmlize-autoloads
inf-ruby-autoloads ipython-autoloads lua-mode-autoloads
mediawiki-autoloads mic-paren-autoloads muse-autoloads nrepl-autoloads
clojure-mode-autoloads org-autoloads pysmell-autoloads python-autoloads
slime-autoloads tabbar-autoloads w3m-autoloads yaoddmuse-autoloads
package tabulated-list jcl-imenu which-func imenu jcl-misc newcomment
jcl-compile jcl-recommended-init jcl-site-start fpl electric cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs jpt-mode erlang-start clearcase tq reporter executable dired
comint regexp-opt ansi-color ring info easymenu jcl-bugfix
jcl-times-init advice help-fns advice-preload jcl-times jcl-util server
jcl-load-path cl time-date tooltip ediff-hook vc-hooks lisp-float-type
mwheel x-win x-dnd tool-bar dnd fontset image fringe lisp-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs
button faces cus-face files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

--=-=-=--