GNU bug report logs - #36401
26.0.90; set-auto-mode uses case-insensitive match for magic-mode-alist

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36401 in the body.
You can then email your comments to 36401 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Benjamin Riefenstahl <Riefenstahl <at> mecom.de>
To: bug-gnu-emacs <at> gnu.org
Subject: 26.0.90;
 set-auto-mode uses case-insensitive match for magic-mode-alist
Date: Thu, 27 Jun 2019 11:30:47 +0200

My use case: I use "magic-mode-alist" to detect a binary file type for
which I have written a mode.  The magic at the beginning is "mTFf", so
it is simple ASCII, but unusually capitalized.  When I add this to
"magic-mode-alist", files and buffers that start with "mtff" are also
put in my binary file mode.

It turns out that "set-auto-mode" in lisp/files.el is the function that
uses magic-mode-alist, and it just calls "looking-at" to test the type.
Because "case-fold-search" is "t" by default, this catches files
starting with "mtff".

Is this intentional?  If so, the documentation for "magic-mode-alist"
should call this out, so users know they might need to work around it.
If not, I could propose a patch to fix it.

TIA, benny

-- 

In GNU Emacs 26.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.14.5)
 of 2017-10-18 built on riefenstahl-linux
Repository revision: b970a4a52a1c04afce4916ca2bf337e2b36d26c4
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description:	Debian GNU/Linux 8.11 (jessie)

Recent messages:
Mark set
Saving file /home/riefenstahl/.emacs.cfg/local.riefenstahl-linux.el...
Wrote /home/riefenstahl/.emacs.cfg/local.riefenstahl-linux.el
(No changes need to be saved)
Auto-saving...done
Saving file /home/riefenstahl/.emacs.cfg/local.riefenstahl-linux.el...
Wrote /home/riefenstahl/.emacs.cfg/local.riefenstahl-linux.el
(No changes need to be saved) [6 times]
Making completion list...
Type "q" in help window to restore its previous buffer.

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 LCMS2

Important settings:
  value of $LANG: C.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

Minor modes in effect:
  global-magit-file-mode: t
  magit-file-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  desktop-save-mode: t
  display-time-mode: t
  diff-auto-refine-mode: t
  delete-selection-mode: t
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
~/site-lisp/css-mode hides /usr/local/share/emacs/26.0.90/lisp/textmodes/css-mode

Features:
(shadow sort mail-extr emacsbug sendmail find-dired thingatpt eieio-opt
speedbar sb-image ezimage dframe find-func ruler-mode hl-line hexl pp
cl-print misearch multi-isearch help-fns arc-mode archive-mode
benny-mtff-mode magit-obsolete magit-blame magit-stash magit-bisect
magit-remote magit-commit magit-sequence magit-notes magit-worktree
magit-tag magit-merge magit-branch magit-reset magit-collab ghub-graphql
ghub url-http tls gnutls url-gw nsm url-auth url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap let-alist json magit-files magit-refs magit-status magit
magit-repos magit-apply magit-wip magit-log magit-diff smerge-mode
magit-core magit-autorevert autorevert filenotify magit-process
magit-margin magit-mode git-commit magit-git magit-section magit-utils
crm subr-x magit-popup log-edit message-x message rmc puny rfc822 mml
mml-sec epa epg gnus-util rmail rmail-loaddefs mm-decode mm-bodies
mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mailabbrev
mail-utils gmm-utils mailheader pcvs-util add-log with-editor cl-extra
help-mode async-bytecomp async dash elec-pair dired-aux make-mode
time-date map sh-script smie executable desktop frameset highline
ange-ftp generic-x cl autoinsert ps-print ps-print-loaddefs ps-def lpr
dired dired-loaddefs benny-x-clipboard disp-table mm-util mail-prsvr
time server protbuf solar cal-dst cal-islam cal-hebrew holidays
hol-loaddefs vc-git diff-mode easy-mmode diary-lib diary-loaddefs
cal-menu calendar cal-loaddefs which-func imenu jka-compr grep compile
delsel cus-start cus-load generic benny-calendar-cfg derived cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs filecache benny-file-cache tramp-sh docker-tramp tramp-cache
tramp tramp-compat tramp-loaddefs trampver ucs-normalize shell pcomplete
comint ansi-color ring parse-time format-spec advice benny-unicode
benny-tools browse-url autoload radix-tree lisp-mnt finder-inf gh-common
marshal eieio-compat rx info package easymenu epg-config url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache url-vars seq byte-opt gv bytecomp byte-compile cconv
cl-loaddefs cl-lib mule-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 418674 115456)
 (symbols 48 37479 2)
 (miscs 40 2593 3720)
 (strings 32 93366 4129)
 (string-bytes 1 2851917)
 (vectors 16 66469)
 (vector-slots 8 1822112 223144)
 (floats 8 700 564)
 (intervals 56 6177 3982)
 (buffers 992 36)
 (heap 1024 70067 2662))

Message #8 received at 36401 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Benjamin Riefenstahl <Riefenstahl <at> mecom.de>
Cc: 36401 <at> debbugs.gnu.org
Subject: Re: bug#36401: 26.0.90;
 set-auto-mode uses case-insensitive match for magic-mode-alist
Date: Sat, 06 Jul 2019 11:36:21 +0300

> From: Benjamin Riefenstahl <Riefenstahl <at> mecom.de>
> Date: Thu, 27 Jun 2019 11:30:47 +0200
> 
> It turns out that "set-auto-mode" in lisp/files.el is the function that
> uses magic-mode-alist, and it just calls "looking-at" to test the type.
> Because "case-fold-search" is "t" by default, this catches files
> starting with "mtff".
> 
> Is this intentional?  If so, the documentation for "magic-mode-alist"
> should call this out, so users know they might need to work around it.
> If not, I could propose a patch to fix it.

I think this is an omission.  Please do propose a patch to fix it.

Thanks.

Message #11 received at 36401 <at> debbugs.gnu.org (full text, mbox):

From: Benjamin Riefenstahl <b.riefenstahl <at> turtle-trading.net>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Benjamin Riefenstahl <Riefenstahl <at> mecom.de>, 36401 <at> debbugs.gnu.org
Subject: Re: bug#36401: 26.0.90;
 set-auto-mode uses case-insensitive match for magic-mode-alist
Date: Sun, 14 Jul 2019 17:23:57 +0200

[Message part 1 (text/plain, inline)]

Eli Zaretskii writes:
> I think this is an omission.  Please do propose a patch to fix it.

See attached.  This applies against master, but the conflict against
emacs-26 is also just minor.  Please advise if you want this
differently.

[0001-Make-REs-in-magic-fallback-mode-alist-case-sensitive.patch (text/x-diff, inline)]

From ea6ea194e8858789dce220e28ff33dcb5e313d30 Mon Sep 17 00:00:00 2001
From: Benjamin Riefenstahl <b.riefenstahl <at> turtle-trading.net>
Date: Sun, 14 Jul 2019 17:09:39 +0200
Subject: [PATCH] Make REs in magic-(fallback-)mode-alist case-sensitive.

These variables are used for well-defined file formats where relaxed
case matching is not wanted usually.

* lisp/files.el (magic-mode-alist, magic-fallback-mode-alist): Update
the doc string.
(set-auto-mode): Make looking-at for elements of magic-mode-alist and
magic-fallback-mode-alist use case-fold-search == nil.
* lisp/files.el (files-test-magic-mode-alist-re-baseline)
(files-test-magic-mode-alist-re-no-match)
(files-test-magic-mode-alist-re-case-diff): Add.
---
 lisp/files.el            | 18 ++++++++++--------
 test/lisp/files-tests.el | 27 +++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 8 deletions(-)

diff --git a/lisp/files.el b/lisp/files.el
index b2249bfcb4..d97f11c32e 100644
--- a/lisp/files.el
+++ b/lisp/files.el
@@ -2945,9 +2945,9 @@ magic-mode-alist
   "Alist of buffer beginnings vs. corresponding major mode functions.
 Each element looks like (REGEXP . FUNCTION) or (MATCH-FUNCTION . FUNCTION).
 After visiting a file, if REGEXP matches the text at the beginning of the
-buffer, or calling MATCH-FUNCTION returns non-nil, `normal-mode' will
-call FUNCTION rather than allowing `auto-mode-alist' to decide the buffer's
-major mode.
+buffer (respecting case), or calling MATCH-FUNCTION returns non-nil,
+`normal-mode' will call FUNCTION rather than allowing `auto-mode-alist' to
+decide the buffer's major mode.
 
 If FUNCTION is nil, then it is not called.  (That is a way of saying
 \"allow `auto-mode-alist' to decide for these files.\")")
@@ -2979,9 +2979,9 @@ magic-fallback-mode-alist
   "Like `magic-mode-alist' but has lower priority than `auto-mode-alist'.
 Each element looks like (REGEXP . FUNCTION) or (MATCH-FUNCTION . FUNCTION).
 After visiting a file, if REGEXP matches the text at the beginning of the
-buffer, or calling MATCH-FUNCTION returns non-nil, `normal-mode' will
-call FUNCTION, provided that `magic-mode-alist' and `auto-mode-alist'
-have not specified a mode for this file.
+buffer (respecting case), or calling MATCH-FUNCTION returns non-nil,
+`normal-mode' will call FUNCTION, provided that `magic-mode-alist' and
+`auto-mode-alist' have not specified a mode for this file.
 
 If FUNCTION is nil, then it is not called.")
 (put 'magic-fallback-mode-alist 'risky-local-variable t)
@@ -3098,7 +3098,8 @@ set-auto-mode
                              ((functionp re)
                               (funcall re))
                              ((stringp re)
-                              (looking-at re))
+                              (let ((case-fold-search nil))
+                                (looking-at re)))
                              (t
                               (error
                                "Problem in magic-mode-alist with element %s"
@@ -3159,7 +3160,8 @@ set-auto-mode
                                            ((functionp re)
                                             (funcall re))
                                            ((stringp re)
-                                            (looking-at re))
+                                            (let ((case-fold-search nil))
+                                              (looking-at re)))
                                            (t
                                             (error
                                              "Problem with magic-fallback-mode-alist element: %s"
diff --git a/test/lisp/files-tests.el b/test/lisp/files-tests.el
index aa5dbe7acf..df2c3f47ae 100644
--- a/test/lisp/files-tests.el
+++ b/test/lisp/files-tests.el
@@ -1282,5 +1282,32 @@ files-tests-file-attributes-equal
   (should (equal (file-size-human-readable 10000 'si " " "bit") "10 kbit"))
   (should (equal (file-size-human-readable 10000 'iec " " "bit") "9.8 Kibit")))
 
+(ert-deftest files-test-magic-mode-alist-re-baseline ()
+  "Test magic-mode-alist with RE, expected behaviour for match."
+  (let ((magic-mode-alist '(("my-tag" . text-mode))))
+    (with-temp-buffer
+      (insert "my-tag")
+      (normal-mode)
+      (should (eq major-mode 'text-mode)))))
+
+(ert-deftest files-test-magic-mode-alist-re-no-match ()
+  "Test magic-mode-alist with RE, expected behaviour for no match."
+  (let ((magic-mode-alist '(("my-tag" . text-mode))))
+    (with-temp-buffer
+      (insert "not-my-tag")
+      (normal-mode)
+      (should (not (eq major-mode 'text-mode))))))
+
+(ert-deftest files-test-magic-mode-alist-re-case-diff ()
+  "Test that regexps in magic-mode-alist are case-sensitive.
+See <https://debbugs.gnu.org/36401>."
+  (let ((case-fold-search t)
+        (magic-mode-alist '(("my-tag" . text-mode))))
+    (with-temp-buffer
+      (goto-char (point-min))
+      (insert "My-Tag")
+      (normal-mode)
+      (should (not (eq major-mode 'text-mode))))))
+
 (provide 'files-tests)
 ;;; files-tests.el ends here
-- 
2.11.0

Message #16 received at 36401-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Benjamin Riefenstahl <b.riefenstahl <at> turtle-trading.net>
Cc: 36401-done <at> debbugs.gnu.org, Riefenstahl <at> mecom.de
Subject: Re: bug#36401: 26.0.90;
 set-auto-mode uses case-insensitive match for magic-mode-alist
Date: Sat, 20 Jul 2019 12:35:06 +0300

> From: Benjamin Riefenstahl <b.riefenstahl <at> turtle-trading.net>
> Cc: Benjamin Riefenstahl <Riefenstahl <at> mecom.de>,  36401 <at> debbugs.gnu.org
> Date: Sun, 14 Jul 2019 17:23:57 +0200
> 
> Eli Zaretskii writes:
> > I think this is an omission.  Please do propose a patch to fix it.
> 
> See attached.  This applies against master, but the conflict against
> emacs-26 is also just minor.  Please advise if you want this
> differently.

Thanks, pushed to the master branch.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.