GNU bug report logs - #15874
24.3.50; exit! not properly font-locked in ruby-mode

Previous Next

Package: emacs;

Reported by: Bozhidar Batsov <bozhidar <at> batsov.com>

Date: Tue, 12 Nov 2013 12:40:01 UTC

Severity: minor

Found in version 24.3.50

Fixed in version 24.4

Done: Dmitry Gutov <dgutov <at> yandex.ru>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 15874 in the body.
You can then email your comments to 15874 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Tue, 12 Nov 2013 12:40:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to Bozhidar Batsov <bozhidar <at> batsov.com>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Tue, 12 Nov 2013 12:40:03 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bozhidar Batsov <bozhidar <at> batsov.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Tue, 12 Nov 2013 14:38:29 +0200
The built-in Kernel#exit! method is not properly font-locked. exit is font-locked
with the built-in face, but the ! is font-locked with the default face.



In GNU Emacs 24.3.50.1 (x86_64-apple-darwin13.0.0, NS apple-appkit-1265.00)
 of 2013-11-11 on Bozhidars-MacBook-Pro.local
Windowing system distributor `Apple', version 10.3.1265
Configured using:
 `configure --prefix=/usr/local/Cellar/emacs/HEAD --without-dbus
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs/HEAD/share/info/emacs
 --without-gnutls --with-ns --disable-ns-self-contained'

Important settings:
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Conf[Unix]

Minor modes in effect:
  diff-auto-refine-mode: t
  erc-truncate-mode: t
  erc-spelling-mode: t
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-match-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-netsplit-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  global-flycheck-mode: t
  which-function-mode: t
  winner-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  global-anzu-mode: t
  anzu-mode: t
  projectile-global-mode: t
  projectile-mode: t
  flx-ido-mode: t
  ido-ubiquitous-mode: t
  shell-dirtrack-mode: t
  volatile-highlights-mode: t
  global-hl-line-mode: t
  recentf-mode: t
  savehist-mode: t
  show-smartparens-global-mode: t
  show-smartparens-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  prelude-global-mode: t
  prelude-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<return> M-> <return> C-y C-p C-p C-p C-p C-p C-p C-p 
C-p C-p C-n C-n C-n C-n C-n C-n C-n C-n C-l C-x C-s 
<down-mouse-1> <mouse-movement> <mouse-movement> <drag-mouse-1> 
<help-echo> <down-mouse-1> <mouse-1> <help-echo> <down-mouse-1> 
<mouse-1> <down-mouse-1> <mouse-1> M-x e m a c s - 
r e p o r t <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> r e p o r t e m 
a c s <return>

Recent messages:

Tramp: Checking `vc-registered' for /sudo:root <at> localhost:/etc/postfix/main.cf...done
Tramp: Inserting `/sudo:root <at> localhost:/etc/postfix/main.cf'...done
Mark set [2 times]
Saving file /sudo:root <at> localhost:/etc/postfix/main.cf...
Renaming /sudo:root <at> localhost:/etc/postfix/main.cf to /var/folders/32/92h89tr96rs0s79j4ckhhm_r0000gn/T/!sudo:root <at> localhost:!etc!postfix!main.cf~...failed
Cannot write backup file; backing up in ~/.emacs.d/%backup%~
Copying /sudo:root <at> localhost:/etc/postfix/main.cf to /Users/bozhidar/.emacs.d/%backup%~...done
Copying /var/folders/32/92h89tr96rs0s79j4ckhhm_r0000gn/T/tramp.92769aBQ.cf to /sudo:root <at> localhost:/etc/postfix/main.cf...done
Wrote /sudo:root <at> localhost:/etc/postfix/main.cf

Load-path shadows:
/Users/bozhidar/.emacs.d/elpa/flycheck-20131107.2307/.dir-locals hides /usr/local/Cellar/emacs/HEAD/share/emacs/24.3.50/lisp/gnus/.dir-locals

Features:
(shadow sort mail-extr emacsbug message rfc822 mml mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mail-utils tramp-cmds tramp-sh
conf-mode tramp-cache ffap url-parse url-vars crm misearch multi-isearch
superword subword ruby-tools inf-ruby ruby-mode-expansions
smartparens-ruby ruby-mode smie view magit-bisect magit-key-mode magit
epa derived epg epg-config diff-mode git-rebase-mode git-commit-mode
server log-edit pcvs-util add-log executable vc-git markdown-mode
noutline outline rainbow-mode color rainbow-delimiters elisp-slime-nav
guru-mode prelude-key-chord key-chord prelude-xml nxml-mode-expansions
html-mode-expansions smartparens-html rng-nxml rng-valid rng-loc rng-uri
rng-parse nxml-parse rng-match rng-dt rng-util rng-pttrn nxml-ns
nxml-mode nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc xmltok
prelude-web prelude-scss prelude-scheme prelude-ruby prelude-perl
prelude-org prelude-js prelude-erc erc-truncate erc-autoaway
erc-spelling erc-notify erc-log erc-list erc-menu erc-join erc-ring
erc-networks erc-pcomplete erc-track erc-match erc-button erc-fill
erc-stamp erc-netsplit erc-goodies erc erc-backend erc-compat
prelude-emacs-lisp prelude-css prelude-common-lisp slime-autoloads
prelude-clojure prelude-lisp prelude-c prelude-programming flycheck
help-mode f which-func imenu prelude-osx exec-path-from-shell
prelude-global-keybindings prelude-editor winner undo-tree diff esh-var
esh-io esh-cmd esh-opt esh-ext esh-proc esh-arg eldoc esh-groups eshell
esh-module esh-mode esh-util re-builder whitespace midnight ediff-merg
ediff-wind ediff-diff ediff-mult ediff-help ediff-init ediff-util ediff
dired-x dired anzu projectile pkg-info find-func grep compile s bookmark
pp expand-region text-mode-expansions er-basic-expansions
expand-region-custom expand-region-core flyspell ispell smex flx-ido flx
ido-ubiquitous warnings ido tramp tramp-compat auth-source gnus-util
mm-util mail-prsvr password-cache tramp-loaddefs trampver shell
pcomplete comint ansi-color format-spec etags ring volatile-highlights
hl-line windmove recentf tree-widget wid-edit savehist saveplace
uniquify diminish smartparens-config smartparens autorevert filenotify
delsel prelude-mode easy-mmode edmacro kmacro prelude-core epl advice
help-fns dash thingatpt prelude-ui zenburn-theme prelude-packages
ack-and-a-half-autoloads clojure-mode-autoloads coffee-mode-autoloads
diminish-autoloads elisp-slime-nav-autoloads erlang-autoloads
exec-path-from-shell-autoloads expand-region-autoloads flx-ido-autoloads
f-autoloads geiser-autoloads gist-autoloads finder-inf gh-autoloads
eieio byte-opt bytecomp byte-compile cconv eieio-core
gitconfig-mode-autoloads gitignore-mode-autoloads rx grizzl-autoloads
guru-mode-autoloads helm-projectile-autoloads ido-ubiquitous-autoloads
key-chord-autoloads logito-autoloads info easymenu
git-rebase-mode-autoloads git-commit-mode-autoloads cl-macs gv
markdown-mode-autoloads move-text-autoloads multiple-cursors-autoloads
paredit-autoloads pcache-autoloads pkg-info-autoloads epl-autoloads
puppet-mode-autoloads rainbow-delimiters-autoloads request-autoloads
rubocop-autoloads ruby-tools-autoloads s-autoloads scss-mode-autoloads
slim-mode-autoloads dash-autoloads undo-tree-autoloads
volatile-highlights-autoloads yaml-mode-autoloads package cl cl-loaddefs
cl-lib time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel
ns-win tool-bar dnd fontset image regexp-opt fringe tabulated-list
newcomment lisp-mode prog-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer nadvice loaddefs button faces cus-face macroexp
files text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
cocoa ns multi-tty emacs)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Wed, 04 Dec 2013 18:32:02 GMT) Full text and rfc822 format available.

Message #8 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Bozhidar Batsov <bozhidar <at> batsov.com>
Cc: 15874 <at> debbugs.gnu.org
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Wed, 04 Dec 2013 20:31:26 +0200
Bozhidar Batsov <bozhidar <at> batsov.com> writes:

> The built-in Kernel#exit! method is not properly font-locked. exit is
> font-locked with the built-in face, but the ! is font-locked with the
> default face.

Any ideas for a regexp that matches (any) non-symbol character?

http://lists.gnu.org/archive/html/emacs-devel/2013-11/msg00684.html

I'm not sure we want to move `!' to the symbol syntax class. Maybe we
even should move `?' back to punctuation.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Wed, 04 Dec 2013 19:21:02 GMT) Full text and rfc822 format available.

Message #11 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15874 <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Wed, 04 Dec 2013 14:20:29 -0500
> I'm not sure we want to move `!' to the symbol syntax class.

How is the Ruby lexing done w.r.t identifiers and ! (and ?) ?


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Wed, 04 Dec 2013 22:34:01 GMT) Full text and rfc822 format available.

Message #14 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15874 <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Thu, 05 Dec 2013 00:33:06 +0200
On 04.12.2013 21:20, Stefan Monnier wrote:
>> I'm not sure we want to move `!' to the symbol syntax class.
>
> How is the Ruby lexing done w.r.t identifiers and ! (and ?) ?

There's no special handling: ? is considered a part of whatever token 
it's adjacent to (even when the token follows it, ex. "?wut"), and ! is 
considered a separate token.

The former is usually harmless, as far as indentation and syntax 
highlighting are concerned (*), but not considering ! a part of 
preceding token makes us mishandle the (rare, to be sure) method names 
like "else!" and "end!".

(*) Counterexample:

irb(main):013:0> def foo?
irb(main):014:1> 42
irb(main):015:1> end
=> nil
irb(main):016:0> begin; foo?end
=> 42

We don't treat "end" in "begin; foo?end" as a keyword, but ideally, we 
should.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Wed, 04 Dec 2013 23:16:01 GMT) Full text and rfc822 format available.

Message #17 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15874 <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Thu, 05 Dec 2013 01:15:37 +0200
On 04.12.2013 21:20, Stefan Monnier wrote:
>> I'm not sure we want to move `!' to the symbol syntax class.
>
> How is the Ruby lexing done w.r.t identifiers and ! (and ?) ?

To answer your actual question: Ruby considers ! and ? parts of 
identifiers only in method names, and only when the name ends with one 
such a character.

The names of local variables, as well as instance and global ones, can't 
use ? and !. We may want to handle this eventually:

irb(main):001:0> @abc?1:2
=> 2

irb(main):002:0> abc = 1
=> 1
irb(main):003:0> abc?2:3
SyntaxError: (irb):3: syntax error, unexpected ':', expecting $end





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Thu, 05 Dec 2013 02:51:02 GMT) Full text and rfc822 format available.

Message #20 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15874 <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Wed, 04 Dec 2013 21:50:19 -0500
> To answer your actual question: Ruby considers ! and ? parts of identifiers
> only in method names, and only when the name ends with one such a character.

Maybe a good way to handle them is via syntax-propertize, then, especially
if we can come up with a good regexp that distinguishes "method name"
from "local variable name"?


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Thu, 05 Dec 2013 03:43:02 GMT) Full text and rfc822 format available.

Message #23 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15874 <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Thu, 05 Dec 2013 05:42:19 +0200
On 05.12.2013 04:50, Stefan Monnier wrote:
> Maybe a good way to handle them is via syntax-propertize,

Huh yes, that's an approach I haven't considered. But it seems more 
performance-intensive than the other options, at least as long as Emacs 
regexps don't support lookbehind.

If we could match "non-symbols", then the font-lock part of the problem 
would be solved, and appending "?" or "!" in the forward-token and 
backward-token functions would take negligible amount of time.

Say, why don't we have a "non-symbol" matcher? I would think that in 
many cases the regexp engine performs unnecessary work when it sees 
"\\_<" or "\\_>" because we often know in advance that the character 
we've already matched, or will have to match next, is a symbol 
constituent. But the engine has to check.

Most uses of `(regexp-opt '("boats" "boats" "boats") 'symbols)' would 
fall under this scenario.

> then, especially
> if we can come up with a good regexp that distinguishes "method name"
> from "local variable name"?

Not really. References to local variables, unlike instance and global 
ones, are indistinguishable from method calls if we judge only by name.

We could backtrack until the beginning of block or method, look for 
assignments, etc, but that's a lot of work for marginal gain: we can 
just assume that each such reference is a method calls, because 
otherwise it would be syntax error.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Thu, 05 Dec 2013 06:34:02 GMT) Full text and rfc822 format available.

Message #26 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Kevin Rodgers <kevin.d.rodgers <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Wed, 04 Dec 2013 23:34:35 -0700
On 12/4/13 11:31 AM, Dmitry Gutov wrote:
> Bozhidar Batsov<bozhidar <at> batsov.com>  writes:
>
>> The built-in Kernel#exit! method is not properly font-locked. exit is
>> font-locked with the built-in face, but the ! is font-locked with the
>> default face.
>
> Any ideas for a regexp that matches (any) non-symbol character?

`\sCODE'
     matches any character whose syntax is CODE.  Here CODE is a
     character that represents a syntax code: thus, `w' for word
     constituent, `-' for whitespace, `(' for open parenthesis, etc.
     To represent whitespace syntax, use either `-' or a space
     character.  *Note Syntax Class Table::, for a list of syntax codes
     and the characters that stand for them.

`\SCODE'
     matches any character whose syntax is not CODE.

...

 -- Syntax class: symbol constituent
     "Symbol constituents" (designated by `_') are the extra characters
     that are used in variable and command names along with word
     constituents.  For example, the symbol constituents class is used
     in Lisp mode to indicate that certain characters may be part of
     symbol names even though they are not part of English words.
     These characters are `$&*+-_<>'.  In standard C, the only
     non-word-constituent character that is valid in symbols is
     underscore (`_').

> http://lists.gnu.org/archive/html/emacs-devel/2013-11/msg00684.html
>
> I'm not sure we want to move `!' to the symbol syntax class. Maybe we
> even should move `?' back to punctuation.

-- 
Kevin Rodgers
Denver, Colorado, USA





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Thu, 05 Dec 2013 10:59:02 GMT) Full text and rfc822 format available.

Message #29 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Kevin Rodgers <kevin.d.rodgers <at> gmail.com>
Cc: 15874 <at> debbugs.gnu.org
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Thu, 05 Dec 2013 12:58:20 +0200
Kevin Rodgers <kevin.d.rodgers <at> gmail.com> writes:

>> Any ideas for a regexp that matches (any) non-symbol character?
>
> `\sCODE'
> `\SCODE'

That doesn't really help, unless we want to enumerate all syntax classes
that we can accept.

Non-symbol character is a character that's not of class "symbol", nor of
class "word". "\\S_" matches words.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Thu, 05 Dec 2013 14:50:03 GMT) Full text and rfc822 format available.

Message #32 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Kevin Rodgers <kevin.d.rodgers <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Thu, 05 Dec 2013 07:50:49 -0700
On 12/5/13 3:58 AM, Dmitry Gutov wrote:
> Kevin Rodgers<kevin.d.rodgers <at> gmail.com>  writes:
>
>>> Any ideas for a regexp that matches (any) non-symbol character?
>>
>> `\sCODE'
>> `\SCODE'
>
> That doesn't really help, unless we want to enumerate all syntax classes
> that we can accept.
>
> Non-symbol character is a character that's not of class "symbol", nor of
> class "word". "\\S_" matches words.

[^[:symbol:][:word:]]

if the [:symbol:] character class actually existed :-)

-- 
Kevin Rodgers
Denver, Colorado, USA





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Thu, 05 Dec 2013 17:26:02 GMT) Full text and rfc822 format available.

Message #35 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Kevin Rodgers <kevin.d.rodgers <at> gmail.com>
Cc: 15874 <at> debbugs.gnu.org
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Thu, 05 Dec 2013 19:25:04 +0200
Kevin Rodgers <kevin.d.rodgers <at> gmail.com> writes:

> [^[:symbol:][:word:]]
>
> if the [:symbol:] character class actually existed :-)

Yep, I like this non-existent feature, too. :)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Thu, 05 Dec 2013 19:03:01 GMT) Full text and rfc822 format available.

Message #38 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15874 <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Thu, 05 Dec 2013 14:02:53 -0500
>> Maybe a good way to handle them is via syntax-propertize,
> Huh yes, that's an approach I haven't considered.

The advantage is that it would also fix the sexp-movement operations.

> But it seems more performance-intensive than the other options, at
> least as long as Emacs regexps don't support lookbehind.

I don't see why we'd need lookbehind.
We can use a regexp like "\\(?:\\sw\\|\\s_\\)\\([!?]\\)" and then place
the syntax-table property on the ? (or !) character.

> Say, why don't we have a "non-symbol" matcher?

I'd rather do something a bit more generic, then, like a \\S which can
take a set of syntaxes to exclude.  Or maybe extend the [:foo:]
character classes to allow [:sw:], [:s_:], etc... so we could do
[^[:sw:][:s_:]].

> I would think that in many cases the regexp engine performs
> unnecessary work when it sees "\\_<" or "\\_>" because we often know
> in advance that the character we've already matched, or will have to
> match next, is a symbol constituent. But the engine has to check.

It's not that big of a deal, tho.  I doubt this would ever appear as
a noticeable slowdown.  And with [^[:sw:][:s_:]] you'd have the extra
work of having to handle the EOB case, so you'd have
\(\'\|[^[:sw:][:s_:]]). 

>> then, especially if we can come up with a good regexp that
>> distinguishes "method name" from "local variable name"?
> Not really.  References to local variables, unlike instance and global ones,
> are indistinguishable from method calls if we judge only by name.

But you said that ! and ? are only allowed in method names.  So somehow
Ruby's parser/lexer distinguishes the two cases.  Would a reference to
a variable "foo?" simply always be parsed as "a reference to variable
foo?"  which would later trigger an error because there's no such
variable (because there can't be any such variable)?


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Fri, 06 Dec 2013 02:46:01 GMT) Full text and rfc822 format available.

Message #41 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15874 <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Fri, 06 Dec 2013 04:44:49 +0200
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> The advantage is that it would also fix the sexp-movement operations.

Two birds with one stone, yes.

> I don't see why we'd need lookbehind.

Not mandatory, just good for performance, AFAICT. Instead of maching all
identifiers that look like methods and then finding out that most of
them don't end with ? or ! anyway, it would be faster to scan the buffer
for ? or !, and then look a bit back and see if the text before such a
character resembles a method name.

> We can use a regexp like "\\(?:\\sw\\|\\s_\\)\\([!?]\\)" and then place
> the syntax-table property on the ? (or !) character.

Yes, something like that.

> I'd rather do something a bit more generic, then, like a \\S which can
> take a set of syntaxes to exclude.  Or maybe extend the [:foo:]
> character classes to allow [:sw:], [:s_:], etc... so we could do
> [^[:sw:][:s_:]].

That looks good, but, like you mentioned later, if we had a dedicated
backslash-sequence, it could also transparently handle the EOB case.  I
guess, the question is, would it see much use. `regexp-opt' seems to be
the primary use case to me.

> It's not that big of a deal, tho.  I doubt this would ever appear as
> a noticeable slowdown.

That may be true. 

> But you said that ! and ? are only allowed in method names.  So somehow
> Ruby's parser/lexer distinguishes the two cases.  Would a reference to
> a variable "foo?" simply always be parsed as "a reference to variable
> foo?"  which would later trigger an error because there's no such
> variable (because there can't be any such variable)?

Eh, I guess whenever it sees ? or ! at the end of the identifier, the
parser goes with the assumption that it is a method call, because
otherwise it would be a syntax error.

IOW, we only have to worry about @instance and $global variables.

irb(main):022:0* def a
irb(main):023:1> 42
irb(main):024:1> end
=> nil
irb(main):025:0> a?1:2
SyntaxError: (irb):25: syntax error, unexpected ':', expecting $end
a?1:2
    ^
	from /home/gutov/.rbenv/versions/1.9.3-p429/bin/irb:12:in `<main>'
irb(main):026:0> a = 5
=> 5
irb(main):027:0> a?3:4
SyntaxError: (irb):27: syntax error, unexpected ':', expecting $end
a?3:4
    ^
	from /home/gutov/.rbenv/versions/1.9.3-p429/bin/irb:12:in `<main>'
irb(main):028:0> $abc?
irb(main):029:0* 2
SyntaxError: (irb):29: syntax error, unexpected $end, expecting ':'
	from /home/gutov/.rbenv/versions/1.9.3-p429/bin/irb:12:in `<main>'
irb(main):030:0> $abc = 6
=> 6
irb(main):031:0> $abc?1:2
=> 1




Reply sent to Dmitry Gutov <dgutov <at> yandex.ru>:
You have taken responsibility. (Fri, 06 Dec 2013 04:24:01 GMT) Full text and rfc822 format available.

Notification sent to Bozhidar Batsov <bozhidar <at> batsov.com>:
bug acknowledged by developer. (Fri, 06 Dec 2013 04:24:02 GMT) Full text and rfc822 format available.

Message #46 received at 15874-done <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15874-done <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Fri, 06 Dec 2013 06:23:34 +0200
Version: 24.4

Well, it was easier than expected. Stefan, thanks for the advice.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15874; Package emacs. (Fri, 06 Dec 2013 04:58:01 GMT) Full text and rfc822 format available.

Message #49 received at 15874 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15874 <at> debbugs.gnu.org, Bozhidar Batsov <bozhidar <at> batsov.com>
Subject: Re: bug#15874: 24.3.50; exit! not properly font-locked in ruby-mode
Date: Thu, 05 Dec 2013 23:57:49 -0500
> Not mandatory, just good for performance, AFAICT. Instead of maching all
> identifiers that look like methods and then finding out that most of
> them don't end with ? or ! anyway, it would be faster to scan the buffer
> for ? or !, and then look a bit back and see if the text before such a
> character resembles a method name.

We can also do that, using a "[?!]" regexp and then do the "look a bit
back" in Elisp.

> That looks good, but, like you mentioned later, if we had a dedicated
> backslash-sequence, it could also transparently handle the EOB case.
> I guess, the question is, would it see much use.

But then it would be a lot more special cased.

> `regexp-opt' seems to be the primary use case to me.

I think the second arg to `regexp-opt' is a misfeature.

> irb(main):026:0> a = 5
> => 5
> irb(main):027:0> a?3:4
> SyntaxError: (irb):27: syntax error, unexpected ':', expecting $end
> a?3:4
>     ^
> 	from /home/gutov/.rbenv/versions/1.9.3-p429/bin/irb:12:in `<main>'

Good.


        Stefan




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 03 Jan 2014 12:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 10 years and 124 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.