GNU bug report logs - #15294
24.3.50; js2-mode parser is several times slower in lexical-binding mode

Previous Next

Package: emacs;

Reported by: Dmitry Gutov <dgutov <at> yandex.ru>

Date: Fri, 6 Sep 2013 21:01:02 UTC

Severity: normal

Found in version 24.3.50

Done: Stefan Monnier <monnier <at> iro.umontreal.ca>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 15294 in the body.
You can then email your comments to 15294 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Fri, 06 Sep 2013 21:01:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Dmitry Gutov <dgutov <at> yandex.ru>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Fri, 06 Sep 2013 21:01:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Fri, 06 Sep 2013 23:59:20 +0300
If I append -*- lexical-binding: t -*- to the first line of js2-mode.el
and eval the buffer, parsing a file takes several (3 to 4) times longer.

To measure:

0. Install js2-mode, e.g. from GNU ELPA.
1. Save
http://mootools.net/download/get/mootools-core-1.4.5-full-nocompat.js to
some local directory, open it in Emacs.
2. If it opens in some other mode than js2-mode, M-x js2-mode.
3. Don't touch the keys, wait until the file is parsed (the parsing gets
interrupted and rescheduled on input). When redisplay works again, we
can re-parse the file again and measure the time it takes.
4. eval-expression (js2-time (js2-reparse t)), wait until the message
area displays the amount of seconds the process took.

To measure the performance in interpreted mode, open js2-mode.el, M-x
eval-buffer and go through the above scenario (1-4).
To measure the performance in compiled mode, M-x byte-compile-file, then
(load "js2-mode.elc"), then go through the scenario.

My measurements:

lexical-binding |       no |      yes |
Interpreted     |    1 sec | 4.26 sec |
Compiled        | 0.63 sec | 1.76 sec |

--
In GNU Emacs 24.3.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.6.4)
 of 2013-09-06 on axl
Bzr revision: 114159 dmantipov <at> yandex.ru-20130906164012-6j0s6otwkypj9s1z
Windowing system distributor `The X.Org Foundation', version 11.0.11303000
System Description:	Ubuntu 13.04




Reply sent to Xue Fuqiao <xfq.free <at> gmail.com>:
You have taken responsibility. (Fri, 06 Sep 2013 23:45:02 GMT) Full text and rfc822 format available.

Notification sent to Dmitry Gutov <dgutov <at> yandex.ru>:
bug acknowledged by developer. (Fri, 06 Sep 2013 23:45:03 GMT) Full text and rfc822 format available.

Message #10 received at 15294-done <at> debbugs.gnu.org (full text, mbox):

From: Xue Fuqiao <xfq.free <at> gmail.com>
To: 15294-done <at> debbugs.gnu.org
Subject: Re: 24.3.50; js2-mode parser is several times slower in
 lexical-binding mode
Date: Sat, 7 Sep 2013 07:44:38 +0800
Fixed in revision 114156.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sat, 07 Sep 2013 03:16:01 GMT) Full text and rfc822 format available.

Message #13 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: 15294 <at> debbugs.gnu.org
Cc: xfq.free <at> gmail.com, dgutov <at> yandex.ru
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Fri, 06 Sep 2013 23:15:15 -0400
reopen 15294 
thanks

> Fixed in revision 114156.

No, 114156 fixes another problem (which affected both lexical and
dynamic binding code).


        Stefan




Did not alter fixed versions and reopened. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sat, 07 Sep 2013 03:16:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sun, 08 Sep 2013 22:33:02 GMT) Full text and rfc822 format available.

Message #18 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Sun, 08 Sep 2013 18:32:02 -0400
A first likely candidate is the bunch of `catch/throw' in js2-get-token,
since catch, unwind-protect and condition-case are all implemented in
the bytecode interpreter in ways which don't work well for
lexical scoping.


        Stefan "Just reporting on his little progress"




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Tue, 10 Sep 2013 02:05:01 GMT) Full text and rfc822 format available.

Message #21 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Mon, 09 Sep 2013 22:04:38 -0400
> If I append -*- lexical-binding: t -*- to the first line of js2-mode.el
> and eval the buffer, parsing a file takes several (3 to 4) times longer.

[ I'm disregarding the interpreted time for now, since the issues are
  likely to be different. ]

In the compiled case, loading js2-mode-lexical.elc and then redefining
`js2-get-token' to the value it has in js2-mode-dynamic.elc brings me
back the same speed as with js2-mode-dynamic.elc.
IOW the slowdown is in js2-get-token.

If it indeed comes from the handling of catch/throw, then there are two
ways to fix it:
- provide a new byte-code for catch which does not require wrapping the
  body in a closure.  This should not be terribly difficult and would
  benefit all existing code, but it does require changes to the way
  catch is implemented.
- improve the byte-compiler so it can compile away cl-block/cl-return.
  This should not require new byte-codes, but requires delving "deepish"
  into the byte-compiler.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Fri, 13 Sep 2013 03:41:02 GMT) Full text and rfc822 format available.

Message #24 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Thu, 12 Sep 2013 23:40:45 -0400
> In the compiled case, loading js2-mode-lexical.elc and then redefining
> `js2-get-token' to the value it has in js2-mode-dynamic.elc brings me
> back the same speed as with js2-mode-dynamic.elc.
> IOW the slowdown is in js2-get-token.

It seems the slowdown is indeed linked to the way `catch' is handled
(indeed, this non-idiomatic ELisp code ends up byte-compiled in a really
poor way).

The trivial patch below brings the time down from 5s to 2.6s (as
compared to 2.1s for the dynamic-binding version).


        Stefan


diff --git a/packages/js2-mode/js2-mode.el b/packages/js2-mode/js2-mode.el
index 3568f18..1d76469 100644
--- a/packages/js2-mode/js2-mode.el
+++ b/packages/js2-mode/js2-mode.el
@@ -5310,10 +5311,10 @@ corresponding number.  Otherwise return -1."
 
 (defun js2-get-token ()
   "Return next JavaScript token, an int such as js2-RETURN."
-  (let (c c1 identifier-start is-unicode-escape-start
-        contains-escape escape-val str result base
-        is-integer quote-char val look-for-slash continue)
-    (catch 'return
+  (catch 'return
+    (let (c c1 identifier-start is-unicode-escape-start
+            contains-escape escape-val str result base
+            is-integer quote-char val look-for-slash continue)
       (while t
         ;; Eat whitespace, possibly sensitive to newlines.
         (setq continue t)




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Fri, 13 Sep 2013 04:01:01 GMT) Full text and rfc822 format available.

Message #27 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>, Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: RE: bug#15294: 24.3.50; js2-mode parser is several times slower in
 lexical-binding mode
Date: Thu, 12 Sep 2013 20:59:52 -0700 (PDT)
> It seems the slowdown is indeed linked to the way `catch' is handled
> (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really
> poor way). The trivial patch below brings the time down from 5s to 2.6s
> (as compared to 2.1s for the dynamic-binding version).
>
> -  (let (c c1 identifier-start is-unicode-escape-start
> -        contains-escape escape-val str result base
> -        is-integer quote-char val look-for-slash continue)
> -    (catch 'return
> +  (catch 'return
> +    (let (c c1 identifier-start is-unicode-escape-start
> +            contains-escape escape-val str result base
> +            is-integer quote-char val look-for-slash continue)

Not really following this thread, but this caught my eye.  Is this
something that should be mentioned as a guideline in the manual?

Or is the byte compiler likely to be improved in this regard, so users
need not be aware of it and take measures manually?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Fri, 13 Sep 2013 04:38:02 GMT) Full text and rfc822 format available.

Message #30 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 15294 <at> debbugs.gnu.org, Dmitry Gutov <dgutov <at> yandex.ru>
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Fri, 13 Sep 2013 00:37:25 -0400
> Or is the byte compiler likely to be improved in this regard, so users
> need not be aware of it and take measures manually?

As indicated in the FIXMEs I added in src/bytecode.c at the time, the
current byte codes for unwind-protect, condition-case, and catch are
very inefficient for lexical-binding code.

I do hope to fix those issues by introducing other byte-codes which will
let us generate significantly more efficient code for those constructs,
but in 24.1, the priority was to get lexical-binding to work correctly,
performance being a secondary concern (tho for most idiomatic Elisp
code, the performance tends to be competitive).

What people should know is that

   (let (x y z)
     ...(setq x ...)
     ...(setq z ...)
     ...(setq y ...)

is often a bad idea in Elisp, and even more so in lexical-binding code
(in some cases, if a variable is immutable it can be handled
significantly more efficiently, so the mere existence of a single `setq'
on a variable can sometimes slow other chunks of code: in many cases
`let' is cheaper than `setq').


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Fri, 13 Sep 2013 05:46:01 GMT) Full text and rfc822 format available.

Message #33 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15294 <at> debbugs.gnu.org, Dmitry Gutov <dgutov <at> yandex.ru>
Subject: RE: bug#15294: 24.3.50; js2-mode parser is several times slower in
 lexical-binding mode
Date: Thu, 12 Sep 2013 22:45:21 -0700 (PDT)
> > Or is the byte compiler likely to be improved in this regard, so users
> > need not be aware of it and take measures manually?
> 
> As indicated in the FIXMEs I added in src/bytecode.c at the time, the
> current byte codes for unwind-protect, condition-case, and catch are
> very inefficient for lexical-binding code.
> 
> I do hope to fix those issues by introducing other byte-codes which will
> let us generate significantly more efficient code for those constructs,
> but in 24.1, the priority was to get lexical-binding to work correctly,
> performance being a secondary concern (tho for most idiomatic Elisp
> code, the performance tends to be competitive).
> 
> What people should know is that
> 
>    (let (x y z)
>      ...(setq x ...)
>      ...(setq z ...)
>      ...(setq y ...)
> 
> is often a bad idea in Elisp, and even more so in lexical-binding code
> (in some cases, if a variable is immutable it can be handled
> significantly more efficiently, so the mere existence of a single `setq'
> on a variable can sometimes slow other chunks of code: in many cases
> `let' is cheaper than `setq').

Thank you for both answers.  Will the second apply even after making the
byte-compiler improvements described for the first?  (I'm guessing yes.)
If so, you might want to mention this guideline in the manual.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Fri, 13 Sep 2013 13:02:01 GMT) Full text and rfc822 format available.

Message #36 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Drew Adams <drew.adams <at> oracle.com>
Cc: 15294 <at> debbugs.gnu.org, Dmitry Gutov <dgutov <at> yandex.ru>
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Fri, 13 Sep 2013 09:01:03 -0400
>> What people should know is that
>> 
>> (let (x y z)
>> ...(setq x ...)
>> ...(setq z ...)
>> ...(setq y ...)
>> 
>> is often a bad idea in Elisp, and even more so in lexical-binding code
>> (in some cases, if a variable is immutable it can be handled
>> significantly more efficiently, so the mere existence of a single `setq'
>> on a variable can sometimes slow other chunks of code: in many cases
>> `let' is cheaper than `setq').

> Thank you for both answers.  Will the second apply even after making the
> byte-compiler improvements described for the first?  (I'm guessing yes.)
> If so, you might want to mention this guideline in the manual.

Yes.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sat, 14 Sep 2013 04:21:02 GMT) Full text and rfc822 format available.

Message #39 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50; js2-mode parser is several times slower in
 lexical-binding mode
Date: Sat, 14 Sep 2013 07:20:29 +0300
On 13.09.2013 06:40, Stefan Monnier wrote:
> It seems the slowdown is indeed linked to the way `catch' is handled
> (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really
> poor way).

What's non-idiomatic about this use of `catch'?

> The trivial patch below brings the time down from 5s to 2.6s (as
> compared to 2.1s for the dynamic-binding version).

Indeed, it does, in the compiled mode (but please don't install it yet, 
because it'll complicate merging the `arrow-functions' branch(*)). It 
does not make much of a difference in the interpreted mode.

Now that we have eager macro-expansion, I was rather happy that 
interpreted js2-mode performance is only like 2x worse than when compiled.

But 2.6 vs 2.1, it still a noticeable regression. Do you suppose the 
usage of `setq' is the main contributor?

Speaking of byte-code changes, that seems like a fine idea, but ideally 
the code should work similarly well on Emacs 24.1-24.3. Or we can just 
hold off on making js2-mode switch to lexical-binding until [long] after 
24.4 is out.

(*) Would you take a look at it, too? It has quite a few changes in 
`js2-get-token' and related functions. They also make performing the 
same change as in your patch more difficult, since I'm actually using 
the value returned by `catch' before returning from the function.

On 13.09.2013 07:37, Stefan Monnier wrote:
> the mere existence of a single `setq'
> on a variable can sometimes slow other chunks of code: in many cases
> `let' is cheaper than `setq').

I see. Does this also extend to `setf' and its defstruct-related 
functionality? It's all over the js2-mode codebase.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sat, 14 Sep 2013 14:28:02 GMT) Full text and rfc822 format available.

Message #42 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Sat, 14 Sep 2013 10:27:17 -0400
>> It seems the slowdown is indeed linked to the way `catch' is handled
>> (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really
>> poor way).
> What's non-idiomatic about this use of `catch'?

The non-idiomatic part is the "one big let on top, with lots of setq
inside".  It's clearly C code in Elisp syntax.

> It does not make much of a difference in the interpreted mode.

The interpreted performance is affected by completely different factors.
My guess for the interpreted case is that there are simply "too many"
local variables: the environment is represented by a simple alist, so
variable lookup time is proportional to the number of local variables.
That fine when there are 5 local variables, but is inefficient when you
have 100 (better would be a balanced tree or maybe a hash table).
This said, I'm not terribly concerned about it: if you need it to go
fast, you should byte-compile the code.  And I hope we will be able to
get rid of the interpreter in some distantish future.

> Now that we have eager macro-expansion, I was rather happy that interpreted
> js2-mode performance is only like 2x worse than when compiled.

Eager macro-expansion indeed speeds up interpreted code, even though the
intention was rather to get one-step closer to the elimination
of interpretation.

> But 2.6 vs 2.1, it still a noticeable regression. Do you suppose the usage
> of `setq' is the main contributor?

The problem goes as follows:

1- Because of how the `catch' byte-code works, for a (catch TAG BODY)
   where BODY refers to some surrounding lexical variables LVARS, the
   byte-compiler needs to turn the code into something similar to:

   (let ((body-fun (make-closure LVARS () BODY)))
     (catch TAG (funcall body-fun)))

2- When a lexical variable is both
   a- caught in a closure
   b- not immutable
   the byte-compiler can't store this variable in the bytecode stack
   (since the closure can't refer to the bytecode stack directly, but
   instead stores *copies* of the elements it needs), so it needs to
   change code like

   (let ((lvar VAL1))
     ...
     (setq lvar VAL2)
     ...(lambda () ..lvar..)...)

   into
     
   (let ((lvar (list VAL1)))
     ...
     (setcar lvar VAL2)
     ...(lambda () ..(car lvar)..)...)

So if you look at js2-get-token, you'll see that the code does not
directly use any closure, but the use of `catch' ends up putting most of
the body into various closures.  And since all variables are declared
outside of the catch but used inside, and they're all modified by
`setq', they all end up converted as above, so that every use of such
a variable turns into "get the cons cell from the environment, then
apply car to it".

By moving the let inside the catch, some of those variables end up not
being caught by a closure any more, so they don't need to be converted
to cons cells, hence the reduction from 5s down to 2.6s.

> (*) Would you take a look at it, too? It has quite a few changes in
> js2-get-token' and related functions.

> They also make performing the same change as in your patch more
> difficult, since I'm actually using the value returned by `catch'
> before returning from the function.

That's not a problem.  The rule to follow is simply: sink the `let'
bindings closer to their use.  You don't need to `let' bind all those
vars together in one big `let': you can split this let into various
`let's which you can then move deeper into the code.  In some cases
you'll find that some of those vars don't even need to be `setq'd any
more.

Note that such a "scope-reduction" can also be done in C and in many
cases it's also a good idea to do it in C, tho the impact on performance
is much less significant because C doesn't have closures.

>> the mere existence of a single `setq' on a variable can sometimes
>> slow other chunks of code: in many cases `let' is cheaper than `setq').
> I see.  Does this also extend to `setf' and its defstruct-related
> functionality?

It has to do specifically with `setq' (i.e. modification of plain
variables): when `setf' expands to `setq', `setf' is impacted,
otherwise no.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sun, 15 Sep 2013 00:12:02 GMT) Full text and rfc822 format available.

Message #45 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50; js2-mode parser is several times slower in
 lexical-binding mode
Date: Sun, 15 Sep 2013 03:11:41 +0300
On 14.09.2013 17:27, Stefan Monnier wrote:
>>> It seems the slowdown is indeed linked to the way `catch' is handled
>>> (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really
>>> poor way).
>> What's non-idiomatic about this use of `catch'?
>
> The non-idiomatic part is the "one big let on top, with lots of setq
> inside".  It's clearly C code in Elisp syntax.

Or rather Java code, considering its origins. In general, we have to 
stay close enough to the SpiderMonkey codebase to be able to port new 
syntax features easily.

You've explained how this is bad when combined with closures, but other 
than gotchas with `catch' (and `condition-case', I'm assuming), we don't 
have any higher-order functions in the parser. No lambda forms anywhere, 
at least.

Can the fact that `dotimes', `dolist' and `loop' are advised with 
`cl--wrap-in-nil-block', which expands into `catch' form, make much of a 
difference?

>> It does not make much of a difference in the interpreted mode.
>
> The interpreted performance is affected by completely different factors.
> My guess for the interpreted case is that there are simply "too many"
> local variables: the environment is represented by a simple alist, so
> variable lookup time is proportional to the number of local variables.
> That fine when there are 5 local variables, but is inefficient when you
> have 100 (better would be a balanced tree or maybe a hash table).

`js2-get-token' has 13 local variables (*). Which is, while not a 
little, far from 100. Most of the other functions have fewer than that.

Are you counting the global variables, too? The dynamic-binding 
interpreter has to work with them, too. How is it that much faster?

(*) According to Steve's notes, symbol lookup in alists is faster than 
in Emacs hashtables until 50+ elements. Or was, around the year 2009.

> This said, I'm not terribly concerned about it: if you need it to go
> fast, you should byte-compile the code.

I guess so. The sloppy users who disregarded the instructions to 
byte-compile the code were actually creating a bad reputation for 
js2-mode 1-2 years ago, but since package.el is much more popular now, 
it should be less of a problem.

>> But 2.6 vs 2.1, it still a noticeable regression. Do you suppose the usage
>> of `setq' is the main contributor?
>
> The problem goes as follows: ...

Thank you for the detailed explanation.

There are a few `catch' forms left there, for tags `continue' and 
`break', used for control flow. So, most of the 0.5s difference left is 
likely due to them, right?

I wonder how hard it'll be to rewrite that without `catch'.

>> (*) Would you take a look at it, too? It has quite a few changes in
>> js2-get-token' and related functions.
>
>> They also make performing the same change as in your patch more
>> difficult, since I'm actually using the value returned by `catch'
>> before returning from the function.
>
> That's not a problem.  The rule to follow is simply: sink the `let'
> bindings closer to their use.  You don't need to `let' bind all those
> vars together in one big `let': you can split this let into various
> `let's which you can then move deeper into the code.  In some cases
> you'll find that some of those vars don't even need to be `setq'd any
> more.

Thanks, will do.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sun, 15 Sep 2013 00:25:01 GMT) Full text and rfc822 format available.

Message #48 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50; js2-mode parser is several times slower in
 lexical-binding mode
Date: Sun, 15 Sep 2013 03:24:10 +0300
On 10.09.2013 05:04, Stefan Monnier wrote:
> If it indeed comes from the handling of catch/throw, then there are two
> ways to fix it:
> - ...
> - improve the byte-compiler so it can compile away cl-block/cl-return.
>    This should not require new byte-codes, but requires delving "deepish"
>    into the byte-compiler.

This would take care of `cl-block' wrappers, but it would do nothing 
about explicit uses of `catch', right?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sun, 15 Sep 2013 05:05:01 GMT) Full text and rfc822 format available.

Message #51 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Sun, 15 Sep 2013 01:04:22 -0400
> Can the fact that `dotimes', `dolist' and `loop' are advised with
> cl--wrap-in-nil-block', which expands into `catch' form, make much
> of a difference?

No, these only expand to `catch' if there's a `return' inside.

> `js2-get-token' has 13 local variables (*).  Which is, while not a little,
> far from 100.  Most of the other functions have fewer than that.

I don't know how many it takes to be significantly slower than `symbol-value'.

> Are you counting the global variables, too?

No.  But arguments, yes, and every (defvar <var>) between point and BOB
as well.

> The dynamic-binding interpreter has to work with them, too.  How is it
> that much faster?

Dynamic binding lookup is done by `symbol-value' which is just a field
access (plus checking that the var is not special (e.g. buffer-local or
a predefined C variable).

>>> But 2.6 vs 2.1, it still a noticeable regression. Do you suppose the usage
>>> of `setq' is the main contributor?
>> The problem goes as follows: ...
> Thank you for the detailed explanation.
> There are a few `catch' forms left there, for tags `continue' and `break',
> used for control flow. So, most of the 0.5s difference left is likely due to
> them, right?

The problem is not just the use of catch, but the combination of "catch"
with all those vars let-bound outside of catch, used inside, and mutated:
- make a variable immutable (i.e. remove the setqs on it) and that
  variable becomes more efficient again.
- move the let binding inside the catch, and the var becomes efficient again.
- don't use the variable inside the catch, and it becomes efficient again.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sun, 15 Sep 2013 05:07:01 GMT) Full text and rfc822 format available.

Message #54 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Sun, 15 Sep 2013 01:06:08 -0400
>> If it indeed comes from the handling of catch/throw, then there are two
>> ways to fix it:
>> - ...
>> - improve the byte-compiler so it can compile away cl-block/cl-return.
>> This should not require new byte-codes, but requires delving "deepish"
>> into the byte-compiler.
> This would take care of `cl-block' wrappers, but it would do nothing about
> explicit uses of `catch', right?

All js2-mode's uses of catch/throw can be replaced with
block/return, AFAICT (not that it's useful right now, since they'll
just be macroexpanded back to catch/throw).


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sun, 15 Sep 2013 16:55:02 GMT) Full text and rfc822 format available.

Message #57 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Richard Stallman <rms <at> gnu.org>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org, monnier <at> iro.umontreal.ca
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Sun, 15 Sep 2013 12:54:26 -0400
        [ To any NSA and FBI agents reading my email: please consider
        [ whether defending the US Constitution against all enemies,
        [ foreign or domestic, requires you to follow Snowden's example.

Perhaps we should optimize the implementation of catch
in bytecode with some new opcodes.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Thu, 19 Sep 2013 04:12:01 GMT) Full text and rfc822 format available.

Message #60 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Wed, 18 Sep 2013 19:48:48 -0400
I now have a tentative patch which introduces new bytecodes for
"catch" which avoid the need to create closures for
lexical-binding code.

Current results:
- dynbind old-bytecode: 0.84s
- dynbind new-bytecode: 0.83s
- lexbind old-bytecode: 2.14s
- lexbind new-bytecode: 0.67s

Of course, code compiled with the new bytecodes won't run on
non-bleeding edge Emacs.

On my "standard test" (i.e. time to recompile all the emacs/lisp files),
the new bytecodes make no measurable difference, so `catch' doesn't seem
to be used very much there.

Similar changes would be welcome for condition-case and
unwind-protect, and since adding new bytecodes is something I'd rather
not do too often (and since there aren't many unused bytecodes left),
I'll try to get a complete patch first to make sure I don't have to
change my mind and use different byte codes.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sun, 22 Sep 2013 04:57:02 GMT) Full text and rfc822 format available.

Message #63 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> IRO.UMontreal.CA>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Sun, 22 Sep 2013 07:56:34 +0300
Stefan Monnier <monnier <at> IRO.UMontreal.CA> writes:

> I now have a tentative patch which introduces new bytecodes for
> "catch" which avoid the need to create closures for
> lexical-binding code.
>
> Current results:
> - dynbind old-bytecode: 0.84s
> - dynbind new-bytecode: 0.83s
> - lexbind old-bytecode: 2.14s
> - lexbind new-bytecode: 0.67s

Looks impressive!

> Of course, code compiled with the new bytecodes won't run on
> non-bleeding edge Emacs.

Sure.

> Similar changes would be welcome for condition-case and
> unwind-protect, and since adding new bytecodes is something I'd rather
> not do too often (and since there aren't many unused bytecodes left),
> I'll try to get a complete patch first to make sure I don't have to
> change my mind and use different byte codes.

There's absolutely no hurry, I'm not going to switch js2-mode to
lexical-binding until after 24.4 is out.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Thu, 03 Oct 2013 05:01:02 GMT) Full text and rfc822 format available.

Message #66 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Thu, 03 Oct 2013 01:00:34 -0400
You can try setting byte-compile--use-old-handlers to nil now to see if
it fixes your performance problems.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Fri, 04 Oct 2013 02:39:02 GMT) Full text and rfc822 format available.

Message #69 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50; js2-mode parser is several times slower in
 lexical-binding mode
Date: Fri, 04 Oct 2013 05:38:20 +0300
On 03.10.2013 08:00, Stefan Monnier wrote:
> You can try setting byte-compile--use-old-handlers to nil now to see if
> it fixes your performance problems.

Indeed, I see the same peformance improvement you wrote about: 0.60s in 
byte-compiled dynamic binding mode, and now 0.48s in byte-compiled 
lexical binding mode. Thanks!

Not sure if I should close this issue now or at least wait until 
byte-compile--use-old-handlers is nil by default.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Fri, 04 Oct 2013 13:53:01 GMT) Full text and rfc822 format available.

Message #72 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Fri, 04 Oct 2013 09:52:40 -0400
> Not sure if I should close this issue now or at least wait until
> byte-compile--use-old-handlers is nil by default.

Your call.  I'm not completely sure when we should change it: setting it
to nil will mean that pretty much any file compiled with the new Emacs
won't run on older Emacsen.  I think I'd like to see 24.4 release before
making the change.  But please do set it in your .emacs ASAP so it gets
wider testing.


        Stefan




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sat, 05 Oct 2013 03:28:01 GMT) Full text and rfc822 format available.

Message #75 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50; js2-mode parser is several times slower in
 lexical-binding mode
Date: Sat, 05 Oct 2013 06:27:19 +0300
On 04.10.2013 16:52, Stefan Monnier wrote:
>> Not sure if I should close this issue now or at least wait until
>> byte-compile--use-old-handlers is nil by default.
>
> Your call.  I'm not completely sure when we should change it: setting it
> to nil will mean that pretty much any file compiled with the new Emacs
> won't run on older Emacsen.  I think I'd like to see 24.4 release before
> making the change.

No problem, guess then I'll leave this open until then.

I'll do the code reshuffling discussed earlier, but actually having a 
performance improvement will be a better reason to switch to lexical 
binding.

> But please do set it in your .emacs ASAP so it gets wider testing.

Good idea, done.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#15294; Package emacs. (Sun, 14 Dec 2014 12:33:02 GMT) Full text and rfc822 format available.

Message #78 received at 15294 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dgutov <at> yandex.ru>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 15294 <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Sun, 14 Dec 2014 14:31:57 +0200
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> Not sure if I should close this issue now or at least wait until
>> byte-compile--use-old-handlers is nil by default.

> won't run on older Emacsen.  I think I'd like to see 24.4 release before
> making the change.  But please do set it in your .emacs ASAP so it gets
> wider testing.

Just a reminder: I've been running with that setting for a while now,
with no problems that I can remember.




Reply sent to Stefan Monnier <monnier <at> iro.umontreal.ca>:
You have taken responsibility. (Sun, 14 Dec 2014 14:09:01 GMT) Full text and rfc822 format available.

Notification sent to Dmitry Gutov <dgutov <at> yandex.ru>:
bug acknowledged by developer. (Sun, 14 Dec 2014 14:09:02 GMT) Full text and rfc822 format available.

Message #83 received at 15294-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dgutov <at> yandex.ru>
Cc: 15294-done <at> debbugs.gnu.org
Subject: Re: bug#15294: 24.3.50;
 js2-mode parser is several times slower in lexical-binding mode
Date: Sun, 14 Dec 2014 09:08:23 -0500
>>> Not sure if I should close this issue now or at least wait until
>>> byte-compile--use-old-handlers is nil by default.
>> won't run on older Emacsen.  I think I'd like to see 24.4 release before
>> making the change.  But please do set it in your .emacs ASAP so it gets
>> wider testing.
> Just a reminder: I've been running with that setting for a while now,

So have I.

> with no problems that I can remember.

Well, I fixed a bug in it a while ago (some issue with "volatile", fixed
before the 24.4 release), but other than that, indeed it works fine.

The default value has been changed in "master" a couple months ago, already.


        Stefan




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 12 Jan 2015 12:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 9 years and 78 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.