JavaScript Function Call Profiling

February 23rd, 2009

With jQuery 1.3.2 out the door I've been looking for more ways to profile and optimize jQuery.

Previously I did a survey of jQuery-using sites to figure out which selectors they were using. This led to the construction of the new Sizzle Selector Engine which targeted those selectors for improvement.

Additionally, I constructed a deep profiling plugin for jQuery which helped to spot methods that were taking a long time to run in live jQuery sites. This helped bring about the improvements in jQuery 1.2.6, 1.3, and 1.3.2.

What do we tackle next? A good place to start would be to tackle optimizing methods that are obviously inefficient - but how do we determine that? One way would be to measure the number of function calls that occur every time a method is run. Firebug provides this information in its profiling data (along with how long it takes to run each method). Unfortunately it's very clunky to manually type out code, check the results in the console, and determine if they're bad or if they've changed. If only there was a way to progamatically get at those numbers.

FireUnit Profiling Methods

Yesterday I did some work to make getting at the profiling data possible, adding two new methods to FireUnit.

fireunit.getProfile();

Run this method after you've run console.profile(); and console.profileEnd(); to get a full dump of the profiling information. For example, given the following profile run:

You'll get the following JavaScript object returned from fireunit.getProfile():

fireunit.profile( fn );

The second method added to FireUnit provides an easy way to execute and profile a single function. Roughly, this method starts the profiler, executes the function, stops the profiler, and then returns the results from getProfile(). Additionally, it watches for any exceptions that might be thrown and makes sure that the profiler is cleanly turned off anyway (a frequent frustration of mine).

You would use it like this:

How To Use

First, you'll need to be sure to have the latest copy of FireUnit installed. I've built a copy of the latest code, into an extension, if you wish to install it:

• http://fireunit.org/fireunit-1.0a1.xpi

When running it you'll need to make sure that:

1. Both the Console and Script tabs are enabled in Firebug
2. That the 'extensions.firebug.throttleMessages' property in 'about:config' is set to 'false'.

The Results

I put up a test page so that I could quickly run through some jQuery methods to see how they stacked up.

Here are the results of running against jQuery 1.3.2 ("Method" is the jQuery method that was called, with the specified arguments, "Calls" is the number of function calls that occurred when executing the method, "Big-O" is a very rough Big-O Notation for the function calls):

We can immediately see, by looking at the big-O notation, that most jQuery methods execute at least one function for every element that they have to operate against. addClass runs about 6 functions per element, filter runs about 2, and 'is' runs only 1.

We can see the problematic functions sticking out like a massive sore thumb: .remove(), .empty(), and .html() - they all run over n² function calls, which is a huge issue. (These numbers are all large for a simple reason: .html() uses .empty(), .empty() uses .remove(), and .remove() is obviously inefficient.) While function calls do not, necessarily, indicate slow code (a lot of jQuery's internal functions are pretty lightweight) it is very likely to indicate inefficiently-written code.

I poked around the code for a little bit and realized that .remove() could be dramatically simplified. I filed a ticket and landed a patch which resulted in these much-improved numbers:

I'm really excited by this new tool. Automating the process of code profiling opens up whole avenues of exploration. Even using nothing more than the above tool I can immediately see room for improving just about every jQuery method.

It's also be very interesting to have this running in some sort of continuous integration setting, to catch any egregious regressions - but I'll leave that for another day.

Tags: fireunit, firebug, javascript, jquery

46 Comments on 'JavaScript Function Call Profiling'

jQuery 1.3 Aftermath

February 1st, 2009

It's really hard to answer the question: Was a release successful? In the jQuery project we try to look at a number of criteria.

• Are users pleased with the release?
• Are users adopting the release?
• Are we meeting the needs of those who don't use jQuery?

It's hard to put exact numbers on those points (we listen very closely to the response on our blog, on twitter, on the mailing list, and elsewhere - and thus far it's been very positive) but we do have a couple tools that we use to try and simplify that process, namely: Google Analytics and Google Trends.

Above is a Google Analytics comparison of Jan 2009 (blue) to Dec 2008 (green) for jquery.com.

We're currently seeing a +30% growth in visitors day to day over December. I'm really pleased to see that the 1.3 release was "sticky" (users liked what they saw and stuck around to keep using it - the daily numbers aren't dropping to their pre-release levels).

Note the increase of bounce rate and decrease in pages/visit and avg. time on site - both were linked to the 1.3 and 1.3.1 releases where people come to check out the release then leave again.

The 14th of January was the 1.3 release (had a lot of traffic that week - we hit Ajaxian, Reddit, Hacker News, and a number of blogs). We hit Digg on the week of 26th and saw no appreciable gain in traffic.

I trimmed out the Christmas-New Years time frame since traffic was very low (and doesn't make for a good comparison).

None of this data includes jQuery UI or static files which are tracked separately.

Google Trends has helped us to learn some things about the use of the library. The biggest of which is the "Christmas slump."

jQuery users largely appear to use it during their day jobs (see the analytics stats to see the weekend slumps). Every year at around the December holiday season (Dec 23rd to Jan 3rd) we see a major drop-off in traffic - we can see a direct correlation within the Google Trends stats, as well.

One thing that I've learned while managing jQuery it's that there's a huge potential to lose users in between projects. A developer ends a project and then re-evaluates his tool chain to see if any improvements can be made. Every time a user finishes a project there's a possibility that they'll leave for another tool - it's our job to make sure that we consistently provide the best tool and experience possible so that the need doesn't arise (better documentation, better code, frequent releases, etc.).

A very similar problem occurs over the holiday break. The users are away from their code for about 1-2 weeks and when they come back they have a chance to choose another tool, pick up where they left off, or to become engaged and continue strong.

The question now becomes: How well can we retain (and hopefully grow) the userbase over this slump?

If we look at the slump from 2006 to 2007 we see an immediate pick-up again after the users return from their breaks. The reason? jQuery 1.1 was released.

But look at 2007 to 2008 - there was almost no pick-up and it took almost half a year to get back to the point at which growth had resumed. Incidentally, there was no significant release in January.

We fixed that this time around - we released jQuery 1.3. Note how we instantly picked up our users and even grew our share during that time period.

From a growth perspective I'm very pleased with the 1.3 release - I think we're setting ourselves up for an outstanding 2009. It's likely that we'll be pushing another follow-up release (1.3.2) this week to address one last 1.3 regression - but other than that, it looks like we're in the clear to heads toward some solid new features and fixes in 1.3.3 and beyond.

Tags: statistics, data, jquery, opensource

41 Comments on 'jQuery 1.3 Aftermath'

CSS Animations and JavaScript

November 11th, 2008

Apple, and the WebKit team, have recently proposed two different additions to CSS: CSS Transitions and CSS Animations.

The two specifications are confusingly named - and it's hard to tell what the difference is between them at first glance. However, to put it simply: CSS Transitions are easy to use, while CSS Animations are made for programmers.

CSS Transitions

CSS Transitions provide you with the ability to force CSS property changing to occur smoothly over a period of time, rather than immediately and coarsely.

For example if you were to set elem.style.width = "500px";, and its current width was 100px then the element would, normally, jump up to 500px wide. With a CSS Transition it would smoothly move from 100px to 500px - the full CSS required would look something like this:

This would make it such that any manipulation of that element's width would be done as a smooth transition. Note that no other properties are listed and, thus, are not affected. You can list as many properties as you wish, in a list: "width, heigh, opacity" (for example).

CSS Transitions complement the existing tools that we have for working with CSS from JavaScript. Any changes to a CSS property still work - they just happen much more smoothly.

Of course transitions can also be injected dynamically from JavaScript, like so:

Since you can hook in these custom transitions it makes it possible to use them directly from JavaScript and within frameworks. Which leads to the question: Can the core of JavaScript animation frameworks be replaced with CSS Transitions, if they exist?

We had this discussion recently on the jQuery dev list and one user, Jonah, implemented a quick proof of concept to demonstrate how it CSS Transitions would work within jQuery. He also wrote up a stress test to see how it scaled.

Give the stress test a try in WebKit (or on the iPhone, if you have access to one) and you'll definitely note an increase in animation smoothness - especially when a large number of animations are being run.

So why not just add in the CSS Transition code today?

There are a large number deal-breaking gotchas:

1. You can't stop an animation that's already running.
2. You get no feedback as to how the animation is running - only an event callback once it has completed.
3. There's no way to synchronize multiple animations.
4. There's no way to specify custom easing functions.

The inability to stop an animation is absolutely killer - and a huge requirement for any JavaScript framework that would be looking to adapt this as part of their code base.

That being said, the current transition code does have its place (namely within iPhone development) so I wouldn't be surprised to see a solid jQuery plugin popup that iPhone devs start to use.

CSS Animations

CSS Animations are a second proposal from Apple/WebKit that embodies a much-more-complex way of doing animations. To give you an idea of the level of power that's provided observe this quote from the proposal:

"Many aspects of the animation can be controlled, including how many times the animation iterates, whether or not it alternates between the begin and end values, and whether or not the animation should be running or paused. An animation can also delay its start time."

The ability to pause animations is crucial - and immediately makes CSS Animations a more-viable candidate for use from JavaScript frameworks.

Here's an example of how you would set up a CSS Animation, from the proposal:

CSS Animations also provide a greater number of callbacks (letting you know when an animation has started, every step of the animation, and when it has ended) which can be important for doing animation synchronization.

Although the CSS Animation proposal has a number of things going against it:

1. Its syntax and usability is far more confusing than that of the, relatively simple, CSS Transitions.
2. It includes the concept of keyframes - while I'm sure this might be useful for someone I just can't see a large enough benefit for such a large feature.

Honestly, at this point, I'd prefer to see CSS Transitions come around, but with a few additions:

1. The ability to pause or stop transitions.
2. Animation start, step, and end callbacks (along with information about how far along the animation is).
3. The ability to provide a custom JavaScript function which would provide a custom easing function.

#1 and #2 are much more important here - but seeing both of those additions would help to make CSS Transitions actually a viable tool for web developers (not to mention that it would provide the best of both worlds - ease of use for non-framework-using developers and power and control for those using frameworks).

Tags: jquery, javascript, animations, css

29 Comments on 'CSS Animations and JavaScript'

Picking Time

November 11th, 2008

It's not often that new user interface conventions are born - or popularized. Even less so within the realm of web development. I'd argue that Sparklines and Lightbox are two of the best examples of UI conventions that were popularized on the web.

Recently Maxime Haineault announced a simple jQuery plugin for inputting a new time of day called jQuery.timepickr.js. Its principles are very similar to jQuery itself: Get users to input the time as simply as possible with as little input as possible.

To achieve this he made a "two click" time picker. The first click is within the time field. This activates the display and allows the user to choose the time - all of which is done by moving the mouse over the times that you desire. The final click is anywhere - filling in the time that was chosen. It's hard to explain, you simply have to try it.

One thing that you'll notice using it is that it's fast. Very fast. I'd argue much faster than clicking into the input area, moving to the keyboard for entering the time, typing the time, then moving back to mouse.

But not only is it faster, but it's also quite intuitive - which is rather rare for something that utilizes a completely new user interface convention.

A nice extra point is that the input is completely styled using the jQuery UI style conventions - which means that you'll be able to customize it completely with the jQuery UI Themeroller.

I love the Themeroller and use it all the time to customize UI controls. It beats the pants off of any other UI customization tool that I've seen. Definitely give it a whirl. Consistently styling JavaScript user interface components can be incredibly annoying, but the Themeroller helps to make it sane - which is just perfect.

Tags: javascript, jquery, ui

27 Comments on 'Picking Time'

Ultra-Chaining with jQuery

October 14th, 2008

We were having a discussion, the other day, on the jQuery-dev mailing list concerning style and jQuery code. There was some discussion about how it could be improved.

One of the points discussed was concerning the use of callbacks and jQuery(this). Callbacks (passing in a function as the final argument to a jQuery method, to be called later) are used all throughout jQuery code. It's a rather core aspect of jQuery (especially the use of closures to pass around information).

We began to wonder: What would jQuery look like if there were no callbacks? A couple solutions were proposed but I posited one that, I think, promotes the idioms present in jQuery.

Its use is simple (although its implementation is definitely a mind-twister). Instead of using a callback you now using jQuery chaining to execute all the methods that you need.

Where previously you would've done:

You would now write:

The end result is quite interesting. It definitely moves jQuery farther into the land of a "Domain Specific Language." I wanted to put this code out there for people to try out and play with even though I have a number of reservations:

1. I'm really not a fan of passing in a method name as a string argument, to another method. This is used a lot in Prototype.js but it just never sat right with me.
2. The .chain() method name probably isn't right. Some other name probably fits better here.
3. The fact that you lose out on the power of normal JavaScript is daunting (you can no longer do something like: jQuery(this).html( jQuery(this).attr("href") ) - you'd have to go back to using a callback).

Nevertheless I think the result holds a lot of potential even if just as a proof-of-concept.

Tags: javascript, jquery

49 Comments on 'Ultra-Chaining with jQuery'

jQuery.embrace().extend();

October 2nd, 2008

These past couple days have been a blast. jQuery Conference this past Sunday followed by 3 days of The Ajax Experience here in Boston.

I ended up giving 9 talks and was on 2 panels in 3 days - completely exhausting. We were able to get video of a bunch of the jQuery Conference so we should be posting it soon. It was great getting to see the whole jQuery team again. We talked a lot, did a lot of planning, and are ready to tackle the work in front of us.

Here are the presentations that I gave, if you're interested:

• State of jQuery '08 (15min - jQuery Conf + Ajax Exp)
• Internals of jQuery + Cool Stuff (60min - jQuery Conf + Ajax Exp)
• JavaScript Library Overview (90min)
• Intro to jQuery (60min)
• What's Happening with Firebug? (5min, All hands-on)
• Visual Programming with JavaScript (60min - jQuery Conf, 90min - Ajax Exp)

I alluded to some big news on Twitter a couple weeks ago and was able to announce it at the jQuery Conference: Microsoft and Nokia are both adopting jQuery as part of their official development platforms.

It's been a crazy couple days tracking all the feedback from the announcement (as the links below will surely attest to) but one thing is clear: People are really excited.

I was amused by the handful of posts from commenters, who obviously had never used jQuery, that were worried about Microsoft "embracing and extending" jQuery. Honestly, there is nothing that I would like better for that to happen. Considering that jQuery has a fantastic plugin architecture in place we fully expect Microsoft and Nokia to join the legion of other happy jQuery users who've constructed powerful jQuery plugins to suit their specific needs.

Considering that it's possible to extend jQuery core, events, animations, and selectors all with explicitly-defined and documented APIs it seems pretty safe to assume that we'll continue to make jQuery as extensible as possible - keeping our core tight, optimized, and designed with the developer in mind.

Here's to jQuery in 2009!

Press:

• Slashdot
• ZDNet
• Information Week
• eWeek
• C|Net
• CMS Watch
• Infoworld

Blogs:

• Scott Guthrie, Microsoft
• Scott Hanselman, Microsoft
• Dare Obasanjo, Microsoft
• Sitepoint
• Ajaxian
• Webmonkey
• InsideRIA

Tags: jquery, javascript, conferences, presentations

17 Comments on 'jQuery.embrace().extend();'

Adv. JavaScript and Processing.js

September 22nd, 2008

Recently I gave two talks at the Web 2.0 Expo in New York City and one for the Boston IxDA.

Learning Advanced JavaScript

An advanced talk on the JavaScript language. Explored functions, closures, function prototypes, and inheritance. The entire presentation was given using an interactive site/presentation (tested in Firefox and Safari).

Feel free to browse through the presentation (I'm not sure how useful it will be without me talking about the particulars - but it may be nice).

There are a number of neat things that I like about the implementation of this talk:

• It's interactive. Each code slide is executable (the user can see the output right away). Additionally each slide is editable - just double-click the code to go into an edit mode.
• Code editing is simple. Basic IDE functions (auto-indentation, proper tabbing, and backspace-to-delete-tab) are included. It's not a ton but it's enough to get started.
• All code slides include syntax hilighting.
• All slides are bookmarkable.

The presentation includes a number fill-in-the-blank quizzes to help test your knowledge of what you just learned. In practice I may save this for situations in which more people have laptops/computers at the talk.

You can download the full presentation as a zip file.

Building a Visualization Language

I gave a talk on my work with Processing.js, together with covering how the Canvas element works and the Processing language itself.

• View on Slideshare
• Download PDF

During the talk I stepped through the construction of a visualization using Canvas:

• View Canvas/JavaScript Version
• View Processing.js Version

jQuery for the Boston IxDA

An introductory presentation explaining how jQuery works.

• View on Slideshare
• Download PDF
• Demo Code

The meat of the presentation was a series of interactive slides which could be run and played with in order to better understand how jQuery works.

You can download the runnable code as a zip file.

Upcoming

I'm going to be giving a number of talks this weekend at the jQuery Conference followed by The Ajax Experience, I'll be sure to post the slides and code from them, as well.

Bonus

Last week was the MIT Career Fair - I stopped by and worked the Mozilla booth with Boris, Brad, and Julie - a good time as usual:

Tags: jquery, javascript, processing, canvas

44 Comments on 'Adv. JavaScript and Processing.js'

querySelectorAll in Firefox 3.1

August 20th, 2008

A brand-new implementation of the Selectors API has landed in the latest Firefox nightlies (and in Firefox 3.1a1) - on track to head your way in the upcoming Firefox 3.1 release.

I've talked about this API before (1, 2) and while I do have some misgivings about the current API (which will be remedied in upcoming revisions of the spec) there is one thing that is undeniable about it: It is extraordinarily fast.

Thankfully, implementations haven't scarified specification compatibility for performance and we can see both the Firefox and WebKit implementations coming in at 99.3% passing the Selectors API test suite. Opera is working on their implementations, slated for Opera 10, and Microsoft has an implementation in beta 1 of Internet Explorer 8. This means that by late this year all browsers will have an implementation of the Selectors API in the market.

JavaScript libraries have already been working to utilize this new API, preparing for when it'll eventually be ready for all to use. The current score is:

• Dojo has querySelectorAll support in Dojo 1.1.1, although support for Safari 3.1 is disabled (there were troublesome crashing bugs in early versions of Safari 3.1 that have since been resolved).
• Prototype has querySelectorAll support in their Git repository (presumably to be rolled into their next release).
• jQuery has querySelectorAll support in an experimental plugin (to land in the next release).

This has lead to some interesting numbers (utilizing the same testing techniques employed by the WebKit team):

That means that libraries that utilize querySelectorAll will be running 2-6x faster than their previous versions. This is already quite impressive.

There are two points to consider when using this API:

1. That you need to try and keep the overhead on top of the querySelectorAll method as low as possible.
2. That it becomes advantageous to avoid the querySelectorAll API in some extreme cases (for example, jQuery avoids it for #id queries, allowing it to go over 10x faster than querySelectorAll).

A lot of bare-bones selectors library implementations are going to look something like this:

Note two points: There's a try-catch block there to capture any syntax errors that are generated by querySelectorAll (syntax errors could be generated by APIs that the implementation doesn't understand - like jQuery's div:first, for example). If no exception is thrown while retrieving the results we need to convert it into an array (most libraries convert result sets into arrays - or bless them in some manner).

Tackling both of these points will introduce some level of overhead in a library (on top of the native querySelectorAll implementation). Of course it's never as simple as it should be, many libraries extend these return sets with additional functionality so the overhead will be that much greater.

Regardless it's readily apparent that this API will be quite instrumental in trivializing one of the most difficult parts of implementing a new JavaScript library. Everything after this is just gravy.

Tags: mozilla, javascript, jquery, selectors, firefox

20 Comments on 'querySelectorAll in Firefox 3.1'