I gave a number of talks this spring on jQuery and especially on some of the recent additions made in jQuery 1.4. Below are all the slides and demos that I've given.
The conferences / meetups that I spoke at (or will speak at, in the case of MIX), and the talks that I gave, are as follows:
Webstock (Wellington, NZ) (Introduction to jQuery Workshop, Things You Might Not Know About jQuery)
Future of Web Apps (Miami, FL) (Introduction to jQuery Workshop, Improve Your Web App with jQuery)
MIX (Las Vegas, NV) (Improve Your Web App with jQuery)
Introduction to jQuery Workshop
This workshop starts with an introduction to the fundamentals of jQuery (1 hour) and continues on with two pieces of hands-on coding (Todo list, 30 min, Social Networking Site, 1.5 hours).
In the workshop I also had two pieces of hands-on coding. The first was an ajax-y todo list the second was converting a functional social networking site into a one page application (making significant use of jQuery UI).
A variety of things that people don't know about in jQuery - including new things added in jQuery 1.4 (and newer), data bindings, custom events, and special events.
For the first jQuery Boston Meetup I built a game using the avatars of everyone in attendance. Sort of a space shooter style game you need to kick and kill the advancing hordes of users. I used this game as a way of demonstrating constructing an application that makes use of custom events, data binding, and building applications in an event-centric manner.
I've been messing around with a new piece of presentation software that I wrote for these talks. It's still terribly crude and buggy (pretty much just got it working enough in order to run my talks in Firefox 3.6 and Chrome) - you've been warned. I hope to refine it at some point and release it for general consumption.
A new method signature is slated for jQuery 1.4: .closest(Array). It builds upon the previous .closest() method and hyper-optimizes the logic needed for handling event delegation (and live events).
closest() (and by extension, is()) has become a critical function in jQuery. With more people using live events reducing any overhead has become of the utmost importance. Every time an event fires that live is bound to (such as click or mousemove) jQuery uses the closest() method to go from the target element and find the nearest element that matches the specific selector. The more handlers that are bound, though, the more computationally expensive it becomes.
In jQuery 1.4 we've added closest(Array) which gives us the ability to batch these selector checks together and reduce the amount of time that we spend traversing the DOM.
For the jQuery project we've run all of our community discussions through Google Group mailing lists for the past three years. At this moment the main jQuery group is the second most popular programming group (next to Android developers) clocking in at over 21,000 members. We also have the jQuery Dev and jQuery UI groups. The main jQuery group averages around 83-143 messages per day. I also use Google Groups for discussion on a number of my other projects (Processing.js, Env.js, Sizzle.js, and TestSwarm).
This post isn't so much about the usefulness of mailing lists as a discussion medium, it's the complete failure of Google Groups as an adequate purveyor of public discussion software. For the jQuery project we're already in the process of moving the full discussion area to a forum that we control. We should have it set up, and everything moved over, within the next month or two.
There is one area in which Google Groups continues to shine: Private, or restricted, mailing list discussions. However any attempts at using it for a public discussion medium are completely futile.
The primary problem with Google Groups boils down to a systemic failure to contain and manage spam. Only a bottom-up overhaul of the Google Groups system would be able to fix the problems that every Google Group faces.
To better illustrate the problem, let's step through the common experience of running a Google Group.
The Beginning
When you create a public group everything will go well for a couple days, at most. Without fail an onslaught of spam will start to come through your group - I've even seen it happen within the first day. It happens to every group and doesn't matter how well you advertise it (or try to hide it). After having watched Google Groups for as long as I have I can only assume that there exists no spam filtering whatsoever. Or, if there is any, it's the most grossly incompetent spam filter I've ever seen.
When these spam messages start to come to your group a couple things will happen. First, you may not even notice the spams coming through. Since you're likely reading the list in a competent email client (such as Gmail) it'll detect the messages and dump them into your personal spam folder. Don't be surprised if you visit your group and see a pile of spammy messages sitting there greeting your new visitors.
Most email client spam detection software is smart. It looks for common points of failure and tries to take care of the root problem. One such tactic is to realize that a lot of spam is coming from a single address (like a Google Group) and start to flag most of it as it comes through (regardless of the actual content). The result is that much of your list is being flagged as a false positive. In the case of Gmail people will then start to un-flag the falsely-binned group messages. This works well until the system starts to think that all group messages are ok - and here comes the spam again.
To fight the spam you'll likely start flagging emails as "spam" in the groups interface. This works well (the user is permanently banned and the message deleted) - until a couple hours pass, that is. You'll see the spammer return, with a slightly different username, posting the same exact spam messages. Flagging a user/message as spam does absolutely nothing to train the groups spam detection system (for reasons that aren't entirely clear and only be explained by incompetence).
It's a horrible game of cat and mouse with the spam destroying the quality of your group. It's at this point that you say "enough is enough" and you turn on moderation for your group.
Moderation
Google Group moderation seems like a palatable idea but in practice is aggravating and crippling. To start, it creates a horrible first-participation experience for your users. For example, let's say you go to bed at the same time as someone in Tokyo attempts to post a message to the group; you won't be able to moderate the message through for many hours (and that's assuming that you moderate messages during your work day). While the experience is much worse than instant posting it is par for the course for most moderation systems.
Of course, this would assume that Google Groups actually informs the users that their message has been held in moderation. Looking through the moderation queue you can see users attempting to submit their message over-and-over again, wondering why it isn't working. Eventually they'll just give up in frustration.
In order to combat this you'll typically need to bring on a bunch of people to help with the moderation duties. In the case of the main jQuery and jQuery UI groups we divvy up the moderation based upon the time of day and week (and where the moderator lives). This is incredibly frustrating but still manageable.
This moderation load also assumes that you are able to successfully navigate the abysmal Google Groups moderation user interface. It's a horrible quagmire of radio buttons and un-evenly spaced rows with no visual delineation. I've provided an example of the interface below (BEWARE: Contains Not Safe For Work text).
When you begin moderating all the radio buttons start on the "Ignore" column, it's your duty to move all the messages to the right columns. The "Spam" and "Always Allow" columns were added just recently (thank goodness) - the moderation process use to be much worse.
With a user interface this bad mistakes happen. Sometimes spam accidentally slips through, sometimes users get completely banned. I estimate that this happens about once in every couple hundred messages. With 84-143 messages coming to the main jQuery Google Group every day that means that there'll be at least a few users banned and a few spam coming through every week.
While the occasional spam slipping through is a reality of the web, accidentally banning users is unacceptable - but it does happen, even when you don't mean to. For example, here's a message that I got from a user just today:
I seem to have been banned from the jQuery Google Group for a reason
I'm not aware of.
> The owner of this group has banned you from this group.
Not sure if this is another Groups glitch or not. I can still access
the jQuery UI group, though.
My account is under email: XXXXX@gmail.com
Please advise. Thanks.
Nothing quite like insulting, confusing, and scaring your users, due to a poorly-designed user interface and abysmal spam detection. This is the reality that Google Group owners have to live with on a daily basis.
All of this changed a couple weeks ago.
End Game
The final straw was placed upon my patience with the Google Groups system a few weeks ago. Spammers are now spoofing the email addresses of existing group participants to sneak their messages through. Previously you would've seen a delightful "FREE MOVIE DOWNLOADS" spam from "freemovies123@gmail.com" - but now you'll see it coming from existing group users - or even the group moderators themselves. This cheat completely bypasses the moderation system since the spammers are pretending to be pre-moderated users.
The Google Groups system is completely fooled. The spam message comes in claiming to be from an existing group participant - and according to the Google Groups interface there is no difference. If you click the user's name you'll be taken to a full listing of that user's posts (with the spam messages delightfully interspersed).
For example, here's a user whose email address is being spoofed and an email that was actually sent to Google Groups. Note that the actual email is coming from a .ch domain and not from the actual Gmail server.
The only "cure" to this problem is to watch for a spam message to come through and then force that user into a permanent state of moderation. Of course, you then have to be careful not to bring that user into a "pre-approved" state the next time you clean up the moderation queue. There is no way to keep track of which users should be kept in a always-moderate state and which should skip moderation.
At this very moment my own email address, and the email addresses of most of the jQuery group moderators, are being spoofed by spammers. This means that we (the owner and moderators of the group) have to moderate our own messages before we post, for fear of letting a spoofed spam through. This will likely happen indefinitely since Google Groups has been notoriously slow to fix problems with the site.
On top of all of this, Google Groups actually strips out many of the original spam indicators from the message when it re-broadcasts it to the full list. This means that when the message finally arrives at a user's email client it actually looks like it came from the spoofed user. Since I'm currently being spoofed I've actually had a bunch of my legitimate email end up in spam folders as a result. Having my email address become flagged as a spammer is positively infuriating. The fact that Google Groups is silently sitting by and blindly letting this happen communicates one thing to me: Google Groups is dead, time to move on as quickly as possible.
Moving On
I've completely given up on Google Groups - and I'm not the only one. Feel free to ask any Google Group moderator and I'm certain that you'll only get a sad shake of their head. The situation is completely untenable - which is why the jQuery team is actively working to get all our lists off of Google Groups as quickly as possible.
To give you an idea of the overall level of quality that Google Groups exhibits here is an anecdote: A couple weeks ago the jQuery UI Google Group was completely deleted for no apparent reason. It was gone for the better part of a day before it was restored. The only mechanism for contacting support, even in a situation as serious as that, is to post on a public Google Group. We never received a response from an admin regarding the missing group. Are there backups of group data? Who knows! Forget it, life is too short for the stress and aggravation that Google Groups provides.
While Google Groups provides a mechanism for exporting a CSV members list it provides no way to export the full message archive for a group. With over 120,000 messages tied up in the main jQuery Google Group alone it's going to take some significant work to get everything out and move on. Our only avenue of escape (short of screen scraping the entire Google Group archive) is doing an IMAP dump of my personal Gmail account and extracting all the jQuery group posts from it. I'm sure that experience will be absolutely delightful as well.
This past weekend was the 2009 jQuery Conference here in Boston. It was an incredible event - 300 people attended and a ton of discussion, collaboration, and learning happened.
Nearly the entire jQuery project team had the opportunity to meet for two days prior to the conference and hash a number of things out - figuring out most of the planning for the upcoming year. The core dev team also had the opportunity to meet and work for two days just after the conference. We're much closer to shipping 1.3.3 now (which is likely to become 1.4, with all the new features that're being added).
(Left to Right: Mike Hostetler (Infrastructure), John Resig (Core), Paul Bakaus (UI), Brandon Aaron (Core), Richard D. Worth (UI), and Scott Jehl (UI and Design team)
A full list of the presentations that were given can be found on the events site. All the presentations that I gave can be found below.
With jQuery 1.3.2 out the door I've been looking for more ways to profile and optimize jQuery.
Previously I did a survey of jQuery-using sites to figure out which selectors they were using. This led to the construction of the new Sizzle Selector Engine which targeted those selectors for improvement.
Additionally, I constructed a deep profiling plugin for jQuery which helped to spot methods that were taking a long time to run in live jQuery sites. This helped bring about the improvements in jQuery 1.2.6, 1.3, and 1.3.2.
What do we tackle next? A good place to start would be to tackle optimizing methods that are obviously inefficient - but how do we determine that? One way would be to measure the number of function calls that occur every time a method is run. Firebug provides this information in its profiling data (along with how long it takes to run each method). Unfortunately it's very clunky to manually type out code, check the results in the console, and determine if they're bad or if they've changed. If only there was a way to progamatically get at those numbers.
FireUnit Profiling Methods
Yesterday I did some work to make getting at the profiling data possible, adding two new methods to FireUnit.
fireunit.getProfile();
Run this method after you've run console.profile(); and console.profileEnd(); to get a full dump of the profiling information. For example, given the following profile run:
You'll get the following JavaScript object returned from fireunit.getProfile():
The second method added to FireUnit provides an easy way to execute and profile a single function. Roughly, this method starts the profiler, executes the function, stops the profiler, and then returns the results from getProfile(). Additionally, it watches for any exceptions that might be thrown and makes sure that the profiler is cleanly turned off anyway (a frequent frustration of mine).
First, you'll need to be sure to have the latest copy of FireUnit installed. I've built a copy of the latest code, into an extension, if you wish to install it:
Both the Console and Script tabs are enabled in Firebug
That the 'extensions.firebug.throttleMessages' property in 'about:config' is set to 'false'.
The Results
I put up a test page so that I could quickly run through some jQuery methods to see how they stacked up.
Here are the results of running against jQuery 1.3.2 ("Method" is the jQuery method that was called, with the specified arguments, "Calls" is the number of function calls that occurred when executing the method, "Big-O" is a very rough Big-O Notation for the function calls):
We can immediately see, by looking at the big-O notation, that most jQuery methods execute at least one function for every element that they have to operate against. addClass runs about 6 functions per element, filter runs about 2, and 'is' runs only 1.
We can see the problematic functions sticking out like a massive sore thumb: .remove(), .empty(), and .html() - they all run over n2 function calls, which is a huge issue. (These numbers are all large for a simple reason: .html() uses .empty(), .empty() uses .remove(), and .remove() is obviously inefficient.) While function calls do not, necessarily, indicate slow code (a lot of jQuery's internal functions are pretty lightweight) it is very likely to indicate inefficiently-written code.
I poked around the code for a little bit and realized that .remove() could be dramatically simplified. I filed a ticket and landed a patch which resulted in these much-improved numbers:
Method
Calls
Big-O
.remove();
298
3n
.html("<p>test</p>");
507
5n
.empty();
200
2n
I'm really excited by this new tool. Automating the process of code profiling opens up whole avenues of exploration. Even using nothing more than the above tool I can immediately see room for improving just about every jQuery method.
It's also be very interesting to have this running in some sort of continuous integration setting, to catch any egregious regressions - but I'll leave that for another day.
It's really hard to answer the question: Was a release successful? In the jQuery project we try to look at a number of criteria.
Are users pleased with the release?
Are users adopting the release?
Are we meeting the needs of those who don't use jQuery?
It's hard to put exact numbers on those points (we listen very closely to the response on our blog, on twitter, on the mailing list, and elsewhere - and thus far it's been very positive) but we do have a couple tools that we use to try and simplify that process, namely: Google Analytics and Google Trends.
Above is a Google Analytics comparison of Jan 2009 (blue) to Dec 2008 (green) for jquery.com.
We're currently seeing a +30% growth in visitors day to day over December. I'm really pleased to see that the 1.3 release was "sticky" (users liked what they saw and stuck around to keep using it - the daily numbers aren't dropping to their pre-release levels).
Note the increase of bounce rate and decrease in pages/visit and avg. time on site - both were linked to the 1.3 and 1.3.1 releases where people come to check out the release then leave again.
The 14th of January was the 1.3 release (had a lot of traffic that week - we hit Ajaxian, Reddit, Hacker News, and a number of blogs). We hit Digg on the week of 26th and saw no appreciable gain in traffic.
I trimmed out the Christmas-New Years time frame since traffic was very low (and doesn't make for a good comparison).
None of this data includes jQuery UI or static files which are tracked separately.
Google Trends has helped us to learn some things about the use of the library. The biggest of which is the "Christmas slump."
jQuery users largely appear to use it during their day jobs (see the analytics stats to see the weekend slumps). Every year at around the December holiday season (Dec 23rd to Jan 3rd) we see a major drop-off in traffic - we can see a direct correlation within the Google Trends stats, as well.
One thing that I've learned while managing jQuery it's that there's a huge potential to lose users in between projects. A developer ends a project and then re-evaluates his tool chain to see if any improvements can be made. Every time a user finishes a project there's a possibility that they'll leave for another tool - it's our job to make sure that we consistently provide the best tool and experience possible so that the need doesn't arise (better documentation, better code, frequent releases, etc.).
A very similar problem occurs over the holiday break. The users are away from their code for about 1-2 weeks and when they come back they have a chance to choose another tool, pick up where they left off, or to become engaged and continue strong.
The question now becomes: How well can we retain (and hopefully grow) the userbase over this slump?
If we look at the slump from 2006 to 2007 we see an immediate pick-up again after the users return from their breaks. The reason? jQuery 1.1 was released.
But look at 2007 to 2008 - there was almost no pick-up and it took almost half a year to get back to the point at which growth had resumed. Incidentally, there was no significant release in January.
We fixed that this time around - we released jQuery 1.3. Note how we instantly picked up our users and even grew our share during that time period.
From a growth perspective I'm very pleased with the 1.3 release - I think we're setting ourselves up for an outstanding 2009. It's likely that we'll be pushing another follow-up release (1.3.2) this week to address one last 1.3 regression - but other than that, it looks like we're in the clear to heads toward some solid new features and fixes in 1.3.3 and beyond.
Apple, and the WebKit team, have recently proposed two different additions to CSS: CSS Transitions and CSS Animations.
The two specifications are confusingly named - and it's hard to tell what the difference is between them at first glance. However, to put it simply: CSS Transitions are easy to use, while CSS Animations are made for programmers.
CSS Transitions
CSS Transitions provide you with the ability to force CSS property changing to occur smoothly over a period of time, rather than immediately and coarsely.
For example if you were to set elem.style.width = "500px";, and its current width was 100px then the element would, normally, jump up to 500px wide. With a CSS Transition it would smoothly move from 100px to 500px - the full CSS required would look something like this:
This would make it such that any manipulation of that element's width would be done as a smooth transition. Note that no other properties are listed and, thus, are not affected. You can list as many properties as you wish, in a list: "width, heigh, opacity" (for example).
CSS Transitions complement the existing tools that we have for working with CSS from JavaScript. Any changes to a CSS property still work - they just happen much more smoothly.
Of course transitions can also be injected dynamically from JavaScript, like so:
Since you can hook in these custom transitions it makes it possible to use them directly from JavaScript and within frameworks. Which leads to the question: Can the core of JavaScript animation frameworks be replaced with CSS Transitions, if they exist?
We had this discussion recently on the jQuery dev list and one user, Jonah, implemented a quick proof of concept to demonstrate how it CSS Transitions would work within jQuery. He also wrote up a stress test to see how it scaled.
Give the stress test a try in WebKit (or on the iPhone, if you have access to one) and you'll definitely note an increase in animation smoothness - especially when a large number of animations are being run.
So why not just add in the CSS Transition code today?
There are a large number deal-breaking gotchas:
You can't stop an animation that's already running.
You get no feedback as to how the animation is running - only an event callback once it has completed.
There's no way to synchronize multiple animations.
There's no way to specify custom easing functions.
The inability to stop an animation is absolutely killer - and a huge requirement for any JavaScript framework that would be looking to adapt this as part of their code base.
That being said, the current transition code does have its place (namely within iPhone development) so I wouldn't be surprised to see a solid jQuery plugin popup that iPhone devs start to use.
CSS Animations
CSS Animations are a second proposal from Apple/WebKit that embodies a much-more-complex way of doing animations. To give you an idea of the level of power that's provided observe this quote from the proposal:
"Many aspects of the animation can be controlled, including how many times the animation iterates, whether or not it alternates between the begin and end values, and whether or not the animation should be running or paused. An animation can also delay its start time."
The ability to pause animations is crucial - and immediately makes CSS Animations a more-viable candidate for use from JavaScript frameworks.
Here's an example of how you would set up a CSS Animation, from the proposal:
div {
animation-name: 'diagonal-slide';
animation-duration: 5s;
animation-iteration-count: 10; }
@keyframes 'diagonal-slide' {
from { left: 0; top: 0; }
to { left: 100px; top: 100px; } }
CSS Animations also provide a greater number of callbacks (letting you know when an animation has started, every step of the animation, and when it has ended) which can be important for doing animation synchronization.
Although the CSS Animation proposal has a number of things going against it:
Its syntax and usability is far more confusing than that of the, relatively simple, CSS Transitions.
It includes the concept of keyframes - while I'm sure this might be useful for someone I just can't see a large enough benefit for such a large feature.
Honestly, at this point, I'd prefer to see CSS Transitions come around, but with a few additions:
The ability to pause or stop transitions.
Animation start, step, and end callbacks (along with information about how far along the animation is).
The ability to provide a custom JavaScript function which would provide a custom easing function.
#1 and #2 are much more important here - but seeing both of those additions would help to make CSS Transitions actually a viable tool for web developers (not to mention that it would provide the best of both worlds - ease of use for non-framework-using developers and power and control for those using frameworks).
It's not often that new user interface conventions are born - or popularized. Even less so within the realm of web development. I'd argue that Sparklines and Lightbox are two of the best examples of UI conventions that were popularized on the web.
Recently Maxime Haineault announced a simple jQuery plugin for inputting a new time of day called jQuery.timepickr.js. Its principles are very similar to jQuery itself: Get users to input the time as simply as possible with as little input as possible.
To achieve this he made a "two click" time picker. The first click is within the time field. This activates the display and allows the user to choose the time - all of which is done by moving the mouse over the times that you desire. The final click is anywhere - filling in the time that was chosen. It's hard to explain, you simply have to try it.
One thing that you'll notice using it is that it's fast. Very fast. I'd argue much faster than clicking into the input area, moving to the keyboard for entering the time, typing the time, then moving back to mouse.
But not only is it faster, but it's also quite intuitive - which is rather rare for something that utilizes a completely new user interface convention.
A nice extra point is that the input is completely styled using the jQuery UI style conventions - which means that you'll be able to customize it completely with the jQuery UI Themeroller.
I love the Themeroller and use it all the time to customize UI controls. It beats the pants off of any other UI customization tool that I've seen. Definitely give it a whirl. Consistently styling JavaScript user interface components can be incredibly annoying, but the Themeroller helps to make it sane - which is just perfect.
