jQuery 1.3 Aftermath

February 1st, 2009

It's really hard to answer the question: Was a release successful? In the jQuery project we try to look at a number of criteria.

• Are users pleased with the release?
• Are users adopting the release?
• Are we meeting the needs of those who don't use jQuery?

It's hard to put exact numbers on those points (we listen very closely to the response on our blog, on twitter, on the mailing list, and elsewhere - and thus far it's been very positive) but we do have a couple tools that we use to try and simplify that process, namely: Google Analytics and Google Trends.

Above is a Google Analytics comparison of Jan 2009 (blue) to Dec 2008 (green) for jquery.com.

We're currently seeing a +30% growth in visitors day to day over December. I'm really pleased to see that the 1.3 release was "sticky" (users liked what they saw and stuck around to keep using it - the daily numbers aren't dropping to their pre-release levels).

Note the increase of bounce rate and decrease in pages/visit and avg. time on site - both were linked to the 1.3 and 1.3.1 releases where people come to check out the release then leave again.

The 14th of January was the 1.3 release (had a lot of traffic that week - we hit Ajaxian, Reddit, Hacker News, and a number of blogs). We hit Digg on the week of 26th and saw no appreciable gain in traffic.

I trimmed out the Christmas-New Years time frame since traffic was very low (and doesn't make for a good comparison).

None of this data includes jQuery UI or static files which are tracked separately.

Google Trends has helped us to learn some things about the use of the library. The biggest of which is the "Christmas slump."

jQuery users largely appear to use it during their day jobs (see the analytics stats to see the weekend slumps). Every year at around the December holiday season (Dec 23rd to Jan 3rd) we see a major drop-off in traffic - we can see a direct correlation within the Google Trends stats, as well.

One thing that I've learned while managing jQuery it's that there's a huge potential to lose users in between projects. A developer ends a project and then re-evaluates his tool chain to see if any improvements can be made. Every time a user finishes a project there's a possibility that they'll leave for another tool - it's our job to make sure that we consistently provide the best tool and experience possible so that the need doesn't arise (better documentation, better code, frequent releases, etc.).

A very similar problem occurs over the holiday break. The users are away from their code for about 1-2 weeks and when they come back they have a chance to choose another tool, pick up where they left off, or to become engaged and continue strong.

The question now becomes: How well can we retain (and hopefully grow) the userbase over this slump?

If we look at the slump from 2006 to 2007 we see an immediate pick-up again after the users return from their breaks. The reason? jQuery 1.1 was released.

But look at 2007 to 2008 - there was almost no pick-up and it took almost half a year to get back to the point at which growth had resumed. Incidentally, there was no significant release in January.

We fixed that this time around - we released jQuery 1.3. Note how we instantly picked up our users and even grew our share during that time period.

From a growth perspective I'm very pleased with the 1.3 release - I think we're setting ourselves up for an outstanding 2009. It's likely that we'll be pushing another follow-up release (1.3.2) this week to address one last 1.3 regression - but other than that, it looks like we're in the clear to heads toward some solid new features and fixes in 1.3.3 and beyond.

Tags: statistics, data, jquery, opensource

41 Comments on 'jQuery 1.3 Aftermath'

HTML 5 data- Attributes

July 13th, 2008

A new feature being introduced in HTML 5 is the addition of custom data attributes. This is a, seemingly, bizarre addition to the specification - but actually provides a number of useful benefits.

Simply, the specification for custom data attributes states that any attribute that starts with "data-" will be treated as a storage area for private data (private in the sense that the end user can't see it - it doesn't affect layout or presentation).

This allows you to write valid HTML markup (passing an HTML 5 validator) while, simultaneously, embedding data within your page. A quick example:

The above will be perfectly valid HTML 5. This should be a welcome addition to nearly every JavaScript developer. The question of the best means of attaching raw data to HTML elements - in a valid manner - has been a long-lingering question. Frameworks have tried to deal with this in different manners, two solutions being:

1. Using HTML, but with a custom DTD.
2. Using XHTML, with a specific namespace.

The addition of this prefix completely routes around both issues (including any extra markup for validation or needing to be valid XHTML) with this effective addition.

On top of this a simple JavaScript API is presented to access these attribute values (in addition to the normal get/setAttribute):

The .dataset property behaves very similarly to the the .attributes property (but it only works as a map of key-value pairs). While no browsers have implemented this exact DOM property, it's not hugely needed - the above code could be done with the critical line replaced with:

I think what is most enticing about this whole specification is that you don't have to wait for any browser to implement anything in order to begin using it. By starting to use data- prefixes on your HTML metadata today you'll be safe in knowing that it'll continue to work well into the future. The time at which the HTML 5 validator is integrated into the full W3C validator your site will already be compliant (assuming, of course, you're already valid HTML 5 and using the HTML 5 Doctype).

Tags: html5, data, dom, javascript

45 Comments on 'HTML 5 data- Attributes'

Overview of Processing

April 7th, 2008

This weekend I gave two talks at BarCamp Rochester (which was very well put together and quite enjoyable) - one on jQuery and a very quick one on the Processing language. I've deconstructed my slides into some bullet points here. If you're not familiar with the language, or what it's capable of, this should give you a good overview.

Processing is a data visualization programming language.

It has three components:

• The Processing language
• The Processing drawing API
• The implementation (in Java - can optionally pass the drawing API through to OpenGL).

The Processing Language and API

• Strictly typed
• Has classes, inheritance
• Includes a bunch of globally-accessible functions (the drawing API - very flat and PHP-like).

Basic Program Structure

Two core methods: setup() and draw()

• Very OpenGL-like
• draw() is called continually at a specific framerate (if none is specified then it goes as fast as possible)

Simple example: Drawing a continuous line with the mouse.

Initialization

• setup() is called initially
• size(...) - set the width/height of the drawing area
• Can include calls to any other number of methods, such as: background(...) - (which draws and fills a background with a specified color).
• Note: All colors are done in RGBA. background(102) is equivalent to background(102,102,102,255) (opaque gray color)

The draw() loop

• draw() gets called as fast as possible, unless a frameRate is specified (with framerate(20), for example). You can disable any looping by calling noLoop().
• stroke() sets color of drawing outline (the color of lines, points, and the outsides of polygons)
• fill() sets inside color of drawing (inside of polygons)
• mousePressed is true if mouse is down
• Very different from typical asynchronous events - since program keeps looping we get state updates automatically. (Unless you specify mousePressed as a global function - then it'll be called as a callback.)
• mouseX, mouseY - mouse position, pmouseX, pmouseY - previous mouse position in last draw() call

Drawing

Different drawing methods: line(), rect(), arc(), ellipse(), point(), quad(), triangle(), bezier(), etc.

All use stroke(), strokeWeight(), and fill().

Can also draw complex polygons using beginShape, endShapre, and vertex - like in this example.

The Canvas

Very OpenGL-like. You can mutate the canvas rendering using: translate(), scale(), and rotate().

You can also save and restore the state of the canvas using: pushMatrix() and popMatrix().

A basic example using pushMatrix/popMatrix: A movable arm.

Classes

Can hold data, do inheritance.

Example: Bouncing an object off of rocky terrain

Math

A whole mess of math functions are provided, as well: dist(), map(), constrain(), abs(), floor(), ceil(), random(), noise(), atan2(), cos(), sin(), pow(), sqrt(), radians().

Images

Can be used to load in external images. Example: Animation of guy dancing.

Demos

Some fun demos that I really like:

• Zipdecode - rendering all zipcodes in the country and searching them in real time.
• Substrate - rendering a piece of art.
• Genetic Trees - selectively breed and mutate trees.
• World is not round - live VJing a song using Processing and a set of physical input controls.

Books

• Processing: Programming Handbook
• Visualizing Data

Tags: graphics, processing, data, visualization, barcamp

9 Comments on 'Overview of Processing'

JavaScript Engine Speeds

September 17th, 2007

Recently, I've been spending a lot of time analyzing the speed of pure JavaScript engines, looking at how well they perform and what their particular strengths and weaknesses are. To start with, I analyzed the bleeding-edge code from:

• Rhino
• Spidermonkey (Firefox 3)
• Tamarin (Firefox 4?)
• JavaScriptCore (Safari 3)

Right now I'm only looking at pure, JavaScript-only, tests (no tests of DOM or other APIs) and am NOT looking at the speed of the browsers' native JavaScript engine implementations. (So, even though you may see a speed for a particular engine, that does not directly correlate to the speed of the JavaScript running within the browser itself. There's always a significant amount of overhead required to run JavaScript code seurely within a browser, thus the efficiency of that security layer will frequently become a deciding factor in the results.

The four engines that I picked all had complete JavaScript implementations and usable JavaScript shells (that way I could feed my tests in and have them cleanly run).

To browse the results I've pulled together a simple application that can be used to view a representation of the data from all the major JavaScript engines paired with the code from the tests which run them.

Right now the browser works fine in Firefox, is quirky in Opera and Safari, and explodes in IE (it requires canvas support). I'll finesse it into shape when I have a little more time this week.

Note: This demo uses a bunch of functionality from the new jQuery UI library, including themes, tabs, accordion, and resizables.

Tags: analysis, speed, data, javascript, ecmascript

25 Comments on 'JavaScript Engine Speeds'

The Netflix Prize

October 4th, 2006

I don't think I could possibly be any more giddy about something, than how I am concerning The Netflix Prize.

In short: Netflix's vote prediction algorithm gets a deviation of 0.95 stars away from predicting your vote for a movie. If you can do 10% better, they'll give you $1 million dollars.

That's awesome and all, but what's really awesome is their amazing training dataset. This is every data miners wet dream: 100,000,000 votes, 17,000 movies, 250,000 500,000 users.

They have two tests that you can run: One against your known data, and one that you'll submit to Netflix. As far as I can tell, your standing (aka, your current deviation) is made public. The lower your number, the higher your rank. Every year that the algo isn't improved by 10%, $50,000 is paid out to the current leader.

Another thing that I find to be interesting: Netflix gets the score that they do without assuming anything about the movie titles, genre, actors, etc. They just do straight number crunching. I'm impressed.

I've already got some techniques that I wanna try. I've got a feeling that I'm overly optimistic at this point, and that I'm going to be highly disappointed when I see my first score. But first, I have to generate my test bed and get to work, this is so cool.

I don't know what it is with me and large, nicely formatted, datasets, but I don't think there's anything that can get me more excited.

Tags: netflix, data_mining, data

13 Comments on 'The Netflix Prize'

Helping People With Data

September 4th, 2005

From a BoingBoing post made earlier today a worthy cause came to my attention. There have been mountains of people in, and around, New Orleans who need to find out if they're relatives are OK - or even tell their relatives that they are OK. This data exists in poor databases that need to be improved - and the quality of the data better structured. If there are any data fans in the audience, then this is for you:

Social Source Foundation, CivicSpace Labs and Salesforce.com Foundation are working with a wide community to solve a simple problem.

Refugee records are in databases spread across the web. What if everyone published their data in a standard format into a central database? A refugee could look in one place for records from across the web.

We have released a data standard and specs for populating a central database. We need community organizers to lead the effort of populating the central database.

I highly recommend that you give this site a visit and see what you can do.

Tags: scrape, database, hurricane, support, data

Comment on 'Helping People With Data'

Dictionaries and Word Lists

August 4th, 2005

The other day I was working on a new application which needed to process large batches of words - as comprehensively as possible. After some quick searches I found that there are (unsurprisingly) a number of freely available dictionary/wordlist files available on the Internet.

The first repository that I tried was that of one hosted on Sourceforge, simply called 'Wordlist'. Many of the lists hosted on that page are spell-checker centric, but the 12 Dicts package, in particular, was rather comprehensive. It originally contained 12 dictionaries, which has since been pruned down. Within the package there are a number of different dictionaries, some contain old English words, some have hyphenated words, some have acronyms, etc. You need to use the grid, that they provide, to determine which package is best suited for you. After doing some work with this list, however, I determined that it simply wasn't comprehensive enough for me (at 74,000 words).

After some more digging I came across the public domain list called ENABLE, which is overwelmingly comprehensive. This particular list is used in just about every word game on the planet - containing approximately 173,000 words! This particular list is very clear-cut and has no limitations imposed as to the words contained within it. If you need a word list for any of your upcoming projects, I highly recommend it!

Tags: data, words, dictionary

3 Comments on 'Dictionaries and Word Lists'

Universal Transverse Mercator (UTM)

July 15th, 2005

A side project that I'm currently working on, involving maps, had a bit of data that I was unfamiliar with. I'll start by saying that I'm very familiar with Latitude/Longitude - it's something that everyone learns in school. However, the data that I was provided with was of an entirely different sort and labeled as 'Easting' and 'Northing'. A couple Google searches later brought me to a an article explaining the concept of something called Universal Transverse Mercator (UTM). The article is very math heavy, but the premise is: The earth is broken down into tiny 'zones' around the ecuator and extending northing and south. Within these zones a measurement, in metres, is taken both to the north and to the east of the ecuator. So your final figure is something like Zone: 17, Easting: 300,000, Northing: 4,000,000 (I just made those numbers up). The reasoning for this is that if your measurements are within a tinier 'slice' of the world, your results immediately become more accurate (especially considering the curvature of the earth). Here's a figure showing all the different zones, taken from this page on different coordinate systems:

Now, using that chart above I determined the zone in which I needed to run all my coordinates through - but the problem is: How to convert them into Lat/Long - which every piece of mapping software understands. There are a number of Java Applets which will do this task, but I wanted something that I could automate. This is where Perl, as always, comes to the rescue - there is a module for it! The module is very lightweight, it simply implements the algorithms specified in the article, mentioned before. Useing that module I was able to quickly run through my data and get perfect numbers out - concluding my UTM adventure. As always, more information can be found on Wikipedia.

Tags: maps, geography, data, conversion, utm, geo

5 Comments on 'Universal Transverse Mercator (UTM)'