Ruby VM in JavaScript

April 8th, 2008

Welcome Waxy.org and Slashdot readers. I blog about JavaScript, like it's my job, feel free to subscribe for a ton more posts like this.

Related Posts:

• Running Java in JavaScript
• JavaScript Talk at Northeastern University
• State of ECMAScript 4
• The World of ECMAScript
• JavaScript As A Language

Note: I'm not the creator of HotRuby, as mentioned elsewhere - it is the work of a highly-skilled Japanese developer.

HotRuby is a project which aims to port the Ruby Virtual Machine over to ECMAScript (allowing it to run, directly, in a browser using JavaScript or indirectly using ActionScript in Flash).

Currently the code works by using Ruby 1.9's YARV (Yet Another Ruby VM) to compile down a Ruby script into opcodes, which are then serialized and passed to the browser for execution. It's a little bit indirect but it still capable of creating a compelling result.

If you were to run one of the examples in your browser the actual chain of execution would be something like this:

1. Script finds <script type="text/ruby"></script> tags and extracts the inline Ruby code from them.
2. The Ruby code is sent to the server via an XMLHttpRequest.
3. The server-side CGI script (in Ruby, using Ruby 1.9) compiles the incoming Ruby into its associated opcodes and serializes it into a JSON data structure.
4. The browser consumes the opcodes, translating it into JavaScript, and executes it.

To observe this full process we can take a look at the code in the provided benchmark and watch its full path through the server and final execution:

For example, observe this portion of the CGI script:

and a sample of the opcode data returned by the server:

["YARVInstructionSequence\/SimpleDataFormat",1,1,1,{"arg_size":0,"local_size":4,"stack_max":3},"","src","top",["startTime","sum","endTime"],0,[["break",null,"label_21","label_29","label_29",0]],[2,["putnil"],["getconstant","Time"],["send","new",0,null,0,null],["send","to_f",0,null,0,null],["setlocal",4],4,["putstring",""],["setlocal",3],"label_21",5,["putobject",50000],["send","times",0,["YARVInstructionSequence\/SimpleDataFormat",1,1,1,{"arg_size":1,"local_size":1,"stack_max":2},"block in ","src","block",["e"],[1,[],0,0,-1,-1,3],[["redo",null,"label_0","label_22","label_0",0],["next",null,"label_0","label_22","label_22",0]],["label_0",5,["getdynamic",3,1],["getdynamic",1,0],["send","to_s",0,null,0,null],["send","+",1,null,0,null],["dup"],["setdynamic",3,1],"label_22",["leave"]]],0,null],"label_29",["pop"],7,["putnil"],["getconstant","Time"],["send","new",0,null,0,null],["send","to_f",0,null,0,null],["setlocal",2],9,["putnil"],8,["getlocal",2],["getlocal",4],["send","-",1,null,0,null],["send","to_s",0,null,0,null],["putstring"," sec"],9,["send","+",1,null,0,null],["send","puts",1,null,8,null],8,["leave"]]]

and you can find the full client-side virtual machine here: HotRuby.js.

Perhaps most fascinating about this, though, is the speeds that are able to be achieved with this script. Granted, the above benchmark is rather contrived, but the end performance results are quite fascinating:

We can see a 2.71x speed improvement from Firefox 2 to Firefox 3 and a 5x performance improvement over regular Ruby 1.8.2, running on the command-line.

It's a fascinating time to be working with JavaScript. The performance improvements that are being provided to us by the browser afford us a realm of possibility that wasn't, previously, viable. The fact that we're even discussing running a virtual machine, implemented in JavaScript, is quite impressive. I'm curious to see what applications end up being built with this implementation - and within what context they end up using it.

Tags: javascript, ecmascript, ruby, vm

39 Comments on 'Ruby VM in JavaScript'

Classy Query

April 1st, 2008

Over the past two years, seeing hundreds of thousands of people use jQuery, a major point has become apparent: jQuery simply isn't able to scale to handle the development needs of most power users. It lacks the clarity and power functionality that most developers need in order build applications in a collaborative environment.

For this reason I've put a lot of work into a new library (which sits on top of jQuery), called:

Classy Query

» View Source to Classy Query

If one thing has become apparent to me it's that users enjoy working with the typical Class-style of object creation and inheritance. Because of this I've constructed the entirety of Classy Query in an classical manner (you can inherit and override any piece of functionality that you desire).

Before we go too far, let's take a look at the type of clearly-defined code that you can now write with Classy Query:

You can even sub-class functionality, creating your own pieces of encapsulation. All of the class and inheritance functionality is taken care of by the Simple JavaScript Inheritance code released last week.

Additionally you can treat pieces of functionality as if they were attachable behaviors, like so:

There's a number of core features encompassed in Classy Query that really help to transform the traditional jQuery experience into something much more usable.

Class creation

The full Simple JavaScript Inheritance API is included in Classy Query, allowing you to extend and create JavaScript 'classes', like so:

Inheritance

It's possible to inherit from any existing piece of Classy Query functionality, supporting a true 'programming in the large' mindset. You can inherit from, or completely override, existing methods with ease (especially since they're all classes).

Behaviors

Behaviors tend to encapsulate a portion of functionality which is applied against an element (such as making a table sortable or making an item draggable). Classy Query supports a technique to make this particular implementation trivial.

No more 'this'

The 'this' keyword is confusing and frequently misused. To counter this all access to elements (within functions) is done by the first argument, rather than through 'this'.

Re-Structuring

The hierarchy of jQuery has been completely re-organized. Rather than having a single, flat, interface through which to access all methods functionality has been broken down into individual groupings of methods. These groupings tend to, also, coordinate with the jQuery Documentation for convenience.

Additionally, nearly all the method names have been re-named in order to provide a greater level of clarity to developers. Frequently it was found that the concise method names of jQuery provided too much confusion to developers just getting started with the library. The end result is a library that is easier to read and understand for developers.

The full list of categories and changed methods can be found in the following table:

querySelector and querySelectorAll

Finally, rather than having a generic, catch-all, method as jQuery selector functionality is deferred to the new querySelector and querySelectorAll methods. These methods only support pure CSS 1-3 selectors (nothing extra) in an attempt to avoid any unpleasant situations when a browser's native querySelectorAll is implemented.

» View Source to Classy Query

I hope this particular library will be of use to the general jQuery-using population. I hope we can start to have an open dialog, moving forward, getting everyone to adopt a more standard approach to JavaScript development. There's no reason why we shouldn't be using this tried-and-true approach to application development. It's worked in an untold number of existing applications, it's bound to work here as well. Enjoy.

Tags: javascript, jquery, programming

65 Comments on 'Classy Query'

The getBoxObjectFor Apocalypse

March 31st, 2008

A long-standing feature within Mozilla's rendering engine has been the getBoxObjectFor method. This particular method was a way for XUL elements to efficiently determine their position, amongst other things. A couple of years ago this feature started to be used by the general web-developer world. This was quickly realized to be a major mistake.

Mozilla developer, Robert O'Callahan, outlined the concerns quite well:

• It's not even a quasi-standard.
• It's not a very convenient API to use or implement because it introduces an
  extra object.
• In Mozilla it's only available when XUL is enabled, so some embedded Geckos
  don't have it --- actually, it's worse, they have the function, but it always
  fails (such as in Camino).
• When it is available, it contains a bunch of XUL-specific methods that don't
  make sense for HTML but people might try to use anyway.

A problem, though, was that no one was particularly excited about removing the method since there was no alternative to jump to. That's no longer the case - now that there's the CSSOM View getBoundingClientRect method in Firefox 3, which fills this gap quite nicely.

And now, getBoxObjectFor is being phased out, in two steps:

1. It's being deprecated in Firefox 3. Any attempts to use it will throw a warning to the console (but continue to work).
2. It will be removed, completely, in the next version of Firefox.

Because of this you should begin moving to getBoundingClientRect tout d'suite. Although, if you've been writing your code using good object detection practices then this shouldn't be much of a problem.

The Relationship between HTML and XUL

In order to understand how a feature like this could have been introduced in the first place you need to have an understanding of how Firefox (and most other Mozilla-platform-based applications) are constructed.

For those not familiar with it, XUL is a markup language designed for constructing user interfaces. In many ways it behaves very similarly to HTML (it renders some things very similarly - and it fully supports CSS, JavaScript, and the DOM). The Mozilla platform uses XUL to render all of its user interface components - and uses the same exact rendering engine to render the interface as to render the HTML pages that you view every day.

This means that there's one rendering engine displaying everything that you see when browsing the web, with Firefox. Because of this tight relationship between the two markups (and their associated CSS properties and DOM methods) there's a lot of work done to make sure that only the correct functionality is exposed in the right situations. However, this was not always the case.

The issue that occurred with getBoxObjectFor isn't that it was accidentally exposed (in that it occurred as an oversight) but that it was made available without first consulting the correct standards bodies or attempting any type of web-accessible-content moderation. The Mozilla project used to be very loose when exposing XUL/Mozilla-specific methods (in my opinion) however that has greatly changed. Newly-exposed functionality is thoroughly examined, analyzed, tested, and vetted for all possible uses - in addition to having active conversations with the appropriate standards bodies (W3C, WHATWG, ECMA, etc.).

This movement is incredibly important for a browser to make. Properly analyzing features will avoid painful situations like the one we're encountering now (in which users of an undocumented method now have to adapt their applications). This is not to say that this is a particularly new thing for Mozilla to do, but it's just that we're getting better at the process (more eyes watching tends to help, as well).

We can see this process in action with a feature that didn't make it in to Firefox 3: Web-accessible native JSON support. While native JSON support was added to the Mozilla platform well in time for Firefox 3 (it's used extensively to power backend features) it didn't make the cut for being made accessible to web pages, mostly for two reasons:

1. Lack of standardization. The ECMA TG1 committee (which is working to standardize ECMAScript 4) was originally attempting to include native JSON support in the next version of the language - however that fell apart and it was deemed to be 'up to the implementors' if they wanted to include it.
2. Lack of proper security vetting. Exposing a whole new API to web applications opens a whole new vector for possible security problems. To counter any issues there needs to be serious analysis before making anything public.

For both of these reasons it was decided that it was better to play it safe with this particular feature and hold off until the next release of the browser (when there would be more time to properly analyze its full impact).

This vetting process is now quite widespread in the Mozilla world, along with the practice of extensive automated testing, and it's really fantastic. The result is a browser that's more stable and secure but still able to provide desirable innovative features that web developers need.

Tags: javascript, mozilla, firefox, xul

3 Comments on 'The getBoxObjectFor Apocalypse'

JavaScript Talk at Northeastern

March 27th, 2008

Yesterday I gave a presentation for the local ACM of Northeastern University. I covered the basics of JavaScript - targeted to a Computer Science major (in the case of the students at NU, they learn Scheme and Java so I emphasized the subject matter to that audience). I did a fast run-through of the whole language, covering the main points, and move on to a quick inspection of jQuery and the DOM - closing with some Q&A. The subject matter of JavaScript 2 (and 1.6-1.8) came up a couple times so discussion of that was interspersed throughout. There were a bunch of jQuery users in the audience and it was great to get to hear their feedback.

JavaScript and jQuery for CS Majors

Also available in OGG Theora.

Here are the full slides, as well. I left in the slides where I typed out code, so that you can be able to reference it.

Download PDF

Tags: javascript, programming, presentation

16 Comments on 'JavaScript Talk at Northeastern'

getElementsByClassName pre Prototype 1.6

March 26th, 2008

There's an important compatibility regression coming up with the releases of Firefox 3 and Safari 3.1 (and any other browser that will natively implement getElementsByClassName) concerning old releases of Prototype (pre 1.6).

In talking with Tobie Langel, of the Prototype team, he recommends the following:

We've already deprecated document.getElementsByClassName and Element#getElementsByClassName [in Prototype 1.6].

The easiest workaround, which does not require upgrading, is to replace calls to document.getElementsByClassName('foo') with $$('.foo') and calls to Element#getElementsByClassName('foo') with Element#getElementsBySelector('.foo') or Element#select('.foo') (version >= 1.6.0) depending on which version of Prototype you are running.

A number of users noticed this problem during the Firefox 3 betas and after the release of Safari 3.1:

• Mozilla Tracking Bug
• Google Groups Discussion concerning Safari 3.1
• Post concerning Safari 3.1

If you have a site that's using an old version of Prototype, and getElementsByClassName, then it's recommended that you either upgrade or perform the above steps (probably sooner, rather than later, so you don't hit compatibility problems).

How This Issue Came About

It's important to realize how a problem like this can come about so that you can prepare for it within your development. This core issue arises from the fact that Prototype was implementing a property on the global document variable which was, eventually, codified into the HTML 5 standard and implemented in some browsers.

Looking at their code we can see something like this:

This code in, and of, itself isn't necessarily the problem. If you implement a JavaScript library that completely, 100%, implements a specification then it's probably safe to do the above in that manner (base2 is a good example of a library that does this well).

A problem occurred because, in their implementation of the method, they were adding new methods to the returned array (such as .each). This meant that users began to use the method in this manner:

Which subsequently broke when an actual getElementsByClassName implementation came on the scene (since the result returned from the method wasn't 'enhanced' with the each functionality - amongst other methods).

Attempting to implement any sort of future specification can be a real fools game. Since standards are often imperfect (such as not providing an .each mechanism) or frequently change - attempting to match that functionality with a library can be incredibly difficult. More often than not it's simply a better idea to implement brand new methods that can never conflict with upcoming functionality (as the Prototype team has done in the 1.6 release).

Tags: javascript, browsers, prototype

22 Comments on 'getElementsByClassName pre Prototype 1.6'

Keypress in Safari 3.1

March 21st, 2008

There was an interesting post the other day which criticized Safari's choice to completely overhaul their event system and, in the process, seemingly cripple the keypress event for non-character keys.

At first glance to me, and to many others, it appeared as if a dramatic flaw had been introduced into Safari's event system - completely castrating its practical use. It was at this point that I contacted Yehuda Katz (Merb and jQuery contributor) - he knows more about the inanity of cross-browser keyboard events than anyone I know. Here's his responses to a few questions that I bounced off him:

John: At first glance through WebKit's changes it appears as if they've significantly crippled the keypress event, is this the case?

Yehuda: People should not have been using the keypress event to get the character that was pressed. That's because the keydown event only provides information about the actual key that was pressed (the A key, the right arrow, etc.), but does not tell the user what character will get added to (for instance) an input box.

Since arrow keys do not get added as text, keydown provides all the information that is needed. There were a couple of quirks with using keydown in certain cases previously which are resolved by their changes that prevent keypress from getting called if keydown's default handling is blocked.

What this means is that if people were using keypress before (and relying on Safari's native results for arrow keys, such as 63232), their code will break. However, this was a bad idea all along; people should use keydown to detect and block non-character keys. My mantra has always been: keydown/keyup if you want to know the key that was pressed; keypress if you want to know what text was detected.

John: Being able to access non-character keys must be useful - granted Safari gave strange results previously - but at least some value is better than none, right?

Yehuda: The keypress event does not provide any additional benefits over keydown for these non-character keys. Keep in mind that IE does not fire the keypress event for some non-character keys (e.g. delete, end, enter, escape, fkeys, etc.).

Because Safari appears to have fully resolved issues with suppressing default events from keydown, there should be no problem with using keydown for non-character keys.

John: So we have charCode, keyIdentifier, and keyCode along with keyup, keydown, and keypress - is there a solution that we can safely use in a cross-browser manner?

Yehuda: There's a reasonably good writeup at: http://unixpapa.com/js/key.html.
Effectively, you can use e.keyCode for all modern browsers in keydown/keyup (although you'll need to consult the chart at the bottom of that page to determine whether the keyCodes returned by the browsers are consistent).

For keypress, e.charCode || e.keyCode should work (Mozilla and Safari define e.charCode, while IE defines e.keyCode which returns the ASCII value). What that means is that you have a close-to-cross-browser keyCode for keyup/keydown and a pretty reasonable charCode for keypress.

As a result, keypress can be used for key events that produce text for which you need to know its ASCII value (A vs. a). keyup/keydown can be used for events for which you care about the physical key that was pressed. Again, there are some quirks, which can be summed up in the tables at http://unixpapa.com/js/key.html.

I'd like to thank Yehuda for taking the time to clear these points up. Effectively if you were doing key-related development, previously, in a non-cross-browser manner then you hit a wall with this recent change. If nothing else this should encourage you to keep a good separation, as Yehuda notes: "keydown/keyup if you want to know the key that was pressed; keypress if you want to know what text was detected."

Tags: safari, javascript, dom, events, webkit

9 Comments on 'Keypress in Safari 3.1'

Simple JavaScript Inheritance

March 20th, 2008

I've been doing a lot of work, lately, with JavaScript inheritance - namely for my work-in-progress JavaScript book - and in doing so have examined a number of different JavaScript classical-inheritance-simulating techniques. Out of all the ones that I've looked at I think my favorites were the implementations employed by base2 and Prototype.

I wanted to go about extracting the soul of these techniques into a simple, re-usable, form that could be easily understood and didn't have any dependencies. Additionally I wanted the result to be simple and highly usable. Here's an example of what you can do with it:

A couple things to note about this implementation:

• Creating a constructor had to be simple (in this case simply providing an init method does the trick).
• In order to create a new 'class' you must extend (sub-class) an existing class.
• All of the 'classes' inherit from a single ancestor: Class. Therefore if you want to create a brand new class it must be a sub-class of Class.
• And the most challenging one: Access to overridden methods had to be provided (with their context properly set). You can see this with the use of this._super(), above, calling the original init() and dance() methods of the Person super-class.

I'm pleased with the result: It helps to enforce the notion of 'classes' as a structure, maintains simple inheritance, and allows for the super method calling.

Simple Class Creation and Inheritance

And here's the implementation (reasonably sized and commented well) - clocking in at around 25 lines. Feedback is welcome and appreciated.

In my opinion the two trickiest parts are the "initializing/don't call init" and "create _super method" portions. I want to cover those briefly so that you will have a good understanding of what's being achieved in this method.

Initialization

In order to simulate inheritance with a function prototype we use the traditional technique of creating an instance of the super-class function and assigning it to the prototype. Without using the above it would look something like this:

What's challenging about this, though, is that all we really want is the benefits of 'instanceof', not the whole cost of instantiating a Person object and running its constructor. To counteract this we have a variable in our code, initializing, that is set to true whenever we want to instantiate a class with the sole purpose of using it for a prototype.

Thus when it comes time to actually construct the function we make sure that we're not in an initialization mode and run the init method accordingly:

What's especially important about this is that the init method could be running all sorts of costly startup code (connecting to a server, creating DOM elements, who knows) so circumventing this ends up working quite well.

Super Method

When you're doing inheritance, creating a class that inherits functionality from a super-class, a frequent desire is the ability to access a method that you've overridden. The final result, in this particular implementation, is a new temporary method (._super) which is only accessible from within a sub-classes' method, referencing the super-classes' associated method.

For example, if you wanted to call a super-classes' constructor you could do that with this technique.

Implementing this functionality is a multi-step process. To start, note the object literal that we're using to extend an existing class (such as the one being passed in to Person.extend) needs to be merged on to the base new Person instance (the construction of which was described previously). During this merge we do a simple check: Is the property that we're attempting merge a function and is what we're replacing also a function? If that's the case then we need to go about creating a way for our super method to work.

Note that we create an anonymous closure (which returns a function) that will encapsulate the new super-enhanced method. To start we need to be a good citizen and save a reference to the old this._super (disregarding if it actually exists) and restore it after we're done. This will help for the case where a variable with the same name already exists (don't want to accidentally blow it away).

Next we create the new _super method, which is just a reference to the method that exists on the super-class' prototype. Thankfully we don't have to make any additional changes, or re-scoping, here as the context of the function will be set automatically when it's a property of our object (this will refer to our instance as opposed to the super-class').

Finally we call our original method, it does its work (possibly making use of _super as well) after which we restore _super to its original state and return from the function.

Now there's a number of ways in which a similar result, to the above, could be achieved (I've seen implementations that have bound the super method to the method itself, accessible from arguments.callee) but I feel that this technique provides the best mix of usability and simplicity.

I'll be covering a lot more of the nitty-gritty behind the JavaScript prototype system in my completed work but I just wanted to get this Class implementation out there to get everyone trying it out and playing with it. I think there's a lot to be said for simplistic code (easier to learn, easier to extend, less to download) so I think this implementation is a good place to start and learn the fundamentals of JavaScript class construction and inheritance.

Tags: secrets, javascript, book

61 Comments on 'Simple JavaScript Inheritance'

getBoundingClientRect is Awesome

March 18th, 2008

getBoundingClientRect is a method, first introduced by Internet Explorer 5 (and now included in Firefox 3), which provides the top, right, bottom, and left offset coordinates of an element. After PPK mentioned this method in a recent post of his, poo-poo-ing its applicability, I thought I'd take some time to show how useful it can be. What makes this method so especially handy is that the alternative means of computing an element's offset position is so horribly slow and buggy that it borders on the comical.

The general purpose of this method, or of similar implementations, is to allow the user to get the visual top/left position of an element (such that if they absolutely position an element from the document it would be on top of the desired element - making things like tooltips possible, to name a single application). This means that you have to take into account a huge number of quirks. In no particular order here are some choice bugs which make this very difficult:

• Handling table border offsets.
• Fixed positioned elements.
• Scroll offsets within another element.
• Borders of overflowed parent elements.
• Miscalculation of absolutely positioned elements.

For example, looking at jQuery's implementation of .offset() (which provides the top/left offset of an element) we can see a number of these quirks worked around. A phenomenal amount of work has gone into the method, by the excellent Brandon Aaron, in order to "get it right" in a cross-browser manner. Check out the extensive suite of tests that are backing it up, as well.

This is where getBoundingClientRect comes into play. For a visual comparison look at the green boxes (representing the getBoundClientRect-based implementation) versus the red boxes (representing the normal cross-browser implementation).

The reason why the getBoundingClientRect implementation is more than 1 line is due to two facts:

1. It doesn't include the document scroll offset (which is a simple addition).
2. Internet Explorer's handling of <html/> element border widths is completely whack.

In comparison to the default technique, however, this is a veritable walk-in-the-park.

Now, not only, is this method simpler to use it's also insanely faster. Stupifyingly so. Considering that you no longer have to perform all sorts of hacks, or walk the tree to rectify miscalculations, the result is quite impressive.

View the following jQuery UI drag-and-drop sorting demo page in Firefox 2 (drag the items in the grid, at the bottom) and then view it in Firefox 3, doing this same:

http://dev.jquery.com/view/trunk/ui/tests/sortable.html

Night and day. I've also uploaded a simple movie demonstrating the difference.

I absolutely encourage everyone to check this method out. It can serve as a near-drop-in replacement for your offset needs - and will gracefully work in new browsers that support the method.

Tags: dom, javascript, browsers

15 Comments on 'getBoundingClientRect is Awesome'