Blog


Spicing Up Embedded JavaScript

June 15th, 2008

One thing that I absolutely adore is the sheer embeddability of JavaScript. It's a (comparatively) simple language that is fiercely flexible. I tend to liken JavaScript to water - alone it's painfully simple but it can take the form of its container - and mixing it with anything enhances its flavor.

Since JavaScript, alone, is so dumb (we've become spoiled by browsers which provide DOM, setTimeout/setInterval, and even a global object reference - none of which are necessarily required my an ECMAScript implementation) we must rely upon the spice of 'larger' languages to help it gain some taste.

We can start by taking a look at some of the most popular languages that are available today: Python, Perl, PHP, Ruby, and Java -- all of which have, at least, one embeddable JavaScript interpreter.

One thing is the same in all the interpreter implementations, as well, they all have the ability to introduce (at least) simple objects into interpreter (from the parent language) and extract values again. Sometimes this may be as simple as executing a JavaScript function and getting its return value (which is often translated from its internal JavaScript form back into the native language).

There's a couple points upon which I like to evaluate the embeddability of a JavaScript interpreter, specifically:

  1. Object Translation: If objects/values are passed to/from the interpreter to/from the native language - is that translation handled automatically?
  2. Simplicity: How hard is it to get up-and-running? (Is extra compilation required or is it written using native language code?)
  3. Bleed-through: Can JavaScript communicate to the base language or is it a one-way-street?

The first point is the easiest one to find compatibility with - virtually all embeddable JavaScript interpreters do some form of object translation. Some do it better (like JE and Rhino) but it generally shouldn't be a problem for simple scripts.

PHP

» PHPJS

A flexible, pure-PHP, JavaScript interpreter that even accepts compiling to an intermediary bytecode. There is no bleed-through but good object translation.

include "js.php";

# Introduce a new JavaScript function, named print,
# which calls a PHP function
function show($msg) {
  echo "$msg\n";
}

$jsi_vars["print"] = "show";

# Add in a new pure-JavaScript function which calls
# our previously-introduced print function
jsc::compile("function alert(msg){ print('Alert:' + msg); }");

# Prints out 'Alert: text'
js_exec("alert('text');");

» J2P5: Pure-PHP ECMAScript Engine

Probably the simplest of the available interpreters - there doesn't appear to be a clear way of communicating from JavaScript-to-PHP (or vice versa). Although, it is implemented in pure-PHP which allows for some nice cross-platform compatibility.

include "js/js.php";

$script = <<<EOD
  print("Hello from JavaScript");
EOD;

js::run($script);

Perl

» JavaScript::SpiderMonkey

An embedding of Mozilla's Spidermonkey JavaScript engine into Perl. Since it uses Spidermonkey it'll require that extra source code and compilation. Object translation is good but there's no bleed-through (JavaScript calling native Perl functionality that isn't explicitly defined).

use JavaScript::SpiderMonkey;

# Instantiate the interpreter
my $j = JavaScript::SpiderMonkey->new();

# Introduce a new JavaScript function, named print,
# which calls a Perl function
$j->function_set("print", sub { print @_, "\n" } );

# Add in a new pure-JavaScript function which calls
# our previously-introduced print function
$j->eval("function alert(msg){ print('Alert:' + msg); }");

# Prints out 'Alert: text'
$j->eval("alert('text');");

» JE: Pure-Perl ECMAScript Engine

A pure-Perl JavaScript engine - shows incredible promise (probably my favorite new engine in addition to Rhino). Has tremendous potential (including the ability to serialize the runtime environment!).

use JE;

# Instantiate the interpreter
my $j = new JE;

# Introduce a new JavaScript function, named print,
# which calls a Perl function
$j->new_function(print => sub { print @_, "\n" } );

# Add in a new pure-JavaScript function which calls
# our previously-introduced print function
$j->eval("function alert(msg){ print('Alert:' + msg); }");

# Prints out 'Alert: text'
$j->method(alert => "text");

# So does this
$j->eval("alert('text');");

# Introduce the ability to do remote requests
use Net::FTP;

$j->bind_class( package => "Net::FTP", name => "FTP" );

# Execute native Perl functionality (FTP module)
# directly from JavaScript
$j->eval(<<'--end--');
  var connect = new FTP("ftp.mozilla.org");
  connect.get("index.html");
--end--

Note the extreme bleed-through that you can achieve with JE (in addition to excellent object translation).

Java

» Rhino

Probably one of the most famous embeddable JavaScript implementations. Implemented in pure Java has excellent object translation and perfect bleed-through. The bleeding can even be controlled explicitly at runtime (to create a more confined environment).

In the following code (using in some of the jQuery build system) two functions are defined which utilize a number of native Java packages - all without ever writing a single line of Java code.

importPackage(java.io);

function writeFile( file, stream ) {
  var buffer = new PrintWriter( new FileWriter( file ) );
  buffer.print( stream );
  buffer.close();
}

function read( file ) {
  var f = new File(file);
  var reader = new BufferedReader(new FileReader(f));
  var line = null;
  var buffer = new java.lang.StringBuffer(f.length());
  while( (line = reader.readLine()) != null) {
    buffer.append(line);
    buffer.append("\n");
  }
  return buffer.toString();
}

Python

» Python-Spidermonkey

Previously this was one of the most depressing embedding efforts (and, really, the only one available for Python) but it has since had some new life blown into it - which is much appreciated. It supports decent object translation and bleed-through (including the ability to call native classes, much like in JE).

from spidermonkey import Runtime

j = RunTime().new_context()

# Introduce a new JavaScript function, named print,
# which calls a Python function
j.bind_callable("print", lambda msg: print(msg));

# Add in a new pure-JavaScript function which calls
# our previously-introduced print function
j.eval_script("function alert(msg){ print('Alert:' + msg); }");

# Prints out 'Alert: text'
j.eval_script("alert('text');");

Ruby

While there was a previous effort to embed Spidermonkey in Ruby it is largely defunct now, superseded by the new (strangely named) Johnson project.

» Johnson

This is a, relatively, new project that's working to completely overhaul how communication is currently done in-between Ruby and JavaScript (using Spidermonkey to power the engine). There is complete bleed-through: Ruby can dig in and manipulate JavaScript objects at run-time and JavaScript can call back and access native Ruby functionality. Additionally they're examining the ability to provide native browser environments using the env.js script that I created last year. While the cross-platform ease-of-use isn't there yet the project absolutely has a ton of potential.

require "johnson"

j = Johnson::Runtime.new

# Introduce a new JavaScript function, named print,
# which calls a Ruby function
j.print = lambda {|msg| puts msg};

# Add in a new pure-JavaScript function which calls
# our previously-introduced print function
j.evaluate("function alert(msg){ print('Alert:' + msg); }");

# Prints out 'Alert: text'
j.evaluate("alert('text');");

Update: Another example of Johnson in action from John Barnette:

require "johnson"

class Monkey
  def eat
    puts "Nom, nom, nom, bananas!"
  end
end

j = Johnson::Runtime.new

# toss a monkey into the runtime
m = Monkey.new
j[:m] = m

# add a JSland function to our monkey...
j.evaluate("m.chow = function() { this.eat() }")

# ...and now it's available as an instance method on our native Ruby object!
m.chow

There's a ton of potential in all of these projects - providing interesting features for blurring the lines in-between JavaScript and different host languages. For your next project it should be pretty easy to find a solution to embedding JavaScript as your quick-and-dirty scripting language of choice.

Tags: ruby, java, python, php, perl, javascript

Running Java in JavaScript

April 28th, 2008

Welcome Waxy.org and Slashdot readers. I blog about JavaScript, like it's my job, feel free to subscribe for a ton more posts like this.

Related Posts:

When I was in Tokyo this past fall speaking at the Shibuya.JS user group I had the opportunity to see a number of interesting JavaScript projects that have yet to make it outside of Japan.

One project, in particular, really caught my eye. It's called Orto [PDF, Japanese] and is an implementation of the Java Virtual Machine (JVM) in JavaScript. This means that you can take an existing Java application, compile it to bytecode, run it through Orto (which produces the JavaScript, and embed it in a web page. While it doesn't provide the full capabilities of most Java code it does provide enough to make for some interesting demos.

The one demo that was presented was that of a real-time, interactive, Tetris game:

If you take a peak at the generated source code you can see some interesting tidbits (obviously all of the code is meant to be 'machine readable' - being just converted bytecode):

"java/lang/Thread 1316742099":function(){var orto333=orto245[0];
var orto336=orto350(orto333);
if(orto336.orto340!=orto310){orto223("java/lang/IllegalThreadStateException",null);
return ;
}
case 117:orto246[orto247-2]={high:(~orto246[orto247-2].high)
  &0xffffffff,low:(~orto246[orto247-2].low+1)&0xffffffff};
if(orto246[orto247-2].low==0){orto246[orto247-2].high++;
orto246[orto247-2].high&=0xffffffff;
orto246[orto247-2].low=0;
}break;
case "CHECKBOX":orto171=orto188["orto/ui/CheckBox"];
break;
case "IMAGE":orto171=orto188["orto/ui/ImageButton"];
break;
case "RADIO":orto171=orto188["orto/ui/RadioButton"];
break;

Unfortunately, the information on the subject is a little scarce (considering that both orto.accelart.jp and orto.jp seem to be missing any relevant information). There are a couple of things that are apparent, however:

  • The result is able to handle threaded application code (translating the threads into a series of yields with setTimeout (mentioned in the presentation and demonstrated in the Tetris example).
  • The application can use regular Java conventions for designing and constructing the UI (as shown here, as well). User Interface components are translated to, similar, HTML ones. It's not apparent to what extent functionality is implemented, but it is to a certain degree.
  • Keyboard interactions are able to be handled and translated to normal Java callbacks.

I was especially pleased with the resulting performance of the demo application. I wasn't expecting a game (translated multiple layers deep) to still be as playable as it is. Obviously the culmination of JavaScript optimization and increases in browser performance have served these types of endeavors well.

Tags: java, javascript, jvm

Why Tamarin instead of...

August 8th, 2007

After yesterday's post on the browser scripting revolution, detailing the new projects being built on top of Tamarin, a number of questions came up concerning the choice of Tamarin instead of other virtual machines. Two engines came up, in particular: The Java Virtual Machine (JVM) - which is already able to run Jython and JRuby, and Mono - which is able to run IronPython and IronRuby.

I'll defer to the words of Mike Shaver and Brendan Eich to explain the reasons as to why, though in a nutshell: The non-technical reasons for choosing Tamarin are over intellectual property and licensing issues and the technical issues are related to compilation speed, file size, and memory footprint.

What's the advantage of Tamarin over integrating the Mono VM into Firefox?

Mike Shaver:

Here are a few, at least as I see them:

* Optimized to run JavaScript and sibling languages, which is our most important language target by a vast margin.
* Licensed appropriately.
* About 1/25 the size, I think (200KB for Tamarin, 5MB for Mono as described by Miguel elsewhere)
* In my coarse measurements, significantly smaller memory footprint.

I was once quite a supporter of getting Mono into our world, including writing a prototype XPCOM binding for it, but I didn't see a path to getting the important factors (performance, licensing, footprint in code and memory) resolved, and I don't think it's much closer today. Nobody in Mono-land was interested enough to contribute to that, which is another counterpoint with Tamarin I suppose, where we have very active contributions from Adobe and others to help us get it in the state we need for it to be a suitable basis for building our whole app on.

It's not like we didn't look hard at Mono, and in the case of many of us lobby hard for licensing and patent concerns to be swept aside. Tamarin is a very good fit for us in a large number of ways, unfortunately including a number of ways in which Mono is not.

[Why doesn't] Mozilla build on the Java platform rather than Tamarin?

Brendan Eich:

Moreover, for Mozilla at least, we absolutely cannot depend on closed source, and we require a non-copyleft BSD license, or at most MPL/GPL/LGPL. Java was not even open source until recently (I don’t remember the date; it was preannounced one too many times :-/), well after we had to make our own plans and commitments.

Finally, in spite of the prospects with JRuby, the JVM really is about Java first and last. Tamarin is about an ECMAScript variant, so it’s a better target now, and more likely to evolve to support JS1 and JS2 in a first class way, than the JVM.

Compilation heroics can help, but the browser will remain an environment where compilation must be very fast. Wherefore our forthcoming work on a trace-based JIT.

Tags: vm, tamarin, mozilla, javascript, mono, ecmascript, java

Bringing the Browser to the Server

July 9th, 2007

This weekend I took a big step in upping the ante for JavaScript as a Language. At some point last Friday evening I started coding and didn't stop until sometime mid-Monday. The result is a good-enough browser/DOM environment, written in JavaScript, that runs on top of Rhino; capable of running jQuery, Prototype, and MochiKit (at the very least).

The implications of this are phenomenal, and I'm not the only one who's interested in it what this could mean for server-side JS development. More on that in a minute, but first here's some sample results from running jQuery:

jQuery

$ java -jar build/js.jar
Rhino 1.6 release 6 2007 06 28
js> load('build/runtest/env.js');
js> window.location = 'test/index.html';
test/index.html
js> load('dist/jquery.js');
// Add pretty printing to jQuery objects:
js> jQuery.fn.toString = DOMNodeList.prototype.toString;
js> $('span').remove();
[ <span#台北Taibei>, <span#台北>, <span#utf8class1>,
  <span#utf8class2>, <span#foo:bar>, <span#test.foo[5]bar> ]
// Yes - UTF-8 is support in DOM documents!
js> $('span')
[  ]
js> $('div').append('<span><b>hello!</b> world</span>');
[ <div#main>, <div#foo> ]
js> $('span')
[ <span>, <span> ]
js> $('span').text()
hello! worldhello! world

On a whim, I then plugged in Prototype and MochiKit, both of which appeared to work OK (I haven't done any significant testing with them - so there's probably gaps). Here's some sample results:

Prototype

$ java -jar build/js.jar
Rhino 1.6 release 6 2007 06 28
js> load('build/runtest/env.js');
js> window.location = 'test/index.html';
test/index.html
js> load('prototype.js');
js> $$('div p')
<p#firstp>,<p#ap>,<p#sndp>,<p#en>,<p#sap>,<p#first>
js> Object.toJSON({foo:'bar',baz:true});
{'baz': true, 'foo': 'bar'}
js> var fn = (function(name,msg){
  print(name + ' ' + msg); }).curry('John');
js> fn('hello!');
John hello!

MochiKit

$ java -jar build/js.jar
Rhino 1.6 release 6 2007 06 28
js> load('build/runtest/env.js');
js> window.location = 'test/index.html';
test/index.html
js> load('Mochikit.js');
js> $$('div')
<div#main>,<div#foo>
js> document.body.innerHTML = '';
js> document.body.appendChild( P( 'test',
  A({href:'http://google.com/'}, 'link')) );
js> document.body.innerHTML
<p>test<a href='http://google.com/'>link</a></p>
js> $$('a')
<a>

I just want to emphasize that these are un-modified copies of jQuery, Prototype, and MochiKit - all running perfectly in this un-natural environment.

When I came up with this idea for an environment, I was mulling over a couple ideas: Namely, better ways of automating tests and ways to bring JS-style DOM/HTML interaction to the server-side. Having a way to bring this popular idiom to established problem sets seemed like a lot of fun.

In short, the following (at the very least) can all get a big dose of JavaScript:

  • Automated Testing
  • Screen Scraping
  • Web Application Development

Now, if you think I'm crazy, I'd like to show you a couple quick examples:

Automated Testing

$ java -jar build/js.jar
Rhino 1.6 release 6 2007 06 28
js> load('build/runtest/env.js');
js> window.location = 'test/index.html';
test/index.html
js> load('dist/jquery.js');
js> load('build/runtest/testrunner.js');
js> load('src/jquery/coreTest.js');
PASS (1) [core] Array.push()
PASS (2) [core] Function.apply()
PASS (3) [core] getElementById
PASS (4) [core] getElementsByTagName
PASS (5) [core] RegExp
PASS (6) [core] jQuery
...

Oh yes, that's right - the full jQuery test suite is now automated and capable of running in Rhino (passing all tests). jQuery served as my initial testbed for development, making sure that I was getting all of my code right. So if you import a copy of jQuery into this environment, it should work "just fine".

By the way, you can try out the automated test suite by getting a copy of trunk/jquery out of SVN, then running make runtest - the results are just awesome.

Screen Scraping

This is one part that works pretty well right now - with the huge caveat that it only works on well-formed XML documents (oops!). I'll be integrating an HTML parser into the code base so that we can make this functionality a little more resilient. In the meantime, here's an example of the sort of scraping that you can do currently:

load("env.js");
window.location = "http://alistapart.com/";
window.onload = function(){
  load("dist/jquery.js");
  print("Newest A List Apart Posts:");
  $("h4.title").each(function(){
    print(" - " + this.textContent);
  });
};

And here's another one that writes the results out to a file:

load("env.js");
window.location = "http://alistapart.com/";
window.onload = function(){
  load("dist/jquery.js");
  var str = "Newest A List Apart Posts:\n";
  $("h4.title").each(function(){
    str += " - " + this.textContent + "\n";
  });
  var out = new XMLHttpRequest();
  out.open("PUT", "file:/tmp/alist.txt");
  out.send( str );
};

Oh yeah, I went there - I made PUT and DELETE requests to local files perform the expected actions. I think the result is hilarious.

Web Application Development

This is still a work in progress, but some of the initial ideas are already at play here in this environment. When I have some time I plan on making a JavaScript-based web app framework out of this - which should be pretty cool.

Here's some psuedo-code for how I think it could work:

window.onload = function(){
  print("Content-type: text/html\n");
  if ( location.href == "/" )
     show_home();
  print( document.innerHTML );
};
function show_home(){
  document.load("index.html");
  document.getElementById("time").innerHTML = (new Date()).toString();
}

Download!

Check out the code - there's still huuuge gaps of functionality missing - I only implemented the bare minimum to get this environment working (and passing the jQuery test suite). So your mileage may vary.

Download: http://jqueryjs.googlecode.com/svn/trunk/jquery/build/runtest/env.js (Formatted)

NOTE (February 2009): The above code is quite out of date. If you're interested in using it in your project I recommend that you visit the following Google group and download the code from the current working fork:

How to Use

To start with, you'll need to have, at least, Rhino 1.6R6. You can download it from Mozilla FTP.

Now download the env.js script and put it in the same directory as the Rhino js.jar.

In order to use it from the command-line, you'll wanna do something like this:

$ java -jar js.jar
js> load('env.js');
js> window.location = 'some.html';
some.html
js> // Your code here!

It's important that you do window.location = "some file" before loading any DOM-dependent code (as the 'document' object doesn't exist before the location request).

A full list of Rhino-shell-specific commands can be found in the Rhino Shell docs.

If you want to write executable scripts, the contents will look something like this:

load('env.js');
window.location = 'some.html';
window.onload = function(){
  // Your code here
};

Which can then run like so: java -jar js.jar myscript.js.

Feedback is very much welcome - I've only thought of a couple use-cases thus far, but I'm sure that the surface is just being scratched.

Tags: firefox, ecmascript, java, rhino, mozilla, javascript

Amazon S3

March 14th, 2006

Amazon S3 was just released today, giving developers/companies the ability to instantly store files/data up to 5GB in size - and as many as you want. They just charge pennies for bandwidth and storage costs.

Are you looking for a new way to make money?

Wrap a quick reverse proxy on top of the public S3 URLs, require any of the above to use a secure password, charge $X dollars/month. Presto, instant data storage company.

Tags: amazon, s3, java, programming, perl, lazyweb

· « Previous entries

JavaScript Books

Secrets of the JavaScript Ninja

JavaScript Secrets

Secret techniques of top JavaScript programmers. Coming Fall 2009.

Pro JavaScript Techniques

Pro JavaScript

The best techniques for professional JavaScript. Published by Apress.

Micro Updates

John Resig Twitter Updates

@jeresig

Infrequent, short, updates and links.

JavaScript Jobs

Ninja Jobs: JavaScript Jobs

Mozilla: Browser Jobs



Hosting provided by: Ruby Hosting by Engine Yard