Projects
December 17th, 2005
This script is capable of taking an RSS (or Atom) feed and converting it into a valid JSON object. I currently cache feed requests once per hour, to save on bandwidth - so keep that in mind when developing an application.
You can use the interface that I have setup, below, or you can download the script itself (written in Perl) and run it on your own server (which is preferable).
Interface
This script currently has a REST interface, accessible via a GET request. The full request would look something like this:
GET http://ejohn.org/apps/rss2json/?url=URL&callback=CALLBACK
the URL parameter would contain the URL of the RSS/Atom feed which you are attempting to convert. The optional Callback parameter would reference a callback function that you wish to have called, with the new data.
You can test this out by visiting the following URL:
http://ejohn.org/apps/rss2json/?url=http://ejohn.org/index.rdf
Sample Code and Demo
A simple, sample, program would look something like this:
getRSS
("http://digg.com/rss/index.xml", handleRSS
);
function handleRSS(rss) {
alert( "Dowloaded: " + rss.title );
}
function getRSS(url, callback) {
feedLoaded = callback;
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = "http://ejohn.org/apps/rss2json/?url=" + url
+ "&callback=feedLoaded&t=" + (new Date()).getTime();
document.getElementsByTagName("head")[0].appendChild(script);
}
I've also setup a more advanced demo, which uses some tweaked code from the del.icio.us JSON page.
Also, the del.icio.us popular widget that I made using the Google Homepage API also uses the RSS to JSON convertor. Taking a quick peak at the source code shows it in use.
Download
If you wish to run your own copy of the RSS to JSON script, you can download it from here. It's a Perl-based CGI script which requires that the following modules be installed: LWP and XML::Feed.
This work is licensed under a Creative Commons Attribution 2.5 License.
Tags: javascript, json, perl, rss, convert
16 Comments on 'RSS to JSON Convertor'
July 10th, 2005
This is a hack that brings the power of address translation (converting a US Postal Address into a Latitude/Longitude) to the Google Maps API - something that wasn't provided in the default distribution.
View the Demo! - Download the Code
This hack, which is completely reusable, is broken down into a couple portions.
Address Translation Proxy (written in Perl)
This portion of this project queries the open API provided by Geocoder, which offers free address translation for any postal address in the United States. (If you live in Canada, you may want to check out Geocoder.ca). The code is very very simple, the only reason why it's needed is due to the fact that Javascript applications can't make queries to services that aren't on the same domain. The code is so short, I can show it here:
#!/usr/bin/perl
use CGI;
use LWP::Simple;
my $cgi = new CGI();
my $a = $cgi->param('a');
my $d = get( "http://rpc.geocoder.us/service/rest?address=$a" );
$d =~ /geo:long>([^< ]*).*?geo:lat>([^< ]*)/is;
print "Content-type: text/plain\n\n";
print "$1,$2";
The above code does the following:
It gets the address from the browser - a query which looks something like this: gaddress.cgi?a=123+Main+St+Anywhere,+NY.
A query is made to the Geocoder service provided at this URL: http://rpc.geocoder.us/service/rest
Finally, the latitude and longitude are parsed out of the results and returned to the Javascript client.
Javascript Addressing Querying
This simple function, written in Javascript, makes a query to the Address Translation Proxy asking it to convert an address into a Google GPoint. This function has two parameters that need to be taken into consideration:
function GAddress( String address, Function callback );
The first argument, address, is a string representing the address that you want to translate (for example, "123 Main St. Anywhere, NY"). The second argument, callback, is a reference to a function which will be called once the translation is complete. That function will be called with two arguments:
function callback( GPoint point, String address );
The first argument, point, will either be a GPoint representing the latitude/longitude of an address OR null, if the address does not exist. The second argument is the same address as what was sent when you called GAddress.
Now, using both of these components, it's time to wrap them together and put them to use! If you're interested to see what a final result looks like, check out this demo.
If you'd like to put this code to use, feel free to download the code below and give it a try!
- gaddress.tar.gz - Contains sample index.html, Javascript Query Function (gaddress.js), and Address Translation Proxy (gaddress.cgi). To install:
- Copy the contents of the archive to your web directory.
- Run the following command, from the command-line (or your favorite FTP client) chmod 0755 gaddress.cgi
- Go to the Google Maps API signup page and generate an API key for the URL where you uploaded the files.
- Finally, get the API key which you generated, open index.html, and change key=CHANGEME to represent your API key.
- You should be good to go! Have fun!
Tags: hacks, google, perl, popular, address, maps, geocoder, geo
78 Comments on 'Google Address Translation'
May 3rd, 2005
This script pulls down the current list of events on the Bug Jar web site - can be manipulated later to fit into an RSS feed or iCal Calendar (for example).
This script utilizes the excellent Webscrape utility.
Downloads
Tags: scrape, events, perl
Comment on 'Bug Jar Event Scraper'
May 3rd, 2005
This script pulls down the current list of events on the George Eastman House web site - can be manipulated later to fit into an RSS feed or iCal Calendar (for example).
This script utilizes the excellent Webscrape utility.
Downloads
Tags: scrape, events, films, perl
Comment on 'Eastman House Event Scraper'
April 20th, 2005
This tool goes through your current Google Search History, grabs all of your recent searches and turns it into an RSS feed. Would work best set up as a nightly/hourly cron job, redirecting to a file.
This tool is written in Perl and uses a few, slick, modules: WWW::Mechanize, XML::LibXML, and XML::RSS. I was influenced by the very nice webscrape tool when building this.
A sample, from my searches, can be found here:
http://ejohn.org/apps/ghistory/google.rdf
And how it looks in my newsreader (Newsgator):
Downloads
Tags: perl, google, search, rss, popular
8 Comments on 'Google Search History RSS'
March 20th, 2005
A CGI application that analyzes Apache logs and finds pertinent session information. I currently have a number of features:
- Support for multiple apache logs (browsable by day).
- Support for Gzip'd logs.
- Ability to block unwanted IP addresses.
- Ability to block unwanted useragents (bots, etc.)
- Ability to ignore certain filetypes.
- Customizable support for search engines.
- Support for RSS feed reader detection.
- Results are groups by RSS feeds, Search Engine referrals, Direct Hits, and Off-site referrals.
- All results are grouped by user, then by session type, only presenting unique sessions.
I'm still in the process of cleaning up the code, but I will be releasing it soon.
Links
Tags: analysis, apache, perl, logs, statistics
Comment on 'Log Session Analysis'
March 19th, 2005
This module is designed to pull down every movie that you've rated within your Netflix account. Utilizes the wonderful WWW::Mechanize module to work its magic.
This module can also be found on my CPAN page.
Downloads
Tags: perl, module
11 Comments on 'Net::Netflix'
March 19th, 2005
This module is designed to convert DVD ASIN numbers to their associated IMDB tag. This modules uses Net::Amazon to find the correct DVDs - but then is forced to make web requests to the associated Amazon page to find the correct IMDB movie. Very handy.
This module can also be found on my CPAN page.
Downloads
Tags: module, perl
Comment on 'Net::Amazon::DVD2IMDB'
·
« Previous entries
