Last week I wrote and released my LOL Feeds site. It takes RSS or Atom feeds from the web and makes a series of lolcat-style images on a web page. It's really way funnier than it sounds.
Initially I wanted to be able to auto-generate Jerk City comic strips based on my friends' twitters, but when that seemed hard I opted for lolcats style images. After all we'd been seeing a lot of the lolcats on twitter - they're displayed when the site is undergoing maintenance.
The original version of the script was very very clever. It used the Google AJAX Feed API and the Flickr API to pull in feeds and random images of cats from Flickr, combine them together with a PHP script I wrote to generate transparent PNGs of text live onto the page. It used the browser's own text-flowing algorithms to lay out the text. It was however amazingly slow.
Browsers only allow a low number of concurrent connections to one site - four or eight I think - and this made the text crawl in. Also while the Google AJAX Feed API and Flickr API are pretty snappy they're way slower than doing it server side. I was sad about this because I'm kind of in love with fully dynamic client-side applications (just look at my home page) but I actually wanted this to see the light of day.
So a rewrite ensued. I pulled down a static set of cute cats from the cute cat group on Flickr, filtered by creative commons license of course. I removed a ton I didn't like and added a couple of pictures of cats I know well. I reworked the script that generated single words into one that could place words on a JPEG. I had to write my own text layout algorithms but it turned out to be pretty simple.
I used Magpie RSS to generate a page that referenced my image generation script. To make the image generation stable (so that everyone looking at the same feed would get the same images for the same text) the image is selected based on a hash of the message. Nothing is actually random.
Then I glued it together with a good helping of mod_rewrite
This version worked pretty well. There were some issues with non-ascii characters and even funny characters like ? and & due to some URL encoding issues but it worked well. I posted about it told some friends and waited to see what would happen.
Things were going fine till I hit metafiler. Dreamhost, my friendly cheap hosting provider (100% carbon neurtral - yay!) noticed my image generation script was slowing down my shared machine and turned it off.
Adding an image caching layer was pretty straight-forward. I turned the image generation script from a .php to a .inc and included it in my feed processing script. Instead of generating an image every time it generates an MD5 of image parameters and uses a cached image if one exists. I was able to update the existing image generation script to use the same code and send a redirect to the cached version. In the past 5 days the script has generated just over 50,000 unique images which have been accessed almost half a million times. That's a lot of cats saying stupid things. And a lot of saved CPU cycles.
I also wrote a Facebook Platform application, but that's a story for another day.