Source Control for your Operating System

I think that I’ve realised why I feel so uncomfortable using MacOS X and Windows for “Real Work”. They’re fine for playing MP3s or browsing the web, but when it comes to developing software I get scared.

For me, developing my own software usually involves installing other software and upgrading existing software on my computer and when you don’t have a package manager that’s really hard to do reliably and reproducably. I’ve tried doing this by hand, keeping track of what versions of what I installed where with what build flags, trying to write scripts that can reproduce my build environment and trying to keep track of which dependencies are build-time and which are run-time. I’m bad at that, but that’s the kind of thing that computers are really good at.

Now, I know that Cygwin and MacPorts each have their own pseudo-package system, but that only covers the UNIX libraries and utilities that are ported, not native tools, and they don’t integrate with the tools you use when producing packages to ship. When you’re developing a Mac or Windows application how do you keep track of which libraries need to be included in your installer or disk image? There must be a trick, but I’ve never come across a tool for the job. The Debian packaging tools do an amazing job of analyzing the dependencies in software that has been built and mapping them to package dependencies. There are even tools for creating VMs that you can use to test that the autodetected dependencies are correct. All automatic, reproducable, testable.

So today, out of the blue, I found a metaphor that I’m satisfied with to explain to my Windows and Mac developer friends the way I feel about developing on their platforms. It’s like programming without source control. Developing on Linux and with source control I feel safe – if anything I do has unintended (or worse, unnoticed) side-effects I know that I can get things back to where they were before, or at least work out what happened. On Mac and Windows I’m back to when I was writing Pascal programs in DOS where I’d type a few lines, save and never be able to make my program work again.

Aside 1: At least with Windows I can do everything in VMWare which gives me snapshots and roll-back, Apple’s DRM prevents me doing the same for MacOS. Fail.

Aside 2: I feel almost the same about Ruby gems. It’s a package system but a terrible one. It’s like being forced to use RCS when I’ve got git sitting right here. I do understand why Mac Ruby developers seem to think it’s the best thing since sliced bread – I remember in 1995 when I was first introduced to RCS.

Posted in Default | Tagged , , , | 11 Comments

Twitter Translation

My friend Britt mentioned today that he was about to launch twitter.jp. How exciting! But I don’t understand Japanese. If only I could easily translate all those tweets in languages I don’t understand.

I played around with Google’s new AJAX Translation API before and I wondered how hard it would be to use that from a GreaseMonkey script. The answer: hard. I’m not sure what the exact problem was but every way I tried to include Google’s APIs into the pages I was manipulating, including creating my own iframe and using document.write failed. In the end I used a static proxy html file (hosted in one of my Amazon S3 buckets for cheap efficiency) with some sneaky cross-site communication (the request goes over in location.hash, the response comes back in window.name).

My script is now up on userscripts.org: twitlator.user.js

To use it simply go to a page listing a bunch of tweets, like the public timeline and find a tweet in a language you don’t under stand. For example:

Click the translate! link and several moments later you’ve got:

Yep. People aren’t posting anything interesting in Japanese either.

I’m pretty sure my script will only work on Firefox 3. I’m using getElementsByClassName which I think wasn’t introduced until Firefox 3. Why aren’t you running it already?

Posted in Default | Tagged , , , , , , | 2 Comments

Ten Years

Ten years ago Netscape released the source code to their browser. They called it Mozilla, I downloaded and built it at my work at the time. I didn’t manage to build it immediately, but I worked it out in the end.

Somehow I’ve spent most of the past seven years working with the platform. I touched on it at Eazel where we used Gecko to display HTML, but mostly left the hard work of integrating a barely-ready gtkmozembed to Mike and Ramiro

Then at Danger I ended up as the main engineer working on our licensed fork of the ProxiWeb web proxy (love those innovative names) which used Mozilla (circa M6 or M8 I think) on Solaris x86 to format web pages for display on hiptops and Sidekicks. Later we built our own web proxy based on Mozilla 1.3, following a similar model but with all the lessons we learned maintaining the old code for years.

Eventually I ran away with the circus as one of the first employees at Flock where we were trying to re-imagine the desktop browser experience, based on Firefox. After a couple of years struggling with vision and execution they worked it out – soon after I left. Now it’s showing the potential of moving beyond the dominant Mosaic-with-tabs user experience that the main players are following.

Now I’m at Songbird, building another kind of browser – one whose focus is integrating media consumption and management. We’re building on XULRunner, the application platform that has been spun out from the Mozilla applications. It’s not my favorite platform for building applications, but how else are you going to build a cross-platform web browser.

Somehow this passing interest I had ten years ago has turned into a career.

Posted in Default | Tagged , , , , , , , , | Leave a comment

I told you so

Six months ago I predicted:

This kind of bundling is often done by the bad guys. If you install Apple’s Quicktime codecs on Windows every update will trigger an iTunes install, even if you haven’t installed iTunes. I’m sure they’ll do the same thing for Safari on Windows. I’m not sure what iTunes’ market share on Windows is but it seems to be significant. If all those users suddenly have Safari installed that could potentially cause a big shake-up in browser market share.

On Friday there was a minor shitstorm on planet.mozilla.org about Apple pushing down Safari to all Windows iTunes and Quicktime users.

If only we had a reusable system-wide XULRunner it would be really easy to do similar but less evil promotion of our growing XUL-based free software suite. Songbird could suggest to users that they might like Firefox – and it would take just a single click and a tiny XPI download to have users running Firefox. We could even get intelligent and suggest Thunderbird to heavy email users or Flock to heavy social networking users.

iTunes’ US market share is around 27% (according to the best numbers I can find). If Apple flips the switch and makes Safari the default browser for all those users Firefox will start looking irrelevant fast.

Posted in Default | Tagged , , , , , , , | Leave a comment

Taming MacPorts

For the record, I don’t like MacPorts. I don’t like building software from sources when someone could have done it for me just fine. It’s a waste of fossil fuels and a waste of my time. Since my Mac doesn’t come with most of the software I’d expect I have to build things from sources. I tried doing this for a long time without MacPorts, but since everyone on the Mac installs all their software through MacPorts none of the portability patches make it upstream.

So I gave into the MacPorts software monopoly. There is, however no chance in hell I’m going to let it run as root and write whatever all over my filesystem – especially while the Mac doesn’t have a decent package management system I can use to restore system files that have been overwritten. MacPorts does, in theory let you install anywhere as any user. Just download the MacPorts tarball and configure with a few options:

./configure --prefix=$HOME/macports --with-install-user=ian --with-install-group=staff
make install

Just add $HOME/macports/bin to your path and you’re off. There do unfortunately seem to be a few limitations. Some packages like openssh want to install things as root. It looks like the MacPorts packaging is too coarse – you can’t install the client binaries without installing the server binaries. I guess I need to look into porting fakeroot.

Posted in Default | Tagged , | 3 Comments

Google AJAX APIs outside the browser

Google just announced their new Language API this morning. Unfortunately their API is another one of their AJAX APIs – that are designed to be used from JavaScript in web pages. These APIs are pretty cool for building client-side web applications – I used their AJAX Feeds API in my home page – but I had some ideas for server software that could use a translation API.

I remembered John Resig’s hack from a few months back, getting enough of the DOM in Rhino to run some of the jQuery unit tests. I pulled that down, wrote the bits of DOM that were missing and now I’ve got a Java and JavaScript environment for accessing Google’s AJAX APIs. Apart creating stubs for some methods that are called the main functionality I had to implement was turning Google’s APIs’ asynchronous script loading into the Rhino shell’s load(url) calls. They use <script src="... and document.createElement("script"), but both are pretty easy to catch. The upshot of this is that everything is synchronous. This subtly changes a lot of the interface. For example, my Language API example looks like this:

load('ajax-environment.js');
load('http://www.google.com/jsapi');
google.load('language', '1');
google.language.translate("Hello world", "en", "es",
  function(result) {
    print(result.translation);
  });

it of course prints: Hola mundo.

I’ve put the source up on github. Have a play, tell me what you think.

Posted in Default | Tagged , , , , | 3 Comments

API Design

Recently at Songbird I’ve been working on some guidelines to help us build better APIs. One of the most exciting things we’re hoping to offer users and developers are APIs that give them access to their music and the ability to extend their and others’ music players. Getting these APIs right can be hard and we think we can do better, so we came up with some ideas and we’re trying to get some wider feedback.

Our guidelines come in two parts – general rules that apply across the board, and platform-specific conventions depending on what the specific APIs are based on (Mozilla Extension API vs Web Page JavaScript). Here’s our general guidelines:

  • APIs should be orthogonal and symmetrical
  • Method naming, argument ordering and error handling should be consistent
  • Code that uses the API should read like prose. Words should not be abbreviated. It should be easy to understand code that uses the API without having read the documentation
  • The API should follow the terminology, conventions and idioms of the platform that is is built on
  • Use cases and target developers should be taken into consideration when the API is being designed and documented
  • Developers should be able to continue to use the tools and libraries that they are familiar with when using the API. Their previous development experience should help them learn and understand the API quickly

These are based on research I did – I distilled half a dozen other peoples’ ideas about good API design into ones that we could easily apply to our own work.

The full document fleshes these out a little more, with examples and then goes into platform specific patterns to follow.

If you’re interested, read it and please send your comments.

Posted in Default | Tagged , | Leave a comment

LUG Radio Live

I’m kind of terrified of public speaking. So I do it as much as possible.

I will be speaking at LUG Radio Live USA 2008 in San Francisco in April. I’m going to be talking about Songbird and the Open Media Web focusing on why what we’re working on over at the nest is especially important to the Linux community. Then I’ll probably show off a few cool things you can do once your digital media platform is open, standards based and extensible.

Here’s what the Lug Radio guys say about their conference:

LugRadio Live USA 2008, the ‘rock-conference’ from the team behind the popular LugRadio podcast, brings the successful and unique formula of the UK LugRadio Live events to The Metreon in San Francisco on the 12th and 13th April 2008. The event is supported extensively by Google and also supported by Dice.

LugRadio Live USA 2008 brings together over 30 speakers across three stages, 30+ exhibitors, a range of BOF sessions, debate panels, lightbulb talk sessions, demos and much more, all wrapped up in the unique event that the UK incarnation has become known for, combining an incredibly loose, social, inclusive, and amusing atmosphere -  you are new to LugRadio Live, it is nothing you will have seen before.

Sounds like fun to me. There will be a huge range of speakers, from kernel hackers like Ben Collins and Kristen Accardi to server folks like MySQL’s Stewart Smith and Samba’s Jeremy Allison. There are library and application developers like Dan Kegal from Wine and Christian Hammond from VMWare.

Apart from me there will be a ton of folks talking about media. David Schleef (of GStreamer and Dirac fame), John Buckman (from Magnatune), Mike Linksvayer (Creative Commons VP) and Aaron Bockover (Banshee hacker) will all be showing me up.

You can see all this and more for just ten dollars on April 12th and 13th at the Metreon in San Francisco. I’m really excited to be part of the conference and really excited to get to hear and meet all the other participants. So if this sounds fun or interesting to you please join me and register.

Posted in Default | Tagged , , , , | Leave a comment

A simpler mobile OpenID workflow?

Chris Messina posted today about the problems with current OpenID work-flows for mobile users. In spite of a long list of chores I was intending to complete today I had a bit of an experiment with an approach to solving this.

The main problem I wanted to solve was to allow a user to prove their identity without having to enter a password. Most mobile devices lack physical alphanumeric keyboards, and without that it’s very hard to fill out password fields.

Read More »

Posted in Default | Tagged , , | 25 Comments

Moving from Drupal to WordPress for blogging

I really like Drupal quite a lot. It’s powerful and flexible, it’s code is clear and well written and it’s extension mechanism is one of the best I’ve ever seen. All this flexibility tends to distract me from actually writing blog posts. So I’ve moved back to WordPress. If you’re reading this over RSS expect the usual disruption.

I first discovered WordPress late in 2003 from Mark Finlay, an Irish GNOME contributor. On December 24th 2003 he posted:

Everyone go look at Wordpress: Seriouslessly sexy blogging Software, looks like it’s gonna kick MT’s ass.

I did, and in the end he was right. Unfortunately he died January 10th 2004. WordPress always reminds me of the three months when we lost Ettore, Chema and Mark.

There were a few scripts around for moving Drupal posts to WordPress but they were all pretty out of date so I took one and updated it. It has a lot of commentary in it. Take a look: drupal-to-wordpress.sql

Posted in Default | Tagged , , | 1 Comment