$ bundle install
Fetching source index for http://rubygems.org/
Gem::Exception: Cannot load gem at [/usr/lib/ruby/gems/1.9.1/cache/rake-0.8.7.gem] in /home/rowan/newprj
An error occured while installing rake (0.8.7), and Bundler cannot continue.
Make sure that `gem install rake -v '0.8.7'` succeeds before bundling.
After reading a StackOverflow post on a similar error message I found that the way around it is to do a bundle install as root:
$ sudo bundle install
This isn't the right way to do it normally, and I don't recommend it, but it does seem to be necessary at the moment because Ubuntu (11.x) has Ruby 1.9.2 packaged under the 1.9.1 label/location.