Chris Lowis

Ruby Manor 2011

2011-11-20T00:00:00+00:00

The third Ruby Manor conference happened in London a few weeks ago. I thought I'd give my reflections on the event in the old-fashioned blog post style.

Philosophy

Ruby Manor is run according to a certain philosophy. Some of it was explained upfront by James, Murray and Tom (this year's organising team), and some of it is implicit - drawn largely from the way previous Manors and the monthly LRUG meetings are run. Ruby Manor is not an unconference, it doesn't have keynote speakers, swag, dinners or t-shirts. It costs less than £15. The speakers and topics are chosen by the attendees and contribution to the process is encouraged.

Organisation

Before last year's conference, speakers proposed their talks on a mailing list. We asked questions, and the organising team selected the final list of presentations. This year, this system was enhanced and formalised as Vestibule - a simple web application designed to encourage and facilitate engagement with the organisation of the conference. The most interesting section is the short motivations given by the attendees for participating. Quite soon after the site launched a number of talks were proposed. As an example, consider the (ultimately unsuccessful) talk I proposed. I gave an outline of the talk I wanted to give and asked for feedback and suggestions. I was encouraged by the level of support and worked further to add more information to the proposal based on the feedback. About 3 weeks before the event, the proposals were open to the vote and whittled down to a final selection.

Vestibule used a points-based "karma" system to rank the participants on their level of engagement. I'm not entirely sure what impact the score had, but I believe tickets were made available in batches to those with the highest scores first. James Adam talks more about the philosophy of conference ticketing on his blog.

I found the Vestibule system quite addictive - it was something I found myself checking each day, especially as new features were rolled out as the process went along. I'd be interested to see some analysis of the level of engagement too - did it demonstrate a typical split in engagement between active and passive, or did the presence of rewards encourage increased engagement?

Venue

The venue this year was the University of Westminster building on New Cavendish street. The lecture-theatre used had a raked seating arrangement with a large projector which meant that everyone got a good view of the talks. The venue was close to a large number of places to grab lunch and coffee during the breaks. I'm not sure what the WIFI was like as I didn't take a laptop, but I don't think people had too many problems. There was very little space at the back of the room which meant that you had to be in your seats in time for the talks and were essentially captive for the duration of the sessions, but the individual talks were of equal length which made shuffling around easy.

Lunch

I experimented with organising "Birds of a Feather" lunches this year. The idea is to have lunch with people who share a common interest, and is organised by having people sign up to lunch-time groups on the day. Although I got a good-sized group of people together for a "machine-learning" themed lunch, I don't think many other topics formed. In retrospect I think it would have been good to organise lunch "champions" in advance to lead a group around a particular topic. Also as the conference was Ruby focussed, I guess people found it easy to find others with at least Ruby as a common interest.

The Talks

The talks were universally of a very high standard. There wasn't a single one that I didn't get something out of, and a couple were among the best I've ever seen at any conference. Keep an eye on the Ruby Manor website for the videos when they're released. For me the real standouts were:

Programming with Nothing by Tom Stuart

Tom introduced us to the Lambda Calculus by considering what is possible in Ruby if we strip the language right back to basics. He implemented a solution to the FizzBuzz problem using just Ruby's Proc object and the methods Proc.new and Proc#call. What started out as a curiosity turned into a discourse on the nature of computation and the power of functional programming. It ended with a roar of approval when the final code was run! The slides were superb, the delivery style was quirky enough to be at home at a Ruby conference but had a real depth - there's no surprise to learn Tom used to lecture Computer Science topics at Cambridge! A real treat.

A Random Walk by Ben Griffiths

I really enjoyed Ben Griffiths's talk on Randomness. He gave a grab-bag selection of slides on such topics as Monte Carlo analysis, Simulated Annealing and a thought experiment on what it would be like to have a CI system that deployed to your production environment randomly. Ben organised his talk as one "idea" per slide, illustrated with occasional Ruby code, and then randomised the order just to keep himself and us on our toes. A fun idea and one that would have tripped up most speakers, I think. Not Ben though, he's a fantastic speaker and one I always enjoy seeing.

The Joy of Text by Sean O'Halpin

I was looking forward to this talk - sitting next to Sean at work, I've seen it come together slowly off the back of a lot of research. Sean talked about text encodings, his ffi-ncurses library and the history of terminal interfaces. He attempted to explain some of the problems we programmers face everyday when we work with text and text-based interfaces. It was a talk which covered a lot of ground, and hid a lot of depth and knowledge of the subject under a veneer of amusing anecdotes and useful tips. I hope the content of the talk makes it out in a longer form (a series of blog articles would be great, if you're reading, Sean!) as there's a lot of very useful content for us all to learn in there.

Conclusion

I hope this post has given you some feel for how great again the Ruby Manor conference was this year, and has encouraged you to seek out the slides and videos when they appear (it looks like the lanyrd page might be a good place to find everything together). It was a conference as much about the people and the community as it was the talks and the programming language. An experiment in self-organisation which showed how difficult it is to get people to step out of the comfort zone and contribute to the shaping of their community, but also what can be achieved when a few of them do so.

Thanks to Murray, James, Tom and all the champs who contributed to this year's event. Well done all!

Great radio interviews

2011-05-18T00:00:00+01:00

I'm a big fan of radio, and one format I particularly enjoy is the long-form interview. Here are some of the interviews I've really enjoyed in recent months, available as podcasts or downloads, and some reasons why you might enjoy them too.

Philip Elmer-Dewitt interviewed by Dan Benjamin on The Pipeline

I'm a fan of Dan Benjamin's 5by5 radio network. I'm less keen on the wide ranging discussion-show formats but when he conducts one-on-one interviews with people he obviously respects his interviewing style really brings out the best in his guests.

In this interview with Philip Elmer-Dewitt they discuss the long career of this prolific ex-Time writer. Elmer-Dewitt is remarkably self-effacing, but I was inspired by the tale of his transition from technical writer/programmer to eventually Science Editor of Time magazine. The day-to-day accounts of his working life and tales of the discipline he's cultivated are truly fascinating. There's a lot of wisdom in this interview!

Morrissey interviewed by Kirsty Young on Desert Island Discs

The recently-launched Desert Island Discs archive website has made hundreds of interviews from the long-running Radio 4 show's archive available to "download and keep". Throwing in some of my favourite band's names into the search box led to this gem from 2009.

Kirsty Young asks pointed, revealing questions often as simple statements to be refuted ("You're not traditionally romantic") but Morrissey is more willing to engage than usual ("I think I am"). He talks freely about his childhood, the cult of celebrity, why four years of the Smiths over-shadows the rest of his career and along the way gets chance to play some great records from The Ramones, Iggy & The Stooges and The Velvet Underground.

Ian Mackaye interviewed by Jesse Thorn on the Sound of Young America

I'd not heard of this show before Dan Benjamin interviewed Jesse Thorn on The Pipeline. It appears to be very popular across the Atlantic, being syndicated widely via NPR. As I clicked through the back-catalogue of shows I came across this interview with Ian Mackaye, guitarist in one of my favourite bands, Fugazi, and head of Dischord records.

In the interview Mackaye talks more about his personal philosophy rather than the music - what it's like to run a business with principles, how it feels to inspire an entire youth movement and the passion he has for the Washington DC community. Mackaye is a great talker with conviction and experience; he rarely gives interviews so this was a real treat.

Carsten Dominik interviewed by Randal Schwartz on FLOSS Weekly

I'm not quite the regular listener I once was to FLOSS weekly. Randal Schwartz is a knowledgeable and competent interviewer, but he doesn't have the easy-on-the ears style of former co-host Leo Laport. I still tune in when the topic or guest looks interesting though, and did so for this interview with Carsten Dominik, creator of org-mode for Emacs. I spend a good portion of my working life inside the venerable old editor, and organise my thoughts and projects using org-mode.

Org is on the surface a simple, useful piece of software, as Dominik makes clear its inception was certainly a case of scratching a personal itch, but its the community that has driven the development of its vast array of other features. Listening to Dominik's passion and theories on community management in open source software was very motivating, and the fact that he developed the software alongside a busy scientific life must say something about the power of org-mode as a time management tool!

Peter Buffet interviewed by Stephen Dubner on Freakonomics Radio

It's difficult to find radio programming that is not scared of numbers or the statistics behind news stories. Stephen Dubner has made a career out of asking the kind of questions that others seldom do, or at the very least presenting the research that others are doing to his audience in a compelling way. He's recently started a radio show too, and some of the episodes are interesting interviews. In this episode he talks to Peter Buffet - composer, musician and youngest son of Warren Buffet.

As you can expect, the topic of his famous father is never far away, but it was the discussion about his early family life that I enjoyed, and the conviction Peter Buffet had to follow his own path. It was an enjoyable interview, Dubner let his guest speak without the clever verbal tricks of some of his other episodes.

Joining BBC Research and Development

2011-04-06T00:00:00+01:00

A couple of months ago I joined the Prototyping team in BBC R&D. It's a great chance to work on cutting edge technologies for broadcast and the web. And a great place to work with some really smart people.

The first thing I've been working on is the fledgling RadioTAG specification, part of the RadioDNS umbrella of technologies. We're looking at adding a button to radios or other radio-receiving devices that will allow a listener to register their interest in a broadcast. Maybe you've heard a track you'd like to listen to again, or you have to leave the house during the Today programme and want to carry on listening where you left off when you reach the office. Interesting stuff.

As my colleague Duncan said, we should all try to blog a little more this year. I've made a start over on our team's blog, but I'm going to try and do so here too.

Using Rake and Git to find out what I did yesterday

2010-11-24T00:00:00+00:00

At our daily stand-ups we have to answer the question "What did you do yesterday". I wrote a little Rake task for the project I am working on to help me answer:

desc "List of all commits commited in the previous 24hours"
task :yesterday do
  git_author = ENV['GIT_AUTHOR'] || "chris.lowis"
  cmd = "git --no-pager log --after=\"1 day ago\" --pretty=oneline --author=#{git_author}"
  puts "Commits in the last 24 hours for #{git_author}"
  system(cmd)
end

This simply shells out to git with the appropriate command line arguments giving me a simple list:

Commits in the last 24 hours for chris.lowis
6663b95b5832sda91998c05fea7970fe44da13af Adding a great new feature
677b2e8092406b3c2031e8f065fg5672d90b8108 Fixing someone elses bug

There's quite a lot of interesting options that you can pass to git log, see the documentation for a list of them.

Laptop Driven Development (with Emacs)

2010-10-26T00:00:00+01:00

I read an interesting post over on Bernerd Schaefers blog the other day where he described his set up for coding when you only have a small screen. He talks about methods for switching between VIM and a terminal. If you're an emacs user, like me, it's more common to run your terminal inside emacs.

I use the homebrew package manager for OS X. One thing this allows me to do is compile emacs from source with support for true full-screen mode

$ brew install emacs --cocoa
$ ln -s /usr/local/Cellar/emacs/23.2/Emacs.app /Applications/Emacs.app

This gives me a recent emacs with a great, full-screen mode which I activate by running M-x ns-toggle-fullscreen

That's my entire screen - no menubars, docks or other OS paraphernalia. I can switch to a shell for running my unit tests by typing M-x eshell (or for a standard bash shell M-x shell)

Since the usable area of the screen is quite large, thanks to the fullscreen mode, I also have room to split the frame in two (M-x split-window-vertically) to see, for example, tests and implementation at the same time

Or, occasionally, to pop open the shell buffer for quickly issuing a command

I find emacs to be a great environment to work in on a smaller screen. You can see more of my emacs configuration on github.

Setting Http Proxy when using the selenium-webdriver gem and Firefox

2010-07-14T00:00:00+01:00

I've been using the selenium-webdriver ruby gem to do some automated Cucumber testing of our application. Sadly, I'm sat behind a firewall which requires an HTTP proxy to access the outside world. Using the Chrome WebDriver bridge worked fine as it inherits the system preferences on OS X. However with Firefox I had to perform some trickery to get it to work:

require 'rubygems'
require 'selenium-webdriver'

include Selenium

profile = WebDriver::Firefox::Profile.new
profile["network.proxy.type"] = 1
profile["network.proxy.http"] = "www-cache.at.my.work.co.uk"
profile["network.proxy.http_port"] = 80

driver = WebDriver.for(:firefox, :profile => profile)
driver.navigate.to "http://google.com"

The []= method on Profile is setting the options that you find by typing 'about:config' in the Firefox URL bar. Take a look there in a working Firefox profile to see what variables you should set for your webdriver-specific profile in the code above.

Growl notifications from Ruby on OS X

2010-07-06T00:00:00+01:00

I wanted to generate Growl notifications from a Ruby script on OSX and had a bit of trouble getting it to work. To save you the same hassle, here's the steps I took:

Install growl

Install Growl if you haven't already. I used version 1.2.

Set some growl preferences

In the growl preference plane (find it under System Preferences)

under network check "Listen for incoming notifications" and "Allow remote application registration"
restart growl

Install the ruby-growl gem

There's a couple of different ruby bindings for Growl but ruby-growl seemed to work for me

gem install ruby-growl

Send a growl notification from your Ruby script

You can now send notifications from Ruby

require 'rubygems'
require 'ruby-growl'

g = Growl.new "127.0.0.1", "ruby-growl", ["ruby-growl Notification"]

g.notify "ruby-growl Notification", "It Came From Ruby-Growl", "Greetings!"

Open-source rails apps to study and learn from

2010-05-31T00:00:00+01:00

If, like me, you learn best by studying other code here's ~~five~~ some open-source rails apps with something to teach. They're a great resource to study and improve your own code, or to use as the starting point for your own applications. All of these applications use Rails 2 but as they are open-source they are a great opportunity to get your feet wet in the Rails 3 world by helping to port them across.

Update: Thanks to the comments here and over on Hacker News, I've added two more applications and made some comments on the non-standard nature of the Loved by Less application code.

Gemcutter

A site familiar to all Ruby developers is Gemcutter - the recent replacement for the venerable Rubyforge. I wasn't aware that the source code of the site was open source until danieldon pointed it out to me, but you can study the code on github. The Gemcutter source is written in a very modern way featuring examples of using Cucumber for integration testing, a simple Rack middleware example and an interesting routes.rb file showing how to cleanly version an API.

Features

A very modern Rails application featuring the latest best practices.
Examples of metal and rack middleware.

Spot Us

Spot Us is a website that allows individuals to commission freelance journalists. It has many social-network type features which make the source code interesting reading.

The comprehensive test suite is written with rspec and the ruby-like templating system HAML is used for the views. It's also an example of how to use jQuery in a rails app instead of the default prototype through the use of the jrails plugin.

Features

Haml
rspec
jquery with the jrails plugin.

Loved by Less

Loved By Less is an open-source social network. I think many Rails freelancers have been asked to create a site with social network features - profiles, 'friending', photo sharing etc. Less Everything were no different and created this open-source platform to help ease the pain of getting started. The source code is available on github. It's unlikely that simply cloning the app will give you everything you need, but it's an amazing resource to read through and take the bits you need for your project.

The loved by less source code is starting to show it's age a bit, and does include some pretty extensive monkey patching which may confuse a newcomer. While I don't think this is bad in itself (it shows the flexibility of Ruby, and also the dangers of not following Rails conventions), it's something to be aware of. I think porting this to Rails 3 would give newcomers to Rails a great resource to learn from.

Features

Search with Thinking Sphinx
Flickr integration
Comprehensive email support
Paperclip for attachments

Saasy

If you're working on a Software as a Service application, take a look at Saasy . It's a template application designed to do billing and authentication tasks 'so you don't have to'.

Studying the code base, you'll see examples of using SSL security and ActiveMerchant. I was also interested in the accounts model which uses the acts_as_state_machine plugin to simplify some of the logic.

Features

acts_as_state_machine
ActiveMerchant

Simply Agile

Simply Agile by Andrew Bruce is an open source agile software project management application. Think of a simple version of Pivotal Tracker. You can add story cards to a backlog, and then assign each card to a sprint. Each time a card is completed you can drag and drop it to the appropriate column in the Simply Agile task board. Andrew has a very dedicated approach to test-driven development and as a result the application is well covered by cucumber and rspec tests. The javascript in this application is written in an unobtrusive style and is a good example of using jQuery for "progressive enhancement". The app is fully functional in browsers without javascript, but where javascript is available so is the drag-and-drop interface.

Features

Cucumber integration tests
rspec coverage
Drag and drop interface powered by jQuery.

Typo

Typo is "the oldest and most powerful Ruby on Rails blogware" and is still under active development. While the 15 minute blog is still a fantastic demonstration of how powerful Rails is for rapid application development, anyone who has decided to take the project further and create their own fully-featured blog system will know how much extra work is needed. From comment systems, to anti-spam protection, restful routing to upload support there are many features that need to be added. Thankfully Typo has already implemented these and many other features, and the source code is available to help you learn how to do the same.

Features

rspec for testing
internationalisation support
email notification
admin and public-facing interfaces

Open Source Rails

Open source rails is a gallery of open-source rails projects. I selected this project for this list because although the code itself isn't well documented or tested, in many ways this is illustrates a great strenght of the Rails framework. It is a good project to study to realise how much can be achieved with a small amount of code and a handful of powerful plugins (not that I'd recommend starting your next Rails project without tests, of course!).

Features

OpenID integration
Paperclip for file uploads
A simple set of routes.

Conclusion

I hope you enjoyed this selection of Rails projects. This is far from a definitive list, and I encourage you to read some of the suggestions of other apps in the comments below and on the discussion over on Hacker News. With the upcoming Rails 3 release I think it's a good time to reflect how far the Rails framework has come, and also to remember to cater for people who are learning Rails for the first time - studying open-source code is a great way to learn.

Do you have a favourite open-source Rails project? Share it in the comments below.

Unfilling regions in Emacs

2010-03-03T00:00:00+00:00

I have the habit of hitting M-q every 5 seconds or so when working with text in emacs. This key combination is bound to "fill-paragraph" and inserts line breaks into the current paragraph to keep the line width to approximately 80 characters.

If you're cutting and pasting the text into an email message or website form though, this is often not what you want. The way round this is to "unfill" the current paragraph or region. The following commands will help:

(defun unfill-paragraph ()
  (interactive)
  (let ((fill-column (point-max)))
    (fill-paragraph nil)))

(defun unfill-region (start end)
  (interactive "r")
  (let ((fill-column (point-max)))
    (fill-region start end nil)))

But now you have lines in your emacs buffer that wrap in an unusable way. Longlines-mode to the rescue (M-x longlines-mode). Longlines mode uses "soft newlines" which will not show up when yanked or saved to disk.

I've recently added my emacs config to github. Feel free to have a poke around and let me know what other emacs tricks I am missing!

Recent Activities

2009-08-25T00:00:00+01:00

Recently…

Back in May I started a new job at the BBC. I've been working on some interesting projects with some great people. I've also been lucky enough to take part in a couple of Hack Days in an official-ish capacity, and I intend to blog about some of the technical aspects soon. For now:

The Music Bore - an automated DJ system that Nick, Patrick, Yves and I built at the Music Hack Day. I wrote some more about this over on the BBC Radio Labs blog Coming soon to an IRC channel, hosted server, or if Patrick and I get our way Art Installation near you!
David and I were invited to the Guardian's Hack Day where we worked on a RDF and Processing powered mash-up of Guardian and BBC content around MPs. David blogged about this on the BBC Internet blog.

Using R and Ruby - slides from February LRUG

2009-02-15T00:00:00+00:00

Introduction

Last week I gave a short talk at the February LRUG meeting about using the scientific programming environment R together with Ruby.

My presentation gave a few examples of the kind of statistical analysis that can be performed with R, and showed how easy this functionality is to access from within your Ruby code.

Slides

Code

I'll post the code for the Twitter analysis to github soon. It needs a little tidying up first.

An initial draft of this presentation used R to develop a recommendation algorithm (using a k-means clustering) using the GitHub api. Although it worked, the recommendations it made were not great, so I removed the code from the presentation and replaced it with the Twitter analysis. I'd like to resurrect this code at some point, so remember to subscribe to my RSS feed if you're interested, or nag me to do it in the comments field below!

Calculating the Pearson correlation coefficient using R and Ruby

2009-01-21T00:00:00+00:00

In a previous article I talked about using the GNU scientific library to implement the Pearson correlation algorithm, as used for example in acts_as_{recommendable} As a prelude to some forthcoming articles, I'd like to show you how easy it is to implement the same thing using the Ruby bindings to the R project.

For this to work you'll need to install R and the rsruby gem. Take a look at the documentation for the rsruby gem, as while installation is straightforward there's a couple of things to be aware of.

Having done that, let's reopen the Pearson class from the previous article and add a new R-based method

class Pearson
  def initialize
    require 'rsruby'
    @r = RSRuby.instance
  end

  def R_pearson(x,y)
    @r.cor(x,y)
  end
end

The new initialize method sets up the communication with the R instance. The actual definition for the Pearson method simple, conversion from Ruby Arrays to R vectors is handled automatically by the bindings, you simply need to know the correct R method to call - in this case 'cor'. Take a look through the R manual to learn more about this powerful tool - almost all the features are accessible through the Ruby bindings.

Here's a quick modification to the benchmark to compare performances:

require 'benchmark'

n = 100000
x = []; n.times{x << rand}
y = []; n.times{y << rand}

p = Pearson.new()

Benchmark.bm() do |bm|
  bm.report("Ruby:") {p.ruby_pearson(x,y)}
  bm.report("GSL:") {p.gsl_pearson(x,y)}
  bm.report("Inline:") {p.inline_pearson(n,x,y)}
  bm.report("R:") {p.R_pearson(x,y)}
end

And the results,


	user	system	total	real
Ruby:	1.590000	0.020000	1.610000	(1.610470)
GSL:	0.010000	0.000000	0.010000	(0.062538)
Inline:	0.010000	0.000000	0.010000	(0.004548)
R:	0.220000	0.010000	0.230000	(0.227184)

The R version is around 7 times faster than the native Ruby version, but not as fast as the C-based approaches described earlier.

The real power of interfacing with R however is in the ability to quickly swap out one algorithm for another, or experiment interactively with your data in an irb console. Once you have an algorithm that works well in your case, then it may be necessary to re-implement in a faster language if performance is a concern.

I'll be talking about R and Ruby a little more in the future, so subscribe to the RSS feed if you're interested.

Identify Programming Languages with SourceClassifier

2009-01-04T00:00:00+00:00

Do you need to identify the programming language used in a snippet of code? For example, in a pastie style application or in your blog comments system. I've just released version 0.2.1 of SourceClassifier over on github.

Source classifier identifies programming language using a Bayesian classifier trained on a corpus generated from the "Computer Language Benchmarks Game":http://shootout.alioth.debian.org/. It is written in Ruby and available as a gem. Out of the box SourceClassifier recognises C, Java, Javascript, Perl, Python and Ruby. A nice advantage of using a Bayesian classifier to identify the source code is that even false matches will still give some usable highlighting. To train the classifier to identify new languages download the sources from github .

Usage

First install the gem using github as a source

$ gem sources -a http://gems.github.com
$ sudo gem install chrislo-sourceclassifier

Then, to use

require 'rubygems'
require 'sourceclassifier'

s = SourceClassifier.new

ruby_text = <<EOT
def my_sorting_function(a)
  a.sort
end
EOT

c_text = <<EOT
#include <unistd.h>

int main() {
  write(1, "hello world\n", 12);
  return(0);
}
EOT

s.identify(ruby_text) #=> Ruby
s.identify(c_text) #=> Gcc

Training

Download the sources from github and in the directory run the training rake test

$ rake train

In the ./sources directory are sub-directories for each language you wish to identify. Each sub-directory contains examples of programs written in that language. The name of the directory is significant - it is the value returned by the SourceClassifier.identify() method.

The rake task populate can be used to build these sub-directories from a checkout of the computer language shootout sources but you are free to train the classifier using any available examples.

Acknowledgments

This library depends heavily on the great Classifier gem by Lucas Carlson and David Fayram II.

Get working quickly with a customised Rails project launcher

2008-12-30T00:00:00+00:00

I just knocked up a quick Ruby script to automate some of the steps I always do when starting work on a Rails project, namely:

open an iTerm tab running script/server
open an iTerm tab running script/console
open an iTerm tab running autotest
open an iTerm tab running a shell window
launch Safari on localhost:3000 for testing.

Customise this little script to suit your own needs. You'll need to install the rb-appscript gem to start with, and obviously this only applies to OS X users.

require 'rubygems'
require 'appscript'

RAILS_PROJECT_PATH = "~/path_to_your_rails_project" # Customise this

def create_iterm_tab( command = "" )
  @iterm ||= Appscript::app( 'iTerm' )
  session = @iterm.current_terminal.sessions.end.make( :new => :session )
  session.exec( :command => 'bash -l' )
  session.write( :text => "cd #{RAILS_PROJECT_PATH}" )
  session.write( :text => command ) unless command.nil?
end

def launch_browser( url = "http://localhost:3000")
  @safari ||= Appscript::app('Safari')
  @safari.open_location(url)
end

create_iterm_tab("./script/server")
create_iterm_tab("./script/console")
create_iterm_tab("autotest")
create_iterm_tab()
sleep(5)
launch_browser()

The sleep(5) is in there to give time for the server to spin up before launching the browser.

Now put all this in a file called "launch_project.command" on the Desktop, and chmod +x it to allow it to allow it to execute when double-clicked.

I got some tips from Dribblings of a Deranged Hermit.

Happy New Year!

Fitting curves to data using Ruby and the GNU Scientific Library

2008-12-01T00:00:00+00:00

In this post I'll show you how to use the GNU Scientific Library and its Ruby bindings to fit curves to data. This technique is useful if, for example, you want to extrapolate into the future on the basis of some past information.

By way of example, your boss approaches you with some historic revenue figures from your new Web 2.0 venture, and asks you to predict future growth. Let's generate some example data to play with:

require 'gsl'
time = GSL::Vector.linspace(0,24,100)
revenue = GSL::Vector.linspace(0,1,100).collect{|yi| yi+rand()/10}

In this code we're using the GSL Vector class to represent time. Here we have a vector of months representing two years of data (24 months). The linspace method causes our vector to have 100 evenly spaced elements between 0 and 24.

We concoct some revenue data using GSL::Vector again. The GSL Vector class does not implement all of methods found in Ruby's native array class, but does have 'collect'. Here we create a random scattering of points.

Let's visualise this data first to see what we are dealing with. To do this we have a number of options, here I'd like to show how we can use Gnuplot to do the visualisation.

First install the gnuplot gem:

$ gem install gnuplot

You'll need to have gnuplot installed to use the bindings. Binary packages are available for many platforms, just make sure that gnuplot is in your path after installation. OS X users should install aquaterm to allow plotting to a desktop window.

Now let's plot the data generated above:

require 'rubygems'
require 'gnuplot'

Gnuplot.open do |gp|
  Gnuplot::Plot.new( gp ) do |plot|

    plot.title  "Company turnover"
    plot.xlabel "Month"
    plot.ylabel "Billions $"

    plot.data << Gnuplot::DataSet.new( [time.to_a, revenue.to_a] ) do |ds|
      ds.with = "points"
      ds.notitle
    end

    plot.terminal "svg"
    plot.output "revenue.svg"
  end
end

The code above generates the following graphic in SVG format.

Gnuplot has a number of output formats, known as "terminals" to choose from including one which plots straight to screen - great for rapid exploration of data.

Looking at the data, we see that a straight line drawn through the points would be a good model for the growth in revenue. We can use GSL's line-fitting tools to perform this fit for us:

(c0, c1, cov00, cov01, cov11, chisq, status) = GSL::Fit::linear(time,revenue)
revenue_fit = (time * c1) + c0
puts c1
puts c0

The value of c1 is the growth rate per month, in this case around 0.04 Billion $ per month! We have used a linear fit here, but the GSL provides many other fitting methods for more complicated data. A quick plot checks the goodness of our fit:

plot.data << Gnuplot::DataSet.new( [time.to_a, revenue_fit.to_a] ) do |ds|
  ds.with = "lines"
  ds.notitle
end

Add this snippet of code to the plotting function above (after the first plot.data and before the call to plot.terminal)

If you need to perform data analysis, provide graphics for your users in your webapp, or produce high quality plots I encourage you to investigate the combination of ruby, GSL and GNUPlot.

Implementing the Pearson correlation algorithm using Ruby and the GNU Scientific Library

2008-11-24T00:00:00+00:00

At Ruby Manor on Saturday Alex MacCaw gave a great introduction to his acts_as_recommendable plugin for Rails. acts_as_recommendable simplifies collaborative filtering for Rails models, automatically generating recommended items, at an on-line store for example, based on a database of user preferences.

At its heart, acts_as_recommendable uses a statistical measure known as the Pearson correlation coefficient to calculate the "nearness" of items to one another. Alex talked about the performance issues he encountered when implementing the algorithm in pure ruby. To allow recommendations to be calculated for the entire database he had to switch to making calculations offline and reimplementing the algorithm in C using RubyInline.

The GNU Scientific Library has an implementation of the Pearson algorithm, and in this post I'd like to show how the Ruby code, or its inline-C equivalent can be replaced with GSL code using Ruby bindings to the GSL.

My naive pure Ruby version of the algorithm looks like this:

def ruby_pearson(x,y)
  n=x.length

  sumx=x.inject(0) {|r,i| r + i}
  sumy=y.inject(0) {|r,i| r + i}

  sumxSq=x.inject(0) {|r,i| r + i**2}
  sumySq=y.inject(0) {|r,i| r + i**2}

  prods=[]; x.each_with_index{|this_x,i| prods << this_x*y[i]}
  pSum=prods.inject(0){|r,i| r + i}

  # Calculate Pearson score
  num=pSum-(sumx*sumy/n)
  den=((sumxSq-(sumx**2)/n)*(sumySq-(sumy**2)/n))**0.5
  if den==0
    return 0
  end
  r=num/den
  return r
end

Here I should note that the acts_as_{recommendable} code is considerably more complicated, however the heart of the calculation looks something like the above. We can replace that with an inline C version using something like:

require 'rubygems'
require 'inline'

inline do |builder|
  builder.c '
    #include <math.h>
    double inline_pearson(int n, VALUE x, VALUE y) {
    double sum1 = 0.0;
    double sum2 = 0.0;
    double sum1Sq = 0.0;
    double sum2Sq = 0.0;
    double pSum = 0.0;

    VALUE *x_a = RARRAY(x)->ptr;
    VALUE *y_a = RARRAY(y)->ptr;

    int i;
    for(i=0; i<n; i++) {
      double this_x;
      double this_y;
      this_x = NUM2DBL(x_a[i]);
      this_y = NUM2DBL(y_a[i]);
      sum1 += this_x;
      sum2 += this_y;
      sum1Sq += pow(this_x, 2);
      sum2Sq += pow(this_y, 2);
      pSum += this_y * this_x;
    }
    double num;
    double den;
    num = pSum - ( ( sum1 * sum2 ) / n );
    den = sqrt( ( sum1Sq - ( pow(sum1, 2) ) / n ) *
          ( sum2Sq - ( pow(sum2, 2) ) / n ) );
    if(den == 0){
      return 0.0;
    } else {
      return num / den;
    }
   }'
end

Which is a considerable amount of code. I love the fact that this code can be embedded directly in the Ruby source code but, as was pointed out to me, for some people this would be seen as something of a maintenance nightmare. If you're prepared to install the pre-requisite GSL library and its Ruby bindings you can actually replace all of the above code with the simple:

require 'gsl'

def gsl_pearson(x,y)
  GSL::Stats::correlation(
    GSL::Vector.alloc(x),GSL::Vector.alloc(y)
  )
end

In this code, the <verb>GSL::Vector.alloc()</verb> method converts a Ruby Array to a GSL Vector class. We then call the correlation method, helpfully provided by the Statistics portion of the GSL.

So, how does this perform ? A quick benchmark :

require 'benchmark'

n = 100000
x = []; n.times{x << rand}
y = []; n.times{y << rand}

Benchmark.bm() do |bm|
  bm.report("Ruby:") {ruby_pearson(x,y)}
  bm.report("GSL:") {gsl_pearson(x,y)}
  bm.report("Inline:") {inline_pearson(n,x,y)}
end

Gives some indicative results :


	user	system	total	real
Ruby:	1.530000	0.020000	1.550000	(1.544765)
GSL:	0.010000	0.000000	0.010000	(0.015925)
Inline:	0.000000	0.000000	0.000000	(0.004115)

While the Inline version in this example outperforms the GSL version, both offer considerable savings on the Ruby version. One of the real advantages of the GSL is it allows you to rapidly experiment with alternative implementations of an algorithm. Also you can be safe in the knowledge that your code is based on a well-tested library of scientific functions.