Effectif Development

Fixing the Xcode Project Templates

2010-01-02T00:00:00+00:00

I recently upgraded my copy of Xcode and installed the iPhone SDK. A short while later, while following an example in the Pragmatic Programmer’s Cocoa Programming book, I found that my copy of Xcode wasn’t creating some key files.

Specifically, when I created a new project from the Cocoa Application template, the app delegate files (ProjectNameAppDelegate.*) weren’t being created. Yet there they were in the book, which has been freshly updated for Snow Leopard and the latest version of Xcode (both of which I have). Hmmm.

I did some searching online, and found that I’m not alone. Others on the Pragmatic forums have had the same trouble, and Martin Wood has written a useful post explaining that the Xcode project template files were messed up. Having read Martin’s post I was curious about what had gone wrong, and dug a little deeper.

Xcode stores template files for creating new projects in this folder:

/Developer/Library/Xcode/Project Templates

The Cocoa Application template (the one that I found to be dodgy) belongs here:

/Developer/Library/Xcode/Project Templates/Application/Cocoa Application

Inside it you should find the following files and directories:

English.lproj/
main.m
___PROJECTNAMEASIDENTIFIER___-Info.plist
___PROJECTNAMEASIDENTIFIER___AppDelegate.h
___PROJECTNAMEASIDENTIFIER___AppDelegate.m
___PROJECTNAMEASIDENTIFIER____Prefix.pch
___PROJECTNAME___.xcodeproj/

I found something rather different; I had several template directories inside my Cocoa Application folder, that should have been in the parent directory:

Cocoa Application/Cocoa Application
Cocoa Application/Cocoa Document-based Application
Cocoa Application/Core Data Application
Cocoa Application/Core Data Application with Spotlight Importer
Cocoa Application/Core Data Document-based Application
Cocoa Application/Core Data Document-based Application with Spotlight Importer

Interestingly, all these template folders (except for Core Data Application with Spotlight Importer) already existed inside Project Templates/Application.

The folders in Project Templates/Application dated from 2007 and used the outdated binary Nib files (with a .nib file extension). The templates in inside Cocoa Application dated from May 2009 and used the more up to date .xib file format. It seems fairly clear that those are the ones we should be using.

I ran the following commands to move the older copies out of the way:

$ mkdir ~/xcode-backup
$ cd /Developer/Library/Xcode/Project Templates/Application
$ for template in \
>   $(find Cocoa\ Application -type d -mindepth 1 -maxdepth 1 | egrep '(Core |Cocoa D)'); do
>     old="$(basename $template)"
>     [ -e $old ] && mv "$old" ~/xcode-backup
>     mv "$template" .
>   done
$ mv "Cocoa Application" ~/xcode-backup
$ mv "~/xcode-backup/Cocoa Application/Cocoa Application" .

I restarted Xcode and made myself a new project and found that the delegate files were present and correct. I also noticed that the Cocoa Application project (which had previously had the standard blue project icon) now stood out with a separate icon:

Ruby Manor

2009-12-12T00:00:00+00:00

Ruby Manor is one of those alternative conferences. It’s not run for a profit, and costs just £8 for a ticket (you pay just enough to cover the cost of the room and equipment). You even get a free beer afterwards in the local boozer (and it’s at least £3 a pint in London these days). Due to the unique way that Ruby Manor is funded (actually, that’s nothing to do with it) you can even check the bar bill.

There were some great talks again this year, and it’s far and away my favourite date on the conference calendar. Congratulations to Murray, James and Kalv on organising such a runaway success.

I spent most of the day typing frantically, but I didn’t manage to capture everything. The photos that are littered through the following articles are all thanks to @glenngillen from Rubypond, who was sat next to me snapping away for most of the day.

So, in alphabetical order, here are my notes from every talk at Ruby Manor 2 (also known as “Ruby Manor Harder”). Each talk gets a page of it’s own this year, mainly to help you see the wood for the trees. You can also find a couple of articles covering Ruby Manor 2008 at the bottom of the list.

P.S. If you’re one of the speakers and spot a mistake, or want me to change anything, just get in touch on Twitter and I’ll sort it out pronto.

3 Unix commands for finding performance problems

2009-11-20T00:00:00+00:00

A user recently reported a performance problem on one of our Rails apps. I pulled up New Relic’s performance graphs and checked what was happening around the time that we received the email. Sure enough there was a massive spike in the time taken to serve a request.

We gave the problem a couple of minutes thought and came up with a hypothesis; we’ve got one long running request in this app that really ought to be run in a background job. Could that request be responsible? Due to the way we’ve configured our web servers, there would need to be at least 8 concurrent requests for that page.

So how many concurrent requests were there? Each one creates a line in the Rails log file like this:

Processing MyController#create (for 172.16.5.45 at 2009-11-16 04:03:44) [POST]

The key thing here is that the time is in the log file, so we can get a rough feel for usage by counting how many requests MyController#create receives in a minute.

You could write a script to work that out for you. It probably wouldn’t take more than 10 minutes, but it feels like a bit of a chore and – frankly – I’m too lazy to bother.

Alternatively, you could spend 10 seconds cooking up a Unix command line like this one:

$ zgrep MyController#create production.log.1.gz | \
    cut -d ' ' -f 7 | cut -d : -f 1,2 | \
    sort | uniq -c | sort -nr | head -n 5
  18 18:21
  16 17:04
  15 18:31
  14 22:47
  14 20:42

Neat. Looks really clever, but it’s actually really easy and anybody can use it.

The output tell us how many times MyController#create was called per minute, for the five busiest minutes in the log file. To give you an idea of how to read it, we peaked at 18 requests per minute at 18:21. Given that we already know that we can only support 8 of these requests at once there’s a very good chance we’ve found our performance problem. It’s probably time to re-implement this rewrite the relevant code as a background job.

To nobody’s surprise, our customer feedback dates from 18:30, which is also when the New Relic graphs indicated that we had a performance problem (requests were backing up in Mongrel’s queue while the Mongrel servers were tied up by our long running create action).

Let’s break that command down a bit while I show you how I built it.

Step 1: Finding the time of each request

Step 1 is to have a quick look at some example log messages to see what information we’ve got to play with. All we really need is the time of each request, and lo and behold, there it is…

$ zgrep MyController#create production.log.1.gz | head -n 1
Processing MyController#create (for 172.16.5.45 at 2009-11-16 04:03:44) [POST]

zgrep is just like grep, but it can read compressed logs.

The time the request was made is the seventh field in the log message, if you split the line on white space. The cut command can do that for us, like so:

$ zgrep MyController#create production.log.1.gz | \
    cut -d ' ' -f 7 | head -n 3
  04:03:44)
  04:03:48)
  04:04:51)

The -d switch specifies the delimiter (a space) and -f tells cut which fields we want. We need to clean “04:03:44)” up a bit. As we’re counting the number of requests per minute we can throw away the seconds. We can do that with cut again:

$ zgrep MyController#create production.log.1.gz | \
    cut -d ' ' -f 7 | cut -d : -f 1,2 | head -n 3
  04:03
  04:03
  04:04

Step 2: Counting requests per minute

We can count each minute with the uniq command. Normally uniq will just remove all the duplicate lines from a sorted list, but with the -c switch we can ask it to tell us how many times each line occurs.

$ zgrep MyController#create production.log.1.gz | \
    cut -d ' ' -f 7 | cut -d : -f 1,2 | \
    sort | uniq -c | head -n 3
  2 04:03
  1 04:04
  3 04:06

uniq has added the count in front of the time. Now we’re getting somewhere (though there’s clearly not a lot going on at 4 o’clock in the morning)!

Note that uniq requires it’s input to be sorted, so I’ve piped the output of cut through sort first. I really didn’t need to do that in this case as the lines in my web logs are already in chronological order, but it’s a good habit to get into when using uniq -c.

Step 3: Finding the hotspots

Okay, so how do we find out when things start to get busy? We can use sort again, and ask it to list the rows in descending order of popularity. sort -r will do that for us, but we also need to point out to sort that it’s sorting numbers (otherwise when sorting “23” and “100” it will just compare “2” to “1” and conclude that “23” is bigger). So that gives us sort -nr:

$ zgrep MyController#create production.log.1.gz | \
    cut -d ' ' -f 7 | cut -d : -f 1,2 | \
    sort | uniq -c | sort -nr | head -n 5
  18 18:21
  16 17:04
  15 18:31
  14 22:47
  14 20:42

And there you have it – it doesn’t seem so clever now does it?

Update: Andrew McDonough got in touch on Twitter to show me how he’d do it with awk. Nice.

Tools of the trade

2009-11-12T00:00:00+00:00

Peter Cooper of Ruby Inside recently asked if people wanted to write about the things that they use in their development work. So I jumped squarely onto that band wagon…

Get organised with Things

I used to manage my tasks in the legendary Lotus Agenda, running in a DOS emulator on Linux. Since I’ve switched to a Mac I’ve stopped using it, but reproducing the power of Agenda in Cocoa would be, without a shadow of doubt, my holy grail of desktop todo list management.

Agenda’s flexibility allowed me to apply GTD the way I wanted; I would setup a list of tasks to work on today, and then a bunch of tasks that I’d get around to later. It didn’t make the mistake of forcing you to assign due dates.

Things is the first modern desktop application I’ve seen that allows me to work this way, and it does it brilliantly. It also synchs with my iPhone. Perfect.

The only downside – you can’t export (i.e. backup) your data in a human readable form. Ouch.

TextMate

I came across TextMate in the first Rails “blog in 15 minutes” screencast. Then I went out and bought my first Mac.

Get yourself a copy of the Pragmatic Programmers’ TextMate book; it’s a great way to both learn the basics and learn about what’s under the hood.

WriteRoom

When I’m writing articles or long emails I find it can be quite difficult to stay focussed on the task in hand. Not only do I have to block out everything that’s going on around me (I haven’t always got those Sennheisers on), I also have to mentally filter out all the distractions on my desktop that are just begging me to click on them (the menu, desktop icons, Tweetie, etc.). Or rather I did, before I started using WriteRoom.

WriteRoom takes everything way and replaces it with a totally blank screen and a flashing cursor.

It’s beautifully simple; you don’t even need to pause to save your work as WriteRoom automatically saves it as you go. When you’re done just copy and paste your text into your blogging software, email client, or word processor. It’s perfect for writing in Markdown.

There are free and commercial versions. I find that the free version does everything I need.

Unix for, er, anything

This is a fairly obvious one, but you really need to be using Unix. And when I say “using Unix” I don’t just mean that you need to be writing your software on Linux or a Mac (you do, but there’s much more to it than that).

I mean that you need to know how to use the power of the Unix command line. Get familiar with your shell and learn how to automate repetitive tasks. I use find, xargs, sed, sort, uniq and bash itself to increase my productivity on a daily basis. You want to get yourself comfortable with writing little snippets of bash into your shell prompt. Here’s an example from my bash history:

for p in $(grep -rl 'Parent: nesta' pages/); do mv $p pages/nesta/; done

Not very complicated, but damned useful.

Install the cheat gem and remember to read the cheat sheets to hand. Spend a spare five minutes reading some man pages.

VMware Fusion

If you deploy your code on Linux but like to use the Mac desktop, you’ve got a problem. Sooner or later you’ll find that something doesn’t quite work when you roll it out, and you’ll wish you’d run your tests on a Linux box first.

That’s where VMware Fusion comes in. I mount a Linux VM over NFS to allow TextMate to access code on the VM, and then ssh into the VM to run the tests, rake, etc.

It’s not free, and there’s no support (unless you pay for it), but it’s still the best virtualisation solution on a Mac.

Omnigraffle

On the rare occassions that I need to capture a new data model I always use Omnigraffle. I find it faster than pen/paper for capturing a rapidly evolving data model.

Most diagrams, wireframes or designs just get sketched on paper.

Make backups with SuperDuper!

SuperDuper! is the best backup utility I’ve ever had (and yes, I also use Time Machine). SuperDuper!’s speciality is in making clones of a hard drive. I regularly make a snapshot of my laptop’s drive to an external disk. If I were to just dump everything on one disk onto another it would take 4 hours, but SuperDuper! can run an incremental backup that takes under 30 minutes.

You can also boot from the backup, which is a life saver if your main disk fails. Hold down Option during start up to boot from a different disk. This trick saved me a fortune recently when my laptop spent two weeks with Apple getting the logic board replaced, and I started using a spare Mac mini.

The UI is a rare example of a simple design that conceals great power. Rude not to buy it. Daft not to use it.

Git for version control

We all know about Git by now. I’ve invested some time in learning how to use some of Git’s more useful features (branches, rebasing, squashed merges, the reflog, etc.) and it’s really paid off.

Deploy with Vlad

Capistrano is neat, but Vlad is cleaner. I still use cap, but for new projects I always choose Vlad.

Web applications

For continuous integration I use Integrity. It looks gorgeous and, being a Sinatra app, is very easy to setup. CI Joe looks nice too, but it came out 1 week after I’d setup Integrity.

I also use GitHub and Basecamp. One of my clients is using Mingle. It’s good, but it’s pricey and I’m not sure if it’s worth the wedge.

I use most of these apps in Fluid.

Hosting

I run most of my web apps on Ubuntu, running at slicehost. I’ve also been toying with Heroku recently and first impressions are excellent. AppEngine is fun (I also love Python), but I usually need to be able to run background processes which often rules it out in production.

Books

Mmm, maybe this should be a separate list, but it could get a bit dull. The short version goes something like this:

If you’ve never read chapters 2 and 3 of Refactoring by Fowler then find a copy now. They’ll make you a better programmer. If you can spare the time read chapters 1 and 4 as well.

15” Macbook Pro

This is the best computer I’ve ever had, bar none. For a long time I toyed with the idea of getting a less powerful Macbook and an iMac. It would have cost about the same money as my MBP and the accompanying 23” screen, but I’m glad of the convenience of having everything on one machine.

Yes, I’ve had my share of Apple hardware problems. The DVD drive gave up the ghost a while back, which I haven’t bothered to fix. The hard disk failed just before it’s second birthday; I replaced it myself with a 7200rpm 500GB drive and the laptop has never felt snappier.

23” Mac cinema display

It’s one of the original screens, with a matte finish. You can’t get them any more and the new model has a glossy finish (lunacy, if you ask me). What on earth were Apple thinking?

External hard drives

I’ve got 4 (3 USB and 1 firewire), all of them 250GB.

The USB drives are 2.5” Formac disk minis. I bought them off Amazon; reasonably cheap, aesthetically pleasing and (so far) reliable. I use one for Time Machine while the other two are in continual rotation as off site snapshots (more on that later).

My firewire drive is a 3.5” Lacie, and to be honest it’s sold me on firewire. I’ll be paying extra for firewire in future. It just feels smoother.

Sennheiser headphones, Bithead amp

Some music (typically electronica) helps me focus, shutting out the hubbub around me without having a negative impact on my ability to concentrate. I’ve got a sneaky suspicion that it helps me achieve flow.

I invested in a cracking pair of high end Sennheiser headphones and a Total Bithead headphone amplifier from headphone.com to power them. Sure, you could just plug your headphones into your computer, but with headphones as good as the Sennheisers the upgraded soundcard in the Bithead (and the higher quality signal that it delivers) makes a big difference. It sounds superb.

If you find yourself working or reading on the train a portable pair of headphones to block out the noise is a good investment. I like the AKG 26P; they’re fairly cheap, have superb bass, and give you a good seal around your ear. I’ve got some Shure in ear phones too, but they’re a bit more fiddly and the bass response can’t compete. They’re good for flying and running though.

Brompton folding bike

I’ve saved the best ‘til last (photo courtesy of Guy Chapman).

Cycling is the only sane way to get around London. A Brompton folding bike is the best way to get between your clients’ offices. You can bung it on the tube/metro, hop in a cab, or if you really need to be somewhere on time, just ride the blessed thing.

You’ll obviously be taking your laptop with you, and if you stick it in the Brompton’s bag (the new S bag is the one you want) then the bike will take the weight.

Get yourself one with a hub dynamo and keep cycling through winter…

URL shortener for Basecamp

2009-11-10T00:00:00+00:00

Basecamp’s UI has one failing that keeps causing me problems – it can’t layout messages that contain long unwrappable URLs. I decided to fix it, and wrote a link shortening user script for Fluid and Basecamp.

Here’s an example:

While it looks like a corner case, it’s a problem that has occurred far too often for my liking. When it does occur, all the comments on a message are similarly affected, making it extremely difficult to follow the thread of a conversation.

To fix it all 37signals would need to do is to abbreviate the text of the relevant link tags, without changing the destination of the link. Incase they don’t ever get around to it, you can use Fluid.app and my new Link Shrinker user script to make the abbreviation for you.

Here’s the same page, fixed up with the Link Shrinker:

To install it:

Open Basecamp in Fluid.
Visit the Link Shrinker home page.
Click the “Install” button.

Reload any messages that contain long links, and you should find that they’ve been appropriately abbreviated.

Benchmarking Ruby exception handling

2009-11-06T00:00:00+00:00

Which is better, handling an exception or explicitly checking to see whether or not your code is going to break? The answer is “it depends”. On the one hand exception handling allows us to write more legible code (often summed up by the saying “it’s easier to ask forgiveness than permission”). On the other, handling an exception is often a costly operation; it can be faster to “look before you leap”.

But just how costly is exception handling? Happily it’s very easy to find out. I was working away with my pair recently when this very question came up. We were calculating a ratio between two numbers, but wanted to modify the calculation so that if the denominator was 0, we’d use 0.5 instead. Our function looked something like this:

def calculate_ratio(apples, oranges)
  apples / oranges
rescue ZeroDivisionError
  apples / 0.5
end

It’s succinct and, I think, very readable. We call this function a lot, and in a large percentage of those calls oranges will be 0. Just how expensive is that? There’s only one way to find out. Test it!

The Benchmark module contains a few nifty functions for testing this kind of scenario. I wrote two functions, ask_forgiveness and ask_permission and ran each of them a few million times with the bm() function.

require "benchmark"

include Benchmark

LOOP_COUNT = 1000000

def ask_forgiveness(a)
  1 / a
rescue ZeroDivisionError
  1 / 0.5
end

def ask_permission(a)
  denominator = (a == 0) ? 0.5 : a
  1 / denominator
end

bm(5) do |test|
  test.report("Forgiveness:") do
    LOOP_COUNT.times do |i|
      ask_forgiveness(i % 2)
    end
  end
  test.report(" Permission:") do
    LOOP_COUNT.times do |i|
      ask_permission(i % 2)
    end
  end
end

We assumed that 50% of our function calls would result in a ZeroDivisionError, which is why we’re passing i % 2 into our two functions (it alternates between 0 and 1).

And the results?

              user       system     total       real
Forgiveness:  6.780000   0.360000   7.140000 (  7.172876)
 Permission:  1.080000   0.010000   1.090000 (  1.090219)

A clear win (in this case) for checking ahead of time. It was worth benchmarking as I’d normally go with the more legible exception handling approach every time.

Exception handling is at it’s best when exceptions really are the exception to the rule…

Installing Nokogiri on Leopard

2009-10-07T00:00:00+00:00

Nokogiri is a popular (and fast) XML parsing library for Ruby. One of the reasons that it’s so fast is that it’s written in C, and makes use of the excellent libxml2 and libxslt C libraries.

I just installed the Nokogiri Ruby gem on my Mac, which runs OS X 10.5.8 (i.e. Leopard). Imagine my surprise (given that I’m supposed to be running one of the most advanced operating systems in the world!) when Nokogiri printed this message in my terminal:

HI. You're using libxml2 version 2.6.16 which is over 4 years old and has
plenty of bugs. We suggest that for maximum HTML/XML parsing pleasure, you
upgrade your version of libxml2 and re-install nokogiri. If you like using
libxml2 version 2.6.16, but don't like this warning, please define the
constant I_KNOW_I_AM_USING_AN_OLD_AND_BUGGY_VERSION_OF_LIBXML2 before
requring nokogiri.

Good one, Apple.

Aaron Kalin tells us how to solve this problem on his blog, by compiling new versions of libxml2 and libxslt from source.

I’d rather use MacPorts. Taking Aaron’s advice and modifying it for ports, here’s what you do:

$ sudo port install libxml2 libxslt -d
$ sudo gem install nokogiri -- \
    --with-xml2-include=/opt/local/include/libxml2 \
    --with-xml2-lib=/opt/local/lib

Yes, this post is largely here to remind me what to do next time this crops up. As you were…

Merging Nesta's categories and articles

2009-09-15T00:00:00+00:00

When I first started writing Nesta I wanted a CMS that would allow me to:

Write articles that would be published in an XML feed, and
Collect my articles on topic specific category pages that would introduce each topic and link to relevant articles (see my Ruby on Rails page for an example).

Articles and categories are a great way to organise a blog; they’re user friendly and search engines love them. So nothing’s changed there; I still want articles and categories. However, I’ve been using Nesta on e-commerce sites recently and have been finding that Nesta, as a general purpose CMS, didn’t quite cut it.

Update: The work described below was originally carried out on a separate unification branch. As of 7 October 2009 the changes described below have been merged into the main code base.

The problem

I made a list of the things that I thought were holding Nesta back:

To make a simple web page (e.g. a /contact page) I had to put a contact.mdown file in the categories folder. It’s not a category page, so this didn’t feel quite right.
You couldn’t host a page at an arbitrary URL; /foo and /articles/foo were supported, while /foo/bar or /foo/bar/baz weren’t.
A category page could collect links to articles, but not to other category pages. This prevented me from building up a hierarchy of topics and limited Nesta’s flexibility.
There was no way to control which category pages appeared in the menu. It hadn’t been a problem on my blog (I don’t have many categories), but for larger sites it doesn’t scale. You need to be able to choose what goes in the menu.
There shouldn’t be any need to specify the “Parent” metadata (it’s used to setup the navigation links at the top of each page). If we’re going to choose our own URLs then we can work out the parent automatically (e.g. /nesta is clearly the parent of /nesta/faq).

The solution

After mulling it over for a while I concluded that the only thing that makes articles different from any other page is that I want them to appear in my atom feed. I’ve also noticed that only the pages that appear in the feed need a date.

Assumption 1: Articles are just web pages that are published on a specific date.

If articles are just pages, what’s the difference between a category page and a generic web page? A page only becomes a category page when you’ve got other pages with related content, and declare them to belong to that category.

Assumption 2: Any web page can become a category page.

If you’ve ever browsed the Nesta code you might have noticed that an Article and a Category were slightly different beasts, and yet we’ve just concluded that they’re both just web pages with different metadata. Whoops – that’s a design flaw.

I’ve refactored Nesta, merging the Article and Category model classes into a new Page class. You can see it (and try it) in the unification branch.

A typical content folder might now look like this:

menu.txt
pages/
  |
  +---- articles/
  |       |
  |       +---- things.mdown
  |       +---- more-things.mdown
  +---- topic.mdown
  +---- topic/
  |       |
  |       +---- stuff.mdown
  |       +---- more-stuff.mdown
  +---- about.mdown
  +---- contact.mdown
attachments/
  |
  +---- foo.png
  +---- bar.png

Note that categories/ and articles/ have been replaced with pages/, and that there’s nothing special about the new pages/articles/ directory; I just want to serve some pages with /articles in the URL. You could serve things.mdown from /cabbages/things if you like; just rename the directory. You can create URLs like /foo/bar/baz simply by nesting directories.

This seems like the way forward to me. I’m planning on waiting a couple of weeks to give people a chance to try it out and give me some feedback, before merging the unification branch into master.

As an added bonus the code is now simpler, runs faster, and uses fewer resources.

Trying it out

If you’re not already running a copy of Nesta you can check out a fresh copy and get started straight away with the new unification branch (you’ll need to install the relevant dependencies first – see the “Quick Start” section on the main page):

$ git clone git://github.com/gma/nesta.git
$ cd nesta
$ git checkout -b unification origin/unification
$ cp config/config.yml.sample config/config.yml
$ rake setup:sample_content
$ shotgun app.rb  # point your browser at http://localhost:9393

If you’re already running Nesta you can get yourself a local copy of the unification branch like this:

$ git fetch origin unification
$ git checkout -b unification origin/unification

If you’ve got some existing content then we’ll need to shuffle it around a bit. I’ve written a migration script to do it for you. Run it from within your Nesta directory:

$ ./scripts/unification-migration

The script automates the following:

Makes a menu file (<content>/menu.txt) from your old category pages.
Moves <content>/categories/* to <content>/pages.
Moves <content>/articles/* to <content>/pages.
Removes “Parent: …” metadata declarations, which are no longer used.

If your content is stored in a git repository the script will stage the changes for you, but will leave the task of committing the changes to you.

If you like the new layout of the content directory you can try merging the new Nesta code into your own site. If, for example, you had your own design changes stored in a branch called mysite you could merge the two together like this:

$ cd nesta
$ git checkout mysite
$ git merge unification
# Review the changes, resolve any conflicts...
$ git commit -m "Merged the unification branch."

After resolving any conflicts that might occur (I only had one when I updated effectif.com which I fixed by removing the category.haml file – page.haml has replaced it) you should find that your site looks just as it did before, but that it’s suddenly a whole lot more flexible…

I’d love to know how you get on. You can get in touch via Twitter or on GitHub. Or just drop me an email at graham@effectif.com.

Testing Rails with Rack::Test

2009-06-12T00:00:00+00:00

The biggest news in Rails 2.3 is its support for Rack, the WSGI inspired Ruby web server interface. Of all the Rack goodness that has come along lately, the one that has me the most excited is Bryan Helmkamp’s Rack::Test library, of which Bryan said “Basically, I extracted Merb’s request helper code into a small, reusable, framework agnostic library.”

I loved Merb’s request specs. I suspect that I’m going to like Rack::Test too.

Installing Rack::Test

We’ll start by installing Rack::Test and loading it into our Rails app. Append this line to config/environments/test.rb:

config.gem "rack-test", :lib => "rack/test"

You can now use rake to install it:

$ sudo rake gems:install RAILS_ENV=test

Running tests with “rake test:rack”

We want to be able to run these tests easily with rake, just like our unit or functional tests. Create a new file called lib/tasks/rack-test.task and paste this code (courtesy of recipe 53 from the Advanced Rails Recipes book) into it:

namespace :test do
  Rake::TestTask.new(:rack => "db:test:prepare") do |t|
    t.libs << "test"
    t.pattern = "test/rack/**/*_test.rb"
    t.verbose = true
  end
  Rake::Task["test:rack"].comment = "Run the Rack::Test tests in test/rack"
end

task :test do
  Rake::Task["test:rack"].invoke
end

Note that we have also extended the :test task so that our Rack tests will get run after our unit and functional tests.

You should now be able to run rake test:rack, but it won’t do anything yet. Let’s test it by dropping this code into a file called test/rack/dummy_test.rb:

require File.join(File.dirname(__FILE__), *%w[.. test_helper])

class DummyTest < ActiveSupport::TestCase
  include Rack::Test::Methods

  def app
    lambda { |env| [200, {}, "Coolness"] }
  end

  test "should be hooked up properly" do
    get "/"
    assert last_response.body.include?("Cool")
  end
end

When you run rake test:rack you should get something like this:

$ rake test:rack
(in /Users/graham/data/effectif/projects/canvas)
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -I"lib:test" "/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake/rake_test_loader.rb" "test/rack/login_test.rb" 
Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake/rake_test_loader
Started
.
Finished in 0.008904 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

Testing your Rails app

You’ll have noticed that the app method (which is supposed to return an instance of the Rack application under test) doesn’t actually have anything to do with Rails. It returns a callable object that needs to be compliant with the Rack API (apps are invoked via their call method, which is also how you invoke a lambda).

Let’s fix that by making the app method return an instance of our Rails application.

For an example of how Rails instantiates a rack app have a look inside commands/server.rb. Scroll down until you find the call to Rack::Builder.new. From there we can see that Rails is using ActionController::Dispatcher.new. That should be enough for most purposes (if not you can see what else you may need to add by poking around in server.rb).

That gives us a template test file that looks something like this:

require File.join(File.dirname(__FILE__), *%w[.. test_helper])

class MyTest < ActiveSupport::TestCase
  include Rack::Test::Methods

  def app
    ActionController::Dispatcher.new
  end

  test "home page" do
    get "/"
    assert last_response.ok?
  end
end

If you want to get a bit more fancy and test things that aren’t handled by ActionController::Dispatcher you can put your app together with Rack::Builder. As a contrived example, if you wanted to test for the presence of static files (I said it was contrived) you could create your app like this:

def app
  Rack::Builder.new {
    map "/" do
      use Rails::Rack::Static 
      run ActionController::Dispatcher.new
    end
  }.to_app
end

This article was written using Rails 2.3.2, Rack 1.0.0 and Rack::Test 0.3.0.

Why Bother?

Shortly after I first posted this article, crispee commented on reddit asking why you’d bother to use Rack::Test instead of the functional tests that come with Rails. It’s a good question, and I completely failed to cover it. Ooops.

Firstly, Rack::Test is more similar to the Rails integration tests than to the functional tests. Functional tests only allow you test the actions and views of a single controller. Rack::Test allows you to write tests that visit pages anywhere on your site – you can simulate the behaviour of a real browser to ensure that your entire app hangs together properly.

Secondly, both the Rails functional and integration tests are closely coupled to Rails. Imagine for a moment that you’ve been running a Rails app for a while now. You’ve invested a lot of effort in your application; users can log in and do all sorts of clever stuff. In the meantime you’ve been hand crafting the sales pages (e.g. home page, tour, FAQ, etc.) have added them to your app. All these static pages are served by one big controller and your routes file is getting to be a bit of a mess. It’s high time that you introduced a CMS.

You’ve also recently stumbled across a hot new rack compatible web framework (for the sake of argument, let’s call it Sinatra). You quite fancy using a Sinatra CMS (sorry, shameless plug) while continuing to develop the main part of your application in Rails. Rack makes this very easy to do in the same Ruby process (see Rails meets Sinatra for details).

Obviously, you wrote a test to demonstrate that a new user can click on the “Buy Now” link on your home page, navigate their way through your signup forms, and end up on the home page. If you were also astute enough to write that test using Rack::Test, that test will still work after you move some of those pages into the Sinatra CMS.

If you refer to Pratik’s “Rails meets Sinatra” post that I linked to above, you’ll see that you’d only have to change the way the rack app is instantiated in your test:

def app
  Rack::Builder.new {
    # URLs starting with /account (logged in users) go to Rails
    map "/account" do
      run ActionController::Dispatcher.new
    end

    # Everything else goes to your CMS
    map "/" do
      run Sinatra.application
    end
  }.to_app
end

How cool is that?

Useful Resources

For more information check out:

Bryan’s original blog post on Rack::Test – a good introduction to the test methods.
The Sinatra documentation – includes some handy pointers on using Rack::Test.
The Scaling Rails screencast on Rack – a terrific introduction to Rack, Metal and the Rails Rack middleware.

Installing the pg PostgreSQL gem on Mac

2009-06-10T00:00:00+00:00

I’ve recently switched from using MySQL to PostgreSQL for my Ruby projects. You can connect to a PostgreSQL database from Ruby using the pg gem, but if the pg_config program isn’t in your path you’ll run into problems during installation.

$ sudo gem install pg 
Building native extensions.  This could take a while...
ERROR:  Error installing pg:
	ERROR: Failed to build gem native extension.

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb install pg
extconf.rb:1: command not found: pg_config --version
ERROR: can't find pg_config.
HINT: Make sure pg_config is in your PATH
...

A few minutes later I realised I’d installed my copy of PostgreSQL using the one click installer, which puts the database server in the /Library/PostgreSQL directory. This sorted it:

$ PATH=$PATH:/Library/PostgreSQL/8.3/bin sudo gem install pg