James on Software

Building clusters in the cloud with conductor

2009-09-28T14:30:53-07:00

I've been using chef in production for a while now. It saves me untold time and energy. Without it, I couldn't possibly manage our 15+ server cluster, in addition to writing and maintaining our code base. As is often the case, however, removing one bottleneck exposed several more.

Currently, we use a set of custom rake tasks and capistrano recipes to launch ec2 instances and deploy our chef recipes. For various reasons, this includes committing a sqlite database to a git repository. The system works, but it isn't pretty. Launching and configuring an instance, for example, requires several commands. And, without a running application, things like auto-scaling are basically out of the question.

I began to see a real need for an always-running application to manage my infrastructure. It would make everything easier, from launching instances to backing up EBS volumes. Today, I'm releasing an early alpha of that application.

conductor

Conductor is the beginnings of an infrastructure management application. The version I'm releasing today has a rather limited feature set. But, the ground work is there. So, most of the fancy stuff should be relatively easy to implement.

Right now, conductor is able to provision instances from ec2 and configure them with chef. You provide a url to a git repository, which contains your cookbooks and some metadata. Conductor will launch your instances and configure them with the cookbooks from that repository. I'm also releasing a basic rails stack that I'll be maintaining. I'm hoping others will contribute stacks for other plaforms.

Currently, things are tightly coupled to the structure of standard web infrastructure. Only two roles are supported: app, and mysql_master. The next major item on my TODO list is to make the launching system far more flexible, so that conductor can be used to manage any kind of infrastructure. I should get to this soon, since our infrastructure still isn't supported.

Roadmap

As much as I hate vaporware, the possibilities for conductor are exciting. Your indulgence is appreciated :-).

On-demand staging. Clone the entire production environment, including databases in one click. Terminate it when you're done. Pay for what you used.
Monitoring and auto-scaling. Pipe metrics, like CPU usage in to conductor. If average CPU usage on the app servers gets above a certain threshold, launch another one. If average CPU drops below the threshold, kill one. Only run the servers you actually need.
Coordinated snapshotting of EBS volumes. One of the problems with running EBS volumes in a RAID configuration is backups. If you are snapshotting more than one drive at a time, you need some way to keep track of which sets of snapshots go together for recovery. Even better, a central app can be smart enough to perform the recovery itself.

Follow Along

As I said, today's release is early alpha. But, I'm going to be putting a lot of work in to this, because it'll make my job a hell of a lot easier, and save us money. So, follow along with the development by following the github project or subscribing to my blog.

If you want to try conductor out, there's a ton of information in the README. I'll be hanging out in #conductorapp on freenode if you're having trouble getting things running.

Observational: Better Observers for ActiveRecord

2009-08-05T18:13:59-07:00

This sort of thing appears in just about every rails app I've ever worked on:

class User
  after_create :send_welcome_email

  protected
    def send_welcome_email
      Notifier.deliver_welcome_email(self)
    end
end

Why would the user have to ask the notifier to send him his own welcome message? That seems like the Notifier's responsibility, if you ask me.

Using an ActiveRecord observer, we can absolve the user of that responsibility. That code would look something like this:

class UserObserver < ActiveRecord::Observer
  def after_create(user)
    Notifier.deliver_welcome_email(user)
  end
end

Note that you'd also have to activate this observer in config/environment.rb.

In some ways, using an observer is better than putting a callback in the User model. It's definitely a better division of responsibilities. But, it feels a little cumbersome.

In the rails app where I spend most of my time, we currently use callbacks to trigger over a dozen mailers, and almost 50 types of activity feed items. So, we'd need a lot of observer subclasses.

A Better Way

The observer pattern, in some form, seemed like the right approach. I wanted to find a way to use it without creating eleventy billion observer classes.

I came up with a gem that I call observational. Here's what it looks like:

class Notifier < ActionMailer::Base
  observes :user, :after => :create, :invokes => :deliver_welcome_email

  def welcome_email(user)
  end
end

This example is functionally equivalent to the first example above. Notifier.deliver_welcome_email will be called during the after_create callback of any User object, with the user object as the first and only argument.

Observational supports any standard ActiveRecord callback. As far as I know, anything you can do with a standard AR observer, you can do with observational. Let me know if I'm wrong about that.

Get It!

sudo gem install giraffesoft-observational

Fork it at github.

Or read the yardoc at rdoc.info.

Introducing Trample: A Better Load Simulator

2009-06-04T11:46:39-07:00

Most load sim tools make requests to a static list of urls. They spawn n threads and make requests to the urls on the list in succession, in each thread. Unfortunately, though, if your applicaition makes use of any kind of caching (including your database's internal caching facilities), this kind of load simulation is unrealistic.

The data required to serve a single set of urls is likely to fit nicely in the database server's cache (even on a small server). So, having a single user hammer a finite set of pages will make your application look much faster than it really is.

Unless your load patterns really look that way, you're testing for a scenario that you'll never see.

Introducing Trample

Trample is a more flexible load simulator. Instead of a static list of urls, trample's configuration language is ruby. Using ruby's blocks (lambda functions), it's possible to randomize the requests that get made in each thread, as well as the user that logs in.

Let's look at an example config:

Trample.configure do
  concurrency 20
  iterations  10
  login do
    post "http://mysite.com/login" do
      {:username => User.random.username, :password => "swordfish"}
    end
  end
  get "http://mysite.com/posts/:id" do
    {:id => Post.random.id}
  end
  post "http://mysite.com/posts/:id/comments" do
    {:id => Post.random.id, :author => "James Golick", :body => "lorem ipsum"}
  end
end

This config will cause trample to spawn 20 threads. In each thread:

Trample will begin by sending a POST request to http://mysite.com/login with the result of the block as its POST body.
Then, trample will GET http://mysite.com/posts/(some interpolated value here, based on the result of the block).
Then, trample will POST http://mysite.com/posts/(interpolated value here)/comments with a POST body that includes the remainder of the parameters returned from the block
Repeat steps 2-3 nine (iterations - 1) more times.

It's possible to create fairly realistic load simulations with a configuration like this. Each thread will start by logging in as a different user. And, the threads will each visit different URLs. The ratio of cache hits to misses should be far closer to what they look like under real load.

Running Trample

To run trample:

$ trample start /path/to/your/trample/config.rb

I've tested trample on ruby 1.8.x, ruby 1.9, and JRuby. JRuby has the advantage of native threading, but the drawback of JVM warmup times. So, theoretically, JRuby should be the best platform for running long tramples. Ruby 1.9 should be fastest for shorter tramples. I could be wrong, though. YMMV.

Get It!

Get trample as a gem:

$ sudo gem install giraffesoft-trample

Or get the source from github.

How to Watch Hulu From Canada (or Anywhere Else)

2009-05-24T17:07:40-07:00

I live in Canada. We don't have guns. But, we do have health care.

Okay, so there are a few other things we don't have — one of them being hulu. Yeah, there's always watchtvsitcoms.com. But, it's slow, full of annoying flash ads, and the quality of the shows is spotty.

A while back, I tried to beat hulu's geofencing (fuck geofencing) by creating an SSH tunnel. That was a miserable failure. Then, recently, I saw somebody tweet about using OpenVPN. So, I gave it a shot, and it worked!

How To Do this Yourself

Now, I could tell you to go and read the official tutorial on setting up OpenVPN. But, I've got something better.

I put together a chef^[1] recipe for installing OpenVPN on a virtual machine. The whole process is automated. So, you don't really need to know anything about OpenVPN to get up and running.

Here's the guide:

Get a linode (or equivalent virtual machine, located in the US).
Setup your vm with Ubuntu.
Install ruby & rubygems on the vm:
```
sudo apt-get install ruby rubygems
```
On your local machine, clone the cookbook repository:
```
git clone git://github.com/giraffesoft/openvpn-recipes.git
```
Or, if you don't have git installed, grab a tarball from github. Then, unarchive it:
```
tar zxvf giraffesoft-openvpn-recipes-xyz.tar.gz
```
You'll also need capistrano (on your local machine).
```
sudo gem install capistrano
```
Now, cd in to the dir you cloned or extracted. Then, edit config/deploy.rb to match your settings.

You'll want to set :host to your server's domain name or ip address. You'll also want to set user to the name of the user you're going to get ssh'd in as. Finally, set the country, province, city, etc variables. Those are for generating the signing authority.

You'll want to setup an ssh keypair on your server, too, to make this easier.
Now that everything is all configured, run chef, by typing this command at the root of the openvpn recipes project:
```
cap run_chef
```
OpenVPN is now installed and configured!
Now, create a client package by typing:
```
cap generate_client_package
```
There should now be a file at build/client.tar.gz. Unarchive this file to wherever you want to keep your OpenVPN configuration. Then, start OpenVPN on your local machine. That's it!

Note: if you're using a mac, you'll want to use Tunnelblick to manage your vpn connection. Just unarchive the client package to ~/Library/openvpn and Tunnelblick will take care of the rest.

You should now have a working VPN. Enjoy hulu!

Also, if you're interested, checkout the code at github.

^[1] Incidentally, if you do absolutely any work with servers or systems configuration and you haven't had a look at chef yet, you're doing yourself a disservice. Chef allows you to keep all of your servers' configuration in a git (or other scm) repository, edit it locally and deploy it out to your server (or cluster thereof). It also does a lot of other awesome stuff. Check it out!

The New and Improved James on Software

2009-04-08T07:26:48-07:00

I finally got around to having this site redesigned — something I've wanted to do for a while. James on Software plays a large role in my personal brand. So, having great design is of paramount importance.

There's not much more to say about the new design, except that I did want to mention the designers.

When I arrived at a good company's site for the first time, I knew I had found my designers. Their work really speaks for itself. Definitely check out their portfolio.

In addition to being awesome designers, they do great XHTML/CSS and even know some rails. They're spearheading The League of Movable Type, an open source font movement. So, if you read this blog, there's a good chance you have a lot in common with them.

If you're looking for a design firm for your next project, you should really check this duo out.

Bringing Computer Savvy & Web 2.0 to Pre-Teen Girls

2009-03-30T16:41:48Z

I'm a huge fan of screencasting. It's a great way to learn about software development. Screencasting is also starting to be used to teach general daily computing topics. Nevertheless, it remains something of a niche. The general public still doesn't really know about screencasting.

There certainly isn't a whole lot of screencast content aimed at kids, except maybe a programming tutorial or two. There's nothing (I can find) aimed at young girls.

Hailey Hacks aims to fill that void in the market. Hailey is a savvy pre-teen girl. She's fun — and a little bit rebellious. You might think of her as a cross between Bart and Lisa Simpson.

In the first episode, titled April Fools, Hailey teaches you how to play pranks, low tech and high. First, she suggests that you fill your father's shampoo bottle with ketchup. Later in the episode, she recommends using wakerupper.com to incessantly ring your father's phone while you're at school. I'll let Hailey show you the rest of them.

So far, we've got one other episode up there, in which Hailey teaches you to make your own lolcats. Lots more are coming. Hear first by following @HaileyHacks or subscribing to Hailey's blog.

Despite the fact that this is web 2.0, we're selling the videos for 2$ each over at the Hailey Hacks site. We'd love to hear what you and your kids think of them!

Firefox-style cmd-1 through cmd-9 shortcuts for switching tabs in Safari!

2009-02-26T14:48:10Z

Safari is by far the best browser on OS X. It's fast, has an awesome UI, and just feels better integrated in to the Mac experience.

As former die-hard firefox users, though, I couldn't live without cmd-1 through cmd-9 for switching tabs. Thanks to Safari Commander, now, I don't have to!

For 5$, one person can use Safari Commander on as many of *their* computers as they like. Two members of the same family with separate computers should purchase two copies.

WARNING: REQUIRES SAFARI 4 BETA. Available for download here. So far, I've found it very stable and worth it for this extension alone.

Buy it at the giraffesoft store!

FLOSS week

2009-02-16T21:02:48Z

We at giraffesoft went over a year without even having a website. Then, we lasted a few months with a website, but no blog. That ends today, hopefully with something of a bang.

We're big believers in extracting functionality sooner rather than later. This practice has resulted in a ton of code that we use in all of our projects — nice and DRY.

Only problem is, we never get around to releasing this stuff. Starting a blog seemed like a good excuse to spend some time polishing up some code and, you know, writing READMEs.

So, starting today, we'll be releasing an open source project every day this week. They'll probably be mostly rails plugins. But, you never know. Something else might float in.

If you're interested in following along, subscribe to giraffesoft, the blog. Or, if like me, you don't really use your feed reader anymore, follow @giraffesoft on twitter or github! See you later with the first project.

One Line Tests Without the Smells

2009-01-16T18:13:26Z

Everybody loves one line tests. That's one of the great things about shoulda. It allows you to test a lot of simple functionality using one liners, called "macros". Unfortunately, though, there are a lot of things about shoulda macros that make them less than ideal as a testing mechanism.

Firstly, they get called at class scope, so the variables from your setup blocks isn't available. A quick read through the archives of the shoulda mailing list proves that this confuses people. The workaround for this issue is a major code smell: string programming.

For example...

should_redirect_to 'post_url(@post)'

Here, 'post_url(@post)' is evaluated at the scope of the controller. A parse error in your string wouldn't point you to your should_redirect_to declaration. It would point you to a line inside of shoulda, where the instance_eval actually happens. Not good. There are lots of other reasons string programming sucks, too, but I won't get in to them here.

The second issue with macros is that failures often happen deep inside of shoulda. Backtraces often become completely useless, forcing you to open up shoulda and wade through it to figure out why your code is failing.

One More Problem

context "A blog post" do
  setup do
    @author = create_person
    @post   = create_post :author => @author
  end

  should "be editable by its author" do
    assert @post.editable_by?(@author)
  end
end

Don't see the problem?

Why bother writing self-documenting test code if you always have to explain it to the reader? Test names are essentially glorified comments and comments are frequently code smells. Furthermore, all the extra code required to create a test (should "" do ... end) almost certainly discourages one assertion per test. If the assertion is one line and the code can explain itself, why bother with all the other crap?

The Solution: Zebra

context "With a blog post" do
  setup do
    @author        = create_person
    @somebody_else = create_person
    @post          = create_post :author => @author
  end

  expect { @post.to be_editable_by(@author) }
  expect { @post.not_to be_editable_by(@author) }
end

But, what about the test name?

I'm glad you asked. This is where zebra gets really cool. The above code will create tests with the following names:

"test: With a blog post expect @post.to(be_editable_by(@author))"
"test: With a blog post expect @post.not_to(be_editable_by(@somebody_else))"

Now, that is self-documenting code.

The right tool for the job

The cool thing about zebra is that it's an extension to context or shoulda and matchy ~~(shoulda support coming very soon)~~. If you have a test that belongs in an it or should block, with a big, old-fashioned test name, you can have it. Just use should or it. When you have a short, self-documenting test, use expect. Best of both worlds.

Get It!

`sudo gem install giraffesoft-zebra`

http://github.com/giraffesoft/zebra

Speaking at CUSEC

2009-01-11T15:20:22Z

If you live in the area and you're not going to CUSEC, you're crazy. The speaker lineup is absolutely amazing, including such names as...

Richard Stallman (yes, the famous RMS)
Dan Ingalls
Avi Bryant (if you've never heard Avi speak, you don't know what you're missing)
Giles Bowkett

Sounds expensive, right? Wrong.

All three days of the conference cost only $150.

Also, I'll be speaking Friday morning. Here's the abstract...

Storming the Java Bastille: a political introduction to dynamic languages

Working with rigid, static languages is like living in a communist dictatorship. Your freedom is extremely limited, there’s a lot of bureaucracy, and it’s often painful.

Working with dynamic languages, by contrast, is like living under libertarian rule. The government is small and doesn’t like to make decisions on your behalf. They provide you a lot of freedom and leave it up to the community to decide how best to do things. But, with great power comes great responsibility. Dynamic languages are like a chainsaw: powerful, but watch out for your extremities.

In this code-heavy session, we’ll take a look at the pros and cons of both styles, hopefully without losing any limbs.

If last year's CUSEC was any indication, this is going to be a great conference. Hope to see you there!