The So-Called Writings of Michael Gorsuch

Sucker Punching the Algorithm with Redis

2011-09-09T00:00:00-07:00

Last night I attended my first OK.rb meeting. I moved back to the OKC area from New York several months ago, but it took me some time to get things in order before I could start attending. You know – kids, man.

Anyway, I have nothing but good things to say about this group. It’s full of smarts and enthusiasm – the sort of thing any hacker-type should be surrounding himself with. Even better, they work on actual code problems. It’s not just fanboy hour; they are actually examining problems and working them out together. A great way to sharpen skills for sure.

The problem presented was clear: understand this code, then optimize it.

# code example provided by James Gray II (http://twitter.com/jeg2)
DICTIONARY = "/usr/share/dict/words"

class String
  def to_signature
    downcase.delete("^a-z").split("").sort.join
  end
end

scrambled = ARGV.first.to_s.to_signature
abort "USAGE:  #{$PROGRAM_NAME} SCRAMBLED_WORD" if scrambled.empty?

File.foreach(DICTIONARY) do |word|
  puts word if word.to_signature == scrambled
end

The code is clear enough: we’re monkey patching the String class, allowing us to generate a simple signature for a given word. We convert the word to lowercase, rip out any non-alphabetic characters, and then place the characters in alphabetically order.

An example of it in action via irb:

ruby-1.9.2-p180 :008 > "test".to_signature
 => "estt"

It’s worth noting that signatures are not necessarily unique:

ruby-1.9.2-p180 :010 > "host".to_signature
 => "host" 
ruby-1.9.2-p180 :011 > "shot".to_signature
 => "host"

So, our code takes a word as an input, obtains its signature, and then finds every word in the dictionary which has the same signature – a very brute force approach, no? This leads to the next part: optimization.

Measure First

The first step is to start measuring execution time. I’m paraphrasing, but our first move was to wrap our algorithm in a method and include some simple debug timing:

DICTIONARY = "/usr/share/dict/words"

class String
  def to_signature
    downcase.delete("^a-z").split("").sort.join
  end
end

def descramble(scrambled)
  File.foreach(DICTIONARY) do |word|
    puts word if word.to_signature == scrambled
  end
end

scrambled = ARGV.first.to_s.to_signature
abort "USAGE:  #{$PROGRAM_NAME} SCRAMBLED_WORD" if scrambled.empty?

start = Time.now
descramble(scrambled)
puts Time.now - start

An initial run looks like this:

$ ruby descramble.rb test
sett
stet
test
2.703979

Alright. Our algorithm takes 2.7 seconds to brute force the word list.

First Improvements

The first improvement that came to the group mind was to see if the first word of each dictionary word was even included in our signature. If it is not, there is no need to run the expensive to_signature method.

def descramble(sig)
  File.foreach(DICTIONARY) do |word|
    next unless scrambled.include? word[0]
    puts word if word.to_signature == sig
  end
end

A clear win:

$ ruby descramble.rb test
sett
stet
test
0.594181

Nice! That’s about 4.5 times faster than the original.

Redis Makes it Fast!

As the session wore on, we were able to work out some seeming improvements, but nothing that made a huge difference. The only other thing we discussed would be to pre-index the word file, but that seemed to be cheating.

In the real world, though, I don’t think that there really is such a thing. We want to make this thing fast, and the only way we’re really going to do that is to index the file before hand. Upon returning home, I quickly whipped up a solution using Redis as my data store.

First, we pre-load the data:

# cheat.rb - preload our word data into redis sets, using the signature as the key
require 'redis'

DICTIONARY = "/usr/share/dict/words"

class String
  def to_signature
    downcase.delete("^a-z").split("").sort.join
  end
end

# our interface to the local redis instance
r = Redis.new
# go ahead and clear out anything that we might have in here
r.flushall

File.foreach(DICTIONARY) do |word|
  r.sadd word.to_signature, word
end

That takes about 17 seconds to run, but we only have to do it once. After that, the indexed data persists and we can query it as needed.

Our app changes to rely upon Redis:

require 'redis'

class String
  def to_signature
    downcase.delete("^a-z").split("").sort.join
  end
end

def descramble(sig)
  r = Redis.new
  puts r.smembers sig
end

scrambled = ARGV.first.to_s.to_signature
abort "USAGE:  #{$PROGRAM_NAME} SCRAMBLED_WORD" if scrambled.empty?

start = Time.now
descramble(scrambled)
puts Time.now - start

The performance improvement is obvious:

$ ruby descramble.rb test
test
stet
sett
0.001002

0.001002 seconds! That’s nearly 2,700 times as fast as the original piece of code. Ah, the spoils of cheating!

Conclusion

Of course, Redis is not the only choice here, but given the scope of the problem, it seemed like a fair pick. It is also such a cheap tool, meaning that there isn’t much to getting it running and putting it to quick use.

Automate Your Development Environments. Please.

2011-05-06T00:00:00-07:00

[note: much of this is inspired from what I’ve learned from my peers at Heroku. If this sounds like your kind of place, we are hiring.]

Your team’s development environments should be managed by automated processes: you should be able to fabricate a clone of production in a manner of minutes. If your team has to invest days in the setup of an environment only to have everyone drift apart a few weeks later, you are probably doing it wrong.

Give strong consideration to employeeing the likes of Vagrant (or something similar) in your environment. If you rely on AWS, put something like CloudFormation to use.

Your production, staging, and development environments should be managed by something like Chef or Puppet (or something else, but whatever you do, don’t roll your own). No one-off machines. As much as I love the variety and differences among my peers, I find it intolerable in systems. And so should you.

If you do this, your development team will be able to walk in each morning and boot a complete development cloud. As each system boots, it will execute the same build procedures as used in production, thus allowing those procedures to be put to the test several times a day. Problems and bad practices will make themselves known quickly. Your development environment will be regularly refreshed, preventing the dreaded drift that plagues so many. Even better, you will be working on an environment that mirrors production, thus eliminating the very cause of most “cowboy” behavior.

Fixing the Author on a Git Commit

2011-03-31T00:00:00-07:00

Ugh. So you just made a commit, take a look at the revlog and realize that you did it as root. How embarrassing. Assuming you haven’t pushed, however, you can fix it without much trouble.

Try this:

$ git commit --amend --author "Your Name <you@domain.com>"

Squashing Git Commits

2011-03-30T00:00:00-07:00

All too often while modifying some chef recipe, I make commit after commit after commit. Many times my local branch’s commit messages look a little like this:

commit 473c878c8bf249a474da9faf57333288aaa0d22f
Author: Michael Gorsuch <michael.gorsuch@gmail.com>
Date:   Mon Mar 21 16:53:10 2011 -0500

    recipes[apache2]: crap. typo.

commit 27bda2b580c8fb697615fca84933d2f89d26c70b
Author: Michael Gorsuch <michael.gorsuch@gmail.com>
Date:   Mon Mar 21 08:58:40 2011 -0500

    recipes[apache2]: added ability to generate sites based on attributes

...

I don’t like merging that sort of mess back in to the main branch, as it can cause some headaches when you have a large team that would rather be doing other things than watching you backtrack all over the codebase.

A colleague asked me to check into git’s rebase functionality. What I found was scary but ever so useful.

Let’s squash these commits together, making it look as if the typo had never occurred.

Use the following command:

$ git rebase -i HEAD~2

-i tells git to use interactive mode, i.e. launch $EDITOR, and HEAD~2 is more or less saying get the past two commits.

When I run this, VIM gives me the following:

pick 0259fff recipes[apache2]: added ability to generate sites based on attributes
pick ad68659 recipes[apache2]: crap. typo.

# Rebase be9f5a0..ad68659 onto be9f5a0
#
# Commands:
#  p, pick = use commit
#  r, reword = use commit, but edit the commit message
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#  f, fixup = like "squash", but discard this commit's log message
#  x, exec = run command (the rest of the line) using shell
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#

Let’s go ahead and do a fixup here, pretending that this change never happened. Just change the top two lines to look like so:

pick 0259fff recipes[apache2]: added ability to generate sites based on attributes
f ad68659 recipes[apache2]: crap. typo.

Now if you exit out of vim, your change will take place. Look at the commit log now:

commit 8654333d73b46f93ae111b40986dff8329b813a2
Author: Michael Gorsuch <michael.gorsuch@gmail.com>
Date:   Mon Mar 21 08:58:40 2011 -0500

    recipes[apache2]: added ability to generate sites based on attributes
...

Fixed and you look more perfect than ever.

Build Debian Packages Easily with FPM

2011-03-22T00:00:00-07:00

Today I discovered that the version of collectd that ships with Ubuntu Lucid has trouble with HTTP basic authentication. To verify, I grabbed the latest tarball, which is several revisions beyond the Ubuntu package… and it worked. So what do I do now? Debian packages are fast, easy to distribute, and work well with Chef… but they are a pain in the ass to build for people who don’t do it regularly.

Enter FPM, or “Effing Package Management”. Jordan Sissel built this so he didn’t have to fight with the package process any longer. Kudos to him, because this freed up my afternoon.

Just so one more example gets out there, here is how you can use it.

First off, grab the tarball of the software you want to convert into a package. I am going with collectd here, but anything that follows the traditional make install route ought to work. Go ahead and run ./configure and make install. I’ll wait.

As it stands now, collectd installs in /opt/collectd out of the box. Assuming that is how we want to keep it for production, we can go ahead and build our custom package.

Now install fpm:

$ gem install fpm

Build your deb with the following commands:

$ fpm -n collectd -v 4.9.4-custom-1 -t deb -s dir /opt/collectd

A little explanation will go a long way here. -n gives the new package a name. -v allows us to state the version. I appended -custom-1 as that should help me distinguish from the vendor. -t lets me specify the package type, such as rpm or deb. -s dir states that I am building my package from the contents of a directory. Finally, the last item tells fpm which directory to build from.

Clear?

Upon completion, you should have a fresh deb in your directory. I installed mine like so:

$ sudo dpkg -i collectd_4.9.4-custom-1_i686.deb

Big thanks to Jordan for making life better for all of us.

A Taste of Cuba

2011-03-22T00:00:00-07:00

Late last week I ran across a neat little web framework called Cuba. I write small tools, so most of my work can easily fit into something like this. Something that it has that I’ve been dying for: the ability to match urls components only once. See my snippit below for an example:

require 'cuba'

Cuba.define do
  on "api/v1" do
    on "rooms/:id" do |id|
      @room = Room[id] 

      # below could be hit like so:
      # /api/v1/rooms/123/messages
      on "messages" do
        res.write "#{@room.name} has #{@room.messages.size} messages."
      end
    end 
  end

  on "api/v2" do
    # ...
  end
end

I’ll be applying this to a real-life production problem this week. For more information, see this great presentation.

Rolling Your Own Self-Signed SSL Certs

2011-03-20T00:00:00-07:00

This isn’t rocket science, but I have to do this so rarely that I end up having to Google it due to lack of familiarity.

Let’s say you want to build your own self-signed certs. Here is how you go about doing that.

First we build the private key:

$ openssl genrsa -des3 -out server.key 2048

This above action will require you to set a password on the key. For most use cases (say, sticking it on nginx), you’ll need the password removed. Do so like this:

$ cp server.key server.key.org
$ openssl rsa -in server.key.org -out server.key

Next, create the CSR:

$ openssl req -new -key server.key -out server.csr

Finally, build the cert from the CSR:

$ openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

There you go. Easy peasy.

Falling in Love With My Work Again

2011-03-11T00:00:00-08:00

In October of last year I made a small note that I had re-introduced myself to the Ruby ecosystem. It was a willed attempt to get some deeper parts of my mind working again after several years of working predominantly with tools that just weren’t doing it for me. Language and platform agnosticism has its place and benefits, but it was time for me to get back to digging a little more joy out of my work.

Once that decision was made, things began falling into place. I crafted a few tools and got my socks blown off by learning a little about behavior driven development.

I was then fortunate enough to pick up a job at Heroku, a shop that has changed the world of application delivery for the better. And it will continue to blow your mind. I promise. I’m focusing on ops engineering, helping craft an environment that lives entirely in AWS, implementing smart configuration management practices using tools like stem and chef, and building sharp, small-scoped components wherever possible. It’s challenging and I think that it will continue to push me hard.

The ops team is entirely distributed, so that’s a sweet perk. Because of that, my family and I can now relocate back to our home town of Oklahoma City. I’m really looking forward to connecting with the budding tech scene there. People are making things.

I am writing this on the airplane as I fly back from my first on-site visit with the Heroku team. I now know that I made the right choice to come aboard as I am now fully convinced that everyone in that building is smarter than I am. Which is great. I have something to aim for.

So things are good. Hope they are for you, too.

An Introduction to Stem - How to Easily Fabricate EC2 Instances

2011-02-27T16:30:00-08:00

When I began working with the Heroku team, one of the first tools I had to learn was Stem. Originally developed by Peter van Hardenberg and based upon Judo, Stem was built to ease the the pain that comes about when building custom Amazon EC2 instances.

Our Simple Example

Assuming that you are like me in that you have both a short attention span and little time on your hands, we’ll keep things simple. In subsequent posts I’ll go into more detail and better practices, but for now we’ll focus on creating a simple ‘base’ AMI that we can fabricate to our heart’s content.

Once complete, we should be able to launch 100 of these suckers if we had the means to pay for them.

Prerequisites

First, install stem:

$ sudo gem install stem

Then add your AWS credentials to your environment (I add mine to my ~/.profile):

export AWS_ACCESS_KEY_ID=XXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXX

Don’t forget to do this:

. ~/.profile

Next…

Start Building!

Run the following to setup the basic directory structure:

$ mkdir -p stems/bare stems/base
$ touch stems/bare/bare.json
$ touch stems/base/base.json

We’ll start by building out a ‘bare’ instance, which will just be a standard public AMI. Once we get this guy up and running, we’ll make a few changes and then build a new AMI out of it for future use.

Let’s take a look at our bare.json:

{ 
  "ami"                       : "ami-4a0df923",
  "instance_type"             : "t1.micro",
  "key_name"                  : "default"
}

There are three parameters here, all of which are fairly self-explanatory. ami refers to the AMI we want to launch, being an Ubuntu Lucid instance in this case. instance_type tells us what sort of system. In our case we’re going to use a t1.micro, which is the cheapest instance size available. No need to break the bank here. key_name defines which SSH key we’re going to use. This name is managed in the AWS Console, so substitute as you need to.

Now that all of this is understood, let us go ahead and launch our bare instance:

$ stem launch bare/bare.json
New instance ID: i-XXXXXX

That returned line, New instance ID: i-XXXXXX, tell us the instance id that we’ll work with. Now do this to get a list of running instances:

$ stem list
------------------------------------------
Instances
------------------------------------------
i-17e5d17b      50.16.66.78     running         default              ...

Great. We have a running system and an IP to connect to. Just do this to get going:

$ ssh ubuntu@50.16.66.78

You should get a shell. Now let us make a minor customization to our instance. You can obviously do something more elaborate, but this will serve as a demonstration:

$ echo FOO > BAR

Now we have a file named ‘FOO’ in the home directory of the ubuntu user. Good enough for me. Let’s build a custom AMI off of this.

First, halt the instance:

$ sudo halt

Run stem list a couple of times to ensure that the instance has entered the stopped state:

$ stem list
------------------------------------------
Instances
------------------------------------------
i-17e5d17b      50.16.66.78     stopped         default              ...

Capture it like so:

$ stem create base i-17e5d17b

After a few minutes (it could take longer depending on how AWS is behaving), you should be able to run stem list and see something like this:

$ stem list
------------------------------------------
Instances
------------------------------------------
i-17e5d17b      50.16.66.78     stopped         default              ...
------------------------------------------
AMIs
------------------------------------------
base  ami-487d8e21

Woo! You now have a base AMI ready with your customizations to be deployed as often as you like. You could do this via the AWS Console… or you could just create a new Stem definition to make this possible. We’re going to go the Stem route.

Create stems/base/base.json with the following info:

{ 
  "ami"                       : "ami-487d8e21",
  "instance_type"             : "t1.micro",
  "key_name"                  : "default"
}

As you can see, the only difference between this definition and the bare one is the AMI.

You can now launch it like so:

$ stem launch stems/base/base.json

If you give it a second to boot and then SSH into the box, you’ll see that our BAR file is in place.

Pretty cool, eh? Once you are done playing around, make sure you go back and run stem destroy against each running instance so you don’t rack up unexpected charges.

In future articles I’ll talk more about better ways to bootstrap these instances, configuration management, and using the stem api instead of the command line tool.

Hi Heroku!

2011-02-27T00:00:00-08:00

Last week I joined the ops team at Heroku. It’s an exciting opportunity and has already paved over my old thinking on operations. I have a lot to learn.

I keep telling myself that I need to port this over to a Heroku app or change the domain or something… but we’ll see. I have enough to think about for the time being.

Cheers!

Diving into RSpec

2011-01-05T17:30:00-08:00

For Christmas, my wife bought me a copy of The RSpec Book by The Pragmatic Programmers. It has changed my entire approach to development.

Over the past several years, I’ve grown apart from TDD. I began using Test::Unit with my early Rails projects, but always found that things became too brittle. My tests would often dive way too deep into the implementation of my classes and then break when I would refactor. After much frustration, I would eventually abandon automated testing altogether. I highly doubt the fault lies with the library — I didn’t really grasp the philosophy and therefore sucked when trying to apply it.

Everything changed with The RSpec Book. I now have a better grasp (maybe): focus on defining and testing behavior rather than the internals. After working through all the examples in the first five chapters this evening, I decided to apply RSpec to Imran’s FizzBuzz problem:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

If we decide that a resonable way to solve this is to create a class named FizzBuzz and a calculate method to do the work, we can start out with a bare spec:

describe FizzBuzz, "#calculate" do
  it "returns 'Fizz' for all multiples of 3" 
  it "returns 'Buzz' for all multiples of 5"
  it "returns 'FizzBuzz' for all multiples of 3 and 5" 
  it "returns the passed number if not a multiple of 3 or 5"
end

This is all very readable, and encapsulates the problem quite well. I can marvel at this for hours…

And when I’m done, I can type rspec <filename> and see something like this:

Pending:
  FizzBuzz#calculate returns 'Fizz' for all multiples of 3
    # Not Yet Implemented
    # ./fizzbuzz_spec.rb:16
  FizzBuzz#calculate returns 'Buzz' for all multiples of 5
    # Not Yet Implemented
    # ./fizzbuzz_spec.rb:17
  FizzBuzz#calculate returns 'FizzBuzz' for all multiples of 3 and 5
    # Not Yet Implemented
    # ./fizzbuzz_spec.rb:18
  FizzBuzz#calculate returns the passed number if not a multiple of 3 or 5
    # Not Yet Implemented
    # ./fizzbuzz_spec.rb:19

Finished in 0.00061 seconds
4 examples, 0 failures, 4 pending

Amazingly clear, right? I have four examples, and all of them are pending further definition. So I don’t bother you with all of the boring details, here is the complete implementation. Since this is so simple of a problem, I decided to combine the spec and class definition in the same file:

class FizzBuzz
  def calculate(n)
    if n % 3 == 0 and n % 5 == 0
      "FizzBuzz"
    elsif n % 3 == 0
      "Fizz"
    elsif n % 5 == 0
      "Buzz"
    else
      n
    end
  end
end

describe FizzBuzz, "#calculate" do
  let(:fb) { FizzBuzz.new }
  
  it "returns 'Fizz' for all multiples of 3" do
    fb.calculate(3).should == "Fizz"
  end
  it "returns 'Buzz' for all multiples of 5" do
    fb.calculate(5).should == "Buzz"
  end
  it "returns 'FizzBuzz' for all multiples of 3 and 5" do
    fb.calculate(15).should == "FizzBuzz"
  end
  it "returns the passed number if not a multiple of 3 or 5" do
    fb.calculate(77).should == 77
  end
end

When running this completed spec, we see that all is good:

....

Finished in 0.00162 seconds
4 examples, 0 failures

Awesome, right? Learn more about RSpec here.

Small Notes on New Year Resolutions

2011-01-05T00:00:00-08:00

What you need to know:

Everyone has them. Everyone blogs about them. Everyone breaks them.

—Me

How to fix it:

If you have plans for the New Year, keep quiet about them. For one reason or another, when we chat about our resolutions or plans, we tend to not accomplish them.

Rather than write a blog post about your intentions, carefully write them down on a piece of paper. Get specific and cut out the dead wood. If you have a list of ten items, reduce it to a list of five.

While you may be able to make progress on your entire list, pick one to really focus on. Consider this another step towards making sure that you actually want what you are asking for. Upon waking, review that item. Upon getting ready for bed, review that item. If that particular goal seems too big, break it up into chunks. It’s like selling software – you don’t launch and then hope for a million loyal customers. You launch and work towards getting your first customer. Then you get your next ten, fifty, and so on.

Good luck!

Introducing logtrend, or HTTP Request Trending With EventMachine

2010-12-03T00:00:00-08:00

I have this problem of not wanting to demonstrate code unless it is prettied up. It probably stems from inexperience and the fact that I work with people much smarter than myself. I told myself that its time to start showing off as soon as something seems useful, so here you go. In this era of social coding, the sooner something is released, the sooner the feedback comes.

The following code demonstrates a simple way to trend the volume of HTTP requests in your environment.

Our Problem

I assume you run a web app of reasonable volume, and stream your access logs to disk. If you’ve ever been fire fighting and scanning logs looking for long running or more-frequent-than-usual requests, you know it is hard and often futile work.

One Possible Solution

Here is logtrend, my contribution to putting these days behind us.

The idea is simple, give logtrend a log to follow, define categories for different types of events, and let it automatically build graphs as you define them. You can then check up on your graphs when you things start to behave funny and see if traffic patterns have changed. For example, perhaps you have a customer abusing your API or hammering on an entry point that is especially expensive.

This can probably be better demonstrated via the following code:

require 'rubygems'
require 'logtrend'

# Invoke this to begin trending your data...
LogTrend::Base.run("/var/log/httpd-access") do |lt|  
  # Set new locations for our graphs and rrds.  defaults to '.'
  lt.rrd_dir = '/tmp/rrd'
  lt.graphs_dir = '/tmp/graphs'
  
  # Add some things to trend.  An RRD is built for each one of these items.
  # Each time we read a line from the log file, we pass it to the block.
  # If your block returns true, we count that as a hit.
  # Every minute, the RRD is updated with the hits for the previous period.
  lt.add_trend(:total) {|line| line.match /.*/}
  lt.add_trend(:fbod) {|line| line.match /fogbugz.com/}
  lt.add_trend(:kod) {|line| line.match /kilnhg.com/}
  lt.add_trend(:long) do |line|
    # Let us pretend that request time is in seconds
    # and is the last item on the log line
    request_time = line.split.last.to_i
    request_time > 10
  end
  
  # Build a graph displaying some of the items we are trending
  # Label it as :requests_per_minute
  lt.add_graph(:requests_per_minute) do |g|
    g.add_point :area, :total, "#333333"
    g.add_point :line, :fbod, "#0066cc"
    g.add_point :line, :kod, "#993333"
  end

  # Build a second graph for our long running queries
  lt.add_graph(:long_requests) do |g|
    g.add_point :area, :long, '#000000'
  end
end

Once started, logtrend will watch the logfile in real time, outputting graphs every minute.

Here is a small sample of our first graph (numbers and other sensitive data have been edited out):

The black background represents the total requests for the given service, and the yellow line represents all API requests coming in. Notice the spike? Fortunately it did not cause any service issues, but had it, we would have had actionable intelligence.

Further Notes

Installation should be a matter of sudo gem install logtrend. Feel free to fork and patch as you see fit. There are still some rough edges, but I have been using it in production for over a week without any issue.

The system is currently retaining a year’s worth of data, but only rendering graphs for the past 24 hours. Expect this to be improved in an upcoming release.

logtrend is built with the intention of staying simple, so it probably won’t ever be a good fit for trending more complicated data than already mentioned.

Credit

As always, this was built on the shoulders of giants. Abstraction upon abstraction made a difficult problem very easy to solve. The most obvious contributions come from the flexibility of Ruby + the awesomeness of EventMachine.

Jordan Sissel’s eventmachine-tail made watching the files a cinch, and Thiago Morello’s rrd-ffi made graph generation and data retention pain free.

Bringing Your Rails App to the Google Apps Marketplace

2010-11-14T10:00:00-08:00

It’s good times. Between amazing frameworks like Ruby on Rails and cloud application platforms like Heroku, lots of heavy lifting has been abstracted away and the friction for bringing your product to market has been seriously reduced. The phrase ‘standing on the shoulders of giants’ sounds more and more applicable as each year passes.

To make things even sweeter, Google comes along and creates the Google Apps Marketplace. The short and easy: Google has millions of customers using Google Apps. You can write applications that integrate with various Google Apps services, and can take advantage of their single sign-on infrastructure. Your app can appear on the Google Apps nav bar, allowing them to switch to your product just as easily as they switch over to Google Docs or Calendar. It looks to be a market begging for lots of attention.

Okay. That’s out the way. Now how hard is it to write an app to integrate with Google Apps? Turns out to be a lot easier than I thought. I spent two hours yesterday morning and came up with a reasonable example. You can find the live version of the app here, and grab the full source via GitHub.

The problem to solve was simple: how do I share code snippets with coworkers? I want a private Pastebin or Gist service, but I don’t want to install anything on my own servers. Enter Styled Bits.

The app layout is very simple: I have three main models: Account, User, and Post. Account represents an entire Google Apps domain, User represents a member of said domain, and Post is a code snippet.

There are two major controllers, Main and Posts. Main handles user login and Posts displays the code snippets.

All of this is pretty standard stuff, the only thing that stands out is how the Main controller actually handles authentication. If you take a peak at the index method and related view, you will see that we display a form asking for your Google Apps domain. You can see it live by visiting the front page of Styled Bits.

After taking your domain name, we redirect you to the login method, where we do something like this:

def login
  url = "https://www.google.com/accounts/o8/site-xrds?hd=" + params[:domain]
  authenticate_with_open_id(
	url,
	{ :required   => ["http://axschema.org/contact/email", :email]}) 
	do |result, identity_url, registration|
    if result.successful?
      # Succesfully logged in
	    email = get_email(registration)
	    account = Account.find_or_create_by_domain(params[:domain])
	    user = account.users.find_or_create_by_email(email)
	    session[:user_id] = user.id
	    redirect_to(posts_path)
    else
	    # Failed to login
	    flash[:notice] = "Could not log you in."
	    render :index
    end
  end
end

As you can see, the simple method authenticate_with_open_id takes care of everything. Don’t you love blocks? Don’t you? That’s all it takes. To get this handy method, just add the following to your project’s Gemfile:

gem 'ruby-openid'
gem 'rack-openid'
gem 'ruby-openid-apps-discovery'

Add the following to the end of your environment.rb to take care of some weirdness with Google OpenID discovery:

require 'gapps_openid'

After that, all you need to do is to install the open_id_authentication plugin:

rails plugin install https://github.com/rails/open_id_authentication.git

With all of that out of the way, you are more or less on your way. Head over here and follow the instructions to become an official vendor and make your app installable. But even without this, Google Apps users will be able to login without issue.

I know that this article is far from perfect, but I hope it will at least inspire others to integrate their apps or create unique solutions for the millions of Google Apps users.

On Kiln and GitHub

2010-11-14T00:00:00-08:00

I work for Fog Creek Software and help maintain the infrastructure underlying FogBugz On Demand and Kiln.

Eyebrows sometimes raise when those in the know discover that I am publishing small amounts of code to GitHub. My imagination hears whispers of treason, and the outcome is this short article.

In my line of work one can either be irrationally loyal to a given tool, or one can choose the weapon that best suits the environment to be conquered. If I was going to be working with a group of developers on a serious project, I would go with Kiln. Mercurial is easy to use and works perfectly across all major platforms. Kiln emphasizes its best features and gives you a fantastic code review tool as well as integration into one of the best bug tracking platforms available.

However, for one who wishes to publish code for the sake of sharing with the world, GitHub is the obvious choice. While I think Git is an all around pain in the butt when compared to Mercurial, it is clear that the open source community has chosen its platform. Kiln is not built to do this sort of thing, and Bitbucket does not have the traction. When I’m looking for code, I more often than not end up on GitHub.

In a perfect world I could do all of my development with Kiln tools and then easily push my open source projects to GitHub as appropriate. Someday, maybe. For now, I use different tools as needed.

Returning to the Ruby Ecosystem

2010-11-12T00:00:00-08:00

For the past year, I have forced myself to work with Python and C#.

Last night I sat down at my computer and typed the following:

gem install rails

After a minute or so, Rails 3 loaded onto my system and I began to remember why I love Ruby so much. After four hours of prototyping, I was able to deploy my code to the real world like so:

git push heroku

I don’t want to go back. Sure, I’ll maintain my old code and use .NET where appropriate, but not on nights or weekends.

To get up to date, I picked up Meet Rails 3 from PeepCode, and grabbed the latest Rails beta book from The Pragmatic Bookshelf.

Serving WOT and SVG Files Via IIS

2010-11-10T17:30:00-08:00

A small one for today. As it turns out, IIS 6.0 only knows about EOT and TTF fonts. In order for your fancypants designers to take advantage of WOFF or SVG, you have to add the following MIME type declarations via IIS Manager (under the HTTP Headers tab):

.woff  application/x-woff
.svg   image/svg+xml

After adding those definitions, all should be well.

Solving libcurl's SSL Issues on OS X

2010-11-10T00:00:00-08:00

Hey! A new blog post.

After getting all excited about Jekyll in my earlier post, I came home to pull down my repository on my wife’s Mac. After gleefully typing the commands in, I received this:

Protocol https not supported or disabled in libcurl

Fortunately, the fix wasn’t too hard to figure out. I just needed to reinstall libcurl with SSL enabled. The following command fixed that:

sudo port install curl +ssl

No more problems!

Hello, Jekyll!

2010-11-09T00:00:00-08:00

After starting and stopping a dozen or so blogs over the past ten years, I finally decided to eliminate the fancy engine and go for a static layout.

I was impressed after hearing about Jekyll and reading Tom Preston-Werner’s post. The entire system is simple: write up your article in the editor of your choice, add it to a Git repository and publish. It’s sort of like the Heroku of static publishing.

I’m keeping the ‘.github.com’ domain name for the time being, and will move to one of my true domains if the writing keeps up.

If you are interested, you can see how easy it was to construct this site by viewing its repository.