Will Leinweber

Some Talks

2010-07-16T15:17:00-05:00

I gave a few talks and just realized I never put them up anywhere. The first talk is one I gave as an introduction to BDD using rspec to some students at uiuc. It seems like giving an "intro to bdd" talk is something everyone has to eventually do ;)

Intro to Behavior Driven Design from will l on Vimeo.

Second is a talk I gave at pecha kucha, which is a lot like ignite. Same auto advancing format, just a different amount of slides and time per slide.

me @ pecha kucha from will l on Vimeo.

Auto Merge Gemfile.lock

2010-07-16T00:17:00-05:00

Here's the problem: you're working on a topic branch and add a gem to your Gemfile, and someone else has added a gem to master before you're done. When you go to rebase, every single one of your commits is going to have a conflict with Gemfile.lock.

All you have to do is run bundle lock to get bundler to relock then add that and continue your rebase. But that's so damn tedious.

Good news! I just figured out how to tell git how to do that for you, and it works great. There are two files you have to edit.

gitconfig

First is your ~/.gitconfig file. Here we're going to give it a new merge strategy, one that will just relock the gemfile. Add this to the end:

[merge "gemfilelock"]
  name = relocks the gemfile.lock
  driver = bundle lock

The driver is what git will use to try and fix the conflict. Its exit status will tell git if the merge was successful or not. Here all we're doing is having bundler relock. You can see it in action in my dotfiles

gitattributes

Next up, we have to tell git to use our new strategy for Gemfile.lock, and we do that with gitattributes. You can either put this in project/.git/info/attributes or project/.gitattributes. I did the .git directory one myself, but it doesn't matter. We just need one line in this file:

Gemfile.lock merge=gemfilelock

results

And that's it! Here's what happen when you rebase now

First, rewinding head to replay your work on top of it...
Applying: commit one
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Your bundle is already locked, relocking.
Your bundle is now locked. Use `bundle show [gemname]` to list the gems in the environment.
Auto-merging Gemfile
Auto-merging Gemfile.lock

Now that, if I may say so myself, is really awesome! (image credit malstora)

Red Dirt Ruby Conference

2010-04-22T01:16:00-05:00

I'll be speaking about using CouchDB from ruby at the upcoming Red Dirt Ruby Conference on May 6th in Oklahoma City. The format is new, at least to me. There are four themes: Ruby, Rails 3, NoSQL, and Servers.

My talk is in the NOSQL theme and it will give you everything you need to get started using CouchDB from Ruby. The rest of the lineup looks amazing, and I'm really looking forward to it.

I'll be sure to post slides and such up here when I have them. Hope to see you in Oklahoma!

Update: And here are my slides:

CouchDB @ red dirt ruby conference

Rails Integration Testing Stack for Winners

2010-05-01T00:17:00-05:00

Full-stack, acceptance, integration, or whatever-you-want-to-call-it testing should be a large part of your test suite. I'm not going to waste time trying to convince anyone of this, since there are probably a million other blog posts doing that (and probably twice as many arguing the different names).

Rather, I'll show you the hands-down best way to do it with what's out there right now. Follow this setup and you too can be a winner. I've written a bit about this earlier, but things have gotten better since then.

Currently, while Rails 3 is close, it's not out yet, so I'm ignoring it for this post. Latest is 2.3.5. Here we go.

Cucumber

Cucumber drives everything. It's awesome, and you should have already heard of it. Everyone uses it. It's surpassed the vegetable on google for the top result.

It ships with webrat, but don't use webrat. It's neat and probably helped a lot to get cucumber popular. But it can't do javascript. And what site doesn't have JS these days. You need to be testing that too, you know.

Cucumber — cukes.info

Capybara

This is what you use instead of webrat. It lets you use a bunch of web drivers behind it. Out out of the box it supports selenium (ugh) and Celerity/Culerity (awesome). Before getting cucumber to work with Capybara was a little hacky, but they're best friends now. Use this.

You tag the stories that need javascript with @javascript, and it'll fire up culerity when it needs to. That's all you have to do. It's fantastic.

Capybara — github

Culerity

This provides a nice bridge between your MRI Ruby rails app, and the jruby-only gem Celerity. It'll fire off a separate server running in the test environment, then start up a jruby process for celerity.

Now this one can be a bit of a pain to set up, since you need to install jruby and then celerity and make sure that's all working and wired up. A nice alternative is Matt Fletcher's fork, which bundles all that into the gem itself so you don't have to worry about it.

There can be some problems with gem bundler, though. The gist of it is that you have to use JRuby v1.3.1 or earlier. I wrote more details about the bug on the mailing list. It looks like just yesterday Matt updated his fork to fix this, so hurray!

Culerity — github

Celerity

This is what's actually doing the work. It's a JRuby-only gem, since it uses the Java HttpUnit headless web browser. This is a complete web browser, with its own javascript engine, except it is just controlled though some API, and doesn't have a display.

Selenium was cool for it's time. Watir too, I guess, though I never used it. But relying on a real browser sucks. It's really slow. You have to worry about browser upgrades breaking the tenuous control. And it makes your CI server a hundred times more complicated than it has to be. Fuck that, when you can use Celerity which is awesome.

For some reason I can't quite remember, I had to stay on version 0.7.6. If you're having weird problems try that version. Of course, if you're using Matt's culerity fork like I suggested, you don't have to worry about it.

Celerity

picture credit: philo nordlund

CI Radiator in Javascript

2010-03-15T01:16:00-05:00

Continuous integration is a must. But if your team can't easily see the current build status, you loose a major benefit of CI: short feedback loops. The longer it takes to notice that the build is broken, the worse off you'll be.

I wrote really simple radiator that shows the current build state and the SHA of the associated commit just using HTML and JavaScript. We have it sitting up in the corner of the room, and it's been invaluable over the last few months. I even made a video showing it off, but never got around to really sharing it. Check it out:

We use Hudson for CI, and if you're using it too, you can just go ahead and use this html yourself with a few modifications. Enjoy!

Why You Should Use CouchDB

2008-11-16T12:00:00-06:00

New Approach to an Old Problem

Your applications need to store and retrieve data. That’s nothing new. CouchDB, and other document databases like Amazon SimpleDB, are new to the game however. Okay, I’ll admit the general concept is some 20 years old, but it’s safe to say that they’ve recently started to gain popularity over the last year or so.

Learning a document database is worthwhile, even if you don’t end up using it for anything serious in the near future. You’ll gain new insights on how to solve your current problems. Don’t take my word for it, there’s that line everyone quotes from The Pragmatic Programmer: Learn at least one new database paradigm a year, or something like that.

What exactly makes CouchDB so great? Read on.

Stable and Scalable

When you update a document in CouchDB, it doesn’t go in and change the existing document. Instead, it just adds a new version. The upshot is that your data is never in a bad state, so even in the case of a power failure, you’re not going to lose data or spend time verifying against corruption when you start up again. All these extra versions are deleted when you compact the database, so there isn’t much of a storage penalty because of it.

Other databases have low erlangs

It’s written in Erlang, and benefits from the stability and concurrency of the language its design idioms. Also, unlike current relational databases that try and make a single query as fast as possible, CouchDB tries to maintain consistent performance as the number of queries grows.

Another big plus is that it doesn’t have a custom binary protocol. It uses regular HTTP. This means you can use existing http infrastructure tools such as load balancers and clustering proxies. Better still, it’s RESTful, so you interact with it using the CRUD operations we’ve come to know and love.

The way you query data from CouchDB is through views which are written using the map/reduce approach. These views themselves are indexed, so they’re very fast as long as you keep the indices fresh. While it’s not an option now, it’s possible that in the future the mapping step could be distributed amongst several nodes which could make it ridiculously fast.

Deep Structures, No Schemas

Data has it’s own natural shape, and I know I’m not the only one who is sick and tired of fighting that fact. It is often an uphill battle cramming models into relational records. Your data is more complex than a few simple strings, numbers, and boolean values. You have lists, you have hashes, and even small variations between instances of the same object. This is unacceptable in the relational world.

ORMs like ActiveRecord and DataMapper have done a lot to to ease the pain and abstract away the nastiness of SQL. It’s not enough though. It’s like treating the symptoms and not the underlying condition. You still have to worry about joins, normalization, and other artifacts from relational databases. These issues leak their way up into your models where they don’t belong, and obscure more important logic.

All of this isn’t an issue with CouchDB, and that’s the biggest selling point for me. The fields in your documents can be hashes, they can be arrays, they can be arrays of hashes. Anything that can be serialized to JSON. It’s about documents — not records.

As a quick example, rather than having to create an assets table for uploads, you can store all the metadata in a single field. That field can live right on the object that is responsible for it, where it belongs. Or you could store all the comments for a blog post inline with the post itself. There’s a good article comparing inline vs. separate storage that is worth a read.

Downsides

Nothing is without drawbacks, and CouchDB is no exception. For one, it’s still alpha. That’s not to say it’s buggy and unusable — far from it — but there are likely to be changes before they hit 1.0. I can’t say to what extent this will be an issue, but you should be aware of that.

There is no security model. This turns out to not be that big of a deal, but it’s far from ideal. You can lock down the port it runs on to only talk to localhost, which I think happens by default.

The first time you run a view it will be slow. It has to go through every document and build the index. After that though, it only has to go through the new or changed documents and is significantly faster. You can make sure your views are fresh by running them after every couple hundred updates or every 10 minutes or so. But if you’re always running unique views, CouchDB probably isn’t a good choice.

Getting Started

Go ahead and compile CouchDB from the svn head rather than going for the last release. You can find instructions over on their wiki. You’ll also want a persistence layer. So far, I’ve been most impressed with Couch Potato. I’ve been using it with Merb, which has been great. You just need to add it to your config/dependencies.rb like this

dependency "langalex-couch_potato", "0.1", :require_as => "couch_potato"

then add this to your config/init.rb

Merb::BootLoader.before_app_loads do
  CouchPotato::Config.database_name = "appname_#{Merb.environment}"
  require 'json/pure'
end

You might not need that json/pure fix. I was having some problem where #to_json was declared multiple times. It’d be nice if there was a merb_couchpotato gem so you could just say use_orm :couchpotato, and maybe use database.yml instead, but it’s not really necessary yet.

That’s really all it takes. Go checkout the readme for Couch Potato and its introduction post.

Time to relax.

Why we're not on Ruby1.9

2009-09-11T01:17:57-05:00

It's been out for a while now, but as a community we just aren't using it. Why?

There have been technical excuses, namely broken gems. It's probably true that this is holding people back from upgrading, but gems can be fixed if we actually cared. So it's not a technical reason.

I've also seen marketing excuses. Odd numbers are generally for developmental releases and even for production. Yeah that's true in some places, but this can't be that big of a hurdle. The other argument is that it should have been 2.0. This holds a bit more weight as a whole number bump would probably make this seem like an important update. Which it is.

We've normally been quick to adopt new practices and tech. Remember when REST was introduced? About a year later, we were all building restful apps and were annoyed when we had to interact with old, non-rest apis. Or the near instantaneous mass migration to git. What was different in those cases?

Evangelists

photo: Esther_G

The guys with the well read blogs, the vocal consultancies, the regular conference speakers. In the past they said, "Hey I'm using this, it's great, you should too". And we did. That hasn't happened this time.

We need I've-switched-to-1.9 posts and talks. Hell, lie about it, if it'll get the community to upgrade. There'll be problems with gems, sure, but that period will be short. We'll make it through.

I'll end by doing my part: I've switched to ruby 1.9.1, it's great, and you should too.

Faking "or shift left"

2008-09-04T01:30:11-05:00

I've often been annoyed at the lack of ||<< after being spoiled with ||= for so long. The workaround I've typically used is to go ahead and use ||= [ ] to make sure I have an empty array. That's ugly though. You're dropping down a level and writing the "how" instead of the "why".

Today some code Jay Fields posted had the answer I've been looking for. It's not quite as elegant as having the operator itself, but it does the job of tucking away the ugly detail of ensuring you have an empty array to shift on.

1 def method_stack
2   @method_stack ||= []
3 end
4  
5 def method_missing(sym, *args)
6   method_stack << [sym, args]
7   self
8 end

We're still using ||= [ ] but it is hidden away in its own spot, which I find to be much cleaner.

Closures in Ruby

2008-09-15T01:30:11-05:00

I remember reading that Proc.new and lambda weren't exactly the same, but I never ran into any problems so I never bothered to look up exactly what the differences were. I just always used lambda, which as it turns out, is probably for the best. Paul Cantrell wrote a article/script hybrid that does a great job showing and telling all of the differences with the 7 (who knew?) different ways you can have a closure. It's worth a read, check it out. Here's the summary table that's about halfway through.


                                                    "return" returns
                                                    from closure or
                                   True closure?    declaring context?     Arity check?
                                   ---------------  -------------------    -------------------
1. block (called with yield)       N                declaring              no
2. block (&b => f(&b) => yield)    N                declaring              no
3. block (&b => b.call)            Y except return  declaring              warn on too few
4. Proc.new                        Y except return  declaring              warn on too few
5. proc                                <<< alias for lambda in 1.8, Proc.new in 1.9 >>>
6. lambda                          Y                closure                yes, except arity 1
7. method                          Y                closure                yes

Reopening NilClass

2008-04-03T01:30:11-05:00

I just read a post advocating changing NilClass#method_missing to always return nil. Their argument is that you no longer have to check if you actually have something before calling methods on it like:

@sun && @sun.still_burning?

Instead, you could just call @sun.still_burning?, which is nice. I usually just check is the sun still burning.com 20 times a day to make sure, but you get the point. Edit: it looks like this site is no longer up, and now I get worried at night, when I can’t tell

Now, it’s a very attractive, very short solution to an every day annoyance — that alone is enough to make me worry. Overriding #method_missing on NilClass just seems wrong_. The better solution is to define Null Objects for all of your classes, and have those returned instead of nothing. Yeah, it’s more work than the methodmissing solution, but it’s a lot safer and doesn’t leave such a bad taste in my mouth.

I do actually reopen NilClass though

Often, actually. Working with rspec and making typos can be very frustrating otherwise. If do @sum.should_recieve(:set) instead of @sun, then you’ll get

Mock 'some mock' received unexpected message :tyop?
# or
Mock 'NilClass' expected :list_pathways with ("hsa") once, but received it 0 times

The second one is easy to realize you have a typo, sure. But I know I’ve wasted time wondering why the expectation failed, and I’m sure you have too. By adding a few methods to NilClass, this is no longer an issue:

1     class NilClass
2       def should_receive(*args)
3         raise "WARNING: you tried to add expectations to nil!"
4       end
5       alias :stub! :should_receive
6     end

Now we get: WARNING: you tried to add expectations to nil! and even a line number to where the typo is.

When else have you found it okay to mess around with NilClass? I’m sure there are a few other occasions where it makes sense.

Sun picture from rodger smith, flickr