Panasonic Youth

Seven Months at Github

2012-11-27T00:00:00-08:00

I just passed my seven month anniversary at GitHub, and working here has been amazing. GitHub is a pretty special company, and I thought I would take a moment to reflect on what makes it special and some lessons I have learned over the past half year.

Great People above all else

The quality of people at GitHub enables every other unique and powerful thing about our company. We don't have deadlines, top down management, or much in the way of rules or process because GitHubbers are self motivated, smart, and passionate. While it is nice to imagine a world where all companies could be like GitHub and optimize for happiness and get rid of management, the reality is that you need awesome people whose happiness is derived from creating a great product for that to work.

Positive Peer Pressure

We thrive on very loose process and self management because of positive peer pressure. GitHubbers want to work on hard, important problems to get their peers to also work on them. We fix nasty bugs and work the support queue because making customers happy is awesome, and it also earns the respect of our peers, even if the work is thankless sometimes.

GitHub Hiring

GitHub hires great people. We put a lot into hiring and screening of candidates. That doesn't mean draining, three day interviews with comp sci heavy interviews. You won't have any GitHubbers glaring at you while you try to implement Quicksort on a whiteboard. There was a bit of coding in my interview, but it was actually just pairing on the GitHub codebase, working on a real problem that my interviewee happened to be working with that day. It was really just an exercise to see how I approached solving problems and worked with other people. By the time you get to the office for an in-person interview, it is assumed you have the basic chops to work at GitHub, and the rest is culture fit and personality fit.

Culture of Shipping

GitHub has a culture of shipping like nowhere I've seen. Even when it comes to things that many may not think of as easily "shippable", like architecture, process change or redesigns. For example:

creating a pull request with a spike demonstrating a small (but critical) piece of an architectural change
drafting a post in our internal ideas app, with references, diagrams, and examples, showing how to improve our teamwork
sketching a redesign idea on paper and uploading a scan to a pull request

So while it can be valuable to have conversations about things like architecture, process, or design, it is more valuable to take concrete action, even as fuel to move the conversation forward. Any experienced software development has seen "bikeshedding" kill productivity - endless philosophical debates and arguments driven by ego instead of practical value. Focusing on actions and shipping cuts through the crap and keeps things progressing forward.

Anarchist Structure

Much is made of the flat, nearly non-existant management style of GitHub. Yes, it really is true that you can work on whatever you want. Yes, that means that I could drop everything and work on a Haskell rewrite of github/github, but in reality that just doesn't happen.

It turns out when you hire self motivated, talented people who want to work with other really other good people, folks generally work on important stuff. Bugs get fixed and crummy things like grunt-work automation and difficult tests still get written. And if those things get missed on the first pass, peer pressure and continual review ensure ensure they get done.

End

It has been a fantasticly good half a year at GitHub. I have learned so much, shipped some great software, and look forward to many more years of learnings and shipping.

Day 3 Craftsmanship Swap at 8th Light

2010-07-11T00:00:00-07:00

(note: this was from my Craftsmanship swap Thursday, July 8th at 8th Light)

My last day of the craftsmanship swap at 8th Light was on Thursday. Doug and I paired more on the DRb/DRbFire fix that we had been discussing and working towards on Wednesday. The fixes can be found in Doug's fork of DRbFire.

The actual code changes resulting from our deep dive into DRb were pretty simple in the end. The hard part was the learning. We had to have enough understanding of everything in place, which made the code changes self-evident. I find with most domains, both technical and business related, this is so often the case -- the solution presents itself once you've learned what you need to know.

For lunch on the last day, I gave a draft version of my talk for DevNation. Got some great feedback and questions, and overall it went well. Post-lunch I was pairing with Doug until it was time for goodbyes and more travel.

Anyways, the 8th Light office has some very unique opportunities for reflection. For example, I took advantage of the in-office hammock for an afternoon break:

Stop - Hammock time

8th Light also has an in-office treadmill that you can use while coding. I tried it for a little while, and its a great break from the monotony and inactivity of sitting all day. Doug setup an adjustable desk over the treadmill, and also wired up the treadmill interface to a LimeLight based GUI. Treadmill based hacking is something I would love to have at the Relevance office.

Doug getting his treadmill hack on

Overall, the 8th Light guys are great, talented bunch. Their apprenticeship program is a great model to follow for how to train young developers, and I plan on following up on quite a few things I've learned from the swap. Thanks for 8th Light for having me as a guest, and to Relevance for helping make this happen.

Day 2 Craftsmanship Swap at 8th Light

2010-07-10T00:00:00-07:00

(note: this was from my stint on Wednesday, July 7th at 8th Light)

For Day 2 of my stint at 8th Light, Skim and I came in early enough to find a relatively empty office. I paired with Skim on an app he has been building as part of his apprenticeship at 8th Light. All apprenticeships have to complete several projects during there time, with their mentor (a craftsman at 8th Light) acting as the product owner.

Steve's application is a Tic-Tac-Toe game built in Ruby, which had the additional twist of having both a console UI and a Limelight UI. Limelight is a UI framework built in JRuby by the 8th Light guys, and I can definitely see myself reaching for it the next time I need to built a cross-platform rich UI.

I helped Steve out with some Rspec issues and also increased his git-fu, and we discussed some of the difficulties in testing threading. A Thread.new call in his game was causing timing related failures, and we tried to extract out the threading implementation enough to allow his specs to work around it.

Lunch was taken out of the office at Thai Gourmet, which was delicious. It was at least on par with Thai Cafe, which is a favorite at Relevance.

I had to fight off curry drowsiness post-lunch to pair with Doug on the same application we had paired on in day 1. The application uses DRb and DRbFire under the hood to manage client-server communication. Based on some issues reported from client installations we suspected some socket or connection related issues in the DRb related part of the codes. And so began a deep dive into DRbFire and DRb itself.

DRbFire is actually a protocol on to of DRb that does a neat little trick to allow DRb to connect across networks, such as when a client and server need to talk through firewalls or a NAT. Given the fact that 99% of clients will jump through a NAT or firewall at some point, what DRbfire does is pretty cool.

Doug and I dusted off a simulator that 8th Light has written to fake out multiple clients connecting to the server without having a ton of real machines around. By the end of the day we had discovered that something strange was definitely going on, as simulating 40+ clients connecting to the server and disconnecting resulted in over 80 socket connections hanging around in a CLOSE_WAIT state on the server. We were able to reproduce the same error that had been reported from client installations, but weren't 100% sure of the exact source yet.

Dinner that night was at a Hibachi-style steakhouse with Skim, Micah, and his family. The Japanese knife dude managed to impress without any injuries, and fine flame broiled food was had by all. I was back to the hotel by 8 or so and I turned in for an early night. Apparenlty the travel and pairing was finally catching up to me.

Day 1 Craftsmanship Swap at 8th Light

2010-07-08T00:00:00-07:00

Tuesday marked the first day of my craftsmanship swap at 8th Light. All in all it was a good day of pairing and learning with a fun trip to downtown Chicago on the end.

First order of business was grabbing a ride with Steve Kim for the short trip over to the 8th Light office, and checking out the area. 8th Light is located in a nice office park in Libertyville, IL, overlooking a marshy area, making for a great view from their office. Libertyville is a suburb about 45 minutes north of Chicago, with a quaint 'main street' type strip not far from the office.

I spent most of the day getting up to speed on a shared workstation/timeshare application that 8th Light has been working on for a while, which involves a lot of moving parts and quite a few languages. As I overhead someone explaining, it's a Java application written in Ruby, with C and Objective-C thrown in for good measure. All told its an impressive app that wrangles a pretty complicated problem domain well.

Doug Bradbury was my patient pair for most of the day and clearly knows both the tech and the domain very well. I helped Doug write some acceptance tests in Fitnesse, which went pretty well once I was able to wrap my head around Fitness. I haven't written Fitness acceptance tests before, but once I understand some of the terms used and basically mapped it to how I think of Cucumber features, things went well. The 'a-ha' moment came when I realized a Fitnesse 'fixture' roughly equals a Cucumber 'step'.

After that Steve, Eric Smith and I headed out to the Metra to grab a train downtown to attend the Chicago Ruby meetup. There were two talks, with the first one on HTML 5 WebSockets really grabbing my attention as having a lot of potential for "real" interactive apps in the browser being possible in a year or two. It will be a happy day for web devs we can drop ajax and polling as our solution for 'live' interaction with the browser and have something more first class and less hacky.

We closed the night with a few adult beverages at the Elephant and Castle, and after a train ride back I was back in the hotel before midnight. All in all a great introduction to 8th Light and the Chicago Ruby community.

New Releases: LogBuddy 0.5 and CapGun 0.2.4

2010-05-28T00:00:00-07:00

Hey now, it has been quiet here for too long, so what better way to bring some life back then open source updates?

First, LogBuddy, the little developer tool that could, has been updated to 0.5. What is LogBuddy? LogBuddy is a very small tool that logs the name of the thing your are logging, plus its value. Stop typing puts my_var => #{my_var} while doing ad-hoc debugging. For example, assuming you have a local, an ivar, and a class var called 'a':

d { a }       => a = "foo"
d { @a }      => @a = "my var"
d { @@bar }   => @@bar = "class var!"

There are a few significant changes in this release.

You no longer have to call LogBuddy.init in order to have the d method available everywhere - now it just happens upon require unless you specifically set SAFE_LOG_BUDDY in your environment. This makes LogBuddy easier to require and use for small tasks.
No more logger method mixed in everywhere - that was a bad idea and just asking for namespace bugs.
LogBuddy will no longer raise if the logged line of code is removed before the output is generated. This probably only happened if you were coding quickly and using a tool like autotest or watchr to run tests on change.

CapGun adds deployment notifications to Capistrano, so you can email blast interested parties on deployments. Version 0.2.4 fixes a bug where notifications would fail on the very first deploy due to a missing REVISION directory. This version also got fixes for Rails 3 using the new ActiveSupport, thanks to Jared Pace for those.

Raleigh.rb Dependencies Talk

2010-03-16T00:00:00-07:00

Last night I gave a talk on Rubygems, Bundler, and dependency management at Raleigh.rb. Below is a list of references from the talk, as well as some extra guides for getting rolling with Bundler.

Thanks to everyone who came out!

Postgresql and Sphinx and fe_sendauth no password supplied

2009-09-29T00:00:00-07:00

Fun fact learned today about interactions between Postgresql and Sphinx. Assuming you are using Postgres as your app's database, and you are wiring Postgres as the Sphinx data source, you must set a database level password for your database user. You also cannot use the 'trust' mode in Postgres and try indexing without a password. This can be especially confusing when you see that your Rails based db operations all work fine with no password, but then Sphinx indexing (via Thinking Sphinx, in this case) fail with this error:

fe_sendauth no password supplied

If you decide to setup your postgres database user to have a null password (the default I believe if you don't specify one on creation), or you turn postgres auth mode to "trust", you can never index data with this user. Sphinx requires postgres users have actual passwords and use them for auth, even if postgres itself does not care. Note that I'm referring to the database user password, not the OS level password.

Moral of the story: if you plan on using Sphinx and Postgres, make sure you have passwords wired for your Postgres user, even in dev/test environments.

Announcing the RunCodeRun Gem

2009-09-28T00:00:00-07:00

Hook up your RunCodeRun builds from your command line.

Install:

gem install runcoderun

Usage:

cd [your-runcoderun-enabled-project] rcr [command] [options]

Right now we just support open. This opens up your project homepage on RunCodeRun in your default browser. It is smart enough to make a best guess at the correct owner and project name from your git config. This requires Launchy, like the Github gem.

Try it out:

$ git clone git://github.com/floehopper/mocha.git $ cd mocha



opens http://runcoderun.com/floehopper/mocha in your default browser:

$ rcr open $ runcoderun open # if you like to type more

Patches or ideas welcome! This is very much in the spirit of "release early". A command to tie in with the API is clearly the next easy win for this.

Rails 2.3, JSON, and lamentations

2009-06-23T00:00:00-07:00

There is a nasty gotcha in Rails 2.3 involving rendering JSON from models and a surprising change in behavior. If you are familiar with how Rails renders xml for models, you may expect json to be very similar. You would probably expect a top level root node followed by the attributes:

{"person" => { "city=>nil, "name"=> "Foo Bar" } }

It appears somewhere on the road to 2.3, the JSON encoding changed so there is no longer a top level node:

{"city"=>nil, "name"=>"Foo Bar" }

The end result of this change means that if you have consumers expecting JSON in a typical "Rails" format, with a top level element denoting the class, there will be pain. We just hit this in a project when trying to use ActiveResource to consume some models exposed via standard restful actions.

The following demonstrates how this breaks down exactly:

# in the backend web app Person.create! :name => "foo"



meanwhile, in an active resource consumer

person = Person.find(1)  # finds a model okay, but...

the below all raise NoMethodError

they should find the attribute from the attribute hash

and quack like an ActiveRecord model

person.name
person.age
person.any_field_on_the_model

person.attributes # returns a hash in the format of { "person" => { "name" => "foo"...}, which is where the problem lies

It appears this issue is fixed or on the way to be fixed in the 2.3 stable branch, but for now be wary of JSON support in general on 2.3 until more of the below tickets get resolved.

References:

<li><a href="https://rails.lighthouseapp.com/projects/8994/tickets/2584-232-activeresource-json-doesnt-send-parameters-with-root-node">the bug in question - still open</a></li>

xml vs json - fight!

<li><a href="https://rails.lighthouseapp.com/projects/8994/tickets/2753-to_json-behavior-still-different-between-rails-2321-and-rails-2-3-stable">2.3 related (?) fix that caused other issues</a></li>

CapGun 0.2.0 - now with all Git commits since the last release

2009-06-21T00:00:00-07:00

Relevance has released CapGun 0.2.0 to GitHub and Rubyforge.

We've added more Git knowledge to this release, courtesy some great work from Muness. Your deployment notifications will now contain the branch they were deployed from and a list of all revisions included in the deployment (since the last deploy, of course).

There was also a overhaul to the internals. Muness refactored all the email logic to a presenter, so it should easy to change the notification email to your liking. CapGun now uses Jeweler and Micronaut, to match the rest of the standards for Relevance open source.

Keep us honest by checking the build status on RunCodeRun, and file feature requests and bug reports on Github Issues.

Pair programming? TDD? Where is the proof?

2009-02-10T00:00:00-08:00

Often while discussing agile practices with agile skeptics, the question of proof comes up. “Are there studies done showing TDD? How do you know pairing works and doesn’t just waste time?”

Those are tough questions. Software is not a science, and there are no double-blind, randomized controlled projects. The study of software development barely approaches the level of a social science. Its easy to discuss the flow and intuitive sense that agile works with other people who have experienced it, but try expressing that to someone who has never stepped beyond the waterfall model.

Things are getting better in the area of real, hard studies done on agile. One example is the work being done by Microsoft’s Empirical Software Engineering Group (ESM). The summary from the TDD study is worth citing:

“The results of the case studies indicate that the pre-release defect density of the four products decreased between 40% and 90% relative to similar projects that did not use the TDD practice. Subjectively, the teams experienced a 15–35% increase in initial development time after adopting TDD.”

I’ve included full links to some of the standout papers on my @to-read list below.

Realizing quality improvement through test driven development: results and experiences of four industrial teams
Do Cross Cutting Concerns Cause Defects? (my alternate title: “why the hell should I care about AOP?”)
Pair programming: what’s in it for me?
The Influence of Organizational Structure On Software Quality: An Empirical Case Study

Upgrading git via MacPorts

2009-01-14T00:00:00-08:00

Git 1.6.1 was released at Christmas, and if you are still on 1.6.0 or lower its well worth upgrading. MacPorts is the way to go for Git on the Mac - the ports team has been awesome in keeping up with the latest versions. Here's how I upgrade:

First, make sure you have the latest portfiles:

sudo port sync

Deactivate any preexisting versions - not sure if this is strictly necessary, but I've seen failures when I don't.

sudo port deactivate git-core

Install the new hotness, with all optional add-ons.

sudo port install git-core +svn+bash_completion+doc

Verify it:

git --version

=> git version 1.6.1

Note that there is a MacPorts 'upgrade' command, but I've seen it fail to actually upgrade things from time to time. Doing the install this way always works for me.

Best Albums of 2008

2009-01-09T00:00:00-08:00

So, here are my top ten albums of 2008, just a bit late. All links go to the album on the Amazon MP3 store (if available) and they are affiliate links.

Fleet Foxes - Fleet Foxes

This album was on everyone’s top albums list of the year for good reason. Whether you like folk, pop, or country, this album has no filler and is great through and through. Pick up their Sun Giant EP as well to round it out.

Bon Iver - For Emma, Forever Ago

I’m a Wisconsin native, so I may be a little biased in picking an album created in the Wisconsin north woods. Still, a superb album for cold winter nights and hot cocoa or coffee at your side.

The New Year - The New Year

Before the New Year, there was a Texan band named Bedhead in the 90s. Bedhead was quiet, contemplative, and perfect for those lonely hours between midnight and dawn. Since then, Bubba and Matt Kadane (real names, and real brothers) formed the New Year, got a little more accessible and upbeat, and released three albums. This third self-titled album is where slow-core and pop come together seamlessly. The Company I Can Get is the closest thing the Kadanes have done to a radio friendly single, while still jaw-dropping beautifully sad songs like The Idea of You.

Underoath - Lost in the Sound of Separation

Metalcore that doesn’t suck and tries new things. Not easy to find these days.

Sun Kil Moon - April

Mark Kozalek released an insane amount of music in 2008, both under his own name and Sun Kil Moon. April is the best of the bunch and will grow on you.

Dodos - Visiter

Catchy, quirky indie pop, producing a sound far bigger then any two humans should be capable of with only drums, guitar, and voice.

Cult of Luna - Eternal Kingdom

Epic post-metal. Recorded on the site of a demolished mental institution, with lyrics from the diary of a madman who was once a resident there. If you are a fan of metal, what else do you need to know?

Adam Beyer - Fuse Presents

Minimal techno is notoriously difficult to translate out of the club and into the living room. Beyer is a top techno producer and DJ, and this mix really captures the banging minimal techno that became so popular in 07/08. Avoid if your responses to techno include “wait, where are the lyrics?” or “is the CD skipping??”. If you can get into the Chemical Brothers, Underworld or even Paul Oakenfold, put this one on and give it some time. You may just find yourself nodding along and digging techno.

Sigur Ros - Med Sud I Eyrum Vid Spilum Endalaust

Sigur Ros finally gets a little less serious and overbearing and make a wonderfully fun album with some actual pop songs alongside the epic twelve minute compositions.

Daïtro / Sed Non Satiata - Split LP

(This came out in Europe in 2007 and made it over here in 2008.) Two French post-hardcore/screamo bands in the vein of Page 99 or Envy (read: much heavier and harder then MTV emo) join for an album that is everything good about hardcore right now. Takes elements of post-rock and math-rock, yet still retain the passion and hooks that made screamo so captivating to begin with. You don’t need to understand French to rock out to this split.

Honorable Mention

MacPorts Ruby performance gotcha

2008-11-18T00:00:00-08:00

If you install Ruby from MacPorts (all the cool kids do), be aware that the first 1.8.7 port had some issues that basically broke performance, making it run three times slower than normal. Verify that you have the 1.8.7-p72_2 version active, and not 1.8.7-p72_1:

~ $ sudo port installed |grep ruby
ruby @1.8.7-p72_1+thread_hooks
ruby @1.8.7-p72_2+thread_hooks (active)

The defect on MacPorts Trac has more details, along with links to a relevant ruby-talk thread.

Its worth pointing out that this issue went from discovery to being fixed and released into the port stream in about a days time. MacPorts has really come a long way from just a year or two ago. Fairly major packages used to lag behind the latest stable releases by multiple versions, but now I find everything I care about stays up to date with ease from MacPorts. They are even able to keep up with git releases, an impressive task considering how fast and furious releases git releases come.

Kudos to the MacPorts team for doing a fine job and making developers’ lives easier.

Tarantula 0.0.5 Released - the "Naked Aardvark" release

2008-09-26T00:00:00-07:00

Announcing version 0.0.5 of Tarantula.

Tarantula is a big fuzzy spider. It crawls your Rails application, fuzzing data to see what breaks. It can verify HTML validation across all your pages, ensure you don't have 404s, and pretty much anything else you want via custom handlers.

Don't let the version number fool you, we've been using Tarantula across many projects at Relevance and its very stable. This release fixed a number of annoying bugs, including namespace conflicts with other classes due to Rails dependency loading, improved gem spec with correct dependencies, and clean up on the html reporter.

Install it via the Github

gem install relevance-tarantula --source http://gems.github.com

or via Rails 2.1+ gem handing:

config.gem "relevance-tarantula", :source => "http://gems.github.com"

Scp or rsync failing with no error message? Check your startup scripts...

2008-09-15T00:00:00-07:00

The other day I was having issues trying to scp/rsync data, with no real error message to try and debug things. Turns out that any output produced by your startup scripts will break rsync/scp hard. I had some simple 'echo' statements print when different scripts were being loaded...turns out scp/rysync don't like that.

My capistrano task was a very simple call out to the 'get' helper, which just uses scp under the hood. The task ran and looked as if it completed, only nothing was ever transferred and the scp progress bar never came up. Sometimes it would block and do nothing, which was real fun, too.

The solution was simple - change all the bash scripts we use to not output any echo anything when running. I deployed the new scripts to all servers I needed to scp with, and the issue was resolved.

Since this is a known issue in the faq, it won't be fixed or improved with a better error message. It's just something you need to be aware of and work around, either via detecting if the session has an interactive terminal before sending output or removing your output statements altogether from you startup scripts.

Git Clone vs cp -R --> WTF?

2008-07-30T00:00:00-07:00

I knew git was fast, and I even knew it was faster than a lot of plain linux local file operations. Still, this still blew me away:

[code]rsanheim@ares:~/src/personal/oss $ du -hd 0 insoshi/ 26M insoshi/

rsanheim@ares:~/src/personal/oss $ time git clone insoshi/ /tmp/insoshi Initialize /tmp/insoshi/.git Initialized empty Git repository in /private/tmp/insoshi/.git/ Checking out files: 100% (2193/2193), done.

real 0m3.826s user 0m0.251s sys 0m0.658s

rsanheim@ares:~/src/personal/oss $ time cp -R insoshi/ /tmp/insoshi_cp

real 0m9.065s user 0m0.114s sys 0m1.442s [/code] Ok, so a 26 meg repo takes almost three times as long to copy via a recursive cp than a local git clone. Thats a fairly small repo, lets try something bigger:

[code] rsanheim@ares:~/src/relevance $ du -hd 0 rails 75M rails

rsanheim@ares:~/src/relevance $ time git clone rails /tmp/rails2 Initialize /tmp/rails2/.git Initialized empty Git repository in /private/tmp/rails2/.git/

real 0m2.321s user 0m0.151s sys 0m0.465s

rsanheim@ares:~/src/relevance $ time cp -R rails/ /tmp/rails

real 0m7.133s user 0m0.067s sys 0m1.505s [/code]

The rails repo at 75 megs is still ~ 3 times faster.

Obviously, this is not scientific at all, but the point is pretty clear. Git is doing some magic that lets it move files around locally 2 to 3 times faster than a plain copy. From looking at the man page, I would guess it has something to do with git using hardlinks for things in .git/objects when cloning locally. My linux fu falls down a bit here -- what are the ramifications of using hard links versus doing a "real" copy?

(This also makes me want to try out gitbak even more...)

Quick: Find the Bug or Gotcha with named_scope

2008-07-23T00:00:00-07:00

Think fast! Where's the bug?

[ruby] named_scope :active, :conditions => ["activated_at < = ?", DateTime.now.utc.to_s(:db)] [/ruby]

Looks fine, right? Maybe you've hit this already, and you see it immediately.

The symptoms are that the DateTime.now always seems to be a bit off - maybe you just restarted your server and its a only a few minutes off.

The bug is that DateTime.now gets evaluated at the time the class is loaded, not when the finder is run. What makes this easy to miss is that it will always work fine in tests and development, as everything is constantly getting reloaded there.

The fix, obvious once you've spent a combined time of over an hour trying to figure out what is going on:

[ruby] named_scope :active, lambda { { :conditions => ["activated_at <= ?", DateTime.now.utc.to_s(:db)] } } [/ruby]

Notes on testing Bj (Background Job)

2008-07-10T00:00:00-07:00

Some thoughts and random notes on testing Bj within a Rails integration test (or spec).

You have to turn transactions off for the scope of the test, or suffer very confusing issues, since Bj itself wraps the job submittal within a transaction. The way I did this was just overriding the use_transactional_fixtures method in the one specific spec. [ruby] describe Foo def self.use_transactional_fixtures false end [/ruby]
Remember, bj = background job. This may seem obvious, but whatever you submit to bj will be running in an entirely different process, so in our spec you need to wait for that job to complete before trying to assert things. You can do something as simple as this: [ruby] MAX_TIME = 10.0 seconds = 0.0 while(job.pending?) do job.reload seconds += 0.5 sleep 0.5 raise if seconds > MAX_TIME end # normal assertions here [/ruby] This gives your job up to 10 seconds to finish, and will timeout if it takes too long, which usually means something has gone wrong.
You now have to watch multiple logs to figure out what is going on. So tail your test.log and tail the bj log as well, and run the script in isolation to make sure you understand where exceptions and syntax errors will go. I wasted some time scanning logs when I really need to check the job.stderr field that bj populates, so be sure to output that for common test failures.

Overall, I've been pleased with bj, besides some open questions I've still been working out by perusing the source. Check it out if you need a easy to use persistent job queue.

CapGun and LogBuddy updated to 0.0.5

2008-07-06T00:00:00-07:00

Some long overdue releases of cap_gun and log_buddy - both have been updated to version 0.0.5. Both are now available as gems on github.com/relevance as well as from rubyforge.

CapGun gives you super simple deployment notifications from Capistrano. LogBuddy gives you a log helper through all objects, and can also log the name of the thing passed in along with its value -- saving you on typing and making debugging quicker.

CapGun got a fix so it does not attempt to display the rails_env if its not defined - this should clean up any strangeness in notifications if you saw something like "my_app was deployed to ".

LogBuddy got some minor tweaks and improved specs.

Both libraries now use Echoe, since Hoe complains about readme.txt when I want to use readme.rdoc, dammit. Both now only have a dev dependency on echoe to play nice with RubyGems 1.2.

You can install them via github or rubyforge:

sudo gem install log_buddy sudo gem install cap_gun

or

gem sources -a http://gems.github.com sudo gem install relevance-log_buddy sudo gem install relevance-cap_gun

Please log bugs or issues at our Trac.

Git 1.5.6 released

2008-06-18T00:00:00-07:00

Git 1.5.6 has been released, and there are a lot of usability fixes and tweaks which should make the upgrade worth your while. Looking at the detailed list of changes since 1.5.5, it looks like submodules have been getting quite a bit of love from many contributors, so it might be time to get them another shot. Scroll down or search in the announcement for the the part starting with "Changes since v1.5.5" and look through there for some of the submodule improvements that are coming.

The directions posted here worked fine for me to upgrade my existing source based installation in /usr/local.

Git lessons learned

2008-06-06T00:00:00-07:00

Lessons learned from day to day use with various ruby and rails projects.

Submodules completely suck when things get complex - I'm moving away from no submodules, and using direct exports for now until I have time to research braid or piston 2.0. For more details on this, see this or this post on the github group.
Use capistrano 2.2, not 2.3! 2.3 breaks git support
Always use :remote_cache for deployments -- super fast with git
If you have weird errors, it probably means you need to pull - when in doubt pull to make sure you have the latest
Branch more locally - I've been burned a few times when I've started work in master and then regretted it later when I wished my work wasn't in mainline (yes, its possible to fix this after the fact, but that gets into more advanced git usage)

Refactotum Rails Conf 2008

2008-05-29T00:00:00-07:00

I'm in Portland for Rails Conf with over 80% of the Relevance crew. We were testing out our "plane number" yesterday, but thank goodness American didn't let us down.

We'll be speaking today at about how to contribute to open source at Refactotum from 1:30 to 5. We will cover some tools to help you find the code with the most technical debt, go over example refactorings, and then spend the rest of the session going from project to project and helping out as folks hit obstacles. Please bring a laptop with any projects checked out that you'd like to hack on during the session (git preferred but not necessary).

Hope to see you there!

Testing Velocity Part 2 - Why do we test?

2008-04-18T00:00:00-07:00

A couple weeks ago, I began a series on keeping your test suite fast and effective. I now am going to digress a bit, take a step back and view the big picture to establish context.

Before addressing test performance and what makes up a good test, we should ask ourselves why is it that we write tests at all? If we want to be effective, we should always stay conscious of the overall goal of testing, as well as the specific goals behind each test in context.

Some would argue that tests are primarily a design tool. Or that tests are a living, breathing, specification for our code. Others would say it's primarily a means to drive and maintain quality. Some may say that tests are useful to ensure that the really difficult parts of our system work, or to keep lax developers in line.

Testing is only valuable and useful insofar as it supports software as a cooperative game. When you think "cooperative game", imagine rock-climbing, or ? To quote Alistair Cockburn:

Software development is a (resource-limited) cooperative game of invention and communication. The primary goal of the game is to deliver useful, working software. The secondary goal, the residue of the game, is to set up for the next game. The next game may be to alter or replace the system or to create a neighboring system.

So is testing for design, or quality, or correctness, or communication? The answer is that it's for all of those things (and more) as long as it helps deliver working software and prepare for the next game! So when someone asks, "Why do you write tests? Why do you care about the speed of your tests?" The first answer is, "It depends." It depends on the software you are delivering, on the teammates and domain you are working with, and on what the next game (if any) is. Since every software project will be different, clearly how you write the code and its tests will differ. An embedded system targeting your phone will have a much different test suite than a large enterprise web app.

This digression is to clarify debates I've heard (and been involved in) over issues like what should a "proper" unit test should do, how much setup is okay, mocks or stubs or fake objects, and when it's okay to mock/stub versus when it's not okay. There are not hard and fast answers to any of these questions. You need to consider context: What is being built? What are the current technical issues? Does the client want to run (or create) acceptance tests? How slow is the test suite currently, and is it impacting dev speed? How large is the system, and how much larger do you expect it to be? Who will be maintaining the system after it's released, and what will their skill level be?

Answer those types of questions before making statements like "this spec is doing too much setup and runs too slow," or "we shouldn't be stubbing in a functional test like this." It will ensure that your discussions and debates will stay grounded and useful instead of becoming endless religious debates. Keep context and the cooperative game model of software in the back of your mind, and I will too as I continue this series and try to lay out some practical, overall guidelines.

CapGun Released! Super simple Capistrano deployment notifications

2008-04-17T00:00:00-07:00

Tell everyone about your releases! Send email notification after Capistrano deployments! Rule the world!

Drop your ActionMailer configuration information into your deploy.rb file, configure recipients for the deployment notifications, and setup the callback task.

Setup and configuration are done entirely inside your deploy.rb file to keep it super simple. Your emails are sent locally from the box performing the deployment, but CapGun queries the server to grab the necessary release info.

This even includes the Net::SMTP TLS hack inside as a vendored dependancy to allow super easy email sending without setting up an MTA.

This is the "Rock n Roll McDonalds" release - 0.0.1.

INSTALL

sudo gem install cap_gun and gem unpack into your vendor/plugins
or just grab the tarball from github (see below)
see the readme and rdocs for more config notes

URLS

Log bugs, issues, and suggestions on Trac: http://opensource.thinkrelevance.com/wiki/cap_gun
View source: http://github.com/relevance/cap_gun/tree/master
Git clone source: git://github.com/relevance/cap_gun.git
RDocs: http://thinkrelevance.rubyforge.org/cap_gun/

history meme onwards...

2008-04-16T00:00:00-07:00

rsanheim@ares:~$ history 1000 | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head 71 gst 67 l 38 **** 38 c 28 git 22 ss 21 gca 13 gsd 13 : 12 rake

I like keeping it short. The 3rd one thats bleeped out is a shortcut to my main client project right now. The aliases can be found in our open source'd etc stash on github.

Tagging Muness, Jason, Aaron, and Chris.

If you think Huddle is ripping off Campfire

2008-04-08T00:00:00-07:00

...then you haven't seen mIRC, an irc client app that has been around for years:

See anything familiar in the screen shot above?

Tabs on the top, list of participants in the right hand pane, and chat in the main window. Change the font to Lucida Grande 12 pt and put in a browser and you pretty much have the Campfire l&f, minus a whole lot of polish and design love. Chat applications tend to follow a pretty standard UI model, and the fact that Huddle took the 37 Signals style is a good thing for Jason Fried and co.

Here's why: The look and feel from Basecamp and Campfire have become the defacto "template" for web 2.0 apps without a design of their own. There is a good reason for that: 37 Signals develops kickass UI's. As long as they stay on top of their game and keep their customers happy, they have nothing to worry about from people ripping off their design. The customer who leaves your app for a free one because the CSS is the same is a customer you don't really want anyways.

Here's the standard process whenever this becomes public and everyone gets up in arms about it:

37 Signals gets a ton of free press and attention.
Paid users for the app in question (campfire in this instance) shrug and say "huh, I'm pretty happy with my service, and I'm not going to switch.
People new to both apps who are in the market try both and choose on their merits.
The user experience for the 37 Signals' apps remains superior in most cases, as they are continuously tweaking and improving while copycats are stagnant or trying to keep up.

What exactly is the problem here?

The more interesting point about Huddle, and Google App Engine in general, is the lack of diversity the platform provides for. But please, drop the "Google ripped off 37 Signals oh noess!!!!" angle.

Zero to a Fully Git Enabled Rails App in the time it takes to drink an espresso

2008-04-05T00:00:00-07:00

updated: now uses the real Rails git master at github, now that its live.

So you want to set up a fresh Rails app in a fresh git repo, with proper ignores setup, with vendor/rails using a git submodule (which enables switching to any Rails branch or tag locally)? This isn't rocket science or anything, but I figured I'd post it to see what things could be done better.

Assumptions

You have a working, recent version of git installed.
You have the git-rails gem installed.
You have the bash aliases at the bottom of the post.

rails app_name cd app_name



git-rails init

gca -m 'initial commit'

git submodule add git://github.com/rails/rails.git vendor/rails

bring in rails from github

git submodule init

Submodule 'vendor/rails' (git://github.com/rails/rails.git) registered for path 'vendor/rails'

git submodule update

gca -m 'bring in rails via submodule'

Created commit 3b67dee: bring in rails via submodule

2 files changed, 4 insertions(+), 0 deletions(-)

create mode 100644 .gitmodules

create mode 160000 vendor/rails

cd vendor/rails gba # pick branch of Rails you wanna use git co BRANCH # where BRANCH is the specific branch you want, unless you want the default of going against edge

You're done!

Here's those aliases you need:

alias g='git' alias gb='git branch' alias gba='git branch -a' alias gc='git commit -v' alias gca='git commit -v -a' alias gd='git diff | mate' alias gl='git pull' alias gp='git push'

log_buddy 0.0.2 released

2008-04-04T00:00:00-07:00

Description

log_buddy is your friendly little log buddy at your side, helping you dev, debug, and test.

Synopsis

Call LogBuddy.init to use log_buddy. It will add two methods to object instance and class level: "d" and "logger". You can use your own logger with Logbuddy by passing it into init's options hash:

LogBuddy.init :default_logger => Logger.new('my_log.log')

Now you have your logger available from any object, at the instance level and class level:

obj = Object.new
obj.logger.debug("hi")
class MyClass; end
MyClass.logger.info("heya")

You also have a method called "d" (for "debug") on any object, which is used for quick debugging and logging of things while you are developing. Its especially useful while using autotest. When you call the "d" method with an inline block, it will log the name of the things in the block and the result. Examples:

Changes

0.0.2 * rdocs * support for multiple statements in one "d" call separated by semicolons

Installation

sudo gem install log_buddy full docs here

Rails is moving to Git (and Lighthouse)

2008-04-02T00:00:00-07:00

The Rails core team is making a big move to Git (and Github) for source control and Lighthouse for issue tracking. I love the move to github - it will make managing patches and your small tweaks to Rails much easier to keep in sync with your chosen branch of Rails. The move to Lighthouse I'm not sold on, only because I haven't felt the big benefits it offers over Trac, though I haven't really used Lighthouse much over the past few months.

We've been moving all internal things at Relevance to Git and Github, and so far we've been very happy with the move. Local commits are great, git is blazing fast, and github is the bees knees. If you haven't learned Git yet, now is the time.

How to move your domains off GoDaddy

2008-03-29T00:00:00-07:00

Its common internet knowledge that GoDaddy is one of the worst, most unfriendly registrars out there. Go to NoDaddy for all sorts of nasty things that GoDaddy that make them completely disreputable, or google "godaddy sucks" for a thousand other reasons.

What's less common is knowing how to actually get your domains off GoDaddy once you've had enough and have smartened up about finding a good registrar. Its simple once you know the process and can ignore the five thousand gratuitous options that godaddy has on their control panel.

First, you must make sure your domain is unlocked.

go to your domain control panel
click the checkbox(s) next to the domains you want to unlock
click the "Locking" button
choose "unlock" and click ok

dotster

Click on the domain name itself that you want to transfer
You are now on the "show details" page from hell -- the link you want is Authorization Code: Send by Email - click on that "send by email" link, then click OK in the confirmation page that comes up.
Check your email in a few minutes, and you'll have the auth code (aka EPP Code) you need! Use that code at the new registar you want to transfer to, and you'll be off godaddy soon enough

There is a helpful page here showing this process with screenshots, but it loads very slow. Here's a cached copy that may work better.

I'm happy to say the last of my domains will be off godaddy by tomorrow, and I'll be 100% godaddy free.

log_buddy Released - your helpful little dev buddy

2008-03-28T00:00:00-07:00

LogBuddy is your friendly little log buddy at your side, helping you dev, debug, and test. It plays well with Rails and plain old Ruby projects. To use it, sudo gem install log_buddy, then require 'log_buddy' and call LogBuddy.init. It will add two methods to object instance and class level: "d" and "logger". You probably only want to use it in non-production environments.

The "logger" method is just a typical logger - it will use the Rails logger if its available.
The "d" method is a special helper that will output the code in the block and its result - note that you *must* use the bracket block form - do...end is not supported.

Examples

[ruby] require 'lib/log_buddy' LogBuddy.init

a = "foo" @a = "my var" @@bar = "class var!" def bark "woof!" end

module Foo; def self.module_method

"hi!!"

end end

d { a } # logs "a = 'foo'" d { @a } # logs "@a = 'my var'" d { @@bar } # logs "@@bar = 'class var!'" d { bark } # logs "bark = woof!" d { Foo::module_method } # logs Foo::module_method = 'hi!!' [/ruby]

More Details

Log bugs/issues/suggestions here: http://opensource.thinkrelevance.com/wiki/log_buddy
Source: http://github.com/relevance/logbuddy/tree/master
git clone git://github.com/relevance/logbuddy.git
rdocs: http://thinkrelevance.rubyforge.org/log_buddy/

Testing Velocity - Keeping your test suite fast, Part 1

2008-03-26T00:00:00-07:00

If you are a Ruby or Rails developer, and you know what you are doing, you are writing tests or specs. Tests express the intent of your code, help verify correctness, and aid in design and exploration. Rails gives you helpful conventions to follow and functional and integration testing support for free out of the box. Ruby's power and flexibility allows you to reach in and alter objects at runtime so you can easily mock and stub external dependancies. In all the discussion and attention giving to tests versus specs, mocks and stubs, fixtures, and test coverage, one thing that doesn't nearly get enough attention is the importance of your test suite's speed.

Why does test speed matter? Consider at the most basic level what a test is: the smallest level of automated feedback in your application. You can view a typical agile application as a series of concentric feedback circles, the smallest being unit tests at the method level, up to story level tests (ie acceptance or integration), on to daily standups, iteration planning/review and release planning and retrospectives. The more you can tighten and tune all those feedback loops, from the micro level to macro, the more you can quickly respond to change and complete quality features that your app's users will really use and love.

If you've ever worked on an larger project with a lot of slow tests, you've probably seen the "slow tests antipattern": developers stop running the whole suite after small changes, because it takes too long and they get bored or distracted while they wait. Changes that require many small steps, such as major refactorings or API changes, become too burdensome to do regularly. Design, quality, and the code base as a whole suffer as a result. In the worst cases, developers stop running the test suite interactively and let a nightly build run it, or maybe the tests are abandoned altogether. Tests aren't run before checking code, and all the value from automated tests/specs is lost. Abandon all hope, ye who enter here.

Ruby, particularly in the current 1.8.6 MRI implementation, isn't exactly known for its peformance. Just ask the Rails Envy guys and they will tell you - "Rails doesn't scale". Maybe Rubinius or JRuby can rescue your slow tests a year or two from now when new virtual machines bring major improvements, but for right now it primarily comes down to you and your team's development practices.

What is a "fast" test suite? It depends on project size of course, but generally unit tests should run in the tens of seconds or less, and the full suite (including functional+integration) should be at most a few minutes. Browser-based acceptance level tests using something like Selenium are typically outside of the "interactive tests" group that I'm considering here, and take much longer. If your unit tests alone take a minute or two to run you will have some serious obstacles, if not now then later as your code base and technical debt increase.

So your test speed matters if you anticipate your app growing beyond 500 lines of code. Ruby is built for developer productivity and not machine productivity, but in order to sustain developer productivity we need to keep our tests running fast.

Future posts in this series will discuss different ideological and technical approaches to keeping you tests running fast. What does Rails consider a "unit" and "functional" test, and does it make sense? What is the actual goal of a good unit test? Should we use unit_record, nulldb, or something else? What about plain Rails2 fixtures versus fixture replacement, or maybe a home grown factory? Do YARV or JRuby offer some hope for the near future for some easy, free improvements in runtime?

I'm curious as to know what the "average" Rails test suite clocks in at, and what level of coverage it attains. Is your test slowness hampering your development? Do you rely on autotest, focused tests, or something else to keep the tests you run relevant? Please do share your thoughts on test speed and what sorts of issues this series should cover in upcoming posts.

Dave Thomas' testing library (thrown in github)

2008-03-14T00:00:00-07:00

Dave recently blogged a very cool little test framework he whipped up. Its under 100 lines of code and he says he definitely does not plan on supporting it publicly.

At the very least I intend on playing with it and incorporating some of the ideas, so I put it up on github. This does not constitute a release or support - I just want to watch the forks develop.

Grab it with

git clone git://github.com/rsanheim/prag_dave_testing.git

Staging Environments in Rails

2008-03-11T00:00:00-07:00

(photo @ flickr)

Most Rails apps that grow beyond the “toy” or “small” stage benefit greatly from the addition of a staging environment. Staging is where you deploy to flush out integration issues, to demo new features to users and clients, and generally put the app through its paces in a “production-like” environment before doing a real production deployment. It should match the production environment as closely as possible, though in practice things do diverge for certain cases and for most apps the cost of truly duplicating production hardware is prohibitive.

Rails lets you setup a staging environment easily. Create a file in config/environments called ‘staging.rb’, and then start your server(s) with RAILS_ENV=staging, and you are all set. Here’s quick steps to get up and running with a staging environment:

copy config/environments/production.rb to staging.rb
add an entry to database.yml for staging
tweak your deployment to respect your new environment using multistage or something simple like this:
[ruby] task :production do set :rails_env, "production" role :web, "prod.ip.here" # other roles... end task :staging do set :rails_env, "staging" role :web, "stage.ip.here" # other roles end[/ruby]

setup and deploy staging:

cap staging setup; cap staging deploy:cold; etc...

tweak and iterate as you see fit

You may ask at this point “why not just a copy of my production stack as staging?” The fundamental reason is that abstractions leak. Your staging environment is similar to production, but its not an exact copy. Think about email notifications - do you really want exception notifications or activation messages to be send the same in staging as production? Usually the answer is no - you want to be able to tell immediately if an error is from prod or staging, which could mean prepending [STG] in the subject and maybe changing the sender to be staging@domain.com. For a lot of apps, you will have tasks that will perform differently between staging and production. Some typical processes like this includes batch jobs, web service calls, and all sorts of notification jobs that would notify users or do a real external call, but in staging should just do a fake call and log the result.

To enable easier testing and cleaner code, a very simple wrapper class around RAILS_ENV is nice to have. It also makes me happier to have less constants in my code, as I feel all caps detracts from the beauty of Ruby. You can use Coda’s plugin or write your own in probably 5 minutes. Since you have a level of indirection around your environment checks, you can write specs like this:

[ruby]it "should only log notifier emails in staging" do Rails.stubs(:staging).returns(true) AppNotifier.expects(:debug).returns(mock("logger", :debug => "some logger call")) AppNotifier.send_reports end

it "should send emails in production" do Rails.stubs(:production).returns(true) AppMailer.expects(:deliver_reports) AppNotifier.send_reports end [/ruby]

So don’t be afraid to create a separate staging environment and give it the respect it deserves when your deployment is complex enough to demand it. It will make your life easier and your app easier to maintain and scale.

Tarantula, Rails super-fuzzer, released

2008-02-26T00:00:00-08:00

Stu has finally released Tarantula over on the main Relevance blog. Tarantula is probably the most exciting open source release Relevance has put out since I've joined the company about half a year ago. It basically will crawl your app intelligently, try putting garbage into forms and query params, and give you a nice looking report of what breaks and what doesn't. It can also validate html as it goes.

We have a lot of good ideas and plans for many more things this tool can do - one big thing I'd like to see is the ability for it to run in a black box mode against your real app (in a staging env, of course) in addition to a running as a Rails integration test.

Learn Git 10 Different Ways

2008-02-22T00:00:00-08:00

No particular order - things near the top just happen to be what I've had time to look into so far.

Git-svn crash course
FLOSS Weekly #19 - Interview with Junio Hamano, maintainer of Git
Git tutorial
Learning git-svn in 5 minutes
Scott Chacon’s Git with Rails screencast
Tech Talk: Linus Torvalds on git
Tech Talk: Randal Schwartz on Git
Using Git without feeling stupid (parts one and two)
Using Git with Django screencast
Git Screencast at Peepcode

Friends don't let friends commit centralized.

Productivity Talk abstract and resources

2008-02-20T00:00:00-08:00

Thanks to everyone who came out to Raleigh.rb last night for my talk. You can hear the talk (mp3 ~25 mb) thanks to Matthew Bass. The sound quality is decent considering it was just recording right into his macbook.

I've posted more details from the talk, the abstract, and references (books, tools, etc) at the main "can has productivity" page.

I Can Has Productivity?

2008-02-18T00:00:00-08:00

I'll be speaking at Raleigh.rb tomorrow on Productivity, Ruby, and how to get more stuff done. Spend more time programming and less time reading about programming (or reading PerezHilton, if thats your poison).

If you are in the area you should come out and say hi. Productivity is serious business.

[ANN] brain_buster 0.8.0 released

2008-02-12T00:00:00-08:00

I'm happy to release version 0.8.0 of BrainBuster.

This is a long overdue release. I've removed all deprecated code, updated for Rails 2.0, improved specs considerably, and cleaned up the api and views.

Because of the way the filter chain is handled in Rails 2.0, you will now have to handle a captcha failure explicitly. Right now thats just done by overriding a method in your controller. See the README for more info.

I hope to post a quick tutorial for getting up and running with BrainBuster over the next few days, so please watch for that if you want to learn how to add an accessible, image free captcha to your app.

BrainBuster has now been moved to github for the dev scm, and to the Relevance Trac for tickets and bugs.

The mailing list is still at google groups.