A key step to debugging any program is replicating the environment to ensure you can consistently produce the bug. In my early Ruby days, to inspect the environment, I used a primitive method: placing puts lines in my code to print values to the console (let’s call them “inspection puts”). It may have looked something like this:

class Buggy
  # assuming perform_operation and special_options exist...
  def buggy_method(param=nil, options={})
    puts "\n\n\nDEBUG"
    puts "param: #{param}"
    puts "options: #{options}"
    @instance_var = perform_operation(special_options(param, options))
  end
end

The case for ruby-debug

This method very easily gives me the information I need, but it has some downsides:

1. To inspect the return value of special_options I have to add another inspection puts.
2. Every addition of a new puts requires that I restart the application to inspect the results.
3. To inspect how special_options and perform_operation are handling the data, I have to add inspection puts inside of them.
4. I must remember to remove all of the inspection puts before I push the code.

If only there was a better way to do this.

ruby-debug to the rescue! By putting a breakpoint in our code, we have the ability to inspect environment state, check the return value of any methods, and step through our code one line at a time. This is much more versatile than inspection puts because the full environment is available to us. The “I wonder what the value of this is” problem is gone since we can inspect whatever we want. The step functionality the debugger gives us is useful as well, allowing us to step inside of a called method while maintaining the interactive environment.

Setting up ruby-debug

To get set up with the debugger, we’ll need to install the gem:

# Using MRI-1.9.2
gem install ruby-debug19

# Using MRI-1.8.7 or Ruby Enterprise Edition
gem install ruby-debug

ruby-debug in action

Let’s update the example from above using a debugger breakpoint instead of inspection puts:

class Buggy
  # assuming perform_operation and special_options exist...
  def buggy_method(param=nil, options={})
    require 'ruby-debug'; debugger
    @instance_var = perform_operation(special_options(param, options))
  end
end

The next time the method is called, the debugger will stop and give us an interactive shell. We can inspect the values of each variable with the eval command:

eval param
eval options

We can also see what the return value is of the invoked methods:

eval special_options(param, options)
eval perform_operation(special_options(param, options))

We can even use the step command to enter inside of special_options and perform_operation to see what they do.

Here are the various debugger commands that I most commonly use:

• list, l – show the code for the current breakpoint
• eval, e – evaluate expression and print the value
• step, s – next line of code, moving within methods
• continue, c – continue in the program until the program ends or reaches another breakpoint
• quit, q – abort the program

Many more commands are available, which can be seen by entering help in the debugger.

Better debugging ftw!

With the ruby-debug gem, we have a better tool for diving in to our code than inspection puts. Using a debugger breakpoint, we can interactively step through our code with less cleanup.

Happy debugging!

Debugging references

• Debugging with ruby-debug in the Debugging Rails Applications guide
• ruby-debug19 gem
• Example code from this post

Who am I?

• @yellow5
• Github

Possibly related posts

• Identify what your Ruby process is doing
• Idiomatic shared state in RSpec
• Quick data import and linking in Rails

For this example, I’m going to deploy a small static site from a private Github repository to a server via ssh using whiskey_disk. There are a few steps in getting this all setup, but once you have the ssh infrastructure playing nicely, a complete deployment of a small static site takes on the order of 2-5 seconds. Stay with me, and I’ll show how all the pieces fit together.

To begin with, I’ll assume you have a site or application already pushed to a private Github reposity that you now want to do a deploy of. You also have ssh access to the server you are deploying to for this to work. For this example, I’m deploying the master branch of the following private git repository:

git@github.com:rud/whiskey_disk-demo.git

Adding the configuration file

Make sure you have your appropriate code pushed to Github, then we’ll add the whiskey_disk config file config/deploy.yml to the project:

demo:
  domain: "deploy_user@example.com"
  deploy_to: "/home/deploy_user/domains/useful_site.com"
  repository: "git@github.com:rud/whiskey_disk-demo.git"

This is a simple YAML file (indentation matters), you will need to adjust the domain and paths as appropriate. I use the name demo for the environment, other common examples would be staging, production, etc. A lot more options are available, see the official whiskey_disk readme, but this all we need to get started.

Running whiskey_disk setup

whiskey_disk has two commands: a setup command, for doing the initial repository clone, and the deploy command used for all subsequent deploys.

With the config/deploy.yml file in place inside the project, we’re going to just try and run the setup command from the local computer, seeing what happens. For now, this errors out:

$ wd setup -t demo
Initialized empty Git repository in /home/deploy_user/domains/useful_site.com/.git/
Host key verification failed.
fatal: The remote end hung up unexpectedly
fetch-pack from 'git@github.com:rud/whiskey_disk-demo.git' failed.
error: pathspec 'origin/master' did not match any file(s) known to git.
Did you forget to 'git add'?

Okay, we’re not quite ready to do deployments just yet.

Resolving all ssh issues

Github hangs up on us – what’s happening? Testing from my local machine as described in the github ssh guide:

$ ssh -T git@github.com
Hi rud! You've successfully authenticated, but GitHub does not provide shell access.

That’s the kind of success we’re looking for on the server as well. Ssh to the server as the deploy user, and try the same command there:

$ ssh -T git@github.com
The authenticity of host 'github.com (207.97.227.239)' can't be established.
RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,207.97.227.239' (RSA) to the list of known hosts.
Permission denied (publickey).

Try the command again, now that the host key is trusted:

$ ssh -T git@github.com
Permission denied (publickey).

Notice how we get a new error later in the connection process, a Permission denied (publickey) instead of Host key verification failed.

For the sake of completeness, this is what this situation looks like from whiskey_disk:

$ wd setup -t demo
Initialized empty Git repository in /home/deploy_user/domains/useful_site.com/.git/
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
fetch-pack from 'git@github.com:rud/whiskey_disk-demo.git' failed.
error: pathspec 'origin/master' did not match any file(s) known to git.

ssh-agent to the rescue

The root cause for this difference is Github knows my public key and has access to the corresponding private key, but this key is not available to my server, so it cannot authenticate against Github. The solution is fortunately simple. Do this on your local machine:

$ ssh-add

This prompts you for the password for your private key, then keeps the private key unlocked in memory for automated access. You need to do this after each time you restart before you can do new deployments, as the unlocked key is only stored in memory. We also need to enable ssh-agent support for the connection to the server – as ssh-agent gives a server the possibility to use our private key without further prompting, you need to trust the administrator of servers you deploy to. With that caveat out of the way, open your ~/.ssh/config file, and add a config block like this:

Host example.com
    ForwardAgent yes  # This enables ssh-agent functionality, making whiskey_disk happy

Adjust the hostname to taste, obviously. You can also add port-number here, if your admin moved ssh access from the default port. While you have that file open, also check out these extremely handy ssh productivity tips for a lot of extra ssh goodness.

Close the previous connection to your server, open a new one (so ssh-agent can do its magic), then re-run the test:

$ ssh -T git@github.com
Hi rud! You've successfully authenticated, but GitHub does not provide shell access.

We have ssh-win!

Deploying

Time to re-try the deployment setup from the local machine:

$ wd setup -t demo
Results:
deploy_user@example.com => succeeded.
Total: 1 deployment, 1 success, 0 failures.

This means the repository was successfully cloned into place. We only have to do this once, from now on deployments are just a matter of:

$ wd deploy -t demo
Results:
deploy_user@example.com => succeeded.
Total: 1 deployment, 1 success, 0 failures.

I’m seeing a total time taken for a deployment in the area of 2-5 seconds. Remember to push to Github before running your deployment, as it is from Github your server is pulling versions.

This method for deployment is extremely awesome, and the amount of awesome grows with the size of your deployments, as server-server connections are most likely a lot faster than your own internet connection.

Next steps

There is a lot more to whiskey_disk that I’ve shown here. In particular I’d like to highlight the “Configuration Repository” section of the official whiskey_disk readme. Editing per-application configuration files on individual servers is effectively a thing of the past.

Possibly related posts

• Modernize Your Git Workflow
• The sneakernet in git
• chop – efficiently dispense with current git branch

I have previously described how I’ve been using small hack && ship commands to simplify and streamline my personal git workflow. Recently I’ve switched to using the Git Flow setup as my branching and release workflow.

Setting up git-flow

Installing the git-flow toolset on OS X is trivial using the homebrew installer:

brew install git-flow

The git-flow project also has instructions for installation on Linux and Windows.

Installing git-flow adds a few helpful commands to the git environment for creating and managing branches for features and releases. A fresh git repository is born with a master branch. In the default git-flow setup, this is where the current production release lives. Furthermore, a branch called develop is created, this is where development takes place. Note that git-flow is just a series of shortcuts to having a development branch and a production branch with sensible ways of shuttling changes back and forth. After installing the git-flow package, configure your local repository for git-flow use with:

git flow init

You can most likely accept the defaults by pressing enter at each question – this also makes it easier for others to initialize git-flow in their working copy, as all are using the same defaults. The only change from running the init command is two [gitflow] sections in your .git/config file looking like this:

[gitflow "branch"]
	master = master
	develop = develop
[gitflow "prefix"]
	feature = feature/
	release = release/
	hotfix = hotfix/
	support = support/
	versiontag =

It’s no more magic than that. Consult the built-in help with:

git flow

If you’ve got the bash-completion package installed, you can use tab-completion for git-flow commands and arguments. Highly recommended.

Working on a feature

Start work on a feature – say feature 77 from your issue-tracker:

git flow feature start 77-speedup-yak-shaving

This creates a new regular local git branch called feature/77-speedup-yak-shaving based on the current develop and places you on it. Want to share your work in progress with others? Use:

git flow feature publish 77-speedup-yak-shaving

This pushes the branch, and sets up your local branch to track the remote branch in one easy step. The regular git push and git pull --rebase work as you would expect here. There is nothing special about branches created by git-flow, they are just conveniently named and managed.

All done with a feature? Rebase your feature on the current develop branch, then merge it in:

git flow feature rebase
git flow feature finish 77-speedup-yak-shaving

With these two steps, you end up on develop with your feature merged in. Run your test-suite, and push to a central repository as appropriate.

Releasing to production version and hotfixes

Production releases are handled quite nicely in git-flow:

git flow release start 2011_year_of_the_yak

This creates a new branch called release/2011_year_of_the_yak, based on the current develop branch. Here you can fix any HISTORY or VERSION file, commit, and the finish up the release with:

git flow release finish 2011_year_of_the_yak

By finishing the release, you now get a tag called 2011_year_of_the_yak, and the temporary release-branch is removed. You end up back at the master branch.

A hotfix is simply a feature branch which is based on the latest production release, and which is automatically merged to both develop and master. Extremely handy.

Rebase considered harmful?

Some developers prefer to only ever merge, but I’m a big fan of rebasing local work before publishing. I’ll leave you with this:

If you treat rebasing as a way to clean up and work with commits before you push them, and if you only rebase commits that have never been available publicly, then you’ll be fine. If you rebase commits that have already been pushed publicly, and people may have based work on those commits, then you may be in for some frustrating trouble.

From Pro Git, chapter 3

Possibly related posts

• Deploy a private Github repository with whiskey_disk
• The sneakernet in git
• chop – efficiently dispense with current git branch

$ git bundle create blog-dump.bundle master

This creates a file blog-dump.bundle with the full history for master, including all parent commits. You now have a single file, which can effectively be mailed, downloaded via SCP or FTP, or simply moved to Dropbox (affiliate link) for safe keeping, or otherwise shared with others. Single files are easy to archive and backup.

Thawing a bundle

When you want to restore from such a bundle backup, you simply clone from it. The steps include creating a new repository, and then pulling from the bundle, like you would a remote repository:

$ git init
$ git pull blog-dump.bundle master

Your repository now holds all commits that were stored in the bundle. For more usage-scenarios see the official git bundle help or just type:

$ git bundle --help

Possibly related posts

• chop – efficiently dispense with current git branch
• Easy local code-review with git
• Simple git workflow with hack and ship

One set of built-in helpers I’d like to cheer on a bit are let and let!, as they can greatly simplify setting up state for multiple examples. They exist to provide a simple and convenient way to create and share state between examples. When not using these methods, specs with a bit of shared state typically look like this:

context "with a bit of state" do
  before :each do
    @user = User.create!
  end

  # The following specs assume the user has already been created
  it "should be the latest user" do
    User.last.should == @user
  end

  # .. more examples reusing the @user
end

This is easy to get started with, and when there are only a few variables in the before block, things are fine. But if you have a typo in an example where you intended to use the @user instance variable in an example, but you accidentally typed @usr, you just get a nil value, instead of a helpful error message regarding an unknown indentifier. This minor problem can be avoided by using let instead, as you get an unknown indentifier error instead of a nil value. Another issue is that all variables in the before block are evaluated, whether or not they are actually used in the examples.

The excellent official RSpec 2 documentation has the following to say on let, before showing clear examples of use:

let and let!

Use let to define a memoized helper method. The value will be cached across multiple calls in the same example but not across examples.

Note that let is lazy-evaluated: it is not evaluated until the first time the method it defines is invoked. You can use let! to force the method’s invocation before each example.

The fact that results are both memoized and lazy-evaluated means no extra effort is expended if the value is not used in a given example. Performance win!

Use let or let!?

Please use the correct helper method – I’ve seen examples like this:

context "with a bit of state" do
  let(:user) { User.create! }

  before :each do
    user
  end

  # The following specs assume the user has already been created
  it "should be the latest user" do
    User.last.should == user
  end

  # .. more examples reusing the user
end

The exact same thing can be expressed directly and become more readable, just by adding a !:

context "with a bit of state" do
  let!(:user) { User.create! }

  it "should be the latest user" do
    User.last.should == user
  end

  # .. more examples reusing the user
end

The let and let! helpers are available in from versions rspec 1.3.1, and rspec-core 2.0.0 onwards. In a nutshell, this means they are available for use in Rails 2 and Rails 3 projects. Take them for a spin if you haven’t already, and tell me how it goes.

I recommend having a quick look at the RSpec 2.5 documentation now, as you’ll probably discover one or more neat new solutions to recurring issues when writing specs.

Possibly related posts

• Getting to know Ruby debugger
• Identify what your Ruby process is doing
• Quick data import and linking in Rails

Soon you’ll be back on the road to awesome.

Simply install and load the xray gem, then add this to your Gemfile (or similar):

gem "xray", :require => "xray/thread_dump_signal_handler"

Next, you need to start the ruby-process, and find the PID (process identifier):

$ ps ax | grep ruby

You can now invoke:

$ kill -3 1234

where 1234 is the PID of the ruby process. Note this does not terminate the process. Instead, you immediately see a full stacktrace for current thread written to standard out of the process. If you’re using the Ruby Enterprise Edition runtime, you will get a stacktrace for all threads, a big win. If you are running the process interactively, you see the output directly in the terminal. For service-processes in production, you will need to check the appropriate logfile.

Thread dumps for Phusion Passenger with RVM

You can see all running Phusion Passenger instances, including their individual PIDs with this:

$ rvmsudo passenger-status

Then move in for the kill -3, and have a look at standard out for Passenger. By default on OS X 10.6, this ends up in the file /private/var/log/apache2/error_log, so have a look there. Notice the use of rvmsudo instead of regular sudo – this is because I’m using RVM to manage my ruby-versions.

I hope you found this useful – what is your stacktrace-aided war-story?

Possibly related posts

• Idiomatic shared state in RSpec
• Quick data import and linking in Rails

It is easy to install with DarwinPorts:

sudo port install p5-app-ack

This puts the ack command in your PATH and you’re ready to go.

Basic usage of Ack

A recurring task is to find all uses of a method or class in a project. As this is simply a substring search, you invoke Ack with:

ack inconvenience_late_payers

and ack quickly starts outputting all the places where this is used. If you add a -i, the search becomes case-insensitive. Use ack --help to see all the many available options.

A bit of configuration goes a long way

Ack is fast because it skips uninteresting files. It is meant for searching for text, and skips version control folders, logfiles, images, and a lot of other stuff you are not interested in when searching through your source-code. However, the backside is you need to configure things once to have support for all your various kinds of source-code.

Adding this small bit of config means .haml, .sass, and Cucumber .feature files are also included when searching.

~/.arkrc:

--type-set=haml=.haml
--type-set=sass=.sass
--type-set=cucumber=.feature


With this in place, you are ready to rock HAML/SSS + Cucumber projects.

Know any extra must-have options or shortcuts I have left out?

Update: added link to official Ack homepage.

Possibly related posts

• chop – efficiently dispense with current git branch
• Easy local code-review with git
• Simple git workflow with hack and ship

Now, meet chop – for chopping down the current working branch after it has been shipped and is no longer needed. The script changes the current branch to master, and then deletes the branch you was previously on. If you give a branch-name as an argument that will be the new current branch.

#!/bin/sh -x
set -o errexit
CURRENT_BRANCH=$(git branch | grep '\*')
git checkout ${1:-"master"} || exit 1
git branch -d ${CURRENT_BRANCH:2}

I use this small script is multiple times every day, and I really like the name of it. There is not a whole lot of functionlity, but as this is an often repeated action, it makes sense to automate it.

Enjoy!

Possibly related posts

• Easy local code-review with git
• Simple git workflow with hack and ship
• Ack! Effortlessly search your codebase

Assume you have a pipeline of post-import steps to run. This can be organized in numerous ways. Simplest is to just have a bunch of methods called one after the other once you have the data loaded:

• link_frobnitz
• spin_really_fast_around_z_axis
• reticulate_splines
• deploy_hamsters

Now, assume once in a while one of the steps fail for an unexpected reason. You know, it’s rare data from external sources is as clean as we’d like. So you need to fix a few things and retry the import. However, as datasizes grow and with that the running time of the import, it can be a huge waste redoing all the work because of a misplaced comma made the final deploy_hamsters step fail.

Exceptions are the obvious way to report fatal data-errors, and implicit or explicit transactions to ensure consistency of the import. But how can this easily be combined for a resume-friendly import mechanism?

Enter the bulk importer step runner with trivial progress reporting:

def import_updaters
  all_steps.each do |step_name|
    run_import_step(step_name)
  end
end

private

def run_import_step step_name
  puts "Running #{step_name}"

  ImportModel.transaction do
    self.send(step_name)
  end

rescue => e
  STDERR.print "\nImport error:  #{e.inspect}\n#{e.backtrace.join("\n")}"
  STDERR.print "Please resume at step #{step_name}"
  exit 1
end

protected

def all_steps
  [
    :link_frobnitz,
    :spin_really_fast_around_z_axis,
    :reticulate_splines,
    :deploy_hamsters
  ]
end

Notice you obviously have to change the model-name (ImportModel above) and provide the actual implementation for these individual steps. all_steps returns the list of methods to run, run_import_step runs a single step with error-handling, and import_updaters runs all the relevant updaters.

Easy performance statistics

As a bit of bonus-functionality, the following can be used for reporting import progress with timing-statistics after each step completes:

def report_progress message, &block
  STDERR.print message

  if block_given?
    time = Benchmark.measure { yield }
    formatted_time = "%.2fs" % time.real

    STDERR.puts " - #{formatted_time}"
  else
    STDERR.puts
  end
end

Usage is simple – just call report_progress with a comment to print and a block of code, like this:

def run_import_step step_name
  report_progress "Running #{step_name}" do
    ImportModel.transaction do
      self.send(step_name)
    end
  end
  //...

What do you use to make data-imports easier to manage?

Possibly related posts

• Idiomatic shared state in RSpec
• Identify what your Ruby process is doing
• Ack! Effortlessly search your codebase

So, assuming a fairly linear history of commits from multiple developers, how do you easily keep track of what you have already read through and reviewed? Easy, use a local branch as a bookmark. This tiny script makes it trivial to add or update such a branch:

#!/bin/sh
NEW_BASE=${1?"Usage: $0 <treeish>"}

git branch --force reviewed $NEW_BASE || exit 1

echo Marked as reviewed: `git rev-parse --short reviewed`

Save this as a new file called reviewed.sh in your PATH.

Usage is extremely simple:

reviewed.sh 369b5cc
reviewed.sh master
reviewed.sh v1.0.7
reviewed.sh HEAD~5


Running one of these commands will mark the given treeish as reviewed, and when you look at your commit-history in a visual tool such as git gui, or gitx, the reviewed branch visually indicates how far you have gotten. Note how both commit-IDs, branch names, tags, and relative commit-IDs can be used as argument.

You can also utilize this review bookmark from the commandline. The following shows you all commits added to master since your last review:

git log reviewed..master --reverse

You can add a --patch to that command to see the full diff for each change. Adding --format=oneline just shows you the commit-IDs and first line of the commit-message.

Once you’ve read all the latest commits on master, simply do a

reviewed.sh master

and you’re done.

Why not use a tag?

I find it convenient to be able to do a push of all tags to the central repository with

git push --tags

and this would share such a private review-tag. As this is my private reminder of how far in the commit-history I have reviewed, sharing it is just confusing to other developers.

Notice: Any commits which are added only to the reviewed branch are unreferenced when you mark a new treeish as reviewed. Just something to keep in mind.

How do you keep track of the flow of changes?

Possibly related posts

• chop – efficiently dispense with current git branch
• Simple git workflow with hack and ship
• Ack! Effortlessly search your codebase