def blog

Developing in Linux on OSX

2015-10-06T00:00:00+00:00

I wanted to move to Linux development environment, but at the same time I wanted to stay on OSX for everything else. So VirtualBox was the obvious option. I also had few extra requirements:

lots disk space
integration with OSX clipboard (so that vim/xclip work transparently)
static IP (for hosts file, nginx conf and general convenience of never having to look up new IP address)

If you happen to need something similar, then simply follow TLDR; below.

TLDR;

Install Packer, VirtualBox, Vagrant
git clone https://github.com/artemave/vagrant-boxes.git && cd vagrant-boxes
cd packer && packer build --only=virtualbox-iso -var 'DISK_SIZE=102400' ubuntu-15.04-server-amd64.json; cd .. (disk size is in megabytes)
vagrant box add --name 'ubuntu-15.04-server-amd64' ./builds/virtualbox/ubuntu-15.04-server-amd64_virtualbox.box
vagrant up
vagrant ssh

Provision script will create a user account on the vm with the same name as the user on the host machine you’re running vagrant up from. Following Vagrant convention, the password is ‘vagrant’.

The default RAM size is set to 8gb. You can adjust this in Vagrantfile. Then vagrant reload.

To get host/guest clipboard integration, install XQuartz, set the pasteboard settings to look like this:

and make sure XQuartz is running whenever you vagrant ssh.

You can also build a VMware box (--only=vmware-iso), but I haven’t tried it.

Notes (for the curious)

When I say “development environment” what that really means is console development environment. I use tmux for keeping sessions/windows per project, Vim as an editor and command line for everything else.

This kind of setup is perfectly achievable in OSX of course. The main reason to move to Linux was the lack of native docker support for OSX. “But there is docker-machine (former boot2docker)” I can hear you saying. And that is true. But there is also a range of minor issues (such as no support for host data volumes, or slow VirtualBox file sharing for when you want to develop against running container, or some more obscure ones) that come with it. It is just one more abstraction layer that will always get its toll.

There are other good reasons to use Linux, but, perhaps, it was just that time of the year when one needs to shave a yak, so I thought why not, let us give it a go.

By the way, nice thing about having all work stuff in a VM is that when you need to restart your laptop, it only takes vagrant suspend to pause the VM and vagrant up after restart to pick things up right where you left them. Also setting up new machine is much faster.

Initially I wanted full GUI experience (with i3wm tiling window manager), but the graphics turned out to be far too slow. A bit disappointing, giving the top dog MacBookPro.

Then I decided to use Vagrant with a server box. However stock boxes are set to have 40GB disk and, as I quickly learned, resizing VB disk images involves more steps than a human can follow.

So I ended up forking the base boxes repository, changing the disk size in packer config and building the box myself.

To keep things simple I also dropped the Vagrantfile into the root of my fork. There are three things to note there:

static IP: config.vm.network "private_network", ip: "192.168.33.10"
ssh with X11 forwarding (for clipboard integration): config.ssh.forward_x11 = true
provision script: installs few useful packages (opinionated) and, most importantly, compiles Vim with X support (for clipboard integration) along with some other batteries (such as Lua support for Unite.vim).

Setup Nginx reverse proxy (with ssl termination) on OSX

2015-04-01T00:00:00+00:00

This is NOT a joke.

To test Google Game Services integration locally, I wanted my local machine to be accessible over https on something other than locahost or 127.0.0.1. So I decided to employ localtest.me and use nginx locally as ssl terminating reverse proxy to my app running on port 9999.

Install nginx:

% brew install nginx

Generate self-signed certificate:

% openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /usr/local/etc/nginx/cert.key -out /usr/local/etc/nginx/cert.crt

Nginx.conf:

# Important! I had to change this to myself (artem) or it wouldn't serve js assets with "permission denied".
# Group is mandatory despite nginx docs claiming otherwise.
user  artem admin;

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    sendfile        on;

    keepalive_timeout  65;

    server {
        listen       443 ssl;
        server_name  abc.localtest.me;

        ssl_certificate      cert.crt;
        ssl_certificate_key  cert.key;

        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;

        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;

        location / {
            proxy_pass http://localhost:9999;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
        # Proxy websocket connections (you probably don't need this)
        location /games/play/ {
            proxy_pass http://localhost:9999;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }
    }
}

Start nginx as sudo:

% sudo nginx

Test it: https://abc.localtest.me

Value Added Tests

2014-12-08T00:00:00+00:00

TL;DR: high level acceptance test should focus, reveal and describe the value that a feature adds to the business.

This statement may be obvious once you “get it”, but may sound remote and abstract if you don’t. So let us leave it hanging around for a while and check out this (slightly adapted) real world example.

Suppose we’re a car retailer website. We’d like to introduce a “personal assistant” feature so that every time someone requests a quote, a member of our sales team gets assigned to it and an email introducing the assistant gets sent to the user. The auto assignment can be changed later from an admin interface.

We want to deliver early so we decide to break up the feature and first tackle the creation of “assistant profiles” for existing sales user accounts. After all, you can’t assign anything if it does not exist yet. But, more importantly, some of the new code will end up in production without any public facing changes. So the rest of the new code will be smaller and therefore less disruptive to release. Plus, while we’re busy with the rest of the feature, someone from the sales team can go away and fill up the profiles.

We practice test driven development and the “outside in” approach, so we start with an integration test that looks like this (if this syntax - ruby/rspec - it does not look familiar, just assume that it does what you think it does):

scenario 'create assistant profile' do
  # test data
  dave = create_sales_user()
  admin = create_admin_user()
  img = test_image()

  # create assistant profile
  visit admin_path()
  login(admin)
  click_link('Sales users')
  click_button('Edit')
  click_button('Create assistant profile')
  fill_in('name', with: 'Dave')
  attach_image(img)
  click_button('Save')

  # check if it has made it into db
  assert_assistant_profile_for_user(dave)
end

Looks pretty straightforward, so we make that happen and deploy to production. And off to implement the assignment part. I’ll spare the details but while working closely through screens and buttons we realise that assistant profile, in addition to what it already has, should also have have a last name. Turns out, while customers should only see first names, if we happen to have two Daves on the sales team, it will be hard to distinguish between them in case of reassigning (think of a drop down with only names vs drop down with “name surname”).

Also, the email that gets automatically sent to a customer upon an assignment, should have a round image of Dave (according to designs) and that image could not be rounded with css since there ain’t no fancy css (round corners that is) in emails. Therefore, we have to generate the round version when the profile image is uploaded.

All that means that assistant profiles created so far will have to be changed to add last name and the images will have to be reuploaded. And the next deploy to production is going to involve down time for db migration (we thought we were done with migrations in the first part).

Well, luckily, none of this is a big deal since no one from the sales team has actually bothered to create any assistant profiles yet.

Those issues are not massive, of course. But this simple example hopefully illustrates how, on a bigger scale, things can suddenly get out of sync when features, broken up into bits based on what developers think would be the best way to deliver them, do not quite match up when you put them back together in the end. This results in extra integration work (by changing something that already is assumed to have been done) or, worse, accepting those inconsistencies for whatever reason, making business to get on with it and building up new features on top. To relate that last one back to our example, we could suggest to use email address in the “reassign assistant” drop down and that way we won’t have to introduce last name at all. It is less user friendly, but does not cost anything. Suddenly, something that we would not even consider in the beginning, becomes a viable option just on that basis.

Now. Let us see how we could have avoided those inconsistencies early at a low cost. Let us rewind back to the point where no code has been written yet and we were laying out the first integration test. Except this time let us call it an acceptance test. And what that implies is not just a change of a buzzword but a change of how we think about it. Let us not think about what links and buttons users are going to click, but, instead, let us focus on what motivates them to take the hustle of doing so. Here is our first scenario written this way:

scenario "create assistant profile" do
  given_jessy_is_an_admin()
  and_walter_is_a_sales_person()
  when_jessy_wants_to_make_walter_an_assistant()
  then_he_can_do_so_from_an_admin_interface()
end

Wait, what? Does Jessy create an assistant profile so that he can… create an assistant profile? That makes no sense. Let us think again. But no matter how hard we try, turns out, on its own, there is no better value in just creating assistant profiles. Which reveals the fact that there is no value at all. And that is why sales haven’t created any profiles yet. They are pragmatic and also busy. They’d only invest time in doing something if the return on that investment was worth it. (of course, in reality, there can be plenty of other reasons, but I am making a point here!)

So let us write something that adds value, shall we?

scenario "customer contacts an assistant by replying to introduction email" do
  given_ned_just_requested_a_quote()
  and_he_realised_that_he_forgot_to_ask_about_finance()
  when_he_receives_an_email_introducing_an_assistant()
  then_he_can_reply_to_this_email_with_his_question()
end

There is also going to be a “customer contacts an assistant from a quote page” scenario. And a scenario for every other way for a customer to exercise “contact assistant” feature. But let us just focus on this one.

Notice how it assumes that “assistant profile” is already an existing thing. By the time we’re done implementing this scenario (and its siblings), there will still be no user interface for creating those. There will, however, be an obvious need to create them. Since, otherwise, none of the above will produce any value. This is a good old “outside in” only this time applied on a higher level where no code, but the features themselves are being designed.

So, perhaps, now it would be the right time to do the CRUD bit? It certainly is a better time, but let us keep exploring what “assistant” is without diving into admin interfaces just yet. Why? Because that admin ui is the direct reflection of our current knowledge of what “assistant” is. When the knowledge changes, the code will have to be changed too. Best wait until our knowledge is complete.

And there happens to be one more feature in the pile that is not about managing profiles - “reassign assistant”:

scenario 'admin reassigns assistant' do
  given_tweedy_has_requested_a_quote()
  and_rocky_has_been_assigned_to_assist()
  when_he_gets_hit_by_a_bus()
  then_an_admin_can_reassign_ginger_to_be_tweedies_assistant()
end

While implementing the reassign drop down, we realise that an assistant also requires a last name. Good thing we didn’t rush with the admin ui. We’d have to go back and add it everywhere.

Ok, now can we do the CRUD? Yes. But it is no longer just about the CRUD. As we’ve already established, there isn’t much value in it on its own. But what is that value then? There is one last piece of functionality left unspecified: initial assistant assignment. Remember that “you can’t assign anything if it does not exist yet” argument? Well now it is the right time to pull it out. So let us connect the two and complete the puzzle:

scenario "admin creates an assistant" do
  given_marty_is_a_sales_person()
  and_rust_is_an_admin()
  when_he_creates_an_assistant_profile_for_marty()
  and_then_customer_reggie_requests_a_quote()
  then_marty_becomes_reggies_assistant()
end

And… cut!

There are tools that support the this style of testing. Most notably, Cucumber (using Gherkin syntax). A lot of people don’t like Cucumber and they have good reasons for it. Still, in my opinion, it is well worth it.

But this is not why I am mentioning it. One part of Gherkin is a feature narrative. It is a blob of text at the very top that is not mapped to any executable code. For that reason, it is often skipped because it is just words. But if you’re sold on that value thing I am talking about, that would be the starting point of exploring it. The starting point here is to understand why the business needs the feature, before getting into the details of individual scenarios. In the hunt for value, this will set you on the right track. And it is still “outside in” by the way, only at the even higher level. Back to our example we could write something like:

In order to reduce the drop out of potential customers,
we’d like to make their experience more personal and engaging by assigning an “assistant” to each quote request.
Assistants will follow up on quotes and will be there to answer any questions.

So, yeah, The Value. It is an incredibly important thing to question. The earlier the better. Doing so costs little compared to what it can save. And going back to our tests, it is kind of hidden when the test is written as a sequence of actions (imperative style), but a lot more obvious when it is written in terms of what users are trying to achieve and why (declarative style).

Ajax in Rails how it is meant to be

2014-07-10T00:00:00+00:00

If you can use turbolinks (e.g. brand new project), use it and read no further. It’ll cost you nothing and will get you a long way before you need anything more sophisticated.

Like so many others, the website I am currently working on is using client side js and ajax to make user experience snappier by not reloading page when they click links and buttons. But, ultimately, every user action results in a server call. No state is client side only.

To do this, we intercept form submissions and link clicks in js, turn them into $.ajax and handle the json or html returned from the server in success handler. Pretty standard. And pretty repetitive too. Why so? Because every time code follows the same pattern of intercepting the user event, rebinding to $.ajax and handing the response. Over and over again.

Rails is good in shortcutting common tasks away. And this case is no exception. Below I’ll demonstrate what rails has to offer using a simplified example of posting a comment into the comments thread (source code)

Let us start with a non-ajax version (commit):

controllers/comments_controller.rb:

def index
  @comments = Comment.all
end

def create
  Comment.create! comment_params
  redirect_to action: :index
end

views/comments/index.html.erb:

<%= form_for Comment.new do |f| %>
  <%= f.text_area :text %>
  <%= f.submit %>
<% end %>

<%= render @comments %>

views/comments/_comment.html.erb:

<div class="comment">
  <span><%= comment.text %></span>
  <span><%= comment.created_at %></span>
</div>

Now let us ajaxify adding new comment.

Json way

Intercept form submission and turn it into an ajax call that returns json (commit):

comments/comments_controller.rb:

def create
  comment = Comment.create! comment_params

  respond_to do |format|
    format.html { redirect_to action: :index  }
    format.json { render json: comment }
  end
end

assets/javascripts/comments.js.coffee:

$(document).on "submit", "#new_comment", (e) ->
  e.preventDefault()
  $form = $ this

  $.post "#{$form.attr 'action'}.json", $form.serializeArray(), (comment) ->
    $text = $('<span>').text comment.text
    $createdAt = $('<span>').text comment.created_at
    $newComment = $('<div class="comment">').append($text).append $createdAt
    $('.comment:last').after $newComment

This might not be the best way of client side rendering, but it immediately highlights the problem of duplicate rendering of the same thing - comment - on the client and on the server. So, instead of returning json, let us reuse server side template and return html instead.

HTML way

commit

controllers/comments_controller.rb:

def create
  comment = Comment.create! comment_params

  if request.xhr?
    render comment
  else
    redirect_to action: :index
  end
end

assets/javascripts/comments.js.coffee:

$(document).on "submit", "#new_comment", (e) ->
  e.preventDefault()
  $form = $ this

  $.post $form.attr('action'), $form.serializeArray(), (html) ->
    $('.comment:last').after html

That is where we normally stop. Our client side javascript is largely a combination of the two approaches above: json and html. But there is a third one where rails returns javascript that automatically gets evaled (by jquery-ujs) on successful response. Sounds evil. But let us see it in action.

Javascript way

commit

controllers/comments_controller.rb:

def create
  @comment = Comment.create! comment_params

  respond_to do |format|
    format.html { redirect_to action: :index }
    format.js
  end
end

views/comments/create.js.coffee

$('.comment:last').after '<%= j render @comment %>'

We also need to add remote: true to form_for in comments index.

So?

So why is this better than returning html?

Less code. More specifically, less glue code. Glue code is repetitive and boring. Good riddance.
No reference to the dom selector (of the comments form). Therefor, less coupling to maintain. Plus you don’t have to make up names. Because naming things is hard.
Better code location. In json/html example the js code can be anywhere. Which means, harder to find. Whereas js template can only be in one place (its controller view path - views/comments) and named after the action - create.js.coffee.
Less client/server side code distinction. Means: less layers in development stack. Apart from returning javascript instead of html, js template is no different from html one. The same erb and rails helpers are available there.
No need to pass server side data to client js. That is where we normally start abusing html5 data attributes or simply store server side data (e.g. path helper values) in javascript variables on page load so it can be later accessed from javascript. Another bowl of glue.

But what about “loading…”?

Sometimes you need to run javascript before the ajax request is sent. Typical example would be to disable user control (link or button) while request is going on and maybe show a loading spinner or something. Rails provides a simple shortcut for disabling controls: remote buttons and links accept disable_with option that will disable a control while request is in progress optionally changing its text.

Also, jquery-ujs broadcasts a number of events which can be used to trigger global loading indicator or something along those lines.

Obviously they can also be bound to specific links, forms and buttons but that is where it stops being pretty :)

UPDATE (17 July 2014)

Josh Chisholm suggested in comments a variation of js approach in which AccountsController#create redirects to index instead of rendering javascript (commit). The nice thing about it, is that there is no need for different respond_to based on type of request (html or js) in create. Request processing follows the same path regardless.

Git a Grep on Vim

2014-06-02T00:00:00+00:00

UPDATE Combination of .gitignore and .agignore will do just fine (though it falls short on subdirectory specific ignores). So ignore the rest of this post.

I’ve been a happy user of ag.vim for years until recently I started to work on a phonegap project. Two things are special about it:

./node_modules
compiled source files

Both of these are included when grepping the project and, man, this is annoying. And slow.

So today I’ve had enough and decided to do something. Surprisingly, quick googling didn’t reveal anything I could just copypaste somewhere, so I had to put on my bash pyjamas and sort this out like a pro.

I’ve heard of git-grep and the plan was to bolt it in instead of a stock ag, but only if current vim folder is a git project (or a subfolder of one). This way none of the aformentioned types of files will be included in the search.

So this is it. Two steps.

First. Save this code to agprg.sh:

#!/bin/sh
# this is used as vim grepprg in conjunctions with ag.vim

if [ -d .git ] || git rev-parse --git-dir > /dev/null 2>&1; then
  git grep -n $1 | while read git_grep; do

    file_and_line=$(echo "$git_grep" | cut -d: -f1,2)
    match=$(echo "$git_grep" | sed 's/[^:]*:[^:]*:\(.*\)/\1/')
    column=$(echo "$match" | awk "{print index(\$0, \"$1\")}")

    echo "$file_and_line:$column:$match"
  done
else
  ag --column $1
fi

Make sure this is in the $PATH and don’t forget to chmod +x it.

Second. Add this to .vimrc:

let g:agprg = 'agprg.sh'

And voila, search. As in, The Search. Fast and furious.

If you don’t use ag.vim then you can still use this with vimgrep. Just add this (in addition to the above) to .vimrc:

let g:grepformat="%f:%l:%c:%m"

The silver searcherer is still expected to be installed.

Faster Chrome extension development cycle

2013-12-28T00:00:00+00:00

Here is how developing chrome extension works out of the box:

make a change
open “extensions” page
find your extension there
hit reload
test the change

Steps 2,3,4 really should not be there. Doing this over an over again just makes me want to stop writing software and get into baby clothes instead.

Turns out I am not the only one to feel that pain. There is an Extenstion Reloader extension that, if combined with Guard, automates those steps.

There is a bit of setup. But really is just a bit, assuming you’ve got a ruby interpreter and a terminal. It works on OSX and, with minor adjustments, on Linux. Not sure about Windows.

Install guard-shell gem:

$ gem install guard-shell

Add Guardfile to the root of your project with the following contents:

guard :shell do
  watch /(js|css|html|json)$/ do
    `open -g http://reload.extensions`
  end
end

In a terminal, cd into project folder and run guard.

That is it. From now on, while guard is running, changing any file inside your project will result into extensions reload in a background.

</br>

One caveat though, looks like Extension Reloader does not reload changes in manifest. Other than that, you got it!

CIborg anywhere (not just on ec2)

2013-07-29T00:00:00+00:00

ciborg lets you easily create CI instances on Amazon ec2. It is truly amazing in the sense that it just works. Having recently been stubbed multiple times to death with knife by chef, I am very grateful that tools like ciborg do exist.

The only downside is that it is coupled to ec2 and ec2 only. Hopefully folks at pivotal labs will sort this out at some point, but as of this writing, that is the reality of it. So whether you want to explore other (cheaper) options such as digitalocean you are out of luck.

I’ve been looking into ciborg source code thinking I could cannibalise bits that are only specifically about provisioning and configuration, rather than creating/destroying instances. I got deep down that rabbit hole before I finally discovered that there is no need to do that at all. The ciborg commands responsible for this (bootstrap and chef) already are operating on the basis that there is a server somewhere and that its address is explicitly set in the config. In my case, all I had to do was to create a droplet on digitalocean and put its ip in ciborg.yml

There a few things however that ciborg assumes about an instance:

it is ubuntu 12.04
it has ubuntu user account with sudo rights (NOPASSWD)
ubuntu user can ssh without password (hint: ssh-copy-id ubuntu@your-ci-host)

After making sure my newly created droplet satisfied those conditions, I ran ciborg bootstrap followed by ciborg chef and watched the magic going. For a long while. Once that had finished, ciborg open popped up a browser with jenkins already running my first build!

In case of a small setup of only one ci instance I reckon losing the ability to automatically create/destroy instances is not a big deal. What is left, on the other hand, is:

jenkins behind nginx secure proxy
jenkins that can pull from your github private repos
bootstrapped ci environment suitable out the box for simple stuff like rails
the multitude of recipes from travis-ci cookbooks pluggable by simply adding them to ciborg.yml

Here is the ciborg.yml (that has gotten smaller as a result of all the this):

---
ssh_port: 22
master: 12.13.14.15
recipes:
- pivotal_ci::jenkins
- pivotal_ci::limited_travis_ci_environment
- pivotal_ci
- mongodb::default # that is how easy to add recipe from travis-ci cookbooks!
cookbook_paths:
- ./chef/cookbooks/
- ./chef/travis-cookbooks/ci_environment
- ./chef/project-cookbooks
node_attributes:
  travis_build_environment:
    user: jenkins
    group: nogroup
    home: /var/lib/jenkins
  nginx:
    basic_auth_user: login
    basic_auth_password: password
  jenkins:
    builds:
    - name: master
      repository: git@github.com:shopa/shopa.git
      branch: master
      command: script/ci_build.sh

TL;DR; Use ciborg! Now anywhere!

Faster Android emulator with Android-x86 on VirtualBox

2013-05-01T00:00:00+00:00

UPDATE Setting up VirtualBox only makes sense on OSX. Lots of people suggested in comments that using KVM on Linux works just fine.

A considerable amount of yak shaving, proportional to how big or new the next task is, normally accompanies my development activities. I am very new to Android development, so a lot of yak shaving has to happen before I start producing anything useful.

This post itself is yak shaving and as such makes a good example of meta yak shaving (it is, and describes yak shaving). The described yak shaving is about speeding up android emulator, as you might have already guessed.

So. The first thing on that quest is a low hanging fruit of Virtual Machine Acceleration. Sadly, it is low rewarding too. It barely promotes emulator from being a joke to “unbearably slow”. Not good enough.

Next stop is more interesting - Android-x86. It ports Android on x86 platform so it can be installed on a PC or, more specific to our needs, on a VirtualBox VM. Let us go through the setup, as there are few hoops to be bear in mind.

Create vm with sensible defaults (Other Linux, 512 Mb RAM, HD 2 GB). Boot with the ISO that you need.

Setup port forwarding:

% VBoxManage modifyvm "VM_NAME_HERE" --natpf1 "console,tcp,,5554,,5554"    
% VBoxManage modifyvm "VM_NAME_HERE" --natpf1 "adb,tcp,,5555,,5555"

so that adb (Android Debug Bridge) can connect to vm:

% adb connect localhost
connected to localhost:5555

At this point we’ve got a fully functional Android VM that we can deploy to from Eclipse. To do that simply use a run configuration with target set to “Always prompt to pick device”.

Fix resolution because it is not mobile by default. Here is how to do it (mostly copied from here):

Add custom screen resolution:

 % VBoxManage setextradata "VM_NAME_HERE" "CustomVideoMode1" "480x800x16"

Figure out what is the ‘hex’-value for your VideoMode:
- Start the VM
- In GRUB menu enter a
- In the next screen append vga=ask and press Enter
- Find your resolution and write down/remember the ‘hex’-value for Mode column
Translate the value to decimal notation (for example 360 hex is 864 in decimal).
Go to menu.lst and modify it:
- From the GRUB menu select Debug Mode
- vi /mnt/grub/menu.lst to add vga=864 (if your ‘hex’-value is 360) to the end of kernel boot parameters. Depending on what device you want to emulate (and probably current state of Android-x86 project) you might also want to change DPI (e.g. add/change DPI=240).
reboot -f

Finally, to enable proper mouse support, in VM settings (System tab) uncheck “Enable absolute pointing device”.

That is it. Fast emulator, comparable to that of iOS. If only Google had something like that in their sdk. But hey, that would mean what, no yak shaving? Unacceptable.

Dependency lookup with binding_of_caller

2013-03-30T00:00:00+00:00

UPDATE Turns out binding_of_caller is not recommended for production apps

Rails project, RetailersController, products action, circa 2013 AD:

def products
  @products = current_retailer.
    products.order('created_at desc').
    paginate(page: params[:page], per_page: 20)
end

That query is not too bad. I can totally live with it. But I’ve been roaming around this controller for the last few days and every time I am passing by this piece of code it just does not make the journey better. The effect is somewhat enhanced by three other actions that look identical to this one (except they query for other things). So today I finally stopped and decided to sort it out like a pro! Well. That, and I had to to put some tests around those actions and stubbing out a query such as the one above is just so much the opposite of fun… But that would be a boring start for a post.

Despite Rails community actively exploring The Right Way™ to deal with the logic that does not fit into Rails MVC, the most common thing amongst those who just need to Get Shit Done™ is to move the query logic into model. Ok. Second most common. The first one admittedly is to leave it as is and get some shit done instead. However, I do not belong to that category of people, since I am writing this post instead of getting shit done. And neither are you, since you are reading this post instead of getting shit done. So let us disregard those lucky bastards and see what is down that rabbit hole.

Ok, moving the query into the model is not cool. So what is? I don’t know! And when I don’t know, I tend to start with making stuff up. What would an ideal client code look like? It would be nice to have something that is easy to stub out in controller test. Also, it would make it more readable to see where the products are coming from. And certainly none of the pagination/sorting machinery should make it through. Something like this:

def products
  @products = PrepareForTableView.relation(current_retailer.products)
end

This way retailer is still aware of its products but on the other hand controller does not need to know how to sort and paginate. Also, it looks like a decorator. Or presenter? Whatever. Design patterns - check.

The slightly annoying problem however is that it is not going to work. At least not without also passing params[:page]. Ok. How does it look now?

@products = PrepareForTableView.relation(current_retailer.products, page: params[:page])

Acceptable. But not as good as before. What can be done?

Conventions work great for Rails. So let us introduce one. The one that postulates that PrepareForTableView is only ever going to be used in a controller. So what? Well that means that its methods will always be called from the context of a controller. If only we could get hold of that context. Oh, wait. Isn’t it what ruby Binding is for?

Quick recap:

Objects of class Binding encapsulate the execution context at some particular place in the code and retain this context for future use. The variables, methods, value of self, and possibly an iterator block that can be accessed in this context are all retained.

Ok, but how is this useful? The profit comes from the fact that ruby can eval stuff in specified context (binding). Like this:

page = controller_context.eval("params[:page]")

So all that is needed for our case is access to binding of caller. binding_of_caller gem does just that (and does it well):

module PrepareForTableView
  module_function

  def relation relation
    page = binding.of_caller(1).eval("params[:page]")
    relation.order('created_at desc').paginate(page: page, per_page: 20)
  end
end

Good.

Now let us finally get some shit done and allow retailer to download products in xls. xls version does not require sorting. Also our client is specific about having more than one page worth of data in there. Having controller context at hand gives our module a lot of flexibility so the above can be achieved without even changing client code:

# renamed as it is no longer just about table view
module WithinRequestContext
  module_function

  def adjust_data relation
    request, params = binding.of_caller(1).eval('[request, params]')

    if request.format == Mime::HTML
      relation.order('created_at desc').paginate(page: params[:page], per_page: 20)
    else
      relation
    end
  end
end

The client code remains the same:

def products
  @products = WithinRequestContext.adjust_data(current_retailer.products)
end

Last, but not least. Unit testing this module is simple. The test just needs to have params and request defined:

describe WithinRequestContext do
      let(:request) { stub('request') }
      let(:params) { {page: 1} }
      let(:relation) { stub('relation') }

      before do
         # relation follows builder pattern, so should the stub
         relation.stub(:paginate).and_return(relation)
         relation.stub(:order).and_return(relation)
      end

      describe "#adjust_data" do
        context "Non HTML request" do
          before do
            request.stub(:format).and_return('some mime type')
          end

          it "returns unmodified relation" do
            relation.should_not_receive(:order)
            relation.should_not_receive(:paginate)

            WithinRequestContext.adjust_data(relation).should == relation
          end
        end

        context "HTML request" do
          before do
            request.stub(:format).and_return(Mime::HTML)
          end

          it "paginates" do
            relation.should_receive(:paginate).with(page: 1, per_page: 20)
            WithinRequestContext.adjust_data(relation).should == relation
          end

          it "orders by created_at desc" do
            relation.should_receive(:order).with('created_at desc')
            WithinRequestContext.adjust_data(relation).should == relation
          end
        end
      end
    end

That is all I’ve got on dependency lookup with binding_of_caller - an awesome little gem used by pry-stack_explorer and better_errors.

It is worth mentioning that it is all fun and games until someone gets to maintain it. I’ll wait till then before start recommending the above approach… But you don’t have to!

Refactoring in command line and vim

2013-02-14T00:00:00+00:00

Refactoring in dynamic languages may seem like a hassle. And it is! There is no IDE to perform the renaming magic, it is all a bit manual, hence error prone and scary. The good news is, the command line almighty is on your side. The not so good news is, its capabilities are so vast that it is often difficult to remember how to do X, Y and Z required to solve a particular problem.

I am working on a rails project at the moment. Every now and then there is a ‘domain object’ (model, controller, views, tests, etc.) that needs to be renamed. It is a good example of refactoring so I figured it deserves a little coverage.

So, to business. There is model called OfflineDeal that needs to be turned into InstoreDeal.

renaming files

Let us find all the files with offline_deal in their name:

% find . -name "*offline_deal*" 
./app/assets/stylesheets/retailer/offline_deals.css.less
./app/controllers/offline_deals_controller.rb
./app/models/offline_deal.rb
./app/views/offline_deals
./db/migrate/20130117121729_create_retailer_offline_deals.rb
./db/migrate/20130118164309_change_vouchers_deal_id_to_offline_deal_id.rb
./db/migrate/20130121150012_add_referrer_reward_to_offline_deals.rb
./db/migrate/20130130172326_add_fine_print_to_offline_deals.rb
./db/migrate/20130131122749_rename_offline_deals_description_to_voucher_text.rb
./spec/controllers/offline_deals_controller_spec.rb
./spec/factories/offline_deals.rb
./spec/models/offline_deal_spec.rb
./spec/routing/offline_deals_routing_spec.rb

And pipe the result of find command into vim:

% find . -name "*offline_deal*" | vim -

Notice that not all of these files require renaming. Migration files reflect change history and should therefor stay untouched. Let us get rid of the lines with those file names:

:g/migrate/d

and change the remaining ones into rename command:

:%s/\(.*\)offline_deal\(.*\)/git mv & \1instore_deal\2

Each line is now turned into git mv command:

...
git mv ./app/models/offline_deal.rb ./app/models/instore_deal.rb
...

And here is the magic part. Run them all:

:w !sh

Voila, files are renamed.

file changes

Inside project files there are 3 things to change: method references, symbol references and class name references.

The first two (including dynamic methods such as path helpers or AR finders) are covered with replacing all occurrences of offline_deal with instore_deal. Let us do it first.

Find all matching files and open them in vim:

% vim $(ag offline_deal -l app lib spec features config)

I am using ag - a faster alternative to ack (a popular alternative to grep). -l option tells it to only print names of files with matches, not the matches themselves (as it does by default). That list of file names is passed to vim so they are all opened in buffers.

Now for each opened buffer let us perform the replace and save the changes:

:bufdo %s/offline_deal/instore_deal/g | update

To rename class names, we repeat the above steps with OfflineDeal -> InstoreDeal.

db changes (rails specific)

This part has got nothing to do with command line refactoring and it is only here for the sake of completeness.

We rename db table offline_deals to instore_deals in a migration. There are other tables referencing it, so corresponding foreign keys and index columns should also be changed.

run tests

No tests? Time to panic!

Better Shopify app integration testing

2012-12-22T00:00:00+00:00

Part of the system I am currently working on is to act as Shopify application. That is to be able to interact with its API.

I have an integration test (cucumber/capybara) that uses real Shopify retailer site, however, that approach has its obvious drawbacks: such tests are slower and they fail every now and then because of network issues. It is a good start while working on a feature, but it is just not good enough to have it in a an everyday test suite.

On the other hand, a test that uses real site is closer to reality. So, ideally, I want to keep both options, but only run the latter one occasionally to check higher lever integration.

My test scenario consists of three steps: application installation, buying something on retailer site and querying api for order details.

Below I am going to cover how to make all three not depend on actual shopify site, but still be able to dynamically switch back to that path.

install application

Shopify is using OAuth2 to authenticate and grant permissions to the application (that is, install). If your app is a rails one, then you must be using shopify_api gem which in turn is using omniauth-shopify-oauth2 under the hood. But you probably know that anyway.

Lesser known that omniauth has a pretty elegant solution for integration testing. All you have to do is to stick the following somewhere in env.rb:

OmniAuth.config.test_mode = true
OmniAuth.config.add_mock :shopify, {
  provider: 'shopify',
  credentials: {
    token: 'test_token_blah'
  }
}

From their docs:

Once you have enabled test mode, all requests to OmniAuth will be short circuited to use the mock authentication hash as described below. A request to /auth/provider will redirect immediately to /auth/provider/callback

So now in the step definition we need to distinguish between authenticating locally or against real shopify. Assuming there is link to install app on a current page, here is helper that does the job:

def install_app
  click_link 'install app'

  if not $http_stubbed # then 'install app' link will bring us to real shopify site admin area
    fill_in 'Email Address', with: 'real_admin_user@example.com'
    fill_in 'Password', with: 'real_password'
    click_button 'Sign In'
    click_button 'Install'
  end
end

$http_stubbed is set via passing an environment variable when running cucumber:

# env.rb
$http_stubbed = ENV['UNSTUB_HTTP'] != 'true'

buying a product

In my application, when user buys something on shopify store, some custom javascript (added to shopify success page) calls back home to trigger order processing. The system does not care if it is a real shopify success page. So all we need is a page that, when visited, runs that script. Having done that, let us introduce a helper that will conditionally buy either real product or a fake local one:

def buy_banana
  if $http_stubbed
    # test route that leads to our fake success page
    visit '/test_retailer/banana_success'
  else
    visit 'http://real-store.myshopify.com/products/banana'    
    add_current_product_to_cart
    purchase_contents_of_cart
  end
end

## querying Shopify API for order details

Once the order is purchased, our system goes away and queries the Shopify API to get order details and do some mumbo jumbo with it. Since the goal is to replace real Shopify, we need to stub out those interactions, but in such way that our system never notices the difference.

Common approach to stub HTTP interactions is to use VCR gem. It records low level HTTP interactions, stores them in files and allows to replay them later in tests.

On the surface that seems strait forward, but doing it right™ ended up being the most confusing and difficult part of the whole setup. Let us cover things I had troubles with bit by bit.

ignore unwanted requests

Once in playback mode, VCR will blow up on any request it does not know about. This is totally sensible, but may (and will) cause confusion at first. Good news is that VCR can be configured to ignore certain requests. Here is an example from my setup:

VCR.configure do |c|
  ...
  c.ignore_localhost = true
  c.ignore_request do |request|
    uri = URI(request.uri)
    # this is just an example obviously
    uri.host =~ /(?<!vasily-on-shopify.)localtest\.me|www.google-analytics.com/
  end
  ...

### binary headers

By default, all headers in VCR fixtures (this is only in case of Shopify though) will be in binary representation. That is not always handy, especially if you need to dynamically change some values in there. This problem is described in more details here. And here is the workaround:

VCR.configure do |c|
  ...
  c.default_cassette_options = {
    ...
    serialize_with: :syck, # so that headers are human readable
    ...
  }
  ...

### binary response body

The above trick does not solve another binary related problem: response body. That is because it is genuine binary data and it took me quite a while to figure out exactly what the heck this binary represents. Turns out the response content is gzipped.

Why is this a problem? It becomes one the second you need to dynamically modify response data. In my case, order created timestamp has to be changed so that order passes some validation inside my app. It is easy to find created_at in human readable response json and change its value to <%= Time.now %>. The same is obviously impossible with binary data.

So here how I deal with this:

VCR.configure do |c|
  ...
  c.before_record do |interaction|
    require 'zlib'
    require 'stringio'

    if interaction.request.headers['user-agent'].first =~ /ShopifyAPI/
      content_encoding_header = interaction.response.headers['content-encoding'].first
      content                 = interaction.response.body

      interaction.response.body = if content_encoding_header == 'gzip'
                                    interaction.response.headers.delete 'content-encoding'
                                    Zlib::GzipReader.new(StringIO.new(content)).read
                                  else
                                    content
                                  end
    end
  end
  ...

The above code unzips content before cassette is recorded.

rerecord cassettes

I want to be able to rerecord cassettes every now and then to keep up with the reality. One problem here is that it erases all manual modifications made to VCR fixtures (e.g, created_at ERBfication).

The solution here is to employ c.before_record hook more heavily. It might end up in more code, you’re going to need to parse json, change it and serialize back, but the result is worth it. Here an example:

VCR.configure do |c|
  ...
  c.before_record do |interaction|
    # change created_at to Time.now so it passes timebox check
    if interaction.request['uri'] =~ /orders.*\.json/
      order = JSON.parse(interaction.response.body)
      if order['order'] # means if not 401
        order['order']['created_at'] = '<%= Time.now %>'
        interaction.response.body = JSON.dump(order)
      end
    end
  end
  ...

Also, when rerecording, do not use VCR record mode :all as it only overwrites the same requests and adds new ones, but does not wipe out the old unmatched ones. Simply remove fixture file instead.

And that is about it! Simple, isn’t it? But jokes apart, I believe fiddling with tests at such level is an investment in understanding your system better which in turn allows to find bugs earlier and fix them while in comfort zone (that is, while the context is in your head and the code is not in production)

SlowDown.vim

2012-11-12T00:00:00+00:00

I am a vim user.

As such, I take continuously enhancing my vim-fu seriously, to keep impressing myself and, of course, others too. It is just fun. And not much beyond. Personally I don’t understand the argument for learning vim ‘because it makes you more productive’. I mean, things do get much faster in vim by all means, but if you are after productivity, mastering a text editor just does not seem to be the top priority. I don’t think vim can (or should) be sold this way these days. Imagine Yoda saying ‘Use the Force Luke - it will make you more productive’. They would have never made it past episode IV!

Learning other languages, reading other people’s code and so on - that is what going to make you more productive.

However. It wasn’t always like that. There were times when mastering vim moves was the matter of life and death. A lot of what I largely consider as a way to show off (perhaps even recursive macros!) was once a prerequisite to getting things done. It was in the era of Slowness. Slow terminals. Or teletypes. Or God knows what else, but the point is that a war could have started and finished before user input would show up as output. No wonder people were getting far more advanced in far shorter time than I am today.

This first occurred to me once when I got stuck editing a perl script over ssh tunnel through two (or three?) servers. Don’t ask me how I got into this situation. Sometimes shit like that just happens. So, things have gotten really slow and by the end of day two I found myself using moves I never even thought I knew. That got me thinking. What if there were some sort of a way to slow my everyday vim down? That would naturally force me to trade the amount of keystrokes to their more efficient vim alternatives.

And that is how SlowDown.vim was born. Perhaps the only vim plugin on the planet that (deliberately) makes user experience worse. And possibly the smallest one too.

Check it out. It is fun!

Special thanks to Adrian Longley for proof reading.

Stub like a surgeon, spy like James Bond with REST-assured

2012-05-27T00:00:00+00:00

This is the story of REST-assured, a testing tool for isolating automated integration tests from external HTTP-based dependencies. Unlike alternatives that I know of (most notably VCR), it does not stub out the underlying HTTP client library, but rather is an actual service that is configurable to respond in arbitrary ways. As an immediate result it is language agnostic, supports spying and… Let me tell you the whole story first.

I work in BBC Future Media. The problem of testing systems talking to each other via HTTP is extremely relevant here. There are a countless number of services that provide more or less RESTful APIs and in turn consume from others. None of the products (that I know of) are standalone.

When it comes to integration testing (in a particular project) we are forced to think hard how to represent those dependencies to our application. It would be nice of course to simply spin up a virtual machine with the services needed next to our code, but that sadly is rarely an option, not the least because some of those services are in development themselves.

Lots of tools adressing this problem arose in such a highly ‘fertilized’ environment. Some are hidden in corporate SVN repos, and some have seen the light of day (rack-stubs, anmo). REST-assured is one of a kind that eventually grew into something more generic and hopefully can be helpful out there in the wild.

When I first faced this problem, I thought for a bit and started using hand-crafted fixtures which were served directly from tests. As the number of tests grew, the focus shifted from creating new fixtures towards reusing existing ones. Further down the line, making them configurable started to sound like a good idea. Then one day it became apparent that it is quite hard to figure out their state during non-trivial tests. And so it went on and on, presenting new challenges.

And it was for that reason (not because programmers love to create frameworks!), it didn’t take too long till I found myself pulling all that stuff out into a framework. Well, OK, not a framework strictly speaking, but something along those lines.

My current project here is a ‘control panel’ type of thing that sits between multiple external systems, presents their data in the browser, issues control requests to some of them based on user actions, delivers responses back to the user and updates the state of things in others.

We keep development within the BDD cycle. Every feature starts with automated acceptance test. For this we are using Cucumber with ruby. Once we get the meaningful failure out it, we go down to Javascript unit tests (using Jasmine) and when those pass, end up on Java layer where JUnit and Mockito help us to test drive the back-end.

On a side note. Nowadays there is a nice alternative for Java projects - cucumber-jvm - so that everything can be kept within the same project, language, libraries, etc. It looks very impressive, but on the other hand I kind of appreciate the fact that there is a ten foot wall between production code and integration tests. We are forced to treat the system as a black box.

With a few exceptions, every scenario in our suite has a Given step that implies data lookup in external systems. And many of our Then steps verify that certain requests have happened in a particular manner. So being able to stub/spy on HTTP interactions becomes a prerequisite to being able to test anything at all.

Below I am going to cover few typical scenarios and how REST-assured comes at rescue. There will be ruby code (just because there is ruby client library), but it isn’t limited to ruby and can be controlled from within any environment capable of making http requests (e.g, curl will do). Or even through web interface.

Let us summon an imaginary client who wants an application that shows most popular tweets for a search criteria. He anticipates that users aren’t generally strong on command line and also won’t be bothered installing a native client. So he wants a webapp. All right. We unfold our cucumber communicator and agree to start with the following simplest feature:

Given there are no popular tweets about "X"
When I search for popular tweets about "X"
Then I should see that there are no popular tweets about "X"

I am not going to go into details of setting up a Sinatra and Cucumber project (check out the demo project here). Except for one thing which is relevant to what we are talking about. To be able to stub Twitter’s search API, let us inject its host as a dependency (so to speak):

# tweet_checker.rb
TWITTER_HOST = ENV['TWITTER_HOST'] || 'http://search.twitter.com'

and configure it to point to the REST-assured instance in tests before loading application code:

# env.rb
ENV['TWITTER_HOST'] ||= 'http://localhost:4578'
require_relative '../../tweet_checker'

Now let us find out what real Twitter returns when nothing is found:

% curl 'http://search.twitter.com/search.json?q=asdfsdfrefsdfsdgs&result_type=popular'
{"completed_in":0.0070,"max_id":0,"max_id_str":"0","page":1,"query":"asdfsdfrefsdfsdgs",
"results":[],"results_per_page":15,"since_id":0,"since_id_str":"0"}

And use the result to craft response in the Given step:

Given /^there are no popular tweets about "([^"])*"$/ do |query|
  RestAssured::Double.create
    fullpath: "/search.json?q=#{query}&result_type=popular",
    content: %({"completed_in":0.0070,"max_id":0,"max_id_str":"0","page":1,"query":"#{query}",
      "results":[],"results_per_page":15,"since_id":0,"since_id_str":"0"})
end

The above means that a GET request to http://localhost:4578/search.json?q=X&result_type=popular will return the JSON for an empty result set. And when our app (driven from When step) hits this url, it will appear to it as if there are no tweets. So that Then step can observe the result page and check for the expected message about no popular tweets.

Next, we want to tackle the case where twitter returns something. Here goes the second feature:

Given the following tweets are most popular mentioning "Y":
  | from_user | text                                 |
  | bob       | check it out, my tweet gets recuked! |
  | alice     | Yo dog, your cuke makes me puke      |
When I search for popular tweets mentioning "Y"
Then I should see those tweets

To be fair, the above scenario should not expose any test data. But since I want to demonstrate usage of more complex fixtures and I couldn’t think of a better example, we’ll stick to this one.

So, again, we take a sample of a real response - only this time, since it is quite a big chunk of JSON, let us put it in a separate file. We also want to substitute bits of it with test data, so let us make an ERB template out of it.

{
  "results": [
    <% tweets.each do |t| %>
    {
      "profile_image_url": "http:\/\twimg.com\/profile_images\/u.jpg",
      "from_user_id_str": "149752077",
      "created_at":"Sat, 05 May 2012 21:37:12 +0000",
      "id_str": "116001342874595328",
      "from_user": "<%= t[:from_user] %>",
      "to_user_id": null,
      "text": "<%= t[:text] %>",
      "metadata": {
      ...

Given /^the following tweets are most popular mentioning "([^""]*)":$/ do |query, table|
  @expected_tweets = table.hashes.clone
  template = File.read(File.expand_path('../tweets.json.erb', __FILE__))
  tweets_json = Erubis::Eruby.new(template).result(tweets: @expected_tweets)

  RestAssured::Double.create
    fullpath: "/search.json?q=#{query}&result_type=popular",
    content: tweets_json
end

The verification step is going collect tweets shown on a page and compare them with those saved in @expected_tweets:

Then /^I should see those tweets$/ do
  actual_tweets = all('#tweets tbody tr').map do |row|
    { 'from_user' => row.find('.from_user').text, 'text' => row.find('.text').text }
  end
  actual_tweets.should =~ @expected_tweets
end

It is worth mentioning that our Given step is already a bit too complicated for a step definition. All the template mangling and creation of a double should be hidden in a domain object. Something like, PopularTweetsSearchResult.new(tweets: @expected_tweets). And with time, as the number of constructor options grows, that domain object creation can in turn be delegated to Factory Girl (or the like). This might sound like overkill, but the truth is that Cucumber is not particularly great at organising steps, so passing state around in conjunction with step reusability pushes complexity through the roof before you know it. So, ruthlessly moving whatever you can away from that context is not a bad idea.

Ok. The project goes on and the day comes when our client shows their true face as the following feature lands on our table:

Given there are popular tweets about "Z"
When I search for popular tweets about "Z"
Then the record should be sent to The Vegan Police Intelligence Service

Well, money has no smell. Besides, we already switched to use Factory Girl, so there is no turning back anyway.

Given /^there are popular tweets about "([^"]*)"$/ do |query|
  # factory takes care of creating double with sensible default tweets
  @search = build(:popular_tweets_search_result, query: query)

  @create_record_double = RestAssured::Double.create 
  fullpath: "/api/tweet_search_records",
    verb: 'POST',
    status: 201
end

@create_record_double is going to match any POST to http://localhost:4578/api/tweet_search_records. It is not going to return any content, just an HTTP 201 repsonse. Ideally it should also be hidden inside its own domain object, but for demonstration purposes let us leave it as is. Much like the Twitter search API, The Vegan Police Intelligence Service host address also needs to be changed to point to the REST-assured instance.

Our Then step is going to analyse what requests have hit @create_record_double (if any) and verify that they match our expectations.

Then /^the record should be sent to The Vegan Police Intelligence Service$/ do
  @create_record_double.wait_for_requests(1) # will raise exception if that does not happen
  req = @create_record_double.requests.first

  # verify that payload contains search query and user
  req.body.should =~ @search.query
  req.body.should =~ @search.from_user
end

The above looks a bit messy. Rspec custom matchers help a great deal with tidying up complex verifications such as this one. I am not going to go into the details of defining one, but in the end it should look like the following:

Then /^the record should be sent to The Vegan Police Intelligence Service$/ do
  @create_record_double.should have_been_requested
    .with_query(@search.query)
    .with_user(@search.from_user)
end

Years have passed since we commissioned our little project. The silver has been spent big time. And millions of people have been brainwashed to absolutely love it. But one day we got an email from a beloved customer to investigate an issue. The problem seems to be that no records are being posted to The Vegan Police Intelligence Service any more.

Say no more. We immediately check the CI build status and observe that everything is happily passing. Meaning that, as far as our tests are concerned, records are being sent. After hours of collective head scratching, we finally go back and read the rest of their smug report where they suggest, amongst other obviously irrelevant things, that it could be due to recent API changes. Blimey! Turns out the new version changed the records API. The easiest fix in our case would be to stick /v1 in the POST URL in order to use the old version. OK. First we change our test double and make sure of the correct failure, then change the implementation. And looks like we are done, right? Not quite so. The bug is fixed but the underlying problem is not. Fixtures will still eventually go out of date. We need a way to validate our assumptions (that is what fixtures are) against something more real.

Unfortunately, it is difficult to address this problem in general (we tried), as the definition of valid is very much context dependant. So even though REST-assured creates this problem, it doesn’t provide any built-in solutions (so typical!) and your tests are probably the best place to host such checks. Nonetheless, forewarned is forearmed and hopefully this is not going to ruin your day. One piece of advice here: consider using tools like VCR to cache HTTP interactions or else things will get REALLY slow.

Speaking of slow, REST-assured adds quite a bit of an overhead to test startup, as it spins up a separate service stuffed with ActiveRecord, ActiveResource and some other things you don’t want to know about. Two options here: consider using spork or start it separately from tests. Both options work great in development (e.g. when you want lots of re-runs) and also give you the benefit of being able to inspect doubles via a web interface after the tests have finished.

Wrap up time. Let us recap what REST-assured is and what it is useful for. REST-assured is a locally deployable service that you can configure at runtime to respond to arbitrary HTTP requests. As such, it can be used to serve test fixtures to an application under test. It also records all requests made to it, so that more control-type requests can also be verified.

So that is it for the introduction of REST-assured. You can give it a try right here right now. Have a nice day. And don’t let them catch you.

Special thanks go to Sam Starling for proof reading.