Musings of a Software Development Manager

6 Months with Vim

Ed Gibbs — Mon, 14 May 2012 13:49:58 +0000

College Roomate: So you know vi?
Me: Of course, that would be command shift-Z-Z

This is a conversation I had about twenty years ago. For most of my Unix editing I got by with pico and I found the modal nature of vi downright frustrating. Hence the importance of knowing the command to escape from modal hell.

At my current company vim was the default text editor, I even paired in the interview on vim. Over the years I’ve bounced between text editors and IDEs depending on the languages and environments. One key point though was that when I reached for a text editor it was always one with a GUI front end like BBEdit or TextMate. Despite working in vim most of the day I still default to MarsEdit for writing blog posts.

I jumped into vim with some lingering fear remembering my dislike of its modal nature and the lack of any mouse support. I tried out VimGolf and spent some time trying to memorize basic navigation and selection commands. It came pretty slowly at first. I bogged down my pair quite a bit fumbling around with arrow keys or splitting the screen the wrong way. About two months in I had developed some basic proficiency, but I found myself wondering if RubyMine or TextMate wouldn’t be a better option.

At about the 6 month mark I looked back and realized I was finally comfortable in vim as I found myself defaulting to it for most of my text editing even when I wasn’t working on code. I still have a few cases where I prefer using a visual editor or IDE for tasks like:

blogging
email
exploring large projects
larger refactorings

Looking back there were several items that accelerated my vim mastery. One was our switch to sharing a single base .vimrc so when you pairing with anyone you had the same configuration of the editor. It also included several features like macros for commenting, formatting, auto-complete, and running inline RSpec. After setting up that .vimrc I saw how vim could be a more powerful tool.

Another big step was learning to love ‘hjkl’ over the arrow keys. That took about a week or so to break the habit and it was painful, but the speed difference was worth it. The mouse is already gone, so might as well toss the throwback to the arrow keys. (I found the hack here on Stack Overflow.)

A critical change that I arrived at fairly late was remapping my caps lock. Normally you rarely touch caps lock, but it is in a very convent position especially compared to the escape key. I remapped my caps lock to tap it to get an escape and holding it down yields a control which are the two most commonly used modifier keys in vim. That made my fingers very happy.

Finally, the biggest learning expeditor was pairing with a number of developers who had more expertise than myself. Since the team had standardized on vim before I arrived they had all gone far past where I was on the learning curve and were able to quickly demonstrate how to use vim or which keystroke to use as I fumbled along. I greatly appreciated their patience and willingness to let me bang something out that they could have typed in half the time.

I still don’t feel anything like a vim expert, but I do feel comfortable in a modal editor. Never thought I would be typing those words all these years later.

Jenkins: My Personal Bodyguard

Ed Gibbs — Wed, 09 May 2012 13:29:45 +0000

I’ve been running Jenkins as my CI solution for years now, but for the first 6 months at my new job I used it in an entirely new way. The typical CI pattern is you setup to run against checkins to your master branch run all the unit tests, and depending on how sophisticated you are integration tests, static analysis, or even performance tests.

In my current shop we had a standard CI build using CC.rb against checkins to our master branch. It ran about 5000+ specs sequentially and if everything’s green master is available for production pushes. That fulfills the standard case for CI just fine, but I quickly found that it a little light for my needs. The issue is we relied on feature branches for rolling out all of our stories and while those stories are being worked there’s no CI job at all. Another pain point is even with parallel specs and a nice 4-8 core processor the full test run takes 5-10 minutes so it doesn’t get run nearly as often as one might like.

I’d let Jenkins be my default choice for CI in Javaland, but I was really open to switching it out for a more Ruby style CI server. I figured the first place to look was our already running CC.rb implementation. I tried out CC.rb long ago and remembered liking it a lot better than the original cruise control, but with years of development on it I figured it had evolved. Turns out CC.rb is a pretty basic solution for CI and it hasn’t had much love over the years. It runs a build showing the console output and a very basic web console. Coming from Jenkins the lack of features and plugins was pretty shocking.

Given the Ruby community’s love for XP practices there must be a better option available. I looked at numerous CI servers for Ruby including CI Joe, Integrity, and Travis. All of them followed the lines of providing a very simple CI server much like CC.rb. Travis CI appeared to be a bit more with its use of a hosted open source solution, but it is completely tied to hosting your project on Github. None of these projects showed the depth of a solution like Jenkins.

So the solution was to run Jenkins as my own personal bodyguard. We were looking at swapping out the CC.rb implementation, but it wasn’t a priority right away. So I dived in and setup Jenkins to run a build on my laptop. Instead of just building master I added whatever git feature branch I was working on and hooked up an audible trigger to play James Brown “I feel good!” for successes and “Houston I think we have a problem” for failures. I expected to use the rake plugin to launch the build, but it turned out it was much easier with using things like rvm and bundler to simply execute a script. My final scripts for the build looked something like the following:

1
2
3
4
5

bash -l -c "rvm use 1.8.7-p330@acme"
bash -l -c "bundle install"
bash -l -c "rake db:migrate"
bash -l -c "rake parallel: prepare"
bash -l -c "rake parallel: spec"

So finishing up a bit of functionality, I’d commit and about 10 minutes later my laptop would exclaim “I feel good”, or occasionally “Houston I think we have a problem” to let me know I missed something. So I had my own personal bodyguard build and I could happily let it run in the background.

This year we replaced CC.rb with Jenkins and dropped feature branches so I’ve retired my laptop Jenkins, but I recommend the practice if you find yourself in a similar spot.

Stubbing Partials with RSpec in Rails View Tests

Ed Gibbs — Sat, 19 Nov 2011 22:59:51 +0000

Working with my pair yesterday we ran into an issue testing a view that pulled in several partials. In the interest of making progress we punted after about half an hour of trying to setup expectations on the partials and just tested the negative cases where we didn’t have to worry about the partials being called. Intellectual curiosity and not wanting to leave the views lightly tested I dived into seeing how we could effectively test that the partials were called as expected.

Say you have a view like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<div id="content">
<div id="page_title" style="padding-bottom: 20px">
<h2>Booksh2>
div>
< %= render :partial => 'tabs' %>

< % if @books.empty? -%>
<em>No Booksem><em>
< % else %>
< %= pagination_links = render :partial => 'shared/pagination', :object => @books %>
< %= render :partial => "book_rows", :locals => {:books => @books} %>
< %= pagination_links %>
< % end %>
em>div>

You want to test the conditional to see that the partials to display the list of books work correctly. You don’t want to worry about concerns like setting up the books collection, and you might even want to stub out the call to the partial that displays the tabs a the top of the page.

The Rspec example group for this looks like the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

require 'spec_helper'

describe "books.rhtml" do
before(:each) do
assigns[:books] = []
end

describe "with no books" do
it "should display 'No Books' message" do
render
response.should have_text(/No Books/)
end
end

describe "with books" do
before(:each) do
assigns[:books] = [Factory.build(:book)]
template.stub(:render).with(hash_including(:partial => "tabs"))
template.stub(:render).with(hash_including(:partial => "shared/pagination"))
end

it "should render book rows" do
template.should_receive(:render).with(hash_including(:partial => "book_rows"))
render
end
end
end

The key find was to use with(:hash_including()) which I found from a helpful link at Stack Overflow.

Breaking the Build at the New Job

Ed Gibbs — Sun, 18 Sep 2011 03:43:55 +0000

My first break of the CI build on a new team came about two weeks into starting the new job. We had made a small change to a dynamically created URL on a single page of the application. Normally this would fall into the category of “too simple to test” for me. My pairing partner pointed out when we started the story that we should put a test in for the change. I argued that changing a URL was too simple a change to bother writing a test for. He disagreed, but let me go ahead after some further discussion. After checking in the feature, about an hour later a developer on the other side of the room piped up with “Someone broke the build.” My pairing partner smiled and said, “I guess we didn’t need to test that.”

It turned out there was another test I didn’t realize had a dependency on that same link and by changing it we had broken an assertion on the specific URL. A bit embarrassed I learned an important lesson. My assumptions about “too simple to break” needed some adjusting.

Over the course of several weeks I came to appreciate that many things were worth testing. A particularly revealing example was testing a new javascript dialog. Javascript is always a pain point when it comes to unit testing, and I’ve often let that coverage of javascript functionality be covered by functional tests usually with Selenium. My pairing partner walked me through a complete TDD approach asserting every part of the function we created and the various DOM elements we used to create it. I did ask a few questions about whether we really needed to test that we had created a DOM element in some exact order at points, but he was firm about testing in every piece. By the end of the feature I had a new appreciation for doing very low level unit testing.

I love that I’m being challenged on not writing enough tests. For so long in past shops I was the hardcore TDD advocate constantly questioned about the real need for testing some component. Now I’m learning to test in even very small changes.

Cardboard Boxes and Modern Web Frameworks

Ed Gibbs — Sun, 07 Aug 2011 14:29:24 +0000

My daughter spent an hour the other day cutting holes, drawing red bricks, and pasting grass along the bottom of a simple cardboard box. It’s a common story among parents, especially of small children that despite spending a lot of money on a gift, the kid ends up enjoying the box and ignoring the toy.

I’ve noticed a common quality of many newer web frameworks is that they provide you with a nice box to build from. Typically:

A default structure so you know where to find the models, views, controllers, extra libraries, etc.
A command line for generating stubs, running tests, and deploying.
A plugin system for easily adding functionality from swapping out javascript libraries to adding a security system.

Though there are probably others I’m aware of this model being used by Spring Roo, the Play Framework, Gryphon, Grails, and Rails. I don’t claim to be a historian on this, but Rails was my first experience with this style and I assume to be the originator of the approach.

The benefits of this approach are obvious from the first time you start off with a Hello World tutorial. For one you generally have a single download for the framework. After unpacking you’re able to use the command line to generate your Hello World controller, find the view in a predefined location, add the Hello World line, and fire up the application from the command line. Passing the 5 minute test is pretty important if you expect developers to give your framework a chance.

As a consultant working on numerous legacy code projects, there’s always the groan moment when you start looking into the code and you realize its non-obvious where to find things. I’ve seen libraries sprinkled about at random, key configuration files that were supposed to be moved into the users home directory, model classes mixed in with controllers, and a host of other inconsistencies. With a framework that reinforces the default structures it becomes easy to find things and much easier for plugin-authors to write plug-ins that make the framework much more valuable.

Finally, the plugin system is a critical part of new frameworks. Being able to add security framework on the fly, swap out your testing framework, or simply add in a nicer date library really starts to make things feel like magic. Indeed among these frameworks they are starting to push more and more modularity into the plugin systems to allow for the framework to evolve better over time.

I hope this trend continues in the world of web frameworks, as I really like a nice box to start a project from.

Mock With Spock

Ed Gibbs — Tue, 07 Jun 2011 03:38:16 +0000

My default rule with mocking is to try to stick to stubs where possible. I don’t enjoy having to setup and verify interactions with mocks, but sometimes you have some code where that’s exactly what you need to do. I’ve used many frameworks in Java over the years from EasyMock to Mockito, but I was quite happy with how easy it was to do in Spock. I recently found myself having to build a test harness around some legacy code. The real world code was more involved, but it looked something like this:

1
2
3
4
5
6
7
8
9
10
11

public void addDefaultQuestions(Category category) {
if (categoryDao.getCategory(category.getId()).getQuestions().isEmpty()) {
for (Question question : category.getQuestions()) {
if (question.isDefaultQuestion()) {
categoryDao.addQuestion(question, true);
} else {
categoryDao.addQuestion(question, false);
}
}
}
}

The method is taking a category, checking if any questions related to the category exist in the database and then saving all the questions with a valid flag set to true or false. Not unusual in a typical corporate application, but I want to test two things:

I can add new questions to the database with the proper valid flag.
If the database category already has some questions then do nothing.

After walking through the Spock mocking documentation I had a pretty good sense of the approach. In Spock it’s referred to as interactions, but it doesn’t follow the typical expect-run-verify pattern. You just verify what you need to if you need to. And given a choice I prefer not to have to verify the mock at all.

With this code I needed to mock the categoryDao which used straight JDBC and made calls to the real database. That meant I needed a way to verify that the questions were added correctly with calls to the categoryDao. Hence I needed the power of an actual mock and not just a stub class.

The first test would show that I could save new questions in a category to the database:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

def "should only insert new default questions"() {
given:
def question1 = new Question(defaultQuestion: true)
def question2 = new Question(defaultQuestion: false)
def category = new Category(questions: [question1, question2])

CategoryDao dao = Mock()
dao.getCategory(_) >> new Category()

CategoryService service = new CategoryService()
service.setCategoryDao(dao)

when:
service.addDefaultQuestions(category)

then:
1 * dao.addQuestion(_, true)
1 * dao.addQuestion(_, false)

}

So the steps are:

Setup a Category object with two questions.
Create a mock dao.
Define a method and its default return value on the mock DAO.
1. We define arguments to getCategory() with a wildcard operator the underscore standing in for an id.
2. Then with the right shift operator (>>) we define that we will return a newly created Category object.
Inject the mock into the service class we’re testing.
Finally, we make verifications on the addQuestion() method by just stating the number of times we expect the method to be called with a given set of arguments, again reusing the wildcard underscore character.

You can even specify the particular order you expect by breaking the verifications into separate then: blocks. For this example it wouldn’t matter on the order, but in case it did the last when then block would change to:

1
2
3
4
5
6
7
8

when:
service.addDefaultQuestions(category)

then:
1 * dao.addQuestion(_, true)

then:
1 * dao.addQuestion(_, false)

And to round out testing the legacy Java code we need to test the negative example where it should do nothing if there are already questions in the database for the given category.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

def "should add no questions if questions are already in database"() {
given:
def question1 = new Question(defaultQuestion: true)
def question2 = new Question(defaultQuestion: false)

CategoryDao dao = Mock()
dao.getCategory(_) >> new Category(questions: [question1])

CategoryService service = new CategoryService()
service.setCategoryDao(dao)

when:
service.addDefaultQuestions(new Category(questions: [question2]))

then:
0 * dao.addQuestion(_,_)

}

So we can test for the negative case by just verifying that addQuestion was called zero times.

Grails Unit Testing: Mocking With MetaClass Stubs

Ed Gibbs — Sun, 17 Apr 2011 21:06:21 +0000

On a recent project I ran into issues with testing controllers in Grails. Starting test first, I spent some early time figuring out how much support there was out of the box for unit testing Grails domain classes and controllers. I setup Spock as a plugin and plunged in. I was dealing with a legacy database which had 100% compound primary keys so many of the findBy type operations that are well supported for controller testing are of no help. Often I found I needed to mock a call to a find or findAll with HQL syntax or a criteria call. I knew metaClass mocking could work here, but I wanted to understand better how that would impact the rest of the code after I started replacing methods on the fly. Turns out Grails unit testing has built in support for cleaning up metaClass hacking after every test with the registerMetaClass() method in GrailsUnitTestCase. Mrhaki has a good post on this. Here’s an Spock example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import grails.plugin.spock.ControllerSpec

class ContractControllerSpec extends ControllerSpec {

def setup() {
registerMetaClass Contract
}

def "validateContract() returns true for an existing contract"() {
given:
mockDomain(Contract)
def contract = new Contract(division: '33', unit: '99', contractNumber: 'C7777777').save(flush: true)
controller.params.division = '33'
controller.params.unit = '99'
controller.params.contractNumber = 'C7777777'
Contract.metaClass.static.find = { String query, Map namedArgs -> contract }

when:
controller.validateContract()

then:
"TRUE" == controller.response.contentAsString
}

}

Grails has support for the same idea using mockFor(), but the syntax is much like EasyMock. The same test would look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import grails.plugin.spock.ControllerSpec

class ContractControllerSpec extends ControllerSpec {

def "validateContract() returns true for an existing contract"() {
given:
mockDomain(Contract)
def contract = new Contract(division: '33', unit: '99', contractNumber: 'C7777777').save(flush: true)
controller.params.division = '33'
controller.params.unit = '99'
controller.params.contractNumber = 'C7777777'

// EasyMock like syntax
def mockContract = mockFor(Contract)
mockContract.demand.static.find(0..5) { String query, Map namedArgs -> null }

when:
controller.validateContract()

then:
"TRUE" == controller.response.contentAsString
}

}

For pure stubs I prefer the syntax of just using MetaClass. I often don’t care about validating the exact calls to the dependent class so I don’t really want a full mock. You don’t need a call to registerMetaClass() because mockFor() does this for you.

Developers and Desktop Databases

Ed Gibbs — Tue, 08 Mar 2011 14:26:07 +0000

Typically development groups fall into one of two patterns:

Shared development database – every developer shares a single centralized development database often managed by a DBA group.
Desktop databases – every developer has an individual instance of the database.

The practice has evolved over time and developers tend to follow precedence on it. Historically, developers didn’t think of running a local database, primarily because of the resources involved. Installing and running something like Oracle locally used to be a significant drag on a development machine. You were already trying to run some sort of IDE that sucked up most of your machines resources so running the database locally wasn’t a great option. If you were lucky enough to be running a pretty light weight database you might think about running a local copy, but otherwise you just setup a connection and shared the dev instance.

As machines have gotten quite capable of running databases, IDEs, and even virtual machines all at the same time the option to run a local copy of the database became very real. In this case a number of developers began to install and run local copies of the database. Running your own local copy quickly makes some development issues stand out:

You have to learn a bit more about how your target database software is setup. Suddenly it’s just not just a set of connection parameters.
You have to think about the best way to coordinate changes to the database model. Making a lot of changes locally to apply later to the central dev integration instance is going to end in a lot of angry emails with how you broke the build.
You can quickly experiment with local changes to the database model without impacting the rest of the team.
You have a truly complete development environment, so if you take your laptop home at night, work remotely for the day, or get called for troubleshooting in the evening you can be effective without having to VPN into the dev database or drive into the office.

I think the long term trend is most developers will run local instances of their target database. This is almost standard in startups and newer organizations. As a consultant I love having a complete development environment since I can run and demo the project at a moment’s notice. The issues with integration are being addressed with approaches like database migrations in Rails and maintenance of DDL in source control, treating it like an equal member of the code. I’ve also seen partial steps towards this model with the idea of in-memory local databases used for unit/integration testing where you use an ORM like Hibernate to build the database model on the fly whenever you need to run tests.

Spock Intro Tutorial

Ed Gibbs — Thu, 03 Mar 2011 14:21:05 +0000

I gave a presentation on Spock a very nice BDD framework in Groovy a few months back to our Groovy Users Group in Sacramento. After using it on a real world Grails project the last few months it has grown on me to become my go to testing framework for Groovy/Grails or Java projects. A typical specification looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

def "a pager should calculate total pages, current page, and offset"() {
when: "count, rows and page number"
def pager = new Pager(count, rows, page)

then: "should return correct total pages, the current page, and the offset"
pager.totalPages == totalPages
pager.currentPage == currentPage
pager.offset == offset

where: "you have a number of different scenarios"
count | rows | page | totalPages | currentPage | offset
100 | 10 | 1 | 10 | 1 | 0
950 | 100 | 5 | 10 | 5 | 400
72 | 20 | 3 | 4 | 3 | 40
}

If that passed your 5 second test take a look at a fuller introductory tutorial I put together.

A Gentle Introduction to Spock

And if you want to try out executing real code the project has a nice browser based environment at Meet Spock.

AgileZen for Solo Remote Development

Ed Gibbs — Mon, 24 Jan 2011 04:55:33 +0000

Over the past few months I’ve been working on some remote projects as a solo developer. Knowing I wanted a light weight tracking tool I took some time to experiment with a few tools and I’ve recently come to appreciate the lightweight character of AgileZen.

Despite many warnings to use the simplest thing that could possibly work, often stickies on the wall or simple index cards, there have always been a number of options to translate these planning/tracking tools to software applications. Despite settling on the power of physical wallboards for collocated teams throughout my career I have looked into using a number of tools for feasibility ultimately deciding none of them worked as well as a cork board and some stickies or index cards. Tactile and visual cards rock.

Some of the tools I’ve looked at in the past include:

XPlanner (an very old tool in the space)
Basecamp
Excel
Rally
Scrumworks
Pivotal Tracker
Trac
Mantis (a bug tracking tool used for planning)

I never felt any of them where worth the effort to maintain in the end despite many high hopes. I’m not immune to tool fascination, but I tried to honestly evaluate the worthiness of the given tool. As a manager much of my focus on this was to please stakeholders and other managers with a nice visual report that could be shared via email and web browsers. In my Agile rollouts it turned out that I was ultimately more successful showing off my cork boards and cards. We did maintain an Excel spreadsheet with backlogs as well to meet some documentation standards for PM groups. It was a small compromise I was willing to live with.

My most recent projects presented me with a different challenge. As a solo developer I’m largely on my own and not sharing the project tasks with anyone else, but I like to have an organized workflow at all times. My home office doesn’t have much in the way of space for a nice corkboard, and again I wanted to fiddle with tools so I tried a few to see how they might work for me.

A first attempt was with Pivotal Tracker. I love the things Pivotal is doing out there with a focus on TDD/BDD, pair programming and producing high quality code. Pivotal Tracker is a nicely polished tracking tool that largely simulates the idea of a wall board with an emphasis on iterations that contain a set number of story points. It’s an opinionated piece of software with this and assumes that you are very concerned with ordering of stories based on points and how you pull them into the backlog. After a lot of work trying to understand how it wants me to work I found it not a great fit as a solo developer. It assumes you want to have tracker plan your iterations automatically based on your velocity. In fact they admit that you can change some settings for the current iteration to allow you to drag stories in or out, but it will still automatically plan all the other iterations in the backlog. In the end I might be up for trying it with a non-collocated team, but for a solo developer the overhead was a bit higher than I prefer.

AgileZen was my second attempt. I remember from some old twitter and blog posts that several people had said it was a really nice lightweight tool. I realized arriving at the site they had been acquired by Rally. I’ve used Rally’s tools, but for collocated teams I didn’t really think the overhead of using Rally was worth it. It appears that AgileZen is their attempt to provide a lighter more Kanban style tool. I liked the little bonsai tree on the login and found more to like when I went in and attempted to setup my current project. I watched the short series of screen casts showing how it worked, maybe 5-10 minutes in all. After that I’ve been very productive without having to dig into any help. The central metaphor is just a digital task board:

Agile Zen focuses on the concept of cards and a board with very lightweight workflow. You create a feature or user story and ‘hang the card on the wall’. You then just move the card to progress it from the backlog to started. You can do some customization, but the default workflow was perfect for me as a single developer. It also looks like it works quite well for remote teams and appears to support more options like marking cards as ready or blocked. I’ve even used the blocked status for a card once where I wanted to run a design idea by a colleague before I continued further on that particular story. And I’m a sucker for some auto-generated stats, but it has a nice page where it calculates your throughput with a pretty graph. It will take more time to see if the stats actually give me useful feedback, but did I mention they look nice:

Overall the tool feels closer to a digital taskboard than anything else I’ve experimented with. I’d still use a physical task board for a collocated team, but for a team working in remote locations or a single developer I think it makes a very good electronic substitute. Now if only it had an iPad version. If you’re a solo developer feel free to try it yourself as it’s 1 project is free for a single developer.