For those who haven’t, spikes are the agile response to reducing technical risk in a project. In case you are not sure how to build something or fear that some piece of functionality might take much more than expected, and you have to gain more confidence about it, it’s time to run a spike. Usually timeboxed, spikes are technical investigations with the objective of clarifying a certain aspect of the project.

In my current team, we are developing a few tools that are quite specific to our context and were not sure on how to solve a few issues, so we have been quite frequently playing spike cards.

This is all not new and I’m sure most of you have done that before. If you did the way I used to do, you would write a spike card, something as “investigate technology X for problem Y“, would spend 2 days doing it and would have a response for it in your head once you were finished.

In our current context, team members were rotating quite quickly, so we were worried that any knowledge we would get from spikes could have been lost if it was just…let’s say.. in our heads.

Not wanting jut to write the findings up, as we first thought about doing, we decided to tackle the problem with the golden hammer of agile development: tests!

So, instead of writing tests to decide how we should write our code, we started writing tests to state the assumptions we had about the things we were investigating, being able to verify them (or not) and have an executable documentation to show to other people.

For example, here is some code we wrote to investigate how ActiveRecord would work in different situations:

it 'should execute specific migration' do
  CreateProducts.migrate(:up)
  table_exists?("products", @db_name).should be_true
  table_exists?("items", @db_name).should be_false
 end

it 'should execute migrations to a specific version' do
  ActiveRecord::Migrator.migrate(ActiveRecord::Migrator.migrations_paths, 02) { |migration| true }
  table_exists?("products", @db_name).should be_true
  table_exists?("items", @db_name).should be_true
  table_exists?("customers", @db_name).should be_false
 end

it 'should not execute following migrations if one of them fails' do
  begin
    ActiveRecord::Migrator.migrate(ActiveRecord::Migrator.migrations_paths, nil) { |migration| true }
  rescue StandardError => e
      puts "Error: #{e.inspect}"
  end

  table_exists?("invalid", @db_name).should be_true
  m = ActiveRecord::Migrator.new(:up, ActiveRecord::Migrator.migrations_paths, nil)
  m.current_version.should == 3

  table_exists?("products", @db_name).should be_true
  table_exists?("items", @db_name).should be_true
  table_exists?("customers", @db_name).should be_true

  table_exists?("another_table", @db_name).should be_false

 end

We have used this technique just a few times and I won’t guarantee it will always be the best option, but so far the result for us is having code that can be executed, demonstrated and easily extended by others, making it easier to transfer knowledge between our team.

1. Work on puppet manifests, making a few changes
2. Commit code to repository
3. Wait for build to finish, which just verified for correct syntax
4. Wait for latest version to be published on the puppet master
5. Wait for next sync between master and client
6. Check that configuration was applied correctly on the client

As you can see, not very simple. If you also consider that I am not very experienced with puppet, you can imagine how I ended up having to retry things in this very long loop, which can end up with anyone’s patience.

Testing Infrastructure Code

Coming from a development background and being used to having very fast feedback about code that I write made me go into a search for testing tools that could ease my pain.

Unfortunately most of the tools I’ve found where not ideal since they focused on unit testing code, as rspec-puppet. Not sure what others think of it, but in the case of puppet manifests and chef recipes, unit testing doesn’t make much sense to me, since there is no code to be executed, and tests end up looking like some version of reverse programming, where you just assert what you wanted to write, but it doesn’t guarantee that the code actually works.

Introducing Toft

Luckily, one of the options I’ve found was Toft, which is a library aimed writing integration tests for infrastructure code using Linux containers. The main idea is that you write cucumber features verifying what you expect the target machine to have (packages, files, folders, etc..) and Toft starts a linux container, applies your configuration and runs your tests against it.

It also can be run from a vagrant box, so you can have your tests running on your mac, which is quite handy.

Features can be created using normal Cucumber steps, and mostly rely on ssh’ing into the target machine and verifying what’s going on in it, so are quite easy to extend and adapt to your needs. Here is an example of a feature verifying if a specific package has been installed.

Scenario: Check that package was installed on centos box
Given I have a clean running node n1
When I run puppet manifest “manifests/test_install.pp” on node “n1″
Then Node “n1″ should have package “zip” installed in the centos box

We’ve started using it in our team when writing new manifests and have also setup a ci build with it, which is quite useful to guarantee our manifests still work over time.

Toft is still in its beginning but I believe it has quite a lot of potential. If you are using chef or puppet you should definitely check it out at:

http://github.com/exceedhl/toft

Organisational learning. Should anyone care ?

My hope is that this goes without saying, but I guess that the first question to be asked is why should anyone care about learning? Some businesses make money staying still, while the world keeps moving. Full stop.

While this could be true for established business in the past, it’s something that is not so clear anymore. Gary Hamel makes a good point in his “Reinventing the Technology of Human Accomplishment” presentation: Change is changing, and it’s much faster now than it has ever been.

The world is changing, and faster than ever, so (every) company has to adapt, not only once but constantly.

Why is it so hard ?

A company has to learn. If this was an easy task, it wouldn’t be so rare, but creating a learning organisation is one of those things that gets harder the more you try and control it. If you want to form a learning organisation, a few questions you might think of are:

• What does my company need to learn ?
• If I had to setup the body of knowledge necessary to run my organisation, what would it be ?

While these could have been easy questions to answer 50 years ago, nowadays, in this ever changing (specially IT) world, whatever you write as being essential today might not be useful at all in one year’s time.

And the big issue is that most companies are not setup to deal with this problem. Instead of an evolving structure, modern organisations still have a very hierarchical structure based on top down control, with the vision of a CEO (put any other senior role here) that drives the strategy while most of the company just follows.

They have Project Management Offices that dictate how to deal with projects, architecture teams that say what can and cannot be done and common policies that dictate behaviour for everyone. Bjarte Bogsnes  has a good tale about how companie’s policies try to establish control over the employees:

“A friend of mine works at SAS. Even if he is trusted to fly planes around the world, if he wants to change his shirt more often than what the policy states, he needs a written authorization”

So the common alternative to evolution in this kind of structure are the well known “Change Programs”. They try to implement a top-down approach to change, that can be explained in two simple steps:

1. Senior people decide how to move forward
2. They teach the rest of the company how to do it

While step 1 is always a success, step 2 is usually more troublesome that people would like to think. And it’s not hard to understand why, since the whole idea of a change program is quite an unfair proposition to the affected employees, which could be described like this:

“You have to embrace change, but only the change we want”

As expected, people resist. Peter Senge has a quote that explains this situation quite well – “People don’t resist change, they resist being changed” – It’s natural that most employees will doubt and resist something that they haven’t been asked about. I would, and I’m quite confident that most people would too.

And as a result, money is spent, time and most importantly people are consumed through the process, and not much is learned in the end.

So is there an alternative to this situation ? Do you agree with this view ? Let me know what you think

Why is it hard?

I still remember watching an old presentation from Dan North where he kept hiting on the same point, which was that there is always a story behind any decision made on a project. And that is the main problem I see when joining a new team: there is always story, and you probably don’t know it.

Working in a new codebase for me feels much like moving to a house in a new neighborhood. It takes some time until you know where all the shops are, what’s the best way to get anywhere and where can you buy stuff in the middle of the night. In a software project you don’t know exactly where things are in the codebase, it takes time to understand the rationale behind the design and you are afraid of making big changes, even when they should be made!

So when ramping up a team with a considerable amount of new developers, there is a risk they won’t be able to be effective for quite some time, or that in the urge to be effective they will end up changing what doesn’t need to be changed, being counter productive after all.

So here we go again

This challenge came up again in my last project, where we were starting to work in our second release and wanted to include a team of developers in China in what would turn into our new distributed team.

Gladly enough we were able to spend 2 weeks together as a colocated team in Xi’an, but still we needed to use this 10 working days to ramp up 5 new developers who had little or no experience in the codebase we had been working on.

This time we decided to try something out of the ordinary with some inspiration coming from my period at Thoughtworks University  (what Sumeet would call workscaping), in what could be described as a mixed of promiscuous pairing and very frequent showcases.

What does that mean ?

The idea was:

• We rotate pairs every 2 hours
• After each 2 hour block the team would stop and every pair would do a 5 min presentation on what they had done
• We have a 30 min slot available at the end of the day for having a more in-depth discussion about topics that the new developers had doubts about.

Some of the benefits we expected to achieve were:

• Everyone would work together quite a few times, improving the relationship between team members
• The new developers would have an opportunity to touch different parts of the codebase quite soon, hearing the stories from the old ones about what was there and why
• By presenting it back to the team, everyone would have a chance to raise questions and understand a little bit more about a specific topic
• The constant sharing of information would enable newcomers to get more questions answered and deliver functionality quicker, which always helps with confidence

So the day would start with a normal standup, and after 2 hours the team would swarm and go around the room, hearing what had been happening. Pairs would then rotate and start again. Repeat that 4 times and the day is over.

How did it go ?

Overall the result was quite good in my opinion. Most of the results we expected to obtain actually happened, specially the communication within the team, which was great after a few days.

After one week of doing it most people started feeling the need of working more hours without interruption, so we changed our pace to 2 rounds of 4 hours and then eventually rotating pairs once a day, where we ended up stabilizing. Despite being earlier than expected, it felt like a natural point where people had enough knowledge to deliver and wanted to focus on that, which can probably be considered as a success.

In our project together we had a team divided between Melbourne and Xi’an and have been running retrospectives every two weeks. To avoid the common problems of distributed retrospectives, our team was using a simple format that worked (and is still working) quite well.

• We began by having a 5 minutes chat with the whole team present and setting the goals for the meeting
• The teams would then go into a 15 minutes offline mode, where they would raise and vote for the topics they wanted to discuss separately
• After the initial voting, we would normally pick the 2 most voted topics on each side and discuss them all together.

While it seems quite simple, the separation for the voting and initial exchange of ideas makes the the retro feel much more focused and better than other distributed retrospectives I have previously attended, and definitely a format I will keep using for distributed situations.

I’ve been doing it for a while, and have tried a couple of ways to actually get the data so far, going from having a spreadsheet where I would keep all the dates when cards transitioned (very detailed and very complex to maintain) to the usual noting down those dates in the card itself (less complicated but still troublesome, since people would forget to write on the card when they moved it).

Since I’ve started my last project, Herry introduced me to a new way, which as most good ideas, is so simple and good that I wonder how I had never thought about it.

The way we have been doing now is using stickers to keep the count of the number of days a card has been on the board. Whenever it starts to get played, we apply one _put your favourite colours here_ sticker to it for every day it is in play until it’s done. We currently just have two states in our board (doing and accepting) so we have two different colours to track the days in each specific stage.

The reason I find it much simpler is because you don’t have the problem of getting dates wrong anymore, or have to remember to note things on cards when they move. The only action needed is to apply a sticker to every card being played after the standup.

At LESS 2011, the organizers ditched the standard forms and went with simple post-its for feedback.

Instead of answering multiple questions, all you had to do is to grab a post it (available in all chairs) and stick it on the right place on the door when leaving (picture below). If you had anything more to say, you could write it in the actual post it, which were then delivered as they were written to the presenters.

   That’s it! Simple and effective.

Here are some examples of the results. I’m sure I could get better at it but the fact that I’m able to look a t it 2 months later and still remember what was presented says to me I’m on the right track.

Apart from that, the act of recording and try to create an explanation of what is being said makes me pay much more attention on the content. And it also helps me perceive when Im actually not understanding what is being said (or it is actually too hard to understand..), which happened a couple of times during the conference, as you can see from this example.

As any presentation write-up, this is just my understanding about what was presented, so please don’t hold the presenter on to what I’m writing here :)

Why ?

The first question Bjarte addressed was why the need for something different. And I believe this sentence explains it all

January-December is artificial for business and just works for accounting. But this is not accounting

He followed by explaining that StatOil is always trying to get the best performance they can (who isn’t!), and coupling everything they do to an accounting mechanism (the budget) doesn’t make sense for them. He illustrated it with a comparison between the use of traffic lights and roundabouts. While the first is easier to use and provides more control, the latter actually provides better results because it relies more in the current situation than on statistical analysis.

We need to find more self regulating ways to manage a business, and beyond budgeting is one of them.

Bjarte used Douglas McGregor’s Theories X and Y to explain how we traditional management doesn’t usually trust people and relies on a stable environment to succeed, but in a dynamic environment in which most companies are inserted today, we need to start trusting our employees much more, and that is where beyond budgeting stands.

He had a nice example to show the lack of trust some companies impose, about a friend who is a SAS pilot, and even when he is trusted to fly planes full of people around, if he wants to change his shirt more often than it’s stated in the company’s policy, he needs a written authorization for it.

How does it work ?

According to Bjartes, the problem with budget is that it’s a single tool used for three separate purposes: Setting targets, creating forecats and allocating resources. Since these results are usually different from each other (targets are what we want, forecasts are what we expect to get), we won’t get an optimal result from using it.

In StatOil they have developed an alternative format of planning called Ambition to Action. These are a few principles that are used when cdefining it:

• Good performance is being better than those we compare ourselves with
• It could be external companies or even ourselves, which means we are learning.
• Do the right thing
• Every new joiner receives a StatOil book (a thin one!) with some of the principles that guide the company, so they can all use their business judgement and make decisions when needed.
• Resources are made available and allocated on a case by case basis
• Bjartes compared having an yearly budget to having a bank that opens for only a month throughout the year. How can anyone make timely decisions with that ?
• Business is forward looking and action oriented
• Performance measurement is holistic, and composed by 50% results and 50% behaviour

These principles are used to define what to do in the company, in a process that goes like this:

1. Strategic Objectives -> 2. KPI’s -> 3. Action & Forecast -> 4.Individual goals

So the company is going to set strategic objectives which are then converted in KPI’s. These are used to define the things that need to be done (Ambition to action), which reflect on how performance is measured for everyone in the company.

Some interesting thoughts he shared when explaining the process were:

• The perfect KPI doesn’t exist, since not everything that counts can be counted.
• StatOil creates around 1100 Ambition to action plans, and these are not a reporting tool, but more a guidance on how you as an employee should manage him/herself. Apart from that, everything is open, so everyone can see everyone’s else goals.
• Performance measurement is usually based on teams and, as said before, divided between results and behaviour. Assuming that the environment very often changes, how someone behaved in that occasion is as important as the results he/she delivered.

The last point that was made during the presentation is how StatOil is moving from a calendar-driven model to a business-driven one. There are no more annual versions of Ambition to Action being created, and they can be changed at any time. The performance review is still happening annually, but Bjartes mentioned they are currently revisiting it now.

Overall it was a great introduction to Beyond Budgeting and how it is applied at StatOil. If you are interested, there is more information here and here.

Bjartes is speaking in the Thoughtworks Live Australia event. If you have a chance to go, don’t miss it.

Our team is quite small (around 9 people) and Im sure a lot of what we have done would be impossible to do in a larger team (I guess the next question would be why do you have a larger team.. :P), however just wanted to share the tools used and how it was all done.

Since we use skype for video communication, sharing a whiteboard would be impossible. The simplest solution found was TypeWith.me, an online collaboration tool that we projected on the wall.

We started by having everyone spending some time writing their ideas/observations on post its, but then instead of attaching them to the wall, we went around the table and everyone dictated their concerns, that were scribed in on TypeWith.me. After sharing ideas we went around again and everyone voted on ideas, which were noted on the online document again. Some reshuffling was done on them based on votes, and the we continued with the standard retro time-boxed discussions.

In general, I believe the retro had a good outcome. We didn’t lose too much time with the communication overhead and having the topics projected on the wall made it clear enough to both sides what was being discussed. Not trying to claim this was revolutionary in any sense, but if you are interested in running a small distributed retro, this might solve your problem.