L'Agilitateur - In English

How does design get done on an Agile Project ?

Oaz — Mon, 07 Jul 2008 12:11:00 +0200

I watched a great presentation by Jeremy D. Miller : How does design get done on an Agile Project ?.

Jeremy shows through multiple examples the kind of design issues you inevitably face during an project and explains how to handle them in an agile way.

I converted the slides of his presentation into a flash document in order to get a side by side view of slides and video on this web page (though you'll have to click on the slides to go to next one).

Enjoy !

Pod pod pod pod

Oaz — Thu, 09 Aug 2007 23:55:00 +0200

Thanks to Jeremy D. Miller I just discovered a place that is worth a web surf: Process, People, and Pods. Fred George started to blog less than a month ago and, in a few articles, he described the very nature of agile software development.
From agility inner secret to encounters of the third kind (of developers) and path to development wisdom, we go through all what we would like to tell about our jobs if we knew the words for that.

Oh, and did I tell you about the pods? You should really read those ones too!

The Time Dilation of an Ideal Day

Oaz — Fri, 17 Nov 2006 01:02:00 +0100

Ideal Day, Story Points, Load Factor, Velocity... Lots of concepts for a simple goal : predicting the amount of work that will get done for a given date. Anybody who went through this kind of measurement system probably tried to grasp its validity. Dave Churchville wonders "How long is an ideal day ?" in calendar time. Ed Gibbs rebounds on the difficulty that people have with story points when time units come more naturally.

I'm no exception. I used plain story points for a while but I recently started to trick this tool to get (I hope) a better matching between the developer estimates and the amount of work units.
Estimating in story points relative sizes is not easy because you quickly loses the connection with calendar time. A developer will provide much more consistent estimates with time units : "I can do that in a couple of hours", "It will take two or three days". These estimates might not indeed be accurate. On one hand, developers are sometimes optimistic because they underestimate the time needed to write the full range of tests, because they forget that some documentation has to be updated, ... On the other hand, developers can be pessimistic : a difficult task can be seen as a huge amount of work, a tedious one might look like lasting forever, ...
The estimates might not be accurate but they will probably be consistent. When someone asks you such a figure, you quickly imagine the size of the code you need to write and you transform this size into your own time unit system. As a consequence, the same task will be given the same figure in time units.

Now, the question is : is there a linear transformation between the developer personal ideal day (DPID) and a "standardized" ideal day?
I assume that this transformation is not linear. I have no evidence for this assumption but, in my current environment, I often get the feeling that short tasks tend to be underestimated and long tasks tend to be overestimated. I think I need some logarithmic transformation.

Actually, I prefer to have a transformation from DPID to story points. The reason is simple. I think that story points are better for a planning game. When you mention a ratio between "ideal" days and calendar days, there is always someone to make it known that the developers are lazy if this ratio is less than 1. The story point is a much more neutral unit because it looks like the cost is inherent to the work unit.

I proposed the following scale to the team and we started to test it:

Basically, the developer has to decide if the work unit:

	Developer Personal Ideal Hours	Points
will take one hour or so	1	1
is between one hour and one day (more or less half a day)	4	2
will take one day or so	8	3
is between one day and one week (more or less half a week)	20	5
will take one week or so	40	8
is really longer than a week (about two weeks)	80	13

I found out that the Fibonacci numbers (which are sometimes used for story points) were perfectly fitting this model. The idea is that we can try to adress the "relative sizes" dilemma by assuming that a 1 hour task + a half day task (in the developer own time system) will actually take as long as a one day task in the same system. Same thing for a 1 day task + a half week task matching a 1 week task.

These figures lead to the following graph. A short period of time represents more points than a longer one.

I don't know if this model will provide better results in terms of stability of the velocity but I hope it will help to remove the potential mismatchs between an iteration full of small tasks and another one containing a few big ones.
Anyway, time (dilated or not) will tell...

Black-box project management

Oaz — Fri, 03 Nov 2006 22:06:00 +0100

I love metaphors on project management. I just read a good one by Tom Looy : "How Long is a Piece of String?". You can get an estimate by looking at it but you will get a more accurate value by measuring it with a ruler. Estimating is not an easy job, especially when it comes to predicting when a piece of software is done. In the early stages of a software project, when you can barely estimate and certainly not measure anything, fixing scope, cost and dates is impossible.

Years ago, I was appointed software project manager for the first time. Being a software developer, I intuitively knew that I should be careful with scope, cost and dates issues. I was reluctant to draw a full gannt chart too early in the project because I suspected that people would rely too heavily on it. The projects director I was reporting to told me : "This is not a way to run a project. You must list every task for the project, estimate them and put them in the chart then you will get a release date. From there you will start tracking the team progress". I complied with the order.
Any experienced project manager obviously knows what happened : as the first tasks were not completed on time, the release date started to slip.
The estimates were indeed very bad. Half of the developers had just graduated from university and relying on their estimates was the worst thing to do but the rookie project manager didn't know that.

Now imagine this project is a big black box with 3 levers...
The first one is labelled 'product'. It shows the number of features implemented by the project weighted by their associated quality level (which is either explicit or implicit).
The second one is labelled 'cost'. It represents the global cost of the project from hardware to human resources including the productivity of each resource.
The last one is labelled 'duration' and is the release date.
As a project manager, you only have 2 hands so that you can grip 2 out of 3 levers and watch the 3rd one move freely...

Listing every possible task for the project means that you set a list of feature and decide some quality for each of them : the left hand is on the 'product' lever.
Estimating these tasks means that you evaluate a cost for each of them : the right hand is on the 'cost' lever.
Now you can start watching the 3rd lever (the release date) moving upwards !

Of course, time is money. If you don't want to see the 3rd lever being stuck centuries from today (which would mean a project that has no way of ending), you've got to move the cost lever upwards for paying the resources extra time.
This is exactly what happened in my first project.
At some point, people said that we could not afford releasing after a given date : we had to catch this damned 'duration' lever and hold it tight.
We had to choose a hand to do the job. As somebody had promised a list of features to the customer, the left hand was not available. The right hand had to let the 'cost' lever go freely and grip the 'duration' lever.

Guess what ? The 'cost' lever started to move upwards !
People were pushed to stay later and come to work on week ends "to give some fresh air to the project", in the general manager own words.
The productivity fell as developers got depressed. They were indeed paying the cost rise.
The customer had to pay too. In this period, I was asked to find every possible woolliness in the customer requirement documents in order to get some extra cash. "Hey Mr Customer ! What you want now is not exactly what you wrote, You've got to pay for it !"

No need to say that when the cost rose too much someone said "Stop !". The right hand being on the 'duration' lever, the left hand had to stop the 'cost' lever before it hits the ceiling...
Cost and duration were now in control. As it was still impossible to tell the customer that features had to be dropped, the quality level dramatically fell as the free 'product' lever went downwards.

Eventually, features were dropped and someone had to explain to the customer why this project was a death march.

Most of the time, holding the 'cost' and 'duration' lever from the beginning is the most sensible solution because :

If you hold onto 'product' and finish below expected schedule, your customer misses the opportunity to get some extra features

If you hold onto 'product' and finish beyond expected schedule, you're late and the customer is not happy

I once read that "project plans are either lucky or late" but, actually, project plans are either lazy or late : fixing a scope does not bring any guarantee that the product will be available on schedule and budget and in the same time, it makes no sense to stick to a scope when there's some room left.
That being said, sometimes feature can be really mandatory (e.g. safety issues). In these cases, focusing on cost and duration is meaningless because being late is less important than being out of scope.

In the end, the only thing we should care about is knowing, at any time, where our hands are, which levers are in control and which lever is currently moving freely so that we can have a look at it.

To Fix or Not to Fix : That is the Question

Oaz — Mon, 12 Jun 2006 20:41:35 +0200

Dave Churchville talks about things you should care about when planning bug fixes. I totally agree with his point. On one hand, it does not make sense to implement more and more features without fixing serious hazards first. This is sometimes called creeping featuritis. On the other hand, stopping any shipping till every bug is fixed can make more harm than good : a buggy software can provide more value to the user than no software at all.
I must confess that I had not fully understood the arguments in his first post. Indeed, one of the biggest misconception in software development is that bugs don't need to be prioritized just like any other feature.
That being said, it is a wise decision to let the business, the customer or the product owner, decide if the development team should implement new features or fix a few more bugs. Now the problem is : does the business have enough information to make a good decision ?
The person, or group of persons, making the decision must:

be aware of the final user needs at a fine level of details

know the technical impacts of an issue whether it is fixed or not

Regarding #1, one could tell that it's the business people job to make decision regarding actual user needs. Things are not so simple.
If we are talking about in-house software development and the final users are a group of people working at the next floor, they are probably involved in the decision process, or, at least, someone will ask them when a decision on a tricky issue is needed.
If we are talking about a software shop and the final users are people working in other companies, things are getting touchy. In this case, the product owner probably has a customer's business background but she might not have the knowledge required by every position in this business. It is ok when prioritizing features at a coarse level but it can turn into dice rolling when dealing with detailed activity of a real user.

Listing #2 as a pre-requisite for business people might look counter intuitive : why should we care about technical details when deciding on which bug or feature the development team should work on ? The answer lies in the nature of a bug.
When a feature is not implemented, there are no side effects. One can find a workaround to fulfill the same needs but does not get unexpected behaviour.
When a bug is not fixed, it is a different story. The bug might be hidden in a lower layer of the software. From there it can have impacts on many features of the application. These "other" bugs were not discovered (yet). It is just a matter of test coverage. Of course, we could just do "more tests" but then the economic argument (do not spend time on fixing bugs because time is money) is no longer valid. Things turn into an arbitration between spending time on tests/spending time on bug fixes/spending time on new features...

I have no answer for these two items but I know for sure that letting the business people unilaterally making the fix/not fix decisions is as bad as letting the developers or the QA people making these decisions. At some point, all these people have to talk to each other to find an agreement regarding the prioritization of bug fixes.

Every Process is an Iterative Waterfall

Oaz — Tue, 23 May 2006 05:55:00 +0200

Dave Nicolette raises concerns about organisations moving to agile development while appointing "traditionally-minded" managers. Risk #1 is that these managers turn the agile processes upside down by transforming them into a Staggered Iterative Waterfall as described by Kane Mar.
I deeply agree with the key statement of those articles : agility's main duty is improvement of interactions between individuals with the help of suitable processes. It is not about focusing on a well-polished process aimed at eliminating every possible waste in human activities. However, I do not subcribe to the RUP-bashing movement that seems to result from it.
It does not take a long time to realize that performing sequentially across consecutive iteration the analysis, design/coding and testing activities is a waste of opportunities. In the Staggered Iterative Waterfall pictured by Kane Mar, feedback that could happen inside an iteration takes three of them to arise.
Nevertheless, performing all the activities in the same iteration is not an easy job. For example, in an eXtreme Programming case :

The iteration plan needs the user stories to be written and estimated before the iteration start (we have to know which stories fit into the iteration). As a consequence, the time spent by the customer to workout the story must be shortened to its minimum. Every minute spent writing a story without interacting with the developers is a loss of feedback.

The customer tests must be run after (and also before) the iteration deliverable is built but before the iteration is over. This implies that customer tests are heavily automated and written as soon as coding starts. Failing to do so means that developers will not be able to produce value while the deliverable candidate is tested.

It is not an easy job but great teams can certainly achieve it to a certain extent.

The question is : can they fully achieve it ?
Obviously not.

In most projects, the best on-site customer (or group of on-site customers) cannot stay on the playground full time and simultaneously being fully aware the underlying customer activities. These on-site customers must be as close as possible to the development team but they have to stay in touch with their own businesses.
The problem is that you cannot put the whole world into a single room.
This is the reason why the stories describing the customer business need to be partly written off-site in an high level analysis activity. This is the reason why the the customer tests cannot be fully written while designing the system. The deliverable must be checked against some off-site exploratory testing activities involving other users.

The combination of those two activities with the software system development enhances the ability of the customer to think about her business process in a true iterative waterfall.
Maybe we can call that "Iterative Business Process Modeling". I'm far from being a BPM expert (just started reading stuff on that subject) but, intuitively, it looks like a good name.
The idea is that two processes, ran on each side, can reinforce each other. Feedback from the real user activity helps the developers to build the expected system while feedback from a delivered system helps real users to think about the way they work.

When going through the iterations, this process can be pictured as following.
The core system development activities are performed in each iteration with agile interactions between individuals at their peak.

Now let's have a look at the RUP-bashing thing.
The four phases of a RUP iteration look like a miniature waterfall: Inception, Elaboration, Construction, and Transition. [...] The reason RUP is used for iterative waterfall processes and not for agile processes is simply that it is unnecessary in a true agile process; its four phases become unproductive administrative overhead.

So the point is that RUP does not work for agile processes. I could just argue that "If you cannot run a RUP framework in an agile mode then you're not running a RUP framework" but I do not like this kind of extremism.
It looks like there's a big misundertanding regarding the RUP phases. Inception, Elaboration, Construction and Transition are not performed inside a single iteration. They are the phases of a release and each of these phases contains one iteration or more. Inside an iteration, you can do strict Waterfall, you can perform a V-shape or you can blend all the activities in the agile way. You could overlap the iterations but this is considered as a RUP Anti-pattern. It is advisable to do frequent reviews and have multi-skilled teams. Both of them are agile practices.
The RUP phases are the "macroscopic tempo". In a simplistic view of an agile instanciation of RUP, the inception phase is merely the first draft of user stories before development begins. The elaboration phase shows an important milestone : when it is over, the team considers that every "major" user story is implemented. The vision is established and the risk of a project failure is now lowered. The construction phase is about polishing the system until every customer need is implemented. The transition phase hands over the application to the users.

RUP is not inherently less agile than methodologies known for their agility. It is just a matter of usage. Its versatile nature support iterative waterfall processes including a bit of administrative tasks that inevitably happen in the real life with its natural time and space boudaries. Hence, one could even argue that RUP is more able to cope with human concerns than strict methodologies.