Business Analysis

Size or Estimate

2008-04-08T12:14:00.000-07:00

Estimation, I think, has been one of the biggest problems of mankind. The Software development industry is no exception to this sad reality. We almost never seem to get it right.

The problem with the word "estimate" is that it's personal. It's relative to the person who is estimating. Moreover, it's a unit of time. That is why I prefer "sizing" stories rather than estimating them. Size is a non disputed (standard) unit, which measures the level of complexity involved in developing a piece of functionality, a story. The size is standard across a particular project team.

Pizzas are a great example of standard size. They come in small, medium and large sizes and, given the context of the vendor, the whole world knows what a small pizza means.

Pizza Appetite is the capacity to consume pizzas. A 5 year old will find a small pizza to big to eat. A teenager will probably find it perfectly filling. At 25, I find the small pizza too small. I can comfortably eat a medium pizza (or 2 small pizzas). I am sure there are people I don't know who can eat nothing less than a large pizza (or 4 small pizzas)!!

Stories are like pizzas. Given the context of a project, they have fixed sizes. As a developer spends more time on a project, getting older (and wiser), his/her capacity to develop complex stories increases. And this is where the trouble starts. At the beginning of the project the developer could do 1 small story every day. But now, instead of realizing that he can do a medium story (or 2 small stories) in a day, he begins to think that the medium story is actually a small one. The thing to remember is that the size of the story remains the same. It's just that the developer can now do more complex stuff in a given time.

Complexity points are never to be estimated (in fact they can't be estimated because they are not relative to individuals). In the OLAP terminology, complexity points are the fact. An estimate is a calculation you (as an individual) do when you are looking at them from the Time dimension.

So if you are estimating units of time, it makes sense for you to re-estimate so that you take into account your improved understanding of the system.

When you are sizing stories in complexity points be very careful not to fall in this trap. Re-sizing even sounds wrong anyway.

Functional Goals

2008-02-15T00:36:00.000-08:00

After this post, I have had a number of conversations with people about what I really meant by setting functional targets instead of story point (velocity) targets. In one such discussion, I was given an example where someone set a goal to run for 90 minutes!!

At first glance you might not find the last statement worth the exclamation marks at the end of it. But this is exactly the difference between functional targets and velocity targets. I think it's wrong to set a target of running for 90 minutes. The real goal is to burn X number of calories. Or to travel X number of kilometers.

Once you set your goal right, you are open to find better ways of achieving it. If your goal is burning calories, you'd probably do it better (faster) by skipping rope for 30 minutes than by running for 90 minutes.

Now all you have to worry about is

the rate at which you are actually burning calories AND
what can you do to improve this rate

So instead of planning to complete 30 points next iteration (going by yesterday's weather, etc), I'd plan to achieve a set of features and keep optimizing the rate at which I am able to produce features assuming that I never compromise quality in wake of going faster.

Iterationless...

2008-02-12T21:46:00.000-08:00

After working in week-long iterations for 2 and a half years, going iterationless left me a little confused on my recent project. Of course when I say iterationless, I mean using extremely small iterations to bring in a flow to stories. Lean? Distributed Lean, maybe?

Firstly, I like the idea of trying something new. The team being small and the project being developed in ruby helps this experiment a lot. There are, however, somethings missing in this way of working. I'll list them down and give my view on each of them in this post.

To realize what we missed, let's look at what we get from iterations. The following are the main motives behind iterations.

Celebrate small successes
Get regular feedback from clients (showcases)
Let the whole team know what we will be focusing on in the next iteration, including the business context around it (Iteration Planning Meeting)
Re-estimate stories based on additional information gathered during the earlier development (Iteration Planning Meeting)

So let's see how we are trying to handle each of these situations.

1. Celebrate small successes.
This is very important. To look back and realize that we actually celebrated achieving a story-point targets each iteration, however, now sounds really lame. The moment you move your focus away from finishing a set of stories in a given time TO finishing a set of features and going to production, the world starts making more sense.

I think in this aspect chucking iterations totally makes sense. What this means is you should have a very strong release plan, with frequent (less than a month long) releases.

2. Getting regular feedback from clients
Showcases have nothing to do with iterations. Iterations only provide regularity to showcases. To say that we will do showcases once a week (and do it) is I think good enough.

One thing to beware of is that showcases-bound-to-timelines (once a week) and development-bound-to-releases (of irregular time intervals) can be quite tricky to handle.

3. Let the whole team know what the focus of the next iteration is
Ours being a small (ideal?) team, I don't see too many problems in this area.

4. Re-estimate stories based on new knowledge
This, I thought, was the most important aspect we were missing out. But once you realize that your story point sizing is only relative to any other story you estimated, the world seems much better. This means that as long as your triangulation is correct, you are ok.

One thing that IPMs give us though, is an opportunity for the whole team to estimate together. Right now I am unsure of how important this method is. I am posting a few questions related to this at the end of this post. It'll be great to get some feedback on these.

All in all I think going iterationless is beneficial in terms of:

Shifting the team's focus from story points to features / business value.
Letting the team focus on estimating complexity of stories much more clearly.

What I am unclear about is estimation.

If we spend 2 hours every week to estimate together, are we going against lean philosophy?
On the contrary, if we estimate just-in-time, in smaller groups (pair to work on the story) are we losing the "group touch" to estimation?
Also if we estimate just-in-time (when we decide to play a story), the business doesn't get an opportunity to re-prioritize based on sizing.

Agile Business Analysts

2007-11-29T04:03:00.000-08:00

Yesterday, Craig Brown asked Do we need Agile Business Analysts?. Two and a half years back when I joined ThoughtWorks, I would have said "No, we don't". Today I strongly disagree with the view that Business Analysts can be done away with on Agile projects.

The objective of an Agile project is to provide maximum value to the client as soon as possible.

Now I agree that the ideal situation would be when you have business users & developers who agree and abide by this sentiment. You can happily get rid of the BA in such a situation. Real life, for some reason, is never ideal.

Anyone who has interacted with business users on a medium to large project will have faced situations where it has been hard for the business users to come to a consensus on what they want. Business users are generally representatives of different functions of the organization. Each of them has expectations from the application which more often than not are conflicting. Keeping these users focused on what we set to achieve when we started building the software is what a BA does.

Being on an agile project gives the business users many chances to dream, many chances to make changes. Developers are human too. They dream about technology. While sometimes this can lead to some really cool/unique/usable feature in the product, it can also lead to "overcoolified unnecessities". Now in an ideal situation, all these dreams would come true. Unfortunately, dreams in the real world are bound by budgets & deadlines. Managing these dreams, ideas, changes so as to keep them aligned to the original business objective and deliver most value in the given time and budget is what a BA does.

What agile aims at, is to start minimal and change as required. When a small bug on a story turns out to be a big chunk of enhanced functionality, we create a new story. The tasks mentioned above, along with writing user stories, supporting implementation of the functionality, planning the iterations and releases, testing the developed functionality, etc. together makes for a considerable chunk of work. That's when you create a seperate role called Business Analysts.

Now this role can definitely be performed by developers. In my experience, though, good developers don't really want to do any of this. They want to write good code. That's what they are best at. That's where they will deliver maximum value.

Hence Business Analysts.

StubbornSoft & MammothSoft

2007-02-14T11:32:00.000-08:00

I met a couple of villains today
a formidable lot.
StubbornSoft was one of them called,
the other was MammothSoft

StubbornSoft had it's own ways
for the user it didn't care.
MammothSoft would smugly say
"Try, change me if you dare."

In a moment I then realized
my dark and ugly fate.
When both of them cried out to me
Why, come let's integrate

We'll make a heavenly trio they said
our technologies cutting-edge
So smart piece of code as us
there'd ne'er be, we pledge

"I am quite small" to them said I,
"my features are too few.
But I let users do their job
and go home happy too"

"And what do we have to do with that?"
They asked to my dismay
"We were made to make the boss happy
not users anyway..."

Software is for users

Mountains to climb

2006-12-03T21:44:00.000-08:00

Business Analysts work closely with the customer. Each new area is a challenge. There are discussions and problems and solutions. Both the analyst and the customer get comfortable with the idea of finding better, simpler solutions. This is especially true for fixed bid projects, as the project is at stake if the analyst fails to find a simpler solution and convince the customer too.

Solving problems is always exciting. For an analyst, this is the juicy part of the job, because no human being can survive for long just writing word documents one after the other. One incentive is customer delight, second is developer delight (scope reduction, etc). Of course the kick that an analyst gets out of solving a problem and offering a relatively simple solution is inexplicable.

However, sometimes it happens so that the analyst gets stuck. There is a complex requirement that he/she gets from the customer and as usual the problem solving starts. The solution however seems complex. The situation is there right in front of him, all the scenarios are known and the functionality required to cover all these scenarios is just simply hugh. So the story is left half way through. More thought is put into the problem in wake of a simpler solution.

What the analyst fails to realize is that there is no simpler solution. Sometimes there's a given amount of work and it just has to be done. The sooner this realization comes, the better.

The time spent on a given story is I think, the best indicator of such a situation. Just measure the average time you take for a story. Probably coupling it up with the dev (complexity) estimate would be even better. So if you are taking much more than this average time, there's somthing wrong.

For Example: You take 6 days to analyze and write a (roughly) 3 point story. If you realize that this one story has gone on for 15 days, it's time to stop and think.

In case of such a situation,

Firstly, raise the fact and see that the whole team knows about this hugh chunk of work coming. Have a domain session and share your knowledge.

Secondly, prioritize this area. Change the release plan if required. This will most probably turn out to be a risky area.

And most importantly, start work as soon as possible. Just write a small, testable story and get the area kicked off. IMHO it is acceptable even if the story doesn't provide much value to the customer. It will provide value to the team by introducing an area.

It's like looking up at a mountain and thinking about a simpler way to reach to the top. Most of the times it's smart to start climbing.

As someone has rightly said, "Thought is supposed to be a guide for action. Not a substitute for it."

How to eat an elephant?

2006-12-02T19:41:00.000-08:00

The only way to do that is to cut the elephant into slices. Slices which are small enough to be eaten because the sole purpose of cutting the elephant is to make it possible for humans to eat it.

You'll need a certain number of people to eat a whole elephant anyway. If you make slices small enough, at least they won't feel sick at the end of it.

There is no overhead in the activities of choosing the piece that looks most delicious, putting it in your mouth, chewing it and swallowing it. There is some, but it definitly beats having to swallow the elephant whole.

So don't try to be subjective or pragmatic or take a case by case approach. Don't fool yourself.

Write small stories.

Applied Integration

2006-10-16T12:36:00.000-07:00

For a while, I have had a question on my mind that I couldn't really find an answer to. I had almost forgotten about it but a friend brought up the same question and before I knew it I was giving him my take on it. Before I get to the question though, I want to write a little about integration.

Last release I was involved in building some functionality to integrate with one of our client's systems. As I understand it, there are four prominent characteristics of integration that should be kept in mind when we decide to integrate systems.

1. Specialization
Integration arises out of the need for specialized systems to come together to attain a common goal.
Although each system can help in a unique way, none can achieve the goal by itself.

2. Communication
Integration is communication and all the rules of communication apply.
- Common language
- Interchange of information
The goal of communication is to get to a win-win deal.
Communication is feedback, well, at least most of it and this feedback is the basis of integration

3. Functionality on both sides

There is functionality on the other side. You are integrating with a system because it has something to offer. The goal is to leverage the information / knowledge / functionality on both the sides in a lesser amount of time than is required for any of the systems to attain the goal by itself.

If the Matrix is ever built (in a reasonable amount of time that is), it will be through integration of various specialized systems rather than all by itself.

4. Compromise
In order to communicate effectively, there is compromise required on both sides.
These compromises are mostly in the form of language & time.

I found myself using this understanding to explain my point of view to my friend. The question I was struggling with was,

If I do my job right, will everything work itself out?

I readily realized that this was stupid. However right I did my job, I couldn't achieve anything in a reasonable amount of time. So I changed my question to,

If we all do our jobs right, will everything work itself out?

This seemed promising. But I realized that, my job is my specialization and that's true for all of us and when working together, feedback is important for us to know that we are doing our job right. Without this basic integration we are hopeless.

Any team brings in integration of many specialized systems. Developers, BAs, QAs, Project Managers & Clients, for example, in a software development team. To take part in this integration, communicate useful feedback and leverage each other's "special powers" by making necessary compromises is the only way to build the Matrix... Or for that matter a library management system.

Embracing Bunker Busters

2006-08-19T11:27:00.000-07:00

For various reasons, there are situations when there is no running away from a Bunker Buster. An agile team should therefore be watchful. Once the analyst finds a potential bunker buster, he/she will have only one parameter to tweak... The timing. Embracing a bunker buster is a a brave step, I would say. But then isn't, bravery only relative to the preparation??

Symptoms
Here are some tips out of my experience for identifying bunker busters. Note that these symptoms do not decide a bunker buster individually. Like gesture-clusters in body language, they work in combinations.

Vagueness
The first and most probable characteristic that you'll notice about a bunker buster is vagueness. This is not the same vagueness that you experience when you don't know the screen where particular information is captured. This is vagueness which has potential to change the way you have been thinking about the solution. Some fundamental assumptions about the way the user is going to use the application could be proven wrong at the and of your analysis. This brings us to the next characteristic, The Paradigm Shift...

Paradigm Shift
Over a period of time in a project, the team develops an idea of how a user is going to use the application. A bunker buster has the potential of changing this basic usage paradigm. Let's take an example here.

Imagine a phone book application. This application allows Adding, Editing, Viewing and Deleting phonebook entries. The team envisaged this application thus:
- The user will search or browse the phone book to find a particular entry.
- The user will then view the details of the selected entry.
- The user then has the option to Edit or Delete an entry
- The function for adding a new entry is available throughout.

Now after some time into the project, the business and the analysts realized that the users that search, browse or view entries are different from the users that add, edit or delete entries. The solution was to select a mode while entering the application. The user would be able to only add entries in the "Add" mode. While the "View" mode would let users only view an entry.

This is a paradigm shift. While most bunker busters we see in reality are not of this magnitude, I hope you get the idea.

Extra Long Analysis Phase
Both the above lead to an unusually long analysis phase for this particular story. Another fact that contributes to this is the fear that there might be exceptions. While the new usage paradigm makes sense from the usability point of view, there might be functionality which will make less / no sense if the user is going to use the application in a different way!

Approach
To get consensus of all the stakeholders on this new paradigm, to clear up confusions, to find and take care of exceptions (if any) and to present all this in the form of a small, independent, testable and valuable story is a mammoth task. To make this a tad easier, here's a list of things to take care of when dealing with a bunker buster. I have divided this by stake holder because the underlying gist is to communicate constantly with everybody.

Client
Communication with the client is always crucial. After sometime into a project analysts generally split into areas and so do the members of the client team. While handling a bunker buster however, constant communication with the complete client team is essential. I would include all business as well as technical users in this. End users, sponsors, client's system administrators, deployment team (in case you are building a product) and of course the complete client team that defines requirements. The shift in usage paradigm is a key decision on which you would need all kinds of input and a consensus from all the mentioned parties.

BAs
Other business analysts are the best people to talk to when it comes to finding exceptions. They will have complete knowledge of the respective areas that they are working on. Also, early communication of this kind will help them analyse the future functionality keeping the bunker buster in mind.

QAs
One thing that closely matches the way an application is going to be used is the functional test suite. Functional tests are the most valuable and the most fragile tests on a project. A bunker buster can render loads of them useless if they are not refactored to accommodate the new usage paradigm in time. Depending on the size of the functional test suite, the QAs will have to decide whether to refactor them or throw them away or leave them untouched. The QAs can leave the tests untouched if the application can provide useful hacks, like switching security off. Of course there are other functional tests that test security. Personally, I think this is the last resort. Functional tests should always be in sync with the way the application will be used.

Developers
Communication with developers is of vital importance because a bunker buster has potential to change the domain model in unexpected ways. Validating the feasibility of the change and it's estimation is absolutely crucial. While the bunker buster is in play, I would suggest even pairing with the developers. One area where it will most hurt is the amount of detail in the story itself. Due to the long phase of analysis, accommodating all the detail in the final story is all the more difficult. Remember that while you are busy preparing this story for play, there is functionality being added to the application. I think this is one more place where it would be much better to leave the story at a high-level, do a careful kickoff and if possible pair up with the developers.

Timing
As mentioned before, timing is possibly the only parameter that we can tweak as a team and taking full advantage of this fact is very important. Due to the consequences mentioned above, I think it is best to play a bunker buster at the very beginning of a release. That way the UAT for the previous release can go smoothly. The team also gets a logical break point to adopt this new way of thinking about the solution... Mentally(analysis) as well as physically (code and tests).

Scary as all this may sound, I guess successfully embracing changes like these is the ultimate value that an agile team delivers.

Avoiding Bunker Busters

2006-08-13T12:20:00.000-07:00

"Prevention is better than cure" and one good way of handling Bunker Busters is to avoid them. Agile teams seem to be wary of handling anything by prevention, as it requires thinking ahead and taking care of things to come rather than things that are. While this makes perfect sense for Grenades and Shells, Bunker Busters have much more potential for damage and it is worthwhile trying to prevent them rather than handling them when they come.

There are areas in every application which can prove to be huge bunker busters if not handled in advance. Teams are even aware of there existence but still choose to handle them "when they come". One such area is "Application Security" and I have heard about (and experienced) this quite a lot so I think it makes a good, simple example.

Let's go back to the way we write stories. The story aims at stating a particular requirement in the form of an action, which a particular role wants to perform to achieve a specific, measurable, achievable, realistic and time-bound goal. The generally accepted form of such a story is a single sentence as follows:
As a <role>
I would like to <action>
So that <immediate goal>

A Grenade in this form would look like:

As an administrator,
I would like to add a new book to the library,
So that the book can be borrowed by the borrowers.

Now this is a good story because it can be easily implemented to achieve the following:

A form which only the administrator can access [role]
Logic to validate & persist the information entered into the form [action]
tests to check if the book can be borrowed by the borrowers (which is another role BTW). [goal]

The "So that" clause here does two things. It sets the testability of the story as well as clearly mentions the dependancy.

Leveraging all clauses of the story in this way (and firstly writing stories in this way) can go a long way in favour of the project.

The same story would become a Shell, like this:

As an administrator,
I would like to restrict other borrowers from adding a new book to the library,
So that I can control the books added the library.

(this will be played in addition to the first story, because we decided to do security later)
Sounds worse, does it?? But it can get even worse like this:

As an administrator,
I would like to restrict borrowers from adding new books to the library, editing book information, deleting books from the library, viewing x,y and z reports and configuring x, y and z,
So that I have full control of the admin functionality of the application.

(this story will be played after the stories for all the above functionality have been played...)
This is a bunker buster; and the worse part is that this could be avoided.

(Note: the second and third stories aim at delivering the same functionality, although the reader might think that these are stories will let the administrator "configure" access restriction to the mentioned parts of the application.)

Grenades, Shells & Bunker Busters

2006-08-09T11:55:00.000-07:00

As agile business analysts, we try to keep our stories small, independent, testable and valueable. Once in a while, though, we come face to face with a Bunker Buster and everything starts looking scary.

Impact

Every story is likely to impact a certain portion of the developed application. Measuring this impact and using it to "time" a story well, is something commonly ignored by business analysts. This doesn't mean that the analyst or the team doesn't know the impact of a story but this measure is not used in planning exercises effectively.

We can almost categorize stories to get a quick idea of the impact that they have on an application. In my experience, there are three such categories

Hand Grenades - Stories that add functionality
This means these stories add completely new functionality. These stories impact a very small part of the application that has already been developed.

Shells - Stories that enhance functionality
There are stories that form latter parts of a series which enhance already developed functionality. These stories bring in minor refactorings and hence their impact is rarely beyond the code written to deliver that particular functionality.

Bunker Busters - Stories that enhance the application
These are the stories that aim at enhancing the "complete" application. A common example of this species is application security. Inevitably, there are areas of the application that only privileged users can access. These kind of stories bring in major refactorings and hence have an impact on a considerably larger proportion of the code already written.

Handling Bunker Busters
In a ideal world, a business analyst will deal with Hand Grenades and Shells. But things are not that simple all the time and we need to handle a Bunker Buster once in a while. There are 2 approaches towards handling these stories.

Avoiding
The first approach is to simply avoid a Bunker Buster by reducing it into several Hand Grenades or Shell stories. Carrying the example of security forward, we will think about the security involved, everytime we add or enhance functionality and thus split security across several Hand Grenade and Shell stories

Advantages:

Less / No impact on existing code
Tests (especially functional ones) remain healthy and effective
Stories become really independent and valuable
Planning becomes simple

Disadvantages:

Requires upfront thinking and design
Inclusion in all stories

There are areas like security, common to most projects we do, which can be handled using this approach looking at the trade off between upfront design and ease of implementation.

Embracing
The second approach is to play a single (generally huge) story which impacts a whole lot of developed functionality. In many cases this option is inevitable because the required functionality was never thought of.

Splitting this story doesn't make too much sense because the result tends to be half-baked functionality which poses even more problems. Timing this kind of a story is something that the team needs to be careful about. This story is likely to impact a large proportion of code as well as tests besides bringing in all the disadvantages of large stories with it.

Considering these impacts, the team may decide to :

play these stories as soon as discovered and refactor code and tests as required
OR
play these stories at the fag end of the project by leaving existing test untouched and testing this functionality seperately.

For more on Bunker Busters, stay tuned...

Deceptively small stories??

2006-07-27T10:45:00.000-07:00

For sometime now I have got the feedback from developers, that the stories I write look small but have loads of work to be actually done and they get under estimated during the Iteration Planning Meeting. Now when I asked for a way in which I could make the story look as big as it is, I was given the idea of putting tasks in the story.

I have used this method for sometime now and I find myself writing things like:

TASK 1: When the user clicks "OK" an empty Word template should be uploaded to the server and then the same file should be opened for editing

whereas I would have generally written:

Clicking "OK" should open MS Word with an empty template so that the user can edit the contents.

While this puts implementation into the story and makes the story really bulky, it seems to really help the devs and may even be of help to a technically inclined client or the client's IT department which is going to take over the maintenance.

I think distributed agile makes us take many such deviations from the pure form of agile ideas we have read about. Most of these are justifiable in terms of the trade-off between concepts like "minimum documentation" and "transparency / knowledge sharing". After all isn't agile about doing whatever it takes to quickly deliver value to the customer??

Communication Patterns

2006-07-23T02:05:00.000-07:00

Incidentally my earlier post was about how Distributed Agile works. My unexpectedly long web silence was infact due to the current DA project that I am working on.

One interesting thing that we get to observe is how the communication patterns change over time, amongst the local team and also with the distributed team.

There are few specific things that I have observed during the last 5 months on this project, which I would like to bring up here. All of these relate to the communication that happens across oceans.

Overview
Last few months I have been busy on a Distributed Agile project. I feel obliged to put down my observations of the changes in the way communication takes place between distributed teams.

The typical setting renders the following arrangement:

Client (since this is about communication we restrict clients to be the people who give requirements. NOT sponsors or end-users)
On-site BAs
Delivery Team. This is the larger team that does the development and consists of BAs, QAs, Devs and an IM/PM

Note that the patterns of communication enlisted here use the above setting as a base. They may not be similar to the way distributed teams work when actual delivery is distributed.(That'll be covered in part 2, if there is one)

Communication
Communication is arguably the most emphasized practice in Agile methodologies. A perfect agile team will communicate in these ways (in order of preference)
- face to face
- video conferences
- voice conferences
- email or other documentation.

Small co-located teams will easily handle communication amongst themselves and with the client. It is the larger teams that find it difficult to manage without emails / documents and it gets worse when the teams are distributed and have to work with each other across seas.

This is a collation of my experiences about how the communication patterns evolve and affect the project.

Context blues
When the project begins, a very small number of people have had a chance to meet the client face to face. To make things worse, the team that ramps up, and starts to increasingly communicate with the client, is the team which is going to deliver the software. This is the 3rd entity listed in the overview.

The BAs in the ramping team lack 3 things that makes communication effective. These are listed below in order of importance.

Business Context
The first thing that the delivery team lacks is the business context in which the solution is to be provided. The Business Objective behind this project and the criticality of the project success are the fundamental things that let the team organize it's thoughts and actions towards a common goal which is more or less inline with the client's expectations.

Client Context
The second thing that hampers effective communication is the lack of knowledge of who the client is. It is of high importance that the delivery team is aware of the client's position and the stake that he has in success of the project.

Client
Knowing what the client is like, is also very important to be able to communicate effectively. Having met the client personally impacts the effectiveness of communication a lot.

This is the first phase of the communication with the on-site team. Another interesting feature of this phase is the importance of the on-site BAs. The on-site BAs do not lack any of the things listed above, which makes them many folds more productive and hence important to the project.

Infrastructure blues
This is the second phase of the communication evolution in a distributed agile project. This is the phase where the delivery team reacts to the earlier phase of lack of context. The immediate solution which is put forth is to strengthen the communication infrastructure so as to get as "close" to the client as possible. By the order of preference given above, the team rules out the face to face communication and promptly moves towards video and voice conferencing solutions.

This phase features infrastructure failures which are acted upon with varying degrees of success and in the end, stabilize.

Delivery team amnesia
This phase is a result of the rising ease in communication with the client. The delivery team now has ease access to the client. It also has most of the necessary context of the project. The development also starts showing real progress by this time and hence the delivery team now gains importance. It has to get critical questions answered, give development support and show progress via showcases.

All this action tends to give the delivery team intermittent amnesia about the existence of the on-site BAs. This is a pattern that continues in varying degrees throughout the project. For countering this particular problem the team comes up with a schedule to communicate with the on-site BAs. This turns out to be a decent solution but it fails every time there is a change in the schedule. Every change then takes time to stabilize.

Guests and parties
By this time the delivery team is using the communication infrastructure to the full extent. But there is still some context that's missing and the delivery team starts becoming more and more aware of this fact. There is a strong strong need to talk to the client face to face.

The project has more or less stabilized by this time and so the cross-pollination as well as client visits to the delivery location take place. Many important decisions, sessions, context & knowledge sharing happens during these visits. There are invariably some tricky areas/issues to be discussed during these sessions. Making the most of these sessions is one responsibility that the BAs should really handle well.

Another amazing effect of the visits is the trust that builds between the client and the delivery team because now the client actually sees the work going on and progress being made. This is a good opportunity for the delivery team to bring to the clients' notice, the implications of their actions (or inaction). Signing off / giving feedback on functionality that has been developed is a major participation required from the client. Demonstrating how this feedback is taken and worked upon helps the delivery team to considerable extents.

These visits also enable teams members to spend more informal time with the client. Team outings and sight-seeing are important parts of these visits.

Stabilization
This is the phase after the client visits are over. Whatever major issues were to be discussed have been discussed and decisions reached (well, almost). This is the golden phase in the project where everyone is much more clear about what they need to do and when to do it and that includes the client.

This is the phase to be reached by a distributed agile project as soon as possible. It is generally not possible to jump these phases and get to the stabilization phase early. It is however, quite helpful to understand that these phases will come and will need to be handled.

Overseas...

2006-04-19T03:39:00.000-07:00

So what happens to the agility when the project is distributed? Well, it suffers...

It's not that it doesn't work in a distributed context. In fact ThoughtWorks has been doing distributed agile for quite sometime now. We talked about some of the best practices of agile earlier. Let us see what problems distributed agile poses to these principles.

Time difference
When working across oceans, the first thing that hits the team is the time difference. The typical arrangement renders the team to be in a low cost country like India and the client in the US of A or Europe.

The time difference is the first major barrier to communication. Because even if we assume that the other barriers like distance, language, culture, etc can be handled, we just wake up and live at different times.

Communication
As much as agile believes in face-to-face communication, it suffers badly in a distributed context. Mitigation consists of:

Cross pollination
Using Voice communication rather than mails
Video Conferencing
Adjusting working schedules on both sides

These techniques are most effective with mirrored teams. Of course, that's the essence of a distributed project, but it gets missed quite often in that the client site is left with a couple of analysts and the development goes on offshore. The mirrored team aims at having all roles replicated on both the shores which encourages more communication amongst the team members.

Documentation
One more thing hampered is the amount of documentation, which increases almost exponentially...

As the customer is sitting elsewhere all the analysts can't talk to the customer face-to-face which adds to the documents and artifacts created. Also for transparency and quicker feedback sake, tools like online bugtracking systems are used.

The whole effort towards reducing the feedback time and working closely with the customer comes with major overheads in distributed contexts.

Cost and Benefits
From a customer perspective, distributed development reduces cost significantly. The organization seemingly doesn't reap the benefits of distributed development in terms of profit margin due to the increased cost of travel, communication, etc.

But the benefit of having a distributed team like:

Better team skillset (as compared to on-site)
Reduced costs (as compared to on-site)
Reduced culture shocks to customer (as compared to offshore)
Improved business context (as compared to offshore) because people living in the same country have definite advantage in understanding the business
General cross-pollination leading to better diversity and a more cohesive work force overall in the organization.

Selling Distributed Agile
There are some important points that we need to keep in mind when selling distributed agile (even agile) to clients. Setting these customer expectations goes a long way in favour of project success. Other than the cost / benefits decribed above:

Expect lots of communication. Mostly in the form of voice communication rather than email
Expect much more participation than any waterfall project your organization has been a part of
Expect having to give feedback right from the beginning of the project.
Expect software to grow (an agile team will rarely go for place holders)

A related and more detailed article by Martin Fowler can be found here

Agility...

2006-04-07T03:53:00.000-07:00

The key practices of agile software development, known and followed are:
1. Face-to-face communication over documentation
2. Quick feedback from end users over project phases (as in waterfall)

Collective ownership, Pair Programming, use of whiteboard, paper, and most importantly the terrific visual representation of the project health and progress... the wall

For an agile team the wall is everything. The action of moving stories & bugs along the columns and the instant snapshot of the project health is fantastic. Here's how it works...

The process :

The Business Analyst analyzes the requirement and writes a story and acceptance criteria
The developers estimate a bunch of such stories in a planning game
The developers (in pairs) then sign-up for a story each morning
When the development is done, the story is tested for desired functionality by the analyst
After the analyst has signed the story off, the tester picks it up
The testers and devs play ping-pong for a while with the issues (bugs) raised by the former
When the tester signs off the story, it's ready for the client to have a look at.

This procedure takes ideally about 3 days per story. The client can then look at the story and sign it off or (ask for some changes) !!!

The documentation per say in this complete procedure is an index card for writing the story, some more for the acceptance criteria and a few more for the bugs.

The client and the analyst discuss the requirements face to face probably using some paper to make rough screen sketches sometimes.
The analyst talks to the dev pair that signed for the card and describes the functionality.
The devs use whiteboards & pair rotation for design discussions, issues and knowledge sharing
The devs keep bugging the analyst all the time for clarifications
The tester talks to the devs about issues and only writes a bug on a card if it can't be resolved then and there.

For anyone roaming around, the color coded, columned wall with the stories and bugs gives a clear picture of the iteration health. And a team can easily setup the wall to show the project health coz it's the team that decides the color codes, the statuses (columns), etc on the wall.

It's the team that decides when to start the day, what coding conventions to follow, the core hours of work and it all works out so smoothly because the client is part of the team!!

Now how much better can it get?

Value..

2006-04-03T07:20:00.000-07:00

Stories being testable and valuable is as important as them being small and independent. Perhaps more... because these are the two important characteristics which in fact drive the size of the the story.

Like we saw that an agile analyst can sometime fool himself by creating independent stories, one can also blunder quite easily when it comes to testability and value of stories. Let's look at an easy example:

Imagine a customer entity in a system with about 50 odd attributes. Creating one story capturing all these attributes will be a real mess. An analyst would (and should) readily want to break the story. What he/she might end up doing, however, is breaking the story thus...

First story for creating the customer domain object, giving values for all the attributes (in code) and saving it to the database.
Second story for creating the screen
Third story for adding validations.

Task & sizewise, this will work fine. But where is the value? What are we actually delivering to the user that he/she can use?

Another problem is the testability. What an analyst should remember is that a story should be testable by the end user... not through code.

So what does one do in these kind of situations? Well, maybe (and I am quite sure) that there are different types of the customer. Breaking the stories so as to capture details for each type can be a great start. This is called a vertical split.

In situations where this is not possible the team might have to resort to internal breaking of stories. The catch is it can be shown to the customer only as a complete functionality, which lengthens the feedback cycle which defeats the whole purpose of breaking the story!!!

These are the situations to beware of... But then that's what we get paid for right??

The perfect split is somewhere out there and finding it gives an analyst immense happiness... and who could help you better at that than the client?

;-)

Chicken and Egg

2006-03-30T22:44:00.000-08:00

Managing dependencies is another tricky issue that plagues agile business analysts. Fortunately it's not as bad as the chicken and egg problem. The secret to successfully solving dependency problems is to start at the beginning... Well maybe an example will help.

Let's consider a system that sells books online. The roles which will interact with this system, will be
- the buyer
- user registration
- the administrator
- the sales team and
- the warehouse manager.

The most important entity in this system in undoubtedly the book.

Starting to write stories for creating a book is an obvious start for anyone. Now this is where the situation gets tricky. If all your initial stories are going to be related to books, they are inevitably going to be dependent on each other. For example :
View book details depends on Listing books which depends on Create book

Writing such stories doesn't make sense from a tactical perspective when we are doing iterative development because we can't develop such stories in parallel.

Instead if we decide to write a Create Book and a Register User story, we are suddenly in business. What this means is that we can start separate areas in the application simultaneously.

Sometimes however things are not as simple as that. There are various restrictions which can lead an analyst to write stories in the same area. What will typically happen is the analyst will write Create Book, List Book and View Book Details stories and get them played with valid assumptions and dummy data. The analyst has thus created independent stories... The consequence is the cost of integrating these pieces together.

There are several points in the application where everything comes together to form a complete picture. Integration of separate stories into a logical area and separate areas into a seamless solution is inevitable. What we need to understand is that reducing these integration points is optimal in terms of overheads.

Thus an analyst needs to take a call in consultation with the developers about finding v/s creating independent stories. Finding them is however definitly advisable.

If I were write user stories for evolution I think I would have developed the Chicken and the Egg seperately and integrated them in the end. Probably that's how it was done anyways!!!

Small Stories, Large Stories

2006-03-27T05:30:00.000-08:00

One question that keeps analysts busy for a most of the time is whether or not to split a story. Here's my philosophy behind my unending support for small stories.

Let's start with one huge story that covers everything that is required to be done in a project. It is estimated by the developers at say 120 complexity points which converts to about 300 days. Following are the consequences.

A requirements document (the story) - coupled with the time taken to prepare such a document
The techinical discussions and decisions for implementing these and of course the time required
The implementation of this story
The testing and
The user acceptance (well usually non-acceptance and it takes very less time too :p )

This is what we do in a waterfall approach towards software development. We are looking a 300 days feedback loop because that's when the customer will actually see the application.

Agile works towards shortening each of these phases and doing them over and over again until all the requirements are fulfilled. Thus everything boils down to the feedback cycle you are looking at. Let's look at a shorter feedback cycle, say 100 days. Thus you end up with 3 stories with an estimate of 40 complexity points converting to 100 days. Which means you get feedback thrice during the project because the customer reviews the functionality delivered by each 100 day story as soon as it is completed. That's somewhat better.

Let's just try and make this a 10 day feedback cycle. We thus have 30 storys with an estimate of 4 complexity points converting to 10 days. What we are looking at is basically a 2 week iteration. Now that's really good. I'd however go for one more split so my stories have a complexity point estimate of not more than 2.

The upside
The upside other than what we established above is that developer estimates are generally exponential (although they are supposed to be linear). Which means that one 4-point is not equal to four 1-point stories. It is exponentially bigger. Thus the smaller stories you have the lesser will be the actual time taken to implement, test, get feedback, make necessary changes and deliver.

The downside
Well, there is no downside. The only thing required is constant interaction with the customer for analysis of new stories and review of completed stories.

Todo
An agile team should set customer expectations appropriately before starting the project. The customer participation is the heart of the project. Successfully establishing a short, reponsive feedback loop is the first and foremost thing to do when doing an agile project... and the stories, keep them short.

Small is beautiful...

Once upon a time...

2006-03-24T05:54:00.000-08:00

User stories are the basis of business analysis in an agile project. A user story is an artifact which captures a small but valuable piece of business functionality which can be implemented and tested independently.

The philosophy behind agile methodologies is basically iterative development with short & fast feedback cycle which is used in improving the solution and accomodating changes (subject to priority) so that the end result is really what the customer wants. Inevitably this creates a unique position of a business analyst as a broker between the customer and the developer. Writing user stories is what this broker spends most of his time doing.

There are numerous ways of writing user stories but the goal is to capture a "role", an "action" and an immediate "goal".

Like:
As a Business Analyst
I would like to write my stories in "As a, I'd like to, so that" format
So that I can capture the role, action and goal easily

Ok, that's not an awfully great example... but the general idea goes thus.

User stories should be

Small
Independent
Testable and most of all
Valuable to business

and of course the next question is why.

This leads us to look at the meaning, purpose & priority of each of the above.

Valuable
The user story should deliver a valuable piece of business functionality. This is important so that the story can be released and used by the real users as soon as it is developed, thus giving the team a faster feedback.

Independent
Independence is a very important characteristic of a user story. When a story is independent, it can be picked up for development anytime during the iteration. It can deliver a valueable business functionality independent of the rest of the feature.

Testable
A story needs to be testable. Which means that the desired functionality that a story aims to deliver should be capable of being demonstrated, for all related business scenarios.

Small
A user story should be as small as possible... possible keeping all the above characteristics in mind. You can choose to split a functionality into shorter stories as long as it remains Valuable, Testable and Independant.

This is the primary job of the broker and what's challenging about it is what I'll be posting hereafter alongwith my version of the solution.