Becoming an Agile Business Analyst

iAnalysis or rake analyze

2009-03-06T19:53:00.000-08:00

Why isn't there something like Intellisense for analysis yet? Surely we've mastered the patterns involved in analysis so far :) Why can't I type a user story and hit ctrl-spacebar and have a series of analysis methods pop down for me to choose or even override? hmmmmmm.

What are we building - documents or software?

2008-12-11T21:17:00.001-08:00

One of the people I like to read and get insight from is Scott Ambler. One of the things he and several other thought leaders in the Agile movement point out is that the end product we are building is software, not a document repository (unless you happen to be building some software that is actually a document repository.. but I digress). I think this thought is generally accepted as part of the agile movement (we value working software over comprehensive documentation). That has been a value of mine now for several months and has been shaping the way I approach business analysis. The important thing to remember here is that we aren't tossing documentation out the window, but we are getting rid of the wasteful documentation that doesn't help get the software developed.

So what do I do with all this free time? If you read up on some of the things Scott mentions, I've picked up some old skills of mine when I was first doing development and help my team out writing unit tests. We aren't doing TDD yet, which is a specific pain point for me because I know how much it cleans up your code and reduces complexity, but these unit tests are giving us some assurance that we aren't breaking code. Another thing I've been doing a lot is reading, which has proved fantastically rewarding. I would recommend a recent book by Andy Hunt that I just read, "Pragmatic Thinking and Learning: Refactor your Wetware". This book is a really good primer on thinking in a different way. If you happen to major in philosophy like I did, you'll see a lot of things that you probably read from Phenomenology, Husserl, Heidegger, and some of the ancient philosophers.

Working for work's sake

The idea of producing less documentation still bothers me from time to time though, not because I think this approach is wrong, but I still have that Industrial age mentality that imposes the mindset that I must always be producing widgets all the time. So I still have to stay on my toes to continually help the software come into being, while making sure I don't fall into the old trap of writing documents now on something that I won't work on for months.

Whats wrong with "getting ahead"?

There are two major pitfalls with creating large amounts of documentation about something that won't be coded/implemented for months.

What you document today will ultimately change (emphasis on will) by the time you implement it for the users to benefit from it.
What you document today most likely won't contribute to what is being built right now. Yes this is tied directly with point 1 as well.

So why is this? For both points, we all know that requirements change, and as an agilist you welcome those changing requirements. However, if you spend time and effort writing documentation now, then that is ultimately wasted when it comes time to implement it because the requirements have to be "re-written" or updated because they aren't what is needed now. Also, that time you spent writing the documentation then prevented you from helping your team on the current software iteration/increment now.

Balance this with Domain Knowledge

There is the idea of domain knowledge that is sometimes regarded as beneficial documentation. I would agree on this with a caveat. That documentation needs to be somewhere that everyone sees readily and quickly - like a Wiki or on whiteboards/Post-It note boards in the room. It should not be secured by the analysts only to be brought out when the sprint is going to implement it, the team should be learning the domain knowledge along with the BAs, though the BAs probably take on more of a coaching job of the domain knowledge to the rest of the team.

Software development is about developing software ultimately, don't waste your time or your client's time by creating documentation that no one will read and that won't help your team.

Should Agile be hard?

2008-12-08T20:02:00.000-08:00

Agile software is supposed to help us deliver quality software that the customer wants as quickly as possible. There are series of twelve principles that have been put in place to help form the idea of agile software development. Shouldn't we be able to utilize this information easily to create software? It depends.......

I find that the principles that make up agile software development are helpful in generalizing what we need to aspire to. They don't give you any specifics really, just beneficial guidelines that should help you get there, something like a small flashlight to help guide you down the path just far enough.

The problem is that if the people involved in an agile project don't understand the meanings behind these principles, they aren't of much use. I think this points towards what you'll find in the Dreyfus Model of Skills Acquisition regarding people's skill in either software development or whatever trade they have that will be combined on an agile software project. The following may or may not make sense:

Agile projects that combine people with skillsets of varying degree might be successful as long as the variance is within the skillset, and that skillset has some people at the proficient level (or competent level at the very least but this introduces a lot of risk).
Agile projects that combine a group of people with a certain skillset where no one has the skillset above Advanced Beginner are in danger of failing.

I say this because I'm starting to believe that agile software is extremely dependent on skilled "artisans" that understand what those 12 principles mean. Those 12 principles are understood within context far better than they are if just considered by themselves. Someone in the Beginner or Advanced Beginner skillset may look at those and take them as hard and fast rules that are applicable everywhere (Best Practices anyone?). Even those in the Competent skillset are in danger of doing this. However, when you crossover into the the Proficient and of course Expert skillset, you don't understand these as hard rules that must followed rigidly in all areas, you understand that things exist in context of something, and thatthey need to viewed in that light to be understood.

So expand this past just the development team, I think we understand what happens when you have a group of junior developers and you want them to try agile with no one versed in the skills that a proficient or expert developer can explain. Apply this thinking to the management team- what happens when you have a group of people in the management role that are fairly unskilled in it. I think this lends to creating environments that are unsafe for agile to be practiced and crafted. To me, people with management styles that are still rooted in the industrial revolution mentality (people are widgets or resources and can be replaced easily) are bound to doom an agile project because they aren't prepared to treat the team as a group of skilled craftsman, but instead as a group of people that replaceable. If you have a management team that contains those at the Proficient skillset, I believe they tend to lean towards management practices that let a team strive to meet a goal, and accept failures while expecting success. Proficient and Expert managers tend to understand that it is much more valuable to have people that are motivated to learn and to do something than it is to have people who only do their job out of fear of what you may do to them. Where a beginning manager may see people as subject to rules that they have encountered in management seminars, proficient managers see people as unique individuals living within unique contexts that don't respond the same way to one stimulus or teaching.

Anyways, I've rambled on long enough... My point to all of this is that if you have the right team setup that is geared towards Proficient skillsets, then agile will probably be easy because chances are you'll probably already be doing it. If you don't have that mix, then expect a bloody struggle and possibly a remake of the Myth of Sisyphus.

Long Delay

2008-12-08T19:46:00.000-08:00

Sorry for the delay, but after attending an agile conference in Orlando back in November, I really had my assumptions and understanding of agile shaken to the core. I find it hard to call what I was doing before as agile, but more like FrAgile. I heard of agile, knew itwas probably good and was striving towards it. My only problem was that I was really focused on the process of Scrum to be my "agile" instead of trying to understand what it means to "be agile".

Of course, I now welcome failures like this. And I don't think of failures as really all that bad (unless you happen to be performing surgery on me at the moment). Falling flat on my face has always been the most rewarding experience for me because I never stay down. I always get up, turn around to see what tripped me, and keep going again and avoid what tripped me.

Anyways, what really kicked me over from the conference was my realiziation that I was focusing too much on doing scrum as a process, that I neglected what should have been my goal - delivering quality software to my clients. I just assumed that if we followed what the scrum books said then it would naturally follow.

So when I got back I began to focus on the principles from the Agile Manifesto (http://agilemanifesto.org/principles.html). From there I've tried reading more theory about agile and its principles and less about how a process can save the day. I'm still battling with how to be a BA, but I'm less focused on the role of BA now. I'm debating the use of roles on an agile team (in some contexts they are useful, in others maybe not so much). I'm really getting interested though in how I can help my team complete their tasks by helping them get the information they need as quickly as possible.

More to come... I want to write something else now.

Requirements: Just in Time - Part Three

2008-10-30T19:56:00.000-07:00

Iterative Requirements.....
How does one accomplish iterative requirements? I almost wonder how one cannot accomplish this, as the process for gathering requirements is in itself iterative. Version 1.6 of the BABOK hasn't covered this yet, though maybe future versions have. As the process of business analysis continues, the analyst is continually refining (refactoring) his/her requirements document each time they meet with stakeholders for a project. The big jump though from the waterfall process to the agile process is that we don't spend code-less iterations doing this prior to code actually being written. We must perform smaller iterations of requirement elicitation concurrently with development.

Here's the big stepping point.... instead of the normal effort spent way up front attempting to discern with infinite precision what the software will do, we are now doing it at the same time the code for it is being written. This gives us immediate feedback about the requirement/product and quickly tells us if we are failing or succeeding.

I'll admit this is a bit of a roller coaster ride as you know very quickly if the requirement you covered was done successfully, or if a developer/qa person uncovers something you didn't think of. -- As an aside, this is the kind of thinking I have to move away from, you'll never catch everything in a requirement. -- As opposed to the waterfall process where you spend a great deal of time basking in the thought that your requirements perfectly explain how a system will function, until you see months later that instead of a perfect billing application, you have a login screen, and two page to enter account information on.

Requirements at the speed of development
Most analysts I know are extremely anal about their requirement documents. They will spend hours upon hours making sure every aspect is covered in excruciating detail so that in their minds it makes perfect sense - so much so that even a developer would understand it! This makes the move to agile for most a painful one in my opinion. Mostly because it brings the developer into concert with stakeholder, no longer placing the wall up between the two. It also makes it painful because we don't have these long timeframes to write and Re-write (using WordUnit of course http://www.waterfall2006.com/beck.html) requirements. The transfer of knowledge happens much more quickly, coming from the stakeholder directly to your developer/analyst.

So how do we accomplish requirements so quickly? I would venture that we have to stop thinking of requirements in the same way that we have for the past several years. The document that used to hold pages upon pages of value mappings, or data constraints, etc., are no longer very useful. What is useful is a database that has value mapping and data constraints. As opposed to the mammoth SRS that used to hold these and never get read, the database would hold these and would constantly be read. If development and the product owners are constantly meeting with each other, then that long document is no longer needed, just the database that does what the document was going to tell it to do.

It is getting late, but I think I'm going in a solid direction for now. I'll pick this up later after I've made my next big mistake. :)

Agile QA

2008-10-12T08:46:00.000-07:00

Before I begin this post, let me point out that I believe QA must exist within Agile. My discussion here is on how I think it should exist if it is expected to be agile.

As every BA knows, he/she will undertake QA to some degree. This is very clear in the waterfall process as the BA is often tasked with writing the tests that will usually be performed and ultimately automated by the QA team. In waterfall, this happens very distinctly at the end of development. Once developers finish what they set out to write, it is tossed over the wall to QA, and the waiting for bug reports ensue. Its a very well laid out process, and has successfully helped delay and hinder projects along with all the other steps of waterfall. Unfortunately, there is very little written on the web that discusses this so I'm tossing an idea out there.

When I look at the QA process in my curent project, it is unfortunately still bogged down with the waterfall process. We have really good QA testers, but we are still following a process whereby a lot of documentation is created up front about what will be tested, and then testing it after the software is developed. While we have focused our attention on the rest of the team to incorporate agile into our process by doing daily builds, increasing discussion with the product owner and building our 'self-organizing' teams, we are still in the mindset of 'tossing' a finished development effort over to QA to find bugs. Even though we are doing daily builds and handing over software iteratively (finished to a point but not yet complete), we are still not getting QA in the loop as soon as I think they should. This process seems wrong, both from an agile standpoint, and from an effort to improve software as we build it.

Think of it this way: If there are people on your team that are responsible for the quality of a product, you would want them involved at the very beginning, right?. By this I mean that their expertise and input are gathered at the beginning as development starts writing code, and is continuously sought after during the development process. What this does, I believe, is give the team the direction and insight that they need to build the product that will stand up not only to what the product owner asked for, but will also become a piece of software that the product owner wants, and will do that much sooner than if we tossed it over to be tested.

Imagine these two scenarios:

Scenario 1:

Team begins development on a product by reading the user story and clarifying the acceptance criteria with the product owner. The teams begins developing based on that conversation, and asking the product owner for clarification along the way until they finish writing the code that they think satisfies it. Meanwhile, QA testers are preparing test scenarios and cases that they intend to exercise against the finished version of the software. During these two seperate threads, neither group really talks with the other about what they are doing. Once the software reaches testable phases, all of that is handed off (that should be a flag right there) to QA so that they can begin testing it and seeing if it meets their perception of the acceptance criteria + their QA insight. QA begins testing it and finds all sorts of issues that those developing it didn't really think of (not because they are bad codeers, but because they think in a different way). These issues are then worked on, sometimes argued about, sometimes deferred till later, sometimes fixed, and then re-tested.

This scenario is what I see as a phenomenal waste of talent within the team. If this is the process we follow, then I wouldn't consider the team very agile but instead falling for into scrum-fall process that will lead to problems.

Scenario 2:

Team begins development on a product by reading the user story and clarifying the acceptance criteria with the product owner. The team then decides on what they will start building and gets further input from those that will be testing it. Those people on the team now know what to build based on the acceptance criteria + criteria that will be put into internal tests to ensure quality. As the software is built, further conversations take place for clarification both with the product owners and QA, and the sofware is built to those expectations (Note that I have not said that what QA wants to test is gospel, that is part of the conversation). As the software reaches testable phases, QA runs thier tests against that bit of functionality. When a test does fail, those who develop it more likley know exactly why and what was missed and can fix it (assuming the test was correct).

The difference in this scenario is that at this point those tests should be more of a formality since development was geared towards making them pass from the very beginning. Thus, the amount of QA's valuable time is not wasted testing for things that the developers never realized they would be expected to build in the first place. The time spent up front in the conversation is very minimal when compared to the rework that may be needed if end the tests are not known ahead of time.

Think of it as though you were in school (to borrow Mike Cohn's analogy of acceptance criteria). If you are studying for something that you will be tested on, you want to know what the test will ask so you know what to study. Knowing that, you are better prepared for the test because you geared your study habits successfully to answer those questions.

This scenario gets us closer to a more collaborative team environment. I also believe that it helps improve on the first scenario's waste by letting developers know what they will be tested on ahead of time so their effort takes that into consideration. At the end of each iterative step, less time is spent going back and forth between QA and development because it was mitigated in the beginning. I don't see this as the silver bullet, as QA will certainly come up with testing scenarios as the software is developed that they may have not thought of at the beginning of development, but it should get us a great deal closer to a more agile process.

I still fully expect the role of QA is to make sure that the client gets not only what they asked for, but what they want. I also expect that QA should try to break the code as much as possible. What I'm advocating for here is a more collaborative approach that makes sure those who will be subjected to a test, know exactly what they will be tested on from the very beginning instead of getting surpised at the end.

Requirements: Just in Time - Part Two

2008-10-12T06:10:00.000-07:00

In the first part of this post, I tried to walk through what I was thinking about requirements gathering/elicitation in the agile sphere. I noted then that it was subject to chanage, and I'll reinforce that note in this post I'm sure.

As I discussed in the 3rd part called "Detailed examination of the requirement", I talked about when it should be be done - as close as possible to the sprint that the product owner wants it to be implemented. That satisifed my post's title. Now I'd like to talk a little more about what is done for requirements gathering/elicitation as the BA (that is after all, our traditional bread and butter).

When I first started out trying to learn agile, I managed to probably make every mistake I could make, or at least put a significant dent in them. The best one was at the beginning. I had just come from a waterfall job, so I was still in the frame of mind that tomes of documentation were a good thing even though the agile preference found value in less of them. So while I knew that the developers would be working from User Stories that barely fit on a notecard, I was convinced that all Agile BAs had a dirtly little secret that they kept detailed requirements hidden away somewhere that backed up each of the user stories (you know the kind... Section 1.A.1.a.ii). As I began asking around the internet of other Agile BAs, I quickly found out that they weren't keeping that secret, that they weren't in fact keeping large amounts of detailed documentation. What they were doing was documenting acceptance criteria in place of those requirements.

To top off the acceptance criteria, some retain artifacts such as use case models or workflows to better describe the acceptance criteria as well as the overall "vision" of the product. This minimal documentation along with the developers communicating with the product owner, does more for a developer than a 75 page SRS can do. This is because something like a simple use case model or workflow diagram gives the developers context, and the acceptance criteria layout what happens within that context. The additional discussion that takes place with the product owner gives even more insight as to what the user story will require.

So all of this is to say that instead of documenting some missive that explains every exact detail of the system, requirements should now come in the form of a simple User Story statement with the acceptance criteria and perhaps a model or two. That's barely sufficient to get it done. The in-between stuff like implementation details, UI functionality, etc. can be obtained through direct dialogue and documented in Unit Tests. This brings up another point.. Things like implementation and UI probably shouldn't appear in the acceptance criteria. I say that for two reasons: 1.) because sometimes acceptance criteria are written way ahead of time, and 2.) you will often find better ideas of implementation and UI come from discussions between those that will develop them and those that want them (and yes, the BA is part of that dicussion quite often).

How a BA arrives to that User Story and acceptance criteria+models is the hard part to migrate to. I don't necessarily think that this is done using traditional requirement elicitation techniques, at least not all of them. It can, but something tells me there is a better way, and I think it will look more iterative in nature. The argument can be made that SRSs are iterative (just look at the change log at the beginning), but I'm thinking much smaller iterations still. This is the topic I will explore in Part Three.

How does a Sprint feel?

2008-10-03T20:40:00.000-07:00

I heard an interesting metaphor to describe how it should feel each time we begin a sprint.

It should feel like you are on a high diving board about to jump into the pool.... and you aren't sure if there is water in the pool or not.

Spiked Punch?

2008-10-03T20:21:00.000-07:00

Every now and then, a team is bound to run into new technology that they haven't covered or had experience with. In Agile teams, this is bound to happen frequently as we are constantly writing software and not wasting time reading requirement tomes.

So what happens when you don't have an SRS with 60 pages of tables detailing what occurs with a new technology and how a developer is supposed to learn it? Let the developer learn it of course. The developer after all is the best person to do so, and why waste the valuable time of a BA who won't necessarily know all of the intricate things that is pertinent to a developer? There is no need for the BA to run off on their own and learn something for a developer, they are the master craftsmen for that trade and will be better utilized in learning that endeavor (generalizing specialist aside for now - I'll talk more about those in a later post).

For the most part, this is accepted practice in Agile. The problem for me came in our last Sprint Planning meeting. We were presented with a story that asked us to deal with a new technology. Unfortunately, we spent too much time in the planning meeting discussing how this technology worked until we finally realized that we couldn't do it in a 2 week sprint. At that point it finally dawned on us that we should do a spike.

We spent way too much time coming to that conclusion in 'retrospect'. There were early signs in the conversation that we didn't know what to do (it could be argued that we saw these signs before we started the planning as well). At those initial signs we should have posited the idea of doing the spike and moved on to other stories. The resulting mayhem ensued when we realized that the user stories we had intricately laid out were done based on us doing the initial story - so we had to redo some of the stories that depended on that one actually getting implemented.

Thankfully, we have been following solid code practices and had the ability to follow a seperation of concern and build outside of it.

Requirements: Just in Time- Part One

2008-09-27T14:15:00.001-07:00

This one is going to be big.

So one of the the things I've had difficulty getting a full grasp on is the actual requirements gathering. It might not be that I'm having trouble grasping it, but rather it might be my interaction with the interested parties that is confusing my understanding.

Let me explain: In the scrum methodology there is this concept of the product backlog and the sprint backlog. In the product backlog are user stories that have vague acceptance criteria. I understand these are usually created from meetings with the product owners where they write vagues stories such as :

"As an admin, I need to be able to view audits of user's searches, so that I can know what they are searching for" (best case here with the "so that").

The acceptance criteria from this early meeting may be something like:

All user's searches are audited by the system.
I can search all users's audits.

So that sits on the product backlog until its priority reaches Release planning. The developers are now expected to say that the story is a certain story point size, so that it can be calculated into the team's current velocity, so that the release date can be either determined or to know if we will go past the release date.

A few weeks later that story is now ready for the sprint and it is placed on the Sprint backlog. The developers now have to communicate with the product owner to find out the details of this user story. This most likely leads to a revision of the estimate done at release planning because they'll find out more from the product owner at this new date, and their own domain knowledge has increased.

In this light, I view requirements gathering/elicitation as a vastly different beast within agile. It is iterative, just like the functionality that is delivered. I see requirements in the agile sphere as following this type of pattern*:

Initial discovery of the requirement
Cursory examination of the requirement
Detailed examination of the requirement

Lets look at each iteration:

Initial discovery of the requirement

This happens when the requirement is first discovered, whether through interviews, meetings, whatever initial form of requirement elicitation is performed. This typically only occurs as a brief mention of it, but can also occur if you have a product owner willing to sit down and tediously go through processes that we try to map. The initial discovery can occur at the beginning of the project, or the first day of a sprint. The result of this isn't always a User Story, but at the very least it is a note to the BA to follow up on this requirement.

Cursory examination of the requirement

Now that the requirement has been identified, the BA has to learn what it really is. For example, its one thing for a product owner to say they want to upload files, it is a vastly larger thing when the client means they want to extract all possible metadata from those files upon upload. So the BA must now pick a moment to meet with the product owner and go over this requirement to figure out if it is something new, or something that they've already gone over before and phrased differently. Upon discovering that it is new, they now have to embark on two paths - 1. Find out more about the requirement at a high level (usually documented with the help of use case modeling) and 2. Find out where this fits in with everything else in scope and priority (ie. where in the product backlog). This stage will usually result in user stories, and they will probably be epics at this point.

Detailed examination of the requirement

Once the BA has identified where the requirement/user story fits in the product backlog, they will probably need to spend some additional time with the product owner to find out the details of these user stories. This should be done as close to the sprint that it is intended for, or at the beginning of the sprint (see related Last Responsible Moment). The reason for this is agile in scope: the one thing that waterfall has taught us is that what we think we want the software to do and what it does when it is implemented much later are two different things . Delaying the detailed requirement gathering until right before the sprint enables the product owner to be fully aware of where the product is and what it is capable of right before they try to implement their new functionality. If they lay that out ahead of time, it runs the risk of coloring their perception of what it should be based on whatever was known at the time, and when the sprint comes to implement it, they find out they wasted their time and the BA's time and must now change what the user story was or perhaps even get rid of it.

This seems beneficial to me for making the transition, I hope it helps someone else.

*I expect this to evolve and change over time.

When your eyes are bigger than your stomach

2008-09-26T14:38:00.000-07:00

So I've been working towards agile now for about 5 months. I've had an incredibly exciting time learning all about it and growing as an anlayst. There is much "theory" to learn about agile, as it isn't an easy methodology to adopt initially. This is especially true for those coming from the waterfall background.

Anyways, we are about half way through our first sprint in our second release, and we've now realized that one of the User Stories was way more complex than we initially anticipated. This has happened to us before, but I believe we mis-handled it by having everyone working tons of overtime trying to finish it before the sprint review - which affected the quality and morale of the team.

So this time, I'm suggesting a new approach, one that I feel is more agile. I'm connecting with the product owner at our earliest realization that this story's size will affect our ability to finish other user stories. I'm hoping that we'll be able to split another story in our sprint so that we can finish this one with quality, and still deliver at least some of the other story in this sprint.

My theory here is this:

Estimation is always an estimate, its never accurate. Because we don't waste countless time gathering anal requirements ahead of time to narrow the estimate risk, we are bound to encounter stories that surprise us once we start them. As a result, we are more susceptible to these suprises. I would expect that this is a common occurance in young agile teams, less so in mature teams because your estimation skills improve with time. My expectation is that a user story will be split along a functional boundary, and placed in the product backlog, so we can deliver the quality functionality by the end of the sprint.

Update:

In a true and agile fashion (I'm learning as I go) we are proceeding with completing the user stories for the sprint as best we possibly can. We will attempt to follow what Jeff Patton has talked about before regarding "thinning" user stories if we can't complete the last one fully. We'll shoot for completion of the full sprint, but this gives us a secondary target to hit if we don't make the first.

There once was a man from Nantucket..err Texas

2008-09-26T05:07:00.000-07:00

So I'm trying my hand at blogging, along with 9 bazillion other people in the ether. I'm attempting this because as a business analyst, I'm finding a few resources on doing analysis in an agile setting, but nothing that really helps me make the transition from a waterfall model to the agile model.

I'm with a startup company now that is essentially adopting agile cold turkey. So as long as we are around, I'll blog about the transition in the hopes that this vein of business analysis can expand, and really so I can improve my own ability to do this.