Do It Yourself Agile

2009-07-07T11:20:00.002-04:00

Originally, my blog was focused on Software Configuration Management. Then I described in a blog post how I realized that I'm not really an SCM person, I'm a software development process person that had been focused on SCM. So, I changed the name of the blog to "Agile Development Thoughts." Today I realized that this blog has changed its focus from random thoughts about Agile Development to a blog about Agile for the Agile DIY'er. So as of today, this blog is now "Do It Yourself Agile."

I still believe that the quickest route to Agile success is to get people involved who have done Agile before. But, that option is not always available and even when it is, it may not be possible to retain an Agile coach for the multiple years that it will take to reach your full Agile potential.

So, the idea of this site is to give the Agile DIY'er a resource to lean on. A good place to start is the web-only book "Do It Yourself Agile." This is an evolving book which is expanding and changing every week in response to questions and discussion from readers like you. If there is a topic that you don't see covered or that you would like to explore more fully, let me know! Just post a comment wherever you see fit or use the Q&A page. I'll answer your question or create a new blog post on the topic as soon as I can.

Check out the web-only book "Do it Yourself Agile" .

Agile Exposes Scaling Problems

2009-07-06T12:42:00.002-04:00

One of the first questions that seems to come up in any discussion of Agile is "How Well Does Agile Scale?" Sometimes this is asked explicitly, but more often there is an underlying assumption that Agile does not or can not scale very well. When I was first exposed to Agile, my first impression was that Agile didn’t scale beyond a small team, say 1-12 people.

I used to try to tackle the question head on, but then I realized that there's something else going on here. Let’s take a step back. What do we currently assume about traditional development? I think we come to the discussion thinking that just because there are teams of 100 engineers, 500 engineers, and even 2,000 engineers doing traditional development, that traditional development is a proven quantity when it comes to scaling. Let’s first ask the question: “How well does traditional development scale?”

To answer that question we first have to define "scaling." I think a good working definition is that as you add new resources, there is a linear increase in effective output with little or no decrease in efficiency. For software, output and efficiency translate to new functionality and productivity. That would mean that if you have a 50 person engineering team and you add 50 more people you get twice the output. But when was the last time you saw that? Have you ever seen it? In my experience, after talking to hundreds of development organizations doing traditional development, productivity falls and thus output does not increase linearly with additional resources. In many cases, output actually decreases as more resources are added.

What do you actually do to "scale" your development organization? Do you have meticulously updated Gannt charts and estimates? Do you schedule lots of meetings? Do you spend a lot of time making sure that requirements and designs are just right? Do you reserve 30% of the development schedule for testing (and fixing)?

In my experience, after talking to hundreds of development organizations, the answer to the question “How well does traditional development scale” is “not very well.” We've just suffered along with this problem, in large part because while the pain was there, the root causes were difficult to trace, let alone address.

The main point here is that if you are in the process of rolling out Agile to a large organization, don't be discouraged when you run into scaling problems. The problems were always there, they just weren't as obvious. Now, instead of wondering why things aren't coming together as expected at the end of the project, you may find out right away that your organization doesn't really have a good way to coordinate API changes for APIs that are used by multiple teams. As Agile exposes problems like this, you can take steps to solve the problems and thus create a more scaleable development organization that just happens to be doing Agile development.

Is Agile Really Any Better? Free Talk with Pizza and Beer

2009-06-02T20:49:00.002-04:00

If you will be in the Boston area on June 16th, I'll be talking about how and why Agile works, pitfalls to watch out for, and why it goes well with beer and pizza. There will also be an opportunity to learn how AccuRev helps in an Agile environment. You can read more details here.

Ken Schwaber to Speak on "Flaccid Scrum"

2009-05-23T01:28:00.001-04:00

Come hear Ken Schwaber's talk on "Flaccid Scrum" at the Agile Bazaar in Burlington, MA on June 18th. Visit our web site for more details.

How Continuous Integration Promotes Healthy Agile Practices

2009-05-14T17:26:00.002-04:00

Come learn "How Continuous Integration Promotes Healthy Agile Practices" in a webinar I'm doing for Version One at noon EDT on June 17th . See you there!

Abstract: Healthy, successful Agile teams rely on a robust Continuous Integration implementation as the hub of their Agile infrastructure. Many of the Agile practices depend on Continuous Integration (CI) in order to provide their full value. For instance, unit tests are the most valuable when all unit tests are run automatically on a frequent basis. Scaling Agile to large and/or distributed teams also depends on CI. Conversely, CI can be used as a leading indicator of the health of your Agile teams. This session will cover best practices for transitioning to Continuous Integration as well as show the strong link between CI and one-piece-flow, unit tests, short iterations, whole teams, refactoring, user stories, shared code ownership, distributed Agile, and large scale Agile.

Product Managers and Product Owners - Newton MA, Thu Feb 26th, 6pm

2009-02-22T09:38:00.002-05:00

Rich Mironov of Enthiosys will be speaking on "Product Managers and Product Owners: Which Do We Need, and What Do They Do". Hosted by the Agile Bazaar.

For more details, please check out the full meeting announcement.

Poll - Short Iterations, The Benefits

2009-02-10T12:51:00.006-05:00

Short iterations are the practice of breaking up a release into 1-4 week development cycles that each produce a shippable increment of work. Some of the benefits of short iterations include working software that you can show to people in order to get their feedback and having nothing in progress at the end of the iteration so you can change your plans to take into account market changes, feedback from customers, or information you discovered while implementing that iteration. Also, all of the tests for the functionality implemented during the iteration have been written and run and you have a well known starting point for the next iteration.

There are some assumptions hiding in this practice. It assumes that you believe that incremental design is possible and that you believe that any refactoring you have to do in subsequent iterations is worth the benefits as described above.

So, if you were starting a company with your own hard-earned cash, would you use short iterations or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Whole Teams, The Benefits

Poll - Product Backlog, The Benefits

2009-02-10T12:46:00.005-05:00

A product backlog is a list of high-level product requirements (or user stories), prioritized by business value (typically determined by a product manager or product owner). It contains entries for any and all work that is performed, including bug fixes. As new requirements are gathered or submitted, they are inserted into the backlog at the position that best represents their business value with respect to requirements that have already been entered. As requirements are met, they are removed from the product backlog.

A backlog gives a clear context to discussions of priority. Anybody can go and look at the backlog to see the position of something they have a stake in. If it is lower than they think it should be, they can examine the backlog to see what is higher in order to help frame a discussion around either moving it higher or moving other things lower.

So, if you were starting a software company with your own hard-earned cash, would you use a product backlog or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Your Preferred Methodology

Poll - User Stories, The Benefits

2009-02-10T12:37:00.004-05:00

User stories are a description of what is needed from the user’s perspective. User stories help to separate business value from implementation and focus all parties on the desired outcome.

User stories are different than requirements. When using requirements, it is likely that the developer implementing the requirement will be presented with an implementation task or a design document and be constrained to implementing as specified or as designed. A user story removes invisible constraints by focusing on the outcome desired by the user. The developer doing the work will see the user story, will be able to better understand what the user needs, and will be able to participate in or even own the specification and design of that story. User stories provide engineers more freedom to utilize their creativity and ability to innovate without the risk of implementing something that the user doesn’t want.

So, if you were starting a software company with your own hard-earned cash, would you use user stories or not?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Product Backlog, The Benefits

Poll - Collocation, The Benefits

2009-02-10T12:34:00.004-05:00

Collocation is simply having everybody on a whole team in close proximity to each other. This compounds the coordination benefit of whole teams. This is not related to using or not using outsourcing. Whether you are outsourcing or not, collocation only refers to whether your whole teams are sitting near each other.

So, if you were starting your own software company with your own hard-earned cash, would you use collocation or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - User Stories, The Benefits

Poll - Whole Teams, The Benefits

2009-02-10T12:21:00.002-05:00

A whole team is a small group of people (no more than 12) that work together towards a common purpose, primarily spend their time as part of the team, and as a team have all of the skills they need in order to be self-sufficient. These skill sets may include server side programmer, web designer, tester, technical writer, project manager, etc. The intended benefit is that you spend less time waiting for other groups and bringing part-time participants up to speed, you lose less time due to communication delays, and individuals spend less time multi-tasking between multiple projects.

So, if you were starting a new software company with your own hard-earned cash, would you use whole teams or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Collocation, The Benefits

Poll - Unit Tests, The Benefits

2009-02-10T12:12:00.005-05:00

Unit tests are simply tests that exercise small amounts of isolated functionality. That is, if you have a function that adds two numbers, instead of depending on running a user function that eventually calls the function, exercise the function directly. This often requires the use of mock objects that pretend to be things that the function needs in order to test the function in isolation from other functions that it depends on. Unit testing and refactoring are often done hand in hand.

The cost of unit tests is in writing the tests themselves and refactoring code as new functionality is introduced to keep the unit tests testing at the right level. The benefit is that you can easily test changes quickly to find simple problems before doing more thorough and slower testing. It also provides a good safety net for refactoring, gets developers more involved in testing, and usually improves the design of the software.

So, if you were starting a new software company with your own hard-earned cash, would you use unit tests or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Stand-up Meetings, The Benefits

Poll - Incremental Design, Possible?

2009-02-10T12:08:00.004-05:00

Incremental design is the practice of designing as you implement. You may do some upfront work to sketch out the basic ideas and groundwork, but don't have a fully fleshed out design up front. The basic idea is that you only implement what you absolutely need to satisfy the user-centered work that you have committed to for the current milestone/iteration. That way, you don't spend time designing for work that you never do.

If you aren't doing incremental design, don’t you end up having to make changes to your software during the next release to take into account all of the feedback and requests for new features that you get from customers? Don’t you have to rework the software (including the design) anyway? Aren't you already doing incremental design on a macro scale anyway?

One of the points of incremental design is that design changes are inevitable and that you are better off writing your code in ways that make it more amenable to design changes. For instance, unit tests encourage the creation of mock objects and the use of mock objects encourage the creation of abstraction layers which as we all know are a good thing.

And let’s not forget that dirty little “major overhaul” secret. You show me some software developed without incremental design that hasn’t had a “major overhaul” and I’ll buy you dinner. So, if you agree that you are going to have to make changes to your design anyway, sometimes major changes, why wouldn’t you want to do incremental design?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Short Iterations, The Benefits

Poll - Your Preferred Software Development Methodology

2009-02-10T11:21:00.005-05:00

Last question. With the exception of change reviews, all of the practices listed are the core of Agile development. Sure, there's that whole philosophy and manifesto stuff. But IMHO, if you are doing short iterations (for real) and many or most of the other practices, you are doing Agile development. No 3x5 cards, pair programming, certification, or manifesto required. Just don't like the idea of a new word or "joining a movement?" No problem, improving your dev process is the goal, not "doing Agile right" or anything else for that matter.

For any of the practices that you are not currently doing but that you believe have a significant benefit, think about what the combined effect would be if you implemented all of them. It may take time to achieve, but it will definitely be worth the investment.

So, you are starting a new software company with your own hard-earned cash. What software development methodology will you use?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Suggest additional practices to add to survey

Poll - Retrospectives, The Benefits

2009-02-10T11:17:00.003-05:00

A retrospective provides a forum for the team to talk on a regular basis about what worked and what didn’t work as a team, to discuss ideas for improvement and to work out problems. Instead of a manager having to unearth all problems and come up with all solutions, a retrospective accomplishes much of this, providing the manager with more bandwidth to deal with sensitive issues and to focus on people management. Retrospectives are particularly effective when combined with short iterations because problems have less time to fester and there are more opportunities to take corrective action.

So, if you were starting a software company with your own money, would you use retrospectives or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Change Reviews, The Benefits

Poll- Milestone/Iteration Reviews, The Benefits

2009-02-10T11:15:00.004-05:00

Milestone, or iteration reviews, are done at the end of each milestone or iteration. It is a demonstration to key stakeholders including one or more customers that you have completed the work for that milestone. It is also an opportunity for the stakeholders to provide feedback on the work. Reviews provide a test of the functionality, show progress towards goals, and build confidence in the work. The demonstration is a visible, trustworthy, and confidence building way of showing progress. If you can’t demo the work, how do you know how much real progress you are making?

So, if you were starting a software company with your own money, would you do milestone reviews or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Retrospectives, The Benefits

Poll - Change Reviews aka Code Reviews, The Benefits

2009-02-10T11:10:00.005-05:00

A change review, aka a code review, is a review of the changes (code or otherwise) that were done in order to fix a bug or produce an enhancement. This can be as simple as having an engineer other than the author review the changes and provide comments to the author to something as sophisticated as using a specific code review methodology and/or a code review tool such as Smart Bear. So, if you were starting a software company with your own money, would you do code reviews or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Incremental Design: Possible?

Poll - Stand-up Meetings, The Benefits

2009-02-10T11:04:00.005-05:00

A stand-up meeting is a quick daily meeting of the whole team where you answer three questions: what did you do since the last meeting, what will you do until the next meeting, and what impediments do you have. Everybody gets a quick sense of how things are going and who needs help. The benefits are that problems don’t fester for long and it removes the need for most other meetings.

So, if you were starting a new software company with your own hard-earned cash, would you use stand-up meetings or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Milestone/Iteration Reviews

Poll: Refactoring, the Benefits

2009-02-10T10:54:00.008-05:00

Refactoring is the practice of continuously improving the usability, maintainability, and adaptability of code without changing its behavior. That makes it much easier to add new and unanticipated functionality. Refactoring has the disadvantage that it takes extra effort and requires changing the code. Any change has the potential to reduce the maturity and stability of the product, especially if you don't have adequate testing in place.

If you were starting a software company with your own hard-earned cash, would you use refactoring or wouldn't you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Unit Tests, The Benefits

Poll: Continuous Integration Benefits

2009-02-10T10:47:00.007-05:00

With Continuous Integration, all work from all teams is integrated into a single codeline as frequently as possible. Every check-in automatically triggers a build and usually a subsequent run of the test suite. This provides instant feedback about problems to all interested parties and helps to keep the code base free of build and test failures. It also reduces the integration headaches just prior to release.

There are two benefits of Continuous Integration. The first is that it forces you to break work down into small, manageable pieces. The second is that it spreads the integration work out over the entire development timeframe instead of just a short period at the end. Thus, you have more time to find and fix issues instead of a compressed high-stress window at the end of the release.

So, if you were starting a software company with your own money, would you use continuous integration or wouldn’t you?

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll - Refactoring, the Benefits

Suggestions For Practices to Add or Subtract

2009-02-10T10:37:00.005-05:00

Thanks for participating in the practices survey.

If there are other practices you would like me to add to this survey, or other general comments you'd like to make, please comment below. If you think there are practices that just don't belong on here, I'd be interested to hear which ones and why.

If You Were Starting a Software Company With Your Own Money, Would You... ?

2009-02-10T10:18:00.014-05:00

If you were bootstrapping a software company with your own hard-earned cash, what software development practices do you believe in enough that you would use them right off the bat?

You have to vote to see the results. There is a link below each poll which links to the next. There are 15 in all. The first is on your role, the rest are about which dev practices you value.

[Note to the jaded cynics: the polls really do only collect votes, your privacy is assured]

Click Here for Poll Survey Software
| Website Polls
| Email Marketing

| Innovation Management
View MicroPoll

Next: Poll: Continuous Integration Benefits

Agile Transformation Seminar

2009-02-05T20:09:00.002-05:00

If you are in the Boston area, consider going to the Agile Transformation Seminar on Thursday February 26th in Lexington. The seminar is a 1/2 day in-depth look at the business benefits of Agile and includes a case study on Avid's Agile transformation. My session will be about the effects of Agile on the needs of your tool infrastructure.

What I Believe About Agile

2009-02-05T13:18:00.004-05:00

Before I get too far into what I believe about Agile, let me give a little background. In 2005, I was a staunch Waterfallian. Stuff happened (blogged elsewhere) and then I turned into an Agile advocate.

There are two important things that I believe about Agile. First, while I do believe that being Agile is more than just a set of practices, I also believe that there are a set of practices which are so good that for me they define what I currently think of as Agile and believe everybody can do them and would benefit from them.

Here are the practices which currently define “being Agile” for me. These are roughly in order of benefit: short iterations (preferably 1-2 weeks), Kanban (as an alternative to short iterations), backlog, one piece flow, continuous integration, whole teams, collocated whole teams, test first/test early, use of a product owner, incremental design, refactoring, unit tests, multi-stage continuous integration, user stories, sustainable pace, task assignment and estimation by team members, stand-up meetings, burn-down and burn-up charts, iteration review, retrospective, and frequent releases. There are other good practices out there, but these are the ones that I believe will become mainstream.

Second, and most importantly:

I truly believe the benefits of Agile are significant enough to focus your resources to get to the point of saying this sentence.

This is based on the belief that the algorithm of Agile is just plain better than traditional development combined with direct observation of the significant benefits experienced by both external and internal teams.

To put it another way, if you are not yet investing in Agile, I believe it is the #1 investment you can make to increase your productivity, quality, revenues, customer satisfaction, profitability, and employee satisfaction. It is possible to invest in Agile but not get these benefits. If that is happening to you, I urge you to stick to your guns and look for help. Getting to Agile is worth the investment. It is worth redirecting resources, changing priorities, and doing what it takes to get there.

The web book “Zero to Hyper Agile in 90 Days or Less” is a good place to start.

You may also be interested in “The Top Ten Reasons I May be Wrong That Agile is Better” .

The Top Ten Reasons I May be Wrong That Agile is Better

2009-02-04T22:12:00.010-05:00

First of all, it is important to note that "Agile" is just a word and means different things to different people. There is a good chance that we have different definitions of what “Agile” means. If you want to know what I mean by Agile, please read “How Agile Works in a Nutshell”.

10. “Customers don’t want frequent releases.”

That’s often true, but it is not actually an impediment. While frequent releases are an important benefit, it takes time to get to that point, isn’t the right option for all situations, is just one of the benefits, and is not required to get the other benefits. Read more in a series of posts on this subject.

9. “The advocates are overzealous and arrogant.”

Yes, some of them are. But not all of them. Many people are overzealous and arrogant, that’s not unique to Agile. It is the concepts of Agile that make Agile worthwhile, not the advocates.

8. “It is all just a marketing ploy. You don’t really believe Agile is any better than traditional development and in fact it is just more of the same stuff in a shiny new package. You just want to hop on the buzzwagon.”

This one is hard to refute. The only way I can dispel that belief is if you get to know me better.

7. “I haven’t tried it, and I believe Agile Development is a good development method, but it is just another flavor. Whatever works for you.”

You may be of the opinion that traditional vs Agile is just a preference/philosophy/ideology thing, like chocolate vs strawberry, Aristotle vs Plato, Democrat vs Republican. But then in that case, why would anybody go to all of the effort of investing in Agile? Tech folks don't generally make massive switch-overs from one way of doing things to another just for preference or just to be 2% more productive, they generally only switch in order to get an order of magnitude improvement. So, if it is just a preference thing, there's not much to talk about (from my perspective).

6. “Yeah, I tried that Agile thing. What a disaster!”

There are any number of reasons for failure. I've seen plenty of cases of Agile failure. But in every case that I've seen "Agile failure" it was either because it wasn't really Agile that was being done, or there was a major impediment. An example of a major impediment would be that the attempt was initiated by executive decree followed by no investment in success and a goal of 100% Agile within an impossibly short amount of time. I realize that saying “it probably wasn’t really Agile” sounds pretty convenient. But, I have yet to hear anybody say "I feel like we were really doing it right, but it just didn't work." If that's something that you would say, I'd love to hear from you.

5. “Yeah, I know a group doing Agile and they are really successful. But they would succeed anyway because they’ve got all of the best people on that team.”

All I can say to that is that for me Agile is only worthwhile if it is something that has mainstream applicability. If it won’t work for the majority of teams, it isn’t interesting to me.

4. “I believe that you believe that it is better, but I can clearly see that it is not. I don’t know what your reasoning is, but it is clearly wrong.”

This is exactly what I was saying in 2005. Well, that works both ways and I’ve been on both sides of it. All I can say is that I believe it is worth your time to consider that if so many people that have devoted so much energy to the software development process believe that Agile is a good thing, then maybe there really is something worthwhile about Agile and it is worth your time to learn more about it.

3. “You don’t really understand how successful traditional development can be, and there are probably a bunch of techniques for success you are not aware of.”

My career for the past twenty years has been to contribute to the software process improvement efforts of countless companies. So it isn’t that. By the way, if you haven’t heard of the Niagra method (the best traditional development method), you should take a look.

2. “You have blind faith in Agile. You want it to be better and you will ignore any evidence to the contrary.”

Well, you'll have to take my word for it that I'm a very pragmatic person. If something isn't working, and it doesn't look like it is going to, I don't have a lot of patience to continue doing it just because I think it should work. I originally believed in Agile when I realized that it is algorithmically better than traditional development. But now I believe in Agile because I've experienced significant benefits from practicing it.

1. And the number one reason I may be wrong is the one I don't know about yet. If you've got a good one that I haven't covered, please let me know. I look forward to your thoughts and feedback.

Next: What I Believe About Agile