Derailleur Blog

Understanding Misalignments in Software Development Projects

Fri, 25 May 2012 20:44:02 GMT

Misaligning the work we do with how we do it:

When introducing Scrum to your managers and colleagues, you may find yourself in the position of having to convince or "sell" them on the merits of a very different way of working. While you may all see and feel the outward manifestations of why things haven't been working well like missed deadlines, runaway budgets, sub-par quality, stress and low morale, changing how you work seems like a very risky proposition - especially when weighed against faulting the team, the customer's unrealistic expectations, poorly-written and/or changing requirements and an unsympathetic QA department. The typical management response is to add more process, more management oversight and exhortations for employees to do things "right" or "better".

However, while in the throes of a critical project with money, jobs and credibility on the line, many of us fail to recognize that these "pain-points", contrary to our ingrained cultural beliefs and schooling, are signals that we need to work smarter, not harder. Much like a door that doesn't shut right or a wobbling bicycle wheel that needs truing, there are fundamental misalignments occurring between ourselves, our work, and how we do it that are preventing us from doing better. They can be ignored, of course, as most organizations do - but this comes at a cost of compensation in other areas. The result is an organization that is imbalanced and will always struggle to meet expectations.

The aim of this post is to explain six of the most critical misalignments that occur between the nature of the work of software development teams and how they are directed to approach them. A subsequent post will explore how agile frameworks like Scrum help to address them with simple process changes that bring about often dramatic improvements.

The Six Fundamental Misalignments

A misalignment occurs when something works against its intended purpose or design; this causes side-effects which we interpret as either indications of underlying problems or an intended consequence. These interpretations take on different characteristics depending on who makes them: The further from the root causes an individual is, the more likely cognitive biases will interfere with correct diagnosis. In other words, those who do the work tend to be more in-tune with what's going wrong than those who are directing them. The caveat to this is when the underlying problem is well-understood.

In traditional software projects, there are six fundamental misalignments that contribute to teams working at cross-purposes, making the goal of delivering valuable software and ROI difficult to almost impossible:

Fallacy of Division of Labour
Defined Process Control Models
People as Replaceable Components (Resources)
Command & Control Team Management
Task Switching
Hero Culture

To make the misalignments clearer below, I've created a notation that takes the form of {Misalignment} // {Is Misaligned With}:

Fallacy of Division of Labour // All Effort is Design

This misalignment along with #2 below are the primary causes of all others because it aligns the "design" of the traditional project lifecycle with that of a real-world engineering project which separates design (performed by architects and engineers) and construction (performed by lower-cost trades) activities. Software development does not fit this model, however, because it is an entirely intellectual, design-driven process where "construction" is performed by machines (eg. compilers and packagers). This misalignment affects every aspect of a project from how it is planned, staffed, delivered, maintained and consequently how it works and responds to various management techniques. In short, it glosses over the inherent complex nature of coordinating a software project, and thus risk.

I've written on this phenomena in a previous post: The Fallacy of Division of Labour in Software Projects.

Defined Process Control Models // Weakly-Defined or Continuously Variable Inputs

As a consequence of the first misalignment, a second is created that aligns a software project using a defined process control model based on product manufacturing schemes with the intent to produce repeatable outcomes from well-known, defined or fixed inputs. Most software development projects are not simple enough to have fixed inputs - they continuously vary based on the degree of agreement on requirements with customers/end-users, certainty on the technologies employed and the skill and abilities of the people who will be creating it. It is a wicked problem that requires an empirical approach based on the principles of visibility, inspection and adaptation at regular, short intervals to tame.

This misalignment, of course, engenders another: The conviction that wicked problems can be reliably estimated and costed well in advance of the actual work being done from an up-front plan that spans the entire project horizon.

I've written about the underlying conditions that create this misalignment in a previous post: No Silver Bullets - Just Hard Work

People as Role-Based, Replaceable Components (Resources) // People as Unique, Skilled Contributors

When a software development project is considered as the outcome of 90% construction effort controlled using a manufacturing-based defined process, the logical extension is to view the people involved as role-based, replaceable components. This misalignment is made explicit by the repeated-use of the HR terminology of "resource" instead of "contributor", which recognizes that people constitute the most significant variable input to a project. As a result, organizations "shop" for team members based on the role-based skills they list on their CV instead of looking at their attitudes and abilities. The resulting frictions create team dysfunctions like heroics, demoralization (why be passionate about what you do when you're just a "Drone in Sector 7-G?"), hiding behind titles ("not my job") and scapegoating.

It's very difficult to promote true teamwork under these conditions because people are disconnected from their work and an innate desire to be recognized for their contributions. For more insights on this misalignment, see my earlier post on Why Financial Incentives Don't Work which references fantastic examples of people and team dysfunctions by Patrick Lencioni and Dan Pink.

Command & Control Team Management // People as Self-Organizing, Rational, Skilled Adults

As a result of viewing people as replaceable components and not unique contributors, the next misalignment of using command and control team management arises. This comes from a mindset/cognitive bias that only through direct and persistent management oversight and intervention can a project be stewarded toward a successful outcome. At the employee level, this has an "infantilizing" effect in an organization, building into people the expectation that they will be told what to do instead of being trusted to organize themselves around and be responsible for their work. Worse: It actively discourages the opportunities for creative thinking by making it explicitly risky to do so. Who wants to lose their job over trying something new that might not work? Overall, this misalignment is about trust and the issues managers have with giving control over to those who are better-suited to solving on-the-ground problems.

For a definitive example of self-organization in action, see how the highly-successful video game company, Valve, operates without any direct management oversight. Also see Ralph Stayer's excellent 1985 essay, How I Learned to Let My Workers Lead, for an example of how a CEO transformed his business to let workers set the pace and direction of change.

Task Switching // Effective Completion of Tasks, Predictable Quality

When we think of people as replaceable components in a fixed system, it's not uncommon for an additional misalignment to occur where we view their individual capacities to respond to ad-hoc or multiple requests as nearly infinite. This is called "task switching": It requires individuals to mentally "unload" one problem domain to "load" or "reload" another and get up to speed quickly. Many studies have shown that this is counterproductive and can lead to severe impacts on the quality of work that is delivered. In traditional and agile software projects, this is the number one contributing factor to missing deadlines and falling short on goals.

This is why agile frameworks like Scrum advocate keeping the team shielded from outside influences when working within a Sprint - to distract them with ad-hoc requests, even minor ones, can derail their ability to meet their goals.

Hero Culture // Teamwork

There's an age-old joke about IT projects being an exercise in cat-herding, ie. that it's a futile effort to control the uncontrollable, with certain developers becoming "cowboys" who ignore or shrug responsibilities to "go it alone" and get things done. This is an outward manifestation of an organizational behaviours and practices that allows this to happen due to the aforementioned misalignments. Specifically, if we create a work system that constrains an individual's ability to collaborate and rewards them for making our dysfunctional defined process control models and resulting practices work in spite of themselves, we also create the misalignment of a culture that explicitly or tacitly endorses heroics over discipline and team-based collaboration - in other words, true 'agility'.

This said, it is also possible to find trace elements of heroics in an agile team where an individual decides to do work beyond the scope of the project with the aim to surprise the customer. This has all kinds of ramifications as the 'hero' has kept their cohort out-of-the-loop on the changes they've made until late in the process and may set expectations with managers and the customer that aren't sustainable by the entire team who will end up having to cover the shortfalls as a result.

Almost Inextricably Out-of-Alignment

In their recent book, Software in 30 Days, Ken Schwaber and Jeff Sutherland remark that, "[businesses] have been ill-served by the software industry for forty years - not purposefully, but inextricably". This is the consequence of introducing layers of cascading and inter-related misalignments into organizations and projects between how software teams work and the work they do. As a result, the organization works against and in spite of itself, often in the dark about the fundamental missteps that are occurring, and often exacerbates the situation by inappropriately treating symptoms.

A critical tipping point is reached when the symptoms of the misalignments begin to overtake the ability of people within the organization and/or their customers to tolerate them. It is at this time that an exploration for new ways to manage the work becomes imperative. Recognizing and understanding the six fundamental misalignments I've outlined above will enable business owners, managers and team members to begin to deconstruct their mindsets, work habits and philiosophies toward their work and start to introduce corrections to bring them back into alignment for greater effectiveness. This is the key to understanding the raison d'etre for almost all agile software development frameworks, including Scrum.

In a subsequent post I will explore how simple changes in behaviour works to realign teams with their purpose of creating valuable software (and thus ROI) for their customers.

Three Contract Models for Scrum Projects

Thu, 24 May 2012 17:21:37 GMT

In this post I want to do a high-level fly-by of the options you have to establish a contractual relationship for your next agile project. Moving toward an iterative/incremental delivery process requires rethinking not only your software delivery practices but the legal framework that will enable them. As you may have discovered, traditional contracts are by definition inflexible agreements intended to apportion risk exposure in an adversarial relationship. Contractors are obliged to "make the date/cost" irrespective of the hurdles thrown in their way, while customers are dissuaded from making any changes mid-stream (irrespective of merit) through costly change order processes.

Obviously, this is the antithesis of agile software delivery which relies on an agreement that promotes a collaborative relationship between contractors and customers to promote sharing of risk with flexible options for accommodating changes while controlling scope. The following contracting models are an intermediate step toward this ideal; they re-balance the contractor/customer relationship by giving customers options for controlling scope and costs in exchange for agreeing to a covenant to abide by the rules of Scrum - which is mutually beneficial. In the case of Mitch Lacey's Ranges and Changes model, risks and latent knowledge are surfaced with a short, two-week exploratory project that can be used to prime an agile project with a preliminary Product Backlog and Release Plan.

Each model starts with using a standard fixed-price contract that includes time and materials backstopping as a starting point, thus addressing contractor risk exposure, and adds on options to accommodate changes and costs, thus addressing customer risk exposure. In turn, these provisions act as legal "interfaces" into Scrum, which governs project delivery on a day-to-day basis.

While these are great options for getting your customers to buy-in to a Scrum project, they should be considered waypoints in a journey toward an entirely different relationship management framework called alliancing, which my friend Paul Culmsee outlines in his recent book with Kailash Awati, The Heretic's Guide to Best Practices. However, this is a story for a subsequent post. As they say, "Start Here" and see where it leads.

As always, comments/suggestions/slings/arrows are welcome in the comments below or via Twitter at @DerailleurAgile.

Change For Free

Created By:

Jeff Sutherland, Scrum Co-Creator

In Brief:

Combines a standard fixed-price contract with an option for customers to make changes in scope (features) without incurring a penalty.

Key Feature:

Change for Free option clause that may be exercised by the customer to make changes in scope without penalty provided that they adhere to the rules of Scrum for each iteration by fulfilling the role of Product Owner and working with the team. In all other instances, changes are charged according to time and materials.

How it Works:

Per Scrum rules, the customer Product Owner is responsible for re-prioritizing their Product Backlog of features according to business value at the end of each iteration. Changes in the backlog's priority are free provided that the total contract work is not changed. New features may also be added for free during Sprint Planning or Sprint Review in exchange for an equal sum of low-priority features.

Customer and Contractor Obligations:

The customer must agree to abide by Scrum Rules and keep their Product Backlog in priority-order and involve their stakeholders and end-users to ensure that only the most valuable features are developed.
The contractor must provide assistance to the customer to help them build their Product Backlog and facilitate working with the development team under Scrum rules of engagement.

Money for Nothing

Created By:

Jeff Sutherland, Scrum Co-Creator

In Brief:

Adds an option to Change for Free contracts that allows customers to terminate a project at any time for a nominal charge of 20% of the remaining contract's value.

Key Feature:

Money for Nothing option clause that may be exercised by the customer for early termination of the contract provided they adhere to the rules of Scrum, otherwise remaining work to meet the cutoff is assessed using time and materials charges.

How it Works:

The customer may, at any time, decide an ROI-cutoff point, ie. the point where the cost to develop the next feature or set of features in the backlog exceeds the value to the business. Accordingly, the vendor agrees to terminate the project for 20% of the remaining contract value while assuming the risk of late delivery for any mutually-agreed upon features to meet the ROI-cutoff.

Example:

You have agreed to a $100,000 contract with a number of features that are being developed across eight sprints/iterations. During development on the sixth sprint, the customer decides that they should have enough features to release the software by the end of the seventh sprint. They request a meeting with the Scrum Master and their manager to announce the invocation of the Money for Nothing option.

Once it is confirmed that the customer has met their obligations and is greenlit for the option, the team is informed that this will be the final iteration and they are committed to complete the work that will be mutually estimated and agreed to for sprints six and seven as per usual Scrum rules and options.

With each sprint costing $12,500 ($100,000/8), the remaining value of the cancelled project for the last sprint is $12,500. After invocation of the Money for Nothing option, the customer retains 80% of this value ($10,000) with the 20% balance ($2,500) going to the vendor.

Thus, the total cost of the contract to the customer is $12,500 * 7 = $87,500.00 + $2,500 = $90,000.00

Customer and Contractor Obligations:

The customer must agree to abide by Scrum Rules and be a full-participant in their project;
The contractor must agree to the terms of the Money for Nothing clause, including the reasonable assumption of risk for features to meet an arbitrary cut-off date. This necessitates having good software engineering practices that keeps the software in a potentially-shippable state at the end of each iteration.

Ranges and Changes

Created By:

Mitch Lacey, author, The Scrum Field Guide

In Brief:

A two-part contract consisting of a discovery phase designed to provide customers with ballpark cost and date ranges for a preliminary set of features which can then be used to determine if it makes sense to proceed with a delivery phase using a Money for Nothing/Change for Free contract.

Key Features:

A Discovery Project Contract for a short, fixed-length investigation of a customer's software requirements and build a preliminary Product Backlog that is assessed a range of estimated costs and dates according to team size and expected velocity in story/feature points. This project should include enough funding for 2-3 developers for two weeks. Deliverables include an estimated Release Plan and Product Backlog along with an expected minimum number of feature/story points that the team could deliver based on the investigation.
A Delivery Project Contract that references the Discovery Project Contract deliverables and establishes a framework for delivery using the estimated ranges as a starting point. Also includes the team's Definition of Done to deliver each feature, how the work will be delivered and an explanation of how the customer can reject an iteration's worth of work. This contract will likely include the Money for Nothing/Change for Free options detailed above.

Customer and Contractor Obligations:

Same as with the Money for Nothing/Change for Free contracts;
Contractor agrees to provide all deliverables from the Discovery Project to the customer;
Customer agrees that all estimates derived from the Discovery Project are extremely preliminary and can be expected to change when implemented in a project.

Business Card "B" Side Agile Quotes

Tue, 08 May 2012 04:39:59 GMT

I've been working on some prototypes for the flipside of my business cards - here's a sampling using quotes pulled from my collection that I mocked-up in Visio and converted into PNGs. Feel free to grab them verbatim or use them for inspiration. Let me know your favourites - I'll mock them up and post them!

There's A Lot of Room for Agile to Grow in 2012

Tue, 24 Apr 2012 03:05:06 GMT

Impatience asks for the impossible, wants to reach the goal without the means of getting there. - Hegel

InfoQ (a clearinghouse blog of current 'good ideas' in agile and software engineering) published a post today by David Bulkin lamenting the current and likely future state for agile implementations in the coming year: Looking Back at Looking Ahead, Gloom for Agile in 2012?

Reading over the choice excerpts, what struck me was not so much the pessimistic characterizations of what's going on "in the wild", but rather that it's not changed much over the past decade. While agile adoptions are definitely on an increasing trend (last year's Version One State of Agile Development Survey showed a number of continued positive indicators), change is proving an intractable problem for a lot of folks.

Industry commentators from Cutter Consortium identified all-too-familiar problems contributing to holding back businesses, teams and organizations from enjoying greater success with agile practices which can be distilled to significant misalignments in understanding and applying agile practices. For example:

Using traditional recruiters/head-hunters to hire agile coaches to "fix their agile implementations" without understanding deeper reasons behind their struggles;
Using Scrum as a local optima only while keeping the rest of the organization fixed in old command-and-control structures and practices;
Avoiding investments in improving developer skillsets and tooling to enable agility over adding bodies to projects riddled with technical debt (thus raising the probability of invoking Brooks' Law);
Investing significant time building massive, overly-detailed product backlogs prior to starting and maintaining throughout the lifetime of the project instead of allowing details to emerge through continuous dialogs - a significant source of muda or waste;
Avoiding changing parts of the organization or practices to enable agility out of fear of taking on the challenge - effectively allowing the impediment to metastasize;
Avoiding getting experienced help to guide adoptions in favour of "going it alone" with inexperienced in-house staff;
Adapting agile frameworks to fit dysfunctional company cultures instead of the other way 'round.

These points underscore just how difficult it is to change and realign bad practices into something more resilient, powerful and enabling. Some organizations can do this, some cannot. I'm an eternal optimist in that I believe any organization can change if they want it badly enough - and if they do, I really want to work with them to make this desire a reality.

However, I make this point to new customers who I am bridging into agile: What we're about to take on is not easy; it's going to take a lot of hard work, but if we persist it will be extraordinarily satisfying, productive and effective. When we allow ourselves to think it is easy or that we can take an easier shortcut, we've already begun down the road to failure. In other words, as Liz Keogh pointed out succinctly in a talk last year on Learning and Perverse Incentives, quoting the opening sentence of Waltzing with Bears: "If a project has no risks, don't do it."

This is a learning opportunity - it's something to embrace boldy, not shrink away from in fear!

This said, change doesn't come for free, nor does it have a lot of handy, jingoistic shortcuts. It goes in fits and starts, has growing pains, moments of uncertainty and then clarity, mirages of objectives that smash upon concrete realities, giving way to even more problems and opportunities for continuous improvement. There's no handbook to tell you what to do 100% of the time - merely the best advice based on what went right for some other people in a different context. This is the space that agile has to grow into in 2012: While there's a lot of continued dysfunction, having this naked and apparent is a good thing: It means that agile has delivered in surfacing (and continuing to surface) all the things we're doing wrong. All that remains is the hard-work of shifting the boulders.

If you're stuck and you're not getting good advice to get unstuck, try another advisor. Learn more about the transformation you're undertaking. Go and visit your teams to understand the nature of their problems. Give them responsibility to really get things done and learn to get out of their way and let them lead.

Scrum Project Safety Checklists

Mon, 23 Apr 2012 21:34:56 GMT

Earlier today I came across a TED talk by Dr. Atul Gawande, "How do we heal medicine?" via Mark Graban's Twitter feed (@MarkGraban). Mark consults to hospitals and healthcare providers on how to use lean practices and techniques to improve their delivery of services under conditions of extreme risk and uncertainty (sound familiar?). I highly recommend following him and reading his blog, LeanBlog.org - really good, mind-expanding stuff. And if you're up to it, I'd also recommend getting a copy of his new book, Lean Hospitals.

In this talk, Dr. Gawande provides a brief overview of the challenges facing healthcare providers in the modern era which has come to be defined by teams of specialists who work under chaotic conditions of extreme risk and uncertainty (again - sound familiar?). He goes on to outline, in basic terms, how a systems based approach to medicine is helping conquer this complexity and leading to better outcomes. However, just assembling teams of expensive specialists to provide care doesn't always succeed - often things are overlooked in the heat of the moment.

The usual strategy has been to try and employ a rigid set of "best practices" or recipes on things like surgery - with the same disappointing outcomes as we see in waterfall software projects.

So, Dr. Gawande looked to another high-risk industry, aviation, for some ideas on how they handle complex situations that can't be pre-defined. To his surprise, he discovered they use simple checklists. Working in conjunction with Boeing, he and his colleagues created a simple 2-minute checklist for surgical patient care that when applied at several hospitals around the world resulted in a 35% decrease in complications and a 47% decrease in deaths. The teams of surgeons and specialists still did what they do best, just with better guidance on the important pre-op/post-op procedures:

Dr. Gawande describes how a checklist helps people handle complexity by implementing pause-points at critical parts of a process to catch problems before they become dangerous and something can still be done to address them. In his example, three key pause-points were identified for surgical safety: Before the induction of anaesthesia, Before skin incision, and Before patient leaves operating room. It also provides timing guidance for administering antibiotics to cut the risk of infection by 50%.

It immediately occurred to me that in the context of improving outcomes in complex systems, a simple checklist could provide similar results to agile software teams using Scrum. So, I set about designing my own Scrum Pre-Flight Checklist based on five critical pause-points:

Before Starting Scrum
Before Sprint Planning
Before Daily Scrum
Before Sprint Review
Before Sprint Retrospective

My intent was to provide key practice reminders that can help them continuously improve their game and the outcomes for their customers by making visible and apparent all the things they are and are not doing. While the consequences for skipping certain items won't result in compromising or ending a patient's life, they could have impacts on the health of their project and relationship with their customer - which may certainly feel like a matter of life and death to them!

See what you think - they aren't exhaustive, and I'm going to be revising them based on in-the-field tests. Try them out on your teams and let me know how they work or did not work for your team either in the comments section below or on Twitter via @DerailleurAgile.

Scrum Project Safety Checklists

Misaligning Incentives with Performance Outcomes

Thu, 12 Apr 2012 22:05:27 GMT

Last August I wrote a post that discussed Why Financial Incentives Don't Work to motivate or incentivize employees toward better performance or behaviours that managers desire. This was in reference to a session that researcher Dan Pink delivered on the science behind motivation and how we've fundamentally misinterpreted what makes people want to do better.

Dan distilled this down to three misalignments between what we do and how we do it (yes, this is my new recurring theme):

Autonomy - The ability of an individual to have control over their destiny;
Mastery - The desire to improve one's skills;
Purpose - The desire to be part of something with meaning.

Dan delivered a second talk expounding on these conditions which was illustrated by the creative folks at RSAnimate which you can see below.

Dan's findings affirm the observations Dr. Deming made which I described in my previous post on The Red Bead Experiment - chiefly that monetary rewards (or any other threat or praise) do not effect changes in reducing variability within a fixed process. The additional vector that Dan adds (and is shown if you carefully watch the clips demonstrating the Red Bead Experiment) is the demotivational impact having an unattainable reward is when you've removed a worker's autonomy, mastery and purpose.

What do you think? Add to the discussion below or via @DerailleurAgile. Looking forward to the conversation...

The Red Bead Experiment

Tue, 10 Apr 2012 21:47:59 GMT

In my recent posts I've been exploring the tensions and misalignments that occur when we confuse the way we work with the work we are doing. This is demonstrated when we try to apply a fixed, single-pass/phased (aka 'waterfall') process based on the division and separation of design and construction activities to software development which has no such divisions because it is an entirely design-driven activity. Inevitably, this misalignment causes unpredictable variations in quality and delivery as things don't go according to an up-front plan, resulting in waste, re-work, employee attrition, litigation and in some cases loss of the business itself.

Because the single-pass/phased process isn't tolerant to inspection or adaptation (since it is predicated on the movement of work products along a virtual, gated assembly line), it contributes to exacerbating an already poor situation. Worse still, the people who are closest to the process and the work are rendered nearly powerless to change it in any meaningful way by management intransigence or ignorance. However, they are still beholden to deliver results within it.

This dysfunction of management/disconnection of employees is hilariously illustrated by the famous chocolate factory scene from the classic TV show, I Love Lucy:

Just prior to the time this episode was filmed in the early 1950s, an American statistician and management consultant, Dr. W. Edwards Deming, was beginning to teach Japanese businesses the fundamentals of quality control techniques which would eventually lead to the ascendancy of giants like Sony, Mitsubishi, Toyota and others who began to manufacture quality goods at a lower price than in North America. Through this work, Deming developed an approach to management that observed the processes and environments people worked within had a significant impact on productivity, quality and cost, and not the exhortations or machinations of management to apply "corrective measures". Moreover, even when using a "fixed" process designed to promote stable, predictable outcomes (like 'waterfall') there will always be variations in results that have nothing to do with an individual's willingness to work hard or inherent skill.

The Red Bead Experiment

To demonstrate the disconnection between traditional management approaches to corrective action and misalignments between workers and their work, Deming devised a simple experiment using colored beads and a paddle called The Red Bead Experiment.

The experiment organizes volunteers into a company charged with shipping white beads to customers: 4-6 "willing workers", 1-2 QA/Test leads and an analyst who records results. The workers use a paddle with 50 indentations to sift out as many white beads as possible over four rounds or iterations. As each worker completes their task, they show it to QA who counts and reports the number of defective red beads they have accumulated and passes the results to the analyst for tracking on a chart:

Worker	Day 1	Day 2	Day3	Day 4
A	8	10	7	16
B	10	7	11	13
C	10	13	7	9
D	9	7	4	4
E	11	10	11	9
F	8	9	9	6

The beads are typically mixed in a 5:1 distribution (2000 white, 500 red) to simulate the recurring nature of defects within a work system. As each round is played, "management" praises workers who produce the lowest number of red beads while criticizing those who do not, sometimes asking patronizing questions like how things are at home, if they need to take time off or perhaps take advantage of the company's EAP to get "back on track". As the defects accumulate, however, "management" begins to threaten loss of benefits, holidays, and layoffs in an effort to stimulate better behaviour.

After each round, "management" introduces new corrective measures to treat the symptoms of shipping defective products: Slogans exhorting employees to "Stop, Think, Act and Review" (STAR) or to be "Intentional" in their work, "Get it Right, The FIRST Time", etc.; Defect targets and KPIs (usually 3 or fewer); Monetary incentives for meeting or exceeding targets, and finally Ranking and Appraising the top performers and dismissing the rest. None of these measures have any effect on reducing the number of defects.

The game finally ends after the fourth round with the sorry news that the customer has cancelled the project due to high defects and as a result the company is closing.

What Does the Red Bead Experiment Tell Us?

In stark terms, that we're profoundly misunderstanding the causality effects of incentivizing good workers trapped within a flawed process that we are unable or unwilling to change. The experiment, of course, places the "willing workers" into an impossible scenario: No matter how well they try to do their tasks, the process doesn't permit them to do better. The point is driven home: Variability in defects, even when using a fixed process, is always present irrespective of how management tries to impose corrective actions that treat symptoms and not root, underlying causes.

It also tells us that within a fixed process we will find defect rates that statistically distribute 50% above the average, and 50% below the average no matter what extrinsic motivation is applied or how intelligent or industrious a worker under this process may be. This suggests a profound misalignment in how we perceive someone who is an "above-average" or "below-average" employee.

This result is present whenever we use fixed processes as a proxy for expertise and prohibit those that are closest to the work from making adjustments and refinements to inspect and adapt processes to drive down defects, re-work and costs while increasing quality. Management is impairing itself and its business when it chooses to try and centralize and direct work processes without providing the tools to employees for making improvements.

To see a demonstration of the Red Bead Experiment in action, check out the following short videos on YouTube: Red Bead Experiment Part 1, Part 2, Part 3, Part 4, Part 5. To read up on how to run the experiment, see these resources.

Footnote:

I'm currently working on an adaptation of this experiment for software teams and organizations and their managers to help stimulate discussion about the misalignments they perceive (or perhaps, more interestingly, do not) and what can be done to help bring about improvements to re-connect employees with a sense of purpose to their work while freeing managers to new opportunities that can occur when you allow workers to lead.

What do you think? Feel free to comment below or on Twitter via @DerailleurAgile. Looking forward to the conversation...

Misaligning How You Work with the Work You Do: Why Waterfall and Agile Don't Mix Redux

Thu, 05 Apr 2012 16:01:37 GMT

A recent blog posting about Top 10 Project Management Trends for 2012 suggests that for the coming year, teams will be blending agile software development practices with waterfall to create a wondrous new "hybrid" model:

4. Agile blends with waterfall for a new “hybrid” approach

Having moved from “manifesto to mainstream,” Agile has confronted project teams with the difficulty of implementing the experimental and hyper-collaborative approach. To transition an organisation into fully adopting certain aspects of Agile, project teams are combining traditional and Agile elements to create their own hybrid approach. In areas such as planning, requirements, and team communication, organisations are designing custom-made methodologies to do what works for them.

As I stated in my post of last October, Five Reasons Not to Mix Agile with Waterfall, this is a flawed approach that won't help teams adopt agile practices - even as a transitory step - because it is in conflict with how both approaches to project delivery work. In other words, when you try to mix agile and waterfall, you demonstrate poor understanding of both systems. Here's why:

Agile == Empirical, Waterfall == Defined

A principal conflict between agile and waterfall processes are their foundational principles. The former is based on a transparent process that is perpetually open to inspection and adaptation; the latter is a closed process based on static phases of execution that are intolerant to inspection and adaptation once the project has started as it causes churn that exceeds the boundaries, and, ironically, necessitates iterating to resolve.

Software projects deal with significant complexity on multiple fronts and scales that are highly resistant to linear planning. This is because they exhibit traits of being wicked problems. This necessitates using an empirical process to continually refine the solutions. I explore this topic more fully in my post, No Silver Bullets - Just Hard Work.

In Software Projects, All Effort is Design

Jack Reeves observed in his 1992 essay, What is Software Design?, that the source code developers create is the de facto design of the system, with construction performed by machines (the compiler and packagers). This has a serious implication in that it effectively negates the whole notion of drawing parallels between the division of labour on real-world engineering projects and software development where "skilled" architects do all the designing that is dutifully followed by "lesser-skilled" workers. I explored this topic more fully in my post The Fallacy of Division of Labour in Software Projects.

Waterfall is predicated on dividing design from construction - this means that when you attempt to blend it with agile frameworks by say, planning everything up-front and fixing delivery dates, budget and scope, you effectively negate the relevance of having a self-organizing team that iterates and learns. Contrary to some popular assertions, you can develop complex architectures using emergent techniques.

Productivity Will Worsen, Risk Will Escalate

Choosing to create an agile/waterfall mashup needs to be examined for the root-cause motivations. If it is to be used as a "stepping stone" to adopting a process framework like Scrum, the rationale needs to undergo a more rigorous examination, eg. Taiichi Ohno's "5 Whys". I'd suggest a facilitated dialogue mapping session using IBIS notation on a projected screen.

My experience and intuition tells me that the driver to not fully transitioning has more to do with mindset, courage and a willingness to enter into a potentially painful cycle of continuous improvement that will involve a chaotic period of successes and failures. This is the "change adoption tax" - it doesn't come for free, it can cost customers but it is only through this effort that you can reach a place where agile practices will benefit your organization.

This diagram shows Virginia Satir's Change Process Model - it illustrates the shifts that occur when organizations begin to adopt and internalize new practices. It begins with the Late Status Quo, a place where people are comfortable in familiar surroundings, which then becomes disturbed by a challenge that reaches a tipping point where the status quo is interrupted and enters into a period of Chaos. During this period, there is a lot of volatility as the organization begins to experiment and learn with how to realign how they work with the work they do. Results are unpredictable due to group and individual dynamics, organizational intransigence or openness, enforcement or relaxation of bureaucratic rules, presence or absence of shared understanding of the motivations and solutions to change and ultimately vision. Moving into the next phase of Integration and Practice, where organizations become proficient, productive and efficient is dependent upon the effort put into continuous improvement during the period of chaotic challenge.

Mixing defined processes with empirical processes will elongate this chaotic period: Productivity gains will be unpredictable as traditional command and control roles and practices conflcit with agile framework practices. The ensuing chaos within the teams that are told to self-organize and iterate against the fixed forces that enslaved them before will drive up risk more than previously. Results will be mixed and almost assuredly will sour managers and customers from commiting to agile adoption.

In other words, counterintuitively, it will make things more painful and more expensive to delay the effort to move entirely toward agile adoption.

Conclusion: Agile Adoption is about Realignment of How You Work with the Work You Do

The take-away from this post and my previous one is that agile frameworks are based on and enact significant changes and realignments in how teams and organizations work with the nature of the work they are undertaking. Defined processes, like single-pass/phased or 'waterfall' are designed to substitute expertise with process by dividing design and construction activities. This model excels when the tasks to be undertaken require very little interpersonal communication or shared understanding of problems and responds well to adding addtional manpower.

Software development, however, does not respond well to this type of organization because it is entirely intellectual, design work. It requires an empirical approach with high interpersonal communication, substituting process with the expertise of a team's aggregate skills. Because problems can shift in unpredictable ways. it benefits from short iterations of planning, inspection and adaptation.

By attempting to mash-up the two models, an unpredictable mis-alignment of how people work will occur, resulting in a protracted period of head-scratching on what went/is going wrong. Change is hard, it isn't free, but it needs to be approached intentionally and with courage. As Ralph Stayer of Johnsonville Foods writes in his 1986 essay How I Learned to Let My Workers Lead (which I HIGHLY recommend anyone thinking about how to introduce change to an organization should read),

"Getting better performance from any group or individual, yourself included, means a permanent change in the way you think and run your business. Change of this kind is not a single transaction but a journey, and the journey has a specific starting point and a clear destination."

If there is serious desire to change, then get on with the change and begin to discard old practices. Otherwise, defer until the last responsible moment when such a move is possible.

How I Deliver Agile Consulting

Mon, 02 Apr 2012 19:28:56 GMT

Over the weekend I read Tobias Mayer's incredibly candid and inspirational blog post on how he delivers his agile guidance as a consultant, entitled Lemon Sorbet (a must-read if you're in the market for an agile consultant). It was inspired by Bob Marshall's (@FlowChainSensei) blog about How to Spot a Lemon Consultant, which prompted Mayer to reflect and introspect on Marshall's unvarnished, satirical assessment of the bandwagon charlatanism that is rising with the popularity of agile processes: Was he, too, a 'Lemon Consultant' ?

In his own words (emphasis mine):

So I’m a consultant who plays games. Am I selling snake oil? I don’t think so, as I have no intention of selling anything. I try to embody the principles I believe in, and engage people in dialog, figuring out what the real problems are that need to be solved, which is almost never the presenting problem. When a root problem is well understood, the solution will emerge from the people engaged in the problem. It will not be solved by management, executives or seasoned experts—and certainly not by the consultant. Give it up...

I may well be a lemon consultant—some clients would certainly say so. But even that is a form of success, perhaps. The client knows better what he wants next time round. I like to think I’m a lemon sorbet, cool, sharp and palette-cleansing ;) Once I’m gone you get to start the next course of real food, refreshed. Occasionally, someone might say, months later, “hey, remember that sorbet we had that time? That was good!”

I quite like that last line as it reminds me of how an old colleague and friend describes my style of influence as "casting a big wake" long after I depart. However, I digress: After reading over Mayer's and Marshall's blogs, I had my own moment of introspection. Here's how I have always approached my delivery of agile guidance to customers - I think it puts me just beside lemon sorbet, perhaps nearer to key lime:

It's a Passion, not just a Profession

I think one quality that separates me from my citrus competition is the professional journey I've taken to arrive at becoming a full-time agile consultant. Since 2001, realigning how I work with the work I am doing has been a passion. It began by reading an article in the Economist about ExtremeProgramming and subsequently discovering Schwaber and Beedle's classic text in a Toronto used book shop in 2002. Since then, I've voraciously read about, practised and experimented with agile practices wherever I have worked.

I took my own initiative to push change in resistant organizations through education and demonstration. Sometimes I succeeded, other times I failed. In either case I pushed on, learned more and continued to expand my knowledge about agile practices of all flavours and why they are better suited to complex, wicked problem-solving than a rigid, up-front plan.

When you work with me, you benefit from my decade of experience. I try to give meaning to the sometimes-trite phrase, "trusted advisor".

I'm excited and enthusiastic to share my knowledge with you, but I also understand that when it comes to choosing a consultant, it's also about chemistry and personality. This is why I offer free sessions like Pitch Your Boss where you get a chance to test drive me before making a commitment.

Straight Goods - No Fairy Tales

Should you engage me to teach Scrum to your teams, you will often get a couple of questions first:

1) What is your motivation to adopt Scrum?
2) Does the transformation have executive sponsorship or buy-in?
3) Has Scrum been forced on people, or have they been asked for their opinion?
4) Is the organization open to change or calcified in their ways?
5) Is the need for change perceived in the organization?
6) On the whole, are the rank and file risk-averse? The organization?

While I strongly promote Scrum as a robust and responsible set of practices for getting software project ROI, I'm also very cautious: I will warn you that this change doesn't come for free - it takes a lot of hard work, patience and perseverance. Yes, I can help you and your team get usable software in 30 days or less, but it's just the first step in a bigger journey for which there isn't a definitive guide, and it can be rough.

This is where I provide value: I help your teams become better software delivery professionals by using Scrum and at the same time help your business and customers become more entrepreneurial by learning how to leverage Scrum for business advantage.

Workshops, Training and Games

I like to use games to help introduce and reinforce points. Some do have predetermined outcomes that I know about - but you may not. I use them to disrupt how you think about work and how you do it. Sometimes you'll like the outcome, sometimes it might seem pointless - until you think it over for a while.

Workshops are like sandboxes for mentally exploring new ideas, strategies and techniques. I employ them to get you actively involved in understanding agile software development concepts and how to use them. I really dislike PowerPoint slide decks and try to keep them to a minimum.

Training isn't just a rote walkthrough of the Scrum Guide - that would be almost as insulting as those folks who put a wall of text on a PowerPoint slide and then read it to you. Expect to be challenged and to challenge me. Surprisingly, I don't have all the answers and might learn something from you.

Tchotchkes, Pamphlets, Brochures and Cheatsheets

Recently I was asked if I carry a line of materials that can be distributed to teams as part of my engagements. I do not, for the simple reason that there is already a surfeit of guides, cheatsheets, pamphlets and Scrum-on-a-Card type documents that can be found with simple Google searches. I provide value through my personal interactions with you and your teams, not through branded reproductions of what is already out there.

This said, I do have a library of materials (free and for-purchase) that I have collected over the years that broaden understanding and provide context (ie. the "why" behind the "how") to agile practices. If you are a fan of agile software development books, I can tell you which to avoid and which are must-haves.

Also: I don't carry a line of pens, planning poker cards, estimating dice, post-it notes or similar marketing tchotchkes - to me, these are just tacky and a sure sign of a snake-oil peddler.

Having Fun Changing Work

In his book, Three Signs of a Miserable Job, Patrick Lencioni describes the disconnection and dissatisfaction people feel with their work through the presence of any of three conditions: Anonymity, Immeasurement and Irrelevance.

Anonymity refers to the need human beings have to be understood and appreciated for their unique qualities by their peers and superiors;
Immeasurement refers to the need employees have to control their own fate by gauging their success and failure through tangible means;
Irrelevance refers to the need of employees to know that their job matters to anyone - a strong connection between their work and the satisfaction of another person or group of people.

In my career I have laboured under and witnessed others endure some or all three of these conditions as a direct result of the misalignment between the type of work involved in software development and how it is structured. Even in the swankiest of shops, with pool tables, beer fridges and loft spaces, a miserable job can exist.

My motivation for becoming a consultant is to help right this ship and put fun back into work for IT professionals by realigning them with their work and kicking the stuffing out of any sense of anonymity, immeasurement or irrelevance. Scrum helps unlock this potential in teams, customers, managers and business owners. Ring me up, and I'll show you a whole new way to work that can raise team morale as well as profits and have even some of the sternest folk smiling again.

What do you think? Let me know your reactions and thoughts below - love to hear them!

Get Trained, not Certified. There is a Difference.

Fri, 30 Mar 2012 13:36:51 GMT

Get professional training.

That's what new teams and organizations are told is a "best practice" when embarking on an agile transformation. It's good advice, because though it seems common sense, there's a huge difference between reading about Scrum and actually doing it. Having an experienced trainer or consultant lead you through the fundamentals of the Agile Manifesto and how it is concretely expressed in process frameworks that use visibility, inspection and adaptation is a good idea - you're bound to have a lot of questions.

But what about becoming a "Certified Scrum Master" ? Or how about a PMI PMP-Agile Certification? Or even, as I am, a "Professional Scrum Master I" or "Professional Scrum Product Owner I" ? If you have this designation, it seems to infer that you're a cut-above because an authority has knighted you as such.

I tend to side with Ken Schwaber when it comes to certifications: They (barely) certify that you attended a class for two days where the topics of agile project delivery were discussed. They do not equal competence or experience or a superior access to knowledge beyond the reach of mere mortals.

Not long after launching Scrum.org, Ken wrote content about the Genesis of Scrum.org and the Professional Scrum Developer program. Within he explains the creation of Scrum and the early Certified Scrum Master (CSM) courses he taught in the early part of the last decade, leading up to the creation of The Scrum Alliance, with folks like Esther Derby and Mike Cohn, and how, not long after the wheels fell off the whole thing.

What Ken began to notice was that instead of improving the quality of projects with knowledgeable and capable Scrum Masters, the market was becoming flooded with a lot of poor-quality CSMs:

By early 2009, there were more than 60,000 CSMs. More organizations were using Agile processes than waterfall processes, and of those employing Agile 84% were using Scrum. However, less than 50% of those using Scrum were developing in incremental iterations, which are the heartbeat of Scrum.Martin Fowler wrote in his blog that he was encountering many instances of "Flaccid Scrum." Teams were using Scrum vocabulary but weren’t able to create a potentially shippable increment of functionality within a single Sprint.

Something to ponder as you consider signing up for that next class so you can hang a "Certified" set of letters after your name on your resume. Ultimately, Schwaber found himself on the side of a losing battle for improved quality vs. volume sales and was forced out of the organization. The story doesn't end here, however, as the situation continued to decline to such an extent that high-profile trainers like Tobias Mayer ended up leaving the Scrum Alliance in a very public way, renouncing all his certifications.

This had an impact on me, as my first Scrum cert was the Certified Scrum Master in 2006, taught to me by a trainer who was trained by Ken Schwaber. I really began to see certifications with a rather jaundiced eye. So, why do I have Professional Scrum I and Professional Scrum Product Owner I "certifications" ? Did I take them to make myself seem better than the next guy? Not really. Let me elaborate:

I ponied up the cash to pay the course fees ($1500), fly from Toronto to Boston ($400), rent a car ($150) and two nights' stay at a nearby hotel ($450) not so I could wear a PSM I designation like a tacky badge, but rather to actually meet the man who inspired me to change my career path when I picked up his book, Agile Software Development with Scrum in a used bookshop near Yonge and Eglinton in 2002.

I, like all my fellow classmates, had years of experience implementing Scrum on a variety of projects. We weren't there for a cert, we were there to learn and ask question from the guy who, along with Jeff Sutherland, had the vision to adapt lean manufacturing techniques to software development in 1993 and the courage to see that vision realized.

(Got my book signed at my PSM I training; inscription reads: Scrum On!!)

My advice: Seek out a good trainer not for the tacky and expensive certification badges they have or are willing to provide to you and your team but the experience and knowledge that they can provide during and after the course ends. Look for someone who can train on-site and is willing to help you in your embryonic adoption stages as you struggle to implement Scrum within your organization - this is where the real value resides.

If you, like me, already have some experience under your belt, I highly recommend attending one of Ken's PSM I courses for the same reasons I did: To meet the man, learn some new things from him and to share and benefit from the wide array of like-minded folks who take his classes. That's worth more than a hundred CSMs and PMP-Agile designations any day.

The Cost of Waterfall Contracts

Thu, 29 Mar 2012 13:13:49 GMT

While reading over Mitch Lacey's excellent new book, The Scrum Field Guide last night, I came across an image in the chapter about traditional vs. agile contracts that I think rather hilariously drives home the point about why customers are losing faith in single-pass/phased (aka waterfall) projects. While definitely understandable from the vendor's point of view, there's something wrong with a process that so severely punishes customers for getting what they want, not just what they asked for.

And as a Demotivational Poster that I created with Despair.com's DIY application:

Ouija Board Team Estimating

Wed, 28 Mar 2012 19:30:28 GMT

Recently I came across a really fun, innovative way that teams can collaboratively estimate user stories for a Sprint called Ouija Board. If you're looking for something to liven up your Planning Poker sessions, I suggest giving it a try:

Set-Up and Rules:

As per Planning Poker, the "game" is played in quick successive rounds to get a team consensus estimate (0,1,2,3,5,8,13,21) of a user story or feature. Break out one of your old decks of Planning Poker cards (or just write numbers on Post-Its) and set up a preferably small, round table like this:

Ouija Estimating:

Place your stack of user stories to estimate on a another table or shelf that's nearby so they will be easy to "draw" and get estimated.

As usual, you can choose to pick an easy reference user story that everyone can agree is worth "2" points. Place this story card next to the "2" card on the table. To begin estimating, gather your team around the table (this works better with small teams of 4-5 people) and draw a user story from the prepared stack, read it aloud and place it in the centre of the table. Next, have each team member place a finger on the card and gently begin to move it to the cardinal position that best represents the estimate they agree on:

Deciding Estimates

If a card is moved to a number on the table without hesitation, everyone can remove their fingers and the estimate written on the card and can be left by the number as a reference story or removed to a pile designated for estimated stories.

If there is hesitation and a slight tug-of-war to get a card to an estimate, the Scrum Master or facilitator can pause the activity and ask everyone to engage in brief discussion on the impasse. As per Planning Poker rules, allow only two, two minute rounds of re-estimating and discussion. If consensus still can't be reached, move the story into a separate pile to review with the Product Owner.

If the story stays in the centre of the table, the team probably doesn't have enough information to estimate. The story either needs to be broken down and re-estimated, or the Product Owner needs to provide more information or they may need outside input to get confidence on the technologies they need to use to implement it. Move the story to a separate pile.

If your timebox allows, return to the stories that were set aside and see if consensus can be reached - otherwise, you should be good-to-go.

Check out the video below that inspired this post for an example of how it's played by a real team (turn on closed captioning to see real-time commentary describing the rules). What do you think? Would this liven up your team estimating sessions? Do you have different games that you play to generate consensus estimates? Tell me more below...

The Two-Week Project Bootstrap Explained

Mon, 12 Mar 2012 19:47:35 GMT

Last week we launched a new promotion to help organizations and their software development teams begin to adopt agile practices called The Two-Week Project Bootstrap. This is the complementary "next step" from our Pitch Your Boss service, which provides a low-key, exploratory discussion about agile practices, to actually doing them in a real-world context. It's very easy to talk about agile, and while we are passionate about communicating our enthusiasm for Scrum to others, we also want to help them put theory into action.

The inspiration for The Two-Week Project Bootstrap comes from Jeff Sutherland's recently-published business fable, The Power of Scrum (see my review here), that describes the travails of a new team ramping into Scrum to rescue a beleaguered project from disaster. We wanted to capture and put into motion the essence of Sutherland's short story in a turn-key engagement, providing customers with a hands-on, real-world bootstrapping experience without having to gamble on meeting an experienced Scrum coach in an airport pub to guide the transformation (the plot device used in the book).

Irrespective of whether your project is in-flight, in trouble or greenfield, The Two-Week Bootstrap is aimed at helping your organization begin an agile transformation for world-class software delivery.

What Happens in a Two-Week Project Bootstrap?

A Two-Week Project Bootstrap begins with an initial consultation to understand the context of your current project and the people who have been entrusted to deliver a release. We examine questions similar to the following:

    1) What products and/or services does your business provide?
    2) What are your expectations for the software product you're developing?
    3) How have past projects been delivered? What went well? What did not?
    4) How do you believe Scrum can help?
    5) Does anyone in the organization have experience with Scrum or similar agile frameworks?
    6) Have the proposed team and your organization been approached about changing direction?

This information is used to help tailor and target our training and guidance over the next two weeks.

Once this is complete, we set two very important dates: When we will commence the two-day Scrum Training session and when we will begin the first Sprint or iteration. Our preferred schedule would look a little like this:

The reason we prefer this chronology of events is twofold: First, we keep training fresh in the minds of the team as they go into their first iteration, second we establish the optimum rhythm for a Sprint (iteration) that allows us to create predictability in our delivery: Mondays we plan; the following Friday, we deliver and inspect and adapt the product and ourselves. Over many projects we have found two-week iterations to strike a balance between the length of time a team typically needs to create a working, tested product increment and the time the business needs to make effective, timely decisions about the product and project.

The final part of the initial consultation involves advising on establishing a Scrum Team Room. In order for The Bootstrap to work, the team needs to be co-located in a space where they can work collaboratively without the obstructions of walls or cubicles. Ideally, this should be a room large enough to accommodate the developers and their equipment comfortably, along with whiteboards and a large screen monitor or projector. Whereas the Scrum Team will be practically living in this space for the duration of the project, its selection must be considered carefully.

Scrum Training for Everyone

The Two-Week Project Bootstrap rubber hits the proverbial road when everyone involved with developing the new product or service are trained in the fundamentals of Scrum. We employ innovative techniques to quickly immerse participants in the rules and then begin to apply them. Our objective is to give context to "getting agile" and how it can be leveraged to enable world-class software delivery. We also want to pump-up everyone for the work ahead!

At the conclusion of training, we ask everyone if they are willing to participate in the Two-Week Project Bootstrap using Scrum. This might seem pointless, but it is a small and important first step toward changing attitudes about project delivery: Empowering teams and individuals instead of ordering them about.

Filling the Roles

Once we understand the rules of Scrum, we need to fill three Scrum Roles: Product Owner, Scrum Master-in-Training and The Development Team. These three roles comprise what is known as The Scrum Team and each have specific objectives:

The Product Owner is responsible for setting the priorities of the project and providing direction to the team to earn ROI with each delivered increment of product functionality. This is an intensive, team position that is roughly analogous to being the General Manager of a sports team or a Product Manager.

The Scrum Master-in-Training is responsible for learning the rules of Scrum and ensuring they are followed by the team AND stakeholders. Working closely with our experienced Scrum Master, they learn the fine art of "Servant Leadership" that keeps their team productive and how to liaise with the Product Owner and Stakeholders to ensure the best possible ROI is being realized every iteration. This is a FULL-TIME role and should not use a developer in a part-time capacity.

The Development Team is ideally comprised of 5-9 software professionals who as an aggregate possess the necessary skills to create the product increment. This includes software developers, testers, UI/UEX, architecture, and the like. It is all dependent on the product or service you're creating.

Planning the Sprint

With the schedule, training and team in-place, the work of the Sprint begins in earnest early Monday morning! Our objective is to demonstrate how to plan a Sprint with the resources you have at-hand. It might feel a little like working without a net, but we find the best way to start working with Scrum is to start working with Scrum. This means we plan only as much as we need to get started - four hours max.

The goal is to build a Product Backlog with just enough work to fill the iteration and then break them down into discrete, estimable tasks that the team can execute.

Executing and Reviewing the Sprint

Over the next ten business days, the team works collaboratively to create their first increment of functionality that will be reviewed on the following Friday. This is a very intensive, hands-on period where we provide the structure and stability to help the team begin to adapt to a new way of working. Our experienced Scrum Master, along with the Scrum Master-in-Training, facilitates each Daily Scrum team meeting, answers questions from the team on process and rules, and helps them prepare for delivering their increment at Sprint Review. Additionally, they work with the Product Owner to help them keep their backlog groomed and prepare for the upcoming Sprint.

After the Sprint Review, the Scrum Master(s) and Team hold a brief meeting (The Sprint Retrospective) to reflect on their experiences of the past Sprint and what should be done to improve for the next one. This oft-overlooked meeting is vital for adopting Scrum (or any agile framework) and is a non-negotiable if a team is serious about their transformation. We provide our customer teams with a number of innovative ways to facilitate Sprint Retrospectives so as to leverage them to full advantage.

Go/No-Go?

Based on the past iteration, The Two-Week Project Bootstrap concludes with a wrap-up meeting with the Team, Product Owner and Stakeholders to determine if Scrum is right for them. This is the go/no-go point. If the effort is greenlit for successive iterations, the option is now open for the organization to either continue engaging us for helping coach a number of Sprints, or to go it alone. In the latter case, we offer two weeks of complimentary phone and email support to get the team and organization through the rough patches.

Want to know more? Contact us to set up a free, no-obligation consultation session to see if The Two-Week Project Bootstrap is right for your organization. In the meantime, get a copy of Jeff Sutherland's The Power of Scrum to get an idea of how a transformation to world-class software delivery is started.

The Power of Scrum Revealed

Thu, 23 Feb 2012 04:22:36 GMT

About a week or so ago, after reading a tweet from Jeff Sutherland, I finally picked up a copy of his new novella, The Power of Scrum, for my Kindle (a steal at $6). Jeff released this in paperback last November and on January 31, 2012 it was released in Kindle format. It has been on my "must-read" list since hearing about it last fall, and having just completed it this weekend, I thought I'd share my thoughts on it.

For the impatient: It's a well-written short story about applying Scrum on a really risky project. Get a copy for yourself and your manager. Now.

What is it About?

The Power of Scrum is a gripping business fable in the same genre as those by Patrick Lencioni, Eli Goldratt and Stephen Denning written by the co-creator of Scrum, Jeff Sutherland. Over the course of ten short chapters it tells the story of Mark Resting, the beleaguered CTO for a software company on the brink of disaster after six months of repeated delays in delivering a promised solution to their largest customer, Logistrux. Failure to deliver now would mean mass layoffs and legal ramifications that would likely take down the entire company. In a Hail Mary pass he travels to Logistrux's London head offices to make a final plea for three more months to deliver a solution - without knowing how he would make good on that commitment beyond driving his team into the ground.

After getting drenched in the infamous rainy London weather and missing his last flight home, Mark makes a chance meeting at the airport hotel pub with Jerry, a San Francisco-based Scrum consultant on layover after delivering a paper at a conference in The Netherlands. With the question, "What does rugby have to do with developing software?", Mark engages Jerry in a dialogue about what Scrum is, how it works and why it works that spans the entire book.

As with many who are new to Scrum, Mark is initially quite skeptical of the things that Jerry describes and pushes back several times during their initial meetings. How can teams really manage themselves? How can a project be successfully delivered when the customer is allowed to make changes along the way? How can we deliver a solution without a big, up-front plan? How can software be released every 2-4 weeks?

Jerry patiently explains the answers to these questions and many more as he introduces the structure and rules of Scrum and how they are applied to help align the team with the customer and their wishes. Reluctantly, Mark agrees to see if his team will adopt Scrum to turn his firm's fortunes around and begin releasing software within two weeks. From this point onward, the story of their successes and setbacks unfold as they race against time to deliver a solution that will meet Logistrux's needs.

Why is it Compelling?

Well, it's a great story that teaches how to deliver ROI under extreme conditions for a start. However, this story has a greater impact in my opinion.

We live in what I've come to see as the Second Age of Scrum. In the First Age (1993-2003) Scrum was born and by sheer dint of perseverance came to be one of the most popular agile delivery practices due to its simplicity and effectiveness in providing real ROI where other practices failed. However, this success came at a price, with sustained attacks on adhering to Scrum's simple rules as being "dogmatic" and not "pragmatic", ironically leading to an "anything goes" ethos and an acceptance of "Scrum-But" implementations (eg. "we do Scrum, but we don't do x, y or z"). Net effect? A confusion in the market about what Scrum is and if it can really be as effective as advocates claim.

By using a powerful narrative/dialogue approach to describe how to apply Scrum, The Power of Scrum turns this tide back (here's to hoping it ushers in a Third Age of Scrum Enlightenment) by making the subject matter easily understood by anyone. Instead of being a staid how-to guide, the material is presented within a believable, real-world encounter that brings the human dimension of collaborative, incremental software delivery to life. The interplay and emotions between the main characters as they struggle to let go of past behaviours while bridging toward new ones is palpable and quite close to what one can expect to see in the field. While some situations do feel "manufactured" for a certain outcome, it's understandable from the perspective of advancing the story and not becoming a John Le Carre novel.

Combined, this makes for a really good read that educates while it entertains.

Significant Highlights:

The Power of Scrum is, for me, chock-full of Scrum goodness: I really liked how common misconceptions and errors are discussed at just the right moments in the story through emphasized, bold-faced dialogues between Mark and Jerry (so much so I've highlighted them all in my Kindle for future reference). I also liked how "the essentials" of each chapter were captured and summarized in bullet points to help reinforce the concepts that were introduced. Additionally, I found several parts of The Power of Scrum interesting from the perspective of Scrum adoption and execution that make it a stand-out effort:

First, and perhaps most significantly, Mark decides to use Scrum on an active project in distress and not a clean-cut, greenfield opportunity or low-risk pilot project. It's a messy situation with the threat of layoffs and lawsuits in the balance that harkens back to Scrum's early implementations on similarly risky, in-flight projects with high stakes.

Second, Mark decides against imposing Scrum on his teams, choosing to let them decide if it should be used to solve their delivery problems after an interactive meeting with Jerry. This sets the tone for one of my favourite Scrum truisms: Treating professionals as rational adults and not infantilized lumps of clay that need to be told what to do every waking minute of their working lives.

Third, the project is started without delay: The team decides its goal (ie. remove everything that doesn't work) for a first release in two weeks and plans their Sprint without a lot of overhead. It takes the better part of a day to sort it out, which is about right for a new team.

Fourth, Jerry shares the Scrum Master role with a young, up-and-coming test lead, George, to help show him the ropes. Mark selects George over some of the more senior staff because he showed an aptitude in the initial meetings about introducing Scrum for being fair and open-minded, and this comes with the real consequence of a senior developer, Vince, harbouring resentment.

Fifth, the current Project Manager, Rick, is cajoled into the role of a Product Owner Proxy instead of getting one from their customer. This is often the necessity for some teams who have customers that cannot or will not provide an individual responsible for the product vision and direction.

Sixth, the description of estimation, task breakdown and the use of a Scrum Board to track progress are covered. While not "strictly Scrum", they are well-established, time-tested practices that make the entire system work.

Seventh, the Sprint Retrospective is emphasized as a critical part of team continuous improvement.
Eighth: A latop gets thrown across a room and wedged into a meeting room wall. While not at all related to Scrum, we've all wanted to do that at one time or another... ;-)

The Cheesy Bits

Of course, there's always a risk with writing a business fable about how to resolve wicked problems using a lightweight framework: You need to provide a backstory that gives depth to your characters. The Power of Scrum suffers only slightly from this with the exploration of Mark Resting's personal relationship with his wife and its loose mirroring in how he approached projects, ie. waiting for everything to be perfect and known before making decisions. While a useful allegory, I didn’t find this to add significantly to the main thread of the story and I could easily see it being the bits that would be edited out for a movie adaptation.

Another weak spot was the ready acceptance of personal fault by the project manager, Rick, for past delays and not being open to fully embracing the role of Product Owner. In real life it is rare to find this level of maturity - it is quite more likely to encounter varying degrees of indifference and subterfuge that accumulate to threaten the project.

Who Should Read It? Why?

Anyone interested in knowing more about Scrum and how it can be applied to turning around projects in distress should read this book, from developers to testers to managers and executives. It is written in a very approachable way that makes understanding the material entertaining and effortless and can begin to open the dialogue about Scrum adoption in practical terms.

Bottom Line?

Despite a few weak spots, The Power of Scrum is a well-written short story about implementing Scrum on a risk-laden project that offers useful lessons in how to apply it in a real-world scenario. While well-versed in Scrum, I found myself totally engrossed in the narrative and how the characters mount an effort to take back control of their project's destiny through incremental delivery and continuous improvement. The challenges, setbacks and wins are all very believable and similar to what I've witnessed as a Scrum Master and coach over many years.

This book needs to be on your bookshelf or Kindle and that of your managers and business owners. Pronto!

The Power of Scrum (Paperback)

The Power of Scrum (Kindle)

Making Sense of Wicked Problems with Issue Mapping and Scrum

Thu, 16 Feb 2012 04:36:45 GMT

Recently, I had the good fortune to be able to attend a class a long-time acquaintance, Paul Culmsee, was offering on Issue Mapping. This was one of those once-in-a-blue-moon events because Paul lives and works on the other side of the planet (he's from Perth, Australia) and he was making a special trip out to deliver the course on the invitation of a local software consulting firm here in Toronto, Navantis, Inc. I've known Paul for several years and have followed his uncommon-sense blog, Cleverworkarounds.com, for some time, and I participated as a reviewer for the book he co-authored with Kailash Awati and had published last year, The Heretic's Guide to Best Practices. It was in this book that he laid out how he uses Issue Mapping to help organizations disentangle their problems.

So, you could say I was a motivated attendee; but I digress: This was an excellent class.

Issue Mapping is a technique for facilitating productive meetings that deal with wicked problems, ie. those that are difficult to clearly identify and resolve because they often have incomplete specifications, contradictory elements and fluid properties that are difficult to discern or categorize and inherent, complex interdependent parts. Agile software delivery advocates have long-known about the wicked problems involved in software development (I previously wrote about them here) - frameworks like Scrum, XP and Kanban are all aimed at conquering and dividing the chaotic forces that wicked problems unleash on teams charged with delivering value. Naturally, I found the topic of techniques for disentangling complexity in groups extremely interesting.

Issue Mapping Explained

Issue Mapping was first devised by Jeff Conklin who leveraged the research of Horst Rittel and Melvin Webber to help groups "unpack" their wicked problems (see his book Dialogue Mapping: Building Shared Understanding of Wicked Problems). It is comprised of three simple elements:

A Shared Display such as a laptop connected to a projector, or the good old whiteboard or flipchart;
A Notation with simple rules for how content is captured into the Shared Display;
A Facilitator who is skilled in capturing group interactions into the Shared Display using the Notation schema.

(**Note: I will be using the terms Issue Mapping and Dialog Mapping interchangeably

Issue Mapping notation is based on Horst Rittel's IBIS or Issue Based Information System argumentation scheme which was first developed in the 1960s and 1970s to support the coordination and planning of political decision processes. The basic premise of IBIS is that irrespective of how complex or wicked a problem is, it can be distilled into four basic elements or nodes: Question, Idea (or Answer), Pro arguments and Con arguments. These nodes are assembled according to a basic grammatical structure that builds the "map" of the issues under discussion by a group:

Like a standard English sentence, the map is read from left to right beginning with a root question that opens the exploration. Ideas are then presented in response to the question with supporting Pro or dissenting Con arguments, which can give rise to succeeding Questions and Ideas. Relationships between the nodes are shown with arrowed lines pointing to the left to show the path back to the root question. While the map can be freehand drawn on a whiteboard, the standard is to have the facilitator use a lightweight tool such as the freely-available Compendium, which I used to make the above map. Once you get the knack, it becomes quite fluid to create the maps as the conversations flows.

Making Argumentation Visible, Creating Shared Understanding

Issue Mapped meetings differ significantly from their stodgy counterparts by being significantly more interactive. Instead of a PowerPoint death march where attendees are talked at or through, they are now engaged as "designers and contributors" to the meeting's content and resolution by the facilitator who helps them see the logic of their problems in real-time, causing a natural shift of their attention toward the screen as the problems are unravelled. This has a powerful grounding effect that helps to minimize the impact of distracting influences (what Jeff Conklin refers to as "fragmentation") that would otherwise prevent reaching a shared understanding of the issues.

How Can Issue Mapping Benefit Agile Projects?

I see Issue Mapping as a natural fit for Scrum teams and projects and especially for Scrum Masters and Coach/Facilitators:

Project Chartering and Vision: Perhaps the most thorny of wicked problems - "What should we build? Can we build it in time? In-house or contractors? A mix? Should we do this at all?" Issue Mapping can help facilitate this decision-making process toward a useful conclusion.

Setting Sprint Goals: Similar to visioning, but with a reduced scope - Issue Mapping can provide a fantastic means for capturing why certain decisions were taken to set a Sprint Goal and to help align the team and Product Owner.

Kickstarting a Product Backlog: Often, Scrum is brought in to help remedy a project that is in dire straits, such as the one Jeff Sutherland describes in his short business fable, The Power of Scrum. In these situations, there isn't usually enough time to do a proper build-out of the Product Backlog as would happen in a greenfield project - you just need to determine what you can accomplish in the next two weeks to show progress. Issue Mapping can help bring everyone into alignment on the objectives they want to set for the first iteration.
Creating a Definition of Done: The #1 gotcha that trips up most new and even experienced agile teams - "How will we know when we've completed a feature?"

Facilitating Sprint Reviews: Capturing the feedback that stakeholders generate while inspecting a Sprint's product increment can sometimes be chaotic. Issue Maps could be used to give some coherence to the commentary and help to inform the next Sprint Planning session.

Facilitating Sprint Retrospectives: This was my first immediate thought for using Issue Mapping within a Scrum project because it has to do with aligning a team's expectations of itself with a future state they want to achieve. Continuous improvement is sometimes a difficult nut to crack for new teams - a brief session with maps could help illustrate what problems the team believes they need to work on next.

Intra-Team Conflict Management: Working on a Scrum Team can be stressful: Take a group of intelligent, creative people, throw them into a pressure cooker where they need to strive together to deliver value and ROI every two weeks and you are bound to have blow-outs. Since Issue Maps are inherently neutral, they can help explore why problems occur and what can be done to resolve them. By holding a mirror up to reflect them back to the team, they can use the session to take the pressure out of the situation and come up with a respectful path forward.

Do I Need a Facilitator?

Absolutely. Just as you need a Scrum Master to ensure the rules are followed and the team and Product Owner are working as productively as possible, Issue Mapping is dependent upon a skilled and impartial practitioner. Because wicked problem solving can involve some heated and impassioned debate, it's necessary to have a facilitator who doesn't have a "horse in the race" so as to avoid situations where they may be accused of bias. While the rules of Issue Mapping and IBIS should take the starch out of these scenarios most of the time, I think it best to have someone who is naïve to the topic so that they can reflect-back statements and arguments to encourage clarity for everyone at the table and not just the acronym and abstract concept junkies.

How Do I Get Started?

I recommend picking up two books: Dialogue Mapping by Jeff Conklin and The Heretic's Guide to Best Practices by Paul Culmsee and Kailash Awati. The first gives you the basics of using IBIS for mapping wicked problems while the latter is a "sequel" that expands on how to use Issue Mapping in contemporary settings. There's more to this than just tossing up some nodes on a screen - there is value to understanding the types of questions that arise in mapping problem wickedness and strategies for gaining insights and clarity. And of course, you can also take an Issue Mapping course from Paul or one of his authorized designates at Seven Sigma, LLC!

Getting familiar with the IBIS notation and using a tool like Compendium to capture notes for Scrum meetings is a good way to begin before trying it out with your team. I'd start with simple dialogues such as roughing in your Definition of Done or your next Sprint Retrospective to get the feel of capturing dialogue in real-time with a forgiving audience. Don't make a big announcement that you're going to do Issue Mapping - just let it happen and answer the questions that naturally arise and follow where the conversation leads.

In Defense of Estimating with Smaller Numbers

Wed, 07 Dec 2011 02:37:54 GMT

Recently, Mike Cohn blogged about his disappointment with how some teams, upon receiving a new deck of planning poker cards, immediately toss out the high-value cards: 20, 40, 100. For the uninitiated, Planning Poker is a tool for group estimation where Product Backlog Items (PBIs) are assessed a point value based on a modified Fibonacci sequence:

While originally created by James Grenning, Mike is largely responsible for popularizing this approach to group-estimation, and it features prominently in his book, Agile Estimating and Planning, and his Scrum certification courses.

In brief, Planning Poker is played in rounds with each team member simultaneously "playing" a card with the point value they think best represents their assessment of the size, complexity, uncertainty and risk of a particular user story or feature, with higher numbers predictably reflecting greater discomfort and understanding.

In subsequent rounds, stories or features are assessed relative to one another to speed things up, ie. if Story A was assessed 8 points and Story B is slightly less complex or risky, the team may elect to assess 5 points. The reasons for using points instead of hours is a well-explained topic which I won't cover here - suffice to say that it helps teams begin to get a handle on what they can accomplish in an iteration over time.

Looking at the illustration above of Mike's deck, it certainly presents a bewildering array of choices for a team: It's like an estimating Swiss Army Knife. In my experience, irrespective of deck size, teams tend to coalesce their estimates within the following scale:

As they become less certain and want to express this in the backlog, they move further up the scale:

21 point stories tend to be those that the team feels are not quite ready for prime-time or haven't matured to be reliably estimated. They may be too vague or the team doesn't possess enough technical or business domain knowledge to break down.

But what about going beyond this? Into the realm of 40, 100 and 'infinity' cards? What distinguishes between them? Personally, I've found that teams tend to play these cards as a form of comic relief - the stories are just so complex, so improbable as to be beyond any sense of reliable conveyance. Often I've been asked if I could include a "WTF?" card to play in these situations.

Basically, when the team plays these cards it's a cry for help: They haven't the foggiest how to deal with the story in its present form, so it should either be decomposed or sidelined until they can become more familiar with what's required to deliver it. However, this raises this question: Why do we have these cards in the series?

These Amps Go To 11

In the famous heavy metal rock band mockumentary, This is Spinal Tap, there is a well-known scene where band guitarist Nigel Tufnel tries to impress an interviewer with how loud his guitar amplifiers are because they can be turned up one more "notch" than standard amplifiers which only go to "10". When the interviewer asks him why not make 10 the loudest setting, a bewildered Tufnel replies: "These go to 11."

While hilarious, Nigel's attachment to his amps illustrate the same cognitive bias that underpins the 40, 100 and infinity estimation buckets: At a certain point both the "loudness" of the amps and the uncertainty and risk of an estimate aren't better represented by inconsequential placeholder values: It's all relative. And that's the point.

Ambiguous Precision

In his post, Mike argues that while not always used, these upper-scale estimates should be retained because to do otherwise "is like deciding to strike 'millions' and 'billions' from our vocabulary just because our bank balances are in the thousands". This "ambiguous precision" is necessary to provide extremely coarse-grade estimates that may not be taken to the bank, but can help to inform whether a release will land in this quarter or one further down the road depending on the customer's tolerance for error.

Perhaps - but I'm inclined to disagree because once we get into the 'millions' and 'billions' in our estimates they begin to mean less and less to us and our customers because we are more concerned with the "thousands". By retaining the additional "zeros" we're not adding any clarity, and indeed it becomes somewhat semantic when we start to try and delineate the difference between a 40, 100 and infinite point story - how many angels can dance on the head of a pin?

What we need to bear in mind is that these are estimates. As Barry Boehm and Steve McConnell pointed out decades ago with their famous Cone of Uncertainty graph, they are most unreliable the further out from implementation that they're devised:

By rationalizing that we need grossly ambiguous numbers to forecast distantly future releases puts a little too much stock into why we estimate with a non-linear point scale in the first place. It's not about any degree of precision - we just need to convey that the further in the future we try to look ahead, the more likely we are to be really, really wrong.

In his paper introducing Planning Poker, James Grenning notes:

As the estimates get longer, the precision goes down. There are cards for 1,2,3,5,7,10 days and infinity. This deck might help you keep your story size under 2 weeks. Its common experience that story estimates longer than 2 weeks often go over budget. If a story is longer than 2 weeks, play the infinity card and make the customer split the story.

Similarly, in his 2005 book, Mike suggests:

Because we are best within a single order of magnitude, we would like to have most of our estimates in such a range. Two estimation scales I've had good success with are:
1, 2, 3, 5 and 8 [Fibonacci]

1, 2, 4 and 8

In other words, a team's confidence in their estimates is expressed as a function of the inverse relationship between uncertainty and risk and reliability and precision - this is the intended consequence of using a non-linear point scale:

However, there is a point of diminishing returns that occurs when we add more numbers to the scale - it becomes increasingly difficult and consequently pointless to assess confidence. While we could argue that a 100-point story is an order of magnitude more complex and uncertain than a 40-point story, or two orders of magnitude more complex than an 21-point story, it doesn't convey what's more important: Irrespective of the value, it's a signal of diminishing confidence from the team.

In my experience, the early warning signal is tripped when I see 21 point stories starting to appear in the Product Backlog that's been prioritized for an upcoming Sprint Planning session, indicating that as far as the team is concerned, the feature may not be in a reliable state for inclusion in the Sprint. This doesn't mean it's an automatic out, but rather one that needs further discussion and a shared understanding before committing - sometimes the high estimate is tossed out just to allow the team to push back on being overloaded This is an indication from the team for assistance. It could remain on the backlog, or become a candidate for refining and decomposing. In any event, it is a signal.

However, even with a "21" point story, teams usually have some idea of what to do. What about features that are added to the backlog as rough ideas?

Introducing The 'X' Factor:

In mathematics, the first three to four letters of the alphabet are designated for known quantities while the last three are reserved for unknown quantities. In keeping with this tradition, I'd like to suggest an alternative estimate signal for features and stories that teams cannot reliably estimate with reasonable precision: x.

By replacing the 40, 100 and infinity cards with an 'x' card, teams have options for how they want to flag stories that need attention:

They can assign'x'an agreed-upon value, eg. 50 or 100, if it makes everyone feel better having this ambiguous certainty, or;
They can leave 'x' as an explicit signal that a story isn't developed-enough for inclusion into a Sprint and requires more work to either break down, refine acceptance criteria or even provide a definition of 'done', or;
They can indicate stories for exclusion from the backlog.

In all cases, seeing an 'x'-estimated story suggests an immediate and natural conversation needs to occur about what to do next.And this is cuts the core about why we use story cards and planning poker instead of byzantine use cases and GANTT charts: They are cues for continual conversations about what needs to be built.

With respect to the forecasting quandry that Mike relates in his blog post, given that any estimate in the upper-echelons is incredibly speculative to begin with, 'x' can be assigned a raw value and added accordingly. However, my preference would be to discuss the situation with the customer: If there is a preponderance of 'x'-factor stories on the backlog that the team and Product Owner aren't able to delineate, we can only reliably forecast against those which we can reliably estimate.

Anything else is pure conjecture - and needs to be explicitly stated.

What do you think? Do you agree with using 'x' as a replacement for the 40, 100 and infinity buckets? Is there a better sequence?

The Fallacy of Division of Labour in Software Projects

Mon, 14 Nov 2011 21:39:28 GMT

One of the most enduring myths in software systems development equates it to the building of a house or similar structure. While a seemingly useful abstraction to express what parts of a solution we're creating (eg. foundation, framework, architecture), it really obscures the actual nature of the work involved by implying that engineering and software projects share a common basis in how they are planned, designed and delivered.

This false equivalence has in turn given rise to the fallacy of division of labour in software projects which presupposes that design activities (eg. requirements gathering, analysis, architecture) can be divorced from construction activities (eg. programming, coding, packaging) thereby enabling the creation of predictable delivery schedules according to team size. In other words, according to the John Heywood proverb, "many hands make light work".

Unfortunately, tasks in software projects do not divide as cleanly as they do in civil engineering projects like house building or widget manufacturing while assuring similar precision. Consequently, there is a rather significant mismatch between the reality of software development activities and the current plan-driven, single-pass/phased methodologies currently used to organize them. In this article, I examine the root causes of this mismatch.

The Default Model: Division of Labour in Engineering Projects

Much of our so-called "modern" ideas about the efficiency and effectiveness of IT project delivery borrow quite heavily from engineering and manufacturing disciplines - which is problematic because the nature of the work, as we'll see, is entirely mismatched to this type of organization. I call this the "Default Model" because it represents the status quo that is found in many IT and software shops.

Under this model, design and construction activities are separated to create a cost-effective, predictable schedule that can employ people with lower skills to build the end-product. Highly-skilled (and therefore expensive) designers and architects create "instructions" that can then be handed off to lower-skilled (and therefore inexpensive) workers to "construct". In terms of effort and cost, this tends to break down as shown below:

Thus:

In a traditional civil engineering project, design activities account for the minority of overall effort and cost while construction activities constitute the majority.

This model presupposes that the delegated tasks undertaken can be perfectly partitionable with no required intercommunication between workers or partitionable with intercommunication between workers. In his 1975 landmark book, The Mythical Man-Month, Fred P. Brooks Jr. described this relationship between workers and months thus:

Men and months are interchangeable commodities only when a task can be partitioned among many workers with no communication among them… In tasks that can be partitioned but which require communication among the sub-tasks, the effort of communication must be added to the amount of work done.

In the first scenario, think reaping wheat, splitting wood or putting nuts and bolts together - these tasks can be divided evenly across a labour force without cross-communication while providing the requisite precision. Henry Ford famously exploited this concept through the use of automated tooling machines that created parts which could be easily assembled by workers into finished products.

In this case, effort effectively becomes a multiple of the men added to a project because there is an approximately one-to-one exchange:

Credit: Fred P. Brooks Jr.

In the second scenario, sub-division of tasks can only be accomplished through intercommunication between workers, creating a "drag" effect on productivity due to the effort required to ramp-up additional workers on tools, technologies, techniques, etc. This blunts the impact of adding workers to shorten schedules:

Credit: Fred P. Brooks Jr.

The time (and thus cost) required to "ramp" and coordinate additional team members comes in pairwise multiples. To illustrate, consider the following cross-communication between two team members:

Now, the same cross-communication that may need to occur between three team members (3x Single Pairwise Communication):

And now with four team members (6x Single Pairwise Communication):

The Fallacy of Division of Labour in Software Projects

In his 1992 essay for C++ Journal, What is Software Design?, Jack Reeves makes two critical observations that effectively blows apart the view of software projects as analogous to engineering projects:

• First, in software, construction is so cheap as to be free;
• Second, in software, all effort is design

Software is "constructed" through the use of automated developer tools that convert source code into machine language and package it for installation on target systems; "design" is performed by a team of highly-skilled (and often highly-paid) developers, business analysts, testers, UI/UEX designers and architects throughout the lifetime of the project. Given that I'm mostly directing this article toward business owners and managers, let me underscore this point:

In a software project, construction is performed by machines; design is performed by the entire development team over the lifetime of the project.

This has a significant impact on the divisibility of tasks in a software project and by extension how the project can be planned and executed. In the default approach described earlier, the dual nature of the effort involved makes the separation of design from construction possible. With 90% of the effort involved in physical construction, the activities readily lend themselves to efficient division across a workforce. As the project takes on added technical complexity, however, intercommunication is required to enable other workers to assist.

In a software project, men and months are not interchangeable commodities because the work involved is by definition technically complex, requiring corresponding interrelationships between team members. The effort required to communicate knowledge to ensure the necessary precision for sub-dividing tasks quickly overtakes any advantages of having multiple workers and further makes it impossible to control schedule or cost through the simple addition of manpower:

Credit: Fred P. Brooks Jr.

This realization is known as Brooks Law: Adding manpower to a late software project makes it later.

This point confounds traditional phased software delivery projects that divide their teams according to specialization and phases of activity with the expectation that men and months are interchangeable and can be done so without much consequence. The necessary interrelationships are weak with infrequent cross-communication between team members in different phases where it would be most beneficial. Activities even within the same phase become siloed according to specialization, leading to the most injurious and risky practice in software projects where all the disparate pieces developed over months or even years come together in a cataclysmic rush in the waning days of the project called "big bang integration".

The imprecisions in technical fit (aka "bugs") arrive in torrents at a time when the team is least able to resolve them due to budget and schedule commitments that were made long ago. The outcomes are unsurprising: Late delivery of solutions, uneven quality, demoralized/disinterested teams, unsatisfied customers and a moribund industry plagued by projects with nearly-ruinous cost overruns.

Coming to Terms with Reality

Software projects are fundamentally different from traditional civil engineering projects like house building or widget manufacturing: The nature of the work is almost entirely design-oriented, making the use of processes and methodologies based on the separation of design and construction activities with little to no inter-communication pointless and futile. By attempting to coerce design activities into phases that disregard or downplay the realities of the work, tension is created that becomes expressed in schedule slippages and poor quality. An automatic reaction is to blame the developers and the managers rather than the process they are working under, which leads to counterproductive behaviours to plan more, be more precise with estimates, add more developers and the like - and all without any commensurate gain in productivity or ROI. This mismatch is the root cause of software project failure.

The remedy is to come to terms with the reality of the nature of design-oriented work and apply a process that permits it to flourish within controlled boundaries. In agile software development this is expressed in frameworks that promote the trinity of visibility, inspection and adaptation.

In the next article, we will delve further into the constituent elements of software development and how agile practices help to focus them for productive gain and resultant ROI.

References:

Brooks Jr., Fred P., The Mythical Man-Month. University of North Carolina at Chapel Hill, Addison-Wesley, 1995.

Five Reasons Not to Mix Agile with Waterfall

Thu, 27 Oct 2011 13:11:19 GMT

A perennial topic within the agile project management community (and that of their customers) is whether a middle ground exists that can "bridge the gap" between single-pass/phased (aka 'waterfall') methodologies and agile (Exhibit A: Agile Hybridization - Novel Experimentation or "They just don't get it.") as a pragmatic approach. In my experience this is a normal reaction when confronted with "radical" change from the status quo in software and IT project management that has ingrained an expectation of simplicity. I've likened this to learning how to swim while holding on to the edge of the pool. It takes a lot of cajoling from the instructor to "let go" and discover your body's natural buoyancy, but once done you wonder what the fuss was about.

Unfortunately, this strategy rarely provides the advantages in productivity or predictability that the project manager or business wants because the two processes, agile and waterfall, view projects, teams and people and how they work together in very different ways. They mix together about as well as oil and water.

In brief, this is because:

Single-Pass/Phased Delivery methodologies are used when the intermediate activities that are involved in a project (ie. the "phases") are well-understood and not subject to much variability or unpredictability, and therefore can be reliably laid out in a linear process that will produce acceptable, identical results irrespective of who does the work. Design activities are separated from construction. Think: An assembly line.

Agile Delivery frameworks are used when we the intermediate activities that are involved in a project aren't well-understood and contain numerous unpredictable and wicked problems that resist traditional linear planning techniques due to high uncertainty and risk. This necessitates an empirical process based on the principles of transparency, inspection and adaptation to manage effectively and outcomes are entirely dependent on the skills of the people involved because all the involved effort is design. Think: New product development.

All agile practices share a pedigree with the lean manufacturing techniques pioneered by Taiichi Ohno for the Toyota Production System. Lean practices view all the activities involved in delivering a project as a work system which can and should be regularly monitored and adjusted according to current conditions and problems; waterfall practices do not share this perspective as they are plan-driven.

Agile framework roles, events and artifacts are designed for maximizing productivity and efficiency within a succession of time-boxes that are regularly inspected and adapted; this is foreign to single-pass/phased methodologies which are infrequently inspected and adapted and generally intolerant to change as they are dependent on rigidly following an up-front plan.

Finally, you're delaying the inevitable decision: The sooner you transition fully to agile, the sooner you can begin learning how to apply the rules to your projects.

If you are introducing change within your organization toward adopting agile, you need to be fully aware of the underlying motivations. If there is a real need to improve productivity, profitability, ROI, customer satisfaction and team morale, agile frameworks like Scrum and XP can significantly help, but only when used in their entirety (ie. the roles, events and artifacts) because they comprise a complete work system. Like anything worth doing in life, it takes time to become proficient and the results are entirely dependent upon your team and organization and their attitude toward learning a new skill.

Visibility, Inspection and Adaptation: Competitive Power Tools

New e-Newsletter Launching: The Shift Cable

Thu, 06 Oct 2011 19:00:48 GMT

When I launched Derailleur Consulting earlier this year, one of the objectives I set for the company was to reach out to business to engage them in open conversations about agile project delivery and how it can help elevate their ability to compete with shorter times-to-market, greater ROI and profitability and higher productivity. While we've already made some progress on this goal, there's a lot more to do and the stakes could not be higher.

As Bent Flyvbjerg and Alexander Budzier illustrated in their article in the September 2011 Harvard Business Review, IT projects now present a "singular new risk" to business and government as a result of their scale, scope and reach not only within their organization but to others as well. Soberingly, they note from their research that it's not a question of if disaster will occur, but when:

"It will be no surprise if a large, established company fails in the coming years because of an out-of-control IT project. In fact, the data suggest that one or more will."

More shockingly, they note that there is an overwhelming reluctance to cancel projects when they are obviously failing. The impact of these catastrophes will be significant, resulting in layoffs that extend beyond the organization itself to others who are dependent on them. In the public sector, it will mean losses that can wipe out large swathes of taxpayer dollars, resulting in core services being cut along with significant tax increases at a time when we can least afford them.

Clearly, the imperative to do better is upon us. We need to begin to seriously address the challenges of delivering complex IT solutions while respecting the fundamental principle of ROI expressed in working software. We feel quite strongly that the time for half-measures is well past; it is now the time to meet the challenges head-on and begin to change the way we think about, plan and execute IT projects.

With this thought in mind we are pleased to announce the launching of a new communications platform to provide customers, colleagues and other interested professionals with access to our insights and analysis on the latest developments in agile software project delivery called The Shift Cable. This FREE email newsletter will go beyond topics that we may blog about to surface and discuss issues that help to amplify the value proposition for adopting agile practices in the enterprise - irrespective of size. We will explore ideas and influences from a wide array of sources to arm you with the material you need to have solid, informed conversations with your customers, managers and teams about why agile practices are more than just a fad, but an effective strategy to turn the tide of project failures and begin delivering real ROI.

The Shift Cable will initially publish twice-monthy, on the 1st and 15th, with additional "Special Reports" going out when there is a breaking issue we feel you should be aware of.

Chris R. Chapman
President and Founder, Derailleur Consulting

Scrum Team Norming

Tue, 04 Oct 2011 19:00:56 GMT

Earlier today I came across a discussion on the Scrum Practitioners LinkedIn group where a member asked "How long before a Scrum team settles in?":

"Initially a new team to Scrum will be unfamiliar with the processes, not good at estimating, and generally speaking a little wet behind the ears. Some people think it takes about 2months before a new Scrum team settles into the process, but how long do you think it takes?"

The responses ran the gamut from 2-3 months to 2 years. Some responded that the real measure is the number Sprints because it helps reinforce the rhythms of Scrum (eg. Sprint Planning, Daily Scrum, Sprint Review). I would tend to concur that this is the more important measure and it is why I often recommend customers to not bother spinning up a Scrum project without a budget for at least four iterations - it takes this long just to get the team to start "gelling" and to begin to adapt toward new behaviours, not the least of which are technical, eg. automated testing.

In this context, Bruce Tuckman's canonical "Stages of Team Development" is often referred to characterize the state of the team's maturity, eg. Forming, Storming, Norming and Performing, with Norming often mischaracterized as the desired state. Tuckman argued that teams will go through these stages as they begin to adapt to their surroundings and coalesce around a goal and begin to tackle problems and deliver results:

In the first phase, "Forming", a new team is just getting to know one-another and understand the reasons why they've been assembled and the goals they have been charged with taking on. It is characterized as a period where team members are in conflict-avoidance mode so as to find compatibilities for the tasks ahead.

The next stage, "Storming", defines the ability for the team to mature to successive levels. It is characterized as a period of increased competition among the team members for advancing candidate solutions and the leadership structure that they will adopt. As a result, there can be a level of tension and discomfort as team members begin to muscle for gains, often chafing those who are less outgoing. This is a critical stage that takes time and patience to resolve - not all intra-team conflict is bad, and can be healthy over the long-term.

If a team survives Storming, they eventually progress into "Norming". In this stage, the team is in full acknowledgement of their goal and work aggressively to meet it, even if it means giving up on their pet-solutions. There is a mutual plan that the team develops and executes, with all members taking responsibility for its success.

High-performing teams may elevate beyond Norming and progress into "Performing" where they function as a highly-autonomous, cohesive unit. Teams in this stage are considered to have reached an elite level of maturity and self-organization, requiring little supervision or intervention. This stage may take months to years to attain and is by definition fragile - it can be easily up-ended with a simple change in membership, throwing the team back into the Storming stage.

All Scrum teams aspire to reach the Performing stage - indeed this has been the experience of early Scrum adopters who found themselves in situations where they were significantly exceeding expectations:

"By having every member of the team see every day what every other team member was doing, we began to get comments from one developer that if he changed a few lines of code, he could eliminate days of work for another developer. This effect was so dramatic that the project accelerated to the point where it had to be slowed down. This hyper-productive state was seen in several subsequent Scrums, but never so dramatically as the one at Easel… [In] the hyper-productive state, the initial Scrum entered the 'zone'. No matter what happened or what problems arose, the response of the team was always far better than the response of any individual." (pp. 12, 15, Agile Software Development with Scrum, Schwaber & Beedle)

Typically, new Scrum Teams will start their first Sprint with Forming and Storming behaviours and remain in this semi-gelled state for the duration of the project. Sometimes they will rapidly move through the stages and settle in Norming or even Performing. Sometimes they will advance and retreat across several stages. Tuckman acknowledged that even high-performing teams will fluctuate according to changes in context and conditions.

To the outside observer, this may seem very chaotic - and in some respects this is true: An organization is considered more chaotic when it employs dynamic configurations with few static patterns. When there is order within this chaos (as with a self-organizing, cross-functional Scrum Team), the organization is considered to be at the edge of chaos. Examples of other such organizations in this state include sports teams like soccer, hockey and basketball, and tactical military and police units.

For Scrum Teams, this fluctuation in team formation and productivity is expected because they are self-organizing against potentially evolving circumstances. Whereas Tuckman saw the Storming stage as a critical one that teams needed progress through and resolve quickly before ascending to higher levels of collaboration and cooperation, for Scrum Teams it is a semi-permanent state that is returned to at the start of each Sprint, before progressing to Norming or Performing because they are always operating on the edge of chaos.

As such, it's not always appropriate to judge a team's effectiveness based on whether they seem to have entirely "normalized" or "settled in". Agile teams expect and embrace change, and this can and will cause shifts in the team's organizational dynamic. Scrum helps teams manage this chaos by providing stable, predictable containers and rhythms within which to work, eg. the Sprint and attendant events to inspect and adapt progress.

In this respect, there's not a hard-and-fast rule that will tell you when you can always expect a team to reach a desired Performing state. It really depends on the individual members and how well they learn to use conflict as a healthy process for driving out the best ways to arrive at a solution that will meet or hopefully exceed a Product Owner's expectations. It takes patience and continuous observation and mentoring to ensure that the team is delivering effectively, with the best measure in the quality of the potentially-shippable product increments that are being delivered.

What do you think? How has your team's self-organization maturity evolved over time? What were the most impactful events? Feel free to leave your comments below.

Derailleur Blog

Understanding Misalignments in Software Development Projects

Misaligning the work we do with how we do it:

The Six Fundamental Misalignments

Fallacy of Division of Labour // All Effort is Design

Defined Process Control Models // Weakly-Defined or Continuously Variable Inputs

People as Role-Based, Replaceable Components (Resources) // People as Unique, Skilled Contributors

Command & Control Team Management // People as Self-Organizing, Rational, Skilled Adults

Task Switching // Effective Completion of Tasks, Predictable Quality

Hero Culture // Teamwork

Almost Inextricably Out-of-Alignment

Three Contract Models for Scrum Projects

Change For Free

Created By:

In Brief:

Key Feature:

How it Works:

Customer and Contractor Obligations:

Money for Nothing

Created By:

In Brief:

Key Feature:

How it Works:

Example:

Customer and Contractor Obligations:

Ranges and Changes

Created By:

In Brief:

Key Features:

Customer and Contractor Obligations:

See Also:

Business Card "B" Side Agile Quotes

There's A Lot of Room for Agile to Grow in 2012

Scrum Project Safety Checklists

Misaligning Incentives with Performance Outcomes

The Red Bead Experiment

The Red Bead Experiment

Worker

Day 1

Day 2

Day3

Day 4

What Does the Red Bead Experiment Tell Us?

Misaligning How You Work with the Work You Do: Why Waterfall and Agile Don't Mix Redux

Agile == Empirical, Waterfall == Defined

In Software Projects, All Effort is Design

Productivity Will Worsen, Risk Will Escalate

Conclusion: Agile Adoption is about Realignment of How You Work with the Work You Do

How I Deliver Agile Consulting

It's a Passion, not just a Profession

Straight Goods - No Fairy Tales

Workshops, Training and Games

Tchotchkes, Pamphlets, Brochures and Cheatsheets

Having Fun Changing Work

Get Trained, not Certified. There is a Difference.

The Cost of Waterfall Contracts

Ouija Board Team Estimating

Set-Up and Rules:

Ouija Estimating:

Deciding Estimates

The Two-Week Project Bootstrap Explained

What Happens in a Two-Week Project Bootstrap?

Scrum Training for Everyone

Filling the Roles

Planning the Sprint

Executing and Reviewing the Sprint

Go/No-Go?

The Power of Scrum Revealed

What is it About?

Why is it Compelling?

Significant Highlights:

The Cheesy Bits

Who Should Read It? Why?

Bottom Line?

Making Sense of Wicked Problems with Issue Mapping and Scrum

Issue Mapping Explained

Making Argumentation Visible, Creating Shared Understanding

How Can Issue Mapping Benefit Agile Projects?

Do I Need a Facilitator?

How Do I Get Started?