Can Icebergs Be Scaled?

Over the years, there have been numerous variations of the Product Backlog model which support the progressive breakdown of features just in time. Why wait to elaborate? As per Lean principles, there are a few reasons:

• We want to wait as long as possible to flesh out the details. After all, we have the most information about these details right before we’re ready to start building them.
• We want to keep the backlog small and minimize work-in-progress so that there is a lower probability that our requirements will fall out of date.
• We want to minimize waste. Why risk investing in defining the details of something we may never build? We all know how quickly customer and market needs can change.

The Iceberg Model

Stories and Epics: Same Backlog, Different Sizes

The Iceberg Model was made popular by Mike Cohn in his book Succeeding with Agile (1). In this model, Epics are just big User Stories which are progressively broken down. There isn’t a clear distinction between an Epic and a Story.

The model is simple and effective for individual Agile teams in smaller organizations. An Epic and Story are expressed in the same format and are part of the same backlog.

In addition, writing Epics and Stories can happen in both directions. Epics may be broken down into Stories, or a group of Stories may be consolidated into a single Epic to facilitate prioritization at a higher level.   Stories can also be grouped together by Themes, which function as categories.  See Mike Cohn’s blog entry on the topic for more information.

Can Icebergs Scale?

In the enterprise, senior leadership evaluates which systems, products, and applications to invest in. How are these decisions made?  Traditional methods include detailed business cases which can be expensive to create and have long review cycles.

However, we can use Agile and Lean practices from the very start. The same techniques we use to define, prioritize, and estimate Epics and Stories can be used for investment decisions.

What are the dangers of not applying Agile and Lean practices at the enterprise level?

• We may put too much effort in determining the cost/benefit of one investment versus another. Forecasting is an expensive process which takes time and is often inaccurate.
• We may be making the right decisions at the smaller-scale Epic and Story level, but may ultimately be making the wrong higher-level investment decisions. We need to define, prioritize, and size at all levels.
• We may have an ambiguous governance structure. Who makes what decisions at which point? When do the decision-making responsibilities of the Product Owner transition to those of a Steering Committee as we scale?

Enter: The Enterprise Backlog Model

Rather than one backlog of Epics and Stories, we need multiple backlog levels, each with the right level of detail and a corresponding governance structure for decision-making. In my next blog entry, I’ll talk about the Enterprise Backlog Model in the Scaled Agile Framework (SAF).

(1) Cohn, Mike. Succeeding with Agile. Addison-Wesley, 2010.

Inspired by Mountains

The First Scaled Agile Summit

When we finalized a date to meet a month ago, I began discussing hotel accommodations and a meeting space with Dean. He cut the conversation short and said he had enough guest bedrooms and a large enough office for all of us. He also had great views of the Flatiron Mountains.

We self-organized quickly, with Dean as Product Owner and his wife, Bec, as our unofficial behind-the-scenes ScrumMaster who created the perfect environment, removed blocking issues, motivated us, and kept us well caffeinated. We followed Agile and Lean practices: building a backlog, pairing, working in timeboxes, and having frequent stand-ups.

About the Scaled Agile Framework

The best way to describe SAF is through our position statement on the Scaled Agile Framework announcement site:

FOR software developers, practitioners, managers, executives, coaches, consultants, trainers and agile working group team members

WHO are involved in planning and implementing Agile software development philosophies, principles, and practices at enterprise scale

The Scaled Agile Framework (SAF) is a proven, publicly available framework for applying Lean|Agile practices at enterprise scale presented in a structured and interactive web format.

UNLIKE proprietary methods promoted by product and consulting firms or methods described in dozens of books

SAF is a synthesized, holistic, integrated and summarized framework available to everyone in a readily accessible public-facing web site, along with supporting resources

Its Purpose

Building Backlogs to Move Mountains

Through our collaboration, we hope the Scaled Agile Framework will:

• Enhance the competitiveness, productivity and delivered software quality of the software industry worldwide
• Help provide the business benefits of software agility to all software enterprises
• Increase motivation, empowerment, and humanity to software development practitioners everywhere

And we surely hope to build good business value along the way!

Moving Mountains

Alex developed a logo several weeks ago which couldn’t be more appropriate: a mountain range. As part of the team Dean assembled, I knew we would move a few mountains of our own.

Watch for our soft launch on December 1st and our complete framework on February 21st. You can sign up to be notified by visiting http://www.scaledagileframework.net and typing your email address in the box at the bottom right.

Dean, Alex, and Mauricio blogged about our first SAF summit.  Check out Dean’s post, Alex’s post, and Mauricio’s post.

Agile Manifesto Signatories in 2001

The most spontaneous and engaging session at Agile 2011 was the “park bench” panel discussion with the signatories of the Agile Manifesto. Ten years later, the Manifesto is still the compass for the Agile movement.

When Bob Martin was asked what he would change about the Manifesto today, he said that there’s only one thing. “I’d color balance the background image on the website. But we all know it’s too iconic to change at this point.”

They agreed that they had no idea that Agile would take off like it did; that 10 years later, a Grand Ballroom in Salt Lake City would be packed with people who wanted to hear them reflect on the 10th anniversary of its signing.

The Next Big Thing

Agile Manifesto Signatories in 2011

Regarding the next phase of maturity, they agreed that scaling and integration will be a major focus. We need to balance emergent architecture with the needs of complex enterprise systems. We need better integration with Operations and User Experience Design. Finally, the most important in my opinion, is that we need a solid framework for middle management.

When Agile Is Just What We Do

My favorite one liner? “At snowbird 10 years ago, it was the last time this group of people agreed on anything.”

The most thought provoking? Alistair Cockburn said that the next frontier for Agile is when the word evaporates. “It will just be what we do.”

What Can Velocity Tell Us?

In my last blog entry, I discussed the guiding principles for Agile metrics.  Before moving on to a birds-eye view of Agile metric, I’d like to establish a shared understanding of a few other concepts.

In this entry, let’s discuss velocity.  In bringing up the topic of Agile metrics, I’ve often heard, “Metrics? Do you mean velocity?”  Yes, velocity is a numeric value which can be tracked, but what does it really measure?

Productivity or Predictability?

The Velocity Chart: Story Points per Sprint

Simply put, velocity doesn’t measure the productivity of a team.  It measures a team’s ability to accurately estimate its work.

What exactly is velocity?  It’s the number of Story Points of effort a team can accomplish in a Sprint.  As you probably know, Story Points are arbitrary units of measure used to “size” one Story relative to another.  Over time (usually 3 – 5 Sprints), a team comes to a shared understanding of what a Story Point is and how many it can consistently execute in one Sprint.

In the chart to the right, you’ll notice a wider fluctuation in the number of Story Points accomplished in Sprints 1 – 4. By Sprint 5, the team begins leveling out at about 16 Story Points per Sprint. What does this mean? It means that the team can estimate and deliver work more consistently.

Starting the Car

Coming up with an initial velocity estimate is similar to estimating how fast a race car can go.  You can invest in collecting and analyzing detailed data to develop your estimate from the bottom up, or you can take a leaner, heuristic approach.   What if you were to say, “Based on my 10 years of experience driving race cars, I think I can comfortably sustain a pace on this race track of 110 mph.”  To validate this estimate, you could then hop into the car, drive it around the track a few times, watch your actual velocity, and adjust your estimate based on real-time feedback.

Estimating Arrival Time

In this unpredictable world we are not all driving race cars on race tracks.  Is a car which can go 110 mph around a smooth track better than a pickup truck which can drive 80 mph over back roads?  It depends on business needs and market conditions.

So what’s the real value of knowing your velocity?  The value is in being able to consistently predict when you’ll arrive at your next destination.  And the next.  And the next.

There will always be anomalies: a key team member is sick with swine flu or a Microsoft critical update downs all database servers.  In such situations, the velocity for that Sprint can be footnoted.  It’s no more than a pot hole on a stretch of highway which blows a tire.  You fix it and keep moving.

Metrics as a Weapon

On many of the Agile discussion boards, you’ll find passionate exchanges about Agile metrics. I think people often have a visceral reaction due to past punishments for missed milestones or processes which have not been followed.

Are metrics a weapon?

I believe in Agile metrics beyond velocity and burndown charts. Before answering the question, “which metrics are most valuable in an Agile world,” we should ask ourselves “what are we trying to accomplish?”

Decision-Making Frameworks

I am a big fan of what I call decision-making frameworks. They can act as a compass to make decisions which support our guiding principles and values.

My decision framework for metrics is simple. Agile metrics are:

• Actionable
  How many PMOs produce voluminous reports which are never read? As strange as it may sound, I’ve seen bug reports circulate months after a product has been retired.
• Timely
  Metrics age faster than requirements… and faster than cheese in the summer sun. By the time they hit the inbox of a person who can take action, the problem may have already been fix. Or it may have gotten worse.
• Easy to produce
  The simpler the formulas, the easier they are to understand and act upon. They are also less likely to be misinterpreted. For those that require a conversation to be meaningful, be sure to add a comment box even if all it says is “see me for more info.”
• Created with minimal effort
  How many ScrumMasters have spent every Friday afternoon compiling the metrics for their status reports? Could their time be better spent removing the blocking issues which are dragging their metrics down?
• Created with minimal handoffs
  As we know from Lean, handoffs mean delays. Delays mean aging. Aging means that our ability to take corrective action deteriorates.
• Created within the context of business drivers
  I’ve worked with companies that must be Sarbanes-Oxley compliant or ISO certified. These are the realities. Our documentation, audits, and reports are as valuable as our primary Agile work product: a potentially shippable increment of software.

The Most Important Measurement

• Value delivery
  In Agile, working software is the most import measure of success. Even if an increment of software can’t be shipped, we should at very least be delivering the highest value features as early as possible.

In a future blog entry, I will talk about applying Agile estimating techniques to business value at the feature level.

Hammering Out UX

I see the same org chart in many companies.  User Experience Design (UX) is nestled tightly within Software Engineering rather than the Product Management team.  Often, they don’t become involved in the iteration until engineering  is involved.

Is this org structure agile?

Pre-planning with the Product Owner

I see the answer to the question in the pre-planning/backlog grooming which takes place prior to a release and iteration planning meetings.  The Product Owner has the inventory of product backlog items, often in the form of User Stories.  The Stories tell the who, the what, and the why without designing the how.  Yet to create the validation conditions, “how” comes into play.

How “how”?  There’s a leap which happens when moving from the Story to the verification conditions.  Let’s use my case study for EcoSense Travel, a fictitious site which is adding a paid subscription plan to it’s free eco-content plan.

As a subscriber,
I can receive my paid subscription content on my phone
So that I can keep up-to-date on high value information when I am away from my computer.

The product owner might then create the following verification criteria:

• Verify that paid content is integrated with the mobile RSS feed’s free content
• Verify that the paid content is branded with the provider’s logo
• …

You may begin to see the problem.  In coming to the Sprint Planning Meeting with the verification conditions, assumptions have been made about the implementation.  Should the information still be delivered through a mobile RSS feed?  Is a logo the best way to indicate a paid content source?

The step between stories and verification conditions

In the pre-planning phase, UX can provide a user-centered reality check on the jump from the story to the verification conditions.  Adding a “solution hypothesis” before developing verification conditions can minimize rework in the Sprint Planning Meeting, or at very least help the team question explicit assumptions.

Let’s take a look at the solution hypothesis for our story:

As a subscriber,
I can receive my paid subscription content on my phone
So I can keep up to date when I am away from my computer.

Solution Hypothesis:

The mobile interface for the paid content will be integrated with the unpaid content’s RSS feeds.

After reviewing the solution hypothesis, UX can then validate whether the hypothesis provides the best user experience, or whether new approaches or delivery vehicles can be leveraged.  If the strategy is to first convert existing users to paid subscribers, more research is warranted.  What if the vast majority are iPhone users?  Android users?  A more usable interface may be possible.

Bring Product and UX together from the start

While it may not be easy to change a historic org structure, UX can still be tightly aligned with the Product Team in the release and iteration pre-planning.  It provides the least risk of wasted work and drives out a more solid user experience.

Bridge the gap and add a bit of “how” to the who, what, and why.

A mind to know, two hands to act

In two previous blog entries, we saw how Lean Software Development, with roots back to Lean Manufacturing, can provide the values which derive the underlying business value. In some ways, it acts as the “head,” telling us the “why” behind that “whats” and “hows” of Agile.

Agile’s Two Hands: Scrum and XP

In addition to the “why,” we also need to know “what” and “how.” I view these as the hands of Agile: one to guide the management practices and one to guide the software engineering practices.

But with so many different Agile methods, how do we choose? Fortunately, the answer is quickly emerging. Scrum has wide market buy-in as an Agile management framework, though it lacks any prescribed engineering practices. Extreme Programming (XP) also has significant buy-in. While XP has a less developed management framework, its strength lies in its quality-driven engineering practices.

Head and Hands: Lean, Scrum, and XP

We previously explored Lean’s values. At a high level, you can see the framework emerge.

Lean Principles:

• Eliminate waste
• Minimize inventory
• Maximize flow / reduce work-in-process
• Pull from demand / decide as late as possible
• Empower workers
• Meet customer requirements
• Do it right the first time
• Abolish local optimization / favor adaptability
• Partner with suppliers
• Create a culture of continuous improvement

Scrum Management Practices:

• Simple, clear roles and cross-trained resources
• Self-organizing teams
• Short iterations
• Daily Stand-up meetings
• Just-in-Time requirements elaboration
• Continuous requirements inspection and prioritization
• Negotiable scope
• Up-front acceptance tests
• Transparency
• Simple management tools
• Continuous process improvement

Extreme Programming (XP) Software Engineering Practices:

• Test-driven development and automated testing
• Continuous integration
• Simple design and incremental architecture
• Refactoring
• System metaphor
• Coding standards
• Pair programming and a sustainable pace
• Collective code ownership
• Spike solutions

This model also allows us to frame Agile readiness and Agile maturity in this way. Is the organization ready to embrace Lean principles? How prepared are teams for Scrum’s management processes? Are the technical infrastructure and software engineering practices ready to become more “extreme” with Extreme Programming?

We’ve already looked at Lean values. In future entries, we’ll dive deeper into Scrum management practices and XP software engineering practices. From there, we’ve laid our foundation for evaluating readiness and measuring maturity.

Birthplace of Lean

In Part 1, we discussed how Agile values can provide a foundation for Agile management practices. However, through the time-tested rules of Lean Manufacturing, we can learn more about the Agile value proposition.

Mapping Lean Manufacturing to Agile

There are ten rules which can sum up the fundamental practices of Lean Manufacturing in the 1980′s. By looking at both Lean Manufacturing and Lean Software Development, the “why” emerges for the Agile management practices. These management practices can then be traced to the fundamental goal of producing more value with less work.

Getting the Value

Rule 1: Eliminate Waste

In Lean Manufacturing…

• Value Stream Analysis is used to identify all activities in the manufacturing process and the value they add to the final product, not the process. The process is then re-evaluated to find the most efficient way to add the intended product value.

In Agile…

• The biggest waste in software development comes from building features which will be outdated or have little value when finally released. A high priority feature today may be irrelevant in a year. In Agile, the business re-evaluates requirements against changing customer needs before each Sprint so that only the highest priority ones are built.
• In Agile, the User Stories are only frozen for the Sprint (though how they are implemented is negotiated throughout). The business is less hesitant to commit to requirements for the Sprint because they have the opportunity to re-prioritize before each Sprint. They waste less time on forecasting needs and the sign-off process.
• The production of documentation which has no value after the completion of an iteration translates to the scrutiny of document production. Which documents add value to the final product, and which simply support the process? Agile replaces verbose requirements documentation with light-weight User Stories and Validation Conditions. The design is then elaborated through conversation in the Sprint Planning Meeting and the daily Scrums.
• In the Sprint Planning meeting, only the highest priority requirements are broken down into tasks so that time is not wasted on features which may never be implemented. This process is often referred to as “just-in-time elaboration.” Time is not wasted on details which may never be needed.

Rule 2: Minimize Inventory

In Lean Manufacturing…

• Inventory is minimized because it costs money, consumes resources to manage, and becomes obsolete.

In Agile…

• The inventory of requirements is kept at a high level and broken down “just in time” so that unnecessary details don’t need to be managed. During the Release Planning process, high level Epics are created, but not broken down into stories until they are targeted for a release.
• Epics and stories are managed by the business and regularly pruned to eliminate the constant review and re-shuffling of the low or no-value items.

Rule 3: Maximize Flow / Reduce Work-In-Process

In Lean Manufacturing…

• Smaller batches and faster cycle times increase the flow of products from the manufacturer to the customer.

In Agile…

• Short Sprint cycles produce completed units of functionality which can be released to the customer. This allows a smoother flow of features, more frequent customer feedback, and faster responsiveness to change.

Rule 4: Pull From Demand / Decide as Late as Possible

In Lean Manufacturing…

• Pulling from actual demand reduces reliance on and investment in forecasting models. Driving from actual demand is more reliable in volatile environments.

In Agile…

• Freezing requirements at the beginning of a long development process relies heavily on forecasting. By re-prioritizing the Product Backlog before each short Sprint begins, features are driven by actual demand rather than speculation.
• Rather than trying to forecast architecture needs, Agile software engineering practices keep the initial architecture design light-weight and allows for refactoring (reworking of code). This keeps designs simpler, but allows additional architecture elements to be added if needed.

Rule 5: Empower Workers

In Lean Manufacturing…

• Teams are trained in work measurement and improvement techniques. Changes which are driven from the workers “on the floor” improve productivity and morale over changes mandated by management.

In Agile…

• Teams are self organizing, and re-evaluate their processes after each Sprint in the Retrospective Meeting.
• The team is empowered to decide how the stories are implemented. Through User Stories, the Product Owner defines the “who,” the “what,” and the “why.” The “how” is collaboratively developed by the team.

Rule 6: Meet Customer Requirements

In Lean Manufacturing…

• This rule is self-evident, and follows Rule 1 (Eliminate Waste). Manufacturing products which no one buys results in inventory which will never be sold.

In Agile…

• In traditional Waterfall, success is measured by delivering the required features on time and on budget. The delivery of the right requirements is often overlooked. As in Rule 1, the creation of low or no-value features is the most wasteful aspect of software development. Success is measured by delivering only the highest priority features each release in a “good” or “good enough” manner.
• Before the Sprint begins, the business creates validation conditions for each User Story. These validation conditions become the business’ acceptance test criteria. This means coding is driven by tests written by the business, not designs and tests created by the developers.
• During the Sprint Review meeting at the end of each Sprint, the functionality is reviewed with the business to confirm that the requirements have been met.

Rule 7: Do it Right the First Time

In Lean Manufacturing…

• Lean integrates quality at the source, putting controls in place at each step in the process. Before Lean, manufacturing was optimized for product throughput. At the end of the process, tests were conducted and defects were fixed at rework stations.

In Agile…

• Tests are written before code, and defects are fixed at each step in the development process. Barry Boehm observed that it costs 100 times more to fix a problem after release than early in the process.
• In an Agile organizational structure, QA is not a silo. Instead, QA personnel are integrated into each team.
• The change control process becomes simpler and less costly because fewer defects are passed on.

Rule 8: Abolish Local Optimization

In Lean Manufacturing…

• For rapidly changing markets requiring responsiveness over mass production, faster machine change-over is more important than the higher throughput of individually optimized stations.

In Agile…

• Cross training and team collaboration are more important than siloed roles. This allows team members to jump in where needed rather than passively waiting for someone else to complete a task or correct an issue blocking their progress.
• Teams are responsible for delivery rather than individuals. Rewarding individual performance (local optimization) rather than team performance detracts from each persons focus on the overall objective.

Rule 9: Partner With Suppliers

In Lean Manufacturing…

• Companies are able achieved the the greatest efficiencies in their supply chain by reducing the number of suppliers and working with them as partners. The efficiencies gained by trust, collaboration, improved product flow, just-in-time movement of goods, and less supplier turnover outweigh the savings which come from competitive bidding.

In Agile…

• Just as moving parts between suppliers is expensive, so is moving tasks between siloed departments. Agile distributes siloed departments into the teams.
• Integrating the business into each team with daily contact creates a partnership which provides just-in-time elaboration of business requirements and more cooperative relationships. Requirements are not “thrown over the wall” with visibility only at the end of a long development cycle.

Rule 10: Create a Culture of Continuous Improvement

In Lean Manufacturing…

• As Rule 5 (Empower Workers) states, everyone involved in the process has the power to improve it. In contrast, a command and control environments rewards adherence to a strict process. Assuming that processes can continuously be optimized, especially in changing markets, a culture is needed to support continuous improvement.

In Agile…

• At the end of each Sprint, the team conducts a Sprint Retrospective, discussing two simple questions: what went well and what can be improved. Software maturity models such as ISO9000 and the Software Engineering Institute’s (SEI) Capability Maturity Model (CMM) reward adherence to strict processes. It assumes that their specific process is right for every situation. Agile assumes that as markets change, requirements change, and the processes to meet those requirements change as well.

Agile: More Value with Less Work

Tracing Agile back to the time-tested rules of Lean Manufacturing, we begin to move from philosophical values to business value.

• Rather than telling a CEO that Agile values individuals and interactions over processes and tools, you can tell her that Agile eliminates waste and creates productive partnerships between all parts of the business.
• Rather than telling a CEO that Agile values working software over comprehensive documentation, you can tell her that Agile speeds up the delivery of valuable functionality to the customer.
• Rather than telling a CEO that Agile values responding to change over following a plan, you can tell her that Agile delivers functionality based on current demand rather than forecasted needs or outdated requirements.

And the list goes on: higher quality, continuous process improvement, and stronger collaboration. Your Agile value proposition is suddenly defined.

Agile Minds, Lean Thoughts

You’ve likely read the Agile Manifesto, the values which were developed by representatives from Extreme Programming, Scrum, DSDM, Feature Driven Development, Crystal, and other alternatives to the heavy, document-driven Waterfall methodology.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Manifesto statements offer traceability to many of the Agile management practices. If we use Scrum management practices for our examples, we can make the following connections:

• The daily 15-minute Scrum meeting with the team standing around the low tech task board supports the value of individuals and interactions over processes and tools
• The Sprint Planning meeting, where the light-weight User Stories are transformed into a shared understanding of the features to be implemented, supports the value of working software over comprehensive documentation
• The application of “good” versus “good enough” in order to deliver on time supports the value of customer collaboration over contract negotiation
• The re-prioritization of the Product Backlog before each Sprint Planning Meeting supports the value of responding to change over following a plan

Agile Values vs. Business Value

Though the Manifesto articulates the values that provide the foundation for the “what” and “how” of Agile, it doesn’t answer the question “why.”

Why are individuals and interactions more valuable than processes and tools? Many people could argue that individuals are inconsistent, their memories are fallible, and their interactions are unpredictable. Shouldn’t defined processes bring consistency and tools record decisions which may otherwise be lost or reinterpreted?

How do we move from Agile values to business value?

Finding the “why” in Lean

When the Agile Manifesto was developed in 2001 at the Snowbird Ski Resort in the Wasatch Mountains of Utah, Mary and Tom Poppendieck had not yet released their book on Lean Software Development.

Lean Software Development draws heavily from the rules of the Toyota Product System, later called Lean Manufacturing. Simply put, it is a time-tested method which focuses on providing more value with less work.

More value with less work. Now that’s something which any CEO could support.

In Part 2, we’ll map the value-driven rules of Lean Manufacturing to Agile management practices.

Scaling the Mountain

To scale to the enterprise, Agile and Lean must span the full value chain: from choosing your portfolio investments to defining and building your products to deploying and supporting them. Quickly. Continuously.

Welcome to the blog, Scaling Lean-Agile!  Existing content is being ported from my recent entries in other blogs, my training material, and my scaling frameworks.

Stay tuned!