Why Risk Management Matters is So Important in Creation

A Risk is a possible situation that could materially affect (i.e., derail) your plans. Risk Management is a set of techniques to reduce the likelihood and/or effect of risks on whatever you are doing.

While any operation or activity has implicit risk, the act of creating something entirely new—be it a company, platform, product or process—is fraught with some of the biggest risks imaginable:

• What if your underlying assumption regarding the need for what you are creating does not match reality?
• What if your select approach to fulfil this need does not fit the market?
• What if customers cannot readily recognize the value of what you are creating?
• What if your assumptions of customers’ priorities differ from what they really are?
• Are your customers do not use your creation in the way you expected?

Get all of these right and you will have created “The Next Big Thing.” Get any one wrong and you will likely have a big failure on your hands: lost or delayed revenue; failed launch; wasted time, work and money; etc. Managing these risks from the start—without undue overhead and complexity—is vital to success. The #LeanStartup easily lets you do this.

Tactic 1: MVP Testing to Avoid the Biggest Risks to Creation

As outlined above, act of creation faces bigger, more fundamental risks than any other endeavor. A core principal of risk management is to “manage the biggest risks first—when they are easiest and least costly to address.” The #LeanStartup does this using the Minimum Viable Product (MVP).

MVPs test high-risk assumptions (regarding market, platform, product position, product scope, etc.) from the start, using rapid, low cost development efforts. This avoids the biggest risk of all: wasting time and money creating something customers do not need, want or use as expected.

However, what is refreshing about the Ries’ approach is how he challenges current conventional wisdom regarding how to do this:

I was a devotee of the latest in software development methods (known collectively as agile development), which promised to help drive waste our of product development. However, despite that, I committed the biggest waste of all: building a product that our customers refused to use. (The Lean Startup, pp. 46-47).

Instead of focusing on eliminating “product development waste” (i.e., figuring out how to build faster), the #LeanStartup model focuses on eliminating “customer learning waste” (i.e., figuring out how to understand your customers faster) using the principal of Validated Learning. Validating Learning first uses MVPs (and later Innovation Accounting, see below) to let you focus your time and energy on creating the things most useful to customer (and vital to your success).

Tactic 2: Innovation Accounting to Continuously Avoid New Risks

Risks are not static. They do not all start at the beginning but can emerge at any time. If you do not continuously keep your eye out for new risks to manage, you are likely to face unpleasant surprises as you progress with creating your new product, platform or process. The #LeanStartup model addresses though use of a new form of metrics called Innovation Accounting.

Innovation Accounting is based on the use of actionable, accessible, and auditable metrics to detect and avoid risks by measuring what customers are doing with your product. It throws out “Vanity Metrics”—both measurable ones such as website hits and anecdotal ones such as “person X said our product was great”—in favor of metrics that focus on how customers proceed through their entire life cycle:

• Of those who looked at our product, which actually bought it?
• Of those who bought it, which used it (and how much or how often)?
• Of those who used it, what features did they use most and least?
• Of those who bought and used it, how many continued to do so? How many recommended it to their friends and colleagues?

Through use of Innovation Accounting (frequently in tandem with A/B Testing, Cohort Analysis and other pilot strategies), you can detect aspects of your product, platform or process that are resulting in undesired outcomes (e.g., abandoned sales, customer turn-over) earlier, addressing them before they become large problems. Furthermore, by tying metric results into your product development lifecycle (another unique change to Agile and Lean methodologies), you can avoid the risks of build whole sets of product features and capabilities that are not useful to your customers (and hence, a waste of time, effort and money to you).

Tactic 3: Pivoting to Mitigate Realized Risks

No matter how skillful you are, some risks will simply “just happen.” Even if your execution is flawless, external events (e.g., new market trend, new competitor, new invention) will change your competitive landscape. When this happen, you may need to change what you are doing if you want to continue to grow your new creation.

The #LeanStartup model introduces the concept of the Pivoting to adapt to change. Pivorting formalizes a fact-based process for change (using Validated Learning from MVP Testing and Accountability Metrics) adding calmness and control when reacting to new information and external events. Ries highlights ten categories of pivots to help you more effectively adapt to a broad range of scenarios, such as:

• Your product is compelling but its delivery model is not
• Your product is compelling but its sales model or channel is not providing the results you need
• Your product is compelling but your valuation for pricing does not match your customers’
• Your strategy is compelling but underlying market and technology changes require a different approach

Inclusion of Pivoting in your overall business model provides a tool to mitigate the effects of late occurring (or late discovered) risks that could not be avoided earlier in the creation lifecycle.

***

Creating anything new is an exciting—but risky—endeavor. Eric Ries’ #LeanStartup model provides three tactics that not only help you guide your venture to success, but also seamlessly harness the most powerful risk management techniques from the start of your idea through realization of your goals.

Steve McConnell, author of books like “Code Complete” and “Estimation: Demystifying the Black Art,” once commented—

There are only a limited number of things that can go right on a project.
However, there are an unlimited number of things that can go wrong

This maxim stands up to real-world experience. Once you start to actively apply risk management techniques to your programs and operations, you will soon find many, many risks to manage. As a result, you can quickly get so “bogged-down” managing risks that you divert too much attention from managing your program or operations (defeating the very purpose that let you to actively manage risk in the first place.)

The Trick is Figuring Which Risks to Manage

Not all risks are the same. Some are large; others are small. Some must be avoided; while others can be accepted. Since active management of any risk comes at as cost, the trick is to figure out where the benefit you are achieving by actively managing risk exceeds to cost of doing this. The easiest way to do this is to apply the Pareto Principle.

The Pareto Principle is broadly asserts that the top (or worst 20%) of any population of items cause 80% of the total population’s effect. While this “80/20 ratio” can sound trite, its use for estimation and prioritization purposes had been born out across a wide set of conditions (1 2 3 4). Applying the Pareto Principle to Risk Management yields the following maxim:

If you focus on continually managing the largest 20% of all risks you will achieve 80% of the total benefit you can possibly achieve using Active Risk Management (ARM)

I have found this as good way not only to prioritize risk response but also a good way to avoid getting caught up in the minutia of managing trivial risks.

Visualizing the Worst 20% Risk Quintile

In discussed in an earlier post in this series, an easy way to estimate the size of a risk is to break it down into two component dimensions:

• P, the probability that it may occur and
• I, impact it would cause if it did occur

However, not only does it make it easier to estimate the size of a risk using tools like sensitivity analysis, it also makes it clearer why one risk is bigger than another. This simplifies prioritization.

The following diagram plots five risks by priority and impact. To make things simple, it plots the size of the impact on a zero to 100 scale (this can be done through normalization of true risk impact sizes or through mapping of Low, Medium and High ratings to Six Sigma 1-3-5-9 Critical-to-Quality (CTQ) ratings:

Using This Visualization to Prioritize Response

With this type of plot it if very easy to break down risks by quintile, from biggest to smallest. Now you simply need to manage each risk from the upper-right-hand corner, inward, in order to use the Pareto Principle to prioritize your risk response. Simply work downward, keeping the cost of managing each prioritized risk below its estimated size. Remember to keep refreshing your list of identified risks on a regular basis (to keep up with the latest information).

A Quick Look at How This Changes Your Response

This simple approach can dramatically change how people prioritize risk response. To illustrate this, let’s take a look at the five plotted risks above:

• Many people focus so much on Impact that they use this single dimension to prioritize risks. Under this model, one would prioritize response in the following Order: Risk 3, Risk 4 then Risk 5. This would be delaying a chance to manage Risk 5—the biggest risk—until later (when it is harder to manage—See Note 1)
• Other people focus on “low hanging fruit” for “quick wins.” While this approach is good for team building, it would start with Risk 1 (and maybe Risk 2). These are low-priority risks that should only be addresses after many larger ones are. (If you continuously look for risks as your program or operations evolve, it may never be worthwhile to waste your time on these risks—See Note 2)
• It turns out the best thing for you to do is manage Risk 5. Then reevaluate which risk is in the worst Quintile based on the most-current information. Then repeat (until your operations or program is over). Under this model, you are always applying resources to address the most pressing risks only. This lets you apply risk management without letting take over your entire focus of operations.

How This Approach Builds Accuracy over Time

While this approach will be 100% accurate on an individual risk-by-risk basis, it will leverage the Law of Large Numbers and the Pareto Principle to balance out bias errors over the course of your entire program or operation. The end result of this will be judicious, cost-effective application of risk management from start to finish.

What’s Next

This post wraps up the last “lesson” in these series on risk management. The next post will share some lessons learned on folding these approaches into your operations in a way that combines discipline with low complexity and overhead.

Author’s Acknowledgement: I would like to credit the following past associates with whom I put this approach to prioritizing risk response together: James Gaines, Simon Grant and Igor Mandrosov.

Notes:

1. Steven Levitt and Stephen Dubner have written extensively about this Impact bias in Chapter Four of Freakanomics.
2. Remember this analysis the next time your by-the-hour consultant explains their focus on “low hanging fruit.”

The four techniques to manage risk

A risk is a possible situation that could materially affect your operation (if it occurs). The size of a risk (and the required level of your response is driven by two factors: the likelihood of its occurrence and the size of the impact if it occurs. The four generally accepted techniques employ different strategies to address these two factors:

Technique Number 1: Avoidance

Avoidance techniques aim at reducing the likelihood that a risk will occur (preferably reducing the likelihood to zero). As such, it can be the most powerful technique for managing risk. Here are some examples of risk avoidance:

1. Operations: Risk that we will be late or over budget because we do not have experience doing X. Avoidance technique: bring in a consulting expert with experience in X.
2. Technology: Risk that implementing a feature will cause us to be late. Avoidance technique: delay implementation of the feature to a later phase.
3. Everyday Life: Risk that I will get hit by a car if I run out in the street. Avoidance technique: look both ways first.

There are few risks where you would not want to apply risk avoidance (the only times you do not want to do this is where the cost is prohibitive).

Technique Number 2: Mitigation

Mitigation techniques aim at reducing the impact that a risk will create if it occurs. As such, it should be used in situations where a risk is both sizable and cannot be avoided (or avoided at low enough cost). Here are some examples of risk mitigation:

1. Operations: Risk that we will not have enough capacity to take customer orders. Mitigation technique: setup of a variable contract support structure that I will only
2. Technology: Risk that my application will not receive the correct data from a client (calling application). Mitigation technique: creation of exception processing to deal with these situations without crashing
3. Everyday Life: Risk that I will get badly hurt if I am in a car accident. Mitigation technique: wear a seat belt and install an airbag.

While people default to using risk mitigation it is not always preferred: would you rather wear a Michelin Man suit when you cross the street (mitigating the impact of getting hit by a car) or simply look both ways first?

Technique Number 3: Transfer

Transfer techniques move the impact of a risk (if it occurs) to an external party. This can get complicated. As such, risk transfer is best reserved for situations where the impact of a risk can be clearly measured and fully addressed by an external party. Here are three examples of using Risk Transfer:

1. Operations: Risk that an employee will burn your store down. Transfer technique:  buy fire insurance
2. Technology: Risk that your customers will not understand how to use your product and will need support. Risk Transfer technique: set up a lower-cost-per-hour Help Desk to provide support.
3. Everyday Life: Risk that I may incur large health care costs. Risk transfer technique: buy health insurance.

There are many risks that you would not want to address through risk transfer. Would you want to buy insurance that pays you out in the event that a customer safety event destroys you brand’s reputation? Unfortunately, most transferred risks are transferred unintentionally (i.e., without managing the risk). A clear example is insufficiently testing a process or application, leaving your customers to deal with the consequences (I am sure you can name a few examples of this).

Technique Number 4: Active acceptance

Risk acceptance is the process of actively deciding that you will accept the consequences (impact) of a risk if it occurs. When applied correctly, i.e., when you actively document a risk and the decision to accept its potential consequences, it truly is risk management. This technique is best when the impact of a risk is small (much smaller than the cost to avoid, mitigate or transfer it). Here are three examples:

1. Operations: Risk that we will not finish all of our tasks on time. Active risk acceptance technique: padding your schedule to account for this lateness.
2. Technology: Risk that using a new technology will cause more overtime debugging and testing. Risk acceptance technique: using the technology and pressing the team to do the overtime work.
3. Everyday Life: Risk that I may get a ticket if I speed on a road but deciding the benefit of going faster is greater than the combination of the ticket price and chance of getting caught

While use of Active Risk Acceptance is often appropriate, formal use of this technique is rather rare. Unfortunately Passive Risk Acceptance, i.e., writing down a potential risk but not estimating its size and actively taking steps to manage it, is all-too common., i.e., “hoping” a risk will not occur.

Which technique is best?

At first blush, it looks like the best risk management technique is Avoidance and the worst is Acceptance. This is not true. Some risks you must avoid (e.g., destroying your brand), others you can easily accept (e.g., getting more purchase orders than you can process). The key is picking your technique based on the size and structure of the risk. This is the very definition of risk management.

What if I cannot determine which technique I am using?

Sometimes I will hear two people debate if something like setting up a Help Desk is transfer (moving the risk from the implementation team to the agents) or mitigation (accepting that the risk is likely to occur but reducing the cost of its impact.) If you hear an argument like this, do not worry. It is a great sign that your team is actively managing risk (something much more important than figuring which “bucket” to use to classify their response).

Can I use more than one technique?

Using more than one technique to manage a risk is more common than many people realize. (It is also very wise.) A typical example can be found in managing technology infrastructure:

You have a risk that a server will fail (a risk that is certain to eventually happened). You employ three techniques to manage this risk. First, you split your traffic across lots of small servers. This reduces the likelihood of failure: a two-CPU server has a much longer mean-time-between-failure than a 192-CPU server. It also reduces the impact: when one server fails it only affects a small percentage of clients. Second, you invest in a Network Operations Center and/or Help Desk to deal with the consequences of server failure: this is a combination of mitigation (of failure cost), transfer (requiring your clients to call for help if they need it) and acceptance (recognizing that failure will occur).

While defining the techniques of risk management is simply, figuring out how to apply them is often complex.

Author’s Acknowledgment: I would like to credit the following past associates with whom I have worked with for years to develop and apply the lessons regarding response to risk: James Gaines, Simon Grant and Igor Mandrosov.

Avoid the trap of analysis paralysis

Many organizations that make the conscious decision to actively manage risk often hit the brick wall of analysis paralysis. They get so paralyzed spending time and energy trying to figure out exactly how big risks are that they don’t get around to responding to most important (i.e., the largest) risks early (when it is easier to address them).

Estimation IS good enough (for comparative analysis)

I found the easiest way to avoid the analysis paralysis trap is simply to decide NOT to calculate exactly how large each risk is. Instead simply estimate the size of each.

I know what you are saying, “Estimating a risk could lease to make very bad investments.” But remember what we are doing. We are not calculating the cost of a Collateralized Debt Swap Obligation (something the Quants did not apparently do well in 2008, even with their advances mathematical models). We are simply figuring out the size of each of our risks relative to each other (so we know which ones to address first).

Since we are using the same approach to estimate every identified risk, we will come up with an almost identical assessment of how big each of risks are relative to each other. However, with estimation, we will do this with much less time and effort.

People are better at estimation than they think

When I ask people to estimate risk, they usually tell me that they would not even begin to know how to calculate the size of a risk. However, people are much better at performing accurute estimates in their head than they think. If you don’t believe me just watch a baseball player estimate the trajectory of a parabolic arc through a near-constant gravity field and a non-linear drag coefficient, i.e., figure out where to stand to catch a pop fly.

The trick to estimating risk: RIP it up first

The authors of Freakanomics (and Super Freakonomics) describe a lot of scenarios in which people make decisions that are not good for them because they do not calculate risk correctly. At the core, these scenarios usually boil down to one point: people forget the size of a risk is comprised of two dimensions:

A RISK is a possible unplanned situation that would
materially affect your plan or operations—if it occurs

To estimate how big a risk is, you need to estimate two things: the possibility and the effect. For the mathematically inclined:

Risk_Size = Impact_Size x Probabilty_{Of Occurance}

To estimate the size of a risk, RIP it in two parts: Impact and Probability. Then estimate each part and multiply the results together.

Estimating (the) Impact (that a risk would cause)

Estimating the Impact of a risk can get as complicated as you want to make. However, it can be easy as well. I use the following approach.

First I ask them what type of impact they are trying to estimate: the cost of lost time, the cost of extra effort, lost revenue, lost market share or lost market capitalization. Each of these has a simple proxy (for estimation purposes), DAYS LOST:

• Cost of Lost Time = Days Lost x Daily Cost of Operations (or Budget)
• Cost of Extra Effort = Days Lost (Extra Effort) x Number of People x Daily Pay Rate
• Lost Revenue* = Days Lost x (Annual Revenue / Sales Days per Year)
• Lost Market Cap* = (Lost Revenue / Annual Revenue) x Current Market Capitalization
• Lost Market Share* = (Sales Days Lost / Sales Days per Year) x Current Market Share

*Sometimes you are only interrupting revenue, not losing it. In these cases I simply calculate the cost of deferred revenue by multiplying these impact by you Cost of Capital (Return on Equity can serve as a proxy)

So I ask this question:

“Roughly how many days will we lose if [X] happens: one day, one week, two weeks, one month, one quarter, six months or a whole year?”

Take the answer and apply it to the above estimation model. This is good enough to get a rough estimate for planning and prioritization purposes.

Note: If you are estimating life and safety, you are in an entirely different spectrum. Most organizations that do this have a model already in place (usually tied to combination of actuarial tables and local liability trends). Contact your Legal or Safety department to find out how they do this.

Estimating (the) Probability (of a risk occurring)

Estimating probability can also get complicated. However, it too can be simple.

I ask people to tell me how LIKELY it is they think [X] will occur:

• Remote Chance (Less than 1 in a 1000 chance)
• Very Low Chance (1 in a 100 chance)
• Low Chance (1 in 3 chance)
• Toss-Up (50/50 or 1 in 2 chance)
• Likely (3 in 4 chance)
• Near Certain (> 9 in 10 chance)

People are usually comfortable providing this kind of estimate.

Putting both estimates together

Once you have your two estimates, simply multiple them together. You may be surprised by the results. For example, that remotely-possible-but-utterly-horrible-thing-that-might-occur could turn out to be a relatively much smaller risk than the 50/50 chance that your vendor will ship two weeks late.

A few best practices to make this easier and more accurate

Since you are performing an estimate, you can use all the tools available for estimation to make this easier. I recommend, Steve McConnell’s Demystifying the Black Art for a fun, informative read on variety of effective estimation techniques. I like to use the following:

• De-composition/Re-composition. Rather than estimate really big risk, I break it down into smaller parts and estimate each (then manage the component risks separately)
• Wide-band Delphi. I never like to estimate my own risks (I might be biased). Instead, I ask my stakeholders to give me their estimates, then bring them together to converge on a single one
• Count (or Measure) Whenever You Can. I revise my risk estimates weekly based on new data. I also measure the accuracy of past estimates to account for future bias

Finally, to make things simple, I like to use a spread sheet-based Risk Register that lets me toggle my DAYS LOST and LIKELIHOOD inputs so I can easily perform quick sensitivity analysis of size estimates. This also lets me sort risks according to Total Size, Probability or Impact.

Author’s Acknowledgment: I would like to credit the following past associates with whom I have worked with for years to develop and apply the lessons regarding estimation of risk: Neal Beliveau, James Gaines, Simon Grant, Jeff Kolar, Clare Little and Igor Mandrosov.

Step 1: Clearly define what a risk is

This may seem like a trivial (or even pedantic) first step. However, if you want to ensure your staff are managing risk in a consistent manner, you need to start with a clear definition of risk that everyone agrees with across your enterprise.

A simple definition of risk

If your organization’s reason for existence is something different than managing risk, it can be helpful to keep your definition of risk as simple as possible. This not only makes people more comfortable adding risk management to their everyday jobs, it also reduces the cost and time required to teach risk management.

I like to use the following “plain and simple” definition for risk:

A RISK is a possible unplanned situation that would
materially affect your plan or operations—if it occurs

I have found this easy to adopt by people from a variety of markets, industries, operations and cultures.

Simple litmus tests to separate risks from issues, rumors and distractions

By using the highlighted key phrases, it is easy to test whether an item of concern is actually a true risk:

• A risk is something that could possibly occur. If something has already occurred (or is 100% certain of occurring), it is not a risk: it is an issue. If something is untrue (or has no chance of occurring), it is a rumor or fallacy.
• A risk is something with the potential to materially affect your plan or operations. If a the occurrence of a situation would not cause a material effect, it is not a risk to you: it is simply an external distraction. (Of course, if could create a risk by distracting your staff from their work.)

Once everyone in your organization begins using the same definition for risk, they can begin exploring and sharing risk information using the same language and terminology when encountering all types of risk.

Step 2: Recognizing the four major types of risk

Many people forget that there are four major types of risk:

Each of these types are defined by the following two quadrant “dimensions.”

Positive vs. negative risk

People tend to focus on negative risks, i.e., risks that would detrimentally affect plans and operations to a material degree. If is prudent to plan responses to negative risk (I have seen this “save the day” many times.) However, simply planning for negative risk is not sufficient. You also need to consider positive risk.

Positive risks are unplanned situations that would create a boon to your plan or operations. If you plan for positive risk, you can take advantage of these situations when they occur. This can be just as beneficial as planning for negative risk. Here is a simple real-world example:

Company A is launching a new release of their product. Based on past analysis, they understand that this product release may lead to short-term increase in customer service calls. To accommodate this, they setup a dedicated Contact Center queue to address this new need.

However, Company A has paid careful attention to design their new release to address many past customer requests. As a result, the release could either increase calls (a negative risk—due to confusion with new features), or decrease them (a positive risk—due to resolution of issues that caused past calls). To manage both aspects of risk, Company A does two things: 1) they use outsourcing to enable them to increase or decrease customer support without incurring a fixed cost and 2) use blended queues to enable them to absorb call volume into their existing Contact Center capacity.

Company A rolls out the new release. Initially, call volume spikes, but it drops below pre-launch levels within 30 days. Because Company A planned for both positive and negative risk, they reactive to both conditions at low cost—without creating long call queues.

Internal vs. external risk

People also tend to focus on internal risk, i.e., situations that could go wrong with their particular project or operations. While it is prudent to think about the risk of your operations, it is not to neglect external situations that could materially affect you.

One of the most successful Program Managers I know pays strict attention to External Risks. Here is his real-world example:

Company B is upgrading their IT infrastructure. This upgrade will bring better features and reliability to its staff. However, Company B’s business is NOT IT. As such, it the upgrade is not the first priority of the majority of Company B’s staff.

The Program Manager for the infrastructure upgrade recognizes that external situations (e.g., end of quarter sales or major product releases) could block or delay infrastructure upgrades at certain departments and locations during key times. To prepare for this, the Program Manager meets with each group to learn about these potential times. He negotiates both first choice and second choice upgrade windows for each department and location. This enables him to steer his upgrades around planned conflicts and—just as importantly—around unplanned ones. As a result, he completes his program on-time (and with high stakeholder satisfaction).

Apply your definition of risk vigorously across all four quadrants

Too many organizations only focus on Internal-Negative Risk (and often in a consistent manner across different projects and operations). As the examples above show, it is critical to start with a clear definition of risk and apply it across your entire enterprise with a focus on all four types of risk. If you do this, you will be ready for nearly any situation that can likely occur.

Author’s Acknowledgment: I would like to credit the following past associates with whom I have worked with for years to develop and apply the lessons regarding definition of risk: Neal Beliveau, Anne Dixon, James Gaines, Simon Grant, Jeff Kolar, Clare Little and Igor Mandrosov.

The first step is a change in culture

The very first step down the path becoming an organization that used risk-based approaches to program and initiative execution is purely a change management one:

Your entire chain of command—from executive to line staff members—needs to embrace the concept of freely discussing risk on a daily basis

This may seem like a very obvious and simple change. It is not. This is fundamental change in how most organizations operate.

You need to encourage free and open discussion of risks and concerns

How many Status Review meetings have we sat through in which nearly everything has a “Green” status? (Even though “everyone” in the room knows which initiatives are having problems—and often what those “unspoken” problems are.) Too many.

At best this is a waste of meeting time. At worst, it lets risks fester and develop into large-scale problems. This happens because—in most enterprises—too many people are afraid to point out risks, concerns and problems (specifically to the very people who are empowered to help resolve them quickly). As long as this cultural phenomenon is in place, it will be impossible to actively manage risk.

To resolve this you need a new paradigm. I recommend employing both the “carrot and stick” (or what Change Management guru, Kurt Lewin, called “Force Field Analysis”). I use three steps to do this:

1. Hold regular portfolio reviews in which you review the risks and issues of each initiative
2. Reward those who raise risks (along with actionable recommendations to address them) with by combining of praise with direct support that helps them resolve their risks
3. Mark the initiatives of those who do not do this as “Red” and require them to review their projects with you in detail (so you can help them see the risks they are missing).

I apply this at all levels: executive through line staff. It is a constructive approach that uses mentoring and support to drive change. Over time (the time will vary based on the speed of your initiatives), it will create the culture you need to actively manage risk.

You need to be comfortable with what this will expose

Making these changes will fundamentally alter how your organization discusses initiatives and their status. You will initially see almost all of your “All-Green” reports turn “Red.” From the outside, things may look worse in comparison to other organizations. You may even get less-than-desired attention because of this appearance.

This discomfort will be rewarded

Even though things may look worse, the opposite will be true. Your entire organization will be concentrating on preventing problems that could stand in the way of success before they will occur. You will create a much more trusting and open environment.

As a result, you will become more efficient, effective and creative at managing risk. The result of this will be increased level of success in your initiatives and far fewer cost and schedule overruns.

These rewards will become visible externally as well. I have implemented this approach many times, across several industries. Every time it resulted in greater trust and respect for the organization that employed it.

Currently, Mr. Roger Baker (CIO for the US Department of Veterans Affairs) is doing something very similar. It is not an easy process. However, it is earning the department a lot of respect for openness and transparency – and enabling them to focus on providing more effective services for their staff and veteran stakeholders.

Because of this, our hopes are larger than average (while our margins for error are smaller.) As a result, we are presented with large risks—both positive and negative. The better we actively manage these risks, the more successful we will be in 2010 and beyond.

What does it take to actively manage risk (ARM)?

1. The commitment to do so. This may sound obvious—it is not. Those who actively manage risk, not only shift their attention from concentrating on “what is going to plan” to “what unplanned things could happen,” they also reward this type of thinking.
2. Continuous focus on identifying, assessing and prioritizing risk in all of its forms: internal and external, actual and perceived, positive and negative. Risk is dynamic. You cannot plan for it upfront; you need to think about it all the time.
3. Systematic approaches to communicating and responding to risk. This is not possible if you have not made the commitment to actively support and reward concentration on “the unplanned.”
4. Built-in ability to adapt and manage the emergence of issues due to undetected or unforeseen risks. (A keyword here is “agile.”) This requires a change in how you plan – and your comfort level with “planned uncertainty.”

Actively managing risk is not a trivial exercise. However, it is also not a monumental one either. By adopting a series of best practices, enterprises can quickly start the new year with an aggressive focus on managing risk. The benefits of this are enormous:

• Increased likelihood of success
• Reduced cost and time overruns
• Faster response to changes in business and environmental conditions

Over the next few weeks, I will write in greater detail on this blog about how I have led organizations across five global industries to perform each of the above four activities.

Even the greatest Program Dashboard will not help if you do not use it to manage your portfolio more effectively. Below are 10 critical success factors I have learned in the process of using dashboards to drive successful outcomes on over 3,300 projects and programs over the past 15 years:

1. Measure What is Important to Your Stakeholders

Don’t just measure what one group thinks is important (as many technology groups do). Ask your critical stakeholders outside your group what they would like to see to understand your progress. (See Tip #1 for more information on this). This will ensure what you measure will be important to all.

2. Be Forward-looking

Dashboards are intended to look where you are going (they are not rearview mirrors). As such, use metrics that look to the future. This lets you use your Dashboard to detect and prevent problems (See Tip #2 for more on this.)

3. Ensure All Use the Same Objective Metrics

If you want to use your Dashboard for than a tiny group, you need to ensure everyone identifies their “apples and oranges” correctly. This can only be done with clearly defined objective metrics. Define these clearly and use them universally.

4. Measure at the Right Level

We do not heat buildings by putting one thermostat in the lobby. We put thermostats in each major area and measure and adjust accordingly. Do the same for your large program: break them down and measure where the work is being done. See Tip #3 for recommendations as to how to do this.

5. Be Accurate and Truthful

For a Dashboard to be effective, it must have the correct data. Avoid using it as a “Sunshine Reporting Tool.” Instead be brutally honest: entering positive and negative data. Share this with your stakeholders (it will give you credibility.) If you do this, your Dashboard will be viewed as a “source of truth” by all.

6. Treat “No Data” as “Bad Data”

Many people simply do not complete all the data a Dashboard requires because they do not have the inclination to monitor as much as you require (or they are trying to “fix” something before reporting status.)  Don’t allow this. Absence of data indicates a problem. Whenever I audit projects due to lack of reporting I almost always find that major items are not being managed.

7. Trust but Verify – Especially When “All News” is “Good News”

Having Green indicators is great. Having them week after week after week is suspicious. While there an limited number of things that must “go right” for a project to be a success, there are an unlimited number of things that can “go wrong.” When everything is Green week after week you almost always have a hidden problem.

8. Match the Timing of Your Dashboard Updates to Your Sensitivity to Problems

Updating your Dashboard a few times a year will ensure it is always stale and out-of-date. (It will also ensure that it will never be able to detect and head off problems before they occur). Updating it daily will create needless overhead. Use the following “rule of thumb” to determine how often you update your Dashboard: How long could I go without knowing about a potential major problem before it becomes a threat to my mission? (Usually this time is between one week and one month.)

9. Do Not Punish Bad News

Do not punish people for reporting problems. Instead, encourage them to highlight their need for support. Asking for help is not a bad thing (it should be rewarded). Covering up problems (or repeating the same mistake over and over again) is entirely a different manner.

10. Use Common Sense

Finally, it goes without saying that you always need to apply common sense to how you are tracking and measuring projects. Every organization is different. Some metrics that are critical one place are useless overhead in others. Pick and measure what is important for your success.

Good luck!

In my last post, I outlined six Dashboard metrics that encourage continuous focus on—

• The critical path required for success
• Looking ahead to see what could create problems in the future

I also outlined measurable, objective criteria that defined when these metrics should be considered Red vs. Amber vs. Green.

One comment I raised was that these metrics appeared to break down for large (i.e., 1,000-person) programs and portfolios. As I promised, this post describes how I address this challenge.

Breakdown Your Program or Portfolio into Manageable Projects

It is next-to-impossible to measure how a 1,000-person or USD $50+ million program or portfolio is doing with a top-down metric. (It is like guessing how many gumballs of each color are in a gumball machine.) It is equally hard to manage the critical path of this from an aggregate level.

The way to address this is to decompose any program or portfolio into a set of nested, manageable projects.

• Start at the top and break down the program into the component modules, work stream or missions that you need to achieve for success.
• Look at each to see how many resources are required for success. If the amount is large, then repeat this recursively until you have a nested portfolio of small projects.

Your program has now moved from one huge, amorphous entity to a set of small projects that are each easy to manage:

Manage and Track Each Discrete Project Using Your Dashboard

Assign a Project Manager to manage and track each distinct project using the metrics define in the last post. Now you can review your entire portfolio from a high-level Dashboard and see exactly where (as Program Manager) you need to jump in and address problems to keep everything on track.

This is not complicated; it is repetition of a simple function many times. The first time you do this, it will take a fair amount of work. However, keeping it up-to-date is fast and simple.

Aggregate Your Projects Metrics for a True View of Portfolio Health

Now you can roll up your metrics for each project into each level of your program. This works as follows:

• Define a numeric score for each metric level (e.g., 0 for Red, 50 for Amber, 100 for Green). Score each metric for each project in the portfolio.
• Combined these scores into a weighted average to measure the true health and status for each level of the program. (I usually weight by project budget).
• Repeat this until you get to the top of the program. Now you have a true, objective view of how the program is really doing

This approach lets you share different levels of your dashboard to different stakeholders.  It also enables these stakeholders to drill down into the program to see where the problems are. Here is a sanitized, real-world example from a 20-country, multimillion-dollar business transformation program I managed a few years ago:

Here is the top-level status of the program I presented to the C-level staff:

Of course, this triggered a dive into the Mission B and Change Management work streams:

The organization leader who was most affected by Mission B (who was present) committed to add staff to serve as leads for his initiatives (this took about three months; each update reflected the progressive improvement of this).

Change Management had larger problems as it required matrix staff from many groups. This took longer to resolve. However, the dashboard drove leaders to act in time to keep this from impacting the remainder of the program.

What is most interesting here is a simple top-level Cost and Milestone Dashboard would have shown this program as Green (especially in terms of budget) until hidden internal dependencies caused major milestone slips in the future. Breaking down the program into initiatives and tracking each with forward-looking indicators prevented this from occurring.

This Approach Is Not New

This approach is not new. It is advocated by IBM in their Zachman Framework and the Object Management Group in its Business Motivational Model. I learned it from Joseph Shea, the original Program Manager of NASA’s Apollo Program (he repeated his break-downs by hand every Sunday night without the benefit of software). It is also how agile companies like Google and Amazon build products (like Gmail and A9).

It Is Not Expensive Either

It doesn’t take millions of dollars to implement this. On four separate occasions I combined use of COTS spreadsheet software with a shared file server and a simple web page service to set this up to manage over USD $800 million of projects and programs.

Remember the Purpose of A Dashboard

The purpose of a dashboard is to provide current information that a controller can quickly use to ensure he or she is on course (and take rapid corrective action if not). It is essential to use this as a guiding principle when determining what you should measure and put on your dashboard.

The Automobile Provides a Simple, Powerful Analogy

When you search for the term “Dashboard” in Wikipedia it brings up an Automobile Dashboard. While there are many different kinds of dashboards, automobile dashboards are likely the most common ones used all of us. As such, it is great place to start a discussion of what you would like on your Program Dashboard.

Do You Want to Know When Something is Wrong?

When you are driving do you want to know when—

• You are going to fast?
• Your brakes are having problems?
• Your oil is low?

Or would you prefer everything to show us a “Green and okay” status regardless of what is really going on under the hood? Would you want it to show real measurements or would you prefer it to express an opinion?

The answers to these rhetorical questions are obvious. As such, why would you not want your Program Dashboard to provide the same service? Unfortunately, Most Dashboards show that everything is Green and okay.

Do You Want to Look Forward or Backward?

Automobile Dashboards are placed underneath the windshield for one purpose: so you can see how your car is doing while you are looking ahead to see where you are going. Your Program Dashboard should do the same: it should show look forward and tell you what problems may occur in the future—not simply report back on past cost and schedule data. Unfortunately most Dashboards simply look at the past (really serving as rear-view mirrors.)

True Forward-looking Metrics That Drive Success

Over the course of fifteen years of program management, I have repeated used six metrics over and over again on my Program Dashboards. These metrics all use objective measures (not subjective guesses) to determine Red-Yellow-Green status. They are easy to understand by all types of stakeholders from mission owners, to engineers to finance professionals. Finally—and perhaps most importantly—they are organized along a forward-looking critical path to provide “Early Alerts” of potential upcoming risks and issues (to they can be addressed before they occur).

Metric #1: Resourcing

The first thing I look at is resourcing (i.e., have you secured the resources, money and critical people and equipment you need to start and execute your project program?) If you do not have these, you will not be able to move your program or project forward. While time –and timelines already published to external stakeholders–will continue to elapse, your progress will not.

Here is how I measure Red-Yellow-Green status:

• Red. You have not secured (i.e., released) the funding for your project or program. You also have not secured critical equipment and functional leads or experts for each area of your program. Without all of this, you are “stuck in the mud.”
• Amber. You have your critical funding, equipment and leads but are not yet fully staffed (e.g., you are hiring or waiting for resources to join your project). As such, while you are able to move forward, you are not able do so at full speed.
• Green. You have all the resources you need. (See Tip 3 to understand how I can make this statement for 1,000-person programs.)

Under this model, every project starts in Red status for resources, them moves to Amber, then Green.

It amazes me how many people skip this metric. If you do not believe me look at all those budgets that allocate money for programs that are started late (or not at all). We have all seen these.

Metric #2: Scope Definition

So once you have your resources you can move forward at full speed, right? Only if you know where you are going. If you have not defined (and gained agreement by all critical stakeholders) on the scope of your project or problem (i.e., what measurable goals you will achieve, problem you will solve of capabilities you will enable) you will aimply burn money and time. (We have all seen this as well.)

As such, once you resources move from Red to Amber on Resources, you need to finalize Goals or Scope. Here is how I measure Red-Yellow-Green status.

• Red. You have yet to complete a measurable Scope document.
• Amber. You have completed a Scoping document but it is under review (either for initial review or due to a Change Request)
• Green. Your Scope is documented and approved by all critical stakeholders.

I also like to keep track of how many Change Requests I have had on the program.

Under this model, every project starts under Red status for Scope and moves to Green only when all critical parties agree on what is needed. Also, every time you go back and want to change scope, you move back to Amber. Why? Because action is required, such as estimation of cost and time impact and determination of whether this change is required for success.

Metric #3: Risk Assessment

A Process Analyst who worked for me at Amgen (Igor Mandrosov) once told me a great expression: “Unmanaged risks grow up to become issues; issues require you change your schedule and critical path.” Risk is not something you should explore at the beginning of a project; it is something you need to manage continuously from beginning to end (I have a whole course on this that I will convert to a blog series one day).

Along these lines, I ask all my project and program managers to estimate risk—both internal to and external from the program. Depending on the type of organization I am, I measure this in currency-equivalence or simple Critical-to-Quailty Six Sigma units. (Contact me if you want to know more). Here is a typical way of using this to measure Red-Yellow-Green status:

• Red. The size of outstanding Internal Risk is 90% or more of the program’s budget OR the program manager is not actively managing risk (see Tip #4 as to how to detect this)
• Amber. The size of outstanding Internal Risk is between 30% and 90% of the program’s budget
• Green. The size of outstanding Internal Risk is less than 30% of the program’s budget

I also like to report the Top 3 or Top 5 risks indicated in the programs Risk Register.

This model forces projects to continuously look at and report on risk to avoid moving into Red status. It also enables executives to focus on programs as soon as large risks are detected (not after they cause cost and schedule overruns).

Metric #4: Execution (or Performance) Assessment

When unforeseen issues emerge (usually do to errors detecting and managing risk) project managers need to begin adapting their plans in response. When issues become large enough (i.e., become Critical Issues) they require changes to the critical path (and milestones) of a project or program.

Along these lines I ask project and program manages to track all issues in an Issues Log. I use the following way to translate this into Red-Yellow-Green Status to detect programs that need intervention before requiring major schedule changes:

• Red. You have re-baselined the schedule three or more times or have five or more outstanding critical issues. (When these numbers reach nine it is time to consider killing the project program. I also automatically mark programs that cannot show me an Issue Log with Red Execution Status.)
• Amber. You have re-baselined the schedule once or have three to five outstanding critical issues.
• Green. You are adhering to your original schedule and less than three outstanding critical issues.

Notice the heavy adoption of Six Sigma Critical-to-Quality measures here. I also require project and program managers to track a count of open Issues (critical and non-critical) in a log or register (and often report the Top 3 or Top 5 on the Program Dashboard.)

This model forces continuous exposure and resolution of issues. This enables you to detect problems earlier (and to drive better risk management), before you miss a major cost or time milestone

Metric #5: Schedule Status

Now we are finally at a (traditional) rearward-looking dashboard metric. This metric is measured using the classic PMBOK definition of Red-Yellow-Green Schedule Status:

• Red. You have missed (or it is no longer possible) to meet your final scheduled milestone
• Amber. You have missed you last milestone (or are late on your current one) but are still able to meet your final scheduled milestone
• Green. You are on time for both your last and current milestones and can still meet your final scheduled milestone

Notice that this allows projects and programs to slip to Amber but return to Green after demonstrated success. (Most Dashboards move quickly from Amber to Red.)

However, if you are using your Dashboard correctly, you should have detected and resolved problems before you reached the point of missing actual milestones. This is critical having a project or program that is considered a success.

Metric #6: Budget Status

This, too, is a (traditional) rearward-looking dashboard metric. It is measured using the classic definition of Red-Yellow-Green Schedule Status (as defined by your Finance department):

• Red. The program is (or is projected to be) over budget by Y% (I typically see 15%
• Amber. The program is (or is projected to be) over budget by between X% and Y% (I typically see 5% and 15%)
• Green. The program is (or is projected to be) less than X% over budget (again typically 5%)

Notice how this measure directly includes input from your Finance stakeholder. Notice also that it is separate cost from time.

The Big Differences in These Metrics

The model uses a series of metrics that require you to prove you are not Red (vs. the other way around).  They also are designed to “Go Amber or Red” early to enable you to take proactive steps to prevent schedule and cost overruns. This enables to provide a true Dashboard (not a Rearview Mirror).

Remember the Purpose of a Dashboard

The purpose of a dashboard is to provide current and accurate information that a controller can use to ensure he or she is on course (and take quick corrective action if not).  As such, your first question is, “Who are the controllers of my program portfolio?” This will be the subject of this post. (The second question, “What information is critical for accurate decision-making and quick corrective action?” is the subject of the next post in this series.)

Your Controllers Are Your Critical Stakeholders

Every organization that I have been part of (or consulted to) contains a set of critical stakeholders whose support you need to ensure success. Boiling this down to single phrase, I refer to this as, “The list of people who must agree to say ‘Yes’ – or agree not to say ‘No’ – for your program to succeed.”

Before you jump in and design your dashboard, ask yourself the above question. This answer to this question will almost contain at least three of the following types of people:

• The person responsible for implementation of the technology
• The business sponsor or manager of the end users of the results of the project (if you are a contractor or vendor, this will be the customer account manager)
• The person responsible for training and support of the end users
• The finance manager who is concerned with cost and budget
• For regulated markets (e.g., Finance, Health Care, Insurance) the person responsible for ensuring safety and/or regulatory compliance

Depending on where you portfolio fits in the overall mission of your organization these individuals may be line managers or executives. What matters is not the level, but rather the involvement of more than simply a technology or program management point of view.

Determining What Your Stakeholders Want to See For Decision-making

Once you have determined who your critical stakeholders are, you will need to find out what they want to see in the dashboard to be informed and be able to make decisions when needed. This is not a hard prospect: you simply need to speak with them (or their proxy representatives if direct conversation is not possible due to market or organization size). Once you complete these conversations, you will have insight into what metrics you need to capture (again, our next blog post) to make the Dashboard useful to them.

Remember to Include Yourself as Well

While it is important to ensure your stakeholders find your Dashboard appealing and useful, it is equally important to ensure it collects the information you need, as Program Manager, to do your job effectively. Therefore, include yourself as a Critical Stakeholder in determining what you need for success. This will ensure your Program Dashboard provides a complete 360-degree view that you and your stakeholders can use to see current and accurate information (and quickly use this to keep your programs on the path to success.)

This is my first tip in my series on creating effective dashboards to ensure program success. The contents of this post draw heavily on the Decision-based Governance Model for Agile Management of large, complex program portfolios.

1. Wrapping up and measuring the success of your 2009 projects and programs
2. Finalizing your budget and planned projects and programs for 2010

As such, this makes now an ideal time to re-examine your existing Program Dashboard to see how you can make it more effective for use next year.

In the past, I have implemented four different program dashboards for use across four different industries: national security, Internet, social media and biotech industries.

However, recently I was fortunate enough to participate in ACT-IAC‘s IT Dashboard Project responsible for recommending improvements for the US IT Dashboard to US Federal CIO Vivek Kundra. This enabled me to learn how over a dozen experts across a variety of businesses are building and using dashboard to successfully manage their portfolios. It also led me to think about how I would help others build successful Program Dashboards for their enterprises. I have boiled this down to four topics:

1. Identifying The Stakeholders (i.e., Customers) of Your Dashboard
2. Defining Metrics that Increase the Chances of Success
3. Mapping Your Portfolio Correctly to Your Dashboard
4. Critical Success Factors to Using Your Dashboard to Drive Success

I will introduce each of these in more detail over my next four blog posts.