I learned a lesson a long time ago – your level of confidence is irrelevant when estimating outcomes. What matters is the justification for your confidence. The Believability Index is the tool to use to better predict, and thereby improve your outcomes.

Decisions

We make investment decisions to support our strategy, seeking to create customer value and competitive advantage. The nature of those decisions blends investment with imagining. We imagine approaches to exploit an opportunity or solve a problem. Then we evaluate those ideas as investments. At it’s core, we decide if an idea is a good one or a bad one. A good idea or a bad idea for us to pursue at this time.

To evaluate an investment, you must estimate what you anticipate. How much do you expect it to cost? How well do you expect it to work? How much do you estimate it will be worth?

I’ve built and seen a lot of estimates informing those decisions. Estimates inside problem statements, business cases, impact maps. Estimates in every unit imaginable. Currencies, team-weeks, t-shirt sizes, click-thru rates, lifetime value. Estimates built through many processes, from Monte Carlo and Markov models to planning poker and value-effort matrices.

Here’s the thing. Not all estimates are equally believable.

Expectations

As product managers, we have the responsibility of making good investment decisions. We take our insights about customers, our anticipations of competitors, and our appreciation of our constraints and use them to generate ideas. Those ideas are the opportunities we choose to pursue and the problems we choose to solve. We collaborate with our teams to discover what we want to build. Through creating differentiated value for customers, we expect to cause changes in behavior. When our product becomes the better choice for more people solving more problems, we expect more sales, leading to more profits.

The best practice in product management is to develop disconfirmable hypotheses describing these expectations. Most of the time, people are not documenting their hypotheses, and instead state their expectations based on their assumptions. A lot of the epics I see have these implicit assumptions of value. It’s the product operations version of a leap of faith. The “build it and they will come” idea, from the movie Field of Dreams embedded in an operating model. A business has to decide if building something is a good idea. What do you expect it will cost to build it? How many of “them” do you expect to “come” after you build it? How much do you expect profits to increase?

You decide based on your expectation of roughly how many people will come, roughly how much it will cost to build it, and roughly how much you expect to benefit after they arrive. Running a business is not a leap of faith. Running a business is a series of intentional decisions based on your expectations. Your expectations are built on what you believe.

Trust and Implicit Risk

There’s a pattern of collaboration I call “T-shaped” where commercial leaders with a wide breadth of perspective collaborate with product managers who have a depth of insight into the domain; the product, the customers, the competitors, and the company’s capabilities. An important aspect of this collaboration is trust.

The commercial leader has to trust the estimates which come from the product manager. Estimates of expected benefit. Estimates of effectiveness. Estimates of cost. Once the product manager wraps up their analysis, assumptions, and hypotheses into an estimate, that detailed work takes a back seat to the estimate.

The estimate becomes the focus of this trust. This is the ideal collaboration pattern. Much better than the order-giver / order-taker pattern of product managers just shepherding the delivery of whatever the leader demands.

There is however, a challenge. Annie Duke, in Thinking in Bets quotes Jeff Yass, Founder, Susquehanna International Group. “The biggest risk is that you have a losing strategy when you think you have a winning one.” There is an implicit risk that the estimate upon which leaders form their beliefs is wrong.

Decision-making in uncertainty is based upon what we believe at the time of making the decision.

The risk Yass and Duke highlight is that the foundations for our beliefs may not be sound.

Believability

As product managers, we need to form and share an appreciation for the believability of our estimates. Not our confidence in our estimates, the justification for that confidence. I built a tool for this in November, 2017, working through this challenge while supporting product transformation at Ford Motor Company. I still have the photo of the whiteboard sketch I did on November 12th. The challenge I was addressing was to be able to express believability about estimates, to avoid introducing hidden risks into the collaboration.

In the collaboration, there is a focus and hand-off around the estimate. A question is asked to determine how much the estimate should be trusted.

Where most people get it wrong is by asking the wrong question. “How confident are you in your estimate?” is the wrong question.

The research from Douglas Hubbard, author of How to Measure Anything, is clear – people are consistently overconfident about estimates. We operate in an environment of uncertainty, with ill-structured problems, undefined variables, and unexpected scenarios. What Kahneman, Klein, and Gigerenzer all point out to us is that we need to leverage heuristics in our approaches to decision-making in environments of uncertainty. Your confidence about your estimate is irrelevant. Instead, people should ask “What is the justification for your confidence?” Or more aggressively, “Why is this believable?”

What matters is the believability of your estimate, not your confidence in your estimate.

The Believability Index

The Believability Index is the tool I developed and have been using with dozens of teams in the almost decade since. What the index does is apply a heuristic to score the believability of any estimate, based upon what underpins the estimate.

The believability score helps people understand estimates. I view believability in this context as the opposite of uncertainty, and built this index to reflect progressive reductions in uncertainty which inform any estimate or belief. Let’s walk through the scores from least believable to most believable.

0. “Let’s try something and see if it works”

A score of zero reflects complete uncertainty – this is just an idea. The CEO came back from a conference, and saw all the cool kids were doing it. “We need to add AI to our ice scrapers.” Trying things is not bad. A tradeoff in portfolio management is balancing the risk of being wrong with the risk of being late. Sometimes the better decision is to just try something.

However, there is no good reason to believe an estimate built on a ” just try something” idea. The idea can be great. The estimate about the outcome of the idea is nonetheless not believable.

1. “Expert” intuition based on domain knowledge

“Trust me, I’m an expert” is, unfortunately, an effective persuasion technique. It can be the embodiment of bravado. Julia Galef warns against it in The Scout Mindset. Daniel Kahneman explored what he called the illusion of validity which stems from the overconfidence of domain experts. Experts can operate on stale knowledge, and suffer from cognitive biases, particularly in estimation. They also tend to be particularly confident, and Kahneman found this confidence to not be a reliable indicator of accuracy.

Having a domain expert tell you how and why something will work is absolutely more believable than a random idea. But it falls short of data in terms of believability.

2. Inference from related or anecdotal data

Many people, for many years, have been telling all of us the importance of being data-driven. Yup. Completely agree.

I think I first heard Rich Mironov say it; when you are building a product or a feature, you’re building it for the first time. In addition to breaking the assembly-line metaphor and associated misconceptions about product development, the lack of repeatability also means there is uncertainty about the effectiveness of whatever you plan to build. The data supporting your beliefs is not perfect. Whatever data you have is, by definition, data about something else. You have to infer the relevance of that data. Knowing that between 1% and 2% of Evernote freemium users converted to paid subscribers 20 years ago does not assure that launching your AI headshot generation service with a freemium tier will drive similar SaaS economics.

Having some data, which you can imagine being relevant, is better than having no data at all. Your estimate is informed – however weakly – by anecdotal data.

3. Extrapolation from correlated data

Correlated data is data much more crisply “about the thing you are measuring.” If you have data about how many people at a train station coffee shop add cream to their coffee, you can build a correlated estimate about how many people at an airport coffee shop will add cream to their coffee.

This is correlated data. Understanding context of usage is important to understanding customers and their behavior. The fundamental question is “Does this data apply?” and is a somewhat subjective interpretation. I think most people have heard the quote “In theory, theory matches practice. In practice, it does not.” I recently heard as an addition, “The difference between theory and practice is what you don’t understand about the theory.” A reasonable example of correlated data would be to look at growth rates for a product in one market, and start with the assumption that growth rates in a new market will match. Unless your markets are very different. Or very strongly coupled.

Another way to think about extrapolation is to look at the past and expect it for the future. If you’ve been growing share by 5% for the last few years, you could extrapolate the curve to expect another 5% in growth for the upcoming year. This is still tricky – conditions may be changing and a continuation of 5% growth is not assured.

Having data of greater relevance is better than less-relevant data. Data is a good foundation for believability.

4. Hypothesis based on extensive research

One of the fascinating things about developing hypotheses is that the act of considering how you would disconfirm the hypothesis triggers an improvement to the hypothesis. We talk about the benefits of testing hypotheses to enable rapid iteration and course correction. This is true, and it is a key element of becoming outcome-oriented, shifting your orientation from “Did we finish?” to “Did it work?” Even more powerful is the experience of anticipating the future evaluation, acting to apply if not force critical thinking to avoid “being wrong.” Don’t underestimate the power of wanting to avoid being wrong.

You develop data – related or correlated – through research. Increasing the level of rigor of research, formalizing into a hypothesis, increases believability further.

5. Indicated by experimental test results

“We ran an experiment and got the following results.” This is the best reduction of uncertainty I know. Therefore, it is the most believable rationale underpinning an estimate.

You cannot completely eliminate uncertainty. You have to – at a minimum – assume the people you experimented with are representative of the people you are trying to understand. Your experimental design will introduce opportunities for what you observe to misrepresent what you will expect. I had a client who used ring-releases, launching features to 1% of the user population, to minimize the risk of doing harm and to gather data about potential benefits. The team expected the results of releasing to 100% of customers to be nearly-perfectly predicted by the results of the 1%.

At a minimum, you have to ask if the results from the past experiment will still apply in the future. The conditions will change. Your new proposition may be a differentiator at the time of launch, but a competitor’s rapid move to neutralize your advantage could still undermine your business case.

This is why there is no certainty, only believability.

Using the Believability Index

The believability index helps us address the hidden flaw in a trust-based collaboration. Without the index, the person making a decision based upon an estimate has no information about the believability of the estimate. Do they consider it to be a sure thing, or a long-shot? They may sandbag or inflate expectations, biasing their decision. As a product manager, when building an estimate, don’t just share a clinical set of numbers.

Apply the believability index to characterize those numbers. You will make better decisions when you share both an expectation (your estimate) and a characterization (believability score).

The shift from talking about what we want to build to talking about the problems we want to solve is easy to understand, but can be difficult to do well. We have to make decisions about which problems we will solve first. We have to make decisions about how grand of a solution we want to create. Talking about value is how we do both.

The Size of the Problem

When a leadership team is setting direction for their company, they start with a vision and their first thoughts about strategy. I love Richard Rumelt’s definition of strategy, defined in Good Strategy Bad Strategy.

The kernel of a strategy contains three elements: a diagnosis, a guiding policy, and coherent action.

Diagnosis is about problem identification. And coherent action is the approach we’re going to take to address it. All told, this is what Rumelt calls “a cohesive response to an important challenge.”

There’s a step of sense making where leaders will simultaneously explore their understanding of the key challenge and a coherent action – identifying the actionable problems which must be addressed. This is high-level problem decomposition. If the challenge is around customer retention, the problems could be around service quality, product fit, pricing, or responses to competition. “Losing too many customers” is too abstract to throw at a product team in a large organization. But improving service quality or neutralizing a competitor’s advantage – those are good meaty problems a team can sink their teeth into.

Capacity Constraints Force Choices

As my friend Rich Mironov has said at least a hundred time, we have no shortage of good ideas. We have to choose which problems we will solve and which we won’t. If we’re smart about it, we will decide which problem to solve first and which to solve second, so we begin reaping the rewards of our first effort while we are working on our second.

Leadership teams want to shape their strategies in pursuit of Rumelt’s “cohesive response”, and should do it in partnership with their product managers. A natural first step towards shortening the list of problems is to eliminate all of the problems which are too small. We also eliminate the ones which are misaligned, too uncertain, or beyond our capability to solve. For now, let’s focus just on size.

Your instinct for where to start is solid – start with the largest problem. Use a good problem statement template and make sure you are quantifying in economic terms, so you can compare the value of solving two very different problems. In a discussion with a client this past week I asked “how do you decide which is more important – increasing customer adoption or reducing customer billing errors?” When you don’t use economics to create a frame for comparison your choices are still driven by intuition.

The Problem of… We have revenue leakage of $10M per year because of billing errors
Affects Whom… Customers, customer support, product line general manager
The Impact of Which is… $2M per year in EBITDA at a 20% gross margin
The Benefits of a Solution are… $2M per year in increased EBITDA through reclaiming $10M in revenue

This looks great. It absolutely defines the size of the problem.

But you can’t always get what you want.

The Size of the Solution

If we had magic wands, we could just choose the biggest problem and solve it. Magic outcomes. But we don’t. Our constraints come into play when we start exploring how we might solve problems. The laws of physics get in the way, first and foremost. The better you get, the harder it is to get better.

In this example, the $10M of leakage is likely to be the aggregate result of a collection of billing issues. Bad addresses, incorrect calculations, pricing mistakes, just as a few examples. Like most problems in the real world, the solution isn’t as straightforward as making one change which drives the totality of benefit. You have to work through a collection of inter-related changes to tackle the problem in aggregate. Then the S-Curve gets you. Resolving all of the sources of leakage is more than twice as hard as resolving half of the issues.

Given that this effort is competing with others for limited capacity, the best use of your limited capacity is to make a smart investment on this problem and smart investments on other problems. This is better than going past the point of diminishing returns on any one problem.

Go back to the article on diminishing returns for the full collection of visuals building the line of reasoning.

Capability Constraints

When I’m helping teams define the work they are going to do to solve problems, we explore not only capacity constraints but capability constraints. Small nimble teams lack scale, large bureaucracies with scale lack nimbleness. The capability gap could be technological, operational, or one of domain expertise.

For any problem you want to solve, between capability and capacity constraints, instead of magically completely solving the problem, what you will instead end up doing is making incremental progress towards solving it. We need to rewrite the problem statement from above to make it useful.

The Problem of… We have revenue leakage of $10M per year because of billing errors
Affects Whom… Customers, customer support, product line general manager
The Impact of Which is… $2M per year in EBITDA at a 20% gross margin
The Benefits of a Solution are… $500K per year in increased EBITDA through reclaiming $2.5M in revenue

While we start by orienting to the scale of the problem ($2M in EBITDA), we have to take into account and talk about what we can accomplish. We form this into a coherent action to address part of the problem. The problem statement helps you with the decision to take this action. When working with teams, I will help to include the problem statement as part of their initiative or program or epic which they use as an artifact to help them make decisions about what to work on and track progress in execution. In this case, the problem statement clarifies an intent to eliminate 25% of the problem.

This container acts as a focusing element for the team to decide if the collection of actions needed to address 1/4 of the problem in its totality is a high priority in comparison with everything else. The team can collaborate to see what the effect of moving the dial on ambition has. Should they tackle 20% of the problem? 50%? Answering the question for this effort requires consideration of the alternative investments being considered and prioritized.

The Size of the Problem Being Solved

Instead of prioritizing based upon the size of the problem in its totality, we have to prioritize based on how much value we are creating. You don’t pay for an all you can eat buffet based on how much food is available, only how much you intend to eat.

There is a huge benefit when designing, scoping, building, and testing, to having a crisp definition of what you are trying to accomplish. You are being unfair to your team if you give them the general direction based purely on the size of the problem.

“We are losing $2M in EBITDA every year due to revenue leakage – try and do something about it.”

You are still being unfair when you give direction without quantification. Now you’re also being unfair to your leadership.

“We are going to reduce revenue leakage by reducing failures to contact the customers.”

This sounds good, you can make a persuasive pitch, even cite the size of the problem. But if you don’t talk about the size of the problem you plan to solve, and only talk about the totality of the problem, you’re not helping anyone.

You lack the information you need to prioritize this work ahead of other work. You lack the information to align the scope of work with the level of ambition, making effective design and execution impossible. All the team can do is build what they want to build – they don’t have a way to answer “did it work?”

Use the The benefits of a solution section of the template to make it clear how much of the larger problem you intend to tackle.

You might find you get what you need.

Vibe coding, as a faster way to tell people what to build is penny-wise, but pound-foolish. Too many people are latching onto a “show don’t tell” metaphor for making it easier and faster to tell their teams what to build. The same forces which make this possible are making this foolish at the same time. We are in uncharted waters, sailing into the storm of an uncertain future. The last thing you want to do is lash the wheel into a fixed position. Trying to go faster in a fixed direction feels like solving the wrong problem. Instead, try to improve your ability to adjust your course in response to the future you encounter.

More Directing When We Need Redirecting Instead

What we see too often in organizations is order-giving and order-taking. Where someone is directing the team on what to build. The common pattern is a serialized, siloed process. First a business case is developed or an idea forms in someone’s head. Requirements are defined, a design is selected, and finally the product is built, tested, and shipped. Product development ‘pipelines’ reinforce this metaphor of certainty. Whether you imagine this as a waterfall, a stage-gate, or a Kanban with definitions of ready and done, the sequence is the same. The work moves through a series of domains sequentially. A stakeholder describes their idea and directs a product manager to elaborate requirements. The product manager directs designers to imagine a solution which meets those requirements. The designers direct the engineers to build the product or feature through a specification. This output is tested for compliance with the spec and launched into the unknown.

The team executes along this path with blinders on, avoiding external signals which might slow them down. Speed is everything. Success is defined as finishing. On time. At quality. Within budget. Do everything we planned to do, as we planned to do it.

As leaders pop up a level and evaluate this process, looking for improvements, it is easy to focus on the concrete. When striving to be better, the easy definitions fall into place. Ship features faster. Ship more features. Ship at higher quality levels. Ship at lower cost. Do a little of each. Do a lot of all of them. More cheaper better faster. The unfortunate, but understandable bias in too many organizations has been to get better at doing things this way, instead of striving to get better by doing things a different way. There’s buzz today (summer 2025) around major organizations abandoning the PRD for the vibe coded “show don’t tell” artifact to give direction.

This form of direction is concrete, visceral, and less time consuming. It also comes with a payload of lost opportunity and triggers cognitive biases while simultaneously disempowering the team, making it harder to discover the storm ahead and redirect the course.

If you’re using AI to make it faster and cheaper to do evaluative research and discovery, I am thrilled. This article is not a rebuke of that. I’m responding to what I’m seeing which is very much not that.

Product Failure: A Systems Failure

As a product manager, I know that we’re building products in conditions of limited information. We have a lot of ways to be wrong. Incomplete information, misinterpretation of the information we do have, and of course making bad decisions based on how we process the information we do have. To make things more difficult, it takes time to build things, so we’re building for a speculated future. We need to use the available information and our intuition to predict what reality will be like in the future. Even more ways to be wrong. Lovely.

I first started building my personal model for why products fail in 2012. At that time, I was focusing on some failures of execution – building a flawed business case, taking the wrong approach to positioning and selling, etc. But I also hit on a couple key failures of decision-making – picking the wrong market or solving the wrong problems.

What I appreciate today is that a lot of product failure happens because of the systems product managers operate within, the processes which define how companies develop products. While execution failures are still sources of product failure, I’ve focused my attention on a cascade of decisions which can cause product failure regardless of how good your execution may be. An attention to that model helps me help companies improve their processes.

I mention it here, because I think the context is needed to express my concern about how vibe coding is emerging in the zeitgeist and how I expect it to do harm.

Building the Wrong Product Right

The model I developed represents a cascade of decisions, each of which, if made poorly, puts you on a path to failure from which the following decisions cannot recover.

• If you choose the wrong market in which to compete, your selection of the right customer group in that market is irrelevant.
• If you choose the wrong customer group, even in the right market, then it doesn’t matter which of those customer problems you choose to solve.
• Solving the wrong problems for the right customers makes the exercise of setting the bar for how good your solution will be (defining “good enough”) pointless.
• If you set the bar too low from your customer’s point of view, it does not matter how elegant your design is.
• With the wrong design, the level of quality, efficiency, and speed of your execution is unimportant.

If you don’t make all of these decisions well, you end up building the wrong product.

(Larger image)

In an excellent article from June 2025, my friend Rich Mironov wrote about the explosion of interest in AI-coding in his article Bottlenecks, AI, and Where Product Adds Value. This is timely stuff. Rich has an optimistic take – by changing the bottleneck associated with the capacity to execute, product managers will naturally gravitate ‘upstream’ to put better emphasis on these weighty decisions. I have a worry that little will change about how we approach these pivotal decisions, while cost-fixated execs are enamored with replacing their payroll expenses with smaller token-expenses.

Remember, it’s the people who sell those tokens encouraging the shift.

The Case for Uncertainty

Underlying the decision cascade in the above diagram are two categories I identify, “Market Uncertainty” and “Implementation Uncertainty”. These are concepts I embraced (co-opted?) from Rita McGrath’s work, shared in The Entrepreneurial Mindset.

To truly oversimplify McGrath’s model, the greater the market uncertainty, the less likely we are to be right in our decisions about what we need to do. Similarly, the greater the implementation uncertainty, the more likely we are to make mistakes about how to do it.

The dominant question you should have in your product development operations is not “did we finish?” but rather “did it work?”

Estimation errors increase because different approaches end up not working. We go back to the drawing board. In the agile world, the notion of a “spike” emerged as a way to account for the capacity applied to learning different approaches to building what the team was trying to build. Often, spikes are architectural in nature, but I’ve also seen them used to reflect explorations of usability. The conversation is easy to understand – the original estimate is for the work required to try the first idea. Those estimates get sidelined when the team discovers they need to try a different approach.

The greater the implementation uncertainty, the more likely our first solution idea is to be wrong.

Requirements errors is such a horrible phrase, but it is the one which has ossified over the years in the order-giver / order-taker world. “That is a requirement error, we built what you told us to build.” Design decisions, in my opinion, are in the domain of “how to do it” not “what to do.” I appreciate that every person on the team may use different labels to describe these decisions (if you want to jump 20 years back on memory lane you can see how I was talking about it in 2006). Regardless of the labels, this where choosing the right markets, customers, and problems lives.

The greater the market uncertainty, the more likely we are to be building the wrong product.

Instead of making my pitch for why uncertainty is higher today than ever before, I’ll just ask you to think about what you see. Is technology evolving faster now in a way your competitors could exploit, or your engineers are embracing – making the impossible possible and the impractical practical? Are these rapid changes causing customer expectations to similarly rapidly change? Are the mechanisms of value creation for your customers changing? Are the problems they face today and will face tomorrow changing? If you’re seeing this, you aren’t in an environment of low uncertainty.

Product Manager as Orchestrator

I’ve seen people share their belief that the product manager is the orchestrator who just needs their team to build what they imagine. This is the product manager I want to compete against. Their individual mistakes will be solidified, unassailable, difficult to discover and correct. Their teams will build the wrong products, even if they build them well.

The vibe coding push explores the possibility for a product manager to act independently to both choose a problem and then choose and prototype a solution approach, asking the rest of the team to “polish” their idea. This approach can be appealing, particularly when you manage product development as a cost-center and not a profit-center.

I think the dictator product manager is a real problem for a couple reasons, centered primarily how we think and how we design. To explain quickly, I need to shift away from an unfortunate thinking frame most product folks have – the double diamond. The double diamond mistakenly describes product creation as a sequence of first selecting a problem to solve, and then selecting a solution to the problem. The research tells us it doesn’t work that way, and my experiences working with teams and in technology for 35 years agrees.

Hat tip to my friend Will Haas Evans at Fugue Strategy Advisors for helping me clarify this insight. It is through our efforts to solve a problem by which we truly understand it. It is in the attempt to design and build the right interaction to help a customer solve their problem that we discover we’re solving the wrong problem. Vibe coding a “show don’t tell” solution dramatically reduces or eliminates that exploration. My problem isn’t that “showing” is faster than “telling.” My problem is that by changing the structure of the communication this way it prevents exploration and displaces collaboration.

A true understanding of the problem to be solved only emerges through exploration of solution ideas and approaches. It is in the elaboration of understanding the solution through which we refine our definition of the problem. This is messy. It needs to be messy.

What percentage of products succeed? Maybe 20% of the low-hanging fruit pursuits of risk averse large companies. Less than 1% of innovation initiatives from startups.

The “should product managers create prototypes on their own?” question feels odd in the double-diamond frame. The product manager eliminates the solution space. This is obviously problematic when acknowledging the emergence pattern of reifying the problem through pursuit of a solution. One of the pieces of research worth exploring if this is your jam is How Designers Work – Making Sense of Authentic Cognitive Activities published by Henrik Gedenryd in 1998. The emergence phase is where we form the insights which improve our answers to the questions on the left.

Having an AI build a prototype for you, regardless of how much re-prompting you do to refine your solution ideas, breaks all of this. So broadly “don’t do that” is my TL;DR

The rush to prototyping is doubling-down on the wrong mindset. For two reasons. First, design fixation will lock everyone into the particular solution approach of (and implied problem definition of) the prototype. David Jansson and Steven M Smith published their research while at the Institute for Innovation and Design in Engineering at Texas A&M. The teams will cycle around making the best version of this first idea – it becomes almost impossible to explore other ideas. If you are operating in a domain with any implementation uncertainty, this is a problem. You essentially eliminate the entire emergence phase of ideation. You’re likely forcing the team into building the wrong solution even if you focused on the right problem for the right customers. And now the process can no longer help you improve those decisions if you got it wrong. Which you probably did based on industry product-success statistics.

Second, you relegate all of your teammates into an order-giver / order-taker pattern, asking them to polish your brilliant idea, to make your perfect prototype “production ready”. This is more than just rude, it simultaneously devalues and dismisses all of the contributions they could be making.

The smartest person in the room is never smarter than the combination of the people in the room.

Enabling Emergence

There are a lot of issues with a lot of organizations, preventing or undermining the emergence phase of cross-functional product development. I help clients identify and address those issues. When I see a product manager proudly vibe-coding their way past collaboration and emergence, I see a problem. The operational patterns which help us better to learn are getting sidelined instead of embraced.

More than ever, we are placing product bets into an uncertain future. Don’t undermine your ability to navigate uncertainty by lashing the wheel and preventing yourself from course-correcting.

How do you debate how to build something you’ve been asked to build? There are several constraints which affect choices – non-functional requirements (NFRs), cost, capacity, and capability are easy ones to discuss. And while all of those are relevant, an altogether different factor has dramatically more impact on your results.

“Can We Do It?” vs. “Will It Work?”

There’s a pattern in most organizations, where people are told what to build, so they build it. They are given some latitude to decide the best way to build it. I’ve written before to make the point that when people ask for things, they tend to ask for the wrong things. But that’s not directly what I want to emphasize here – I want to dig into what happens on the receiving end of the request. These are the two sides of how things play out when the pattern of orientation towards creating value in the organization anchors on creating outputs. People ask for things, assuming they are valuable. People then receive the request and deliver those things, assuming they are valuable.

Yes, the assumption of being valuable causes problems, because the value is often not sufficient to justify the expense. Even if the thing being requested could result in value creation, it may not. Value escapes the team because of how they treat the request. More specifically, because they are unable to treat the request the way they should.

When asked to build something in this way, these are the questions which unfortunately are the common core of the negotiation which unfolds. These are output questions.

• Can you build it?
• Who will build it?
• When can you deliver it?
• What will it cost?

Those output questions should be peripheral questions. These outcome questions need to be at the center of the collaboration.

• How is value created?
• How will we measure success?
• How much will we benefit from having it?
• When do you need to have it?
• What are we willing to spend to build it?

For too many people, the output questions are familiar, and the outcome questions are foreign.

In the output-focused scenario, the questions are asked by the speaker. In the outcome-oriented scenario, the questions are asked by the listener.

The answers to the outcome questions are the ones which matter for making good decisions and ultimately creating great products.

A Semantic Sidebar on How

There’s a semantic debate I think we can avoid about how we describe what we create. Arguing about the difference between “the thing we are building” or “the way we are building the thing” is – for this article – only semantic. People often talk about “what, why, and how” and this sidebar is digging into what “how” really means, so we can avoid doing that through the rest of the article.

Here’s an example – think about enabling on-the-go order-status checking for your customer.

You could build a mobile app which provides order status information to the customer. Is the mobile app “the thing you build” or is it “the way you build it”? An architect would probably say that an order-status-sharing capability is “the thing” and the mobile app is “the way.” This architect would suggest that you could enable order status updates via text message, or a mobile app, or a website, or a call center, etc. Each of those implementation choices, to the architect, is “the way.” Others on the team would likely describe the mobile app as “the thing.”

You could look at the mobile app solution-approach and declare that you will build a native app unique to iOS or Android, or a progressive web app which works on both, or a hybrid app which forces different compromises than either of the other approaches. Is the cross-platform app “the thing you build” or “the way you build the thing”? You can keep going – does the app include authentication features? Does the app provide a view of order history? Are you building C++ code or React functions?

All of these descriptions of how to build are both “the way” and “the thing”, depending on where you are anchored.

I’m referring to this generally as “the way” in the rest of the article to simplify the narrative, because we need to focus instead on “the why.” It is “the why” which needs to drive “the way.”

Deciding the Best Way to Build

To decide what the best way to build would be, you have bring the right information into the decision. Yes, all of the information from the output questions is relevant, but it is secondary. You should be focusing on the more important questions.

How Is Value Created?

The mechanism for value creation is what you need to understand to have a sense for the right way to build something which creates value. This is about cause and effect. You need to know the effect you’re trying to create to choose the right way to cause it. Use a solution hypothesis to describe your belief. We believe if we solve a customer’s problem, by building some thing, it will result in a change in behavior. Take a look at the Assumption of Causality section in this earlier article (link).

Let’s look at a concrete example for this abstract concept. Imagine we believe a customer has a better shopping experience when they know the status of their order, because it eliminates anxiety. This is what Indi Young might classify as a serious harm (link). Different people in different situations may experience different versions of this. Someone living in an apartment may worry about a package being delivered in a public area where it could get stolen. Someone away from home when placing an order may worry about the package being delivered before they get home and being damaged by bad weather or wildlife. Someone ordering medication may worry about exposure to the elements ruining their shipment.

That is the situation of the example – now imagine we also believe that improving the customer’s experience results in an increase in customer loyalty – they will be more likely to purchase from us in the future. There’s a value proposition to the customer (reduce anxiety about delivery) which drives a value proposition for the company (do this and increase loyalty to increase revenue).

Instead of “build a mobile app which provides order-status information” now we have “reduce customer anxiety about order status, to increase customer loyalty.”

The first declaration forces you to build a mobile app and makes it impossible to answer the next question. The second objective gives you choices and the context you need to measure the success of your product. This is a big deal. If you stop reading the article at this point, and just do this, your products will be better. But don’t stop, there’s more you can do.

How Will We Measure Success?

With a directive to build a thing, all you can measure is if you finished building what you planned to build. The definition of success is absent. It does not exist.

With a customer goal of reducing anxiety about order status, we can now explore – how much must we reduce anxiety? With a desired outcome of increased loyalty, we can explore – for how many people must we reduce anxiety?

In Discussing Design, by Adam Connor and Aaron Irizarry, a key concept for critique is suitability. A colleague of mine, Dennis Stevens referred to this as fitness function – how well the capability you’re building fits the need you are addressing. The choices you make about the way you build something influence the effectiveness of what you build. With an objective like “reducing anxiety for this many of these people in this situation by this much” you can critique solution approaches. You have a framework for deciding if “the way” is a good one, and you can compare different approaches and choose the best one. You also have the opportunity to say “we cannot imagine a solution approach which reduces anxiety enough for enough people to warrant building anything.”

How Much Will We Benefit from Having It?

When accepting direction to only build what is requested, you are absolving yourself from responsibility for delivering outcomes. Just build it. If it doesn’t generate value, that’s someone else’s problem.

I’m writing this because it is your problem, even when you don’t acknowledge it. Here’s a little thought experiment. If all the requests were valuable, where the ROI was 2x or 5x or 10x for every feature being requested, how would things play out? As my friend Rich Mironov famously says “there’s never a shortage of good ideas” (one of several links). If every idea has high ROI, and we’re capacity constrained, what will leadership do? Increase capacity. Like scaling up the GPUs in a server farm, bigger is better. Micro-economics tells us we should keep scaling until the marginal cost of increasing scale equals the marginal benefit of the additional deliverables. This is definitely not happening.

What if every idea isn’t valuable? Then there would be downward pressure on costs and headcount. Looking to sustain business as usual, to “make do” with fewer people and keep the lights on. Sound familiar? I believe the pressures on budgets and team sizes and increased throughput exist because only some of the requests generate returns. I also believe the way most companies try and solve it is misaligned with the root cause. Instead of making it cheaper to deliver the wrong things, I believe we should make it more likely we deliver the right (valuable) things.

Increase the size of the top line instead of decreasing the size of a line item.

When subjected to this level of thinking, many requests to build something get cancelled. Because it becomes clear they aren’t worth building. This came up twice last week with two different product managers I’m helping. Both of them offered examples of work which just evaporated when value was explored.

You cannot assert or assess the value of the request, but you can assess the value of the objective.

With an objective like “reducing anxiety for this many people by this much” you can critique solution approaches. You can develop an outcome hypothesis before you begin exploring solution approaches. “If we reduce anxiety by this much for this many this often, we will see a revenue increase of $X…” You now have some guidance for imagining a solution – it must reduce anxiety by “this much” for “this many people” with “this regularity.” You can develop solution approaches to meet those observable, measurable changes. Fitness of the design to the job at hand requires criteria. And these are the criteria. You also have the opportunity to say “we cannot imagine a solution which reduces anxiety enough for enough people to warrant building anything.” Explore cost-effectiveness of the solution by defining effectiveness.

When Do You Need to Have It?

We’ve been trained to ask and answer this question the wrong way. Every organization I’ve worked in or with in the past 35 years has done it, and still does it. We answer this question in terms of scheduling and planning. Imagining the effort required to deliver the solution, thinking about who is available to build it, and the logistics and dependencies which are part of our operational reality.

The problem is, none of that stuff is actually answering the question – all of that stuff is answering a different question “when can we do it?” Again, useful info, but the wrong focus.

When someone needs the solution is an entirely outside-in consideration. There are market rhythms, like launching before a trade show, or back-to-school shopping season. There are milestones in our customer’s or supplier’s timelines, like a key piece of infrastructure being turned off (sunsetted or EOL’d are common jargon terms for this). There are often compelling events which have an impact on the value of solving a particular problem. Delivering a solution after the deadline is sometimes pointless. We should know that as a constraint on how we would approach solving the underlying problem.

Even in the absence of deadlines or cliffs, there is the time-value of money. Once you identify and quantify a problem, you can now identify the cost-per-month of not solving the problem. The faster you can deliver a solution, the more value it has. In agile, the ‘focus on finishing, not on starting’ principle increases the value of whatever is being built by doing this. Assuming the work is valuable, which it often is not.

Understanding the influence of time on value is also an important factor in deciding “the way” you build a solution. An approach could be 50% as effective but launch in 1/10th of the time. Is that a good thing to do? The economic analysis of value (for your customers and for your business) is how you can explore.

What Are We Willing to Spend to Build It?

In theory, this is a redundant question. By knowing how much we benefit, our willingness to spend is a foregone conclusion. Companies have rules of thumb – establish a particular payback period, ROI, NPV, internal hurdle rate, etc. And at times those rules of thumb are ignored to make exceptions for strategic initiatives, like an acquisition or a race to “do AI.” You simply do the math for what qualifies as an acceptable investment, except when a leader says we can ignore the math.

In practice, I have found this question to drive better decisions. I think it happens because people bundle together a bunch of different emotional approaches to operating in uncertainty. Every investment is a gamble. Prospect theory tells us that losing hurts more than winning feels good. Different people get different levels of satisfaction from placing sure thing bets vs. long-shot bets. What I want to do is activate those pathways (whatever they are, for whichever team or leader), because it improves the thinking about the value proposition.

When asked “what is it worth?” people tend to estimate a number. With training, they develop an estimation range. Knowing the range, however, doesn’t tell us everything we need to know to decide to make the investment. Cost estimates have uncertainty just as value estimates do. Just as with value, people tend to estimate with a number, and with training, a range.

If the high end of the cost range and the low end of the value range overlap, what do you do?

When asked why it is so hard to create a bear-proof trash can, as the joke goes, it is because there is a significant overlap between the smartest bears and the dumbest people.

How should you deal with the situation where the cost estimate range for “the way” you chose to build something overlaps with the value estimate range for solving the problem? Ultimately, the choice about how to build something is part of the decision to build it or not. When the highest costs overlap the lowest values in your estimation range, you should explore different ways to build.

When you start with a definition of your objective, you can ask the question – will an alternative solution approach, while lowering costs, also lower the expected value? This is every bit as hard as designing a bear-proof trash can. But it is a necessary consideration when trying to create value for your company through creating value for your customer. A different approach may undermine the realization of value.

Conclusion

Focusing on “the way” must be done in conjunction with a focus on “the why” if you want to create value for your customers and your company.

If we don’t agree on the problem we are solving, we cannot agree on the solution.

I have a bit of a history with the concept of ‘good enough’, from satisficing in sprint planning (2008) to Kano analysis for product managers (2009) to trying not to miss the mark in problem solving (2012) to understanding intent and satisfying customer needs (2014). I explore the concept from different perspectives in product workshops I run, and in my Competitive Product Strategy class. In this article, I want to dive into combining internal and external perspectives to make investment decisions.

I first learned about diminishing returns as a formal concept in economics class. The concept was easy to grasp, as there are so many practical examples which match intuition. An example I use in my class applies utility theory from economics to eating bananas.

Imagine you are hungry, and would benefit from eating a banana. The benefit you get from eating that banana is a reduction of your hunger. How much benefit would you get from eating a second banana? You get less benefit from the second banana than you got from the first banana. This makes sense – you’re less hungry than you were before. A third banana would provide even less value than the second banana. The amount of benefit which comes from each additional banana diminishes.

The reason this works is because the problem you are trying to solve is getting progressively smaller. With each banana you eat, you become less hungry. Therefore the value to you of becoming less hungry becomes smaller. Starvation is a large problem and the value of solving it is large. Feeling a bit peckish is a small problem and the value of solving it is small.

We have to think about diminishing returns in product management because people keep assigning value to the banana. The value is in solving the problem.

Laws of Physics

Every problem reaches the point where continued investment to reduce the size of the remaining problem is not worth it. And this comes from two insights I think of as ‘laws of physics’ when it comes to problem solving. These two laws act as a double-whammy, forcing you to find a definition of ‘good enough’ for every problem you choose to solve.

• Diminishing Returns: It takes progressively more objective improvement to realize a subjective benefit.
• S-Curves: Objective improvement becomes progressively harder to achieve.

Diminishing Returns

These graphs highlight the nature of the model of diminishing returns. We start with a goal for how much we want to increase value for the customer. As a function of how much problem remains, we identify how much problem-solving is required. If the customer is very hungry (the problem is large), then we can create disproportionate value for the customer with a modestly sized solution.

If however, there is not much remaining problem, the customer is approaching being satisfied, then conditions have to get notably better to create an equivalent increment of value. We have to make a bigger dent in the problem to achieve an equivalent increase in utility for the customer.

Problems rarely have binary characteristics, they tend to follow what Kano called “More is Better.” He was looking specifically at feature-level analysis, and when abstracting up to utility or value, I believe diminishing returns is necessary to acknowledge.

S-Curves

S-curves are a model for considering the costs associated with actually making something which solves problems. In the diminishing returns graphs above, we are looking at the impact of somehow solving the problem. In the S-Curves, we are looking at what is involved in solving the problem. We’re double-clicking into the “somehow.”

Think about your ability to perform some function as a capability. The way you should measure a capability is always context specific, as a function of what you’re trying to accomplish. What you can do, to avoid that complexity and keep going is simply think of it this way – you solve a problem (with a capability) by improving the capability. You could invert this as well, to say the better the capability is, the less problem there is remaining to be solved.

You could judge a city’s traffic system in terms of how many people can commute from outside the city to a stadium for a sporting event. The more people you can get to the event in a given timeframe, the better the system is.

When coming at this from the point of view of solving a problem, you would define the problem with something like this problem statement.

The Problem of… The stadium averages 80% capacity for sporting events due to traffic congestion preventing people from attending
Affects Whom… The owners of the sporting team and stadium, and city residents who benefit from tax revenue
The Impact of Which is… 20% of seats are unfilled on average at events, resulting in $100M in lost revenue per year
The Benefits of a Solution are… increasing revenue by $50M per year through reducing congestion enough to convince an additional 12% of people to attend

In this situation, you might define the capability as being better when it has higher throughput, or you might define it when people’s perception of congestion is improved. Let’s go with throughput for now, because it is easier to imagine how those investments work. The main idea about an S-curve is that the better you get, the harder it becomes to get even more better.

It could be hard to add a lane to an existing road to increase its capacity. Adding another lane becomes even harder. Roads tend to be built with easements and shoulders, allowing for “easy” expansion in the future. But continued expansion beyond what was anticipated can require relocating utilities, or even businesses and structures. The more you improve things (capacity), the harder it becomes to improve them. The top of an S-curve looks like diminishing returns, because it is the same pattern, but considered from a different point of view.

What makes an S-curve different is how it begins.

There is, in most of the situations I’ve encountered, some initial work which must be done to lay the foundation for building something meaningful. A software developer may have to invest in their development environment, an organization may need to build their pipeline for deploying software. A city adding high-capacity transport with a light rail system has to build a lot of infrastructure before even the first train can run.

Those investments enable future investments to complete the work of delivering something which is potentially valuable. A manufacturer who assembles drones may need to make all of the logistics investments to get all of the drone components into one location before the work to assemble those drones can begin. Once that up-front work is done, the model highlights where modest incremental investment can yield large increases in capability.

Once you’ve built the rail line, the incremental expenses of adding trains or train-trips to the schedule yield large increases in capacity. This is where the bang for the buck really pays off. There is real potential value in making these investments, tied to a measure of the capability as a presumed source of value. Many engineering teams still struggle with the decoupling of potential value from realizable value.

Escaping this struggle requires combining the S-Curve (familiar to engineers) with the diminishing returns model (familiar to product managers).

The last portion of the S-curve is where the compounding effects put you at a disadvantage.

It simply becomes harder and harder to become incrementally more capable. Adding the initial rail line and train schedule creates an improvement in capability. As you saturate the schedule, it becomes harder to add additional trips, or additional trains, or additional tracks. The more you increase capacity, the harder it becomes to increase capacity. The crux of the S-Curve.

The Double Whammy

Now combine the increasing difficulty of becoming more capable with the decreasing incremental value of becoming more capable. This is the double whammy. These laws of physics are one of the two reasons why companies taking an inside-out approach to developing products are at a disadvantage competitively. Without considering these two principles, you are more likely to overinvest.

You are more likely to enhance the capability beyond the point where it is a smart improvement. You will make unjustified investments of time and money without yielding sufficient benefit to customers.

Conclusion

Combining these two concepts allows us to shift our thinking. We can reorient from an inside-out question of “What do we expect it to cost?” to an outside-in question of “How much should we be willing to spend?”

Every meaningful investment decision we make should anchor on the value of the investment, not solely the cost of making it.

In the early modern period clocks worked through an engineering miracle of pendulums and springs. You would wind a clock, to tighten the spring, which provided the tiniest of nudges to keep a pendulum swinging. This regular, periodic swing of the pendulum is what allowed clocks to tell time. People relied on the clocks, which relied on the regular period of the swing of the pendulum. This predictability and the ability to tell time made much of the modern world possible.

As an engineering student, I found the law of conservation of energy to be really amazing. It’s what drives a pendulum clock. At the top of the arc, right as its velocity reaches zero before swinging back down, then pendulum has potential energy but no kinetic energy. At the bottom of the swing, when the pendulum is at its lowest point, it is moving the fastest it can move – it has kinetic energy, but no potential energy. The notion of trading potential energy for kinetic energy sticks with you. Energy is conserved. Potential energy becomes realized energy. Predictably. With certainty.

A Cultural Metaphor

In a conversation with a colleague earlier, it occurred to me – maybe the power of this metaphor influences why so many teams operate as feature factories. I’ve seen individuals, teams, and entire parts of organizations absolve themselves of responsibility for business success. I usually refer to this as the order-giver / order-taker model.

Engineering teams ask “the business” to tell them what they want, and then they build it. The customer gets what they ask for. The team pats themselves on the back for delivering it on time, on budget, at quality. This is where the metaphor of the pendulum comes to play. If the team were asked to swing the pendulum, raising it up and putting potential energy into the system, they expect with certainty that upon release the pendulum will create kinetic energy as it swings through its arc. After all, it is a LAW of physics.

The team is asked to lift the pendulum. The assumption is that it will result in a fast swing. If the team is asked to build something, it must be valuable; it must be worth doing. Building what is asked for will result in value. Engineers are wired to trust this connection in physics class. It may even have been on a final exam, where their conviction that potential value always converts into realized value was put to the test.

The metaphor of conservation of energy in a pendulum undermines our ability to create valuable products.

The Longitude Act

In 1714, the Parliament of Great Britain passed the Longitude Act, offering a prize to anyone who could help ship navigators know their location without landmarks in sight. Different prizes had been offered for a couple hundred years by different monarchs, in pursuit of a solution.

Without a landmark in sight, a sailor can use a sextant to determine how far they are from the equator; the latitude of the ship. They cannot, however, determine their location East-West; the longitude of the ship. Because the earth spins on it’s axis, to look at the stars in the sky and calculate your longitude requires you to know what time it is. At different points in time, the stars are at different locations relative to the horizon. You need a predictable clock to calculate longitude.

Guess what doesn’t work on a ship. A pendulum. It still swings. It no longer swings predictably. It swings uncertainly. Clocks of the time didn’t work on ships, hence the prizes.

In physics I, engineers learned about pendulums on land, not pendulums on ships. Unless we stumbled upon the history of navigation, we were unlikely to consider that Conservation of Energy comes with some caveats, and that pendulums are not, in fact, certain.

When asked to build something (to swing the pendulum) we should not assume that the thing we are building will be valuable.

Hypothesized Value

While there is no explicit discussion about hypothesized value, there is an assumed potential value. This output-oriented way of working is comfortable. The implicit assumption is that the output has potential value and that value will be fully realized, just as the pendulum converts potential energy into kinetic energy.

Two things happen when teams orient to the work this way. Both are bad and one is hidden.

The visible bad thing is the team operating as order-takers absolving themselves of responsibility for the outcome. They are able to wash their hands of it. Deliver on time, meet the spec, and if it doesn’t yield results, that is not their fault. As a result, the work is less enriching for the team because they lack purpose; the team is stacking bricks, not building a house. There are consequences for the organization as well. All of the insights of the talented people on the team go untapped. Great ideas which could have happened don’t happen. As Whittier told us, the saddest words are “it might have been.”

The invisible bad thing is the inability of the organization to do any course correction because the team is not aligned to the outcome, only to the output. I wonder if this feels ok to the team because the latent metaphor of the pendulum makes it ok. It becomes comfortable to presume that whatever they release – however much potential energy they create – must result in realized value, just as releasing the pendulum must resolve into kinetic energy. The team is trapped in the metaphor of a pendulum on land, but the reality is the team is operating on a ship, and pendulums are no more certain than anything else.

This is a comfortable, but unfortunate way to work. It isn’t collaborative. It doesn’t align the efforts of the teams to purpose. From a management point of view, it is treating the people building the product as cogs in a machine. The order givers require nothing from the order takers aside from delivery.

Conclusion

When value is not part of the conversation, the organization may as well be buying lottery tickets. Immediately jumping to planning and execution, without first doing some evaluation. The presumption that the requested deliverables are valuable is no more true for the team than the presumption that a pendulum clock will work on a ship.

What we want from our leaders is for them to set a direction. To inspire us to advance a strategy or realize a vision. What we get too often is dictation, where they tell us what they want us to build. Changing this relationship requires us to change how we interact. As product managers we can change the conversation. Just as you change collaboration patterns through changing how you develop roadmaps, you change the conversation with leaders through how you orient towards supporting them.

I use two characteristics to differentiate giving dictation from setting direction. First, the scope of work has already been defined and the team is being asked to deliver it. This is an order-giver / order-taker pattern. The thinking has been done up-front, and only the doing remains. There’s a not-so-subtle flaw in the first clause, however. The work is never sufficiently defined. Second, there is no invitation to discuss if it is in fact the right work. There are two ways the product manager and team can think about the work. “This is the work we’ve been told to do, so we will do it” which is contrasted with “this is the purpose of the work we’ve been told to do, so we will achieve it.” Is the team given the agency and responsibility to achieve the purpose (usually “solve the problem”), which requires an invitation to discuss the purpose. When there’s no discussion of purpose, there is no setting of direction, there is only dictating.

Note that I never said “solution” even though that is the language I see universally applied in this situation. You cannot have a solution without a problem. You only have work, or deliverables; scope or outputs; things.

The Dictation Cycle

“Build me {this thing}. When can I have it?” Some dictators are benevolent dictators, who care about their people, but who “see the light” and operate with an authoritarian clarity about what needs to be done. They may ask “When can you build me {this thing}?” In my experience in business, people dictating to their teams are usually well-intentioned. I once heard the advice to assume that all of the people you encounter are smart people trying to do the right thing. While I have been proven wrong a few times, this isn’t a discussion about bad actors. This is an exploration about how the nature of this conversation undermines those smart people, and can even prevent them from discovering if what they are requesting is the wrong thing to request. This isn’t about name-calling a leader as a dictator, it is about honestly acknowledging that the dialog is a dictation – orders are given and orders are taken, without more than a superficial interaction.

Dictation feels like micro-management, where the person setting direction is also telling you how to do your job. As a product owner, you would never tell your developers to add a new feature, to write the code in Haskell, to use CamelCase variable names, to indent code with 3 spaces instead of tabs for each line. This is clearly insane. When a leader who is responsible for setting direction tells you what to build, I assert that they are doing your job every bit as much as the product owner (PO) who requires developers to type one-handed.

The framing of a dictation conversation bypasses “Should we build this thing?” and instead teleports to “What is this thing?” and “How should we build this thing?” There is a progression of framing which is healthy – why, what, and how. In dictation, the product manager is uninvited to the ‘why’ conversation, is asked to participate in the ‘what’ conversation with the dictator, and participation in some version of the ‘how’ conversation is complicated because the ‘what’ conversation is always incomplete.

With dictation, there is no ‘why’ conversation involving product managers. The ‘why’ conversation happens behind closed doors or inside a single person’s head. This is a problem, because the double-diamond is a lie. The lie is that our ability to navigate the problem space can be decoupled from our navigation of the solution space. In language we talk about first understanding the problem, and then working to imagine and execute a solution to the problem. The dictator believes they have already navigated the problem space, and are engaging the team to then begin navigating in the solution space.

To quote Will Hass Evans, Author of Designing Resilience, “So neat. So simple. So wrong.” It is through the exploration of possible solution approaches that we truly develop an understanding of the problem. Janson and Smith explore this in their research on design fixation and describe the concept space and configuration space as interdependent, where spending time in each affects your understanding of the other.

Imagining how you would solve a problem changes your understanding of the problem.

(Larger image)

When leaders force a decoupling – when their process looks like the double diamond, everyone else is excluded from collaboration. The emergence phase does not happen. The leader doesn’t get to participate, the product manager doesn’t contribute, the rest of the team don’t contribute. It is the act of trying to solve a problem which drives not only an improved understanding of the problem, but the discovery of ways to improve the framing of the problem. This collaboration is the necessary activity to increase the potential value of the work to be undertaken.

The product manager is responsible for shaping the work to be done. The product manager is responsible for increasing the potential value of the work being undertaken, not merely fulfilling orders for deliverables.

Signs of Dictation

Understanding how the system is operating starts with understanding how work gets done, what individuals are expected to do and not do. When sitting with a product manager, I will ask if individual success for them is defined as solving the problem behind a request, or if their performance is assessed in terms of delivering what was requested (on time and on budget, at quality, of course). When success is tied to “did we finish?” the dictation pattern is almost certainly in place. Operational metrics (flow metrics) are typically in place, and “did we finish?” is not a bad question. The problem arises when it is the dominant question. So I continue my exploration with the following:

What should you be willing to spend to deliver this project?

There’s only one reasonable line of reasoning to get to an answer for this question. Some version of “This project is intended to solve that problem, which is worth $X to the company, so we should be willing to spend up to Y% of $X to solve it.” When the product manager can answer this question, and the rest of the team knows where to find the answer to this question, you’re in great shape because you’re not trapped in the dictation antipattern. You’re in a world of delegation if not autonomy, where direction-setting is integral to how work gets done. Based on my experiences over the years, you are unlikely to be in this situation.

What I commonly hear when asking about value is explanations that the team was just told to get it done. It is a mandate, a must-do, or the most important thing to some stakeholder (or their boss). What I regularly hear is people answering this question based on the cost estimate. The project must be done, the team has estimated the cost of doing it, budget has been allocated. So the team should be willing to spend whatever budget was allocated. In a lot of organizations, approaching this question with a frame for economic decision-making is simply a foreign concept.

You should not be willing to spend more to deliver the project than the benefit expected from the project.

When people lack the data, or lack the agency, or lack the safety to have this conversation, this organization is operating in the dictation antipattern. The consequence is a double-whammy; the work takes longer to elaborate and costs more to execute, and the works is less valuable than it could have been.

Shifting the Dialogue

What a lot of teams, and expert project managers, have done is harden the existing process with higher expectations about the state the work should be in before advancing to the next step. These are potential improvements to the flow of the process which fail to address the underlying issues. To improve this situation and break from the dictation antipattern, the organization needs to change to a direction-setting pattern. One of my best early recommendations to product managers who find themselves in this situation is to introduce a coherence check into the conversation with stakeholders.

It isn’t helpful for a product manager to jump up on the table and declare the team won’t work on a project without knowing how much benefit is expected from the project. A product manager is responsible for understanding the value of the project. The product manager needs to provide clarity of purpose to their team. The cool thing is that even in the absence of direction and purpose coming from the stakeholders who asked for the work, the product manager has the ability to create this clarity. I would argue the product manager has the responsibility to create clarity around the benefit, value, or outcome expected for any given body of work.

The common dialog, when responding to dictation, is to ask for clarification. A dictation request comes in to build something, or even to enable something to happen. The clarification questions will feel familiar

• What about this situation, or that situation?
• When scenario 1 happens, how should the system respond?

In the old days, we wrote use cases instead of user stories, to describe how we intended the system to respond to stimulus. This is inside-out thinking. The classic “build it and they will come” failure, Costner movies notwithstanding. All of the clarification questions reinforce the design fixation issue, making it harder to course correct and eliminating opportunities to learn.

Different conversational techniques can be used to get to the underlying purpose. Writing a problem statement is the grown-up version of the toddler’s five whys. In a world of unicorns and rainbows, the project request would come with a problem statement which could be evaluated and improved. The next best thing would be to collaborate to write the problem statement together with the stakeholder as part of reviewing the project request. What most product managers have to do – at least initially – is attempt to write a problem statement on their own.

This is where product managers need to understand how the business runs. The product manager will leverage their understanding of the mechanisms of value creation to reverse engineer the project by asking “and then what happens?” Attempting to populate the four fields of a good problem statement template is a fantastic prompt for the necessary critical thinking.

The Problem of…
Affects Whom…
The Impact of Which is…
The Benefits of a Solution are…

The first two fields, describing the framing of the problem and who it affects, are really effective focusing lenses. Attempting to answer the economic questions is where the coherence check comes to life. By quantifying how much value the project is trying to capture, as well as how much potential value exists (by quantifying the impact of the unaddressed problem), you gain so much insight into the potential outcome. Even being able to have a dialog about trying to solve the totality of a problem, or addressing part of the problem incrementally – this is a compelling realization.

This is a critical element of the emergence portion of understanding problems and solutions. There is now a frame for understanding the project. “If” that is the scope, what is the potential benefit? You now have an approach to matching the scope of the work with the (unexpressed) level of ambition underlying the project. You can have a dialog to make sure they match. You are connecting a couple things.

1. Your belief about the outcome a project will yield, given an understanding of the scope of work.
2. Your belief about the mechanism of value creation, through which a scope of work will yield value.
3. Your stakeholder’s belief about intended outcomes with your belief about expected outcomes.

Coherence

This is where coherence comes into play. By comparing your expected outcome with the stakeholder’s intended outcome, you can see where there is misalignment and how to address it. It may be that the stakeholder has a different understanding of how value is created. This is great – you get a better understanding, refining your beliefs. You update your expression of the expected outcome, in the ‘benefits of a solution’ field. Now the entire team can align to this updated belief.

It may be that the stakeholder, with a bigger picture view, is informed with an improved appreciation of how the scope associated with the project is likely to lead to a different level of value than what they intended. The project might deliver more than they intended – allowing them to refine to a smaller project without undermining their original intent. It may be that the project – the implementation originally defined – will fall short of the intended outcome. What the product manager brings to bear is a deeper understanding of the specific mechanism of value creation being potentially exercised. Combining this depth of insight with the breadth of perspective of the stakeholder allows the two together to collaborate to reshape. This reshaping can both alter an understanding of the problem and alter the imagining of a solution approach.

Conclusion

This coherence, matching scope to ambition, refines the shared understanding of the underlying problem through simultaneous exploration of the problem and the solution. This coherence provides better clarity of purpose to the team, useful to inform decisions during execution. This coherence provides clarity of purpose to the stakeholder, who can now express prioritization, selection, and sequencing desires against an economic frame. Both of these changes lead to an increase in the potential value of the work being done. Both of these changes lead to an increase in value realization which results from the work.

“I don’t get to set the direction. Leadership tells us the big projects they’ve decided to do, it’s up to us to elaborate and execute.”

This is a common situation for a lot of product managers. They’ve been told that some big initiative is happening, and it’s up to them to “make it work.” Without all the details needed to unambiguously define what “it” is.

Here’s an example – A company is growing by acquisition, and is now faced with the integration / consolidation of two different products. Merging into a single customer base, looking for cost efficiencies from a combined team, increasing profits because of the reduction in price-competition in the market.

The problem statement for that initiative is likely undefined or unshared with the product teams, but we could imagine something like:

The Problem of… Our widget-product is not growing as we lack the scale to fund the product improvements we need to gain share
Affects Whom… Prospective customers for whom our current product falls short of their needs
The Impact of Which is… Uncaptured servable obtainable market share (SOM) of ~20%
The Benefits of a Solution are… A gross-margin increase of 50% through allocation of fixed costs across a larger customer base

This may be muddled, as the benefits are around profitability, but the problem is around lack of competitiveness. These are intertwined concepts, and if this were an article about better shaping of the central problem statement, we could dive into it. This, instead, is an article about how to better operate within the context of someone else’s decisions.

Inside the Mandate

The leadership team has decided to acquire a widget-product competitor, buying the additional market share, bundled with the additional costs associated with maintaining the acquired product and servicing the existing customers. Most product managers I work with are operating in organizations where they are not participating in decisioning like the above, and instead are operating within the context and consequences of those decisions. This article is a focus on the “now what?” parts of making the intended benefit of the acquisition real.

In rough terms, the company expects cost-efficiencies to drive an increase in profitability, and that the increase in revenue (once the paired increase in costs is eliminated or reduced) will fund investments to improve the product(s) to improve competitiveness and gain additional share.

The solution selected to address the original problem was “acquire competitor Zeta.” Within that context, there are problems which exist which the team must address to make it work. The challenge for the team now becomes identifying those problems within the operating context. To do this, the team must align on the definition of success for the acquisition.

You cannot define the solution of a problem without first defining the problem.

Even in a situation like this, where the team lacks the agency to define the outer problem, there’s still a need for a coherence check to understand how ambitious to be in identifying problems and solution approaches within the outer mandate. To define what needs to be repaired, improved, or created, the product manager needs to know what the definition of success could be.

A common pattern is to start with an imagined comprehensive solution, and then break down that solution into independent projects. Those projects can then be allocated to discrete teams, slotted into incremental delivery schedules, and project managed through delivery. The challenge is, instead, to identify the comprehensive problem (we need to consolidate the experience to realize any efficiencies), and then break it down into a series of independently addressable problems. This is problem decomposition not solution decomposition, helping to avoid design fixation* resulting from prematurely defining a bunch of projects and then focusing efforts on delivering those projects.

*Design fixation is when all of the team’s decisions moving forward fixate on the solution design in front of them. The team becomes unable to shift focus back to the underlying problem to be solved, unable to consider alternative design approaches.

Immediately jumping to an imagined implementation supporting the entire experience, then elaborating details and assigning to teams is the wrong approach. The consequence of jumping to solutioning at this point is you may not make a difference, focusing on areas of the solution which are devoid of problems or stopping short of completely addressing needs. The result is an overpriced and underwhelming deliverable. I hesitate to call it a solution, because it might not even solve anything. By failing to deliver on the value proposition for investment and acquisition, the entire thing can become a boondoggle.

I’d lay the blame on the organization collectively rushing to build some “thing” instead of planning to solve the problems preventing the benefit. There is often a pathological desire to appear busy which disrupts attempts to make a difference.

Finding Problems Inside Projects

The “acquire Zeta and sell their widgets” context has been set. Within that context, there are plenty of problems which might be worth addressing. Because you’ve established the context, you can now identify problems within that context. Given that the customer support experience is going to be supporting customers of both Alpha and Zeta widgets, what are the identifiable problems you might solve. Even within this mandate to consolidate the process, there is agency. It is bounded, but it is present.

Hypothesize that there are issues with trying to leverage the existing processes to support Zeta widget customers in addition to Alpha widget customers. The first question becomes how to identify those issues.

An obvious place to start in this example is to look at the value stream of how the company makes money in the widget business, to see where the gaps might be associated with selling Zeta widgets. The danger of this approach is that it reflects an inside-out view. Given how your process works today, what scenarios can you imagine, and what problems can you anticipate. The danger of this approach is that the scope of work is bounded by what you can anticipate and imagine. What’s missing is the outside-in view. Understanding how your customers experience your company, product offering, and services.

By using a customer journey map as a framing tool, you are constantly reminding yourself and everyone else that the secret to creating value for customers lies within the customer’s experience. I am consistently working with organizations who have not had experience developing customer journey maps. Often, their decisions – about which problems to solve, about selecting solution approaches, and even about confirming their effectiveness happens without discussion of customer needs.

To introduce the concept, I start with an easy to imagine, high level description of a customer’s experience in interacting with a company. This process is likely not a perfect fit for the scenario I’m helping a team to address. It is, however, very approachable and easy for them to appreciate and edit. For this article, I’ll use it unmodified.

This is a model of a customer’s comprehensive experience. Breaking it into stages might feel artificial to a customer, but it is still helpful for problem identification and evaluation. The first stage in this journey is a customer becoming aware of an unmet need – a desire crystalizes, a problem surfaces, etc. Once that awareness is present, the customer shifts into shopping and buying a solution. Cognitively, these are different – think of it as choosing among multiple products, then purchasing one of those products. Product usage, maintenance, and repair are all different elements of the experience of owning a product. At the end of life of a product, you may replace it and you may dispose of the original.

Aware > Shop > Buy > Use > Maintain > Repair > Replace > Dispose

1. Aware – become aware (of your needs, of the product)
2. Shop – make purchase decisions (compare, review, evaluate)
3. Buy – transacting (paying, warranty, add-ons, account creation)
4. Use – out of box experience (OOBE), first use, ongoing use
5. Maintain – required to enable continued use (incl. preventative maintenance)
6. Repair – fixing something which has broken to restore ability to use
7. Replace – replacing broken, lost, obsolete with newer or better alternative
8. Dispose – getting rid of the previous one (repurpose, recycle, destroy, discard)

Within each of these stages, you can look for problems which need to be addressed – because what the company built to support this experience with Alpha widgets may fall short of expectations when it is leveraged to support Zeta widget customers.

Let’s look at the ‘Shopping’ stage in this experience. When someone is shopping for a widget, part of what they are trying to do is decide which widget to buy. Previously, the prospective customer would have been comparing the Alpha widget, the Zeta widget, and perhaps another competitor, the Beta widget. The Alpha company shares information which positions the Alpha widget as being better than both the Zeta and Beta.

Since the acquisition, the company would be excited if the customer purchased either the Alpha or the Zeta widget, as long as they don’t purchase the Beta. Is there a problem which needs to be addressed? Does positioning the Alpha widget against the Zeta widget help sales or hinder them?

The Alpha and Zeta widgets are competing for the same customer, and having two alternatives available makes it hard to choose – potentially driving more business to Beta. The problem here may be to eliminate the choosing (between Alpha and Zeta) to stop accidentally nudging prospective customers to Beta.

There are multiple ways to approach solving this (I know you’ve mentally started solutioning), from killing one of the products to better managing customer flow so only one widget option is presented to any given customer. Another approach is to invest in both Alpha and Zeta products, in a way which differentiates them from each other, making each appealing to a distinct group. This is a great time to remember that you cannot competently design a solution without a definition of “solved” for the problem. Stop unspooling the solutioning thread for a moment and frame the problem.

The Problem of… Forcing customers to choose between Alpha and Zeta causes more people to choose neither and pick the Beta competitor
Affects Whom… Prospective customers shopping for widgets
The Impact of Which is… At least 10%* of the people who should have chosen either Alpha or Zeta end up choosing Beta instead
The Benefits of a Solution are… An increase in profits of $100K annually (10% increase in revenue of $1M at 10% net margin)

*Note – 10% is not meant to be a made-up number, it is simulating for this article the result of quantified assessment. I’m including a number to reinforce the need for quantification when developing problem statements for real, even though this example is not focusing on quantification.

This would enable an initiative to be developed to solve this problem within the solution. Treating the decision to purchase Zeta as an externality does not force you into order-taker mode, it merely clarifies the level of agency you have. As a product manager, you’re still responsible (even if not accountable) for creating the value which comes from solving the problem.

You are constrained to operate within a context. This doesn’t force you to be a cog in the feature factory machine. Identify and solve the problems which exist within the context of the overarching directive.

With this example from the shopping phase, the layers of context are interesting. This is an opportunity to increase profits by changing a characteristic of the anticipated experience. It is a problem because it is undermining the financial premise of the outer context. When Zeta was acquired, there was an untested and unexplored assumption that the combined sales of Alpha and Zeta would not be negatively affected by selling both through a combined experience. After the acquisition, it was discovered that there is a dynamic which undermines the original investment premise. This is an unanticipated problem which exists because of the acquisition.

Defining the problem statement like this within the broader solution helps to provide real clarity on the size of the problem, which helps in both selecting the problem and sequencing the solution of it.

Consider another example, this time inside the ‘Repair’ stage. There is consistently a lot of interesting infrastructure associated with supporting customers after they’ve purchased a product. From stocking replacement parts for physical products to partnering with and certifying service providers who perform maintenance to staffing a call center to provide support. When the company acquired Zeta, they may have decided part of the cost savings would come from consolidating two call centers into one. This problem is anticipated and must be addressed. Problem statements are also helpful in this situation.

Part of the premise for consolidating was that the Alpha and Zeta widgets are similar enough that the same call center agents can support customers who purchased either. Further, leadership assumed there are more support agents than needed. With a little bit of investigation, it becomes apparent that the Alpha widget experts don’t know enough about the Zeta widgets to help customers with issues and vice versa. As part of the acquisition, all customer support calls have been merged into the same hotline. Previously, Zeta had their own call center, and Zeta customers would call that number. Now, all Alpha and Zeta customers call the same phone number.

The question to ask is what are the problems preventing this call-center consolidation from working as expected? This example could be considered an anticipated problem, which people knew needed to be solved all along. From an exec’s perspective, this is just part of the scope – ‘figure it out.’

How might you describe the problem? Here are two valid considerations – immediate cost penalties and delayed revenue penalties. Immediate costs are the ones associated with triaging the incoming calls to route them to the appropriate agents – this takes extra time, may require additional fees for the call-center infrastructure, and may have a small cost associated with configuring the system and training agents to route calls to the right agents. Losses in revenue result from a degraded support experience depressing future sales which would have come from repeat purchases or word of mouth recommendations.

The Problem of… Incoming calls for Alpha and Zeta product support need to go to different agents but all come in on the same line
Affects Whom… Call center agents who handle product support calls for Alpha and Zeta
The Impact of Which is… An additional 20 seconds per call of handle-time reduces effective call center capacity by 10%, increasing cost-per-call by 10% (which works out to a 1% increase in net profit, as allocated support costs are roughly 10% of COGS)
The Benefits of a Solution are… An additional $10K in annual profit (increasing annual profit from $90k to $100K at $1M revenue and 10% net margin)

The Problem of… It is hard for customers to get effective support for Alpha and Zeta products because they cannot easily reach knowledgeable agents
Affects Whom… All customers with Alpha or Zeta products in need of repair
The Impact of Which is… The company gets a reputation for poor product support, degrading word-of-mouth ($200K) sales by 10% and repeat-purchase sales ($150K) by 10% as customer loyalty declines
The Benefits of a Solution are… Protecting forecasted revenue through sales both via recommendation (+$20K) and repeat purchases (+$15K) – yielding annual net profit of $3,500 (10% of $35K)

Your instinct may be to lump these together into a single project, responsible for delivering both outcomes. That instinct can make sense if the same solution approach can be imagined to solve both problems. I’ve seen situations where the same implementation leads to the resolution of multiple problems, but it made sense to treat them as separate efforts because the testing needs are different. Using the example above, solutions which reduce the increase in handle time might avoid degrading the support experience, or they might not.

In this example, there is a pretty clear alignment, where a solution can simultaneously reduce handle-time (cost) and improve the support experience. It may be that what you have to implement to solve both is more complicated than what you would implement to completely solve either. You could choose to focus on the first (reducing call center costs), while anticipating (hoping) it also reduces the size of the second problem by being “good enough” for a large slice of those people who be dissatisfied with the experience.

Iterative and incremental development is not “what do we build next?” it is “what problem do we tackle next?”

Once the solution to the larger problem (costs) is in place, you can evaluate if the secondary problem (lost future revenue) is still actually a problem. It may be that the solution to the first problem reduces but does not eliminate the second problem. It may be reduced so much that it is no longer the next most important thing to work on. This is an example of how problem statements help you to shape your product roadmap by helping you to refine your product roadmap as you go. Before you did any work, you saw two problems, both worth addressing. Your clever approach to solving one of the problems reduced the size of the other problem so much that it was no longer worth addressing. You update the roadmap to reflect a choices about what is most important at that time in the future.

You operate as a product manager with a degree of agency established by your org design and process definition. By introducing problem statements within the context of a mandated solution, you are playing the hand you’ve been dealt and doing good product work. As a product manager you can, through the articulation of problem statements:

1. Provide clarity of purpose for your team to LI, highlighting the rationale for building something.
2. Provide a crisp definition of success, shifting the operational from “did we finish?” to “did it work?”
3. Support economic prioritization decisions when determining how best to allocate capacity and sequencing decisions when determining when to allocate them.
4. Avoid jumping to solutions which may fail to address the problems, allowing teams to contribute to collaboration around achieving successful outcomes.

Conclusion

You don’t have to have own a product strategy to get value from writing problem statements. Defining success in terms of outcomes and aligning teams towards achieving success is what product managers do, and can do even inside feature factories. Identify the problems which need to be addressed within the context of mandates. Those mandates could be very high level (company strategy), high level (market segment defined), or mid level (high level solution approach selected). Write good problem statements within the context of what you’re being asked to achieve.

Which is the bigger risk for your product right now? Shipping with a feature your customers don’t value? Or delaying your release in order to ship with something they do value?

User Feedback vs. Deadline

I recently answered a LinkedIn Pulse Question – “How would you navigate incorporating user feedback when facing tight deadlines for product releases?” https://www.linkedin.com/advice/1/how-would-you-navigate-incorporating-user-feedback-as3kf?trk=cah2

I feel this question needs more visibility, and a bit more discussion than we can have in 750 characters in that section of their site, so I’m expanding my thoughts here and digging into some of the context of why this isn’t a trivial question.

Business Agility

I define Business Agility not in terms of process cadence, but in terms of three characteristics of how a company operates. If you don’t have all three, you’re not agile, regardless of how you manage your projects. If you disagree, I’d love to discuss it with you. I’ll be at Agile2024 this week – and you can probably find me near the C4G booth most of the time. Or start the discussion here or in the comments on LinkedIn when I share this post.

Three Non-negotiables of business agility

1. You have to create opportunities to learn within your process. Having an opportunity to incorporate user-feedback is a great example.
2. You have to perform the activities which generate learning during those opportunities. Asking users for feedback is an example. Many teams don’t do this or don’t know how to do this well.
3. You have to update your approach when you learn something. This is the pressure-point where far too many organizations get in their own way, unwilling to change the plan.

Risk Management

The pulse question, about how to deal with new information in the face of a tight schedule is something which pokes at the third characteristic. The implication of this being at the top of the list on LinkedIn Pulse is that this is a decision people don’t know how to address. I have a point of view – this is a risk-management decision.

You are balancing the risk of being wrong, with the risk of being late. And you have to decide which is worse. When raising our daughter, I would sometimes say “you have to choose your hard.” When you don’t do one thing which is unpleasant, it often comes at the cost of dealing with some other unpleasant thing. Cause and effect. So decide which one is worse and make your choice.

An economic analysis of the decision is reasonable, but people often screw it up. Estimating the cost of being late is something we have well established models for. We use the cost of delay to calculate the cost of delaying a ship-date. A mixture of delayed cash flow and lost revenue makes up the mechanisms of opportunity cost in this decision. People can develop what feels like a concrete assessment of the cost of shipping late.

There’s a minor issue in that by presenting the findings as a calculation, it engenders a sense of false precision in the analysis. Really, there is a range of uncertain consequences associated with schedule delays – between uncertainty both about the impact of, and the magnitude of the delay. That’s a thread worth pulling another time.

The major issue is in the other evaluation. What’s the cost of being wrong? This is so much significantly harder to assess that most people just don’t do it. My sense from working with so many teams focused on efficiency (at the expense of considering effectiveness) is that they don’t even consider this as a criterion for success. With an output-orientation, you win simply by shipping. “Did it work?” is a question you don’t even ask. And in too many organizations, “did it work?” is an unwelcome question. It is too uncomfortable.

Making this decision is problematical – comparing an ill-defined cost of being late with an undefined cost of being wrong. Too many teams pretend that “undefined cost” is the same as “no cost” and just choose to keep the schedule.

For those teams, the plan has become the goal. What happened to the goal?

I start building my model of the cost of being wrong with the premise – “no one will buy the wrong product.” This is both extreme, and wrong. Some people will buy it. But instead of asking people to share their beliefs about how many sales (relative to the established expectation) will be lost from shipping the wrong product, I ask them to explain how anyone buys it.

This is actually useful, because “wrong” is not a binary attribute either, but rather something much more nuanced. You have to genuinely look at the customer feedback, and then decide how this will affect your business model. Will ignoring this cause you to lose renewals or attached sales, driving customer lifetime value down? Will you miss out on referrals and organic growth through word of mouth? How many people are there who would buy if you address this but would not buy if you don’t?

All of this is uncertain, and you need to express your beliefs as estimates and ranges. But now you are at least making a decision comparing two ill-defined costs. You’ve levelled the playing field. You can make this decision well. Which now allows you to consider investing to improve the quality of future decisions.

Getting Help

This is the kind of stuff I help teams do, if you’re new to me and someone shared this with you. If you want help solving these problems, reach out and let’s discuss how best to make an immediate improvement.

Going to Market

You will develop a go-to-market (GTM) approach which includes multiple releases of your product. Each release needs to be a distinct, coherent, valuable offering. There needs to be someone who wants to buy the first release. Shaping your product requires you to add the dimension of considering multiple interim versions of the product, each more ambitious than the previous one. Like the Amazon mobile app, initially focused on the spear fisher, and gradually growing to be more valuable to more people in more situations.

Too many teams try and use reskinned project-management tools which are helpful when sequencing instead of being helpful when shaping. As a result, I think the “what can I do first?” and “when can I do the next thing?” questions displace the more important questions of “what value will the first version create and for whom?” To tease apart the difference between shaping releases vs. sequencing work, it helps to contrast two different ways to think about timing. I’ll demonstrate with a facilitation exercise I ran for a leadership team where we shaped a product – in this case, a bespoke transformation services engagement. I find this to follow the same patterns as developing a product roadmap to operationalize a product strategy.

Shaping an Engagement

Several years ago when beginning a multi-year services engagement with a large corporation, I helped the leadership team through the process of both selecting and sequencing the problems they would want to address as part of their transformation. The first step, problem discovery, was to elicit the collection of problems which were of concern for the leadership team. Recognizing that the benefits which come from solving problems varied, they identified which problems were more valuable and which were less valuable to address. I believe the leaders expected me to use the Eisenhower matrix as a prioritization tool to map importance and urgency of all of the problems to build a prioritized list. In the context of organizational change, I find there is something more important to consider than explicit urgency – appetite.

This group of leaders was operating within part of an even larger organization, and the problems they identified were not solely within their control to fix. Every potential improvement required some form of change beyond their scope of direct responsibility. Sometimes solving an identified problem would require significant or disruptive changes in other parts of the organization. In all cases, solving a problem at least required leaders or teams to collaborate and operate differently across those organizational boundaries as a result of doing things differently within their own areas. In a large organization, political clout is needed and political capital is sometimes spent to drive change outside a leader’s area of responsibility. Leaders may not feel like it is the right time to make waves, or may not feel like they are personally interested in leaning into another leader’s space at a point in time. So instead of combining importance and urgency, I combined value with “appetite.”

Transformation efforts will not succeed without leaders stewarding the changes. You cannot grass-roots your way to a meaningful change at scale. From a shaping point of view, what we were looking for was the highest value problems to solve which leaders had an appetite for solving. Some high value problems were pointedly unappetizing for them at the beginning of transformation; it was helpful for everyone to understand this. What we discovered was which of the highest-value problems this leadership team was willing to push to solve. Committing their energy, their political capital, and possibly their promotions and bonuses to the change. Now with a 2×2 matrix, we identified the most valuable and most appetizing problems to address. Effectively defining the scope of the desired transformation. We were ready for the next step.

The next step was to determine a “when” for solving the problems. Instead of building a timeline-roadmap for the transformation, I had them build my version of what Janna Bastow would call a “time horizon roadmap.” Janna has been promoting the time horizon approach for years, which puts things into a coarse-grained timeframe. Two approaches she has shared are to label investments as “now, next, or later” or to use the categories “current, near term, future.”

I have a variation I find I enjoy more. “Now | next | not yet | never.” While the alliteration is fun, the real value comes in having explicit conversations about problems which will not get solved. Every problem which made it onto the list had someone who experienced it, if not championed its resolution. In a collaborative setting, it is helpful to acknowledge and remember when the decision has been made to not solve a problem. It helps facilitate “disagree and commit” behavior in a healthy way. If strategy really is about saying no, then this gives you a great place to store the ideas which don’t get pursued.

As the leaders took the high-appetite, high-value problems and mapped them into time horizons, it allowed healthy conversations about feasibility – capacity of the team to deliver and capacity of the organization to absorb simultaneous change. The leaders were also able to unearth some sequencing dependencies, where solving one problem would yield limited value if another problem were not already solved first.

The collection of problems which landed in the “now” bucket represented a coherent and valuable offering – a first release. Adding the scope of “next” problems unlocked more value, representing the next discretely valuable offering – a second release. As time progressed, the “next” problems would become the “now” problems, and the murky future unfolded to enable potentially changing the shape of the offering. Some things would not yield as much value as expected and the team would go back to the drawing board to try again. Other problems surfaced which weren’t originally considered, driving discussion to reshape the transformation.

Pruning the Product Tree

In Innovation Games, Luke Hohmann introduced an elicitation exercise called Prune the Product Tree, where you get customer groups to identify the features they want to add to (or remove from) a product. In that exercise, customers are encouraged to demonstrate their desired time horizons – with explicit guidance to identify nearer term and longer term timeframes for the features. In Software Profit Streams Luke and Jason Tanner use the metaphor with a subtle shift – describing the growth of a product in terms of the benefits it creates for customers. They refer to this as a “growth centric” roadmap, differentiated from a time centric one.

I like the metaphor of a growth-centric view of shaping (pruning) your product as a way to consider a time-horizon view. Products, like transformation projects and trees, grow organically, and not always at a predictable pace or in exactly the expected ways. Agreeing on “these things first” is the principle goal of this exercise.

[larger version]

While people are talking coarsely about timing, the real signal in this exercise is relative importance – which problems are the more important ones to address? When I used it in the example above, I was combining discovery of the key organizational issues with alignment and collaboration to shape the bespoke services offering.

When running this exercise about a product, with different customer groups, you get localized signals about what matters to each group, allowing you to think in aggregate terms about your market. For the people you’re working with, time horizons act as a proxy for an undefined combination of importance and urgency. Not only are you not your customer, but not all of your potential customers are the same.

You will find not only that different groups of customers care about different problems, but that they care about the same problems differently. A problem which is “now” for one customer group can be “next” or “not yet” for another. Problems can also be never for a particular customer group, even though I didn’t show that in the visual below.

[larger version]

These signals help you decide which customers to help first, which problems to solve first. It also helps you to evaluate some assumptions about problems – you may feel a problem is shared by multiple customer groups, only to discover that only one group values solving it.

As a result, you can shape how your product will grow. For which customers will you try to provide value first? Which of their problems will you try to help them solve first? And maybe most importantly, operationally, which problems are you choosing to not solve. You are now able to focus on the immediate.

The “next” problems are likely to evolve, your understanding of them certainly will. The “not yet” problems are more of a signal of directionality for the moment. You have to develop a product plan which is feasible, taking into account both your capabilities and capacity to deliver.

Regardless of what tools or techniques you use, the important thing is that you are shaping your product in terms of the value you intend to create for customers in a way you believe will lead to value for your company. When you move from hand-wavy philosophy about value and desirability into operationalizing, you need some form of artifact which manifests this purpose, which you can use to align the organization and clarify for the teams actually building solutions. You need to combine knowing how to write a problem statement with knowing how to use the problem statements to set direction – to both shape and sequence the development and release of your product.

Feasibility Rears Its Head

Shaping your product starts with choosing to solve problems which are valuable to customers, making the solution desirable to them; but must also be done in a way which creates value for your company, justifying the investment. Product investments which are not simultaneously valuable and desirable are bad ideas. However, this alone does not assure the idea is good; some ideas can be objectively good, but not good for your company to pursue. They may be infeasible for you to execute.

Feasibility concerns are concrete and immediately visible. They get the lion’s share of attention within the organizations I’ve helped. The initial go / no-go decision includes an ROI analysis, with a typically unstructured estimate of potential value and a well structured estimate of costs, capacity, and schedule. Through execution, managers focus on output-delivery schedules and measures of feasibility; almost universally bypassing measurement of desirability or value.

The first pass of roadmapping yields an ambition, the desired shape of your product. The customers generate desirability signals for you. Then you develop an approach to meeting those needs which you believe will also drive business value assuming the approach is realistic. This is the right time to push on that assumption of feasibility. Your teams have limited capacity and capabilities. You have developed a solution approach for each problem you choose to solve. Those approaches represent a way to achieve the [[Clarity of Purpose|purpose]] embodied in the problems you are signing up to solve.

The reality of execution is that it takes time to create solutions. When your capabilities are limited, it can take you more time than it might otherwise take – a feasibility double-whammy. It may be that some solution approaches are effectively impossible to execute in the time horizons you desire when treating the organization’s capacity and capability as fixed variables.

When you shape your product approach – and in this visual depicting a choice to only solve 3 problems “now” – you create a signal for conversation with your leaders or execs. Your choice to address 3 problems and not 5 in the first release was likely heavily influenced by the constraints preventing you from solving all 5 in the first release. Can you increase capacity and capability to be able to do more now? Should you make those investments?

This is where you benefit from conversation centered on the value you expect to generate from solving the problems you intend to solve. You are limited in how much value you can create by the capacity available to allocate to the solution approach you envision. When your conversations are focused on outputs or features, and not outcomes or benefits, you are hamstringed in your ability to make a good decision. You can endlessly discuss “can we deliver more now?” and “how can we deliver more now?” You are unequipped to ask the real question, “should we deliver more now?”

Should you keep the missing resources allocated to a different pursuit? Are they even available? Can you create or acquire them? The answer will be yes – if you’re willing to spare no expense. But should you? Only if the benefit of doing it outweighs the cost. Only if the benefit of doing it exceeds the opportunity cost of whatever you’re delaying to acquire the additional resources. You cannot answer those questions without an estimate of the value you expect.

Conclusion

The right frame for decisions is to ask “should we solve this problem now instead of solving a different problem now?” The reason we are constrained in our ability to solve problems as rapidly as we desire is because of our limited capacity to implement the solutions. It is easy to abandon the discussions about problems and center conversations around the solutions. But you lose the ability to make the tradeoffs. The problems are primary and solutions secondary. Not the other way around.

I’m excited to be joining my friend Jon Harmer, Lead Product Manager at Google, in teaching Product Mangers how to drive business impact. In our cohort-based class on Dec 2-3, 2023, with live instruction and hands-on work, we will teach the critical practices to being effective at meeting customer needs in a way which drive business outcomes.

In the class, we will teach you how to use journey maps as product managers to form actionable insights on customer needs; how to use impact maps to drive business outcomes through what you build; and how to discover the greatest sources of jeopardy in your plan so you can derisk the plan in parallel with executing the plan.

Please join us on Dec 2-3, 2023!

Product strategy manifests as a collection of bets, investment decisions to do something or not, to do things now or later. A business case requires you to compare the predicted costs with the expected benefits. Your problem statement must articulate the expected benefit in economic terms to support your decision to place the bet.

This article is the second in a three-part deeper-dive on the importance of using economic measures when writing the problem statements which are the key unit of shaping and operationalizing a product strategy.

1. Undermining Your Ability to Prioritize Your Portfolio
2. Stunting Collaboration which Undermines Your Effectiveness
3. Mismatching Your Scope of Effort with Your Level of Ambition (this article)

Answer if You Can (You Cannot)

In the first part of this three-article focus on using economic measures in problem statements I introduced the challenge of choosing among three different problem statements as the one to pursue first. Without an economic framing of the problem, the decision can only be made leveraging intuition – and the decision is unlikely to be one where intuition is a good approach. Whichever problem you choose to solve first you are choosing because it “sounds good.”

Here’s what the process looks like at a high level once the decision has been made.

1. Decide to solve a problem
2. Design a solution to the problem
3. Estimate the cost of solving the problem
4. Decide if that amount of investment is a good decision

Step 4 is where you’re in trouble again. Because you don’t have the information you need to make the decision well. Consider this example problem statement without an economic measure.

The Problem of… Our call center agents are unsupported in solving the problems customers ask them to solve
Affects Whom… Call center agents
The Impact of Which is… Our employee turnover rate is 40% per year.
The Benefits of a Solution are… Our call center employee turnover rate drops to 20% per year.

The benefit is clear, but not in economic terms. What is it worth to your company to cut call center agent turnover in half? You have to do some analysis. Far too many companies will say “yes” without having done the analysis. Steven Haines has been helping us all get better at product management for decades. The surprising lack of business acumen in organizations (and among product managers) inspired him to found the Business Acumen Institute specifically targeting this fundamental gap. The absence of financial reasoning within and about problem statements is one manifestation of this. Making good business decisions is making good decisions.

Deciding what to do is different than deciding how much to spend to do it. Using economic measures helps you decide what to do. Deciding how much you’re willing to invest without determining how much you expect to benefit is the problem. This is a bad decision process.

What Will It Cost?

Here’s how this plays out when you don’t understand the potential return which comes from solving a particular problem. You start with something which sounds good, like reducing agent turnover. You come up with an idea about what you need to do to achieve this (updated software, new training, higher pay, better conditions….). You can establish great clarity about a potential cost. Set aside for a moment that your cost estimate will be at least as wrong as your initial solution approach is wrong. The decision to place a bet is based upon what you believe at the time of making the decision.

The leader or product manager who has picked a problem to solve because it sounds good now asks “what will it cost?” Whatever the number is, the only other input into your decision-making is the budget. How much is available to be spent? Does this feel proportionally like a good use of some portion of the available budget? There’s no way to evaluate if it is “worth it.” This process is only a budget-allocation process, not an investment-evaluation process. With no mechanism for defending the budget from cuts, no frame for arguing to increase the budget. All of the decision-making is limited to a intuitive allocation of resources, in an environment where intuition is demonstrated to be an unreliable predictor of success.

This frame plays out as “how should we (best) spend what we have?” and I’ve seen it in everything from the annual budgeting process at companies like Dell to PI planning events in companies focusing on their operations ahead of the reason for their operations. “How best to fill the program increment” is a delegated, smaller version of annual planning – where the challenges of political reality must also be addressed. Too often, the leadership heuristic is “make sure everyone is busy” instead of “make sure we work on the right things to advance our strategy.”

What Are You Willing to Spend?

The earlier description of a four-step process is naive, it implies both linearity and sequence. That process description treats variables as if they were constants.

Rewriting the process like this provides a hint:

1. Decide to solve a problem
2. Design a solution to the problem
3. Estimate the cost of solving the problem with the design approach from (2)
4. Decide if that amount of investment is a good decision

Step 4 now becomes a bit more nuanced. Instead of a decision to say yes or no, you are faced with a decision of yes, or no, or not with that approach. You can loop back to step 2 to try and develop a more cost-effective approach which still solves the problem. Back to the drawing board. The first idea is unlikely to be the right one, and too often the urgency pressures on teams require them to pursue the first idea regardless, because there is too much to do.

How do you decide in step 4 if you should loop back? At a bare minimum, you should not be willing to spend more than you believe the benefit to be. The expected cost is an economic measure. You cannot determine if a reduction of turnover to 20% – even quantified – is worth $100K. You cannot determine if a 20% reduction of turnover is worth 50% of your available capacity for the quarter.

This should be enough to decide to estimate – even with uselessly broad estimate ranges – the economic benefit which you believe will come from solving a problem.

Conclusion

There is much more to making a good investment decision than simply assuring you have a positive return on investment (ROI). There are multiple improvements which can be made – all of which require you to use economic measures to express your beliefs.

That’s fuel for another fire. Describe your benefits, as well as the scale of the problems you might solve with economic measures. It is a step in the right direction.

Your process can prevent collaboration. The language you use in your problem statements can stop collaboration too. When you use proxy variables instead of economic measures of outcome you prevent your teams from collaborating and you reduce the likelihood of achieving product success.

This article is the second in a three-part deeper-dive on the importance of using economic measures when writing the problem statements which are the key unit of shaping and operationalizing a product strategy.

1. Undermining Your Ability to Prioritize Your Portfolio
2. Stunting Collaboration which Undermines Your Effectiveness (this article)
3. Mismatching Your Scope of Effort with Your Level of Ambition

Collaboration, Commitment, and Consequences

To imagine and execute a successful product you have to identify and deliver the right things to build to realize your desired outcome. You create outputs in hopes of creating outcomes. You control the outputs you choose to create but you only influence the outcomes you achieve. You will develop two hypotheses to stitch the red thread from outputs to outcomes – the solution hypothesis and the outcome hypothesis.

The solution hypothesis is the one closest to the outputs and the easiest one for teams to start thinking through, using a couple simple questions. Start with one output you plan to build. This could be a new or improved capability, product feature, product, service, marketing copy, imagery, etc. Any THING you invest to create or improve. The first question is “Why do we need this THING?” The reason is consistently to address a particular need. The NEED, or proximate problem, is fairly self-evident. “We need to update the database to track product pairings because without the change we don’t know which products are sold together.” “We need to add a safety switch to the treadmill so that it stops immediately if someone falls.”

So far this is only an assumption, you need more to develop it into a disprovable hypothesis. This is done through the next question – “How will you know if it worked?” Or more formally, what do you expect to observe which indicates you’ve sufficiently addressed the NEED? There is an OBSERVABLE CHANGE which you would expect to see as a result of addressing the NEED by correctly building the THING. This is your solution hypothesis.

This OBSERVABLE CHANGE is not, however, an outcome. You still have to make another jump. It is not an outcome because it is not an economic benefit to the company. The change is a consequence of having addressed a need by building a thing. The OBSERVABLE CHANGE is a leading indicator of an OUTCOME – there is a second assumption – achieving this particular change will lead to the desired outcome. This is your outcome hypothesis.

You must treat these as hypotheses, because there is uncertainty within each assumption. Will the THING you build sufficiently address the NEED? Are you addressing the right NEED to the right degree to achieve the OBSERVABLE CHANGE? Will causing or enabling that OBSERVABLE CHANGE lead to the desired OUTCOME?

Language Anchors

With this model there are three key conceptual attractors for collaboration

1. The THINGS you build
2. The OBSERVABLE CHANGES you strive to create
3. The OUTCOMES you hope to realize.

In James Gleick’s The Information, I was introduced to a key concept – the languages people use to communicate influences the nature of their thinking. The expressiveness of a language and the concepts within the language both influence what you can think. I believe this extends to organizations and how communication and collaboration happen. Every organization I’ve worked with has a process focusing operationally on one of these three attractors; THINGS, OBSERVABLE CHANGES, or OUTCOMES. This choice of focus starts for people with the language in problem statements. Teams stick with this choice as they operationalize the creation of products. The language which teams use determines everything; the concepts of success and effectiveness, the definitions of done, and the sense of purpose people embrace.

The three different operational patterns

• The Builders of Things pattern – when your team fixates on the THINGS they have decided to build.
• The Makers of Change pattern – when your team focuses on the OBSERVABLE CHANGES as the driver of choice of which THINGS to build.
• The Creators of Value pattern – when your team aligns their purpose to the desired OUTCOME which they hope to achieve through creating OBSERVABLE CHANGES.

It is important in any system to establish clarity of purpose for your teams. You can establish clarity of purpose using any of these three languages – one choice is bad, one is not good enough, and one is good. Operating as Builders of Things is a problem for a number of reasons – I want to focus right now on the reason why the Makers of Change pattern is problematic.

Constraints

The concept you use to describe your purpose (the outcome, the change, or the things) is the concept you use to collaborate and ideate. Builders of things are given a list of things to do. Presence (or absence) on the list is effectively fixed. Teams are not equipped, not empowered, and not expected to question the scope of work they have been given. The benefit to the teams is in their lack of accountability for the consequences of what they built. The hidden cost for these teams is that they are unable to influence the effectiveness of their products for their customers or their employer. They are uninvolved. The process drives execution against the list and erects barriers to improving the list.

Collaboration is constrained to discussing the best way to do the things, from planning to designing to delivering. The team’s talents go untapped.

While the OBSERVABLE CHANGE is the expected result of the solution hypothesis, it is the only the input to the outcome hypothesis. The OBSERVABLE CHANGE is a leading indicator of the desired outcome, the outcome is not assured. In the Makers of Change pattern, teams have advance beyond the list of outputs and fully embrace the perspective of operating within a solution hypothesis. This shift from what they can control to what they can only influence (the targeted changes) is a great one. Engineers make things because they want to help people and solve problems. The solution hypothesis encapsulates the proximate problem or need which must be addressed to cause change.

When you use proxy variables in your problem statement, you are using the language of change but not the language of outcomes. The language of change empowers your team to explore solution approaches to deliver on the solution hypothesis. The team is unfortunately not invited to participate in the decision about which changes are required.

Barry Staw first explored the cognitive bias known as Commitment Bias where people struggle to deviate from previous commitments even when they know the commitments should be changed. With a problem statement using the language of change, your frame for commitment is to achieve the change. Even if you get smarter along the way and realize either more or less – or an entirely different – change would be better. If you start with an effort to achieve a higher conversion rate, and later realize you should be trying to create a larger average transaction amount instead, the commitment bias makes it hard to make that change.

Where I’ve seen this happen in organizations is where product people act as order-givers, directing the teams towards creating changes in a “build it and they will come” type pattern. There can be cross-functional collaboration about different approaches to achieving a desired change, but again – no collaboration to refine the thinking on desired changes. Without making space to consider the possibility that a different change is needed, the system operationally will not explore the possibility.

The underlying concept is simple – at the start of a project, you don’t know what the right approach to achieving success might be. This is why you should develop hypotheses. This is important not only for describing your understanding of cause and effect, but to make it possible to question that underlying causality. Without questioning, there is no possibility of learning. Without learning, there is no possibility of course correcting. Without course correcting, there is no possibility of improving the odds of succeeding.

Conclusion

The ability to collaborate on the approach to achieving the desired change of the solution hypothesis is necessary to improve the odds of success. The ability to collaborate on the approach to achieving the desired outcome of the outcome hypothesis is necessary to further improve the likelihood of success. When you are anchored in a language which biases you away from exploring either hypothesis you are limiting your ability to deliver a successful product.

The language which permeates operations starts with the problem statement. You need to use economic measures to describe the problems and benefits which come from addressing them in order to propagate the orientation throughout the organization. This is what best positions you for product success.

When deciding what to measure, we often choose metrics which sound good or metrics which are easy. These mistakes can make a product strategy incoherent, excessively expensive, and ineffective. How we talk about what we choose to do sets our teams up for success. Or failure.

Unpacking Three Ideas

There are two broad categories of measures – economic measures and proxies for economic measures. We operate with proxy variables all the time, sometimes for good reason and sometimes with hidden consequences. Leading indicators are proxy variables which provide rapid feedback when economic measures are delayed. Vanity metrics are the proxy variables which make you feel good, in spite of their questionable connection to economic outcomes.

Three key areas where incorrectly using proxy variables in your problem statements are worth exploring in depth. The measure you use to describe the impact of a problem and the potential benefit of solving it matters. I’ll review each of the three areas, and then dive into the first one in this article, leaving the others for follow-up articles. The right answer is to use a measure of economic benefit to describe the problem. The following dives into the reasons why this matters.

The final two sections in the problem statement template, The Impact of Which is…, and The Benefits of a Solution are… must use the same measure – a common denomination. Even if you cannot shift to an economic measure, the two fields must at least use the same measure. This is important to support the process of deciding how much of the problem you want to address, or how much of the problem you want to address right now. When developing a product strategy, we are not only activating our teams to go “in that direction” but also establishing “how far to go.” When they don’t share a common denomination, the problem statement is incoherent.

**Three issues you introduce when you focus on proxy variables instead of economic value when describing problems:

1. You constrain the product strategy shaping activities to be purely based on intuition, in a situation where this approach is expected to be ineffective. By asking leaders to make choices among problems to be solved without a common denomination for comparison you undermine their ability to reason when making the decision.
2. You hamper collaboration on achieving outcomes. When teams are disconnected from the development of outcome hypotheses, they are disempowered. By scoping their objectives to achieving a change in a proxy variable, you remove their ability to contribute to assuring their work will be consequential.
3. You eliminate your ability to make rational choices about how much to invest to solve a particular problem. It is inappropriate to spend $1,000 to solve a $20 problem; there is a necessary level of business acumen necessary to make these decisions responsibly.

Value Seeking

In a recent webinar on developing hypotheses and measurement, I taught folks how to use the mechanisms of value-realization to define their approach to observation. The theme of my talk was measure what matters and the feedback from attendees was positive. When you’re writing problem statements, you are describing today and you are communicating a vision for how you will find value in a better tomorrow. Here we need to shift to an earlier stage of the process, not talking about how to discern value in what you’ve built, but rather how to think about value before you decide what to build.

When you describe your understanding of value, there is a best practice and there are consequences for doing it other ways.

A major element of prioritization is making comparisons – should you solve one problem ahead of another or instead of another? The problems you are considering solving will all look very different. You’ll have exploitable market opportunities, internal capability shortcomings, product improvements to name a few. Each problem could be described in terms which make sense within those local contexts (grow share of wallet, shorten a cycle time, increase usage of a new feature). Using those terms prevents you from making comparisons and runs the very real risk of misdirecting you towards creating something which doesn’t matter.

The local context of executing on something can be effectively influenced by measuring a vanity metric, and you may never know that it led you astray. The best way to douse a forest fire is to prevent it from starting. The best time to stop wasting time and energy build the wrong things is before you get started, otherwise you’re faced with the challenge of un-ringing the bell which originally placed you on a bad path.

When you start describing problems in terms of proxy variables is where you run into your first issue. Proxy variables exist because they are easy to measure, at the expense of being impossible to compare. By displacing economic measures with proxy variables, you eliminate a shared framework for making decisions.

The lack of an economic framework does not prevent people from making economic decisions. It simply causes these decisions to be driven underground and to be based on intuition instead of analysis.

Donald Reinertsen, The Principles of Product Development Flow

The Apples and Oranges of Proxy Variables

Imagine your product is a mobile app which helps people to reserve parking spots ahead of time near entertainment venues. This allows people to avoid allocating extra time before an event to try and find parking, frees up congestion on the streets near venues, and helps people to better enjoy their experience. You build the app, set up partnerships with parking space providers, and train up people in your call center to field support calls for users of the app / service.

Your team comes to you with an update – things are not going well, and there are three key problems which have been identified. You only have enough budget to address one of them. Or your team only has the available capacity to solve one of them. How do you choose which of the three situations to address?

Situation A: Only 2% of people are still using our app after their first experience.
Situation B: We spend an average of 20 minutes per support call.
Situation C: Our call center employee turnover rate is 40% per year.

These all represent genuine problems worth solving, yet you cannot rationally choose to prioritize one of them ahead of the others. You can only apply intuition. Based on your past experiences, one of those situations may sound worse than the others. If you elaborated each of these situations into problems using the problem statement template, but retaining the proxy variables, it becomes easier to use intuition, but still no easier to make a rational choice about which problem to solve.

The Problem of… 98% of new customers stop using our app because they aren’t able to find and reserve good parking spaces
Affects Whom… Drivers looking for parking solutions
The Impact of Which is… Only 2% of people are still using our app after their second experience
The Benefits of a Solution are… We could increase our total active user population by 10x

OK, great – now you have enough context to inform an intuitive decision – because you don’t actually help with the job to be done, people to drop your product. If you fix the problem, you could increase your user base dramatically. Sounds important and valuable. But you don’t have the information needed to know if you should prefer to solve this problem at the expense of not solving the other problems.

The Problem of… Our customers call us for help when the parking space they reserved is already occupied when they arrive
Affects Whom… Drivers who reserved parking through our app prior to attending an event
The Impact of Which is… We spend an average of 20 minutes per support call.
The Benefits of a Solution are… We could reduce the duration of these calls to under 2 minutes

Again – you have a sense of a real problem, reservations are not being honored and our customers call the support staff. Your customers scramble to find parking while likely risking being late for whatever event they are attending. The value proposition for your customer is in not having to allocate time to find parking before an event, because parking is there waiting for them, making for a hassle-free experience of enjoying a night out. It feels like this sort of problem needs to be addressed.

The Problem of… Our call center agents are unsupported in solving the problems customers ask them to solve
Affects Whom… Call center agents
The Impact of Which is… Our employee turnover rate is 40% per year.
The Benefits of a Solution are… Our call center employee turnover rate drops to 20% per year.

This problem looks inward – where you discover some assumptions about how to deliver a solution have proven to be wrong. When modeling the costs associated with providing good service to your customers, you did not account for the high expense of continuously training new call center employees to become genuinely helpful as well as good brand ambassadors. The cost of training the volume of agents is much higher than it should be for the size of the customer base. It feels like your business model may proved to be unsustainable.

Each of these problems could be critical to address. And each of them “sounds like” or “feels like” a big deal. My friend Paul Young, VP at Pragmatic Institute,
delivered a presentation on the product management “X-Factor” at one of the early Product Camps in Austin. Part of the X-Factor is the ability to be persuasive within your organization. A good product manager could pick any one of the three problems and convince the leaders to invest to solve it. When people describe the responsibilities of a good product manager, they often list the activities – including persuasion. Being good at product management also requires us to develop the skills and practices which enable us to avoid bias in the decisions which define and create our products.

Just because a product manager could convince leaders to invest in any of the three does not mean they should convince them. The product manager is also subject to bias. When asking product folks and leaders to compare apples and oranges like this, you are forcing them to make an intuition-only decision. It is almost impossible to avoid bias in these decisions. In Strategic decisions: When can you trust your gut?, Gary Klein and Daniel Kahneman explore the situations where it can make sense to rely on intuition in decision making. The article is noteworthy in that Klein and Kahneman generally approach decision making from ‘different camps of psychology’, and the article focuses on areas of shared understanding.

First, there needs to be a certain structure to a situation, a certain predictability that allows you to have a basis for the intuition. If a situation is very, very turbulent, we say it has low validity, and there’s no basis for intuition. For example, you shouldn’t trust the judgments of stock brokers picking individual stocks. The second factor is whether decision makers have a chance to get feedback on their judgments, so that they can strengthen them and gain expertise. If those criteria aren’t met, then intuitions aren’t going to be trustworthy.

Gary Klein, Strategic decisions: When can you trust your gut?

This example is definitely not a situation which clears the high validity hurdle. Any given problem a product manager is considering is unlikely to meet Klein’s criteria. The consequence of using intuition exclusively to make the decision is that bias will run unfettered and decision quality will be degraded.

The Solution

The solution is pretty simple, but not necessarily easy. Describe each of the problems in terms of a shared measure of economic benefit.

This is hard, because it requires you to describe your outcome hypotheses. Each of the proxy variables from above is believed to lead to a potential benefit. You should write out the hypotheses associated with problem statement, in order to reframe the problem in terms of economic benefits.

The Problem of… 98% of new customers stop using our app because they aren’t able to find and reserve good parking spaces
Affects Whom… Drivers looking for parking solutions
The Impact of Which is… Only 2% of people are still using our app after their second experience
The Benefits of a Solution are… We could increase our total active user population by 10x

What is implied when the problem statement uses proxy-variables is that building whatever needs to be built to cause 20% of users to continue to use the app, instead of 2% will result in some sort of benefit. What you need to do is describe how much benefit you expect, in order to know if this is the right problem to solve. An outcome hypothesis is the tool to use.

We believe if we increase the new customer retention rate from 2% to 20% it will result in a tenfold increase in billings in the second month. We will know we are right if we see at least a tenfold increase in receivables at the end of the second month. We will know we are wrong if see an increase of less than 1,000%.

The problem statement can be updated.

The Problem of… 98% of new customers stop using our app because they aren’t able to find and reserve good parking spaces
Affects Whom… Drivers looking for parking solutions
The Impact of Which is… 90% of potential revenue from motivated customers is lost
The Benefits of a Solution are… We could increase our revenue by 10x

With almost all of the clients I’ve helped, using revenue as the common denomination of economic value is sufficient. Revenue is not the only choice – the right choice for a given company may not be the right choice for another. You may need to describe economic impact in terms of profits. You may need to describe it in terms of billings. You may get the best insight by looking at problems in terms of per-customer profitability, because that correlates best with enterprise valuation in your unique situation.

The trick is to find something broadly-enough applicable (like revenue or profitability) to apply to both capability improvements and product or service improvements. Customer lifetime value (CLV) is a metric you often see, and this helps with comparisons associated with increasing share-of-wallet, or improving loyalty, or contrasting market segments. CLV is harder to connect to process improvements, as an example. You have to make a couple assumptions of causality – improvement of a process results in a change in customer behavior because… Whereas if you’re looking at profitability, you can more easily isolate investments which reduce your cost to serve a customer.

I’ve been asked before – what do you do when profit is not the motivation? If you are a non-profit, or a public-good company, or otherwise driven by some other purpose? Find the common denomination – which problem has the greatest impact on reducing child hunger? Which has the greatest impact on coral reef bleaching? There is a context within which you’re making decisions – find the lowest common denomination which allows you to make those decisions.

Conclusion

Use a common denomination of economic value for all of the problem statements you use to shape a product strategy.

Estimating with ranges requires a level of transparency which may be uncomfortable because you are acknowledging what you don’t know. Doing this, however, cascades into multiple positive consequences. This is also a necessary component of outcome orientation.

Using Problem Statements to Make Choices

I talk about using problem statements to “shape” a product strategy. It occurred to me that if you aren’t already doing this, then my assertion may not make any sense. I want to take a step back and explain. Using problem statements is an operational approach to shaping and expressing a product strategy which makes it actionable. When developing a product strategy you have to do a couple things – you have to both develop a strategy designed to support the company’s strategy, and you have to express it an a way which makes it actionable. I’ve found problem statements to be an artifact which can be used to both support the thinking process and meet the direction-setting needs.

I landed on this approach as a consequence of seeing so many organizations who expressed a product strategy, completely decoupled from the work they do on their products. I had a client who would refer to the “red thread” connecting her intent with the ideation, decomposition, and execution of work by the teams in her organization. As a leader, she could express intent and rationale, and her teams could trace their efforts back to her purpose. The red thread. Most teams lack this.

Teresa Torres shared an article in 2016 about the opportunity solution tree, a technique she uses to weave the red thread from desired outcomes to solution ideas and the experiments you run to evaluate them. Continuous Discovery Habits was released in 2021 and lives on my short-list of books I recommend to product folks.

It’s working so well that I feel compelled to write a book about it. But that’s going to take time and I want you to have it today.

Teresa Torres, 2016 Article, Why This Opportunity Solution Tree is Changing the Way Product Teams Work

In her article, she identifies four gaps in systematic thinking we have to address to effectively connect that red thread from strategy to execution. The first gap Teresa identified is that we don’t examine our ideas before investing in them. The problem statement is an artifact you can use to wrangle, examine, and improve the ideas before deciding to invest in them. The cascade of actions associated with “what’s next?” can come from the problem statement.

Before we climb up the opportunity-solution tree, we have to decide if we are going to pursue this opportunity in the first place. This is where the shaping begins. The problem statement is evaluating the opportunity (I’ll switch here into “problem to solve” language vs. “opportunity to pursue” language for the rest of the article to help with readability) in the context of a number of possible pursuits.

In the previous article I touched on the importance of using a range to express your estimation instead of asserting with a discrete value. I poked at comparison as one of the decision-making patterns, and showed how the use of a range can inform the choice between an aggressive posture and a conservative footing. With the acknowledgement of the range of possible values, you can make a better, more elegant decision.

Fishing to Win

When I was a kid, my dad liked to go bass fishing and was a member of a fishing club. Sometimes on the weekends, we would go to a lake and “compete” with the other members of the club to see who could catch the most fish on a particular day. The prize for the competition was really just bragging rights within the club, and the purpose was to enjoy a day of fishing. Nonetheless, at 5 am or whatever unreasonably early hour the competition started, everyone was in their boat at the same starting point, and they all raced off to different parts of the lake to get to each person’s favorite secret fishing spot.

When I asked my dad why we spent 20 minutes racing to some arbitrary spot instead of leisurely driving the boat to the location, he explained that there was urgency – the fish were more likely to bite (and then get caught by him) early in the morning. It was a race against the clock. Kids can be annoying early in the morning, and I was no exception. I understood the urgency – but now I wanted to know if time was so critical, why we didn’t just start fishing immediately right at the starting point; everyone else was wasting time getting to their favorite secret cove, we could get a 20 minute head-start.

Dad explained the fish were unlikely to be at the starting point – the conditions were wrong. Too much direct sunlight, all of the commotion from all of the boats, the wrong depth of water, not enough underwater debris and hidey-holes for fish. I wasn’t thinking about any of those conditions – the lake was one big lake, the fish could be anywhere (TAM). Dad had a spot in mind where all of the conditions he considered made it more likely for fish to be present (SAM). If we could get there quickly enough, they would be still be active (SOM). Our best chances for winning the competition were to be where we had the best chances of catching fish.

You fish where the fish are.

You use the impact section of the problem statement to describe your understanding of where the fish are.

Reluctance When You Have No Idea

I had no idea how many fish might be in the lake – thousands? Millions? I still have no idea. ChatGPT estimates between 100,000 and 300,000 fish which met the rules of the competition could be in the lake. No one likes to be wrong. Even a large-language model AI is uncomfortable giving estimates – ChatGPT generated a 200-word-long caveat about the uncertainty in it’s estimate. Fine. We can work with this.

Every time – literally hundreds of times – when I help someone draft a problem statement to describe the work they are doing, they create a qualitative description of the problem.

The Problem of… We lose customers because our slow approval of claims causes delayed reimbursements
Affects Whom… Customers filing out-of-network service claims
The Impact of Which is… Our customers find other insurers because they struggle financially due to our delayed reimbursements
The Benefits of a Solution are… Higher retention rates of our insured because of improved service experience

Once you get to a well-formed, but only qualitative description of the problem like this example, you transition into trying to quantify it. And it is consistently a struggle. There is often some quantification which is available, to help shape the conversation, but not often the right information. In this example, someone may know that reimbursements on average take 120 days. Which is obviously not good. If a patient is floating the cost of weekly rehab sessions, the size of outstanding payments due could really become significant. This is a great example of the curse of knowledge problem. You know objectively 120 days is a problem, you know there is value in solving it. But you don’t know enough yet to commit to solving it, or even to design a solution.

You can start to fill in some quantification with a couple easy pieces of research. What is the churn rate? 80% of customers renew during open enrollment and 20% leave. You know (in this example) that 10% leave because of circumstances beyond your control (people relocate outside of our network, take new jobs which have different insurance plans, etc.). So the hypothesis becomes that no more than 10% of all customers leave because of delayed reimbursements. You also know that only 10% of customers file out of network service claims within a given year.

Now you can rewrite the problem statement, with quantification. Something people are reluctant to do.

The Problem of… We lose customers because our slow approval of claims causes delayed reimbursements
Affects Whom… The 10% of customers filing out-of-network service claims
The Impact of Which is… 10% of our customers find other insurers because they struggle financially due to our delayed reimbursements
The Benefits of a Solution are… Higher retention rates of our insured because of improved service experience

Now you’re making an assertion – 10% of all customers who leave do it because of the delayed reimbursements. This was actually the upper-bound of possibility. 10% of (all) customers leave every year, 10% of (all) customers experience delayed reimbursements. What I’ve seen consistently is that people are uncomfortable with documenting the (obviously incorrect) discrete value, so they just simply don’t do any quantification at all.

With this reluctance, product folks often aren’t encouraged (or required) to ask “what is the range, because I know 10% is wrong?” In the companies where they work they don’t have to. This is one of those places where the system – the process in place, the people around the product person, the policies and leaders – undermines good product work. No one requests or expects quantification. A bad process biases teams towards bad practices which drive bad decisions which result in bad products. When you invest to fix the process to support good practices, you also have to learn and perform those practices.

As an individual product person you can start doing things better right now, and your corner of the company will start improving. When you don’t know, what should you do?

What to Do When You Have No Idea

Once you apply probabilistic thinking and describe a range of possible values, you have a problem statement which looks like the following:

The Problem of… We lose customers because our slow approval of claims causes delayed reimbursements
Affects Whom… The 10% of customers filing out-of-network service claims
The Impact of Which is… Between 0% and 10% of our customers find other insurers because they struggle financially due to our delayed reimbursements
The Benefits of a Solution are… We will reduce abandonment rates of our insured by between 0% and 100% because of improved service experience.

Note: If all of the people who leave are leaving because of this problem, then completely solving it would result in no one ever leaving.

This is an exciting and productive way to say “I have no idea how many people left because of delayed reimbursements.” This is exciting, first in comparison with the alternative, and second because it helps you see what to do next.

The alternative is to just start building something which reduces the size of the problem, without actually knowing the size of the problem. Should you shorten the period of delay by speeding up the current process? If so, how much shorter does it need to be? Should you pull more service providers into your network so that fewer people file claims? Should you establish some maximum-reimbursement-due value, where reaching the threshold triggers a check disbursement?

This is what organizations normally do. They start with a poorly-formed surface-level description of the situation and run full speed towards doing something. Being busy feels better than not. How much should you be willing to spend to do whatever you picked? Just use up the available budget. How do you declare success? Launching what you built. That’s it. Without a definition of what it means to solve the problem, you can pat yourself on the back for having done something. It doesn’t matter if it wasn’t “good enough” because “good enough” was not defined.

Contrast that approach with a better, higher quality, decision-making process – where you are faced with the questions:

• Should we solve this problem?
• Is this problem large enough to deserve our attention?
• Is this problem larger than the other problems we might choose to solve instead?
• Is the potential benefit of solving this problem high enough to justify the expense of solving it?

The Uselessly Wide Range

The current problem statement describes a range of possible benefits from no benefit to effectively infinite (because retention becomes 100%). What I find helps is to treat the problem statement as describing your current beliefs. Because you (so far) lack the information to refine your beliefs, you acknowledge that you have no idea. I describe this as a “uselessly wide range.” And that’s good. It is healthy for the system because you are being transparent and explicit about what you believe. You are acknowledging that you lack the information to make this decision well.

You don’t know how to answer any of the “should we do it?” questions, because you don’t have enough information. If the benefit is 0% – a possibility identified in your problem statement – then definitely no, don’t do it. If the benefit is 100% – then definitely yes, you should do it, do it now, and move heaven and earth to do it.

The first question you need to answer is “should we do it?” and as a binary-decision, you will have a go/no-go line in the sand. There’s some value, somewhere between 0% and 10% where the problem is big enough that you should commit to solving it. You’re placing a bet, based on your belief that the value is actually above your line. But right now, the actual value is just as likely to be below your line as above it.

You’re making a decision where the likelihood of being wrong is equal to the likelihood of being right.

Imagine, given other constraints, that it would be a good decision to solve this problem if and only if you could reduce abandonment rates by at least 50%. What this means in this example is that for the 10% of all customers who leave every year, half of them – 5% of all customers – will have to choose to stay instead. Your estimate expresses what you believe. OK, you’ve got some clarity of purpose – if you can cut abandonment rates in half by addressing this delayed reimbursement problem, it is worth doing.

Your current belief is that it could go either way. You need to do something to update your beliefs, to improve your ability to make the decision.

Invest to Remove Uncertainty

My second reason for excitement is that by identifying the reason why you cannot make the decision well, you identify the steps to take to improve your ability to make the decision.

If the outcome of a decision in question is highly uncertain and has significant consequences, then measurements that reduce uncertainty about it have a high value.

Doug Hubbard, How to Measure Anything: Finding the Value of Intangibles

A major improvement in my personal thinking about this stuff came when reviewing some of the work Hubbard delivered for a shared client – once you know the value of making the right decision versus the wrong decision, you know the value of doing something to get smarter. If you can update your beliefs so that you believe the truth is more likely to be on one side of your go/no-go line than it is to be on the other side, you’ve created value. This is the cost benefit of measurement and experimentation during the decision-making process.

You can think of this like stacking the odds in your favor. By learning more, you update your beliefs – and therefore you improve your ability to make the decision. You aren’t changing the situation, but you are changing your understanding of the situation – you are reducing uncertainty.

You have data about how many people leave, and you have independent data about how many people experienced reimbursement delays. 10% in each case. And you know that you are willing to invest to solve this problem if you believe at least half of the people who left did it because of the delayed reimbursements. You could do something as simple as surveying a sample of people who left.

The question you ask yourself is “what should I expect to find if at least half of the people who chose to leave (versus those who left because of uncontrollable factors) did so because of delayed reimbursements?” This is the input to your survey-design. If your belief is correct, at least 25% of respondents would cite this as the reason. If your survey has a +/- 5% confidence interval, then you can refine your thinking.

The result you would expect would be ‘anything greater than 25% of survey respondents, among all those who left’ – half of a half. Your survey may have a +/5% confidence interval, so might approach things by forming the hypothesis:

• We believe at least 25% of people leave during open enrollment because of delayed reimbursements for service claims. We will know we are right if we see at least 30% indicating this when asked. We will know we are wrong if no more than 20% respond accordingly.

What this hypothesis tells you is just enough to know if it is a good idea to consider solving this problem. Imagine the result of your survey is that 5% of respondents indicate the reimbursement-delay problem as being the reason they left. You now update your beliefs based on the new-to-you information. Your problem statement would look like the following:

The Problem of… We lose customers because our slow approval of claims causes delayed reimbursements
Affects Whom… The 10% of customers filing out-of-network service claims
The Impact of Which is… Between 0% and 1% of our customers find other insurers because they struggle financially due to our delayed reimbursements
The Benefits of a Solution are… We will reduce abandonment rates of our insured by between 0% and 10% because of improved service experience.

You now know (you believe) that at most 1% of your customers are leaving because of the reimbursement delays. In this example, what we’ve stipulated is that this is not a big enough problem to invest to solve. What you might be thinking is that we make those investment decisions based on cost-benefit (among other things), and none of this article talks about benefit at all. And that’s correct.

The impact you identify is a description of the magnitude of the problem – and it represents an upper-bound of potential benefit. No possible solution can actually exceed in value the scale of the problem it is trying to address. There’s a lot more nuance in diving into the solution piece – you have to introduce two key concepts, first around solvability of the problem, and the second around incremental efforts to progressively reduce the size of the problem.

Almost always, the right investment decision is to reduce the impact of a problem, not eliminate it. Every investment to make things better comes at the opportunity cost of not making some other investment to make things better in some other way. What is particularly useful about the impact section of the problem statement in practice is you can identify the upper-bound of realizable value from solving the problem, and make better decisions about which problems to pursue.

Conclusion

Don’t let high degrees of uncertainty about the nature of a problem dissuade you from describing what you do understand and believe. Use that description – and the power of quantification – to identify when there is value in learning before making a decision. This is the kind of shift which markedly improves your product operations by improving your decisions about what work to put into the system. It requires your organization to be willing to be transparent about knowledge and open to learning when learning has value. This is part of what it means to be outcome oriented, as well – if you’re not doing these things your process is probably still output-oriented, you are operating a feature-factory.

Product management is fundamentally a discipline of decision-making. Which investments to make, which problems to solve, which customers to serve, etc. The approach we take to decisions is fraught with peril, and we benefit from removing unconscious biases – improving our ability to elegantly make decisions to improve and advance our products.

Thinking in Absolutes

n the previous article about developing economic framing through the impact section of a problem statement, I introduced the following example problem statement.

The Problem of… We cannot market to the 20% of repeat customers abandon the new product registration process
Affects Whom… 200,000 existing customers making additional purchases each year
The Impact of Which is… We lose $400K – $500K in incremental attached-services sales per year
The Benefits of a Solution are… We increase revenue by at least $400K per year in incremental attached-services sales

In order to keep the topic focused on articulating economic outcomes, I avoided the topic of discussing how to describe the impact. Usually, people make the mistake of estimating a number instead of a range. Commonly, the impact section would read as follows:

The Impact of Which is… We lose $450K in incremental attached-services sales per year

Estimation is necessary, but estimating with absolute number has consequences which affect both your current thinking and the pursuant collaboration. The number generates a false sense of precision and misleads everyone who uses the problem statement to inform future decisions. Product management, and problem framing in general, happens in an environment of uncertainty. The only thing certain about creating an absolute estimate is that you can be certain that your estimate is wrong.

A False Sense of Certainty

When people read a single number, they read it with an unwarranted sense of certainty. The single number does not provide enough information and introduces ambiguity in pursuit of clarity. In this example – is the impact expected to be $450K +/- $5K? +/- $50K, or +/- $500K? The individual value does not shape the conversation correctly – making it impossible to prioritize effectively.

Even intellectually appreciating that the single number is meant to represents a range of possible values, it cannot convey enough information about that range. Is the single value meant to be a maximum impact? A minimum? The mean, median, or mode impact of all the expected values? Without understanding something more the spread of possible values, you will struggle to make good prioritization decisions.

Consider two problems you are considering solving – where they have identical estimated costs and estimated opportunity costs, everything is effectively same, the only difference is in the estimated impact.

Impact A: $450K
Impact B: $500K

With the limited information presented, you would choose to invest to solve problem B – because you expect your investment to be more impactful.

In 2010 when Dr. Jose Briones introduced me to the concept of probabilistic decision analysis in the context of estimation and forecasting, a seed of an idea was planted which would lie dormant until I combined it with the work of Doug Hubbard while I was helping a mutual client. Doug wrote How to Measure Anything, where he really leans into the science of decision-making, the value of information, and again – using probability distributions to inform decision making in an uncertain environment. Those two moments, for me, established a more elegant view of estimation than the underpowered “expected value.”

Learning how to quantify your current uncertainty about any unknown quantity is an important step in determining how to measure something in a way that is relevant to your needs.

Doug Hubbard, How to Measure Anything

When you compare two ranges – you may make a different decision between them than you would when comparing two discrete values. This is subtle, but very important. You already know that the single value actually represents a range – but viewing the range as if it were a value will bias you towards a choice which eliminates your ability to choose.

The essence of shaping your product comes from choosing which problems to solve, which problems to solve first, and the degree to which you will aspire to solve those problems. You want to eliminate the biases which affect your ability to decide.

When comparing ranges, you want to have a sense for the shape of the distribution – is it uniform (all values are equally likely), or skewed to the left of right (very high or very low values within the range are more likely than the opposite), or a distribution like the normal distribution (values in the middle are more likely than values at the extremes). When your decision-making is nuanced enough to require you to know the shape, but you don’t know the shape yet, research or experiment to better improve your beliefs. In practice, I almost never see product-teams doing this.

A simplified approach is to use the same thinking which “won the battle” in the science of estimating work-effort. Coopt the simplicity of a PERT analysis. Ideally, like a true PERT estimate, you will identify upper and lower bounds for your range using the following mental model: You believe there is no more than a 5% chance the actual number is lower than your lower bound, and no more than a 5% chance the actual number is above your upper bound. You are expressing that you believe the likelihood of the actual value being between your upper and lower numbers is 90%.

Consider how you make approach the decision when presented with “the same information” using a value versus a range.

Impact A: $450K
Impact B: $500K

The Impact of Which is… We lose $450K in incremental attached-services sales per year
The Impact of Which is… We lose $500K in incremental attached-services sales per year

When all else is equal, the clear answer is to try and make things better for Impact B.

Impact A: Between $350K and $550K
Impact B: Between $495K and $505K

The Impact of Which is… We lose between $350K and $550K in incremental attached-services sales per year
The Impact of Which is… We lose between $495K and $505K in incremental attached-services sales per year

Now, with ranges, you are presented with a different choice – one which at a minimum allows you to incorporate the characteristics of your risk tolerance into your decision. Do you pursue the problem which is assured to have the highest possible minimum impact (Impact B), or the one which has the highest possible maximum impact (Impact A)? In game theory, these are known as maximin and maximax strategies. John von Neumann referred to these as the conservative and aggressive strategies when he described them almost a hundred years ago (1928).

You may still choose option B, of course – but now you are making a different decision entirely. Much more elegant than “big number better.”

The weird thing about decision-making – even if you make the same choice (Impact B in both cases), the second decision is actually a better, higher quality, decision than the first one.

Conclusion

You undermine your ability to make good prioritization decisions when you compare estimated values and ignore that those estimates really represent ranges. We operate in an uncertain environment – not only are many things rapidly changing, but our degree of understanding of those changing things is changing. Articulating your estimates as ranges will help you make better, and less-biased decisions.

When you first start to describe a problem, you may skip the notion of quantifying entirely, and instead describe an abstract scenario like “repeat customers abandon the new product registration process.” This is the not nearly enough, and most teams will catch it quickly – incorporating the magnitude of the situation, either in absolute or relative terms.

The Problem of… 20% of repeat customers abandon the new product registration process
The Problem of… 200,000 repeat customers abandoned the new product registration process last year

Using relative terms is helpful, as you get a better informed intuition about the situation. When considering 200,000 customers abandoning the process, is that 1% of 20M? or 80% of 250K? Knowing that 20% of people – at whatever scale – are abandoning is a useful hint at significance. However, even adding some numbers to describe the situation is not enough. When shaping what your product will be, you are making a collection of investment decisions – prioritizing some work ahead of other work, discarding some work. Your investment decisions should be framed in terms of which problems you choose to solve. When you are describing situations, you are comparing apples and oranges. You need a lingua franca, a common language or perspective to use when making those decisions. You use the impact section of the problem statement to do that, without losing the character of the problem.

Describing the impact of the problem is where you shift from describing the situation, to assessing the importance of the situation. It is important to identify problems, not their manifestations. The situation – customers abandoning a registration process – is just that, a situation. With the curse of knowledge, you know this is bad because you know why it is bad. For you, this is self-evidently a bad thing. If you’re like me and most of the people I’ve coached, you will describe the situation and move on to the next field – yes, it is bad, but how widespread is it?

The Problem of… 20% of repeat customers abandon the new product registration process
Affects Whom… 200,000 existing customers making additional purchases each year

OK, now you’ve captured a sense of scale – a lot of people are abandoning the process. Still, you’re describing a situation, and now you’re clarifying how widespread the situation is. You still haven’t answered the question “why should we care?” To decide if you should care, your first question should be “how much is this hurting us?” and this is exactly what the impact section of the problem statement declares. This is the information we need to compare the importance of solving this problem with the importance of solving some other problem.

The Problem of… 20% of repeat customers abandon the new product registration process
Affects Whom… 200,000 existing customers making additional purchases each year
The Impact of Which is… We lose $400K – $500K in incremental attached-services revenue per year

Now we understand the magnitude of the problem, using a common language – revenue* – to compare the different things we might do. There’s a bit of an issue here, but I’ve only seen this affect decision-making at two companies – revenue is not a good comparison, because profit is what really matters, and different revenue streams come with different profit margins. Attached-services revenue dollars may come at a very high gross margin, while incremental product sales may come at a much lower margin; $500K in incremental services revenue may be more profitable than $5M in incremental product sales. More commonly, people are making these decisions within an area of the company where margins are similar enough as to be ignorable for ease of analysis. The services team and the product teams are making decisions within their own silos. Yes, this is a problem. But not one you can solve with problem statements. My suggestion – use gross profits when you are comparing choices which have revenue implications.

The economic framework also improves decision making in a more subtle way. As we try to analyze economics, we uncover those assumptions that have the greatest impact on our answers. This permits us to invest more effort on getting these assumptions correct. Thus, the economic view regeneratively improves our understanding of economics. We can create enormous improvements in decision making with surprisingly imperfect answers.

Don Reinertsen, The Principles of Product Development Flow

Problem statements using this template really can help improve both your product thinking and your communications. What you’ve described is a leap – making a connection from the situation of process abandonment to the consequence of lost revenue. You know, but aren’t sharing with anyone why you can make the leap. Selling attached services is a cross-selling activity. Some people who buy the underlying product will also purchase an attachable service. Buying Apple Care for your new iPad, paying in advance for car-servicing (tire rotation, oil changes, etc.) when buying a car, purchasing additional attachments for a tool.

All of the customers who register the product are now qualified leads in your marketing database. Your company can and will target and tailor marketing messages and promotions to those customers. Every customer who abandons the registration process is unreachable (your assumption). You lose out on the opportunity to have a conversation with that customer, and therefore cross-selling opportunities are lost. The customers who abandon the registration are no longer qualified leads because you don’t know how to contact them. The actual problem is that you cannot market to those customers.

The jump in reasoning from your description of the problem to the consequences is too far for someone who doesn’t already know what you know. Without closing that gap somewhat, you undermine your communication of purpose to the teams doing the work.

The Problem of… 20% of repeat customers abandon the new product registration process

Should instead be:

The Problem of… We cannot market to the 20% of repeat customers abandon the new product registration process
Affects Whom… 200,000 existing customers making additional purchases each year
The Impact of Which is… We lose $400K – $500K in incremental attached-services sales per year

Assumption of Causality

Between the problem section and the impact section, you now have the elements of an assumption of causality. What this problem statement is asserting is not only that we cannot market to existing customers who abandon the registration process of their new product, but that we lose money because of this. The implicit assumption is in the reverse – if we could market to those customers, we would capture incremental revenue. These aren’t unreasonable assumptions, but ultimately they do need to be tested.

The first step to testing it is to take what you’ve captured and write it out as an assumption

We assume if we prevent existing customers from abandoning the registration process, then we will cross-sell incremental services to them.

This assumption, however, is anchored in the original situation – abandonment. The following assumption is more powerful because it anchors on the problem and not the situation.

We assume if we could market to existing customers who abandon the registration process, then we will cross-sell incremental services to them.

The first assumption biases teams towards developing a solution which reduces the rate of abandonment. The second version allows the team to consider reducing the abandonment rate – but also leaves open the possibility of solving the problem in a different way. Consider a couple possible solution approaches. For the first version, update the product registration flow with a “use your existing information” checkbox. Arguably this is an improved experience with a better flow for the customer and would reduce the rate of abandonment.

The second assumption does not have the same implicit constraint. The team may consider prepopulating the interface, to reduce abandonment rates. However, the barrier is removed to make it easier to ask the question – if we can prepopulate the UI, why don’t we just automate the product registration and eliminate the flow entirely? At least for existing customers who already have information on file. Registration can be automated using the same contextual information which would have been used to prepopulate a form. Why don’t we bypass the registration pattern entirely, and simply record internally who owns what? For products with an aftermarket, different company-models should be considered.

It is this act of explicitly looking at the implicit assumption in the problem statement – “abandonment causes lost revenue” vs. “inability to market causes lost revenue” – which helps you write a craft a problem statement. It also empowers your team – your collaborators – to make more meaningful contributions to solutioning because they are not unnecessarily constrained with an overly narrow problem framing.

The Problem of… We cannot market to the 20% of repeat customers abandon the new product registration process
Affects Whom… 200,000 existing customers making additional purchases each year
The Impact of Which is… We lose $400K – $500K in incremental attached-services sales per year
The Benefits of a Solution are… We increase revenue by at least $400K per year in incremental attached-services sales

Diving into crafting a non-trivial benefits section is a topic for a future post.

Conclusion

There are two different benefits of quantifying the impact in a problem statement. First – you need a common denomination for making comparisons across problems, when choosing which ones to solve and which ones to solve first. Second – by translating your understanding of a situation into economic terms, you improve your thinking about the underlying problems, and expose the assumptions you are making.

Good product management is understanding the problems your customers want to solve, how your customers get value from solving those problems, and figuring out how best to help them. We need a little help to actually understand those problems from the customer’s point of view. Even good product managers will start with the wrong thinking, and organizations are commonly solving the wrong problems because of this.

Anchoring

Good product management is choosing the right way to solve a problem; great product management is choosing the right way to frame the opportunity. Thomas Wedell-Wedellsborg has a great example in What’s Your Problem? where he contrasts approaches to the problem of slow elevators in tall hotels.

Adding additional elevators, or upgrading the existing elevators to be faster are both prohibitively expensive solutions for an existing hotel, and often impractical when planning the design of a new hotel. Instead of framing the problem as “people wait too long for the elevator,” you can reframe the problem to “people are unhappy with waiting too long for the elevator.” By adding a distraction – in particular, a large mirror – people no longer perceive the time spent waiting as wasted; instead, people use the time to their advantage and stop complaining. These are two perfectly valid approaches to framing the opportunity. Additional elevators might be a good solution, adding mirrors in the hall by the elevator door is a great solution.

You can see the difference by considering two different problem statements

The Problem of… Guests wait too long for the elevator
Affects Whom… All guests of our hotel on upper floors
The Impact of Which is… Our guests are less unlikely to return or recommend us in the future
The Benefits of a Solution are… Our guests are more likely to return and to recommend us to their friends

The Problem of… Guests are unhappy about waiting for the elevator
Affects Whom… All guests of our hotel on upper floors
The Impact of Which is… Our guests are less unlikely to return or recommend us in the future
The Benefits of a Solution are… Our guests are more likely to return and to recommend us to their friends

How you frame the problem, how you write the problem statement, will anchor your teams in a way of thinking about the situation and thereby narrow the field of exploration for possible solutions. Tversky and Kahneman developed the idea of anchoring bias in their research – we know that how you set it up when communicating is how people will interpret it. You are biasing explicitly when sharing clarity of purpose.

Perspective Taking

If Henry Ford had been an architect, we would have a quote about hotel owners demanding faster elevators. Many leaders of product teams I’ve met also focus on their version of a faster elevator. This is another consequence of taking an inside-out view of the problem. Perspective taking is the act of putting yourself into someone else’s shoes – and a necessary step in generating an outside-in view of the problem. The absence of this is why the vast majority of user-stories are poorly written (the focus of the linked article). When considering how product ops work within organizations, when accounting for anchoring effects, it becomes critical to acknowledge you are establishing the perspective everyone will share from that point forward.

When you are developing a problem statement you have the opportunity to improve your thinking. At this point, you can retain your initial inside-out perspective, or you can shift to your customer’s perspective. Making the shift will help leaders develop an informed point of view about what to include, exclude, and prioritize for the product. When people talk about customer-centricity within organizations, this is what they (should) mean – focusing on what matters to customers, and prioritizing investments to make a meaningful difference for those customers.

Wedell-Wedellsborg proposed a two-step shift which I found personally effective. Your first step is to imagine yourself in the situation of your customer. How would you see the situation from their point of view? Because of your own anchoring, you may still see the elevator as being too slow. You’ve got the curse of knowledge, you understand how things work and how they could work. Your second step, however, is to try and imagine how your customer sees it from their perspective.

Perspective taking may improve business outcomes not only by giving us access to more information than we would have without it, but also by ramping up activity in core brain regions involved in creative problem-solving and innovation.

Michael Platt, Director of the Wharton Neuroscience Initiative, Perspective Taking: A Brain Hack That Can Help You Make Better Decisions

Another bonus highlighted in the article – the more we exercise perspective-taking, the better we become at perspective-taking.

Perspective Seeking

Adam Grant reminds us to take things a step further – to genuinely do some research.

What works is not perspective-taking but perspective-seeking: actually talking to people to gain insight into the nuances of their views. That’s what good scientists do: instead of drawing conclusions about people based on minimal clues, they test their hypotheses by striking up conversations.

Adam Grant, Think Again

There’s always a catch – we cannot afford (in time or resources) to do all the research we want to do. So we have to be selective about it. I coach people to consider a good-better-best approach to practices when shaping how they approach creating products.

Bad – an inside-out assertion about an unacceptable situation – usually a problem manifestation.
Good – an outside-in imagining about how you would experience the situation to discover the underlying problem you want to solve.
Better – anchor in imagining the context in which someone finds themselves, and imagine how they would experience it, not how you would in their place.
Best – find and talk with people who do, will, or might experience the problem you imagine them experiencing.

As you progress along the continuum from bad to best, you are incurring additional costs, while also realizing additional benefits. Pragmatically, you can decide in your context where you should stop. This applies to customer research generally (a good better best hidden inside a best), and competitive analysis, prototype-evaluation, etc. Everything which potentially de-risks your product can be considered in this way.

I’ve said to executives before – you’re making tradeoffs between the risk of being late and the risk of being wrong. This isn’t quite a binary, although it makes for a great soundbite – “how late do you think?” shares the spotlight with “how likely to be wrong do you think?”

Conclusion

When writing problem statements, assuring you have an outside-in perspective comes without substantial incremental cost. If you don’t have people in your organization with these skills, then there will be a cost associated with improving your product management capabilities (and possibly changing your operations to incorporate problem statements). But once you’ve made that investment, the incremental cost is insubstantial. You can move all the way from “bad” to “better” across all of your product ops. Then you can selectively apply research where “best” is appropriate to de-risk your investments.

There is more to identifying who’s problem you’re trying to solve – you need to also have a sense for the context in which they experience the problem. Problem statements solve for someone, and good problem statements also solve for somewhen.

The Situation

The situation your customer finds themselves in changes your customer’s goals and their definition of success at achieving those goals. Consider Amazon’s first mobile app, developed ~15 years ago. Amazon at the time made it possible for shoppers to shop for and buy most anything. Part of shopping is evaluating things they might buy – reading reviews, comparing features, etc. For the ‘everything’ store, how do you build an ‘everything’ app? “How do we enable all of our customers to do all these things on their mobile phones?”

The next question feels obvious after you ask it, but so often, people do not. “Of all those things (we know) people want to do, when do they want to do them?” I was working at another Amazon-owned company a couple years after Amazon had launched their mobile app and learned the story of the approach Amazon took with their mobile app. As told to me, this was the story of the spear-fisher. The level of focus applied to creating this app was not something I had seen before.

To get a handle on what it means to focus within the context of a shopping app where anyone can buy anything anytime, you need to dive into a model of understanding consumer behavior. “Shopping” is too broad and vague to describe it, just as “moving” would be too broad and vague to describe both crawling and running. You can think about consumers as having one of a couple different shopping intents. Someone when shopping can have a specific purchase intent or a nonspecific purchase intent. A nonspecific purchase intent could be something like “I need a new pair of shoes to wear to the upcoming formal event.” A specific purchase intent could be “I need a new pair of Jimmy Choo Didi 45s to wear for the upcoming formal event.”

When modeling someone’s purchasing decision-making process, you find that people with nonspecific purchase intent start with an understanding of the need they are trying to satisfy (new shoes for the event) and want to know what their choices might be. Those consumers will then discover what might be good choices, compare those choices, and choose one. Someone with specific purchase intent has already made this choice – and at the time of shopping is bypassing any comparison or discovery activities. They just need to complete the transaction, they’ve already made the decision.

Folks within the Amazon space at the time had personified the idea of someone with specific purchase intent as a spear-fisher. You can imagine a guy knowing he needs a new bar of soap, walking into and through the store directly to the personal care section, grabbing a package of soap (the same brand as he already uses), and proceeding to the check-out counter. Hyper-efficient shopping for the spear-fisher. The process was optimized as a transaction. Success is defined in terms of how easy it was to find the soap, and how long the entire process took. Contrast this with a different shopping experience, of a guy wandering through the store casually, becoming inspired by the skin-care products positioned at the end-caps of the aisles, reacting to in-store promotions, and imagining possible purchases for a new skin care regimen. This process might be optimized by creating inspiration, or reinforcing a positive self-image. Discovery of the perfect product becomes more important than efficiency of transaction.

This is the same person, in two totally different frame of mind. Sometimes this person is a spear-fisher.

For Amazon, the specific scenario driving the original mobile app was supporting the spear-fisher. Just as Amazon started with books, they started in the mobile space with supporting spear-fishing. When the person is looking for inspiration, they could continue to do it on the website from their computer. The description for a successful experience was described to me at the time as follows: A spear-fisher gets in line at the coffee shop, and remembers something they needed to purchase (specific buying intent). By the time the place their coffee order, they will have launched the Amazon app, found the product they wanted, and completed the purchase transaction. Optimizing for efficiency in the transaction, not discovery or decisioning about what to purchase.

This provides amazing clarity of purpose. With this articulation of a somewhen you know how to decide if particular features should be included or excluded. Product-comparison features could be excluded. Comparison is helpful to someone when they are making comparisons or discovering alternative products. Searching by product name would be more helpful than searching by product category. Searching through previous orders would be exceptionally helpful for someone making a routine repeat purchase of consumable items. Need to restock your preferred daily vitamin?

When in line in the coffee shop, Amazon targeted the specific context in which the consumer found themselves, and not simply the consumer. In the prioritization 2×2 introduced previously), we discussed the trap of trying to be all things to all people at all times.

We explored escaping the trap by focusing on solving a problem for only some of the people. In this situation with the spear-fisher, you are not isolating for “some of the people” because spear-fishing is not an identity, it is a contextually relevant condition. Spear-fishing is “some of the time.”

A good problem statement template creates space for describing this context.

The Problem of… It is hard to purchase something from Amazon while waiting in line and away from my computer
Affects Whom… All existing customers when they have specific buying intent
The Impact of Which is… People consider purchasing from brick and mortar retailers while away from their computer
The Benefits of a Solution are… An increase in share-of-wallet from Amazon customers

This problem statement draft captures the situation described above, and provides clarity of purpose to the team envisioning different ways to address the problem. This draft still suffers from many mistakes. For example, this problem statement describes a situation when it should describe the reason why being difficult is a problem. This draft also fails to quantify either the impact or the potential benefit of solving the problem, making it unusable for shaping product strategy – lacking the needed information to prioritize solving this problem relative to doing something else.

When people talk about being data-driven in their product investments, what they unfortunately are more commonly referring to is evaluating the results of investments and not proactively evaluating the (data-informed) potential direction of investment. Both are needed. I don’t have any of the data which would have shaped Amazon’s investment decision to originally build an app for spear-fishing, but a semantically improved problem statement would look like this – and these numbers are all fictional.

The Problem of… consumers don’t consider buying from Amazon when away from their computer because the mobile shopping experience is frustrating
Affects Whom… All existing customers with specific buying intent when away from their computer
The Impact of Which is… Amazon is excluded as the potential retailer in $100B of specific-purchase-intent transactions per year
The Benefits of a Solution are… An increase in revenue of $10B per year as existing customers shift ‘transactional’ purchases from brick and mortar retailers to Amazon

You can imagine with this example the improvements to product thinking which come from writing problem statements. What if the original problem statement for the ‘everything’ mobile app had been written like this:

The Problem of… consumers don’t consider buying from Amazon when away from their computer because the mobile shopping experience is frustrating
Affects Whom… All potential customers when away from their computers
The Impact of Which is… Amazon is excluded as the potential retailer in $1T of spontaneous consumerism transactions per year
The Benefits of a Solution are… An increase in revenue of $100B per year as spontaneous purchases shift from brick and mortar retailers to Amazon

The level of ambition is audacious, and magnitude of the scope of corresponding work is commensurate. All of the scope we could previously cleave from the deliverables is back on the table – account creation, product comparison, cross-selling and recommendations, bundling and upselling… What this problem statement lacks is focus. Desirable to the customer, for sure. Valuable to the business, absolutely. Feasible to execute, not so much.

Narrowing the scope of the problem to be solved to some of the people (existing Amazon customers) allows the team to immediately remove from scope any of the account-creation functionality. This allows the mobile team to avoid both the initial and ongoing costs (and delays, and opportunity costs) of building account creation functionality. Around that time, there likely would have been three teams – building natively for Apple, Android, and Symbian mobile phones. Plus the team building the web experience. Every broad area of functionality would either require a “write once, run everywhere” solution, or multiple independent implementations. At that time, the “run everywhere” experiences were noteworthy in their lack of polish and quality in comparison with native applications.

Narrowing the scope of the problem to be solved to some of the time (when people have specific purchase intent) further reduced the scale of ambition for the project. Each area of functionality ultimately competes for scarce resources, and teams are forced to make choices which compromise the business case, compromise the functionality, compromise the quality, and compromise the customer’s experience. The last three have a knock-on effect of further compromising the business case, by undermining its premise. Too much scope causes delays (first-order undermining), and cutting corners during the delay result in a degraded product (second-order undermining through invalidating the hypotheses which built the case).

The shaping of purpose within a well-structure problem statement also enables the team (and in a healthy environment encourages the team) to debate potential value props. Consider functionality associated with upselling – convincing someone to purchase a more expensive, better alternative to the product they have already selected. There are potential profit benefits associated with convincing someone to buy a “better” version of a “good” product. Amazon has had a feature presented as “people who looked at this product ultimately purchased this other product.” The potential benefit of increasing profits in this way apply when solving for the spear-fisher, just as they do in other consumer contexts.

Providing Focus

With a clear problem statement, you can decide to solve for this adjacent opportunity now and update the problem statement accordingly. You can decide to address this opportunity later with an additional problem statement. This is a healthy discussion. Clear problem statements – especially in the context of a product roadmap – help the teams understand the choices and direct their energies towards achieving the outcomes articulated. Clear problem statements provide focus.

There’s a difference between who is exposed to a situation (many people) and who experiences that situation as a meaningful problem they would like to solve (a select few). It is important to not describe a situation as self-evidently bad, but rather to reshape your framing to discuss the problem which attends. Once you start reshaping into a description of a problem, a good problem statement template nudges you towards more in-depth discussion of who is affected.

For Whom are You Solving?

When working to identify a problem to be solved, part of what we want to think about is for whom we are solving. There are usually multiple groups of people who use our products, and who face different problems or different versions of the same problem. Each group of people has a specific goal they are trying to achieve, a specific circumstance in which they are pursuing that goal, and a specific thinking style which reflects how they conceptualize and approach the goal. Each of these aspects affects the definition of what it means to solve the problem. Each of these distinct groups (combinations of people, contexts, and purpose) will experience a distinct level of impact associated with not-solving the problem and each will contribute a distinct aspect of benefit coming from having solved the problem

I saw a post recently where someone said that product managers operate within ambiguity. I don’t think that’s right. Product managers operate within an environment of uncertainty – and embracing uncertainty is critical to creating value, but product managers have a responsibility to eliminate ambiguity. We don’t want our teams to operate with a lack of clarity, we want them to operate comfortably with a lack of certainty.

Many problem statements start with complete ambiguity about who is affected by the problem to be addressed. You will struggle to solve a problem without knowing for whom you’re solving – because you have no way to understand their definition of ‘solved.’ This is the reason the problem statement template distinctly calls out “Affects…” as a field, versus relying on product people to mention who is affected when describing the problem itself.

Imagine road construction on your commute to work in the morning. For you, the problem may be that your commute was unexpectedly longer, and you risk being late for work. For the truck driver next to you the problem is safely navigating the narrowed road with their large vehicle. It is the same situation – but you experience different problems, potentially solved in different ways. Making the rerouted lanes wider would solve the truck driver’s problem without solving yours. You should not be ambiguous about the people affected by the problem you intend to solve – this insight will help you shape the definition of the problem, and avoid delivering a solution to the wrong manifestation of the problem and ultimately failing your target customers.

You want to solve for all of the customers – the commuter and the truck driver, as well as the construction workers, the city planners, dispatch operators, mapping services, civil servants, etc. You probably don’t have the resources to solve for all of the customers. Even if you did, each incremental solution is developed with an opportunity cost of not solving some other problem. Even in the unlikely event that you have the luxury to solve for everyone, and you have nothing better to do, you still need an approach for sequencing the work into incremental deliveries – and a very strong incremental delivery pattern is to do something meaningful for one group prior to another instead of something partial for all groups. Each group faces a different problem, so you need to identify for whom you will solve, and for whom you will solve first.

All of the People All of the Time

A brilliant colleague of mine, Jeff Vogelsang, once shared a version of this 2×2 matrix to help with assigning a priority level to defects.

This intuitively makes a lot of sense – the “showstopper bugs” are the ones which affect all of the people all of the time. Like the mobile app which crashes on the loading screen. Issues which are less pervasive are lower priority. When something affects only a few people, and then only infrequently, it is intuitively something we address far later than the other issues.

We bring this intuition to defining the problems we are considering solving as well. When coaching someone, I will tell them that it is fine for the first draft of the problem statement, but it is ambiguous because not all people experience the same situation in the same way. Removing ambiguity is something which happens through multiple clarifying steps of rewriting and reshaping your view of the problem. A very common challenge I see facing organizations is jumping to the solutioning aspects of product development while their understanding of the problem to be solved is still overly ambiguous.

You will not have a productive ideation session when designing solutions to an ambiguously defined problem. And here’s the catch – you won’t know you were unproductive, because you will produce a lot of solutioning ideas. You may feel productive. But all of your ideas will be valid, because each of them can be interpreted as addressing some aspect of some problem. The odds that you accidentally addressed the problem you needed to address, and not something else are low. Until you add some constraints to the search space – from “doing something” in an abstract way, to “doing something for someone distinct” you will lack the criteria for ideating to find a solution to a clearly defined problem.

The product development process, when operating with ambiguity, has no feedback mechanisms available to help you course correct. There are no signals to tell you that what you are doing is wasting time and money building something which does not matter. There is a solution – to manifest signals through evaluation of the effectiveness of what you are building at solving the problem you are trying to solve. But those signals can only be generated, those evaluations can only be interpreted, within the context of a clearly defined problem. A problem statement to address a problem for all of the people all of the time is too unfocused to establish enough clarity of purpose to support effective development. A problem statement which addresses everyone’s problem is also likely to be too large to incrementally deliver within your cadence of operations, and will need to be split into multiple independently addressable problems.

Solving for Some of the People

I was recently working with a client who has a customer base of nearly 200 million users. The product manager had a particular problem in mind and was working to formulate a problem statement. Their initial point of view was that it was not helpful to identify a subset of users for whom to explicitly solve the particular problem because the problem was being experienced by all of the users. The following example illustrates the process of focusing on a subset of users without revealing the details of the specific problem they were solving.

“The story is true…the names were changed to protect the innocent…” as the old Dragnet TV show narrator would have intoned.

The Problem of… It is hard to configure a color printer for optimal output
Affects Whom… All color printer users
The Impact of Which is… 5% of customers call tech support for help setting up their printer because of dissatisfaction with the default print quality, requiring roughly $20M USD per year in costs to staff sufficient call center capacity.
The Benefits of a Solution are… More than $10M USD savings per year in call center labor costs and a simultaneously improved setup experience for all customers.

Product managers are afflicted with the curse of knowledge – in this example, objectively knowing that configuring color printers for optimal output is hard with the current solution. Application of the problem statement template to describing the problem immediately triggers a reframing of the problem. The template helps to emphasize that while all users interact with the existing product in their experiences, only a subset of people are sufficiently negatively affected by it as to call it a problem. This gives them the first opportunity to apply a constraint to remove some ambiguity.

If 5% of customers are calling tech support, then you can assert that at least 5% of customers are having a problem with configuration. 100% of color printer users experience the current solution but only 5% of those customers reach out to tech support. This clue helps form the hypothesis that only a subset of users – somewhere between 5% and 100% consider their difficult experience to be a problem worth solving, and a subset of those users (exactly 5%) call tech support.

In the first draft above, one mistake in the problem statement is that it describes the situation (it is hard to configure), when a better problem statement would describe the problem (many customers cannot configure without calling tech support). Just like the the road construction example above – not everyone experiences the difficulty of the configuration experience as the same problem.

Rewriting this problem statement less ambiguity results in the following:

The Problem of… 5% of customers cannot configure their color printer without calling tech support.
Affects Whom… More than 5% of color printer users
The Impact of Which is… 5% of customers call tech support for help setting up their printer because of dissatisfaction with the default print quality, requiring roughly $20M USD per year in costs to staff sufficient call center capacity.
The Benefits of a Solution are… More than $10M USD savings in annual call center labor costs and a simultaneously improved setup experience for all customers.

This is clearer and more focused because it no longer is trying to improve things for all 200 million customers. But the draft above still falls short of being unambiguous. Which 5% are failing? Is it a lottery? Does the application randomly just not work? A reasonable assumption is that for the group of affected customers is somehow different from the other groups – you are clearly in the “some of the people” column in Jeff’s diagram above. The next step is to build a mental model identifying a subset of the users as being somehow distinct from the rest of the users – in goal, in identity or thinking style, or in context.

Imagine four populations of users (in this fictional example) as a mental model. Even some lightweight research could be performed to inform a sense of relative sizes of the different populations. You already know the size of one group (the 5% who call tech support)

1. 75% of users successfully configure the printer without assistance, even though the experience is hard.
2. 10% of users get assistance from someone else, without calling our tech support.
3. 5% of users call tech support, and spend 10 minutes on average getting the printer configured.
4. 10% of users never configure their printer for optimum output.

With that bit of modeling – and ideally of research (which could be as simple as a survey of people who purchased their color printer between 1 and 2 months earlier), you can now improve the problem statement further to focus on groups 2 & 3 from your model.

The Problem of… 15% of customers cannot configure their color printer without getting assistance.
Affects Whom… 15% of color printer users
The Impact of Which is… 5% of customers call tech support for help setting up their printer because of dissatisfaction with the default print quality, requiring roughly $20M USD per year in costs to staff sufficient call center capacity.
The Benefits of a Solution are… More than $10M USD savings in call center labor costs and a simultaneously improved setup experience for all customers.

Some of you, reading the above, will immediately see that this framing of the problem abandons the 10% of users who had to get assistance somewhere else. Yes, that is exactly the point, and you should be getting excited. This framing of the problem provides clarity of purpose to declare that the goal is to reduce the load on the call center. There is no “credit” with this framing for making things better for those users who needed help and found it elsewhere.

With this problem framing, an acceptable solution could be one which reduces the amount of time it takes for a tech support agent to help the customer. An also acceptable solution is one which eliminates many of the calls in the first place by making it easier to configure. Yet another acceptable solution is to provide improved call-center call routing (to more efficiently route setup-assistance calls to dedicated experts) and improved call-center training materials (to more quickly handle setup-assistance calls). Success is measured in terms of reducing the costs associated with call-center support of color printer configuration. You could solve the problem – as defined – without improving the configuration process. Technically, you’ve improved the experience which includes getting support by improving the efficiency of the support experience.

This is an intentional choice, shaping your product strategy by documenting an unambiguous problem statement. There is a cascade of clarity which results – the outputs of ideation sessions are now measured in terms of their anticipated impact on the problem as defined and not arbitrarily in terms of how they make people feel in terms of their impact on the situation originally observed. You could write a very different problem statement, where you explore the consequences of 15% of customers needing assistance, the impact of which is that for every 20 customers, 3 of them have a bad experience, and when asked, share that their experience was bad.

The research shows (and in particular I found these 2011 findings from Stan Maklan and Phil Klaus to be compelling) that process ease has a correlation of 0.69 on the CXQ (customer experience quality) measure, which then likewise extends to high correlations (over 0.9) on both loyalty and word of mouth behaviors in customers. You could develop a problem statement which clarifies that the bad experience (for at a minimum those 15% who needed help, and the subset of the 75% who eventually succeeded independently, but did not enjoy the experience) has a consequence (“the impact of which”) of degrading customer loyalty or on detrimental word-of-mouth ultimately affecting either repeat sales to existing customers, or respectively affecting market-share of prospective customers choosing a competitor’s color printer because of the reputational impact resulting from bad word of mouth.

This is a big deal. Your product strategy is shaped – are you reducing the cost consequences of providing a safety-net to catch people who are struggling (the 5% who engage the call center), or are you reducing the revenue consequences of existing customers choosing competitors for future purchases, or are you reducing the revenue consequences of reduced acquisition of future prospective customers because they anticipate better configuration experiences when purchasing a competitor’s product.

For Whom You Solve Shapes Your Choice of Problem

Your choice about which problem to solve drives your ideation about possible solutions. Your choice about which problem to solve drives the metrics you will define to measure success. Your choice about which problem to solve has emergent properties in manifesting an implicit product strategy in the absence of direction from leaders. Your choice about which problem to solve can be evaluated in its effectiveness in aligning to an explicit strategy when established by your leadership.

Your choice about which problem to solve results from your choice about the customer for whom you are solving.