Why do products fail?  Trying to organize all of the reasons that your product might fail is a Herculean effort.  Understanding how your product did, will, or might fail will help you focus on what you need to do next.

An Exploration of Product Management

A personal goal for me is to become better at product management, so that I can help create better products.

As a product manager, the most important thing you should be doing is understanding the problems that your customers face.  If you treat “improving at product management” as the product, you should start with an exploration of the problem space.  I’m framing that problem space as products that fail.  I think it is also fair to also think about products that under-perform.  They “succeed” given the goals by which they are being measured, but they never realize their full potential.  I’ll keep the “not as good as it could be” notion in the back of my head, and talk to “failed” as the larger issue.

There are conversations, blogs, books, processes, and frameworks for “how to be a (good) product manager.”  I suspect looking at this from the outside-in (problem first, solution second) may yield some interesting insights.  Thanks to Leisa Reichelt for her article on why most UX is bad, which inspired me to have the conversation here, approaching the problem as a root cause analysis.

As an agile product manager, I’m going to approach this iteratively.  I hope that you will provide insights and corrections, helping to adapt this as we go.  Your contributions will make this better, faster.

Root Cause Analysis of Failed Products

Ishikawa diagrams were originally created to allow engineers to figure out why something broke – a visual tool for organizing root cause analyses.  I’ve been using this powerful decomposition tool for several years to solve problems and organize my thoughts.  It may provide the perfect framework for gaining understanding about why products fail.

Let’s dive right in.  Here’s my first crack at the very top level of an Ishikawa for product failure.

[larger version]

Looks pretty sparse (for now).  I will fill it in, as I drill down into each area.

If you’re like me, you’ve got some “reasons for failure” in your head right now – maybe from past experience, maybe from watching products in the market today.  Do any of those reasons not map into the categories above? Tell us about it in the comments (or tell me privately, if you must) – that’s the first vector for informing an improved version of this diagram.

Here are some prose descriptions of what I’m thinking about (for now) for each of those branches on the diagram – I expect them to fill out with smaller branches attached to each of the main branches.

Product Fails in the Market

• Business Case is Flawed – This is where we would capture things like a product strategy that is not profitable, for example, your model was dependent on exponential growth – so even though you had consistently growing market share, it wasn’t enough for the product to be considered “a success” by you.
• Picked the Wrong Market – Maybe this market is about to go away, like buggy whips or audio cassettes.  Maybe the competitors in this market are just too good.  Maybe entering this market is too divergent from your corporate strategy and dilutes focus and investment in your company’s other products.  Another example would be if your team does not have the skills and resources needed to win in a particular market.
• Takes Too Long to Enter Market – Whatever it is you’re doing to enter the market, it took you too long.  Competitors have “out-gunned” you, your customer’s needs have changed, etc.  This is to capture causes where even if everything else was good, you simply didn’t move quickly enough.  At first blush, I expect organizational problems (lack of alignment, bureaucracy, insufficient resources) will all land here.
• Doesn’t Solve the Right Problems – This is where most product managers focus most of their time – making sure we’re actually solving the right problems (the ones customers are willing to pay to solve).  Problems of design – where we intended to solve the right problem, but the proposed solution doesn’t cut it – would not be in this branch – they would be in the “not good enough” branch.
• Positioning & Sales Approach is Wrong – For the first iteration, this is my catch-all for marketing and sales.  All of the problems that are “your potential customers don’t think of your product as a solution to their problems (even though it is).”  Also the problems of “your potential customers decided not to purchase (when they should have)” and “your potential customers have never heard of your product.”  This is definitely an area where you can contribute more than I can to the framework.  How would you (product marketing managers, I’m looking at you) frame this?
• Product is Not Good Enough – Execution is key here.  Not solving problems completely (although that might move to the “right problems” branch) is an example.  Bad design is an example – both bad user-experience and bad architecture.  Poor execution also lands here – broken windows, sloppy implementations, poor quality.  For this branch to work, “product” is not just the product that your development team builds, but also your customer relationships, distribution, services, etc.

Open Questions in This Model

There are definitely some design decisions I made in the approach above that are worth questioning.  Here are some that come to mind for me – add your own in the comments.

• Designs that fail to solve the target problems – I put this in the the “not good enough” bucket, because I felt like there was value in grouping the “how” separate from the “why.”  Many times, an inadequate design is a design that works great for inadequate requirements – which means the problem is in the “why” column.  How would you organize “bad requirements execution” and “bad design execution” in this model?
• Pricing and Cost – together, they reflect profitability.  ”Not profitable” implies a business case problem.  Incorrect pricing implies a positioning problem or is a red herring that is masking a sales problem.  High cost is reflective of bad design decisions and/or execution problems (both of which are in the “not good enough”) bucket.  Perhaps if you can’t create the product at the cost you need to, for your business model to work, when selling at a given price in order to compete, you have picked the wrong market.  Where would you put pricing and cost issues?
• Team Capabilities and Support. Not having (enough of) the right skills on a team limits what solutions you can create.  Not having enough support to gain needed skills constrains the solution space as well.  When your team does not have what it takes, or have what they need, to create solutions that will succeed in a given market, where would you put this?  For now, it is in the “picked the wrong market” bucket, because trying to compete in that market is infeasible.  There’s probably a better way to frame this – how would you do it?

A Litmus Test

One experiment to validate this model is to look at the main causes of project failure and see if they map well into this space.  In the Chaos Report findings from the Standish Group, the following are the top ten responses from the companies they surveyed, along with my thoughts about how they map into this framework.

1. Lack of User Input – In my experience, this manifests both as not solving the right problems and as delivering a product that is not good enough.  The mechanisms are different flavors of “not listening.”
2. Incomplete Requirements & Specifications – Ends up causing delays, and possibly not solving the right problems, or having designs that are not good enough (because the requirements that drove the designs were not good enough).
3. Changing Requirements & Specifications – I put this squarely in the “your mindset is broken” bucket.  Requirements change, even if your requirements document does not.  Depending on how this plays out (for your team, for your process), you either slog on and deliver solutions to the wrong problems, or you take too long to deliver.
4. Lack of Executive Support – This can be a takes too long problem.  Or it could be that lack of support causes a product to be abandoned (even if it were succeeding), or constrains options for your team to the point that you aren’t able to succeed within the parameters.  This one definitely doesn’t fit the model.  Should we update the model, or treat this one as “out of scope?”
5. Technology Incompetence – insufficient skill results in not good enough and usually delivery takes too long.  If really bad, it results in “impossible (for this team) to deliver.”  For the really bad cases, does it land in picked the wrong market, because that market is infeasible?
6. Lack of Resources – Just a subset of lack of executive support.
7. Unrealistic Expectations – in the worst cases, it is a problem with the business model.
8. Unclear Objectives – lands in not solving the right problem.
9. Unrealistic Time Frames – same as unrealistic expectations.
10. New Technology – yes, change is hard.

Not surprisingly, the list from the Chaos report uses a project-centric language.  Other than the (very valid) “failure profile” that lack of executive support causes projects to fail, the model seems to hold up reasonably well.

Your turn – what would you add or change?

The ideal agile team is made up of specializing generalists – but what does that really mean?  The goal isn’t to prevent functional silos of expertise, it is to allow people to cover for each other.

Great Conversation

Elena Yatzeck (@eyatzeck) posted a comment on an earlier article about agile maturity models.

In terms of refinement, I’m thinking a lot these days about “staffing the engineering team correctly.” I’m not sure I agree in practice that you can or should try to staff all teams with “specializing generalists,” or at least not as taken to an extreme. (If you’ll forgive the self-promotion, I talked more about this here: http://pagilista.blogspot.com/2012/01/no-blender-zone-cross-functional-doesnt.html.)

I’ll not only “forgive” the promotion, I’ll re-promote it.  Good stuff.

When re-reading the maturity-model article, this snippet popped out at me:

People over process is the right emphasis. If you can’t find people that are “good enough” you might as well go home. Doesn’t matter how agile you are if you don’t have the horsepower. You also need people who are excited to “do agile” – they like to communicate, they enjoy the project and team dynamics of an agile process. You’re also better off with specializing generalists – ideally, every member of the team can do any work that is needed. This is an efficiency play – you risk introducing bottlenecks when you have a specialist who is the “only one” who can do particular types of work – because you will not have a consistent mix of types of work from release to release.
Agile Maturity Model

Thirty months later, my experiences have increased my conviction that this is true – and have realized that the way I wrote the quote above fails to provide a key clarification.

Following that link to an (even earlier) article on specializing generalists, brings the following (emphasis added):

The idea of specializing generalists is easiest to grasp by first saying what it is not. It is not staffing a team with a database expert, a user interface coder, a SOA (service oriented architecture) guru and an architect. With four specialists, each development task has an obvious owner. Database changes and refactoring go to the database expert. Reworking the UI goes into the queue for the AJAX hotshot. The problem is that this approach is only efficient when each team member is equally loaded with work. Since an agile team is continuously reprioritizing their work based on repeated feedback cycles as part of each release, this doesn’t work. The team will never face a situation where the (for example) four most important things to do are one item for each specialist. You can very easily have a release where all of the most important tasks are focused on the user interface. So all of the non-interface-experts are either working on lower-priority tasks, or even worse – they are idle. And you delay the most important work until the specialist can get to it.

By staffing a team with people who have an area of expertise, but can do anything, you can maximize the value of each delivery cycle. In our example, where all of the tasks for a release are UI tasks, they can be interchangeably assigned to any of the developers. The UI expert may suggest an implementation approach, do code reviews, or provide guidance to all the other developers. But every developer (including the database guy) can sling code effectively to get the job done. Specializing generalists.

This is very effective for making the “development engine” a black-box. Feed it the highest priority stuff, and it all gets done. We can take that approach to the next level. Designers can implement, project managers can design test plans, and yes, product managers can specify design. Twitch. Back up a sentence and read it again.

Specifying design is not the job of the product manager. True. Very true. Emphatically true. But specifying design can be what a specializing generalist does, even when that person is also responsible for defining market needs.
Specializing Generalists 2008

Elena’s article identifies a common misconception – that “specializing generalist” is a fancy way of saying “a bunch of people who can all do everything:

It’s a seductively simple fallacy of division to interpret the concept of “cross functional” team to mean a “collection of cross-functional individuals.” New agilists are quick to apologize that “we still have functional silos here” as though it would be much better if everyone could do all the same things. Grab some equally skilled poly-functional people, have them all take turns doing all of the jobs as needed, and you’ll all laugh your way to on-time, high-quality, and valuable working software.

Not so fast!

The power of an effective agile team, like the power of any other effective team, doesn’t come from its homogeneity, but from its ability to harness its diversity.
No Blender Zone: Cross Functional Doesn’t Mean Homogenous

Elena goes on to say (emphasis mine)

Team members shouldn’t attempt to Harrison Bergeron themselves into a mish-mash of mediocre (but working!) software. Someone needs to facilitate the stakeholders into some sensible semblance of a business case. Someone needs to build functional test suites that mercilessly beat on the code to prevent it from breaking in production. Neither of these are exactly the same skills it takes to gradually evolve the design of a complex system in modules of 100 lines of code or less. If people want to try new things, that’s great, but it needs to be with the realization that other jobs on the team are actual professions with skills and the need for experience in order to excel.

I completely agree.

Specializing Generalist

Specializing Generalist.

• Not a specialist.
• Not a generalist.

You need best of breed team members who specialize in areas of experise – “actual professions with skills,” as Elena puts it.  Without people who excel in the needed areas, you end up with a mediocre product.  How many times have you gone to the store and asked for the “middle of the pack” product?

That’s not even table stakes anymore.  Just the ability to create “something” isn’t interesting in the market, and isn’t interesting to the members of the team.  How many times have you heard someone brag “I love my job, I’m a cog in the machine?”  You have to have people who specialize in all of the needed areas (interface design, market insight, coding, quality, etc) in order to create a viable product.

If you staff your team with (only) generalists you will fail.

Pure generalists cannot create a product that is “good enough” – because they aren’t good enough at the creating the parts, from which the product is the sum.  You have to have people who specialize in creating great “parts” of the solution.  That’s what you need to have a shot at creating a great product.  But it isn’t really enough.  The problem is in how you define “great.”  Great means that customers buy it, users love it, and your competition is knocked back on their heels by it. Everyone agrees on this, but most people miss one thing.

Your market is changing – you also have to be fast.  You can’t solve the right problems if you aren’t fast, because the problems that are “right” are constantly changing – your market is a moving target.

Specialists, as individuals, are capable of creating great “parts” in their silos, and those parts all add up to a “great” product, so what’s the problem?  The problem is that collectively, by the time the specialists are done, they are no longer solving the right problem.

If you staff your team with (pure) specialists you will fail.

The most important tasks for the team, in any given sprint, will not balance into a perfectly allocated workload, where each “part” is worked on by each specialist, where no one is idle, and no one is a bottleneck.  It just doesn’t happen.  I haven’t seen it in 15 years in the software world, or in my prior decade as a mechanical engineer.

When one specialist is waiting for something important, she isn’t idle, she’s just working on something that is by definition not as important.  OK, you’re minimizing the damage – but you’re still taking damage.  When another specialist is the bottleneck, you lose.  Nothing magical to do here.

If you staff your team with specializing generalists you may succeed.

The work that piles up in any one specialized silo is of varying degrees of complexity.  The “UI specialist” may be backed up with a bunch of CSS tweaks, some straightforward AJAX calls to write, and a gnarly refactoring of the model-view-controller model to adapt to changing understanding of market needs.  No one can solve the MVC problem without specialized skills – but with guidance from the UI expert, one of  the other team members can handle the AJAX calls and CSS updates.  Extend this same model across other aspects of the product.  Your database expert may be needed to optimize query performance or resolve locking problems, but other members of the team could make straightforward schema changes.

It is the collective ability of the team to optimize what they collectively work on that accelerates the team’s delivery of the most important capabilities.

You have to have people who specialize, in order to optimize individual performance.  But your team needs to be built with specializing generalists in order to optimize for team performance.

T-Shaped People

From an HR perspective, I was taught about “T-Shaped People” – people who have breadth and depth of skills.

• Specialists are “I-Shaped People” – people who have depth of expertise, without breadth
• Generalists are “Minus-Shaped People” – people who have a breadth of skills, but no depth of expertise.
• Specializing Generalists are “T-Shaped People” – people who have depth of expertise in one area, combined with a breadth of skills across many areas.

These are the people you’re going for.

Thanks Elena for re-invigorating the discussion!

After building an understanding of which problems are important to your each customer you want to serve, and rating each competitive product , you’re ready to tally the scores and see how your product compares with your competition.  This tells you if you’re likely to crush it, and if not, lets you know where you should invest later.  This series on comparing products starts here if you need to get caught up.

And now, on to the finale…

Overall Product Comparison Process

This is a relatively long series.  Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems they care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers. (This Article)

With this information, you can create a point of view about how your product compares to the others.

What You Know

During the series, you’ve pulled together a lot of market information, and probably had several insights.  [Reminder so that no one accidentally misinterprets this article - this is a hypothetical analysis with fake data, intended to teach how you do this - it is not an analysis of the Kindle Fire, specifically]  Along the way you’ve determined

Who your target customers are, in order to make sure you design your product for the right people, instead of trying to design for everyone (which is the same as designing for no one).

1. Tina - A hi-tech prosumer who is using the device to get smarter about the latest trends in her industry
2. Tim - A hi-tech prosumer who is using the device to enjoy niche fiction content, particularly comics, e-zines and self-published works
3. Kenny - A typical kindle user who is using the device for his work in the finance space, studying proposals and business plans, etc
4. Karla - A typical kindle user and voracious reader who is using the device to eliminate the large pile of books on her nightstand
5. Chris - A basic consumer who would is studying business in college
6. Christina - A basic consumer who is in a book club, and who is always reading the latest best seller

Comparing Products Part 3 – Market Problems

How important each of those customers is to your strategy, to give you a means of determining the relative importance of each solution.

[larger image]

[larger Blue Ocean image]
[...]
Important Customers – Comparing Products Part 5

Which problems those customers care about solving, to minimize the amount of effort you put into capabilities that don’t effect the purchase decision for those customers.

The importance of each of those problems to each of those customers, so that you know which problem to solve first (or best) for each customer.

1. Read Anywhere – Be able to read content in multiple physical environments / on multiple devices, and not lose my place in the book.
2. Annotate – Be able to annotate / highlight what I’m reading for future review.
3. Talk About It – Be able to have conversations with other people who are reading what I’m reading.
4. Find More to Read – Make it easier for me to find other content that I would like to read.
5. Subscribe – Be able to subscribe to magazines / newspapers / blogs / serial publications.
6. More From My Network - Be able to read what people I trust are reading.

[larger image]
Comparing Products Part 3 – Market Problems

Your goal is to find the right level of abstraction at which to determine who your competition is.  If we consider Tim, the hi-tech prosumer who consumes niche content, we would see that the decomposition of his high level goals looks like the following:

[larger image]

Know Your Competition – Comparing Products Part 6

5. How effectively each competitive product solves each of those problems, so that you know who your customers consider to be your competition.

Scoring All of the Capabilities for All of the Products

Applying the same process (determine the nature of each capability, determine the criteria for each capability, assign a score to each product for each capability) will result in something that looks like the following [manufactured to illustrate the concepts] data:

[larger image]
Rating Your Competition – Comparing Products Part 7

Now you’re ready to combine this information to see how your product stacks up.

Rolling Up Scores – The Mathy Bits

First, don’t worry – there is a little bit of math here, but it is baked into the process so that you only have to worry about it once or twice, and then it just helps instead of getting in the way.  The math serves to answer several subordinate questions along the way to answering the questions that lead up to “which is the better product, and by how much?”

Cognitive Bias in Customers, but not Robots

For each customer, you know how important each problem is to solve.  You know how effectively each product solves each of those problems.  If your customers were robots, this would be easy – each customer would rationally compute the utility they get from each capability from each product, and optimize their overall utility by picking the product that maximizes this value.  You could just multiply the scores and get the answer.  Here’s one reason why that won’t work (there are others).

We saw with Kano Analysis of realistic more-is-better capabilities, people get a non-linear sense of satisfaction from solving problems, which we reflected in our [made up] data by mapping of the scores (from 1 through 9) against the overall curve for each individual capability.

[larger image]
Rating Your Competition – Comparing Products – Part 7

This approach normalizes the scores within the pseudo-math for comparison. Don’t lose sight of the fact that it is probably harder to improve your product from a “6″ to a “7″ for any one score than it is to improve it from a “2″ to a “3.” What this approach does embody, is that improving a single capability from a “7″ to a “9″ is more valuable than moving from a “5″ to a “7.”

Because your customers are not robots, they will have cognitive biases that cause them to overweight (mentally) their personal “most important problem” relative to the “less important problems” when determining their overall score for a product that solves all of the problems.  In other words, a customer will overweight the “most important capabilities” and underweight the “still important” capabilities, from a purely rational utility-maximization perspective.

It is more important to your customer to be a little bit better at solving their most important problem than it is to be a little bit better at solving their least important problem.

This is true both for robots and customers, and the “just multiply” approach captures this relative importance.  However, when you include the cognitive biases of people, you have to go one step further.

It is more important to your customer to improve your solution to their most important problem than it is to improve your solution by twice as much, to a problem that is half as important.

What the pseudo-math does is change the process from prioritizing by the user’s (robot’s) utility, to comparing products based on the buyer’s perceived utility at the time of the purchasing decision – incorporating the cognitive biases into the way the scores are calculated.

As with any numbers-heavy technique, you run the very real risk of creating your own sense of false precision. This entire exercise is the formulation of a model (in Lean terms, a testable hypothesis) of how to compare competing products.  If you don’t trust (or if you disprove) the approach that I use as a starting point, then change it to reflect what you think (or what you find).

To incorporate this bias that overemphasizes the solution of the most important problems, I square the values of relative importance of problems (to users).  The original data rates the importance of solving problems on a scale from 1 to 5.

[larger image]

When squared, the values become:

[larger image]

I chose to use squaring of the values because it intuitively feels right, when using a 1-5 scale for the (actual) weightings.  There may be a better formula, and the ideal adjustment may be different for each product domain, or even for each persona within that domain.  My gut tells me that if you invest the time in determining the “right” way to account for biases, you will have lost the opportunity to compete in your market, although you will have gained the opportunity to publish a useful psychology research paper.  Use squaring, ignore cognitive bias, or use some other calculation that passes your “sniff test” – it’s up to you.  This is another example of why the Art of Product Management is an art.

Scoring Each Product for Each Customer

When you multiply the scores for each product (for each capability) with the relative importance (of each capability) for each persona, you end up with a table of values that reflect the perceived score for each product for each persona.

For Tina, that table looks like the following

[larger image]

For Kenny, that table looks like the following:

[larger image]

Notice that the ranges of values (for Total score) are different for Kenny and for Tina.  That’s because Kenny cares about fewer product capabilities.  The “Max Possible” column has been added to each table, to reflect the maximum score that each persona would place for any given capability, for any product that scored a “9″ on that capability.  The total-per-product scores for each persona are then reflected as a percentage of the maximum possible score for each persona.  This allows you to manage the comparison across customers effectively.

The table showing the final score per-product, per-customer looks like the following:

[larger image]

What you can see with this view is that none of the products is ideal for any of the personas.  All of the products are pretty good for Kenny, and none of them is very good for Christina, although the iPad 2 is noticeably better than any of the others for Christina.

You can also look at this the other way (columns-first, rows-second), and see that the Nook Tablet is better suited to Kenny than anyone else.  If Barnes & Noble’s strategy involved a focus on Kenny, that would make sense for them. Choosing Kenny may have been a bad idea, however, since the Nook [in our made-up data] doesn’t do a better job – for Kenny – than the other products.

This is the view that is most useful in making product decisions moving forward – making this product comparison process useful to you.  You can use it as a starting point for doing what-if analysis. You can see what you need to improve to improve your product (relative to the competition) for any particular persona.  You can hypothesize about what your competitors might do to improve their products, and what competitive responses you might need.  You can determine if you have a good opportunity to differentiate your product – for a particular persona – or if that part of the market is too crowded.

In practice, you can use this view to (a) assess the viability of a particular strategy (that involves “winning” with a particular persona), and (b) perform sensitivity analysis for introducing new (or improved) capabilities.  To do the sensitivity analysis, start by asking “how much will this improve the rating (for each persona) for the capability?”  Then see what the impact is (in this table) of changing your product’s score(s).  Intuitively, this is asking the question – “Would improving (or adding) capability X turn the dial in the market?”

You can also use this framework to imagine the future positioning impacts of anticipated or likely improvements in competitive products.  This gives you a mechanism for organizing a strategy of competing with what your competitors will be doing (by the time your product launches), instead of trying to compete with where your competitors used to be.

While this is the most useful view for you – informing your individual decisions – you probably need to roll it up to a crisp answer for your boss – answering questions like:

• Which product is best?
• How far ahead (or behind) are we?
• How much do we need to invest to “win”?
• Should we pivot, or should we persist?

Scoring Each Product Across Customers

Earlier in the process, you established a measure of relative importance of each customer, for executing a given strategy with your product.

Using the numbers for the Blue Ocean Strategy, we found [manufactured] the following relative importance of each customer:

[larger image]

When you multiply each per-product, per-customer score by the relative importance of each customer (in a blue ocean strategy), you get a table that looks like the following:

[larger image]

For the Blue Ocean Strategy, the iPad2 and the Kindle Touch are “better” products than all the others, whereas the PC is by far the worst.

If you were to use the Red Ocean Strategy, where we found [created] the following relative importance of each customer:

[larger image]

When you multiple each per-product, per-customer score by the relative importance of each customer (in a red ocean strategy), you get a table that looks like the following:

[larger image]

In the Red Ocean Strategy, the iPad2 is the clear winner, with only the Kindle Touch threatening them.

You can tell your boss, as the product manager for the Kindle Fire:

• We have to pivot away from the Red Ocean Strategy, to the Blue Ocean strategy, if we want to stand a chance.
• We’re still in 3rd place with the Blue Ocean Strategy, but the key to winning it is to win for Tina, where’ we’re pretty much tied.
• The way for us to win with Tina is to exploit the iPad2′s blind spot – finding more to read.
• Here’s my proposal for how we tackle the Finding More to Read user goal for Tina.  In 3 months, we’ll have a better set of features, and 6 months after that the ecosystem around those features will result in us having the better product.

Summary

This is a process for bringing some order, structure, and testability to the process of comparing products – as a tool for informing better future product decisions.

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems your customers care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers. (This Article)

With this information, you can create a point of view about how your product compares to other products.

This model has evolved quite a bit since I first used an early version of it in 2009.  Now that you’ve looked at it too – how would you improve it?  When you use it, let me know what you find to be the strengths and weaknesses of this approach.

And thanks for sticking with me over the last 9 weeks, 8 articles, and 16,751 words.  Please let me know what you think, and share this with your colleagues if you think they would value it.

At this point in the product comparison series, you know who your customers are, which problems are important to them, and which products compete to solve those problems.  It’s time to score the competing products and see how the solutions your product provides (or will provide) will stack up.  This is the latest in a series on comparing products, jump back to the start of the series if you came here first, but hurry up :).

Overall Product Comparison Process

This is a relatively long series.  Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems they care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem. (This article)
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to the others.

Summarizing Effectiveness

Earlier in the series, we identified (and refined) the list of important, relevant problems that our target customers have.

[larger image]

This is the “ruler” by which each competitive product is going to be measured.  We also identified several competitors.

[larger image]

The next step is to assess how effectively each competitive product (including your own) solves each important problem.  Then you need to assign a “score” for how effectively each product solves each problem.  To do that, for each problem you have to articulate an opinion about what it means to solve the problem poorly or completely, or anywhere in-between.

Read Anywhere – Previously clarified as “Be able to read content in multiple physical environments / on multiple devices, and not lose my place in the book.”

Environments can be indoors/outdoors, extremely cold to moderate to extremely hot temperatures, with variable lighting from a dark room to direct sunlight.  It might also capture environmental context – sitting, walking, riding on a bus, driving, etc.; quiet to noisy; physically serene, or getting bumped a lot (like in a crowded coffee shop).

Start by defining the endpoints.  I’ve been using a 9-point scale in this type of analysis, to provide enough granularity to make relative comparisons.  For read anywhere, a score of 1 would mean “can be used to read in a single, idealized environment / location.”  A score of 9 would mean “can be used to read in any realistic situation.”  Mapping out the scores in-between 1 and 9 requires you to think about the nature of the problem being solved – and here’s where Kano analysis is useful (again).

This capability is a good example of an extreme more-is-better capability.  Increasing the range of environments where the product can be used (to read) provides a perceivable benefit to customers, but with diminishing returns.  Also, there is some minimum bar, or table stakes, of environments where the user needs to be able to read, or the product is not considered a viable solution.  On the high end, being able to read literally anywhere, would truly distinguish one product – making that capability a delighter and a strong differentiator.

[larger image]

How do you decide “What is a 3 score?” You inform these relative scores based on user research (ideally), and your subjective opinion (when you don’t have research).  For folks who haven’t been reading Tyner Blain articles for the last few years – a product manager is market driven, which means you need to use market data to do this as a product manager; however, using your own opinion as a product designer is better than having no data at all.

For this example, my [manufactured, invented, made up for this series of articles,] data defines scores for the Read Anywhere capability as follows:

1. Below The Bar - User must be seated, and have power and internet connectivity (at the time of reading) in order to use the device.
2. na
3. Usable but Very Annoying – User can read while standing without being connected to power or the internet.
4. Not Quite Happy, but Whatever - User must be indoors, with reasonable lighting and temperature.
5. Meh - User can read while riding on the bus or in a car.
6. OK, but Nothing Special- User can read in outdoor lighting.
7. Good - User can read pretty much anywhere except really noisy, jarring, and / or low-light environments.
8. na
9. Wow! - User can read anywhere that the user would want to read.

Note that there are no entries for 2 or 8, to reflect that there’s a step-function decrease (or increase) in perceived value at this point in the curve.  Plotted on the Kano analysis extreme more is better curve, this rating scale looks like the following:

[larger image]

One of the things you can see when looking at the scoring system visually is that you may have to make significant investments in order to make incremental improvements, depending on where you are on the curve.  For example, moving from (4) to (6) looks hard – you are objectively increasing the capability measurably – and increasing the subjectively perceived value by a comparable amount.

A small shift in scoring – for example, from (7) to (9) – can have a dramatic impact on perceived value (specifically, satisfaction received from improving the capability).  Conversely, improving from (6) to (7) may be really hard (expensive) to do, but realistically only improves the way your market perceives your product by a marginal amount.

This view can help you answer questions like

• Why does the iPod shuffle outperform the Sansa Clip so dramatically? Because the iTunes-centric ecosystem turns the dial from (7) to (9).
• Why doesn’t our detergent outsell theirs, when ours gets clothes cleaner in soft water?  Because moving from (6) to (7) doesn’t make much of a difference.

Zooming in on the low-end of the curve:

[larger image]

And at the high end of this capability, we see:

[larger image]

Applying this rating scale to the competitive products [more made-up data here] we see

[larger image]

It may be that different personas would use markedly different criteria for scoring relative capability.  So far, when I’ve done this, I’ve found that different persona care different amounts about the scores for a particular capability, but generally use the same approach to scoring.  When I do come across different personas that would use markedly different scoring criteria for the same capability, I will create use the appropriate scoring criteria for each persona, and add more complexity to this process.  To date, I haven’t had to do that.

Scoring All of the Capabilities for All of the Products

Applying the same process (determine the nature of each capability, determine the criteria for each capability, assign a score to each product for each capability) will result in something that looks like the following [manufactured to illustrate the concepts] data:

[larger image]

Alternate Scoring Method

You could also approach determining the score for a single capability like this by giving points for each characteristic, versus trying to define a continuum like the example above.  For example, you could give

• 2 points for being able to use the device without power.
• 1 point for being able to read when you are not connected to the internet.
• 1 point for being able to read in bright sunlight
• 1 point for being able to read in a dark room
• etcetera

Then tally up the score for each product, to describe how well they meet the need that users have to read anywhere.

Interpreting the Comparison

If you were to stop here, you would just conclude that the iPad2 is the best, and that the two Kindle products are “close,” and that the Nook and using a PC are in last place by a wide margin.  However, you aren’t going to stop here.

Stopping here would ignore completely that different customers care different amounts about solving different problems.

In the next article in this series, we’ll see how to incorporate that info, and answer the two questions you need to be able to answer:

1. Which product is best for a specific persona?
2. Which product is best overall.

Summary

To create a competitive product, you need to know how your product stacks up against the competition – and that means you need to know how effective your solutions are (or are perceived to be) at solving the problems your customers will pay to solve.  You can compare today’s product to assess your current position, and you can inform the decisions about what your product needs to be.

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems your customers care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem. (This article)
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to other products.

Taking it to the next level, as a product manager, your decisions about tomorrow’s product should be in the context of where you expect tomorrow’s competition (and tomorrow’s customers) to be.  There is a danger – especially after investing in the “state of the industry” analysis above – that you will continue to compete in yesterday’s race, when you should probably be innovating to redefine the game.  The comparison you just did is not a waste (because it looks at today’s problems – they become the table stakes for tomorrow).

Remember, this exercise informs future product decisions, it does not define them.

Attributions

Thanks Rick Cox for the height comparison photo.

You start with a point of view about what makes a minimum viable product.  When your product launches, it is your customer’s point of view that matters.  You must understand which problems your customers care about solving, and what solutions are available to your customers today.  You need to understand your competition to make informed decisions about your product.  This is the latest in a series on comparing products – jump back to the beginning of the series to catch up, we’ll wait.

Overall Product Comparison Process

This is a relatively long series.  Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems they care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition. (This Article)
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to the others.

No Competition

If you think your product has no competition, you’re wrong.

The photo above shows an El Camino, a combination of a car and a truck.  The front of the vehicle, the cabin interior, and the overall height / profile of the vehicle is that of a car; but the vehicle has a tailgate and a truck bed for hauling stuff.  A pretty novel idea at the time.  There were no other car-truck combination vehicles, but that doesn’t mean the El Camino had no competition.

You should not define competitive products as the products that exactly match the features of your product.  You should not define competitive products as those that solve (exactly) the same set of problems that your product solves.  Every interesting problem already has a solution – and therefore, you have competition.  It may be that there are not any particularly good solutions in the market.  It may be that the competitive products are not well known, and would not be likely to compete effectively.

It may be that the solution your customers choose is “deal with it.”  The “deal with it” solution, at first glance, from a B2C (business – to – consumer) perspective feels like “there is no competition,” but I believe that embracing that point of view puts you at risk of being myopic about your market.  In the B2B (business – to – business) area, “deal with it” is one version of the “in house solution” competitor – a very real scenario.  Stacking ridiculous amounts of stuff on top of your car is analogous to managing your accounts receivable in a spreadsheet.  You can do it, but it is a solution that is fraught with other risks.

Your customer generally has four options from which to choose when deciding if they should buy your product.

1. Buy your product.  Hooray.
2. Buy someone else’s product.
3. Build their own solution.  Your customer believes that rolling their own is a better alternative to purchasing a solution from someone else.
4. Deal with the problem.  The cost of the solution is believed to be higher than the cost of the problem.

How do you identify which products are compelling competitors?

Problem Abstraction

One thing you learn as a product manager, when eliciting requirements, is to keep asking “why?” (and more and more on asking “why?” – check out the comments on these articles).  The goal in elicitation is to find the actual underlying problem, and not just the obvious manifestation of the problem.  One technique you can use is to create an Ishikawa Diagram that helps you define the problem(s) your product is attempting to solve.

Your goal is to find the right level of abstraction at which to determine who your competition is.  If we consider Tim, the hi-tech prosumer who consumes niche content, we would see that the decomposition of his high level goals looks like the following:

[larger image]

There are several benefits to using this approach of describing the decomposition of Tim’s goals:

1. You get traceability of your requirements, allowing you to see why each goal matters, from Tim’s perspective.
2. The graphical layout forces you to be concise in how you describe Tim’s goals.
3. You have a framework for validating that you have defined the complete set of requirements needed to satisfy Tim.
4. Looking at “everything” in one view helps you assure that your requirements are consistent.
5. You know “when to stop” in decomposition when your requirements are atomic.
6. You have a framework for assuring that you focus on those goals that Tim is passionate about.
7. You have tractable goals that you can use to think outside of the box when identifying competitors.
8. You have a clear and visceral tool for communicating with your team, to inspire innovative designs and clever solutions.

Finding Competitors

Some of the competitive solutions will be very straightforward, and obvious based on your understanding of your product domain.  Others you will discover through research.

When we look at the Kindle Fire as a product, we know from Amazon’s positioning that Tim will be able to read books & magazines (in color), consume multimedia content, browse the internet, stream movies, and borrow books.  He’ll have access to “millions of books,” be able to “read anywhere,” and connect with people over email from the device.  The Barnes & Noble Nook Tablet is positioned directly as a competitor to the Kindle Fire (and can even be purchased through Amazon).

It is occasionally annoying to read reviews of the Kindle Fire that basically say “not an iPad killer,” where the reviewer compares the features of the Kindle Fire, a single purpose device with multiple capabilities, with the iPad, a multi-purpose device that can be used for (many) single purposes.  The iPad is a viable competitor for the Kindle Fire, but a comparison in the other direction does not make sense.  The products are asymmetric substitutes – they are only substitutes for each other in a particular context.  If you are Tim, trying to enjoy niche content, then the iPad is a viable competitive product.  If you are a 15 year old boy who wants to play a first person shooter when you’re being shuttled to soccer practice, the Kindle Fire is a non-starter.

Finding a needle in a haystack is easier than identifying competitive products.  You know there is only one needle – once you find it, you stop looking.

Identifying competitive products is more akin to finding out how many needles are in the haystack

You can do research on the internet – start with the positioning from already identified competitors, look for editorials, rumor articles, reviews, research papers, customer reviews, discussion forums.  Search social media streams – find conversations (and broadcasts) about the product and the domain.  This gives you either minimal or overwhelming information, depending on how many competitors are already positioning themselves as products in this market segment.

Interview people who share goals with your personas – they don’t have to match your personas (but they can) – you are trying to discover solutions that you don’t yet know about.  Find out how they address those goals.  Maybe they use a web browser on their desktop computer, cobbling together a cabal of single-purpose solutions.  If they already own a competitive product, find out what they did before they bought that product.  As an example, someone like our Tim persona, before buying a purpose-built product, did the following:

• Used The Sword and Laser podcast to discover new content and get recommendations.
• Used Facebook to get and share suggestions with his friends, and have conversations about his favorite works.
• Used wikipedia and artist individual websites to find other works by his favorite artists.
• Subscribed by email to artists that published serials, blogs, other recurring content.
• Purchased tangible copies of works from Amazon.com.
• Borrowed tangible copies of works from IRL (in-real-life) friends.
• Purchased ebooks in pdf, epub or other formats that could be read on his computer and phone.
• Used Dropbox (or drag-and-drop by USB) to synchronize content across devices.

The problems that people are solving did not come into existence when the solutions were invented. The problems predate the solutions. [Note:if you have a Forrester subscription, check out Mary Gerush's Exploit the Requirements Lifecycle for a more detailed discussion].

The roll your own solution is a viable “competitor” for a B2C product.  In the B2B world, this is usually called best of breed. [Note: it is usually called this by one of the vendors of one of the pieces of the cobbled-together solution].  Using these amalgam-of-products solution strategies can make sense – and for some customers it will be the best solution.

We know that Christina and Time (two of our personas, defined earlier in the series) are different personas – they value different goals differently.  They will also use different approaches to create their own mix and match solutions to their problems.  Previously, I said that you didn’t have to interview people who represented your personas in order to identify competitors.  You do, however, need to figure out which product bundles represent the roll your own solution for each persona.  That is best done by interviewing representative customer prospects.

Our Example Competitors List

In this series, we are showing the mechanics of performing a product comparison, not sharing an actual analysis of the Kindle Fire.  As such, this is (again) where I get to manufacture illustrative data.  Here’s the list of competitive products that could have resulted from the research and analysis described above:

1. The Kindle Fire
2. The Nook Tablet
3. The Kindle Touch
4. The iPad 2
5. A laptop running windows, with a pdf reader, the free kindle app, and an HTML 5-compliant web browser.

The list above is not exhaustive.  Like most analysis activities, you can go on forever, but with diminishing returns.  Where you draw the line is a matter of risk mitigation.  Add as many competitors as you feel you need to accurately assess how well your product (or a future version of your product) will compete in the market.  There is a risk associated with both too much analysis and too little analysis.

Too much analysis slows you down and is expensive.  You will get a comprehensive view, and minimize the risk of missing something, but you run the risk of taking too long.  You also run the risk of never stopping – you can never find all of the needles in the haystack – and therefore you run the risk of never shipping or shipping too late.

Too little analysis saves money and time, but you run the risk of delivering the wrong product.  If you happen to identify all of the competitors that have (or will have) success in the market, you’ll be sufficiently informed (because your view will be a super-set of your customers’ views).  If you overlook a product that is driving the market, you run the risk of watching the world go by through rose colored glasses – you’ll watch people buy that other product, and not understand why they aren’t buying yours.

Here’s where agile provides a perspective that lets you escape the catch-22.  If you do “too much” you can never recover.  The money you spent is a sunk cost.  The time you lost is gone.

If you do too little, you can always do more later, when you realize you need to, as long as your team is capable of embracing change.

Summary

To create a competitive product, you need to know how your product stacks up against the competition – and that means you need to know who the competition is.  A product marketing manager may take this information to emphasize strengths and “turn weaknesses into strengths.”  A product manager can use the insights from this analysis to paint an informed picture of what the product needs to be (and why).

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems your customers care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition. (This article)
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to other products.

Attributions

Thanks Matthiew Yglesias for the El Camino photo.

A good product is one that solves valuable market problems.  To be successful in the market, a product needs to solve the problems that the right customers are willing to pay to solve.  To know if those customers are willing to pay, you need to understand how they perceive your product relative to alternative solutions.  If you’re new to the series, head back to the intro article on comparing products, and catch up with this article, where we look at pulling together the information about which customers are important.

Overall Product Comparison Process

This is a relatively long series.  Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems they care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers. (This article)
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to the others.

Target Personas

Previously in the series we identified (and refined) a set of personas that make sense for comparing products that compete with the Amazon Kindle.  [Note: for people new to the series, this is a hypothetical analysis - using manufactured data - with the goal of showing the mechanics of how to do the analysis, not an actual analysis of real market data.]

1. Tina - A hi-tech prosumer who is using the device to get smarter about the latest trends in her industry
2. Tim - A hi-tech prosumer who is using the device to enjoy niche fiction content, particularly comics, e-zines and self-published works
3. Kenny - A typical kindle user who is using the device for his work in the finance space, studying proposals and business plans, etc
4. Karla - A typical kindle user and voracious reader who is using the device to eliminate the large pile of books on her nightstand
5. Chris - A basic consumer who would is studying business in college
6. Christina - A basic consumer who is in a book club, and who is always reading the latest best seller

[...]
We also identified a set of problems that these personas would want to solve.

1. Read Anywhere – Be able to read content in multiple physical environments / on multiple devices, and not lose my place in the book.
2. Annotate – Be able to annotate / highlight what I’m reading for future review.
3. Talk About It – Be able to have conversations with other people who are reading what I’m reading.
4. Find More to Read – Make it easier for me to find other content that I would like to read.
5. Subscribe – Be able to subscribe to magazines / newspapers / blogs / serial publications.
6. More From My Network - Be able to read what people I trust are reading.

[larger image]
Comparing Products Part 3 – Market Problems

You can’t create great solutions for all of these problems, for all of these customers, at the same time.  This is one of the two main failings of the bureaucratic approach I’ve seen large companies take (the other one is using a long waterfall process, that even if it succeeds, results in a product that solves yesterday’s problems by the time it is released).  The old process would have you gather inputs from multiple stakeholders, identify multiple target customers and the capabilities needed to support them, and roll it all up into a giant document with MoSCoW reflections of the importance of each capability.  Since every capability is important to someone, every capability ends up being “important.”

If you’ve worked on a non-agile project, you’ve seen this:  Everything is high priority.

Except it isn’t.  The problem that the monolithic document, waterfall processes does not take into account is that while everything may be important, not everything is urgent.

Only items that are both important and urgent are high priority.

Every problem identified in our example is a “five” to at least one of our personas.  If you’re trying to be all things to all people, they are all important.  Figuring out which ones are urgent requires you to form a strategy on which problems you need to solve first.

Solve for One Persona First

This process is informed (biased?) by insights that resonated for me from the writings of Seth Godin, Al Ries & Jack Trout, Geoffrey Moore, Hayden Christensen, and innumerable conversations with other product professionals.  In summary, it has bubbled up into the following perspective from me.

• A product that is great for some people, even if unusable for other people, is a great product.
• A product that is usable by many people, but great for none of them, is a bad product.
• If you wait until your product is “perfect,” or even “great for a lot of people,” at best you will deliver yesterday’s product and no one will care anymore.

Believing this, my approach then is to ask:

• For whom should we build first?
• What do those customers expect and what will they love?

Those expectations are set, in part, by your competition – your market is a moving target - which is why comparing products is important to figuring out what problems to solve first, and how well you need to solve them.

Another way to think about it – success in your market is a reflection of aggregate sales.  But sales don’t come in aggregate.  Sales are made one at a time.  In any given sales conversation, your customer prospect is comparing your product with your competition.  If your customer has two choices – the first being marginally interesting (even if it happens to be marginally interesting to “everyone”), the second being “perfect,” which one will the prospect choose?  For any given sale, you want the reasons why that customer would purchase your product to be overwhelming.  You want to be the product that is perfect, not the product that is tolerable.

Even if your goal is to sell to “everyone,” you can’t launch your product with it being the “perfect” product for everyone.  You can launch with a focus on a single persona.  Note that I didn’t say with a product that is “perfect” for a single persona.  You probably don’t need to wait until it is perfect to start selling it.  You certainly shouldn’t wait until it is perfect before introducing it (and getting feedback that helps you improve) to people that are represented by your target persona.

When you think about the minimum viable product (MVP), the minimum is not “be tolerable to everyone” – it is “be good enough for someone.”  Focus first on one group of people.  And figure out where your minimum bar is.  That minimum bar is defined by a combination of

• Which problems are important to solve – if you don’t solve the “showstopper” or “table stakes” problems, you don’t have an MVP, and won’t even be part of your customer’s decision process
• How well do you need to solve each of these problems – the next article in this series gets into this, but it is primarily a matter of matching or exceeding customer expectations, which are informed (or anchored by) how well competing solutions solve those problems today.

Picking the Right First Persona

There are a bunch of factors that can feed into the appeal of targeting any particular persona first:

• Market Size - Fish Where the Fish Are.  All things being equal (although they never are), the largest market is the most attractive market.  How you define “size” may influence which market you pick however.  Is it the market with the potential for the most units sold?  The one market with the highest revenue potential or the one with the highest profit potential?
• Cross-Market Influence - After you’ve launched, you need a plan for growth.  Do you grow by gaining market share with your target persona, by growing share of wallet (sell more stuff) with your existing customers, by growing into adjacent markets (other personas).  Does picking one target persona first, and making them rabid fans, help you transition into other markets?  The best way to sell me a gadget is to convince the digerati (The Verge, Engadget, etc) that it is the best product ever.  The best way to sell my mom a gadget is to convince me.
• Higher-Level Strategy – Your product may only be part of a portfolio of products and services.  Your company may be focused on owning a niche, like being the technology supplier to all health care professionals.  The strategic value of being a “full service provider” or of preventing the entry of a competitor into a market segment by satisfying the personas in that segment may dwarf the value of any siloed value of your product.
• Risk Mitigation – This is the Lean approach.  With which group of customers can you most quickly, most cheaply, and most effectively test your hypothesis that you have a great product that will satisfy needs and succeed in a large market?  Early adopters are the most forgiving technology customers, and often provide the most insight about what problems the larger markets will care about later.  There’s also implicit value in the “gee whiz” factor of novel solutions to existing (and new) problems.  You can put out a beta version of a product to early adopters, where they are ok with some bugs, workarounds, and missing capabilities.  This isn’t likely to be your end goal – you’re serving these people first in order to more effectively serve another market next.
• Ease of Entry – Nature Abhors a Vacuum.   There may be a persona for whom none of the existing (competitive) products provides particularly compelling solutions.  You may want to attack this segment first, because easy early sales can fund your company and keep the lights on while you tackle the larger, and likely more competitive, market.  I think that one of the reasons the Innovator’s Dilemma happens is because there is a niche market with unsolved problems, into which companies introduced products that eventually (through improvements in the product, and changes in the needs of the larger market) becomes competitive and eventually displaces the original leaders in the larger market.

Jason Cohen, founder of Smart Bear Software, wrote a great piece on finding the “perfect customer” – an example of the risk mitigation approach.  The conversation on Jason’s article is great too – and I love one of his comments: “There’s no such things as ’5x better’ if there’s no customer in mind. Because “better” is in the eye of the beholder.”

Your overall approach will determine the different weightings you apply to each of these factors.  As a couple straw men for this example, Amazon might be taking a red ocean (beat the competition in an existing, defined market) or a blue ocean (“create” a new market by solving different problems) approach.  Those approaches might yield the following initial relative weightings:

For the personas in this example, I’ve made up the following reflection of their relative attractiveness along each axis.

[larger image]

These 1 to 5 values reflect the relative attractiveness of each persona for each strategic approach.  For example, the size of the potential market of people like Christina is much larger than the size for any of the other personas.  Note that the Ease of Entry field is shown in orange versus green to emphasize that these initial values reflect a guess at the ease of entry – a general perception of how effectively existing products address the needs of these personas for the problems we previously identified.  When you do the more detailed analysis in future steps, you should circle back and revisit these values.  In Marketing Warfare by Ries and Trout, they point out the importance of the linkage between strategy and tactics – each informs the other.  A tactical understanding of your market (like an understanding of the competitive products)) will inform the best strategy – the strategy that comes from an ivory tower will fail in the field.  That philosophy is consistent with the iterative process of refining a comparison of products, as part of informing your strategy.

The strategic approach you choose will impact the “pseudo-math” of which persona is most important.  Consider the two examples – when you use the percentages from the first two tables to apply a weighting to the assessment of each strategy aspect, you get the following:

[larger Red Ocean image]

A Red Ocean Approach would indicate that Christina is the most important persona on which to focus.  The combination of the large market size of basic consumers reading for entertainment, along with an overall strategy to focus on the broad market makes Christina your target.

[larger Blue Ocean image]

A Blue Ocean Approach would indicate that Tina is the most important persona on which to focus.  The perception that no other products address her problems dominates the other factors.  Once we confirm that no one is solving the market problems that we know are important to Tina, we will have market data that validates that a blue ocean strategy is viable.

There are a lot of numbers here, and definitely a risk that you let the numbers overwhelm decisions and drive everything mechanically, like a computer algorithm.  That would be bad.  Where this data-driven view is useful is in giving you a framework for questioning the strategic approach.  The “math” highlights that a strategy of going toe-to-toe with competitors, and attacking the largest market first says Christina is the persona on which you should focus.  A blue ocean approach would drive you to emphasize investment in solving the problems that are important to Tina.  You also get a feel for how many “happy accidents” you might get along the way, by also “accidentally” creating a great product for Kenny & Chris while focusing on Tina, because the problems that Tim cares about also happen to be ones that Tina cares about.

[larger image]

Remember, though, that the Ease of Entry initial guesses will change as soon as you pull together detailed information about how effectively competitive products address the problems that are important to each persona.  Next we’ll look at the competition.

Summary

To create a competitive product, you need to know which customers on whom you should focus and for which problems they would value solutions.  There are always more potential customers, with differing needs and perspectives, than you can build product to satisfy.  You need to figure out which customers to focus on, and which ones you consider a “happy accident” if you acquire them.  Even if your management team insists that your product is “for everyone,” you need to know for which customers you will solve best and deliver first.

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems your customers care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers. (This article)
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to other products.

Attributions

Thanks Jon Worth for the skater photo.

Thanks Larry Lamsa for the skateboarder photo

If you understand the important market problems, you can make a good product.  If you understand how important each problem is, for each group of customers, you can make a great product.  If you’re new to this series, go back and start at the first article, we’ll wait for you right here.

Overall Product Comparison Process

This is a relatively long series.  Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems they care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers. (This article)
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to the others.

Important Problems

In the previous article on defining market problems, we identified 6 personas / contexts by which we would compare the kindle fire with other products.

1. Tina - A hi-tech prosumer who is using the device to get smarter about the latest trends in her industry
2. Tim - A hi-tech prosumer who is using the device to enjoy niche fiction content, particularly comics, e-zines and self-published works
3. Kenny - A typical kindle user who is using the device for his work in the finance space, studying proposals and business plans, etc
4. Karla - A typical kindle user and voracious reader who is using the device to eliminate the large pile of books on her nightstand
5. Chris - A basic consumer who would is studying business in college
6. Christina - A basic consumer who is in a book club, and who is always reading the latest best seller

The reason it was important to identify contexts is that the important aspect of personas is to identify groups of users with homogeneous perspectives on the relative value of solutions to particular problems – and the three personas previously identified would place different values on the solutions, depending on the context in which they were using the device.

Revisiting the Market Problems

We also identified a set of problems that these personas would want to solve.

1. Read Anywhere – Be able to read content in multiple physical environments / on multiple devices, and not lose my place in the book.
2. Annotate – Be able to annotate / highlight what I’m reading for future review.
3. Talk About It – Be able to have conversations with other people who are reading what I’m reading.
4. Find More to Read – Make it easier for me to find other content that I would like to read.
5. Subscribe – Be able to subscribe to magazines / newspapers / blogs / serial publications.
6. More From My Network - Be able to read what people I trust are reading.

This process is iterative, and in reviewing the 6 problems above, a valid question is – are problems (4) and (6) different versions of the same problem?  When writing requirements, it is important to specify the problems and not the design.  This is a tricky one, as it blurs the line between requirements and design.  Reasonable people can make either of the following interpretations:

• The market requirement is to “find more to read” by any means necessary – it could be through receiving recommendations from the user’s network, or it could be based on some not-specified “black box” heuristic and scoring algorithm.  These two requirements are duplicates.
• Yes, ultimately the user is trying to find more to read, but this is actually too abstract – consider the goal of “enjoy using the device,” which is also the ultimate goal, and too abstract to be useful in guiding the product creation.  Asking people you trust for recommendations is a well established practice for finding reading material, and people make the distinction that it is inherently different from, and provides unique value relative to other approaches (like finding similar products, or “people who bought this also bought this other thing“).

Based on research I’ve done for previous clients (primarily on findability and recommendations in an eCommerce context), my personal perspective is that these problems are distinct, and should be treated differently.  This is one of those Art of Product Management situations, where different product managers can, will, and should reach different conclusions.  For this analysis, the problems will be treated distinctly, and the data that I invent will reflect that I believe these problems would manifest with different importance for different personas in different contexts.

In contrast to the above analysis, problem (1) conflates the two notions of “multiple devices” and “multiple environments.”  People familiar with eInk technologies and Amazon’s Whispersync technology may see these as independent solutions to reading in direct sunlight, and moving from device to device respectively.

I’ve combined these two ideas, primarily because of the influence of this presentation about multi-screen strategies from Precious , a design and strategy consultancy.

[image from Precious article]

My interpretation is that the notion of device shifting is something that is intentional – use the right device for the right environment – and not arbitrary.  Based on that, and the technologies that enable reading in different physical environments – like direct sunlight or a dimly lit room – should be evaluated as part of implementing a device-shifting strategy, and not independently.

Combining these two (in this example) is an opinion-based decision, just as keeping problems (4) & (6) distinct is an opinion based decision.  If the data you gather in identifying problems leads you to combine or separate those problems, do so.

Gathering the Data

In the previous article on identifying market problems, I indicated that you are using a mix of qualitative and quantitative data-collection techniques to identify the problems.  It is during those activities that you also quantify the relative importance of solving these problems.

Innovation Games are a particularly engaging way to gather this type of information [disclosure: while I haven't formally done work for the company, I have worked with and know and respect the founder Luke Hohmann , and likely will work with him again in the future].  I’m including them here because they have worked for me.  A couple examples:

• 20/20 Vision – get customers to put solutions to the problems into relative order (for them).  Effectively, a card-sorting exercise.
• Speedboat – the relative priority insight you get from this game comes when people put anchors at different depths, indicating the severity of a particular problem.
• Whole Product – this brainstorming game can also be used to identify what people perceive to be Must Be (table-stakes) capabilities of your product.

Surveys can also be used to gather data when you need to get feedback from larger numbers of people, or are operating in an environment that is less collaborative.  A simple approach – ask the question “How important is it to you to solve problem X?”  Read up on Likert scales to understand the dangers of asking this type of question, and determine if you need to bring in someone who is an expert at survey design to minimize the impact of bad survey design in your results.

Quantifying the Results

Using the word, quantifying, may not be wholly accurate here – it implies something scientific, when really you are synthesizing the data you have received, in order to eventually determine the relative importance of solving (or improving your solution of) specific market problems.  A made-up word like numberifying might be better.  If quantifying, used in this context, causes your head to explode, please comment below – I suspect this might be a hot-button for folks, but I’m not really sure if it is.

[larger image]

The above table shows hypothetical data representing the “quantified” relative importance of having a solution for each problem, to each persona / context.

For example, Kenny wants to use a device to review and annotate proposals and business plans – internal company documents.  He can do this on any of a number of devices, but the ability to do it “anywhere,” as long as he can annotate, is what really matters to him.  He’s not at all interested in capabilities to find more to read, discuss what he’s reviewing, or subscribe to new content.

For Christina, however, the act of reading is more social than private.  She is not worried about making notes in what she reads or subscribing to new content – she’s reading books and talking about them with her friends.

Gaining Insights

We’re not to the point in the series where we can actually compare competing products effectively yet, but you can already start to get insights.  Imagine you are designing a product for Christina.  Your business case is about going after the “Oprah book club” audience, and Christina is your target user.  You already know which problems are important to her, and more importantly, which problems are not important to her.  You can completely eliminate the subscription and annotation capabilities from your product and create something that Christina would love.

This step alone allows you to avoid the elastic user problem.

Usually, you aren’t able to design a product for just one user.  You can’t be all things to all people.  This is part of the analysis you have to do to determine what to build first – identifying the relative importance of problems to each persona.  The next step is to figure out the relative importance of each persona.  Combining the two allows you to understand the overall relative importance of each problem.

Summary

Creating a great product for your customers means not only knowing who your customers are, but also knowing which problems they want to solve, and among those problems, which ones it is most important to solve or solve first.  When comparing your product to competitive products, the best measure is one that compares the products based on the most important problems for each group of customers.

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems your customers care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers. (This article)
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to other products.

Comparing products without an understanding of the important market problems by which to compare the products is a waste of time.  This is the third in a series on comparing products - jump back to the introduction if you haven’t already read the previous articles.  Go ahead, we’ll wait, then come back.

Overall Product Comparison Process

This is a relatively long series.  Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems they care about solving. (This article)
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to the others.

Too many product comparisons are actually mislabeled positioning papers, highlighting differences, and putting one product in the best possible light.  An honest product comparison must compare products in the context of problems that customers are willing to pay to solve.  Those are the important market problems.

Important Market Problems

Probably everyone reading this has heard the Henry Ford quip about how if he had listened to his customers, he would have built faster horses.  A great marketing sound-bite, but not actually true.  The market problem that cars initially addressed wasn’t “get there faster”, the market problem was “spent fuel” disposal (for horses) in major cities – 2.5 million pounds per day in New York City (hat tip to Super Freakonomics for the first reference I saw).  If cars had generated as much manure as horses, do you think they would have been as successful?

The first step in identifying the important market problems is to identify a set of market problems that might be important. Then you can form an opinion about which ones are important, based on which problems are important to the potential customers in the markets on which you are focusing.  Identifying problems is an unconstrained activity, like collecting seashells on a beach.  There will always be more shells.

You will find shells that aren’t very pretty or are broken.  You will find some shells that are “perfect”, and there will always be some shells that exist, that are “perfect” that you won’t find.  The tough part is knowing when to stop looking for shells and move forward with the ones that you’ve already found.  You have to decide when your shell collection is “good enough.”  You can’t have a collection that includes all of the seashells, you don’t want a meager collection with too few shells, and you want the best shells you can find to be the ones you display on your coffee table.

This is a good example of the debate between big up-front design (BUFD) & big up-front requirements (BUFR) and incremental product development.  In The Design of Design, Frederick Brooks provides a great discussion about this debate between rationalism and empiricism.  The best approach I’ve found to date is to use a combination of agile and waterfall techniques to strike a balance between good enough and right now.

The best approach that I’ve found to do this to combine incremental development techniques to the activities of product management, combined with timeboxing to impose practical, artificial constraints on the process.  An agile product management process looks roughly like the following:

[larger version]

The steps of market research, analysis, and synthesis are the activities you perform.  Data is what you collect, and insights are what you form.  Analysis of data also leads you to perform more research.  Synthesis of those insights into a cohesive vision for your product will lead you back to more analysis and more research.  Note that this process has no ending – it goes on forever – or for as long as you are trying to solve these particular market problems.

As an example – in the previous article we identified a set of personas for our Kindle Fire example:

For this series of articles, I will pretend that three personas were developed, in order to demonstrate this approach to comparing products:

1. Hi-Tech Prosumers – people who get excited about convergence, highly value additional multimedia capabilities, live a multi-device existence, and are acutely aware of competitive products.
2. Typical Amazon Kindle Users – people who placed high value on the reading experience, the convenience of being able to easily get new content, and the absence of a subscription fee.
3. Basic Consumers – people who read primarily on paper media, have recently started consuming multimedia on their computers, and anticipate becoming part of the “post pc era.”

I will also invent additional data as we explore the rest of the product comparison process.
Comparing Products Part 2 – Who Are Your Customers?

However, defining requirements for those personas risks creating a product that only satisfies the basest of goals – the lowest common denominators.  These examples don’t take into account the context of use – e.g. why is someone reading on their device? – and thus we are at risk of not creating the right product for any of them.  Even introducing two simple contexts

• Education - Reading for work or school (directed study) or to gain knowledge about a topic (undirected study)
• Entertainment - Reading for pleasure, the literary equivalent of watching a movie

You’re probably not going to want to take notes on the uses of foreshadowing and allegory when reading for entertainment.  It may be critically important to annotate key concepts and ideas when reading for education.

This single distinction should be clanging a bell in your head – it is very easy for one product to be better than another in one context of usage than in the other.  Looking at the three example personas from the previous article, there is not a good way to say “educational use is more important” for one group of people than for another.  If we iterate and refine our list of personas it will lead to a better comparison of products.

1. Tina - A hi-tech prosumer who is using the device to get smarter about the latest trends in her industry
2. Tim - A hi-tech prosumer who is using the device to enjoy niche fiction content, particularly comics, e-zines and self-published works
3. Kenny - A typical kindle user who is using the device for his work in the finance space, studying proposals and business plans, etc
4. Karla - A typical kindle user and voracious reader who is using the device to eliminate the large pile of books on her nightstand
5. Chris - A basic consumer who would is studying business in college
6. Christina - A basic consumer who is in a book club, and who is always reading the latest best seller

Identifying Possibly Important Problems

Now that we have a (better) idea of the customers for whom we are solving market problems, we need to create a list of candidate problems that might be worth solving.  I am not an expert on user research – so there may very well be a better way to do this, and hopefully someone will share that with us in the comments section – but here’s how I’ve learned to do it.  It is at least “good enough.”  The approach I’ve used is a 3-step process.

1. Seed the Cloud – Start with what you think you know already
2. Refine the List – Validate and enhance that list through qualitative interviews
3. Empirical Validation – Assess the relative importance of those problems through quantitative research

Seeding the Cloud

I like to start by capturing my initial ideas in a mindmap or concept map – an ad hoc, semi-structured view of ideas.

[larger image]

I will then drive some brainstorming (more on brainstorming) or idea-seeding or Innovation Games exercises to develop a refined list of market problems that might be worth solving.  These processes may lead to other artifacts and usually result in updates to my mindmap.  These exercises bring others into the exploration, so that it isn’t The World According to Scott – but it is still the world as seen “from the inside.”

Refine The List

What we’ve done at this point, while creating a set of questions worth asking, possibly, is still heavily biased as an inside-out view of the market.  It may be informed by previous research and synthesis of ideas, but it still reflects what we think about the market, not what customers in the market thinks about their problems.

The next step is to gather qualitative information from customers and potential customers that represent your target personas.

Anecdotally, this is the same process for driving outside-in innovation from a product management perspective.

Mostly-open-ended conversations with people that are reflective of your personas is how I transform from an inside-out view to an outside-in view.  I’m not asking people “what they want” (remember the faster horse), I’m getting validation that the problems they face are the ones I think I understand (waste disposal).  This usually uncovers additional problems I have not thought of, and discounts problems I may have thought were important.  Ethnographic research provides this type of information – and when I’m working with a team that does this work, I utilize it.  Usually, I’m not, so I will find representative users and ask them questions, or observe their behavior.  Essentially, I’m playing the 80/20 game – some insight is better than no insight.

Empirical Validation

At this stage, you still only have a list of possibly important market problems, but now you have much more confidence that the list is valid and mostly complete.  You also have some qualitative insights into the relative importance of problems to be solved (maybe you did a card-sorting exercise, or created interrelation digraphs, or played the 20-20 Vision game, to get those inputs).

Now you can gather empirical data, to gain some statistical significance to your analysis.  You can do surveys that help you characterize and profile user behavior.  You can even do analysis of social networks, to see which problems resonate and why particular products get good (or bad) word of mouth.

Amplified Analytics has a presentation on slideshare showing how they can mine statistically significant information about features in the context of products. [Hat tip to Gregory, for the pointer to this].

I would suggest using a similar approach to try and understand which problems people face (and solve) versus which features they prefer (as solutions).  But the same applies.  I’ve had good results creating surveys and using Google Docs to aggregate results that allow me to characterize problems, (reported) behavioral patterns – like frequency of use, and (reported) relative importance of solving particular problems (like performance times for specific actions), and quantifying data-characteristics (e.g. how many emails someone sends, and to how many different people, in a given day).

Example Problem List

Given that we’re not actually doing a rigorous (and free) analysis of the Kindle, I get to “pretend” that I did research following the examples above, resulting in the following list of market problems, that are of (at least some) interest to our target personas.  We’ll use this list moving forward in the rest of the series.

1. Be able to read content in multiple physical environments / on multiple devices, and not lose my place in the book.
2. Be able to annotate / highlight what I’m reading for future review.
3. Be able to have conversations with other people who are reading what I’m reading.
4. Make it easier for me to find other content that I would like to read.
5. Be able to subscribe to magazines / newspapers / blogs / serial publications.
6. Be able to read what people I trust are reading.

Summary

It is important to be creating a product with an identity.  That means knowing for whom the product is designed, and for which uses.  You have to know who your users are, and which problems they want to solve.  If you’re going to compare products, you should compare them based on their suitability as solutions to these important problems.

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems your customers care about solving. (This article)
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to other products.

Attributions

• Thanks Stéphane Vandenwyngaert for the Model T photo.
• Thanks michaelaw for the smiling coffee drinker photo.
• Thanks Dora Pete for the self-portrait photo.
• Thanks Benjamin Earwicker for the spontaneous photo.

The first step to comparing products is understanding your customers.  This may seem counter-intuitive, but your product’s capabilities are meaningless unless you are comparing them from your customer’s point of view.  This article is part 2 in a series on comparing products.  Check out part 1, then continue with this article on the first steps of comparing products.

Overall Product Comparison Process

This is may be a long series.  Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers. (This article)
3. Articulate the problems they care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to the others.

From that point of view, you can determine where to invest in your product.  That’s the whole point of doing the comparison – not to be self-congratulatory, but to guide forward thinking.  I’ve stopped being excited when I find existing “product comparisons” and game plans when starting to work with a product team.  To date, they have always been “positioning papers” designed to emphasize the product’s strength – usually prepared for internal sales teams, but unfortunately, sometimes prepared for executives (‘Look!  We’ve done a great job!”) Those documents are a useful source of information to get you started down the product comparison path, but if you let them guide you, they will take you down the primrose path.

Identify Your Customers

You want to have a model that represents how you are approaching your market. As a product manager focused on solving customer problems, you should have a customer-centric market model.

Your goal is to identify groups of customers that share sets of problems, and share a point of view about how important it is to solve those problems.  Effectively, this is a definition of a persona (yes, I know there are better ways to define personas).

The important thing is that you identify groups of people that will reach the same conclusion about your product, by answering the following questions similarly -

• Which problems that are important to me can I solve with this product?
• How well does this product solve those problems?
• How does this product compare to other products?

Personas, Goals, and Contexts

The image above is adapted from Robert W. Bailey’s diagram from Human Performance Engineering, (although I first saw it in the Handbook of Usability Testing).  One use of Bailey’s diagram is to understand that to assess the usability of a product, you have to take into account the human being who is using the product, the environment in which they are using the product, and the actions the person is trying to perform.

My extensions of the idea are that

• Groups of people with similar sensibilities (personas) will form similar conclusions about how much they like a product when,
• They are using the product in similar contexts,
• To achieve similar goals

One benefit of abstracting this way is that you don’t get caught up in the mechanics or procedures of how someone is solving a problem (the actions).  Focus on the action is appropriate when evaluating the usability of a particular product, but not when comparing how different products solve the same problem.  I also like something that reminds me that people use products in different environments and situations.

Example Personas

Usually, when I see personas they are focused purely on marketing and demographic data, or they are based solely on contextual inquiry intended to inform a design aesthetic.  The former may tell you where to market a product to groups of people (magazines, tv ads, luxury travel websites, etc), and the latter informs how to implement solutions.  What we need is something that helps us assess the relative importance of solutions to problems shared by groups of people.

This series of articles is looking at the Kindle Fire as a proxy product by which I will explain the techniques (without actually performing a data-driven analysis – I’ll invent data for the purpose of this article series).

I found a presentation on Slideshare:

BlackBerry Consumer Preferences and Segmentation

Where some MBA students documented the research and analysis of the market of prosumers – “professional” consumers.

What I particularly like is the way they presented a summary of part of their analysis on slide 27.  That’s the part we’re going to pretend applies to the Kindle Fire (because it might just).  Note that their presentation includes a “RIM Confidential” clause in the footer, so it might have been removed by the time you read this article.  I’ll briefly describe what they did.  They did an analysis of survey results, identified groups of people that placed similar importance on different activities and aspects of smartphone usage, and identified three “clusters” of people with statistically significant shared sensibilities.

1. Hi-Tech Prosumers – people who placed high value on most features, anticipated high usage of most features, were price-sensitive, had lower incomes (less than $75K USD per year) and were the younger members of the target demographic.
2. Typical Blackberry Prosumers - people who placed high value on network features, moderate value on other features, made heavy use of “standard” capabilities, but did not plan to use apps, or multimedia.
3. Basic Prosumer - people who placed high value on phone features, were heavy users of text and voice features;  had low use of other capabilities, but anticipated becoming heavy users of “smartphone capabilities” (email, calendar, browsing, apps, camera, etc) when they purchased their next phone.

What’s notable is that they did the right type of analysis – finding clusters of people that can serve to quantify market value of different personas – this will inform the determination of the relative importance of different groups of customers (step 3 in the process of comparing products).  By interviewing individual survey respondents that are in each of the clusters, they can get the additional information they need to develop personas.

For this series of articles, I will pretend that three personas were developed, in order to demonstrate this approach to comparing products:

1. Hi-Tech Prosumers – people who get excited about convergence, highly value additional multimedia capabilities, live a multi-device existence, and are acutely aware of competitive products.
2. Typical Amazon Kindle Users – people who placed high value on the reading experience, the convenience of being able to easily get new content, and the absence of a subscription fee.
3. Basic Consumers - people who read primarily on paper media, have recently started consuming multimedia on their computers, and anticipate becoming part of the “post pc era.”

I will also invent additional data as we explore the rest of the product comparison process.

Developing Personas

Please don’t finish this article believing that persona creation is trivial – it isn’t.  I’ve heard the saying before that a bad persona is better than no persona at all, and I believe that.  Without a target persona, you will almost certainly create the elastic user anti-pattern.

Persona creation is more of a precondition of this process than a part of this process – you should already have personas developed for your product.  This article is not intended to go into the details of how to create a persona, but rather a review of the aspects of that persona development that are needed for the product comparison process to be effective:

• Identification of people in your market that share perspectives on problems – e.g. knowing that for one group of potential customers solving problem X is important and problem Y is not, even when problem Y’s solution is important to another group of customers.
• Being able to estimate the relative number of people that are represented by each persona – e.g. knowing how much revenue potential exists for sales to each group of people.

Summary

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers. (This article)
3. Articulate the problems your customers care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to other products.

Attributions

* Thanks Alaska Dude for the Lovely Lass photo.

Recently, the gadget-reviewer crowd has caught on to something we’ve known for a long time.  Comparing products is not about comparing specs, it is about comparing how well the products solve problems that customers will pay to solve.  That begs the question – how should you compare products?  Read on to see the product comparison technique I recommend.

Inspiration

Writing about how to compare products has been on my backlog for about two years, as a key component of how to perform competitive analysis.  This topic is easily a chapter-length discussion, perhaps that’s what delayed writing an article-length discussion.  A flurry of recent articles, The Death of the Spec, Do Device Specs Really Matter Anymore, and Device Specs, from Techcrunch, iDownloadBlog, and Daring Fireball, respectively, all discussed how the Amazon Kindle Fire will likely succeed, in spite of not having particularly good specs (specifications).  The Techcrunch article also opines that Consumer Reports, because of its fixation on specs, has made itself irrelevant as a company providing advice to consumers.

John Gruber hits closest to the mark in the Daring Fireball article where he says

Specs are something the device makers worry about insofar as how they affect the experience of using the device. Just like how focal length and lens aperture are something the cinematographer worries about insofar as how they affect what the viewer will see on screen.
John Gruber

Combine this with a conversation I had last night after recording the latest Start With the Customer Podcast, about writing more frequent articles on Tyner Blain.

Product Management and Specs

The discussions are mostly centered on the utility of specifications, speeds and feeds, in informing a customer’s buying decision.  As Pragmatic Marketing espouses, we should be market-driven, with a focus on solving the problems that customers will pay to solve.  A screen resolution does not solve a problem, but an easy-to-read screen does – it prevents eye-strain and makes a long-session reading experience (like you would have when reading a book, versus reading this article) better.  Avoiding eye-strain is a problem people are willing to pay to solve – we’ve seen that with the success of products that use e-Ink technology.

Let’s look at how this example impacts what you do as a product manager.

You want your team to create a product that avoids eye-strain.  But you also know that you need to write requirements that are unambiguous and measurable.  We know from research that higher resolution displays (to a point) delay eye fatigue.  We also know, from Apple’s successful marketing, that promotion of a retina display (326 dpi, for a phone) is effective, at least with buyer personas, at addressing the perceived problem as well.

A “normal” consumer is not going to be able to make an informed comparison of the likely difference in eye strain over time – the problem the consumer actually cares about – when looking at the specs for a 225 dpi device and a 250 dpi device.  The authors of the articles are exactly right about that – but we, as product managers, already know this.

The problem comes when writing the requirements for your product team – do you just say “create a low eye-strain screen” and trust the team to pick a resolution that is effective? In an ideal world, yes, you would.  Someone (that someone may have to be you) on your team would do the research and come back and tell you that a 225 dpi interface will cause moderate eye-strain in 80% of people after 12 hours of continuous reading (but only 20% of people after 4 hours), and that that number drops to 20% of people experiencing eye strain after 12 hours when using a 250 dpi screen**.  You have an understanding of the value of a 250 dpi resolution over a 225 dpi resolution – an additional 8 hours of reading time for the majority of your customers.  Your customers will not know this (unless you tell them).  But you know it, and that’s enough.

Now you have to understand the incremental cost of creating a product with a 250 dpi resolution versus one with a 225 dpi resolution.  In this example, the incremental device cost is $25 per device for the next 12 months (given projected manufacturing levels).  At your target margins, this would reflect in an increase of $40 in device pricing to your customers.

This higher-capability, higher-priced product will simultaneously appeal to more customers (people who read for more than 4 hours at a time), and fewer customers (people who are price sensitive).  Your hypothesis (backed with market-research) indicates that you will generate 50% more profit by offering the lower-capability resolution (225 dpi), because most of the people in your target market do not regularly read for 12 hours at a time – and those that do will not “blame” your product for their eyestrain – they will “blame” their own behavior.

Now you’re ready to to specify that your product will be built with a 225 dpi resolution interface.  Not because a 225 dpi interface is in any way intrinsically valuable, but because it is the measurable and unambiguous requirement needed to satisfy your goal (achieving profitability) based on a solution to a more is better (see more articles on Kano analysis) market problem (eye strain that occurs in long reading sessions).

Specifications are useful, because they help you characterize capabilities.  The diagram above depicts a more is better characteristic, as described in Kano analysis.  The resolution measurements (dpi) give you a measurable criterion for what you are building, that translates into a measurable criterion (hours of continuous use without eye strain) that your customers will realize.  That criterion reflects the horizontal axis of the diagram.  The longer your customer can read without straining her eyes, the more capable your product is.

The vertical axis reflects how much your customer cares about solving the eye-strain problem.  Note that there are diminishing returns.  Enabling 4 hours of use (without eye strain) versus 2 hours of use is more highly valued than enabling 6 hours of use versus 4 hours (or 8 versus 6).

The above analysis compared the selection of discrete components, but it also applies when looking at incremental investment.  The speed at which the screen refreshes when turning pages is a good example.  You can have an e-Ink screen that refreshes in 1 second, or one that refreshes in 0.5 seconds, each time your user turns the page.  There is no difference in incremental cost, and your team is operating with a fixed budget of time and resources – so there’s no impact on the allocated fixed costs (or margins).  You are faced with a different set of compromises – what are you willing to give up (by reducing investment in other aspects of your product) to achieve incremental improvement in page-turning responsiveness?  It may be that page-turning snappiness is the one idea of your product (it was a differentiator for the Barnes & Noble Nook, but now the e-Ink Kindle has achieved parity).  Or you may be dealing with one of the “host of others” capabilities, in which case you simply want to satisfice.

Bigger Picture

The above example shows how specs matter (indirectly) in the everyday lives of product managers.  Specs are the measurements by which we evaluate the effectiveness of our products at solving the problems our customers care about.

If you are designing a product that has no competition – for customers that have no alternatives, this would be enough.  It would be more than enough, actually, because you could create “any” product, and it would sell.  Your customers always have alternatives – so you always have competition. Sometimes, your competition is “build your own” or even “tolerate the problem.”  But for any interesting market problem, you have at least one competitor trying to solve it too (or you will.  Very soon).

Comparing Products Matters

As a product manager, you need to know what your product needs to be (or do) to be competitive.  That’s where comparing products matters.

The above analysis looks at a single problem (eye strain) for a single product (yours), to try and determine what specs to give to your team.  In a competitive environment (that means you), you need to put the effectiveness of your product in context from your customer’s point of view.  That involves identifying

• Who are your customers and what problems do they care about solving?
• How important, relative to each other, are the solutions to those problems, to your customers?
• How important, relative to each other, is each group of customers?
• What solutions (products) do your customers consider as alternatives (competition) to your product?
• How effective is each product at solving each problem?

From that information, you can synthesize a point of view on how competitive your product is (or will be).

That point of view helps you identify

• Which problems you need to invest in solving, or solving more, or solving more effectively.

Summary & Series

Specification is not a good tool for helping non-expert customers compare products.  It is, however, a good tool – an unambiguous measurement tool – that product managers can use when specifying how a product should be built, or how one product compares to another.

Products must be compared based on their effectiveness (and perceived effectiveness) at solving problems that customers will pay to solve.  Those comparisons should also take into account the relative importance of those problems, as well as the relative importance of different groups of customers, to the success of the product.

Even at 1600 words, this article barely introduces the topic of comparing products.

Recapping the overall flow of this series of articles on product comparison (I’ll update this article with links to future articles in a series on comparing products.):

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers) (This article)
2. Identify your customers.
3. Articulate the problems your customers care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to the others.

Attributions & Notes

* Thanks Roger Wollstadt for the original Volkswagon Assembly photo

** The dpi-related eyestrain statistics are fictional, and written to demonstrate the importance of measurement only

Your boss wants a commitment.  You want to offer a prediction.  Agile, you say, only allows you to estimate and predict – not to commit.  ”Horse-hockey!” your boss exclaims, “I want one throat to choke, and it will be yours if you don’t make a commitment and meet it.”  There’s a way to keep yourself off the corporate gallows – estimate, predict, and commit – using agile principles.

This is an article about agile product management and release planning.

Change and Uncertainty

In the dark ages before your team became agile, you would make estimates and commitments.  You never exactly met your commitments, and no one really noticed.  That was how the game was played.  You made a commitment, everyone knew it would be wrong, but they expected it anyway.  Maybe your boss handicapped your commitment, removing scope, lowering expectations, padding the schedule.  Heck, that’s been the recipe for success since they planned the pyramids.

It makes sense.

1. Your early estimates are wrong. When you add them up, the total will be wrong.  If you do PERT estimation, the law of large numbers will help you in aggregate.  But you’ll still be wrong.
2. The outside demands on, and availability of, your people will change. Unplanned sick time, attrition, levels of commitment over time, lots of “people stuff” is really unknown.
3. The needs of your customers will change. Markets evolve over time.  You get smarter, your competitors get better, your customer’s expectations change.

Agile processes are designed to help you deliver what your customer actually needs, not what was originally asked for. Contrast the two worlds.

In the old world, you would commit to delivering a couple pyramids. After spending double your budget, with double the project duration, you would have delivered one pyramid. When you deliver it, you find out that sphinxes are all the rage. Oops.

Your team changed to agile, so that you could deliver the sphinx. But your Pharaoh still wants a commitment to deliver a couple pyramids (the smart ones will be expecting to get just one).  You can stay true to agile, and still mollify your boss’ need to have a commitment, if you take advantage of the first-principles of why agile estimation works.

Estimation

A commitment is a factual prediction of the future.  ”This will take two weeks.”  Nobody is prescient.

A factual prediction has to be nuanced.  ”I expect* this will take no more than two weeks.”

*in reality, this is shorthand for a mathematical prediction, such as “I expect, with 95% confidence, that this will take no more than two weeks.”

Few non-scientist, non-engineers, non-mathematicians understand that 95% confidence has a precise meaning.  People usually interpret it to mean “a 5% chance that it will take more than two weeks.”  What it really means is that if this exact same task were performed twenty thousand times (in a hypothetical world, of course), then nineteen thousand of those times, it would be completed in under two weeks – do you feel lucky?

To make a statement like this, you actually have to create a PERT estimate – identifying the best-case, worst-case, and most-likely case for how long a task will take.

Unfortunately, we’re rarely asked to make a commitment about a single task – but rather a large collection of tasks – well-defined, ill-defined, and undefined.

You can combine PERT estimates for the individual tasks, resulting in an overall estimate of the collection of tasks.

The beauty of this approach is that the central limit theorem, and the law of large numbers, work to help you estimate a collection of tasks – you can actually provide better estimates of a group of tasks than a single task.  This obviously helps with the well-defined tasks that you know about at the start of the project.  This even helps with the ill-defined tasks.  Rationalists will argue that the key, then, is to do more up-front research to discover the undefined tasks – and then we’re set.  As Frederick Brooks (Mythical Man-Month) points out in The Design of Design, this debate has been going on since Descartes and Locke.  It is not a new idea.

Big Up-Front Design and Requirements (BUFD & BUFR) hasn’t worked particularly well, so far.

Don’t throw out the baby with the bath-water, however.  The math of estimation is still important and useful.  Even if empiricism is not the silver bullet.

Prediction

Estimation is a form of prediction.  Even agile teams do it.  In Scrum, you estimate a collection of user stories – in story points that represent complexity, and you predict how many points the team can complete in this sprint.  Note the time factor.  If you’re working a two-week sprint, there is very little risk of changes in staffing during a two-week period.  There’s also very little risk that your market will change significantly in two weeks – and if it does, what are the odds that you will notice and materially change your requirements in two weeks?

Visually, let’s take that PERT estimate and turn it sideways – so we can introduce the dimension of time.  Imagine you estimated all of the tasks (well-defined, ill-defined, and a guess about the undefined), as if they were all to happen in the first sprint.  Ignore inter-task dependencies, and pretend you had unlimited resources and the ability to perform all tasks in parallel.

The graph above shows the aggregate estimate – the circle is your best prediction, with error bars representing your confidence interval in the estimate.  If you were using PERT estimates, these could represent that 5% and 95% confidence lines.  Subjectively pick something based on your team’s experience in the domain and your confidence in your guesses (about the undefined tasks).

We need a segue into the “best of waterfall” approach to estimating projects, to steal and invert a good idea.

The Cone of Uncertainty

The folks at Construx have published a nice explanation of the cone of uncertainty – an adaptation of an idea from Steven McConnell’s Software Estimation: Demystifying The Black Art (2006).  That article uses his imagery with permission – so please go look at it there.  The idea is that as the project becomes better defined (e.g. during the project), the amount of uncertainty is reduced.

The findings show that initial estimates are off by 400% (either low by a factor of 4 or high by a factor of 4)!  Even after “nailing down” requirements, estimates are still off by 30% to 50%!

As bad as that sounds, it is actually worse.  This is a prediction for the original project (delivering pyramids).  Not only are your estimates wrong – but they are bad estimates for delivering the wrong product.

But – the core idea is sound – the further into the future you have to execute, the greater the mistakes in your estimate.

Taking that concept, and applying it to our diagram, we get the following:

The further into the future you are trying to predict, the less accuracy you have in your prediction.  This reduction in accuracy is reflected as a widening of the confidence bands for your estimate.

• A couple sprints’ worth of work is not much different than one sprint – so your estimation range is not much changed.
• An entire release of sprints (say 6 to 10 sprints) has much more opportunity for the unknown to rear its head.

Now, your prediction is (probably) unusably vague and imprecise.  ”This set of tasks will take X plus or minus a factor of two.”

That’s the reality.

Note: This has always been the reality.  People have historically reduced this “risk to timing” by hiding the “risk of change” aspects – and waterfall processes encourage you to deliver the wrong thing, as close to on-time as possible.

That’s not what we want to do, however.

We still want to deliver the (not-yet-defined) right product, as efficiently as possible.  That’s the goal of agile.  (For folks who haven’t been here at Tyner Blain for long – “right” includes both value and quality).

Refinement

Because we’re agile, and we’re willing to “get smarter” about our product over time, we have an opportunity to improve.  Because of the nature of compounding estimates and the cone of uncertainty, our uncertainty gets smaller over time.

Let’s remove our artificial simplification that we could do everything “right now” and look at what we think we know right now, about the end of the release.

Our ability to predict the amount of effort (for today’s definition of the product) at the end of the release is not very good.

Our ability to predict (today’s definition of the product) one sprint into the future is much better.

After completing the first sprint, we are a little bit smarter – the ill-defined tasks are better defined.  Maybe some of the undefined tasks are now ill-defined.  The same cone of uncertainty is now a little bit smaller – we are a little bit smarter, and the time horizon of the release date is a little bit closer.

The trend continues – each sprint gets us closer to the release date, and with each sprint (assuming we get feedback from our customers, and continue to study our markets) we get a little bit smarter.  We also get better at predicting the team’s velocity (how much “product” they can deliver during each sprint).

Commitment

Your boss still wants a commitment, however.  And that’s where we get to change the way we look at this (again).

The above diagrams all display how we converge on an estimate for a stable body of work.  However, we know that the body of work is constantly changing.

Backlog! [you say]

Yes!  The backlog.  The backlog is an ordered, prioritized list of user stories and bugs.  I was talking with Luke Hohmann of Innovation Games last month, and one of the most popular online Innovation Games is now the one they created based on prioritizing by bang for the buck.  Play it today online (for free!).  How cool is that?

The backlog represents the work the team is going to do – in the order in which the team is going to do it.  Over time, as we get smarter, we will add and remove items from the backlog – because we discover new capabilities that are important, and because we learn that some things aren’t worth doing.  We will even re-order the backlog as we recognize shifting priorities in the markets (or in our changing strategy).

As this happens, it turns out that the items at the top of the list are least likely to get displaced, and therefore most likely to still be part of the product by the time we get to the release.

Instead of thinking about uncertainty in terms of how long it takes, think about uncertainty in terms of how much we complete in a fixed amount of time. In agile, generally, we apply a timebox approach to determining what gets built.

Now, uncertainty, instead of manifesting as “when do we finish?” becomes “what will we finish?”

Your boss is rational.  She appreciates the constraints, she just wants to know what you can commit.  Every boss I’ve worked with has been willing (sometimes only after much discussion) to treat this uncertainty in terms of what instead of when.  They acknowledge that they need to translate (usually for their boss) into a “fixed” commitment.

The solution: commit to a subset of what you predict you can complete.

At the start of the release, you may have 500 points worth of stories.  Based on your team’s expected velocity, and the number of sprints in the release, you predict that you can complete 320 points worth of stories (5 people on the team, a team velocity of 40 points per sprint, and 8 sprints in the release).  Starting at the top of the backlog and working down, draw a cut-line at the last story you can complete (when you reach 320 points).  This is your prediction.

Now the commitment part.  You’ll have to figure out what you’re comfortable with.  Maybe for 8 sprints (say, 16 weeks into the future), you may only be comfortable committing to half that amount – 160 points.  Go back to the top of the backlog, and count down until you reach 160 points.  Everything above the line is what you commit to delivering.

Maybe you are comfortable committing to 240 points, maybe only 80.  This is like playing spades.  The more you can commit to, without missing, the better off you are.  Your tolerance for risk is different than mine.

You can also negotiate with your boss.  Commit to 160 points now, and provide an update after every other sprint.  More likely than not, you will be increasing the scope of your commitment with every update.

Mid-project updates of “we can do more” are always better than “we can do less.”  And both are better than end-of-project surprises.  This also allows you to have updates that look like this:

We didn’t know this at the start of the release, but X is really important to our customers – and we will be able to deliver X in addition to what we already committed.  Without slipping the release date.

Conclusion

Making commitments with an agile process is not impossible.  It just needs to be approached differently (if you want to stay true to agile).  The end result: better predictions, more realistic commitments, and the likelihood that each update will be good news instead of bad.

[Update: Changed initial image.  Thanks Dennis for the great photo!]

Requirements Management – I’m embarking on a journey to help several teams manage their requirements with their existing systems and tools.  This is the first in a series of articles, where the rubber meets the road.  I’ll look at both the theory and the realities of what works (and doesn’t) in practice.  I hope you’ll come along for the ride.

The Idea That Makes it Tricky To Manage Requirements

OK, so I’m working with a product management team that is working with multiple software development teams.  The development teams already have tools and systems in place for tracking work within projects.  The development teams are mostly using Atlassian’s JIRA for tracking activity.  The teams also have Atlassian’s Confluence wiki for capturing “other stuff.”

The development team has a good working process for managing their delivery activities within the “JIRA world.”  From the top down view, you find a project (in JIRA) that represents what will be delivered in a particular release.  Within that project, you see the issues (or user stories or requirements) and associated tasks that were previously scheduled for that release.

This is pretty effective for getting a handle on what’s happening.  Where it falls short is in getting a comprehensive understanding of why it is happening.  And that’s the view product managers need.

Why something needs to happen is completely different from when something needs to happen.  That’s what makes this tricky.  Consider the following analogy.

You’re driving across the country from Los Angeles (LA) to Manhattan (NYC).  Your goal is to get to Manhattan.  Your product manager creates a map, breaking down, roughly, how you will get from LA to NYC.  Your product manager is riding shotgun in the car, and he’s responsible for telling the driver where to go.  Your developer is driving the car, and is responsible for getting the car wherever the product manager tells him.  They share the responsibility for getting to NYC – neither of them can do it alone.

They climb in the car, and start driving.  When the developer has a question about a particular way to go (which highway to take, etc), they communicate.  Together, they make great progress.  After a while, the gas tank is empty, and they stop to fill up.  This is a logical break in the drive, and they stretch their legs, get some dinner, whatever.  With a full tank of gas, they get back in the car and start driving again.

Managing requirements inside the context of a release (in JIRA) is like saying every time you stop for gas (finish a release), you should create a new map.  The goal (destination) is independent of the release cycles (gas tanks).  The requirements should live outside the release.  However, the turn-by-turn information is useful inside the release.  The product manager needs to provide the next level of detail (first, get to Las Vegas, then stay at the Westin,…) for each “tank of gas” – after which, that information is not very valuable, and can be discarded.  But the goal (reach NYC) stays alive.

A process and system for managing requirements should not force product managers to recreate documentation for each release.  Your stakeholder (perhaps you’re delivering a donated kidney to a hospital) needs to know when you’re going to deliver the kidney.  The last place they should have to look for that information is within the context of the current project.

This is what makes it tricky.

JIRA and Confluence

This is not a series of articles around picking the best requirements management solution available today.  These articles will cover the exploration of using JIRA and Confluence to make (1) the product management teams more effective, and (2) the combined product delivery teams (management, marketing, development, quality) more effective.  We’re faced with a real-world constraint that we will use these tools that are already in place.

Product Management Goals

There are a series of goals that we have, that will inspire our implementation choices (of how we use Confluence and JIRA), and ultimately provide the measure of our success.

• We will not remove or regress things that are working today.  The implementation teams (development and test) are delivering products, and using JIRA to track issues and tasks.  Whatever we do must not break this current world.
• As product managers, we are continually investing in, and updating, our understanding of our markets.  We need a solution that allows us to record that information in a way that we can (a) share with each other effectively, (b) remember what we need to remember, (c) make changes as needed so that the view we have documented is always reflected of the understanding we have “in our heads.”
• As a delivery team, we need to connect the two worlds (“what am I doing” and “why are you doing it”), so that the product managers can let the implementation team know what to do next (and what they will eventually be working on).
• As a company, we need to set and manage expectations – internally to stakeholders and externally to customers and partners.

These are the problems we’ll be tackling.

This series of articles will capture elements of implementation choices, why we made them, and the rationale “behind the whys.”

I hope you’ll come along for the ride.

In last week’s article (and the GrandView webinar) I talked about using models of customer behavior as a method of understanding and investing in your markets.  One example I used is what I call a trust pyramid – representing how people have different levels of trust in the assertions of others.  This article explores the idea of the trust pyramid in more detail.

Customer Model Development

There are two challenges to developing customer models.  The first is coming up with an idea that is sufficiently representative of behavior or biases.  The second challenge is to keep the model simple – at the risk of being simplistic.

Another difficulty with modeling in general is to admit (and never forget) that a model is only a representation. A customer model is no more equivalent to your customers than a map is equivalent to the landscape being mapped.  It tells you something about your customers (probably), that allows you to form a hypothesis about their behavior.  You can then apply that hypothesis when prioritizing requirements and designing solutions.

I love that Rich Mironov’s book is titled The Art of Product Management, because I think of product management as the artistic application of scientific tools, and the scientific analysis of artistic insights.  Product management is both an art and a science.

Trust Pyramid

[larger image]

The above visual is a simple representation of a trust pyramid, showing different levels of trust based on the nature of the relationship with the trustee.  Pyramids are built from the bottom up, so let’s review ideas of trust from the bottom up too.

The purpose of the trust model is to understand how much trust a potential customer puts in a positive review or recommendation of a product.

• Trust of the Manufacturer.  People will put the least amount of trust in the manufacturer of the product.  Of course the manufacturer will tell you his product is the best.  Maybe the manufacturer is wrong – believing the product is “the best” even when it isn’t.  Regardless, the manufacturer is not going to tell you to buy someone else’s product (usually), so a recommendation from the manufacturer is almost a no-op – it doesn’t hold any sway.
• Trust of the Retailer.  People trust retailers more than manufacturers – because the retailer picked the manufacturer’s product as the one to sell, presumably because it is the best option.  But the retailer also stands to profit from the sale, so of course the retailer will tell you that the product is great.
• Trust of a 3rd Party Reviewer. Think of Consumer Reports.  They have no financial motivation to promote a particular product.  They also have a long term financial motivation to provide “good advice,” so you are more likely to trust their opinions.  The key element here is that the potential customer believes the reviewer to be free of bias.  An unbiased review is more credible, and therefore more trusted than a biased review from the retailer or manufacturer.  Consumers who write reviews about products are also 3rd Party Reviewers.  Barring astroturfing campaigns, these sources of recommendations are also good examples.
• Trust of Someone Like Me. Thanks Thomas for helping refine my earlier model with this addition! The problem with a 3rd party reviewer’s opinion is that they are expressing an opinion from the reviewer’s point of view – not mine.  A secretary may rave about the comfort provided by a pair of shoes, but she does not stand on her feet all day, so her review is less valuable to me, as a TSA agent, than the review from someone else who stands on her feet all day, like I do.  The reviewer is just as trustworthy as any other 3rd party reviewer – but the message is discounted as being somewhat-less-relevant to me.
• Trust of Someone I Know.  Even 3rd party reviewers can be disingenuous.  Someone I know and trust already will provide an opinion that I trust more than any other.  Especially when that person knows what is relevant to me – that I stand all day.  If I know the person is not credible or trustworthy, I can discount that data explicitly.

Building the Trust Pyramid Model

Yes, it’s a pretty model.  Seeing and understanding the model may inspire you to create models that help you understand your customers.  Seeing how this one was created may help you approach the creation of your own models.

In 2009, I was developing some models for visualizing how Word-of-Mouth marketing works.  That led to development of a conversation ecosystem model of engagement between companies and their customers – a conversation economy.  The final visualization of that model is conversation circles.

The idea is pretty simple.  When you create fans of your company, they promote you, quite effectively.  Seth Godin has been shouting this for years – but a lot of companies still have cotton in their ears. I was working to help people who were new to the idea of engaging their customers to see the value to their business. Part of this work put the question front and center  -why does this work?

The question stuck around with me for a while.  As part of trying to understand the underlying dynamics of social networks, and how the reputation-building game mechanic works in reputation systems, I came across a fantastic presentation, The Real Life Social Network by Paul Adams.  The next time you have a few hours to spend getting much smarter about social networks, read his presentation.

The key ideas that helped me with understanding trust relative to the types of relationships you have can be found on pages 27-30; 103 & 111-113; 126-137; 147-151; 159 & 160; and pages 168-172.

Meanwhile, I saw a great presentation by Chip Conley, where he adapted Maslow’s Hierarchy into an “Employee Pyramid” that he used effectively to help manage his company.

The pyramid visual, combined with Maslow’s “each step is greater than the next” lead to my attempt to organize the different types of trust into a pyramid.  I smoke-tested the model with user experience, user research, and community management people who I trust – leading to improvement of the model.

I’ve been able to validate some aspects of the model by reviewing AB test results of the effectiveness of incorporating elements of marketing-copy, ratings, and reviews into eCommerce sites.  This is anecdotal data, but it is consistent with what the model would predict – “more trust” leads to “more sales.”

Do You Have a Favorite Model?

Do you have a similar model you’ve developed and would be willing to share?  Any other models (from the webinar) that you would like to see explained in more detail?

Intellectual Property.  The legal jargon definition of this term has come to effectively mean “something I’ve patented, copyrighted, or hold as a trade secret.”  A more general interpretation is “an idea.”  For product managers, the most valuable ideas are insights.

The Value of Insight as Intellectual Property

Last week, the folks at Ryma Technologies Solutions invited me back to present another webinar as part of their GrandView Product Management View webinar series.  [My previous PMV webinar was on Kano Analysis] Thanks to Val and Bradley and Johnny for the invitation, and for making it a great experience!

You can watch the video version, listen to the audio, or get the slides in pdf format from the GrandView site.  If you want to get a 5-minute sample to try before you buy (with your attention, that is – this is 100% free), check out the sampler on youtube.com.

Or – if you want to flip through the slides (also on slideshare) now, and then decide – go right ahead:

Or go directly to the page on slideshare.net.

The main idea in the presentation is that your understanding of your market is the information that is most valuable.  There are several examples of this type of information in the presentation – you can see the visuals in the slides, but listening is probably required to explain most of them.

The Contentious Idea

I got some feedback that my central theorem – that insights are more valuable than patents – would be controversial.  I addressed this a little bit in the webinar.  Expanding on those thoughts here might serve to get a debate and conversation going.

I’m a former engineer (electro-mechanical controls design), and I hold 4 patents.  Each of those patents represents a body of “protected information” describing a solution to a problem.  A very specific solution.  Those patents have value.  But those patented solutions are not the only solutions to those problems.

When I was taking economics classes in college, I learned about the notions of substitute and complementary goods.  Once I became a design engineer, and learned how patents work, I also learned how to get around patents.  By designing perfect substitutes.  The process is pretty simple:

1. Pick a patented device.
2. Understand what it does – what function does it perform (and precisely how – this is the patented part)?
3. Understand what problem it was designed to solve – note: this part is not protected information.
4. Determine the important performance characteristics for solving the problem in (3).
5. Invent another way to solve the problem from (3), that meets the criteria from (4), without copying (2).

That’s it.

Every problem can be solved in many ways.  And when those alternatives all solve the problem equivalently, those solutions become perfect substitutes.  That makes any single solution effectively a commodity.  You can’t intentionally create a valuable solution to the problem without understanding the problem.

The value is in the understanding of the problem, and your market’s willingness to pay to solve it.  Not in having protection of one of the infinite number of possible solutions.  That’s the point I make in the webinar, before quickly moving to examples that show ways of developing insights about your customers [note: the examples are influenced by my approach using a customer-centric market model].

Disagree?

Chime in below or on the GrandView site, and let’s have a great discussion.  And as always, thanks for listening/watching/reading!

Thanks everyone for the great conversation in the most recent #prodmgmttalk chat session!  This week, Roger Cauvin inspired us to think about product managers as innovators – or enablers of innovation.  Each week, I find myself thinking about at least one of the #prodmgmttalk questions long after the hour is over.  This week’s thought – organizations prevent product managers from innovating.

Innovation and Product Management

What a great topic.  I’ve never met a product manager with a goal of solving valuable problems who didn’t aspire to creating innovative products.  Maybe you’re out there – say howdy if you are.  Roger’s topics, Cindy and Adrienne’s moderation, and the great community that is forming around the weekly #prodmgmttalk chats created a great hour of thought provoking discussion (brainmates’ summary;  full transcript).

Before diving into how organizations prevent product managers from innovating, I want to frame the discussion around a particular definition of innovation.

While there were several different definitions of innovation proposed during the session, I’m partial to one that I articulated during an  Accept 360 webinar last year on Innovation and Transparency (video; slides).

From the presentation on Innovation and Transparency:

Innovation = Valuable Invention

Putting it visually:

For a product manager – the focus is on discovering the value.

Roger has a great perspective on this product-management-centric view of innovation:

True innovation comes from understanding the problem in solution-neutral terms. So Product Management first attempts to understand the problem thoroughly & communicate it to designers. By framing the problem clearly, Product Management enables designers to unleash their creativity & skills. Peter Drucker speaks of “purposeful”, systematic innovation. Product Management is a big part of systematising innovation.

Roger Cauvin via Brainmates

However, innovation is not just about solving valuable problems (that’s just good product management) – and does not have to start “market first.”  Innovation can also come from discovering applications of novel technology – invention.

A researcher discovers an adhesive that is not strong enough for tape – but can be used over and over – and we have sticky-notes.  Another scientist accidentally leaves a chemical mixture in a beaker overnight and tada! – bullet-proof glass.  This is the view of innovation that seems most pervasive in the collective consciousness – someone invents some awesome thing and then finds a good use for it.

You can have a sequence that works either way – discovering market applications for a new technology, or inventing solutions that address market problems.  Innovation is what you get when you do both.

Another fun example to think about – the light bulb.  Did Mr. Edison start with a market problem (candles are not a good solution to “see stuff at night”) and end up with a light bulb?  Or did he start with an invention (distributable electricity as a service), and try and “invent” a market use?  I never met the guy :), but my hunch is the latter.

The problem is that product managers aren’t inventors looking for market opportunities.  We don’t create hammers and run around looking for nails (or things to hang up).  We engage our market, discover that bare walls are a problem (and people are willing to pay to solve it); then we work with our teams to find ways to hang pictures.

Common Organizational Approach to Innovation

I’ve worked both for and with several companies – large and small – that take this inside-out approach to developing innovative products.  Google is famously known for their “twenty percent time” – which gave us GMail and Google News.  Microsoft now has a policy where employees get to keep the intellectual property (and a share of the revenue) from their side projects.  Inventor-inspired (or developer-driven) projects are more likely to create something novel, and less likely to create something valuable.

As a related anecdotal data point – what percentage of start-ups actually succeed in the market?  I bet there are hundreds or thousands of failed start-ups for every winner.  Most small businesses close their doors in less than five years.  However, that stat includes all businesses, as this great discussion on Quora highlights.  Still – even those statistics reflect business failures, not product failures.  Most businesses have a failed product or two (or two hundred) in their past.  Even Facebook’s first “product” failed, before Mr. Zuckerberg created The Facebook.

Back to organizations.

In the crazy stream of conversation, the following caught my eye:

Whether Product Managers are innovators or innovation enablers depends on the organisation & size of your group. Additionally some orgs have a Product Planning role which precedes Product Managers, moving them to an innovation enabler role.

Rich Velazquez via Brainmates

Yup. Seen that a lot. Don’t care for it.  I think it is dysfunctional to decide what products to create / problems to solve without product management.

That neuters your product managers – making them order takers – and they lose the ability to birth innovative products, left to only participate as midwives in the process.

Bill Bliss wrote an article with great visuals describing the product planning process, (check it out) where he shows how the flow from “vision and mission” breaks down before it gets to products.  He highlights that responsibility shifts from one group to another, and that the process typically breaks during that translation of goals into products (he says ‘projects’).  He references a great article from the Silicon Valley Product Group on establishing a product council that assures product management participation (among other things) in that key “decide where to invest” step.

When product managers don’t participate in the planning process, we don’t have strategic relevance, and we aren’t innovators – we’re order takers.

Call to Action

If you’re in an order taker role with a product manager title, your goal should be to inject yourself into the decision making process.  The specific steps you need to take depend on your specific circumstances.  The skills you need are the ones you already have – story-telling, inspiration, leadership-without-authority, stakeholder management, executive communication.

OK.  So – go do it.

ps: If you weren’t here (at Tyner Blain) in 2006, go read these Top Ten Tips for Preventing Innovation.  Another tip would now be “prevent product managers from deciding where to innovate.”

Nokia, the Finnish mobile phone manufacturer, is getting clobbered as their market rapidly moves away from them.  Recent analyst reports show that Android and iOS (Apple’s platform) based phones are rapidly gaining market share.  Nokia sells neither.  Nokia has a major press event in a few hours, where they will announce their smartphone strategy.  I think a maximin strategy is both likely and correct.

Nokia’s Press Event on Friday 11 Feb, 2011

Reuters is reporting a press event from Nokia in London, at 0730 GMT (just over 4 hours from now) – where new president and CEO, Stephen Elop will announce Nokia’s smart phone strategy.  Previously, Elop was the head of the Microsoft business division.

Nokia’s Current Situation

As a very quick background – Nokia already has a smart phone platform, called Symbian, which was dominating the market until manufacturers began shipping iOS and Android based devices.  They “owned” the market, but don’t any more.  This is a great example of the Innovator’s Dilemma.

Keeping things “as is” is not a viable option for Nokia – they have to change.  Significantly.

The Wall Street Journal is reporting that they received a copy of a leaked memo where Mr. Elop telegraphed that things will be changing.  From Jennifer LeClaire’s reporting of The Journal’s reporting:

According to the Journal, the memo describes Nokia as cornered by competitors and in need of a major transformation. Elop compared Nokia to a man standing on a burning oil platform who jumps into icy waters to escape the flames, the Journal reported.

Jennifer LeClaire, Newsfactor

Several great journalists and technology reporters have articulated Nokia’s current situation and the state and direction of the mobile phone industry – including the “inevitable” cannibalization of feature phones by smart phones.  Instead of repeating their synopses – I’ll link to a particularly good analysis from Peter Rojas at GDGT.com.

Mr. Rojas sums up that Nokia is presented with the following choice of two options, and the possible outcomes of both (bold is mine):

Nokia now has to either dig in and find a way to create its own mobile ecosystem, complete with amazing handsets and a world-class OS that developers want to make apps for (something which is becoming harder with each passing day as iOS and Android further entrench themselves), or it needs to suck it up and work with an existing platform and try and use its massive resources to become the dominant player on that platform.

[...]

If Nokia bets big on being able to create a game-changing ecosystem and it fails to catch on in the market, there will be no salvaging the company at that point.

[...]

I have no doubt that the prospect of a Nokia handset running [Android or Windows Phone 7] worries Samsung, HTC, Motorola, Sony Ericsson, and LG, all of which are trying furiously to compete with the iPhone right now. The last thing they want is for Nokia to have a serious option in the high-end of the market. [...] The market is overcrowded with Android phones right now, and Nokia isn’t necessarily going to dominate it or even stand out, at least not without a ferocious fight.

[...]

So it may be that Nokia hedges its bets and introduces either Android or WP7 on a smattering of handsets in order to buy itself some time while it works on creating its own ecosystem. [...] Buying some time like this isn’t an elegant strategy, [...] but it’s probably the least bad of Nokia’s bad options.

Peter Rojas, GDGT

Summarizing Nokia’s Strategic Choices

There are many factors and variables that make this situation look complex – but they roll up into a choice for Nokia between two strategies (excluding the uninteresting strategy of business as usual):

1. Aggressively invest in their own software platform so that it becomes competitive with alternatives that are available to their customers.
2. Adopt one of the competitive software platforms, and focus on differentiating their hardware.

The first strategy has the largest possible downside, and the largest possible upside.  If the first strategy fails, Nokia becomes irrelevant in the smart phone market.  The definition of success for this strategy leads to massive relevance (and profits).

The second strategy has a downside that keeps Nokia in the smart phone business, but as just another one of many manufacturers.  Not great, but not as bad as exiting the market.  The upside of the second strategy is also not as appealing as winning with the first strategy.  The best Nokia can do is be the “hardware winner” but without control over the software environment.  Even if there were as much profit opportunity, the risk that comes with lack of control would discount the value of winning with this strategy in Nokia’s eyes.

The Maximin Strategy – Game Theory

When you focus on the two choices – one with large up and downsides, and one with moderate up and downsides, you see that Nokia is facing a choice, in game theory, of adopting either the minimax or maximin strategy.

Nokia can either pick the strategy that has the highest possible best case, or pick the strategy that has the highest possible worst case outcome.

A maximin strategy maximizes the “minimum guaranteed payout” in game-theory-jargon.  Translated, it is the conservative approach – minimize the downside of failing in the execution of the strategy.  Mr. Rojas describes Nokia s a company that has “internalized a culture of mediocrity” (see his previous analysis for more details).

If Mr. Rojas’ analysis is even close to accurate, it seems like any strategy other than maximin would be foolhardy – almost as bad as changing nothing.

Nokia should definitely take the conservative strategy, using maximin to attempt to ensure that “the worst” is not that bad.  And that means embracing [at least*] one of the other already successful smart phone software platforms for their future devices.

Nokia’s Next Move

In a few hours we will learn what Mr. Elop has decided to do.  Or, if you heard about his decision first and are reading this second, you’ll now have either a favorable opinion of my analysis or not.

Last week, on This Week in Tech, John C. Dvorak made an eloquent argument as follows:

• Nokia can’t survive with business as usual.
• An “invest in Symbian” strategy is actually a red herring – Nokia is already doing that – but failing – so it would lead to assured destruction, and Nokia therefore must embrace another platform.
• Mr. Elop, as a former Microsoft person, would lose credibility if Nokia were to choose Windows Phone 7 as their platform.
• Mr. Elop, as a former Microsoft person, would not choose “only Android” because of the fallout it would cause in injuring Microsoft (as a perceived gesture of “lack of faith in Windows Phone 7″).
• Mr. Elop, therefore, will choose to introduce Nokia smart phones based on both Android and Windows Phone 7.

While I happen to agree with Mr. Dvorak’s analysis of the futility of doubling down on Symbian – primarily because of the challenges Mr. Rojas articulated in his article, that almost doesn’t matter.

Even if Nokia does have a viable strategic choice of investing in Symbian, they shouldn’t.  The risk of failing to overcome a culture of mediocrity is too great.

Nokia’s “best” choice is to adopt either (or both) Windows Phone 7 or Android as their official platform moving forward.  As Mr. Dvorak points out – doing both could be very smart.  The two platforms were built with different personas in mind, and Nokia should offer phones for each group of people, based on the platforms designed for each group.

In this week’s #ProdMgmtTalk, one of the livelier discussion topics was around gaining insights into your market – and what does that mean (to you)? Steven Haines was the speaker for this session who prompted us to think, and pushed us to rethink our views on market insights.  What a great example of collaboration among product managers!

Collaboration

In an email-interview with Craig Brown, creator of the Better Projects site, for an upcoming article, I admitted to Craig that I felt one of my mistakes was not taking advantage of enough of opportunities to collaborate.

#ProdMgmtTalk, a weekly Twitter-chat session on product management, co-hosted by Cindy Solomon and brainmates‘ Adrienne Tan, had its inaugural session this week. It was a great forum for collaboration and connecting with some obviously very sharp and seasoned product managers!

I’m thrilled to be the Catalyst of Discussion next week (definitely a more apt title than speaker).  Pretty intimidating to be in the slot between @Steven_Haines (transcript of session) and @Jim_Holland.  The rest of the lineup looks stellar too – I’m thrilled to have had the opportunity to work with, learn from, grab a beer with, or at least talk with most of these folks already; and hope to do the same with the rest!

One great thing about a Twitter-chat session like this is that when you’re passionate about the topic and want to join in the discussion, you just do – without having to bide your time or risk talking “over” someone else.  Everyone’s thoughts are shared and consumed – often simultaneously.  Occasionally, some contributions go by too fast – but it is great to be able to carry multiple-threads of conversation at the same time, without ever feeling frenetic.

One downside – thoughts are expressed in 140 character chunks.  For many things we do, and think, care, and talk about in product management; 140characters are inadequate.  And yes, Stewart, I know that 1,000 words probably aren’t ”required”.  There’s probably a happy medium.

Market Insights

Steven Haines, author of The Product Manager’s Desk Reference, got the discussion started with some great questions around “Best in Class Product Management.”  A great conversation thread (multi-thread, really) was started around what it means to have market insights.  In her writeup of the session, Adrienne pulled together a lot of great quotes around the theme.

Combining some of the ideas from the folks in the session:

• Market data = (data about) the industry, market trends, your competitors, your (existing and prospective) customers, your market segments, and your competitors’ products.
• Insight = understanding that is distilled from market data.

Unfortunately, the conversation only lasts for an hour.  Not enough for me.  I think there’s an important next topic for the group:

• OK, you have market insights.  What do you do with them?  And how?

Next Week

Make sure and join in the #ProdMgmtTalk conversation next week (follow the link for times and instructions).

And here’s a tip – there will be a link to this article, as “prep material” for the session.

If you want to seed the conversation with some long form (more than 140 character) thoughts and answers – add them to the comments here.  Folks will read them before the session starts.

A lot of teams that I’ve worked on and with get hung up when thinking about defining requirements for “migration projects” and “system upgrades.”  There’s some intangible barrier to being market focused when it comes to improving existing internal systems.  Every new product represents a solution to an existing problem.  Why do so many projects move forward with teams that are blind to the actual requirements?

New Products

Even “revolutionary” products like the iPod are just addressing existing market problems.  Part of what makes this statement true is looking at things from a market perspective – thinking about the valuable problems that people are willing to pay to solve.  Even if the iPod were the first mp3 player (it wasn’t), it would still only be an improvement.  The first mp3 player was not new, it provided an improved way to listen to your music on the go.  You could put your whole music library in your pocket.  Much better than a stack of cassettes melting in the glove box of your car, or a tape getting caught on your keys and unraveled when you pull it out of your backpack.

What’s important, and difficult (especially for people with technology backgrounds), is to think about it in terms of what people are trying to accomplish – not how they are trying to accomplish it.  In a business process view, it is the difference between process (why) and procedure (how).

There’s a continuum of migration projects, ranging from completely new to identical processes.  Neither extreme technically exists – think of it as a range from infinite change in the existing process to 1 / (infinite change) in the existing process.

Near the “completely new process” /infinite change end of the spectrum are processes that are completely new to you.  Remember, you’re solving a problem, perhaps in a very innovative way, that people are already solving some other way.  You’re just providing a better solution approach.

Near the identical process end of the spectrum are projects are “pin-compatible” platform migrations and near-sighted legacy system migrations.  Moving from an old gas-guzzling car to a new, more efficient model is a good example.  I mention “near sighted” because that old system was designed to meet an old set of market needs, so the new system will, by definition, not meet current market needs – it will only meet the old market needs.  Pragmatically, when considering organizational change, it may make sense to do your system upgrade in two stages: migrate the systems (“nearly identical process”) and deal with all the gotchas of migration first, then start re-engineering the processes and optimizing the procedures to address new market needs (major and minor process changes, respectively) second.

Set aside the possible, possibly rational rationale* to migrate the implementation first, and the functionality second.  Consider that a deployment and logistics detail and get back to the problem at hand.  Most of the times I’ve been involved in system migrations, they have been initiated as cost-savings platform projects.  Over the last few years, more often the migration project gets prioritized as a means to an end – “doing new thing X is prohibitively expensive on the old system.”  That opens the door to talking about goals.

* Couldn’t pass that one up.

Capturing Goals

On a recent project to migrate part of a company’s operations from one platform to another (system consolidation, after an acquisition), I created the following Ishikawa diagram to represent the goals for the migration project.

Even a “don’t change anything” project has real underlying goals.  Once you discover them, you open the door to having conversations about making things better.  Might be a “do it later” situation, but often, there is an opportunity to grab some of the low-hanging fruit during the system implementation.

The most important reason to capture the goals when “everyone already knows the goals” – aside from the fact that that is never true – is to make sure decisions are being made eyes-open.

Requirements live outside of the timeline of a particular project.  Once you’ve identified those requirements (aka market needs / company strategy), the next question is to determine which of those goals this project is intended to support or advance.  That question drives a lot of clarity into strategic thinking, and those answers drive a lot of clarity into project execution.

When you don’t share the context of the goals of the project, you are effectively blindfolding your team.  You prevent them from discovering opportunities to make things better.  You prevent them from making the right choice, when decisions are otherwise (without the context of goals) arbitrary.

For example, we talk about satisficing as a means to know when to ship – when it is “good enough.”  Without context, the opinion of “good enough” comes from someone on the team, without guidance.  In an agile environment, with self-directed teams, you’re making a trust-based decision to explicitly empower the team to make those decisions.  Don’t you want to give them some information, to help them make that decision?

Conclusion

This article really only explores part of the process of making real, large, projects happen in large environments.  That is a topic I’ll be talking about in a couple months.  You won’t get a sense of sated accomplishment from reading just this article – too much of the end-to-end, start-to-finish of real projects is unaddressed.

You will, however, know how to start the project.  Define the goals.  Someone will tell you the goal is to “Copy what the old system did.  We don’t have time to re-engineer.  Why are you bothering me?  Let me know if I need to find someone who can get the job done.”

Now you know how to respond.

**Attributions

Thanks foxtongue for the blindfolded typists photo

Estimating the “value” of features is a waste of time.  I was in a JAD session once where people argued about if the annoying beeping (audible on the conference line) was a smoke alarm or a fire alarm.  Yes, you can get to an answer, but so what?! The important thing is to solve the problem.

Solutions Versus Features

Everyone on that conference call had an immediate and visceral appreciation of the value of making the beeping stop.  That’s the power of solving a problem.  The methods of solving the problem – mute the offender, replace the battery, throw the alarm out the window – do not have implicit value.  They have an indirect value, in an “end justifies the means” kind of way.  But not direct value.

The same sort of thing applies when talking about prioritizing features.  Eric Krock (@voximate) just wrote a really good article, Per-Feature ROI Is (Usually) a Stupid Waste of Time, where he does two great things, and (barely) missed an opportunity for a hat trick.

The first great thing Eric did was look at the challenges of determining relative (ordinal or cardinal) value of “several things.”  He points out several real world challenges:

1. When you have a product with several things already and you want to determine the value of yet another thing – how do you allocate a portion of future revenue to the new thing versus the things you already have?
2. When thing A and thing B have to be delivered together, to realize value, how do you prioritize things A & B?  Relative to each other?
3. The opportunity cost of having your product manager do a valuation exercise on a bunch of things is high.  She could be doing more valuable things.
4. You won’t perform a retrospective on the accuracy of your valuation.  So you won’t know if it was a waste of time, and you won’t get better at future exercises.

The second great thing Eric did was reference a Tyner Blain article from early 2007 on measuring the costs of features.  I mean “great” on three levels.

1. As a joke (for folks who don’t know me, figured I’d mention that I’m kidding, just in case you get the wrong idea).
2. There is some good stuff in that earlier costing article about allocation of fixed and variable costs (with a handy reminder.
3. Eric’s article gives me an opportunity to shudder at the language I was using in 2007, see how much some of my thinking has evolved in four years, and improve a bit of it here and now.

What Eric slightly missed is the same thing I completely missed in 2007 – features don’t have inherent value.  Solutions to problems do have value.  He only slightly missed it because he got the problem manifestation right – it takes a lot of effort, for little reward, to spend time thinking about what features are worth.  I also missed the opportunity in an article looking at utility curves as an approach to estimating benefits, written two days after the one on cost allocation.  We were both so close!

People don’t buy features.  They buy solutions.

Valuing Solutions Instead of Features

Estimating the value of solutions addresses a lot of the real problems that Eric calls out.  It also has a side benefit of keeping your perspective outside-in versus inside-out.  Or as others often say, it keeps you “market driven.”

Anything that you’re doing, as a product manager, that has you focused on understanding your market and your customers and their problems is a good thing.  It may even be the most important thing.  I would contend that it eliminates objection 3 – the opportunity cost of estimating the value of solutions is minimal or zero.  There may be activities with more urgency, but off the top of my head, none that are more important, for a product manager.

Comment if I’m missing something (it’s late and I just got home from another week on the road).

The way I approach determining the value of a solution is by developing a point of view about how much incremental profit I will get when my product starts solving this additional problem.  Revenue can increase from additional sales, or from the ability to increase prices.  Cost can increase if new marketing and other operations (launches, PR campaigns, etc) are required to realize the incremental revenue.

I start with a customer-centric market model.

A given solution, or improved solution (as in “solves the problem better,” or “solves more of the problem”) – which only applies to some problems – is interesting to some customers, in some market segments.

A solution has value when it brings in incremental customers, in a targeted market segment.  It also has value when it reduces or prevents erosion of your current customer base (in a SaaS or maintenance-revenue model) to competitive solutions.

The time you spend thinking about buyer and user personas, the problems they care about, and the nature of those problems (which varies by persona) is not time wasted – or even spent “at the cost of doing something else.”

To make this useful, you have to have a forecast – without solution A, we will sell X; with solution A we will sell Y (and to whom).  A good product manager will be looking at sales, and will be able to reconcile the sales with the projections.  That helps with objection 4 (but doesn’t completely address it – you don’t know if your projections were accurate, so you can’t really know if your estimation is accurate).

This also helps you deal with challenge #1.  You’ve got a model that says “the current product works great for high school students, but not college students, because they also have problem A, which they solve today by…” Your intention is to create solution A, making your product viable to college students.  Allocate the incremental profits from college-student sales to solution A.

My approach to challenge #2 is a little more tactical.

Coupled Solutions

There are a couple ways that Eric’s “must deliver A and B” scenario are interesting, when looking at the value of solutions.

Scenario 1: Solution A solves part of problem X for persona M.  Solution B solves part of problem X for persona M.  Combined, they solve more of problem X for persona M.

This makes sense for “more is better” problems – where “more” solution yields “more” value.

In this case, I have a forecast (the more time I spend on it, the better it will be) that maps incremental sales to improved solutions.  The “first” solution to be released will have more value than the second.  If they are being released together, then I don’t care about the allocation – I combine them.

Scenario 2: If, however, the two solutions are valuable to different personas, then I treat them separately – even if they solve “the same problem,” it is not the same problem (for the same person).

Conclusion

Prioritization by “Bang For the Buck” is worth doing.  Just make sure you are prioritizing solutions, not features.

Also note: this article talked about valuation – what you do with that valuation, prioritizing by market, can be trickier.

Does product management slow down your company?

What Causes Your Business to Be Slow?

Paul Young put out the call for the third annual You Might Be A Product Manager… list.  If you are spending your holiday wondering if Jason Calacanis is right, and product management is actually preventing your company from being successful, you might be a product manager.

Facebook’s success — and mistakes — are based on its developer-driven culture, not because Zuckerberg is some evil mastermind.

The Zuckerberg Doctrine: Developers design products with significantly improved speed and functionality compared to product managers and designers, outweighing potential mistakes and drawbacks.

Jason Calacanis, Launch newsletter 002

Sound like link bait?  Maybe, but you’re probably a product manager :).  Jason’s put his money where his mouth is – he changed the way things are done at Mahalo (his human-curated search / answers company).

In under 30 days, we completely overhauled our product-development process, removing everything between the developer and iterating on the product.

We eliminated positions and process. We made it clear the developers were to make the decisions even if those decisions resulted in a developer being 50 percent slower because they were busy *thinking* about the product (as opposed to just transcribing features from the product manager wireframes).

Jason Calacanis, Launch newsletter 002

As you read through the details of the analysis in Mr. Calacanis’ newsletter, you’ll see that his position is not ultimately as generalized as the above quotes appear to be – Mr. Calacanis (who I’ve seen, and to whom I’ve listened for several years, but never met) is talking specifically about startups.  However, he uses AOL, Yahoo, MySpace and Google as his “bad” examples – not exactly startups.

I think this logical flaw may be leading Mr. Calacanis to demonize the wrong bad actors.

ProductCamp Austin 6

If you’re planning to attend ProductCamp Austin 6 – January 15th, 2011 (register here if you haven’t already), then consider attending a session that John Milburn, Roger Cauvin, and I are going to host – discussing this topic.  If you can’t attend, then jump into the conversation here.  John and Roger and I have been talking about this and exchanging some ideas since the newsletter came out – and we really know how much better ideas get when we open the conversation up to the community.  We’re looking forward to doing that at product camp, and I’m writing this to open it up now – so please, chime in!

Implicit Product Decisions

When I was still writing software and leading teams that were writing software, I would occasionally point out that all software is designed – even if someone sits down and just starts typing code.  There is, at the minimum, implicit design happening in the mind of the programmer.  It might not be “good” design, but there is always design.  There is always a method to the madness, just not always a considered method.  The same is true of product management.

• The creation of every feature and capability, in every product, is preceded by the notion that having this capability is a good idea.  That’s what product managers do – decide which capabilities a product should have.
• Eliminating product managers does not eliminate product management.

If “developer led” companies are still doing product management, how then, are they moving faster?  Mr. Calacanis addresses valid points – his “bad examples” do seem to move pretty slowly, and his “good examples” do seem to move faster.  So what’s really different?  John and Roger and I will be framing the discussion at ProductCamp around how to move faster, and not throw the baby out with the bathwater.

I might describe what Mahalo has apparently done as follows:

• Mahalo, with product managers involved in product decisions, was not moving as fast as Mr. Calacanis desired.  So they reorganized so that product managers were no longer involved in the process – in hopes of having a faster process.  Mr. Calacanis indicated that in a trade-off between “better” and “faster,” he would prefer “faster.”

Agility

The core of the agile development philosophy is – “fail fast. learn. improve.”

All of the other stuff in the implementations of agile comes back to this – consider the main ideas in the agile manifesto*

• People over process: empowerment to fail and learn and improve.
• Value working software: learning is experiential, and you can’t fail or improve without shipping.
• Collaboration: You have to understand someone else’s problem before you can solve it.  Too many products emerge from insular and isolated “exploration.”
• Encourage, don’t inhibit change:  If you punish failure you prevent learning. If you prevent that new knowledge from being applied, you make learning irrelevant.

*Agreed – those aren’t the words used in the manifesto.  Same ideas, though.

When you talk about business agility, while the mechanics might be different, the goals are the same.  Fail fast.  Learn.  Improve.

Mr. Calacanis, in my interpretation, is saying that this is exactly what he wants the Mahalo team to do.  I applaud that goal.

Does he have to “kill all the product managers” in order to infuse his company with (business) agility?

Is product management the antithesis of agility?  By definition, product management is still happening – just without product managers.  Maybe the product management process has room for improvement…

Market Driven

As a good product manager, you are market driven.  What does that mean to agility?

• Fail Fast.  Maybe you’re making decisions that delay launches until you know “the right product.”  That would hurt agility.
• Lean. Are you listening to your customers, and learning from them?  Great!
• Improve. Does what you’ve learned lead to trying something different, with a hypothesis that it will be better this time?

OK, so being market driven will help your company learn and improve.  That’s the up-side.  It may also enable (but not cause) some corporate dysfunction. If your organization punishes failure, or is afraid of mis-steps, and you’re market driven, you have already heard this conversation – don’t (schedule | design | release) until we get [feedback X] or [insight Y].

A tight coupling with your market is a powerful tool.  It can be used for good (learning) or evil (avoid failure).

Whew.  Good to know that being market driven is not the source of the problem.  An organization that is afraid of failure is the problem.

Mahalo’s experiment may work – Mr. Calacanis clearly intends to encourage the “fail fast” element.  So, when his developers are doing product management, they will have an opportunity to succeed by being market driven.

Bureaucracy

In fairness, Mr. Calacanis is really only prescribing the “no product managers” approach for startups.  Startups are not particularly bureaucratic.  Perhaps Mahalo was becoming bureaucratic.  It is easy to see the big companies mentioned in the newsletter as being rife with bureaucracy.  Even if you could fail fast, learn, and improve in a world of t-crossing and sign-offs, your definition of “fast” would not match your competitors.

Piloting your company would be like flying a Cessna twin-prop airplane in a world of super-sonic Gulfstream jets.

How are product managers introducing overhead into your product creation process?  What parts of product management are “not worth the delays?”

There are very real “slow things down” activities in the product creation process.

• Some activities can be removed.
• Some activities can be improved.
• Most activities can be done in parallel with the product creation process, eliminating delays.

What Have You Seen?

There are a lot of stories from the trenches out there – what are you seeing?  Have you tackled this already?  How did you make it better.

I hope that our session focuses on helping attendees, in a very real way, make their product creation process more effective, and make their businesses more agile.

Of course, we don’t have to wait until January 15th – we can start talking about it right now…