Writing Verifiable Requirements should be a rule that does not need to be written.  Everyone reading this has seen or created requirements that can not be verified.  The primary reason for writing requirements is to communicate to the team what they need to accomplish.  If you can’t verify that what the team delivered is acceptable, neither can the team.  This may be the most obvious of the rules of writing requirements – but it is ignored every day.

Verifiable Requirements – Revisiting

In 2006, I first looked at how and why writing verifiable requirements is important - as part of the ongoing series on the rules of writing good requirements.  The focus of that article was on testability, as is this one.  When visiting verifiability again, however, I will add that not only must specifications be objectively measurable, but so must goals be written so that you can identify when a solution has met the objectives that justified its creation.

Directly Verifiable Requirements

Most of the requirements we come across can be directly verified.  The only problem with these requirements is that they lack specificity.  The following example is almost verifiable, and only needs a little help:

• The web site must make it easier for users who use site search to find the products they want to buy.

This requirement presents the challenge of resolving the ambiguity of the requirement. This lack of specificity prevents the requirement from being verifiable.  You have to identify how much easier the site search process must become for users.  Findability is a more is better characteristic in the Kano Analysis model for defining requirements.

You have to acknowledge that making it easier to find what you’re looking for is better for users than making it harder – there is an upward slope to the function.  However, there are also diminishing returns.

[larger image]

• A search that includes “the perfect item” is massively better than a search that does not include what you are looking for.
• A search that includes the perfect item on the first page of results is significantly better than one that returns the perfect item on the second page of results.
• A search that includes the perfect item as the first result is only marginally better than a search that returns that perfect item as the second or third result.

Improving findability as articulated above, from a user perspective, manifests in value to your company in two ways: immediate revenue impact and indirect revenue impact.

The immediate impact can be measured in terms of increases in commerce.  The indirect impact results from improving the user experience.  These improvements in experience result in increased word of mouth, as users altruistically encourage others to visit your website, and also result in an increase in satisfaction causing users to return to your site more frequently and make more purchases from you.

You can measure these impacts in terms of

• Conversion percentage (percentage of people who search and then purchase the products found in the results)
• Revenue attributed to users who search (an absolute measurement of purchases of products found via search)
• Site traffic levels (the number of people that visit your site over time)
• Visitor-recency statistics (the amount of time that elapses between return visits for returning visitors)

Conversion percentage, for users who search, normalized against users who do not search, is the most isolated (from other variables and noise) and fastest responding (as a delayed measurement of impact) measurement of the value of making it “easier” for users to search for the products they want to buy.

Your team will design different approaches to achieving these improvements.  You have to estimate both the cost and the potential benefit of each approach.  Balancing cost estimates with potential benefits will yield the ideal requirement – perhaps a 10% improvement.  [Note: I've collapsed at least another article's worth of balancing cost versus benefit, and multiple articles of "understanding and measuring site search" into one paragraph here, in hopes of staying on task with writing verifiable requirements.]

Rewriting the requirement as follows makes the requirement verifiable:

• Before: “The web site must make it easier for users who use site search to find the products they want to buy.”
• After: “Users who search on the site will be at least 10% more successful at finding the products they want to buy when using site search.”

Impossible to Verify Requirements

Often, stakeholders will present us with requirements that are impossible to verify (as requested).

• The home page needs to load fast.

At first blush, this looks just like the previous requirement (easier search is similar to faster page load).  You can use the same techniques to determine a measurable “requirement” like “the home page needs to load in under 1 second.”  However, that’s not a good requirement, because it is not an implicitly valuable requirement.  This statement provides no insight into why a fast-loading home page might be valuable to a particular user or why it would be valuable to the company.

To address this issue, you have to meet with the stakeholder(s) and understand the actual underlying requirements.  You will ask why, determine motivations and underlying problems, and apply active listening skills to discover the underlying requirement.

The problem is that the statement, “the home page needs to load fast” is a specification.  In Requirements Documents – One Man’s Trash…, I first used the following diagram to show how different people in the process of designing software view their piece of providing a solution.

A developer may receive a spec – “home page must load in under 5 seconds” and feel like the why component is perfectly well defined – it is a matter of context and perspective.  Another way to think about this is that the problem is with the problem statement.

A product manager, however, needs to do research that follows a path like the following:

Given that we are building an eCommerce website, we know that people have expectations around page load times (per Forrester / Akamai report (requires registration)).

[larger image]

Further, those expectations manifest as people abandoning slow-loading websites (from the same report).

[larger image].

We know from this market research that page-load times (for websites) represent another more is better Kano attribute, but one that has must be characteristics at the extremes.

[larger image]

While more speed is better, what the market research reveals is that for any given person, there is a minimum-speed threshold, at which speed is a must-be characteristic – too slow, and you lose customers (immediately).

A  combination of market research and elicitation will reveal that there is a true underlying goal of being fast enough to satisfy as many users as possible.  Combining this with practicalities of scaling your solution, and the associated costs; in the context of a particular solution design (an eCommerce website), you will arrive at a goal that allows you to rework your requirement so that it is verifiable.

Note: The requirement in this example is a subordinate requirement to a goal focused on maximizing conversion percentage (the percentage of customers who visit the site making purchases) – with a recognition that a major source of non-conversion is abandonment of the site, and that a large contributor to visitor abandonment is page load times.  An Ishikawa (or other model) will help you articulate this complexity and formulate requirements at a level of abstraction that is both valuable and actionable.

Rewriting the requirement as follows makes the requirement verifiable:

• Before: “The home page needs to load fast.”
• After: “No more than 10% of potential site visitors will abandon our site before viewing the page.”

Indirectly Verifiable Requirements

Some requirements are impossible to literally verify, and must be inferentially verified.

• The site must support 1 million visitors per day, with a peak of 10,000 simultaneous with page-load times under 5 seconds.

Models are Models, Not Reality.

Imagine trying to create a test by organizing a million visitors to hit your SaaS web solution on the same day, with at least 10,000 of them hitting the site simultaneously.  That is completely impractical.  That doesn’t however, invalidate the requirement or make it untestable.  The combination of modeling, hypothesis formulation, and extrapolation gives you a reasonable way to verify that your solution probably meets this requirement.

You’re already used to the concept that models are representations of reality – think about the maps you use – a map may have a scale like one inch equals one mile.  The map is a model of reality.  A map where one inch equals one inch would be impractical.

An effective approach to measuring this type of scalability requirement is to simulate users (with load testing and realistic user-behavior scripts) with automation.  That takes care of most of the coordination problem.  However, the cost of simulating a million users and 10,000 simultaneous users is high.  You can, more realistically, measure the site’s performance when 10, 100, and 1,000 simulated users simultaneously access a server.  You can extrapolate from those results to estimate how the system is likely to perform under 10,000 user loads.

Note: Extrapolation is dangerous (follow the Elvis link below)- you have to justify why extrapolation holds true, and identify when it does not, to minimize the risk of invalidating your hypothesis.  Make sure you have confidence in the engineering prowess of whoever is doing your extrapolation and modeling.

[see tip #5 of these ROI calculation tips for the Infinite Elvis anti-pattern]

While this requirement does not to be rewritten in order to be verifiable, it does need to be augmented:

• Original: “The site must support 1 million visitors per day, with a peak of 10,000 simultaneous with page-load times under 5 seconds.”
• Additional: “Testing of the site must show that 500 simultaneous users following <user script X> will yield no more than 1% page load times over 200 ms.”

Conclusion

Every requirement you write must be verifiable.  When you can’t verify something, and can’t rewrite it into a verifiable form, that should be a sign that either it is a vision statement or a red herring.  Vision statements guide how we approach creating products and engage markets – they are valuable, but they are not requirements.  Red herrings are well-meaning but ill-advised inputs into the process that need to be culled – they are neither valuable nor requirements.

Make sure all your requirements are verifiable.

People who already use Scrum will only find one new thing in this article – a way to communicate what happens inside a sprint that has proven effective for me.  People who are new to Scrum who wonder “how do things work inside a sprint?” will see how things work in a way that avoids hyperbole and is easy to map to what they already understand from traditional software development processes.

Two Teams Separated by a Common Process

George Bernard Shaw was an Irish author in London who memorably said “England and America are two countries separated by a common language.”  The same can be said about teams developing in agile and waterfall processes.

The story of agile adoption is one that has many colorful examples of adoption and of spreading the gospel.  Some practitioners traveled across the ocean from the agile continent to the lands of agile and opportunity to reap the rewards.  A few of those, after realizing the benefits of agile, invade our consciousness with a hellfire and brimstone tale of doom for anyone who doesn’t convert to the new religion.

Others seem hell-bent on kidnapping entire development teams, smuggling them across the ocean in the belly of steamer ships and unceremoniously dumping them in the land of agile, ready or not.  Scrum is but one branch of the agile movement.

Stalwarts of the old way, which has worked fine for me, thank you very much, refuse to leave their comfortable lives to step into the unknown wilds of agile.  Open-minded but responsible potential converts ask questions to gain an understanding of what life in the new world might be like.  What will life be like if I join this Scrum congregation?

How Does Scrum Really Work?

The best explanations of how scrum works that I’ve read come from Mike Cohn and Mountain Goat Software.  The training, videos, and explanations they share provide fantastic top-down introductions (as well as guidance after adopting Scrum).  His book, User Stories Applied, is the ultimate reference when gaining an understanding of the mechanics of using user stories as the central artifacts for developing software with scrum.

[larger image]

The process diagram above provides a good outside-in, top-down view of how “this newfangled process” results in shippable product, incrementally.  It is a great way to present the concept of Scrum, and incremental development in general.

When people I’ve worked with first gain this understanding, they often ask, what are the artifacts that live in these backlogs?  User stories live there.  Sometimes use cases make more sense than user stories for Scrum- it depends (read the article to see which works best in your circumstance).  The common element is an outside-in, user-centric approach to describing which problems the software is intended to solve.  This is also known as goal-driven development.

Once people understand the top-down view of the process, and have an idea of what user stories are, the next question they tend to ask is “how do stories move through the Scrum process?”

The first attempt at answering this question leans on the process diagram above:

1. The product owner takes the most important stories from the product backlog and puts them into the sprint backlog – collaborating with the team to agree on the scope of work for this sprint (based on the amount of work, capacity of the team, real-world constraints, etc).
2. The team “works the stories“, meeting every day to communicate effectively and provide progress updates – best visualized with a burndown chart.  This chart says “we started with X work to do, and every day, we track how much of X remains to be done.”
3. At the end of the sprint, the work is done, and the software (or an update to the software) is ready to deliver.
4. Sometimes, multiple sprints happen between releases – launches of the updated software, because customers (and the rest of your organization) incur a cost when changing to the latest version.  But the output of each sprint is deliverable – that’s a key concept – even if you choose not to deliver it just yet.

What Does it Mean to Work The Stories?

I’ve repeatedly had people ask, after absorbing the above explanation, “what does it mean to work the stories?”

I’ve had consistent success (in bridging the language divide) using the following diagrams (drawn on a whiteboard) to explain how stories flow through the sprint process in Scrum.

The first diagram shows that user stories have a structure – the story itself, and the acceptance criteria for the story. I also establish that the story is going to go through a life cycle within the sprint.

[larger image]

Next I remind folks that the user stories make it into the sprint backlog because they are the most important (highest priority) not-yet-implemented stories in the product backlog.

[larger image]

Developers on the team pull stories from the sprint backlog and begin working on them.  These stories are work in progress, and are owned by individuals.  Some teams maintain that priority order within the sprint backlog (e.g. implement the most important story in the sprint first, etc.), while other teams use a more coarse-grained approach, relying on the product backlog prioritization to assure that everything in this sprint is the next most important stuff, and the order that the team implements, within the sprint, is up to the team.  Some folks like to strenuously debate this decision (don’t prioritize within sprints vs. do prioritize within sprints).  Both approaches are valid, and you should choose the right one for your team.

[larger image]

As each user story owner believes that her work is complete (all unit tests pass, and the developer believes that the acceptance criteria have been met), the story is ready to be tested by QA.  This step, in particular, helps people who are new to scrum and who think of things as “code and test”, to make some sense of the inner workings of a sprint.  When working with a team that was under-staffed in the QA department, I discovered that calling out this queue of user stories waiting to be tested helped convince management to increase QA investment in the team.  When too many stories pile up here, you know you have a problem.

[larger image]

The scrum team member responsible for testing the user story has the responsibility of assuring that it meets all of the acceptance criteria.  He pulls the story from the queue of “done” stories, and tests.

[larger image]

If the story fails to meet any of the acceptance criteria, it moves back into either the sprint backlog or the being developed column, where a member of the team resumes ownership of the user story and works to resolve the issue.  Once the issue is resolved, the user story is re-queued for testing.  When the user story meets the acceptance criteria, it is moved into the done done column.

[larger image]

Conclusion

If you are new to Scrum, and wondered how the sausage is made inside a sprint, this gives you a framework for understanding what is going on, and how the team delivers.  If you already know or use Scrum, you may have learned a couple things.

• Using this framework to describe how a sprint works is effective when explaining to people who do not have any experience with agile.
• Laying things out this way visually, will give you an easy-to-see signal if your team is unbalanced between developers and testers.

I’ve been involved with agile teams and processes for just over 11 years now, and incremental development is burned into my brain.  If you are new to scrum, please let me know if this seemed like a good way for you to consume this material.  If you’re an old hat, and have other suggestions or presentations, please add them in the comments below, because future readers will benefit from the additional ideas.

Writing unambiguous requirements is about understanding what is written, and what is read.  Without a clear understanding of your market, you can’t write unambiguously.  Even when you understand your market, you risk writing something that is ambiguous to your readers.  Documenting requirements is about communication.  Don’t break this rule, or you’ve wasted all the energy you spent understanding your requirements.

Unambiguous Requirements – Revisiting

Fifty months and five days ago (another grin), I wrote about the importance of writing requirements so that they can be clearly understood.  Writing requirements so that there is one, unambiguous, interpretation of what you wrote.  This article is the seventh in a series on the rules of writing good requirements.  In that version of the article, I focused primarily on language ambiguity, and framed the discussion in terms of ambiguity to the writer and to the reader.

As part of improving that series, I am revisiting each of the articles now that I have fewer hairs and the remaining ones are more gray.  The same framework still applies, but will be expanded in scope, using the following modified definitions of the framework.

• Ambiguity for the Writer – not having a clear interpretation of the requirements.
• Ambiguity for the Reader – requirements that, as written, can be interpreted in more than one way.

It takes two to tango.  To lead, you have to know where you’re going.  To follow, you have to know where you’re being led.  The same is true with requirements – you have to know what you’re writing, and how your writing will be read.  Misinterpretation of requirements is the source of over 40% of all software defects.

Ambiguity for the Writer

When you don’t know why you’re developing a particular set of requirements, it is because you either don’t have sufficient understanding of your market, or of your corporate objectives.  While it is always a lack of knowledge, sometimes, that lack of understanding is insidiously hidden.  You, as the writer of requirements, are the reader of the market – and what you’re reading is likely to be ambiguous.

For Whom You Are Designing

The most overlooked element I’ve personally seen in product creation is failing to understand who the users and buyers of your product will be.  I wanted to type “forgetting to understand,” but I struggle to imagine someone who understands the value of defining user personas and buyer personas, who then forgets to define them when gaining an understanding of their market.

The reason that this is an ambiguity issue is that when you fail to define personas explicitly, you define them implicitly.  An implicit persona is an ambiguous persona.  Alan Cooper named this anti-pattern the elastic user, and the problem manifestation as failing to provide insight to guide design decisions.  When you have an ambiguous user, you lose what Frederick Brookes calls the clarity of design. This results in at best a mediocre product without a core idea or aesthetic.

Different users value the same solutions differently.  Users vary in competency, and therefore have varied personal goals as they learn how to better use your product over time.  When you don’t identify who the user is, you prioritize requirements without insight into what is important to the undefined, elastic, and ambiguous user.

The readers of your requirements are trusting that you will prioritize requirements correctly – but you can’t, because you lack an understanding of who the users will be.  You therefore lack an understanding of which problems are valuable, and which problems are more or less valuable than other problems.  This ambiguity makes it particularly difficult to write valuable requirements.

Precision in Language

The other aspect of writing unambiguously is in using language that has meaning.  Using words like “can” and “may,” is imprecise in requirements.  ”The system can respond in under three seconds.”  Does that mean that it must respond in under three seconds?  Does that mean it is acceptable if the system responds in twenty seconds?  This is an area of overlap with the rule of writing measurable, verifiable requirements, so I won’t go into more details here.

Another source of imprecision in language, not only for the reader, but also for the writer, is jargon.  You should never use jargon, as you have no way to know if your reader will actually interpret the jargon terms correctly – as readers often vary in the depth of their domain knowledge.  Jargon is also a high risk of ambiguity source, precisely because it is jargon.  Jargon is short-hand for communicating efficiently among domain experts.  Are you enough of a domain expert that you are confident that you absolutely, unambiguously understand the accepted meaning of the jargon term?

Hopefully, you’ve achieved a perfect understanding of your market.   Long-time readers will know that I’m joking.  Market analysis suffers from it’s own Heisenberg Uncertainty Principle – if you actually did manage a perfect understanding, the market will have changed – the best you can hope for is a statistical understanding of the behavior of your market.  You can document your interpretations, assumptions, and decisions – achieving clarity in your requirements – at least from your point of view.

Ambiguity for the Readers

As I mentioned earlier – using requirements is a dance – and it takes two, both the writer and the reader.  You need to write requirements so that they can only be interpreted in a single way, or you will stumble.

“You keep using that word.  I do not think it means what you think it means.” – Inigo Montoya (IMDB)

Shared Language

The word “shall” as in “the system shall…” can be misinterpreted when your readers don’t share the same native language as you – which is why you must always use “must.“

Some good research on vocabulary size data for comprehension of english can be found here. The average native english speaker knows 20,000 word families. With a vocabulary of 5,000 english words, only 98.5% of the words in a given text will be understood. 5,000 words is a lot for a speaker of a second language (3,000 words is considered aworking knowledge).

An analysis of the reading-level at which a document is written can be helpfull in identifying if the language is likely to be challenging for readers with limited vocabularies. The Gunning-Fog Index provides a measure of the education level at which a text is written.

Writing Unambiguous Requirements (2006)

Can’t write that section any better :).

Ambiguous Use of Language

I had the privilege to attend a presentation by professor Daniel M. Berry in 2007, on the ambiguous use of language in business rules and requirements.  In that presentation he referenced The Ambiguity Handbook that he authored (80 page pdf), which I just re-read last week.  There are many good examples of ambiguity in the use of English in business documents.

The following are some of the sources of ambiguity, for which I entirely credit professor Berry for pointing out to me.  I’ve only added some illustrative examples:

• Plurality Causes Ambiguity.  Consider “Emails must include sender addresses” and “emails must include recipient addresses.”  Must each email include one sender address and one recipient address?  Must each email include multiple sender addresses?  Solution – don’t use plural subjects.  ”Each email must include a sender address” and “each email must include recipient addresses.”
• Associative Ambiguity.  Professor Berry calls this the parenthesis problem.   What does “A and B or C” mean to you?  Does it mean “A, and either of B or C” or does it mean “either C, or both A and B” when you read it?  In math and programming, we learn very specific rules for how to interpret these types of structures.  We are also given parentheses, that we can use when we want to be specific and unambiguous.  The reader of your requirements is not a compiler, so don’t assume that she will interpret this the same way that you do.  Solution – use parentheses and explicit language to eliminate ambiguity that would arise from ignorance of the associative property.
• Anaphor Ambiguity.  For non-linguists: “don’t use pronouns.”  I love the example that wikipedia uses – “We gave the bananas to the monkeys because they were hungry.  We gave the bananas to the monkeys because they were ripe.
  
  We gave the bananas to the monkeys because they were here.”  The pronoun, “they,” has an ambiguous binding.  Should “they” be a reference to the bananas or the monkeys?  In each of the first two sentences, you could reasonably assume that the reader will properly bind “they” to the monkeys and bananas respectively.  In the third sentence, an assumption of how the reader will interpret “they” is not reasonable.  Reasonable assumptions are still ambiguous – and the context is less likely to be obvious – so don’t use pronouns and rely on readers to bind them to the appropriate nouns.  Solution – repeat the noun for every reference.

Incompleteness Ambiguity

Unwritten requirements are by definition ambiguous.  You must not assume that unstated requirements will be implemented.

A security exploit was recently discovered (and eliminated) in Facebook.  Previously, if you attempted to login to Facebook with an email address and a password that did not match the email address, Facebook would present you with a typical “failed login” screen and a link to reset your password.  However, in an effort to be friendly, Facebook would also include the first name and last name that were associated with that email account in the friendly warning message.

That is nice, except it is a privacy problem, and anyone who knew about this before could use the login page to discover the first and last name to associate with any email address.  ”Bad people” with email address lists could discover the name to associate with many of those email addresses – allowing more sophisticated phishing or social engineering attacks.

The requirement to not provide any personal information when a user fails a login attempt was almost certainly an unwritten requirement.

Conclusion

Precision in your writing will reduce ambiguity in your requirements.  Active listening will help you discover when ambiguity has slipped through your editing net.  You can never completely eliminate this problem, but you can improve.  You must.

Accept has invited me to participate in their webinar series on Transparency and Innovation – this Wednesday, July 28, 2010 (10AM Pacific, 1PM Eastern).

Join us and join in!

The Importance of Innovation and Transparency

Cool, huh? :)

Accept is hosting a free 5-part series on transparency, and has invited me to be one of their speakers.

The other speakers are rock stars – Nils Davis & Alex Lobba [replay available], Tom Grant, Roy Wildeman, and Brian Lawley.

Each of us will be looking at the importance of innovation and transparency from a different perspective.  Combine that with encouraged audience participation, and we should really be able to tease out some powerful ideas!

Join Us at 10AM Pacific on Wednesday, July 28th, 2010

I’ll be talking about how transparency (of the needs of your customers) affects the entire product creation team – not just product managers.  Register today for free, and I hope to get some great questions from you.

Thanks!

What happens when billionaire media magnate, Rupert Murdoch, pits his idea against a Nobel-prize winning idea from the beautiful mind of economist and mathematician John Nash?

When you act on what you hope your market will do, instead of what you predict your market will do – you’re in trouble.

This is a story about understanding your market, and an example of using game theory – specifically, the Nash Equilibrium in “non-cooperative game theory” to predict market responses to your products.

Murdoch’s Idea

To sum up the particular idea I’m talking about – Mr. Murdoch has put the Times of London behind a pay-wall (you have to subscribe to access the content online), and removed the articles from Google’s search index.  He did this because he believes that people accessing the content for free are “stealing it” and that the only way to make money in the news business online is by forcing people to pay for content.  For more nuanced and deeper discussions of what Mr. Murdoch is doing, check out these articles from The Atlantic explaining what he’s attempting, the Editor & Times report of a 66% drop in traffic, Newser’s investigative journalism of what’s happening inside the company, or Business Insider’s take on the matter.

My opinion is that Mr. Murdoch expects his market will respond the way he wants them to – by subscribing to the online version of the Times of London – in stark contrast both to pundit predictions and what professor Nash’s Nash Equilibrium model of non-cooperative gaming predicts will happen.

As results have finally started surfacing, they appear to completely support what game-theory would predict and utterly refute what Mr. Murdoch would have hoped for.

Game Theory for Predicting Market Behavior

I was enthralled by Scott Steven’s lecture series, Games People Play: Game Theory in Life, Business, and Beyond – a DVD set of a series of lectures from The Teaching Company [link goes to Amazon].

Understanding your market involves not only knowing what problems your customers face, but also predicting how your competitors will behave.  Competitive analysis is not just capturing a snap-shot of their products and positioning today, but also forming predictions of how they will respond to the disruptions you will create in your market by innovating.  Markets are not static – you have to understand both how your customers’ needs will evolve and how your competitor’s offerings will change, in order to understand how your product will perform.

One of the “games” in game theory is called The Coordination Game, and it is the one that captures Mr. Murdoch’s conundrum.  The coordination piece of this game is in acknowledging that what you do (to pay-wall or not to pay-wall), and what your competitors do (pay-wall or not) are interdependent decisions, of which you only control your own decision.

You can’t (successfully) unilaterally decide what you are going to do, without taking into account what your competition is going to do.  And vice-versa.  That’s the part that makes it hard – there is a circular reference that has to be resolved.  I suspect that’s the source of elegance in the Nash Equilibrium – he realized that there are “stable” and instable combinations of the decisions – yours, and your competitors.

The Pay-wall Coordination Game

Mr. Murdoch and “everyone else” (his competition for consumers of news) are both faced with a choice – should they allow free access to the articles, or not.  Free access to articles can generate revenue by two mechanisms.

The first is one of the freemium business models – billing Peter to pay for Paul – charging money to advertisers for access to viewer’s attention.

The second mechanic is by leveraging word of mouth – people reading your free article, then encouraging others to read your other articles.  This approach can either be used to get a percentage of people as paying customers (with a “leaky pay-wall” model that allows some free views, but encourages paid-subscriptions), or to get additional traffic that increases revenue from advertising.  One way to maximize this approach is by making your product implicitly viral so that it promotes itself.

Alternately, Mr. Murdoch, or any of his competitors can create a pay-wall, where only paying subscribers will have access to the content.  Presumably, certainly historically, a subscription model with the same level of readership – if that could be achieved – would generate more revenue for the publisher than either freemium model.

The crux is that we have tons of market data to show that “the same level of readership” simply will not happen.

There are four possible scenarios that can happen in this market -

1. Both Mr. Murdoch and “everyone else” institute pay-walls, making all online content available only to paying subscribers.
2. Mr. Murdoch erects a pay-wall, but everyone else continues to allow free access to content.
3. Mr. Murdoch allows free access to content while everyone else erects pay-walls to prohibit access to content.
4. No one erects a pay-wall, and free access is allowed to all content.

[Update 2010.07.21: Why "everyone?"  See comment #2 below]

In scenario 1, assuming people don’t just stop reading online, everyone wins – everyone charges subscription prices, achieving the maximum amount of revenue.  So why didn’t this happen?  Because the decisions are interdependent, and because the market is starting out in scenario 4.

Mr. Murdoch has moved the market (from his perspective) into scenario 2, which is clearly bad for him, in hopes that everyone else will move as well, to reach scenario 1.  Once Mr. Murdoch put the Times of London into scenario 2, then everyone else’s best move is to stay there.  No one else will join him, because everyone else never will.  The market, once stabilized in scenario 4 (the starting point) can never move to scenario 1 – without collusion, which is conveniently illegal.

Someone will always be there to refuse to move to scenario 1.  ”Everyone else” is destined to allow free access, or exit the market.

Understanding Your Customers

It is possible that Mr. Murdoch looked at the Wall Street Journal, which has a pay-wall, and said “I want to do that too.”  And perhaps he discounted all of this game-theory “nonsense” because if it worked for them, it can work for him – the model is BS.

However, I believe the Journal’s market is different, and it works for them for different reasons.  The market for business news pays a premium for immediacy and quality of information – because getting the best information first is worth a lot of money to them.  There are “free” immediate sources of business news information, but the market indicates that it perceives that information to be of lower quality.  And there are free high-quality information sources, without the immediacy.

The reality, which you can only get from understanding your customer’s goals – is that the business-news market is stable in scenario 1. It could be destabilized, by someone offering free, timely, high quality information.  If that happened, as professor Nash showed, the market would either re-stabilize in scenario 1 (the company that changes says “oops” and reverts their policy), or the market would transition to a stable state in scenario 4 (everyone offers free content).

The more concentrated the market [see: measuring market concentration], the more likely a reversion to scenario 1.  The more competitive the market, the more likely that it will stabilize in scenario 3.

Non-business news on the internet?  Very competitive.  And stable in scenario 4, no matter what Mr. Murdoch might hope.  Sadly, there are profits to be made in scenario 4, but they aren’t as appealing as the double-rainbow of scenario 1 – however impossible it is to achieve.

Attributions

The two photos at the start of this article of Mr.s Murdoch and Nash are licensed through the Creative Commons Attribution Share-Alike 2.0 Generic and the Attribution Share-Alike 3.0 Unported licenses, respectively.

All other content in this article is licensed per the Creative Commons link at the bottom of this page.

Everyone who thinks about what it takes to create great products needs to read The Design of Design: Essays from a Computer Scientist.  Frederick P. Brooks, Jr., author of The Mythical Man-Month, has released this inspiring collection of essays about the nature of design.  By Brooks’ definition, design includes the notion of intent and definition of goals, requirements, and implementation choices.  Everyone responsible for creating products, but with fewer than Brooks’ five decades of experience really needs to read The Design of Design.

Disclosures

My first disclosure is that I received a complimentary review copy of The Design of Design.  If you feel that that biases my review, stop reading now, and come back for the next article.

My second disclosure is that I receive a lot of complimentary book offers (and books), and while I read and review them, I only publicly review the books here when I believe you will benefit from reading the book.  In almost five years of blogging, this is only the eighth book review published here.  When I don’t care for a book, I don’t publish a negative review – I simply opt out.

First Impression

In the forward of The Design of Design, Brooks references J.R.R. Tolkein – “J.R.R. Tolkein suggests that God gave us the gift of sub-creation, as a gift, just for our joy.”  This from a man with massive experience in the science, engineering, and architecture domains.  With his acknowledgement of the essence of sub-creation, Brooks got my attention, and he never lost it.

What Brooks truly has a gift for is distilling insight from the vapor of real-world projects.  He is incredibly effective at presenting his (and others’) ideas, constructs, and perspectives.  He succeeds with analogies, logical analysis, anecdotal examples, and an incomprehensible number of references to market data and analyses thereof.

Brooks is probably a polymath, and certainly a Renaissance man.  He puts the “agile versus waterfall” (as a design process) debate in perspective by framing it as a variant of the contrast in point of view between empiricist John Locke and rationalist Rene’ Descartes.  He draws on contemporary, recent, and ancient references and perspectives to reveal insights and support his positions.

The Design of Design

The book itself is a collection of essays – each is a chapter in the book – that flows logically, ideas building upon previous discussions.  He almost creates a story arc as he progresses through his topics – starting by helping you get in the right frame of mind as you leap down the rabbit hole of deep thinking about the creation of products while holding his hand.

Brooks has both affirmed my experiences in product creation and challenged my conclusions and biases.  I read an essay and a I feel a mixture of empowerment, encouragement and humility.

Another thing Brooks does really well is focus on the important ideas (great design comes from great designers, not great design processes), while avoiding entanglement in the pedantic (should we label desiderata as requirements or designs).

I fully expect several future articles here (and many of my future work products) to be heavily influenced by Brooks.  His most significant ideas are meaty, and discussing any one of them would distract from a review of the collection.  For now, I’ll just list some of the areas of thought and ideas that he explores directly or indirectly:

• Models for the product creation process – choosing the right problems to solve, inventing, and implementing a solution.
• Finding the “right” design process – or if there even can be a “right” process.
• Agile versus waterfall – empiricism versus rationalism.
• Elegance of design and the importance of understanding your users.
• Collaboration in design, distributed teams, and the impacts of corporate culture and policy on design.
• Constraints on design – how they can empower and how they can neuter designers.
• Studying exemplars of design – learning from the past, with an emphasis on understanding why design choices were made.

He also provides what seems to be hundreds of fantastic references – I’ll never work through the queue on my nightstand now!

Dave West has also reviewed professor Brooks’ book, and included a set of follow-up questions and answers with professor Brooks in his article on InfoQ

Conclusion

In the first 200 pages – 100 “two page spreads” – I have 35 post-it notes in the book  to flag important ideas.  It is the rare book that gets a dozen such markers.  The professional books that I enjoy are the ones that present ideas that encourage me to pursue new avenues of thought.  The Design of Design inspires me to not only do that, but re-think my definition of “avenue.”  I can’t recommend this book highly enough.

Thank you, professor Brooks, for writing it!

As product managers, we talk about creating the right solutions with our products. Understanding the very real problems our customers face, understanding the very real opportunities our markets present, and manifesting that understanding in a product roadmap.

Other than being “not as good,” how expensive is it to build the wrong product?

The Cost of Poor Quality

There’s an analog to the market dynamics of making poor product decisions – executing with poor quality. Many research studies and articles have identified the market impacts of poor quality.  This has become so well accepted that people today cite it like a law of physics (one example here based on this 1988 IEEE research by Barry Boehm and Philip Papaccio) as the “1-10-100 rule.”  The primary conclusion of that research is that ten dollars spent on fixing bugs:

• Costs and saves $10 when you catch (and fix) the bug during implementation.
• Avoids $100 in costs when you catch the bug during QA and send the product back to development (then test again).
• Avoids $1,000 in costs versus waiting until your customers catch the bug in the field, causing the team to remedy the problems, rush out a patch release, and/or go to heroic lengths to manage a PR problem.

This is an opportunity in front of your product team – a 100x payback from investing in quality during the development process.  Of course, be pragmatic about it - if the cost of testing exceeds the cost of bugs, don’t test.

This is not a solved problem, by any stretch, but the solutions and methods to solve this problem are well understood now.  In fact, a 2001 article by Barry Boehm and Victor Basili shows that in some cases, the labor costs to resolve bugs can be as low as 5:1 – when considering a subset of smaller systems, when using more “agile” processes.  That lowered ratio does not take into account the lost market opportunities and the costs of cleaning up collateral damage to your product – just the immediately realizable (and measurable) costs of resolution.

One very real problem, when talking about “bugs” is in defining what a “bug” is. And the definition of a bug is a matter of perspective. A developer can reasonably assert that “if it meets the spec it is not a bug, it is working as designed.” What if the spec is wrong? The developer may not be guilty, but collectively, your team screwed up. There’s a “bug” in the requirements.

What Is A Requirements Bug?

Now things are getting interesting.

If you wrote a requirement that you interpret as “A” and your developers interpret as “B” – you definitely have a bug – the team won’t build the right product. For each $1 you could spend making sure you have bug-free requirements, you could:

• Make sure you have a shared understanding of the documented requirements through active listening before development begins ($1). Following the Rules of Writing Requirements will help prevent this miscommunication.
• Wait until the engineering team is ready to demo their progress ($10). They will have to build it again, because they built the wrong stuff.
• Wait until development is complete and QA is validating that the code meets the spec ($100). This gets tricky if you are thinking “A”, the developers are thinking “B”, and QA is thinking “C.”
• In classic throw-it-over-the-wall mode, wait until the product is launched, and it is the wrong product ($1000). Assuming “A” was the right problem to solve, the cost of entering the market with a solution to “B”, leaving “A” unaddressed, is impressively high.

This gets interesting because the above assumes that “A” was the right problem to solve. What if “G” was the right problem to solve, and “A” was the wrong market problem? Even if everything (else) is working perfectly – you document requirements for “A”, the engineering team creates a marvelous “A” and it launches without implementation errors – you still fail, and incur the 1,000x cost of a failed product launch.

There is an even larger opportunity in front of your product team – a 1,000x payback on discovering and choosing to solve the right problems for your customers and markets.

• Would Palm still be independent if the Pre had solved a compelling problem?
• Why did Intuit have to buy Mint.com – could they have embraced the same customers with Quicken?
• What is Garmin going to do now that “free” GPS mapping and turn-by-turn directions are becoming ubiquitous? If it is “more of the same,” how much are they wasting?

Wrapping Up

I’m not aware of any studies that show that “requirements bugs” fit the same 1/10/100/1000 cost explosion model that “implementation bugs” exhibit. Emotionally, it “feels about right” to me – it passes my “sniff test.”

There are times when days of research would have been required to avoid or redirect a few hours of implementation effort on projects I’ve worked on. And I’ve seen man-years invested solving problems that didn’t involve much more research.

My intuition from products and teams I’ve worked with is that it probably averages out somewhere around 10x.

What does your gut (or your data – if you have some, post a link below!) tell you?

Most companies ignore their markets – and they will struggle to survive.  Some companies listen to their markets – and they have an opportunity to succeed.  You have the opportunity to understand your market, and transform it into your market – but you can’t get there just by listening.

Don’t listen to your market, understand your market.

Zappos All-Hands Meeting

Thanks Thomas Knoll (@thomasknoll) for pointing out that Zappos’ all-hands meeting was being live streamed today!  [Aside - how cool is that?!]

Chip Conley (@chipconley) spoke to the Zappos (and Bellagio) folks about several things, including being customer-driven.  He was able to pull several ideas from his book, Peak: How Great Companies Get Their Mojo from Maslow, but one of them stuck out in particular as being critically important to product managers.

Transformation Pyramid

There’s a copy of this on slide 28 of this presentation on Slideshare, essentially an adaptation of Maslow’s hierarchy of needs, applied to customer relationships.  It is not clear if this is an approved (by Mr. Conley) posting, so I’m not going to include a fair-rights copy of the image.  You can find out more by reading his book, visiting his website, or dive right into an application of this model to Wall Street from Mr. Conley’s blog.

The basic idea (for focusing on your customers) boils down to this:

1. At the base of the pyramid, you have companies that meet the expectations of their customers- they provide what customers expect (and no more).  In his presentation today, Mr. Conley pointed out that these companies will struggle for survival.  Good enough isn’t anymore.
2. At the next level of enlightenment are companies who meet the desires of their customers.  These are companies Mr. Conley points out as ones who will succeed.
3. At the top of the pyramid are companies who meet the unrecognized needs of their customers.  These companies, through understanding their customers, satisfy needs that people didn’t know they had.

In an earlier article on sustaining a market-driven competitive advantage, I wrote about listening to your customers (#2) and innovating in order to uncover the latent requirements.

You can argue that these problems were introduced because of innovative solutions, or that the problems were latent, and were only discovered because of the innovations.  It doesn’t matter.  What matters is that these problems were previously unaddressable, and now solutions to these problems have value.

That article was written with an inside-out perspective – essentially answering two questions -

• “Why should you innovate?”

• “Why Should You Innovate Repeatedly?”

As you repeatedly invest in understanding your customers, you gain insight into their latent requirements and needs – and get an opportunity to address them.  When you do this repeatedly, you become a thought leader for your markets, which you can leverage as a distinct competitive advantage.

Conclusion

Mr. Conley is approaching innovation, with respect to customers, from an outside-in perspective – and reaching a similar conclusion.

When you understand your customers, well beyond meeting their must-have minimum expectations, and beyond addressing their explicitly stated desires, you have the opportunity to solve the problems they don’t realize that they had in the first place.

From my perspective, his insights make it even more important that you continually focus on what your customers really need, and let your market guide you.  Not to be confused with “give your customers what they ask for” – they’ll only ask for a faster horse.

“For what one idea do you want your product to stand in the mind of your customer?”  I heard Roger Cauvin ask that question at the most recent ProductCamp Austin [correction - he said it here - thanks Roger], and the quote has been jumping to the front of my mind almost daily ever since.  Maybe by writing about it I can exorcise the demon and get back to using the idea instead of being haunted by it.

The One Idea

You’ve just been given a new assignment – your company needs a new software thingamajig so that you can play in the whatchamacallit space, and you’re going to drive product management for it.  You’ve also been asked to deliver the first version in six weeks.  Of course you’ll be able to do follow-on releases, after all, you are agile.  That’s how “agile” works, isn’t it?

Cool.  Exciting.  Challenging.

You sit down with the folks who will be building the product, and find out that they have already spoken with several stakeholders.  Excellent. You circle back with the stakeholders, and start to gather data about your market and audience.

Your schedule is aggressive, so you think about what is realistic to get done in a very small timebox.  With the current schedule, you can’t afford to get caught in analysis paralysis, and you can’t realistically do in depth market analysis, persona development, hyper-accurate requirements prioritization, etc.  You do, however, have time to do the most important thing – define the one idea that will define your product.

If you cannot come up with this idea, you need to push back on your management team, and delay the launch of your product until you have that one idea.

Don’t Abuse Agile

Agile is designed to help you rapidly create iteratively better products, with iterative development and continuous infusion of market feedback and data.  Agile is not a process by which you start typing without any idea of what you intend, releasing it and then getting feedback in an iterative process.  If that’s how you’re approaching agile, your process is broken.

[image from Market Driven Competitive Advantage]

Unless you’re very lucky, your first iteration will be a waste of time and money.  Wouldn’t you rather be intentional than lucky? Make sure you have that one idea before you start developing.

Strategic Alignment

You need to come up with a one idea that is aligned with and supports your corporate strategy.  At a minimum, you have to make sure that you have an idea that is aligned with your stakeholder’s goals, based on the assumption that those goals are aligned with corporate strategy.

Wow

Your one idea really needs to make your users say “Wow!”

I was explaining the importance of this to a colleague (who is not a product manager), and his response was “Oh, you mean the hook.”

I was reminded that you not only have to provide a wow experience for your users, but you have to present a hook that will capture the imagination of your buyers.  Remember that buyer personas make decisions based on their perceptions of user problems [more on convincing buyers].

Satisfice

Remember that you don’t want to try and do everything perfectly in the first release – you want to satisfice.  You may only be able to target a single persona with a solution to a single problem.  You just need to make sure you are solving the right problem – which is where Kano analysis is very handy for identifying the Minimum Market Acceptance criteria.

[From Minimum Market Acceptance]

Conclusion: A Balancing Act

Defining the one idea for your product is a balancing act.

• You have to align the one idea with your corporate strategy and stakeholder goals
• You have to solve a valuable problem for your users, presented in a way that captivates your buyers
• You have to create a product that is compelling right away – good enough in execution to enter your market
• And you have to get the first release out in six weeks!

Good luck!  Or better yet – Good Intent!

Consistency in writing requirements is important on two levels – strategic and tactical.  Tactically, you need to write your requirements with grammatical consistency, so that potentially ambiguous statements will be interpreted similarly.  You also need to write requirements that are logically consistent, so that you avoid “impossible” requirements and gaps of unspecified meaning.   Strategically, your requirements need to reflect a focus on markets and problems that are consistent with your business objectives and the vision your company is manifesting

Consistent Requirements – Revisiting

Thirteen hundred ninety-six days ago (grin), I wrote about the importance of grammatical and logical consistency in writing requirements – an emphasis on the tactical aspects of consistency in writing requirements.  This article adds a brief discussion on the consistency of the requirements you write.  This article is the sixth update of a series of articles on the rules of writing requirements.

Grammatical Consistency

[source]

Writing requirements is not like other forms of writing in one key way – you don’t want to vary the terms and language you use in your requirements documents.  Remember the critiques you received when you studied writing in school for a moment.  Your teacher told you not to repeatedly use the same word, but to use synonyms for that word – walked, ambled, stepped, capered, etc.  Those synonyms allowed you to create imagery and helped your readers avoid boredom.  Your teacher encouraged you to use metaphors that created even stronger imagery – flew, slogged, muddled, danced, etc.  Those metaphors allowed you to infuse emotion into your narrative and helped your writing come to life for your readers.

As Yoda said, “Unlearn, you must!”

Requirements are not a narrative.  Your primary goal is not entertainment, it is enlightenment.  Don’t use varying terms to describe the same action or object – use the same terms over and over again.  Don’t vary the modifiers of terms except when you are explicitly varying the context and referenced items – and then make certain that you use the same modifiers consistently for the same context and references.  When dealing with anything that has some complexity, provide a figure or diagram (possibly in an appendix or glossary) that explains the explicit meanings of each term and term-modifier.

For example, if you are defining requirements for a classification system for authored works, be specific about the definitions of author, editor, publisher, and reviewer.  You may also need to define a term that is used collectively for all of them, like creators.

You must consistently use the same voice  - ideally the active voice.  And of course, you must not use ‘the system shall.’ [There's a great debate in the comments on that article - one thing to note - all of the people on both sides of the debate emphatically expressed their consistent use of one form or the other.]

Logical Consistency

[source]

Logical inconsistency most commonly comes from expressing the same requirement twice.  If you use the exact same words to express a requirement twice, you’re being redundant and repetitive (grin #2).  Most likely, the second expression of the same requirement is both unintended, and worded differently – running the risk that it will be interpreted differently, and therefore inconsistently.

You will also create logically inconsistent requirements when you express mutually exclusive requirements.  These can be directly incompatible, like writing “the system must infer the value…” and “the user must provide the value…”   Or they can be indirectly inconsistent like “the animal must bark” and “the animal must be a cat” – which is logically inconsistent because cats cannot bark (thereby making at least one of the requirements infeasible).

Strategic Consistency

One of the elements that defines valuable requirements is that they “support your business strategy.”  In the context of value, that means you have created a decomposition model for how you intend to realize value – manifesting a strategy or a component of a strategy.

A strategy, however is not always defined by a single measure.  Your strategy could include financial metrics and targets, as well as a stated mission regarding customer service levels, and a long-term investment that is hoped to eventually return long-term rewards.  Your requirements, while not only completely defining the mechanisms for achieving value, must also be consistent with the other elements of your strategy.

For example, you should not take advantage of uninformed costumers when you have a core value of integrity – regardless of how effective that may be at meeting financial goals.  The requirements that articulated the (subordinate) goal of grifting your customers (to maximize revenue) would be inconsistent with the goal of operating with integrity.

Conclusion

Consistent Requirements are requirements that

• Are not in conflict with any of your strategic objectives, vision, or corporate goals
• Avoid logical inconsistencies
• Utilize consistent structure, terms, and grammar

April Dunford just presented Startup Marketing 101 at DemoCamp Toronto.  Great ideas from the ‘marketing and your startup’ point of view.  I’ve often said that product managers and product marketers care about much of the same market data, they just do different things with it.  The idea of minimal feature set came up in April’s presentation – this article talks about product management, agile, and initial market acceptance.

Initial Market Acceptance and Minimum Market Acceptance

April mentions (in slide 4) that an important event to the timing of marketing activities is achieving the “_minimal feature set (for certain markets)_.”  April organizes those marketing investments in terms of three stages of “product marketing lifecycle”:

1. What to do when your startup has a “pre-product” product.
2. Where to focus when you’re focused on early adopters.
3. How to invest in your marketing programs when your focus is on scale / growth.

Coming from an agile background, I think about two distinct notions of “minimum” or “initial” when it comes products and markets.  The following definitions are in a B2B context, as I’ve been focusing on this recently with a client in the B2B space.  The same ideas are relevant to B2C and B2B2C companies, but the language would be slightly different.

I’m assuming that when April used the phrase “minimal feature set”, she was really talking about “minimum market acceptance” in the way that I’m describing here.  The distinction is that features describe what a product does, which is an inside-out view of the product.  To be market focused, you have to think about which market problems are being solved, for whom they are solved, and how well they are being solved – an outside-in product view.

If you focus on, and organize around your features, you are likely to miss the mark.  You must focus on the problems that your customers need to solve.  In your product, you will prioritize the development of capabilities (embodied through features) that are designed to help your customers (buyers and users) solve their problems.  Your market analysis should be geared around understanding how many customers in each segment face each problem, and how much they are willing to pay to solve those problems.

Agile or Waterfall or Waterfragile?

As an agile product manager, and former developer, I know that when you do “everything we need to be successful” in the first release, you’re not being agile – we’re being waterfall.  A slight extension of this is “everything we need to be minimally successful” – and that is still waterfall.

An agile team should focus on the first release addressing the initial market acceptance criteria – the minimal set for a single customer.  The goal is to get the product in front of customers as soon as possible – this starts your revenue stream earlier, gives you valuable market feedback, and gives you the opportunity to establish momentum through repeated incremental releases of your product.  Each release will solve addressed problems “better” and / or address additional problems, until you reach minimum market acceptance.

Kano Analysis for Understanding Problems

Last year, I was thrilled to share my approach for applying Kano Analysis as a product manager with the PMV webinar audience as well as the Austin IIBA chapter.  One of the key ideas in the application of Kano Analysis to understanding your market is developing the personas within that market, and understanding how each of them treats each problem / capability.

[slideshare presentation & PMV webinar]

This analysis, in addition to being useful for understanding the market (or a segment) as a whole, also helps you identify your first customer.  Once you know who your first customer is, you can determine the initial market acceptance criteria for that customer, and that determines the must have capabilities.

Note that finding the initial solution (that the first customer will buy) does not mean releasing a poor quality product – it just means releasing a product that solves enough problems (well enough) for one customer.  With the feedback from that customer, you can drive the prioritization for your next iteration – making the next iteration better for that customer (which can really help your word of mouth), and gaining more customers.  Repeat this process until you get to minimum market acceptance.

Timing Marketing Investments for Minimum Market Acceptance

[slide 9 from April's presentation on slideshare]

Since I wasn’t at April’s presentation, I don’t know exactly what she would think of the following, but I believe it makes sense:

• Initial Market Scceptance (one customer would buy your product) marks the transition from pre-product to early adopters.
• Minimum Market Acceptance (enough to succeed in the market) happens before the transition from early adopters to scale.  Note that it may not (probably doesn’t) mark the transition, but I suspect happens mid-stream.

Would love to see what other folks think about mapping this product management centric view to the marketing timeline.  Please chime in below.

ProductCamp Austin is here again!  The Spring 2010 session is this Saturday, 27 March 2010 at the AT&T Conference Center on the UT campus in downtown Austin.  Make sure and say hi when you’re there!

ProductCamp

ProductCamp is an unconference for product managers and product marketing professionals.  This is the fourth time Austin has had one – we were fast followers of the wildly successful Silicon Valley PCamp in 2008, which Rich Mironov was instrumental in getting off the ground and growing.  Paul Young is the guy who brought together the people in Austin to get our events off the ground.  Paul graciously deflects the credit for getting things going, but he deserves it regardless.

Check out Paul’s great description of ProductCamp. Here’s the key paragraph, if you’re on the fence about attending:

The sessions and networking are the main reason that people keep coming back. Over three ProductCamps, we’ve achieved over 98% of participants answering “yes” to the questions “would you come back to ProductCamp” and “would you recommend ProductCamp to a peer?” This high customer satisfaction is why our local community loves to come out – in droves. Last time, we had over 500 sign up and over 300 participate. This time we are limited by our venue and won’t be able to hold more than 300 or so, so register today and reserve your space. It’s free of course.

Sign Up Now

Today (Thursday) is the last chance to sign up, to give folks a few minutes to finalize planning for space, make name tags, etc.

You can sign up at the official website for PCamp Austin.  The doors open at 8:30, things get started at 9:00, sessions are from 10-4 (with provided lunch at noon), then awards and a happy hour at 4:30.

View Larger Map

Sessions

41 Sessions have been proposed for this ProductCamp, so make sure and come, vote for your favorites, then enjoy them!

Leaf Launch

Very exciting news for this PCamp – Paul Young is launching his new product, Leaf!

From Paul: “Information overload is a problem that every product manager has.  Between email, phone, IM, tweets, status updates, geolocations, and keeping track of who knows what, just communicating can feel like a full-time job.  Social networking has amplified the problem – we now have access to more information than ever before – but how’s that working out for us?”

Does that sound like you?  You should attend his session, then.  I certainly will.

See you there!

The Laudi Group and Red Canary organized and shared a great set of questions for product managers and answers from a panel of product management leaders.  Steve Johnson, another leader in our space shared his answers to the same questions, and in this article, I share mine.

The Product Manager Questions

Hat tip to Steve Johnson at Pragmatic Marketing, for extending the discussion and providing his answers to the questions.  And thanks to the folks at Red Canary and the product managers who originally shared their answers.

I really enjoyed reading their answers – thanks to all of you!

Here are the questions that were put forth and answered:

1. Tell us about the best product you’ve ever encountered.  Why do you like it?
2. How do you know a great product manager when you meet one?
3. What’s your favorite interview question?
4. When is the best time for a start-up to hire a product manager?
5. What has been the defining moment in your career?
6. Mistakes.  What was your biggest?

I’ve personally enjoyed and grown from the great writing that many product managers have shared with us over the last few years.  I’d love it if they would share their answers.  So, either share your answers, or pester your favorite writers to share theirs.

Without further ado, here are my answers to the same questions.

Why Do You Like the Best Product You’ve Ever Encountered?

I love when products solve obvious (in hindsight) problems with elegant designs.  The products where, once they exist, you say “well, duh” or slap your head and ask “why didn’t I think of that?”  These innovative products tend to be disruptive, redefining markets.  Some of the products are rather mundane – like the ketchup bottle where the lid doubles as the base (reducing waste and preventing countless red-water-stained buns).  Other products are more technology-dependent – like Tesla’s radio, xerographic photocopying, solid-fuel rockets.

I think my favorite for elegant design might be Velcro – if you don’t know how it works, or just want to know how it was invented, check out the wikipedia article.  The time that parents save tying tiny shoes is value enough, but it is far from the only (or first) use of Velcro.  An accurate clock that could be used on board a ship was a biggie too, although I never personally encountered one, at least when it mattered.  Accurate time-telling on a ship was the key to dramatically improved navigation back in the days of sailing by the stars.

How Do You Know a Great Product Manager?

Great product managers are polymaths, with several areas of deep expertise and skill.  They are Renaissance women and men, with many areas of interest.  They are great communicators.

Most importantly, they are sooth-sayers.  By the last, I mean that they not only understand the big picture and context of their markets and their business, but they know what is likely to change their business, and where their markets are sensitive to or ripe for disruption.  One definition of sooth-sayer is “one who tells the truth.”  You can’t do that without data, and the ability to understand what the data is telling you.

Add to this a dash of humility and a full dose of open-mindedness, and you have a great product manager.

Of course there’s all of the esoteric skills that the role requires.  When they aren’t present yet, I feel like I’ve met a great product manager to be.

I know it when our conversation rambles all over, goes “depth charge deep” in areas, and then bounces back to the broad view, all with an eye for the relevance and insights that matter for the topic at hand.

What’s  Your Favorite Interview Question?

“Why?”

I almost don’t care what the context of that question is – reviewing a candidate’s previous experience, asking them to provide a “fresh view” on my current situation, or convincing them to dance through a hypothetical situation.  What I want to know is why they think there is value, or a problem , or an opportunity, or … whatever.  A collection of well-dispersed, and sometimes-immediately-sequential “Why?” questions can tell me more about how someone thinks, and more importantly, how they are likely to solve problems, create solutions, and dominate markets than any other question I’ve found.

When is the Best Time for a Start-Up to Hire a Product Manager?

Not counting the founder?

Three to six months before the first product peaks.  All successful start-ups have one good product – solving a single valuable problem, for a single market.  Most (my opinion / observation) start-ups don’t have a second good product or market.  A passionate and insightful founder can spend a long time understanding a market, a problem, and a solution.  That knowledge and passion can yield a successful product.  When is that founder, who is busy running the company, going to find a new problem to solve, a new market to dominate, or a new solution to replace his original idea?

Alternately, I guess the founder can hire someone else to run the company, and anoint herself “president of the product.”  That still counts as hiring a product manager.

What Has Been the Defining Moment in [My] Career?

Switching from electro-mechanical design engineering to software development.  The shift in innovation time scales, the different approach to problem solving, and the markedly different economics of product creation had a profound effect on me.  The evolution from “creating solutions to (defined) problems” to “identifying problems worth solving” was more gradual, as I evolved into a programmer-analyst and consultant and product manager.

“Going agile” half-way through my software development career was pretty eye opening too.  I’ll throw that in as my backup answer.

Mistakes.  What Was [My] Biggest?

From someone else’s perspective, it was leading a team down a six-figure software-development path that solved a problem no one really cared about.  That’s probably defining-moment number 3, when I started including validation of value as part of my scope of engagement as a programmer.

From my own perspective, as they say, it’s the train you don’t see that hits you.  I’ll guess that it was something I didn’t do, not something I did, that would turn out to be the key plot device in my Frank Capra movie.

One of my colleagues got into a debate with one of his colleagues about the differences between goals and requirements.  His opponent fired the following salvo: “[That] is not a business requirement in any company of the world…”

What you call your requirements is less important than how you communicate them.

Labels

I’ve worked with a lot of companies that use different labels to describe their descriptions of the problems that need to be solved. This is not another article about requirements versus specification (specification = design constraint, btw).

If you’re not a long time reader who’s followed the debates and discussions on this topic in the past, check out these articles and comments, where we’ve had some great discussions on that topic – with particular thanks to Roger Cauvin for keeping the debate front and center.

• Requirements Versus Design: Which is Which and Why
• Requirements Docs – One Man’s Trash is…

The central theme is that the debate about “what versus why” centers around differences in people’s perspective on the problem.

This challenge is not really about that (but it makes for good background).  The complaint my colleague received was really a critique of the way that the why of a set of requests was communicated and labeled.

Actionable Goals

Companies have strategic goals (increase shareholder value, gain market share, etc) that serve to guide the investments and activities of the organization.  However, for the teams that are creating products, those goals are too vague to be actionable.  I guess this is where my set of labels comes in.  I don’t think the labeling of a goal as a goal (versus labeling it as a requirement) is abstractly important, however, when people value making a distinction, here’s how I do it.

A goal is something your business (or user) is trying to achieve.  It is too abstract to have a single valid solution approach.  Once your business stakeholder(s) decide on a strategy by which they will achieve that goal, the business will define requirements for the actionable project(s) that execute this strategy.  This is an ideation step that lives inside the “why” box from a business analyst’s perspective.  Here’s an example:

Warehouse Manager

You’re in the IT department of a large company.  You’ve been assigned to support the warehouse manager (or COO, or vice president of order fulfillment or whatever).  Your warehouse manager is responsible for procuring the products that your company sells, keeping inventory on hand, and delivering those products to customers when they have been purchased.

Your warehouse manager is measured (or promoted) based on her performance relative to the following:

• The time that a customer has to wait between placing an order and receiving the product.
• The cost impact of warehouse operations on the products.

Your warehouse manager focuses on these two metrics because of your company’s goals – provide good customer service, be profitable, and grow revenue.  Those abstract goals influence product pricing – it must be low enough to grow revenue in the target markets, and be high enough to be profitable.  Since market pressures drive pricing decisions, typically a profitability target will then create cost-reduction pressures (profitability also influences pricing decisions, selection of markets, positioning within that market, etc).  A customer service strategy can also include a “you get your product quickly” component – as it does in this example.

Your warehouse manager now comes to you and says “I have an initiative to lower costs.”

Is this a goal or is it a business requirement?

Ultimately, it doesn’t matter what you call it, but for you to actually do something you need to know more.  That’s why my approach is to call it a goal.  Your warehouse manager’s problem is to lower costs.  You need to decompose that problem to understand how you (and your team) can do something to help her achieve that goal.  The reason you have to do that is that you need a problem statement at the right level of abstraction.

In elicitation conversations with your warehouse manager, you learn that all of the costs of operating the warehouse are allocated to the different business units, who operate as profit centers.  The warehouse operates as a cost center.  The warehouse manager treats each business unit as a customer, and provides a service of “fulfilling product orders” in exchange for funds that cover the costs of operation.  The allocation of costs is done per product that appears in the warehouse.  Since allocation is per product, it could be something like $1.30 per product in “warehouse costs.”  There are multiple sources of those costs (that are then allocated per product), and there are multiple ways to approach reducing the cost per product.

Your warehouse manager develops a strategy (probably a “mini-strategy” from the company’s perspective) to lowering costs.  She rules out “sell more products” as an approach, because she can’t control that.  She also can not affect her fixed costs (rent, depreciation, labor).

Eventually you discover that there is another cost hidden in the provisioning model – products are purchased, they sit in the warehouse for a while, then they are sold and delivered, and eventually money is collected from customers to pay for the products.  Your company has to pay in advance for the products, and doesn’t get their money back until funds are collected from the customers.  That money is “tied up in inventory” when it could have been invested otherwise – so there is a cost associated with high inventory levels, that directly correlates with the amount of inventory in stock.  When a new version / revision of a product is introduced, your inventory may become obsolete (it is definitely devalued).  And since markets are moving targets, your prospects of selling a product change over time – a risk that you will not be able to sell all of your inventory.

At the end of the day, the amount of inventory you have on hand has a cost.  The more you have, the more it costs.  A rule of thumb I heard 20 years ago was that inventory costs could be as high as 1/3 of the cost of the product for each year you keep it around.  If that is true, that is something like 0.5% added cost for every week the product is in inventory.  Here’s something the warehouse manager can potentially control.

Currently, your warehouse manager has 8 weeks of inventory on hand (on average) for products, which allows her to ship quickly.  She realizes she is adding 4% to the cost of the product.  She decides that if she can cut those costs in half, she will be a heroine.

Her business requirement is to reduce inventory levels from 8 weeks (on average) to 4 weeks (on average).  She also has a constraint that no more than 1% of orders be delayed by more than a day (due to running out of inventory).

Now you have requirements that are unambiguous and measurable.

Conclusion

Don’t trap yourself in worrying about the labeling of goals / requirements, but if you have to work with someone who does find that really important, before you call it a requirement, make sure it is at the right level of abstraction.  And of course, make sure you follow the rules of writing requirements.

I came across a really interesting article LukeW.com, showing how making changes to the way an input form on a website increased interaction by 25 to 40%. The changes reflect the value of thinking outside-in, investing in user experience, and performance measurement.

Bonus: the idea is cool.

Mad Libs Input Form

[full size image available in Luke's article]

The idea being presented is to replace the old boring web input form designed for a computer. The new, fun form is a fill-in-the-blank (aka Mad Libs) layout. The article is on Luke Wroblewski’s site.  The team at Vast.com created and tested a version of this form for the Kelly Blue Book site. [Luke cites Huffduffer.com as the first place where he saw this technique.] Thanks, Luke for sharing this with all of us!

Outside-In Development

Product managers are acutely aware of the need to solve market problems. We are regularly reminded that we’re creating solutions for people in our markets who use our products to solve their problems. When we write requirements, we avoid writing design specifications, and thereby lose the ability to enforce that the proposed solutions also take an outside-in approach. We maintain an outside-in perspective, but we lost the ability to influence an outside-in aesthetic.

Note: Outside-In Software Development is a great book, you should read it.

That’s one area where user experience works well in concert with product management – assuring that the same drivers of what to build are informing the design of how it is built.

The genius (or at least elegance) of this Mad Libs form from Vast / Kelly Blue Book is that it humanizes the experience of requesting that a nearby dealer contact you to discuss a possible purchase of a car from them. The forms we’ve all had to fill out on countless web sites are very inside-out. They expose the inner workings of the program – “gather this info, and that info, and this other info.” Those forms force users to do what the program wants. Blech. The form that Ron Kurti designed, however, forces the program to do what the users want. Huzzah!

Bad Requirements Prevent Elegance

I’ve written about the importance of writing design-free requirements several times.  Avoiding design constraints in your requirements is even one of the pillars of the big 10 rules of writing requirements.

If you had written the requirements for this contact the dealer page, would you have specified the design of the form?

Most of the requirements analysts I’ve worked with would have. I’ve even worked with companies that have developed templates / stencils defining how these interface specifications must be documented – requiring that all the fields be aligned, with specific pixel gaps, etc.

If you had specified the design (read: limited the choices of the designer), you would have prevented this solution.

Mr. Kurti’s elegant solution is clearly better.

One way to write the requirements that would not have prevented this solution would be with a user story and acceptance criteria:

As a car-shopper, I want to engage a car dealer, once every few years, so that I can purchase the car I selected.

• The car-shopper can provide his contact info (name, email, phone) for the car dealer.
• The system will display the information identifying the selected vehicle.
• The system will display the information identifying the dealer being contacted.
• The system will include an opt-in newsletter-subscription option.
• The system will notify the specified dealer when the car-shopper indicates.

The user story nudges the developers into an outside-in perspective by emphasizing the user’s goal. The acceptance criteria make it clear that any solution that meets those criteria is acceptable to the product manager. It allows the interaction designer to create a compelling solution, rather than forcing him to recreate a boring experience.

Measurement Rocks

The Vast team measured the impact of making this change to their form, and saw between a 25% and 40% increase in conversion (people contacting dealers versus people abandoning the process when they see the form). That’s a lot more leads for the dealers, and presumably a lot more money for Kelly Blue Book and Vast. This is a great example of how you can measure the impact of investing in user experience, because it should spark ideas for you.

What can you measure and test and improve?

Now that’s ROI for you.

You give your requirements to the engineering team, and they look complete.  The team builds your product, you launch it and the market soundly rejects it.  Why?  Because your requirements weren’t complete – they didn’t actually solve the problem that needed to be solved.

Complete Requirements – Revisiting

Going back almost four years, I wrote Writing Complete Requirements, as part of the Big Rules of Writing Requirements series.  That article centered on two key ideas of requirements completeness.

• Data, not opinion, is the most important input into determining completeness.
• There is no absolute way to determine the completeness of your requirements in advance.

Completeness is best measured by market acceptance, so technically you can’t know if your requirements were complete until your customers buy (or fail to buy) your product.  This doesn’t mean you should give up on this rule of requirements – you can still focus on, and improve, the completeness of your requirements.

Objective and Heuristic Assessment

Given a market problem and the associated requirements (which you can argue are actually the same thing), you can review those requirements to determine if they objectively appear to be complete.  This assessment is based on what you objectively know about your market (needs, opportunity costs, competitive alternatives, etc).  You can also apply heuristic, or logical analysis to identify gaps in the language of your requirements.

Objective assessment is the application of market knowledge (data) to determine if the nature of the problem is being addressed completely by your requirements.  A bird-feeding maven would know that the requirements for your bird feeder would be incomplete if they didn’t address the problem of squirrels stealing food from the birds.

Heuristic analysis is the identifications of logical omissions in your requirements, without the need to apply market insights.  A logician would know that the requirements for your bird feeder were incomplete if they didn’t address the need to refill an empty feeder.

When you’re writing requirements, you need to do both.  Heuristic analysis of your requirements will identify where your requirements do not fully solve the problems you already know about – an eCommerce checkout process without the ability to receive payment, for example.  Objective assessment will help you identify the problems that you overlooked but need to solve – the need to allow customers to have different billing and shipping addresses when placing an order online, for example.

Complete Requirements for Incomplete Solutions

Don’t confuse having complete requirements (a good thing) with completely solving a problem (not always a good thing).  In agile development, the idea of satisficing is key to scoping individual iterations.  The idea, simply put, is to solve enough of the problem, and not delay delivery (and value) until you can build a solution to the entire problem.  Because of the law of diminishing returns, for many market problems, there is little or no incremental value in solving the entire problem.  Those resources can be better applied to solving additional problems.

Kano analysis provides a framework for identifying which market problems must be completely solved, and which ones have a more is better characteristic.  In practice, more is better problems exhibit the law of diminishing returns, and should not be “completely” solved.

[larger image, or view the entire Kano analysis presentation]

Conclusion

Complete Requirements are requirements that

• Identify all of the market problems that need to be solved and their nature (absolute vs. diminishing returns).
• Are logically complete in their coverage / articulation of those market needs.

To objectively improve the completeness of your requirements, apply market data in an assessment of your requirements.  To heuristically improve the completeness of your requirements, look for logical inconsistencies and gaps in reasoning or coverage.

Engagement – that’s what this whole product management blogging thing is about.  Check out what Tyner Blain readers found to be the most engaging articles in 2009.

Deep Dives

If you’re new to Tyner Blain, you may be surprised by the length of the articles here.  I joke that they are long because I don’t have time to edit.  Stewart Rogers jokes that they are long because I’m incapable of writing a short article.  If you’ve been here a while, you know what you’re in for.  If you’ve been here a long while, then you’re glad (like me) that I don’t write one per day any more.

Product Management is simultaneously a broad and deep discipline, requiring us to have a breadth of perspective combined with a depth of insight.  We then have to apply that in a market context, effectively navigating the political waters of our organizations.  Most of the articles here either try to skim the breadth of a range of related topics, or plumb the depths of a single topic.  Doing that in under a thousand words is pretty hard.  Most of the articles here also link to other articles, to try and provide even more depth and context, and encourage additional critical thinking.

One measure of the quality of the articles here is how often they stimulate readers to read further, or dive deeper into the topics.  While web analytics won’t allow us to measure how thought-provoking an article is, we can look to see how many people dive into the linked articles, versus how many people move on to something else.  We can measure the bounce rate of an article to see how often people leave a page without following any of the “tell me more” links.

Looking at ~170,000 page views at Tyner Blain in 2009, narrowed down to only those articles with at least 100 page views, here is the top-ten list of most engaging (or “least abandoned” if you’re a “cup is half empty” person) articles:

1. Use Case Series: Introduction: A collection of articles on the “traditional” forms of use cases – informal, formal, UML.
2. Agile Development of Use Cases: A dive into the dynamics and cadence of an agile process for developing use cases.
3. How Do You Manage Market Data?: A collection of articles exploring different market-immersion techniques in depth.
4. Scheduling Requirements Changes – Part 2: A focus on practical techniques for managing change with an agile development process.
5. Yesterday’s BPMN Post Was A Big Fat Lie: A mea culpa and clarification of interest to folks following the BPMN Series.
6. Everything I Needed To Know I Forgot in Kindergarten: Why asking why? is important.
7. Plan Your Next Sprint by ROI – Part 1: Prioritizing by Bang for the Buck, not just Bang.
8. Ten Common Mistakes of Going Agile: A collection of articles about common pitfalls encountered when adopting agile practices.
9. Perpetually Almost Finished Projects: Organizing a project with discrete deliverables and rolling-wave planning to avoid the “90% done, 90% remaining” problem.
10. How To Use Timeboxes for Scheduling Software Delivery: One of my personal favorites.  A rational approach to making trade-offs when your “perfect” plan has to change.

OK Stewart, that’s only 519 words.

Why does cross-selling, the process of selling something additional to someone who is already making a purchase, work?  This article explores some of the theory and rationale behind cross-selling – from qualification to motivation and profitability.

Cross-Selling is Big Business

You have an eCommerce site where people purchase products from you.  Adding cross-sale capabilities to your site can have a significant impact on your bottom line.

The e-tailing group’s 2009 report shows (by survey of eCommerce executives) that 55% of online retailers will include cross-sell and upsell capabilities in their sites in 2009 (already there or being added).  For retailers that measure the data, cross-sale promotions result in a range of conversions (additional sales) from under 1% to over 10% – with a plurality of responses in the 1% to 4% range.  That represents additional revenue the retailers would not get without using cross-selling.

According to the Get Elastic blog, Amazon reported that cross-selling accounted for 35% of their 2006 revenue.

Understanding why cross-selling works requires understanding customer’s purchasing processes work.

Customer Purchase Process

The following is a model for how customers make purchases.

• Customers start by thinking about their problem – what problem are they trying to solve (with a purchase)?
• Some people will try and understand the problem space, while others are happy to jump to solutions.
• Understanding the solution space (what are my options?) is next – and may be a shallow or deep analysis.
• Within the solution space, customers will select a product and decide to buy it (from you) or not.
• Customers who reject their first product choice may select another product or may abandon your store.

You can take a higher-level view of these customer activities and decisions, and categorize them into areas:

• Learning - Your customer is learning about her problem and possible solutions.
• Choosing - Your customer is comparing possible solutions, with the intent to purchase one of them.
• Buying - Your customer has selected a solution, and is trying to buy it.

Customer Qualification

In direct sales, one of the first steps a sales rep will take is to qualify a prospective customer – how likely is this prospect to make a purchase?  A similar model can be applied, when treating your website as a sales rep, to understand how likely it is that a visitor to your website will make a purchase.  There is a ton of additional qualification (assessing a prospect’s ability to pay, for example) that we won’t talk about in this article.  In this article, we’ll focus just on this simplified purchasing model:

• Customers who are learning are more likely to buy than visitors who are browsing (window shopping).
• Customers who are choosing are more likely to buy than customers who are learning.
• Customers who are buying are more likely to buy than customers who are choosing.

That last bullet seems silly – of course customers who are buying are more likely to buy – like 100% likely, right?

Wrong.

Prospective customers abandon the buying process all of the time.  SeeWhy reported ~70% abandonment rates during 2009’s post-Thanksgiving online shopping spree.  They also used the phrase “a relatively healthy 63%” to describe abandonment rates in August 2009.  SeeWhy is specifically measuring abandonment of the shopping cart (inside the “buying” area), but there is abandonment (often called leakage) throughout the process above.

The further a customer is into the purchase process, the more likely they are to actually make that purchase.

Clearing Hurdles

As a customer moves through the purchase process, they are clearing hurdles that would prevent them from making the purchase.  Each time they clear a hurdle, the pending “mental cost” of making the purchase gets smaller.

One of the hurdles is making a decision to purchase now, another is making the decision to purchase from you.

Offering cross-sale promotions to customers who have already (probably) decided to purchase from you, now, means that you’re offering the promotions to customers who are qualified.

Relevance

A defining element of a cross-sale is relevance.  You’ve got a customer who is purchasing a product, to solve her problem.  You want to increase the value of the transaction – both for your customer and for yourself.  To do that, you have to offer a relevant cross-sale item.  Selling french fries with a sandwich makes sense.  Selling car wax with a new car makes sense.

Selling car wax with a sandwich?  You probably won’t have a lot of success with that.

How do you determine relevance?  You can start out with an “expert opinion” – car wax is relevant to cars, for example.  Or you can do some data mining of past orders – “7% of people who bought cars also bought car wax.”  That lets you start out with a hypothesis (that car wax is a relevant cross-sell item for purchasers of cars).  If you’re data mining past orders, however, you may not know the direction of the cross-sell.  Cross-selling works because the two products are complementary goods – usually asymmetric complements.

Complementary goods are rarely symmetrical.  Peanut butter and jelly is a good example of symmetric complements – they have comparable price points, and both generally can be improved by purchase of the other.  In the mixer-cookbook example above, the cookbook is a great complement product for the mixer.

However, if your customer had selected the book initially, the encouragement to “add a stand mixer” would fall on deaf ears.

Foundation Series: Substitutes and Complements

You can measure behavior on your site to determine the nature of the complementary goods relationship.  For the orders that include two particular products (that are offered as cross-sale promotions for each other), in what percentage of orders with both products was each product selected first?  Alternately, what is the conversion % of product A when added to a transaction for product B, versus the conversion % for product B when added to a transaction for product A.

Measurement of Cross Selling

The obvious metric that most people think of when evaluating cross-sale promotion effectiveness is conversion rate.  Conversion rate identifies the percentage of people who, when buying the “original” product, choose to also buy the “promoted” product.  Making changes that raise your conversion rate increase your sales of the promoted product.  However, this conversion ratio is more a reflection of the relevance of the promoted product to the original product than it is a measure of profitability.

Your goal is to maximize profitability, while providing additional value to customers (who are better off or more satisfied with the original purchase when they also purchase the promoted item).

Introducing a cross-sale promotion can increase, decrease, or have no effect on the rate of purchase (conversion percentage) of the original product.  You need to measure the original-product conversion rate for customers who were shown the cross-selling promotion versus those that were not.

Second, you’ll want to know what the ideal products to promote are.  Typically, more than one cross-sale promotion is presented to a customer at a time.  For this example, assume 3 promotions are displayed.  Also assume that your data-mining exercise has identified 5 products that could be promoted as cross-selling items for this “original” product.  Which three of the five do you select?

You should select the three that you expect to be the most profitable.

“Most profitable” can be calculated as (profit per promoted item) x (conversion % – of the promoted item in the context of the original item).  When you don’t have conversion percentage data, you can either gather it (through testing) or predict it (through modeling).  There are many aproaches for predicting the degree of affinity, or implied relevance, of one product to another – but all of them are too detailed to cover in a blog article.  When you don’t have a model, you can test the possibilities to see which ones perform the best.

Some companies also report average order value (AOV), but that’s not necessarily an indicator of profitability.  It may be a component of profitability, but not necessarily.

Product Managing Cross-Selling

If you’re product managing your website, and exploring adding cross-selling capabilities, or enhancing current capabilities, here are some things to keep in mind:

• Your customers are in the middle of a buying process, and will be interested only in complementary products that would give them a better purchase – more value, even if it means more money.  This drives both the importance of relevance and value as criteria for selection of products to promote.
• There may not be any one person who is responsible for (rewarded for) the profitability of your cross-selling capabilities, as the complementary products being promoted may be “owned” by different parts of your organization.
• You will want to test the effectiveness of the approach you use for promoting particular products – which original products, promoted products, and unique combinations of the two are the most profitable?
• You will want to be able to test the effectiveness of changes in the presentation of promotions (images, page location, text, etc) on profitability (or on conversion percentage as an isolated variable.
• You may want to offer discounts that apply only in the context of the cross-sale promotion (e.g. “Buy these together and save!”) and measure the impact on profitability – do the added sales offset the reduced profit per sale?
• You may find that a given complementary product has a different degree of relevance (and therefore effectiveness) depending on the market segment to which you are promoting it.  As an example, a game controller may be more relevant to consumers buying a large computer monitor than small business owners buying the same monitor.

Conclusion

Cross-selling works when you promote a relevant product that is complementary to the original product.  It works because the prospective customer is already in the process of purchasing the original product, and is therefore already qualified.  You should only promote products where the additional sale increases value to your customer and increases value to you.

Ultimately, cross-sale is about profitability.

Do you know about substitute goods and complementary goods?  If you’re doing any eCommerce, and are thinking about cross-sell and upsell, you should understand the basics about substitutes and complements.

Substitutes

You’re considering purchasing a specific product (e.g. a blank compact disc from Memorex).  You could consider purchasing an alternate product (e.g. a blank compact disc from TDK), substituting the alternative for the original.  At a high level, that’s the definition of a substitute good.  Any product that could be substituted for another product, and still satisfy the needs that the original product was intended to address.

Complements

You’re purchasing one product (e.g. a slice of bread with peanut butter).  The value you get from the original product would be increased by the purchase of a complementary product (e.g. a slice of bread spread with jelly).  The basic definition of complementary goods is “products that are purchased together.”

Economic Models: Substitutes and Complements

The definitions for substitute products and complementary products come from the world of micro-economics.  Substitutes and complements are used to model the interdependent nature of the changes of prices on the supply and demand of “related” products.

You can imagine a junior economist who tried a pricing experiment to determine the price elasticity of demand of Jif brand peanut butter.  He predicted that by raising prices on the peanut butter, that the store would sell less peanut butter.  Sure enough, demand (at the higher price) was lower, and sales dropped.

However, he also discovered that sales of Skippy brand peanut butter went up, almost by exactly the amount that Jif sales dropped.  This is because Jif and Skippy peanut butter are substitute products (economists call them “goods”).

Later on, this economist tried another experiment.  He raised the prices on both Jif and Skippy at the same time.  This time, the store sold less peanut butter in total (there were no other brands), and the economist thought his experiment was concluded.

Another surprise for the economist – sales of jelly dropped too.  Sales of peanut butter and of jelly are tied together – when you sell more of one, you sell more of the other.  Economists, trying to be difficult, would say that when you raise the price of one, the demand for the other falls.  Technically true, but a little confusing.

Thus entered substitute and complementary products into the world of economics and pricing.

Substitutes and Upselling

You know that you’re upselling – trying to replace one product that is about to be selected with an alternative product – when your customer is considering substitute products.  Notice in the screenshot from amazon.com (above) that ~9% of the people who were otherwise going to buy the original product instead were convinced to buy a more expensive substitute product.

The goal in upselling is to create a win-win situation.  Your customer gets more value from the substitute product, and you get more profit from the sale of the substitute than you would have received from the original.  Everyone wins.

Complements and Cross-selling

The number one mistake I see when people talk about cross-selling is they call it “upselling [sic].”  Its enough to make me Cranky. The example above, also from amazon.com, shows an encouragement, when purchasing the first book (in the series), to also purchase the second and third books. What’s especially clever is that there is no discount – the total price is the same as the sum of the prices if purchased individually.

You can take advantage of the complementary product model to create product bundles, identifying products that should be sold together.

Although that may not be the best idea.

Often, you will see bundles created by combining products that do not go well together – products that are not complements.  This is a sneaky way for companies to sell products that otherwise would not sell as well.

The recording industry made money in the 1950s and 1960s selling singles.  The recording industry then made a lot more money selling albums that “bundled” 8 or 9 mediocre songs with one or two hits when compact discs hit the market.  Digital downloads have re-introduced the popular purchasing of singles, and revenues are declining – not because of piracy, but because a “take advantage of our customers” bundling practice is now broken.

Complementary goods should be used to create bundles that increase value for your customers.

A baking cookbook is a good complementary product for a customer purchasing a stand-mixer (a key appliance for baking).

Symmetric Substitutes and Asymmetric Complements in Context

Substitute products are symmetric – either product works effectively as a substitute for the other – in a specific context.

If you need a new computer to use at your desk, then a desktop computer and a laptop are symmetric substitutes.  Regardless of which one is your originally selected product, the other is a valid alternative.  If however, you are looking for a computer that you can use while travelling, the desktop is not a valid alternative for the laptop (nor would it be your first choice).

When you’re considering the travelling scenario, the products are not substitutes.  Economists will say that they are substitutes as long as they share common uses.  But economists are looking at aggregated behavior.  You have to make decisions in context – and when the context implies that products are not substitutes, they are not substitutes.

Complementary goods are rarely symmetrical.  Peanut butter and jelly is a good example of symmetric complements – they have comparable price points, and both generally can be improved by purchase of the other.  In the mixer-cookbook example above, the cookbook is a great complement product for the mixer.

However, if your customer had selected the book initially, the encouragement to “add a stand mixer” would fall on deaf ears.  Surprisingly, amazon.com still recommended adding a mixer to my book purchase.  Technically rated a “toss up” by GetElastic in their great cross-selling do’s and don’ts article, but I put it squarely in the don’t bucket (until measured and proven otherwise).

Summary

• Complementary Products – consider cross-selling an additional product.  Complements are usually asymmetrical.
• Substitute Products – an upsell is a suggestion to replace the original product with an alternative product.  Substitutes are symmetrical, but only in a context where both products address the relevant needs of your customer.

Unless you live in a world filled with unicorns and rainbows, writing realistic requirements is critical.  When you set unattainable goals, the best result you can hope for is a frustrated engineering team.  Write requirements that are attainable, and your team will surprise you with what they can achieve.

Attainable Requirements – Revisiting

A little over three years ago (ten unicorn years), I wrote Writing Attainable Requirements, as part of the Big Rules of Writing Requirements series.  In that article, we focused on the unique situations that every team faces – politics and people, and the shared challenge of implicit practicality.  Those factors are as important today as they were then.

Defining Attainable

Attainability, for requirements, is simply the answer to the question – can this be reasonably done?  Most, but not all, of the answer to that question comes from understanding if your team can build and test a solution to the problem you identify with your requirement.  Being able to create a solution requires that the problem you’re solving is tractable, and that the team creating the solution has the skills to solve it.

Being able articulate a problem and it’s solution is necessary, but insufficient – solving the problem requires that there be a feasible solution.

As the founders of the agile development movement explained in their pitch for “low overhead” process – people trump process.  I think it was Alistair Cockburn who added, “but politics trumps people.”

Finally, process is the often-overlooked element of determining attainability.  Realizing the benefits of a novel solution to an existing problem usually involves process changes – sometimes significant ones.

The next few sections go into more detail on each of these – tractability, feasibility, people, and process.

Tractability

Goldilocks ruled out the large bed because it was too large.  Eating the elephant, drinking from the fire-hose, boiling the ocean – these are all common phrases in the American vernacular, because so many people try to do too much (at one time).  There are many ways to decompose a large, intractable problem into smaller, addressable components.  Practicality requires that the problem you’re trying to solve is not too large.

If everything you do is small, however, you won’t ever accomplish anything big.  That’s where strategic product management shines.  As part of understanding your market, you identify large, valuable problems to be solved.  To solve those large problems, you have to break them up into smaller problems, and then address them individually.  You also have to make sure your team understands the big picture and the larger problem that ultimately needs to be solved.  A great way to share context while decomposing problems is with the use of Ishikawa diagrams.

And zooming in…

Maintaining context is a particularly critical component of working with agile teams, where each deliverable’s possible size is constrained further to what a team can accomplish in an iteration.

[click for larger version]

For some problems (and some teams!) this may be enough information to provide the context to develop a product backlog.  And your agile team will move forward into managing their own execution from the product backlog stage.  For larger or more complex problems (such as this example), you will need more detailed communication before diving into user stories / use cases.

The same Ishikawa diagram, with more detail, looks like the following:

[click for larger version]

Agile Product Management: Providing Context

This reduced granularity represents the smallest go-to-market atomic requirement.  Further decomposition results in smaller requirements, but they become too small, because they are no longer independently valuable.  I’m not talking about “the sum of the whole is greater than the sum of the parts” stuff here – I’m talking about “if you only get X, without Y and Z, it has no value” stuff.

Feasibility

Even after decomposing market problems into addressable chunks, you still have to make sure that there are feasible solutions.  One of the brighter software developers I worked with back in the day used to say “We’re writing software – it’s all ones and zeros – we can do anything.”  He also followed that up with “…but should we be doing that?”  The team I was working with at the time probably could do just about anything.  But not everything can be done with a given budget and a specific deadline.  Some things are simply infeasible.

Scrum teams measure velocity – a reflection of how much software they deliver in a given iteration, as a function of how much work they believe is involved in delivering.  This is a great “protected” measure of efficiency.  The team is protected from requests to do work that has little or no value – by measuring throughput in terms of effort, a team can get more efficient at doing more.  A requirement is infeasible, and therefore unattainable, when it exceeds what the team can accomplish given a fixed amount of capacity.  Project managers refer to this as the Iron Triangle – given scope, cost, and quality, pick any two as inputs and the third will be a variable output (absorbing the impact of uncertainty that arises during the development process).  I prefer to use timeboxes rather than that rusty old triangle, to emphasize my perspective that quality is inherent in delivery, and should not be a “variable output of uncertainty.”  Because quality is what usually bears the brunt of uncertainty.

plus  yields

When the solution to a defined problem is prohibitively expensive, it is not attainable.  The notion of cost should also be considered in the context of value.  Requirements with higher value can justify implementing solutions with higher costs.

Prohibitively expensive solutions make requirements unattainable.  Relatively expensive (relative to their value) solutions also reflect unattainable requirements.  You need to collaborate with your implementation team to understand the costs associated with any particular requirement to know if it is attainable.  This is one of the strongest arguments against throw-it-over-the-wall interactions between product managers and implementation teams.  Without that feedback about feasibility and cost of a solution, you can’t assure that the requirements you are proposing are attainable.

Regular readers of Tyner Blain will realize that this is just an alternate way of stressing the importance of considering ROI when defining requirements.

Politics

There are political realities that every organization faces.  It could be a benevolent dictator CEO, or an executive with an agenda or pet project.  It may simply be that there is value in the problems you’ve identified, but solving them is not aligned with your corporate strategy.  You may be focusing on dominating an existing market, where expansion into new markets is the company’s focus – or vice versa.  You may be focusing on top-line growth opportunities, while the company is fixated on cost-reduction in a time of economic contraction.

Each of these scenarios causes you to be fighting an uphill battle internally.  When that hill is too steep, you’re writing unattainable requirements.

Process

When deciding what to build, you also have to think about how you launch.  Launching, as Dave Daniels regularly points out on his Launch Clinic blog, is not just putting your product up on your website for sale.  You have to think organizationally about all of the other moving parts:

• Do your customers have the ability to absorb an update to your software?
• Can your customer service reps be trained in time to promote and support the new capabilities?
• Will your sales team be able to communicate a message that helps them close deals based on what you’ve just built?

Process is more than just the process of typing, compiling, and testing code.  Process includes everything needed to successfully go to market and sustain your product.  If you’re defining requirements that are too far ahead of your market they are not attainable.  If your requirements embody process changes that are too difficult or expensive for your customers or your organization to absorb, your requirements are not attainable.

Conclusion

Attainable requirements are requirements that

• Have a reasonable cost to implement,
• Are aligned with your company’s strategy and stakeholder’s agendas, and
• Result in solutions that can be consumed by your company and your customers.

Make sure you’re writing attainable requirements.