User Experience Failures

Will Facebook Home Fail or Succeed?

2013-05-21T02:29:21Z

I don't have a case study of Facebook in my book (lucky them) but I do cite them for being quick to react to customer backlash before things get too ugly (e.g. Beacon). So when a friend mentioned Facebook Home as a potential failure, I perked up and decided to start collecting research on the system.

Here's Facebook's Home section. At launch, the New York Times said, Facebook Home "effectively turns the Facebook news feed into the screen saver of a smartphone, updating it constantly and seamlessly with Facebook posts and messages." TNW noted, "passed the 500,000 download mark on the Google Play store after a little over a week in the wild, though it continues to accumulate mostly negative reviews and has sunk to an average rating of 2.2."

Walt Mossberg concludes, "Facebook Home is a very clever and very well-done product that will delight Facebook fans. If you aren't in that category, or prefer the standard Android user interface, it won't be right for you." David Pogue asks, "Why?"

Satell in Forbes argues that Home will probably fail but it doesn't matter, it's part of Zuckerberg's iterative path toward great products. Kumpurak in TechCrunch makes a similar argument, but theorizes the iteration is at the partnership level as well.

Constine in TechCrunch delves into the design decisions influencing the customer experience and finds that iOS-weilding Facebook developers left out some vital Android bits.

Personally, I'm surprised there hasn't been more comparison of Home to PointCast, if only to refresh our memory of what not to do.

Keeping a secret from your customers

2013-04-16T20:31:43Z

The Segway is a good example of the kind of product I profile in the book: it was well-designed. It had great technology. And many smart, experienced business people believed it would succeed. But customers didn't like it.

So why didn't I include it in the book? Well, the lessons just seem so obvious. I think Paul Graham summed it up nicely in a one-page article: "Someone riding a Segway looks like a dork.... The reason you look like a dork riding a Segway is that you look smug. You don't seem to be working hard enough."

In the book I delve deeper into this type of failure in a case study on the Microsoft Zune, a more nuanced failure. As designers, I think it's helpful to look at these failures through Don Norman's emotional design framework, particularly the part about reflective design: "how it makes one feel, the image it portrays, the message it tells others about the owner's taste."

Graham understands this, and when he offers design advice it's not about the function or usability, it's about the styling:

So there may be a way to capture more of the market Segway hoped to reach: make a version that doesn't look so easy for the rider. It would also be helpful if the styling was in the tradition of skateboards or bicycles rather than medical devices.

In my book I go beyond the design cause of the failure and try to expose the underlying organizational problems. Unlike many analysts, I don't think the hype preceding the Segway launch was the problem, because the product could have lived up to the hype (in the book, I discuss this regarding Google Wave). I agree with Graham's argument that the design process was flawed because it was what I call "Keeping a secret from your customers":

Curiously enough, what got Segway into this problem was that the company was itself a kind of Segway. It was too easy for them; they were too successful raising money. If they'd had to grow the company gradually, by iterating through several versions they sold to real users, they'd have learned pretty quickly that people looked stupid riding them.

Developing in secret isn't always a problem. But if you're creating something entirely new that relies on the customer experience for success, you need to test the actual customer experience.

What are your questions about the "Why We Fail" book?

2013-04-15T19:23:19Z

There's an FAQ section in the front of all Rosenfeld Media books (here's an example). I could take the easy path and just make up the questions and then answer them, but unfortunately I take this stuff way more seriously than is good for me. So please help me by telling me what questions you have, and I will duly answer them, both in the book and here on this website.

You can get snarky too, e.g. Isn't learning from failure overrated?

It's a History Book Too

2013-04-06T13:37:36Z

I'm in Baltimore for the Information Architecture Summit and one compelling theme to emerge is about what our field must do to make progress. To paraphrase Matt Nish-Lapidus, we need to be able to give and receive critique to learn from others. To critique, we need a shared language. To have a shared language, we need some standard vocabulary and history. We don't all need to agree to share the same school of thought, but we should be able to communicate across schools of thought.

Though I haven't thought of myself as a historian of our field, in some sense that's what I've done. I've rallied points of views like Scott Berkun's who said the greatest disease is a "lack of sharing lessons from failure".

Everyone loves to make fun of Microsoft Bob, but few can articulate why it failed. If you don't understand why it failed, you don't have any reason for laughing so hard, and you likely aren't half as smart as you think you are.

At the end of each of my case studies of failure, I tried to both objectively sum up what the experience failure was, and subjectively suggest what we can learn from each story. I hope that will be my small contribution to building a shared history of our field.

Platform Follows People

2013-03-21T16:23:08Z

James Whittaker blogged about why he left his position as development director at Google+ and I thought his daughter had the biggest insight:

As it turned out, sharing was not broken. Sharing was working fine and dandy, Google just wasn't part of it. People were sharing all around us and seemed quite happy. A user exodus from Facebook never materialized. I couldn't even get my own teenage daughter to look at Google+ twice, "social isn't a product," she told me after I gave her a demo, "social is people and the people are on Facebook." Google was the rich kid who, after having discovered he wasn't invited to the party, built his own party in retaliation. The fact that no one came to Google's party became the elephant in the room.

Building a platform from scratch is an enormous uphill battle. I argue in my book that part of the reason that the personal finance service Wesabe lost to Mint is that Mint focused on the customer experience while Wesabe spread it's more modest resources very thin, trying to create a platform. Platforms often start with small, strong features like Google's search and Facebook's updates. Once you have a large audience returning regularly, you can offer them new abilities, and possibly leverage that popularity into a platform.

Google+ was trying to build a sharing platform from scratch, a steep uphill battle, without having an audience first. Platform follows people.

Some nice comments on Chapter 10

2013-03-19T02:12:46Z

I plan to post book reviews, good or bad, on this blog as they come in. Here's the first, from one of my tech reviewers who sent some gushing nice comments on my last "recommendations" chapter.

I've read it. I love it. I think as I thought before that this chapter ties everything that comes before it together so nicely -- and in a practical sense. You have laid out a loose template so that this can be put into practice.

The steps you take, Step 0 - Step 4. Love. Love. Love. You take the discussion in the previous chapters and set it up perfectly for how it can be done. It's basic, but not dummied down, so you will reach a wide variety of levels with your audience. This is a difficult balance to achieve, but you have done it.

Also, interspersing your thoughts based on your expertise/knowledge with the factual information personalizes this chapter. It could have been really dry, but doing that kept it from going there.

I like that you have a "template" of sorts so people can take this and apply it in a way that will be meaningful for them and their work.

Finally, I like that you have driven home that it's the experience, not the product. You have also made my reading list even longer as I want to delve further into this through the resources you have listed. In my opinion, as a person who reads a lot, you have achieved greatness. I want to learn more. Your book has set that up.

Thank you for sharing this with me!

How do we decide what to test?

2013-03-18T18:33:24Z

Here's a bit of a book spoiler: I think methodologies that are based on the scientific method like Customer Development and Lean Startup are a big part of the solution to avoiding customer experience failure.

But one of the most difficult and ill-defined part of scientific method-based methodologies is uncovering implicit assumptions and determining which ones are critical enough to test with customers. I've attended Lean workshops, and when we reach this step the advice is a disappointingly simple, "Where's the most risk?"

Working backwards from the failures profiled in this book, I created four questions that can help us find relevant hypotheses and prioritize them. My questions are still fairly simple, but at least they are based on what caused actual products to fail in the marketplace. Starting with these four can point us in the right direction to ask more specific questions.

1. What does our design do differently than the currently existing successful designs?

For example, the key difference might be the product concept (iDrive, Wave), the interaction (iDrive, Wave, OpenID, Final Cut Pro X, Plaxo, Symbian), the audience (OpenID, Pownce), or the features (Final Cut Pro X).

2. We believe we have a particular competitive advantage, and that our customers' experiences are somehow better or different with our product. Do customers actually experience this difference?

This question, in my experience, isn't asked nearly enough in contemporary experience design practice, especially given how many failures are due to losing to competitors (Zune, Wesabe, Pownce, Symbian).

3. Do our customers' experiences rely on technology performing in a new or better way?

For example, Symbian's touchscreen interface suffered from delays between the touch and the on-screen feedback. Another classic example is Apple's Newton MessagePad, which when first released featured handwriting recognition that often produced unintelligible results.

4. Where could our current customer experience improve?

From your observation, testing, and customer feedback, you should have an idea of which part of the experience is not pleasant, where there are too many steps, what abilities are missing, and so on. These are the places that competitors will target, as we saw with Wesabe.

Is double-loop learning built into the scientific method?

2013-02-11T18:50:23Z

There was a fun little article on double-loop learning recently in The New York Times...

...the most common response [to obstacles in our path] is single loop learning -- an insular mental process in which we consider possible external or technical reasons for obstacles.

LESS common but vastly more effective is the cognitive approach that Professor Argyris called double-loop learning. In this mode we question every aspect of our approach, including our methodology, biases and deeply held assumptions. This more psychologically nuanced self-examination requires that we honestly challenge our beliefs and summon the courage to act on that information, which may lead to fresh ways of thinking about our lives and our goals.

The book Blind Spots does a great job of exposing our cognitive biases that lead to failure and what to do about them. But what I learned is moving a team or a company from single-loop to double-loop learning ain't easy.

And that's another reason I like the scientific method-based approaches so much, like Customer Development and Lean Startup. They don't try to get inside people's heads psychologically, they just rationally institutionalize the stating of our assumptions and the mandatory rigor to test them. It's worked for other fields and I think it's going to work for experience design too.

Experiential Products Fail Experientially

2013-02-06T02:05:01Z

Consumer electronics and software used to suck because they were underpowered (at least, that's one reason). The interface on your VCR was where "the inmates ran the asylum" — low-powered, generic chips were massaged into a modicum of functionality. The result was barely usable.

Now, we have lots of power: fast chips, lots of memory, all sitting beneath touch-sensitive gorilla glass. It's much more power than the Apollo missions used to land on the moon. In the old days, like 10 years ago, we compared processor speeds. Now it would be silly to say "my Android processor is faster than your iPhone processor" because no one cares. And jamming the phone full of features, as Nokia did with their Symbian phones, doesn't make it a better product. We've stepped over a performance threshold where more features and performance offer diminishing returns unless the design helps people have a good experience.

Once a product passes this threshold, I call it an experiential product.

While the qualities of a design can be objectively described on a list of specifications, the qualities of our experience — our thoughts and feelings about a complex, multifaceted product — are subjective.

And because we relate to experiential products differently, they fail differently. They might fail because we feel overwhelmed (Google Wave) or underwhelmed (Microsoft Zune). We might feel cheated (Classmates.com) or annoyed (Plaxo). We might feel frustrated (OpenID) or maddened (BMW iDrive). We might become enlightened to a need we didn't know we had, and then realize that another, different product would actually satisfy this need better (Pounce, Wesabe).

If I were a young designer looking for the next big challenge, I'd look for product categories that are just about to pass this experiential threshold. Maybe:

Television (Netflix, Apple, Google)
Musical instruments (touch-screen driven)
Home automation (Nest)
Automobile telematics (suppliers for all automakers)

Can you think of others?

Failure for Kids!

2013-01-30T16:04:10Z

Some people think it's a waste of time to study failure, we should just study success. Here's a bit of what I say about that in the book...

Instead of studying failures, can't we just study success and then do whatever led to those successes? Yes, this is a good tactic in very simple situations, like learning how to tie your shoes. There's not much to be gained by looking at how many ways it's possible to tie your shoes incorrectly. Because if you fail, nothing terrible happens.

Any industry that's important, complex, and dynamic takes the time to examine failures. Aviation, medicine, and manufacturing are three fields that have taught us an enormous amount of lessons by studying failures.

Somewhere between tying our shoes and flying a plane we cross a line where it does become important to study failure. Where is the line? I think it's when a failed outcome is bad enough to justify the effort to study failure. In the case of my book, you could spend $22 and 30 minutes reading one chapter that could help you save an entire project. I think that makes sense.

This morning I was reading books to my daughter's class and discovered a book that takes a point of view on the matter: The Bike Lesson.

Another in the series of classic Berenstain Bears books from the 1960's, in the story Papa Bear gives Brother Bear a brand new bike. But before Brother Bear can ride it, Pape Bear demonstrates several lessons of everything that can go wrong while riding a bike. Finally, at the end (spoiler alert), Brother Bear is so knowledgable he gives the beaten and bruised Papa Bear a ride home on the bike.

Of course I think Papa Bear is a pretty smart, generous father to ensure his son doesn't have to learn all mistakes the hard way. And the book is a helpful tool in teaching kids when it makes sense to study failure.

The courage to blow the whistle

2012-12-26T19:35:18Z

I've said before that one of the best ways to avoid failure is to say something. Out loud. We might be the least experienced person around a conference room table full of experts and we might be afraid, but chances are if the concern is warranted other people will have it too. And managers need to be open to this kind of transparent communication in order to avoid failed projects.

As illustration, see Snow Fall: The Avalanche at Tunnel Creek, a sad, gripping story of a group of 16 skiers and snowboarders on top of Cowboy Mountain near Seattle in February 2011. The group was filled with world-champions, local experts who had skied the mountain many times, and expert skiing journalists and equipment representatives. They carried safety gear. And they read the avalanche report that told them there was a considerable chance of snow sliding that day.

As they neared the top of the mountain, at least a few of them had concerns about their safety. But no one said anything for fear of looking stupid in front of such an experienced group.

"If it was up to me, I would never have gone backcountry skiing with 12 people," Michelson, the ESPN journalist, said. "That's just way too many. But there were sort of the social dynamics of that -- where I didn't want to be the one to say, you know, 'Hey, this is too big a group and we shouldn't be doing this.' I was invited by someone else, so I didn't want to stand up and cause a fuss. And not to play the gender card, but there were 2 girls and 10 guys, and I didn't want to be the whiny female figure, you know? So I just followed along."

"I've been riding Stevens Pass since I was 3 years old," Dessert said. "I can tell circumstances, and I just felt like something besides myself was in charge. They're all so professional and intelligent and driven and powerful and riding with athletic prowess, yet everything in my mind was going off, wanting to tell them to stop."

"I could see the others when I cut over," Wangen said. "I thought: Oh yeah, that's a bad place to be. That's a bad place to be with that many people. But I didn't say anything. I didn't want to be the jerk."

Elyse Saugstad, a professional skier, wore a backpack equipped with an air bag, a relatively new and expensive part of the arsenal that backcountry users increasingly carry to ease their minds and increase survival odds in case of an avalanche. About to be overtaken, she pulled a cord near her chest. She was knocked down before she knew if the canister of compressed air inflated winged pillows behind her head.... At first she thought she would be embarrassed that she had deployed her air bag, that the other expert skiers she was with, more than a dozen of them, would have a good laugh at her panicked overreaction. Seconds later, tumbling uncontrollably inside a ribbon of speeding snow, she was sure this was how she was going to die.

30 minutes later, three of them were dead (Saugstad survived).

What's Taking Your Book So Long???

2012-12-16T17:14:34Z

"Yo, wasn't the publish date 2012? Now it's 2013. 'Sup wit dat??"

Mmm, yes. I started the proposal for this book over two years ago, and have been working on it for a year and a half, much longer than planned. Some of the book is about a method you can follow, but most of it is stories about what failed and why, and that required a lot of research. Sometimes validating one sentence can take 1/2 hour. I can now empathize with all those doctoral candidates who are ABD.

On the bright side, the extra time has undoubtedly resulted in a better book. The themes have had time to simmer in my head and turn into something delicious. If I finished in half the time, the book would have only been half as good. So there's that. I hope you'll agree when the book comes out in ~~2014~~ 2013.

Change Sucks: Final Cut Pro X

2012-12-10T15:51:57Z

In the book I have a chapter on Final Cut Pro X (the "X" is pronounced "Ten") and how it caused a hurricane in the video editing world. Some of the pain was due to Apple releasing the software without a bunch of features on which professional video editors depend. But some of the pain is because the interface is radically different.

Earlier this year seven professionals offered their take on Final Cut Pro X. I like how Michael Friedman, a 'grizzled veteran' with shows like Project Runway on his resume, puts it all in perspective:

Working in film and television for 20 years I'd personally seen editors unhappy with change many times before:

In 1995, I knew a flat-bed film editor who was resistant to non-linear editing and said 'if I wanted to work on a computer, I'd be an accountant!' In the years that followed, he worked less and less.

When Final Cut Pro came along, I trained many seasoned Avid editors how to use the new software More than a few said said it was 'not suitable for professional use.' But in 2001, my first primetime television editing job was on Final Cut and, as recently as 2011, I edited Project Runway on FCP.

When FCP X came along. The grumpy and resistant editor was... me. While the new approach and interface intrigued me, I felt totally disoriented and was unable to perform even the most basic editing tasks. Mind you, I beta-tested version 1.0 of Final Cut in 1999 and loved it. Since then, I have spent more than 20,000 hours editing, mostly on Avid, but with healthy doses of FCP and Premiere as well.

In the 90′s, I was an aspiring editor, but now I am a grizzled veteran. So, with version X, I said "why should I bother to learn yet another interface? Where's my FCP 8? What can X do that I can't do now? What's in it for me?."

However, having witnessed the waves of resistance to change in the past, thinking 'I may be missing the next big thing', I've forced myself to work through the discomfort and learn Final Cut Pro X.

What he learned about FCP X is that, "instead of driving a car, it feels more like skiing or surfing.... When you get over the frustration, there is something fast, fluid and flexible in FCP X that I haven't experienced before."

If you think you should rehearse, you're right.

2012-11-20T14:17:31Z

Here's a story of one of my failures, one you won't find in the book because there's just not enough room for them all.

I was working as a product manager of an international website. I had inherited a legacy product that was popular (there were over 10 million people registered) but with a rat's nest of old code, obsolete middleware, and expensive hosting, all of which we were replacing.

The site served three countries, and among them the members were conveniently divided into small, medium, and large groups. When it came time to migrate the database records onto the new system, we started with the small country. We took the site down in the middle of the night, moved all the data over, turned on the new site, and everything worked fine. A couple weeks later, we migrated the medium country the same way, and everything went fine.

The big country was our main money maker, so it had to go smoothly. We all traveled to that country, had a big meeting, and the technical guys I worked with were confident this migration too would go smoothly, since it was the same as the other two, just more data. "Don't you need a common checklist or something, like NASA does?" No, I was told. They weren't sure how long it would take, but everyone knew what to do. I felt a bit uncomfortable, but I had to trust my partners in IT.

When the time came, we took the site offline and started transferring all the data to the new database. After a few hours, the systems just stopped responding. I've been in this situation before as an IT person: the system is trying to cope with so much data it reaches a point of overload. And you're not sure if the system is still working and just too busy to respond or if it crashed and it's not coming back. You can wait and see if it finishes, but for how long? The site is down and we're losing more money the longer we wait.

The lesson here is that quantitatively bigger events can be quantitatively different. A 2-minute dance routine may not require rehearsal, but a 2-hour play does. We should have made a copy of the database and rehearsed the migration, but the previous migrations made us overconfident. Sigh.

This isn't a customer experience lesson per se (although customers did suffer somewhat since the site was down) but you can certainly apply it to experience design. Small changes are easy to do ad hoc; big changes should be rehearsed and tested.

Team Romney's "Orca" System Felled By Hubris

2012-11-12T20:53:46Z

I cover a couple case studies in my book about companies with too much hubris, and this recent story gives you a flavor of that kind of failure.

Mitt Romney's campaign team created a system called "Orca" to track local polling analytics and help get-out-the-vote efforts in key battleground states. It was rolled out to volunteers on election day without any prior learning or debugging time. You don't really need to know anything more than that to explain the failure, but the story goes on to include networking mishaps, misplaced server redundancies, lack of stress testing, lack of customer experience testing, and more.

As one Orca user described it to Ars, the entire episode was a "huge clusterfuck." ...This sort of failure is why there's a trend in application testing (particularly in the development of public-facing applications) away from focusing on testing application infrastructure performance and toward focusing on user experience. Automated testing rigs can tell if software components are up to the task of handling expected loads, but they can't show what the system's performance will look like to the end user. And whatever testing environment Romney's campaign team and IT consultants used, it wasn't one that mimicked the conditions of Election Day. As a result, Orca's launch on Election Day was essentially a beta test of the software--not something most IT organizations would do in such a high-stakes environment.