User Experience Failures

Test Your Reaction to Experience Design Failure: Air France 447

2012-05-01T12:54:02Z

In the book I sometimes illustrate an experience design failure and then ask, "If you were in charge, what would you do next?" Then I tell you what the company did next. It's a great way to appreciate not only how difficult it is to avoid failure but also that having full hindsight doesn't make it obvious how to change a design to avoid failure again. Here's an example I'm fascinated by that won't be in the book.

Let's begin with the summary from Wikipedia:

Air France Flight 447 was a scheduled commercial flight from Rio de Janeiro -Galeão (GIG) to Paris -Roissy (CDG) involving an Airbus A330-200 aircraft that crashed into the Atlantic Ocean on 1 June 2009, killing all 216 passengers and 12 aircrew. The accident was the deadliest in the history of Air France, and has also been described as the worst accident in French aviation history.

After years of searching the black box was found and investigators were able to report what happened. This article by Nick Ross and Neil Tweedie in The Telegraph, Air France Flight 447: 'Damn it, we're going to crash', is a great summary. Go read it now then come back.

I discovered the article via Daring Fireball where John Gruber quoted this part:

As forward thrust was lost, downward momentum was gathering. Instead of the wings slicing neatly through the air, their increasing angle of attack meant they were in effect damming it. In the next 40 seconds AF447 fell 3,000 feet, losing more and more speed as the angle of attack increased to 40 degrees. The wings were now like bulldozer blades against the sky. Bonin failed to grasp this fact, and though angle of attack readings are sent to onboard computers, there are no displays in modern jets to convey this critical information to the crews. One of the provisional recommendations of the BEA inquiry has been to challenge this absence.

Then Gruber wrote, "User-interface design is, in some cases, life or death." I loved reading that statement because it supports the importance of my book, and I tweeted it. But on further consideration I'm not sure UI design is at fault here, for three reasons.

First, the above quote from The Telegraph article is technically incorrect. The angle of attack information is available on the display, but not by default. As this article points out, "it is possible to call up angle of attack on the ACARS (Aircraft Communications Addressing and Reporting System) display."

Second, and most importantly, the key reason this doesn't happen in Airbus A330 flights more frequently is that the problem was pilot error:

Bonin's instinct was again to pull back on the control stick. He left it there despite the stall warning that blared out some 75 times. Instead of moving the stick forward to pick up speed, he continued to climb at almost the maximum rate. If he had simply set the control to neutral or re-engaged the autopilot, all would have been well.

Third, the A330 cockpit today is already rather complex. I'm not convinced we should further complicate it with yet another data point if this problem is mainly due to pilot error.

On the other hand, research may support adding the angle of attack information: "Years ago both Boeing and Airbus consulted with management pilots worldwide on cockpit display preferences. Because the US military is such a strong influence in the United States, they asked for an angle of attack indicator; however neither manufacturer decided to install it..."

We might also assume that there will always be less experienced pilots in stressful situations in the cockpit, and we need to take all reasonable steps to insure the same problem doesn't happen again.

You're probably not a airplane cockpit designer with a Ph.D in human factors, and yet considering what you would do with this information may tell you something about how you would manage a similar situation in your work. Would you:

Add the angle of attack information as a standard data point in the cockpit
Somehow communicate how one pilot is manipulating the side stick to the other pilot (see The Telegraph for the side stick explanation)
Improve pilot training
Run simulations and usability tests
Something else?

Transparency at Work: Aviary

2012-04-18T13:38:43Z

Last night I went to hear Avi Muchnick, founder of Aviary, talk about success, failure, and how he created a transparent workplace, enticed after finding other evidence that transparency helps avoid failure.

Aviary started as web-based media editing tools (think: Photoshop on the web, but much easier) but when that business model didn't bear fruit they later pivoted to create a mobile SDK for developers with similar media editing functions.

Avi had previous startup experience, and liked the idea of transparency, that is, of having no secrets at work, within the limits of reason and legality. In the beginning they even extended this outside the company, sharing roadmaps and product development details, until an advisor persuaded them that was going too far and could help a competitor crush them. They now keep the transparency internal, and Avi says when he's up front with vital company information it builds respect among his employees.

He benefits too. Once when a new employee demonstrated some strange behavior, he received immediate feedback from others in the office. The situation was so bad Avi fired the person after five days. Compare that to a previous situation where Avi debated what to do for nine months before finally letting someone go.

Some Failures You Won't Find In My Book

2012-04-16T18:45:38Z

People love to tell me about their favorite customer experience failures, which has come in handy when finding the case studies for this book. Inevitably I've had to go down the path of researching some of these products only to find out they didn't fit my criteria. Here's a few of those dead ends, leave a comment if you disagree:

Microsoft Bob and Clippy got hammered in the media, but it's not clear if customers actually had a bad experience with them or not. If that happened today, I could do a study to see how customers reacted, read reviews from customers themselves, or do some sentiment analysis of social media, but since Bob came out so long ago all I have to go on is what mass media said, and that's not a good substitute for customer experience.
MySpace - what this a failure? I think MySpace served us well. Rather than frame its demise as "being beaten by Facebook" I think MySpace was something different and unique, and they simply had their time when that product made sense to that audience.
QuarkXPress, the Tablet PC, Wordperfect, the ESPN phone, Pointcast, Facebook Beacon, and many video game consoles failed or fell behind in customer experience, but other factors were so prominent that I can't pin most of the blame on the customer experience.
Blackboard creates terrible customer experiences and deserves to die, but doesn't because the people who purchase it (IT and administrators) aren't the people who suffer from using it (students and teachers) so it'll probably take a competitor with a good design and a great strategy to actually kill it.
RealPlayer had agressive customer acquisition tactics that created an annoying experience, but they probably failed because competitors had superior pricing schemes.
Technorati offered an OK experience, then seems to have pivoted into irrelevance, perhaps a failure of strategy.

How Did the Ford Motor Company Avoid Failure?

2012-04-02T00:42:14Z

In 2008, the Ford Motor Company seemed caught in a death spiral.

The company was hemorrhaging cash -- more than $83 million a day -- as the bottom fell out of the car market. In late autumn, Ford's stock price bottomed out at $1.01.

Move forward three years. For 2011, Ford turned a net profit of $20 billion on sales of $128 billion. It distributed profit-sharing payments of about $6,200 to each of 41,600 eligible employees. On Friday, its stock closed at $12.48.

Wow. How did they do that? In Bryce G. Hoffman's new book, "American Icon: Alan Mulally and the Fight to Save Ford Motor Company," as reviewed by Nancy Koehn for the New York Times, he gains extensive access to the company and comes away with an interesting thesis centered on the methods of the CEO...

Mr. Mulally knew that Ford could not hope to improve its market performance without simultaneously changing its culture. Some of the book's most interesting passages deal with his efforts -- often one person at a time -- to improve accountability and to foster commitment among executives.

Mr. Mulally's chief instrument here was data-driven management, in which each executive was responsible for consistently knowing and reporting how his -- very few women appear in this story -- department was performing. Concentrating on consistent metrics, he argued early on, would focus managerial attention on the big picture while increasing transparency.

That T word, transparency, popped out at me because I've been blogging about other examples (one, two, three, four). Not all the examples are about customer experience failure, but I think the approach of using transparency as a culture change tool is applicable: if we openly discuss problems as they occur, we're better able to fix them before they lead to failures.

The review sums up one pivotal incident where talking out loud about a problem opened the gate for more transparency...

Early on, Mark Fields, head of the Americas business, tested the new chief. Mr. Fields wondered: Did Mr. Mulally really want business plan review meetings to be forums where problems as well as achievements were discussed? If so, then the gatherings ran counter to Ford's culture, in which formal meetings were "political theater," according to Mr. Hoffman, and side discussions were "where deals were cut and truths too painful to put in a PowerPoint presentation were shared."

Mr. Fields told colleagues in a business plan review meeting that he was delaying a product introduction. (The vehicle had a possible problem: a test driver had noticed a grinding noise coming from the suspension.) Mr. Fields and some others around the table thought that he might be fired, the book says. Instead, Mr. Mulally started clapping, praising Mr. Fields's willingness to report the delay. Two weeks later, other executives' PowerPoint slides were filled with examples of specific problems in their departments.

Is Google+ a Failure?

2012-03-27T17:52:31Z

Google+ has existed for less than a year and it's already being judged a failure. If we judge it by traffic in the stream on the Google+ website, traffic seems low compared to direct competitor Facebook, but Vic Gundotra, Google's vice president for engineering, says G+ is a "social blanket that envelopes the entire Google experience" so I'm not sure what we see on the website is the right metric.

Nick Bilton says G+ fails on design, but I can't find a single specific point in this analysis that demonstrates how the design leads to a bad customer experience. Yes, there are places where the design is obviously lacking like in the iPhone app, but by comparing it to Instagram and Path he fails to make his case of how that design interfacing with that customer actually fails.

Dave Moon thinks G+ failed at the concept stage: "Google+ failed because they tried to make a slightly better version of Facebook, and no one wants a slightly better version of Facebook." This argument resonates more for me, and reminds me of the Zune vs. iPod case study that will be in my book. Google+, like Zune, might need more to differentiate it, to stop trying to be a Facebook-killer and try to be a great Google+, whatever that is. A social blanket and some video conferencing may not be enough.

I'm not sure what kind of customer experience Google intended to facilitate, or that they don't have enough time to iterate on the design and/or concept to produce something more popular, so I'll withhold judgment for now. Besides, my friends who work on G+ will kill me.

I'll be writing about Google Wave instead.

Why Sneezing is Bad For My Book

2012-03-12T02:25:59Z

This winter I suffered through both an upper respiratory infection and bronchitis, stealing precious healthy time from writing. One theory for the prevalence of these afflictions was that our warm winter made it easier for bacteria etc. to spread. But what I didn't get was the common cold or bouts of sneezing, which is good because I sometimes organize my ideas like this:

There's a threshold where there are too many points in a story to organize them on screen. So I print out a rough outline, cut up each individual idea, and sort them all on my dining room table. The photos here are from the chapter I'm writing about Symbian. It's a complex story and I found myself finding several improvements on the telling as I moved the paper around. The next step looks like this:

Under each logical collection of ideas I slide a sheet of paper and tape all the bits down with one long piece of Scotch tape. Then I feel I have something that looks and feels like the prototype of a chapter. The next step is to write a shitty first draft.

Cabin in the Woods, Days 4 & 5

2012-02-10T22:52:35Z

Over the past two days I had a very productive day and a half. Then I think I reached my burn out point of being alone in a house and writing.

Besides, that's just enough time to revert to bachelor pad habits. Not pretty, time to go home.

Cabin in the Woods, Day 3

2012-02-09T16:20:07Z

Journalism can feel like hunting for treasure. Some days you do your research and find a few trinkets and you're satisfied, and sometimes you find big booty. Yesterday was one of those big booty days, when I pump my fist and realize the story is even better than I had planned.

I started a rule yesterday of No TV. While I need some chill time, it's dangerous to take a guy like me that usually doesn't have a signal and grazes on Netflix at home and lock him in a house with 200 channels and a nice LCD. Too many good movies, too many fun shows about hot cars. So No TV. That enabled the aforementioned booty finding and most of a new chapter.

Cabin in the Woods, Day 2

2012-02-07T16:30:48Z

This morning when I woke up I listened to my body and my body said, "I'm kinda sluggish and my back is stiff, though the brain is ready to go." So rather than exercise I had a coffee and a smoothie and went to work for an hour or so, and then went to the beach to run and do push ups. That worked well so far.

Truth be told, it's a little scary being in a house by yourself in a deserted area. The first night I locked the bedroom door. And the creaking of the house freaks me out a bit.

I was about as productive today as yesterday. Despite the schedule I set, I think I'm simply a night owl and have to use those hours to get work done, especially if my body's union demands a nap mid-afternoon. Night time is productive time.

Cabin in The Woods, Day 1

2012-02-07T01:21:28Z

I've scoured the Internet for evidence, interviewed characters in my story, and compiled hundreds of notes. Now I am at the hard work of writing, and that requires serious focus. Just seven days ago I playfully put out a call on Twitter:

Can anyone spare a cabin in the woods near NYC for a writer needing to hit a deadline?

And you, beautiful UX community came through for me! Even better than a cabin in the woods, I'm now sitting in a house at the beach, coincidentally on the same island where I've vacationed most of the last 20 years.

I sought the advice of friends and family and fellow Rosenfeld Media writers: how can I make the most of this week? Armed with their wisdom I created a schedule. Here's the plan and some notes on day one.

7:00 wake, have coffee and bite to eat, put on exercise clothes I slept badly my first night here, to be expected. I managed to get out of bed by 7:20, though groggy and with a headache.
7:15 go to beach, exercise, meditate Alone with the beach and the Atlantic Ocean, wonderful even in the coldness.
8:00 shower and dress
8:30 make daily pomodoro list Pomodoro has been great to keep me on track, though it may take a while to get the rhythm of dividing my time into 25 minute segments. Trello, which I usually use for my task lists, has been my Pomodoro list tool. I use my phone as my timer.
8:45 start working I managed to arrive here just 15 minutes late.
10:00 have a snack
10:15 work I wrapped up a chapter that was in progress, feels good.
13:00 eat lunch, walk On my first night I cooked enough food for a few days, that's been a big time saver.
14:00 work In reality, given the rough sleep last night, a nap was inevitable.
16:00 snack, chill
16:15 work I'm into the meat of another chapter, hopefully I can finish it tomorrow.
18:00 review work, revise schedule, start Pomodoro list for next day
19:00 dinner
20:00 chill
22:00 go to bed

I'm hoping for a better night of sleep and more productivity tomorrow, so I'm going to try this schedule again.

My Writing Process

2012-01-27T16:13:48Z

At the moment I'm deep into writing the book, most of the research having been done. I've had a background process running in my head about how to organize and write so that I'm more productive, and when the topic came up recently on our internal author mailing list I took the opportunity to write out the process. Your feedback is welcome:

I collect source material (text and images), random thoughts, and start writing outlines and drafts in Evernote. The tagging and searching help me organize by chapter. The syncing helps me access stuff across devices. The lack of formatting helps me focus on the ideas.
To track images, I note the source right in the text. When I drop in an image, I'll write a caption and insert a footnote number (since my book is researchy, I'm using footnotes) and put the source info in the footnote.
Lately I'm been revising the process for actually writing the chapter. Here's what I'm trying now:

Capture enough research that I feel like I understand what happened
Summarize the research in one Evernote file
Think about how I want to tell the story
Shift around the summary to make an outline
Set a time limit of about 3 hours to write a shitty first draft
Reread and tweak the draft a bit until it feels like it works
Move the draft to Word and refine language
Post on Basecamp for my editor

In Word I do simple version control, renaming major versions v1, v2, etc.. The Word files are in a Dropbox folder so I get access across devices and backup, plus my Mac is periodically backed up with Time Machine.

The greatest disease at Microsoft

2012-01-05T14:17:40Z

Scott Berkun illustrates one case of why it's vital to learn from customer experience failure:

The greatest disease at Microsoft is lack of sharing lessons from failure, especially where innovation is concerned. Microsoft has made many big, visible bets. Many of them have failed, but that's par for the course. The problem is these expensive lessons are swept under the rug, encouraging others in the company to repeat the same mistakes. Everyone loves to make fun of Microsoft Bob, but few can articulate why it failed. If you don't understand why it failed, you don't have any reason for laughing so hard, and you likely aren't half as smart as you think you are. A case study on Vista, MSN Search, Microsoft Bob, The Tablet PC, etc. should be produced by an outside consultant, and stapled on the forehead of every manager at the company, once a day, until they read them all word for word. Then they'd take advantage of Microsoft's so called experience and wisdom. Otherwise, they are being set up to make the same expensive mistakes again and again.

Say Something. Out Loud. (Part II)

2011-12-08T15:28:29Z

I recently wrote about times when we strongly suspect something will fail but we say nothing. And then the thing fails. Here's another big, timely example: The entire Greek economy.

It's not a customer experience example, unless you consider that a lot of honest, hard-working Greek citizens were screwed by the government, and that the same lesson can be applied to our work.

Reporting by Chana Joffe-Walt at NPR's Planet Money sums it up. When Greece fabricated statistics to show they were eligible for membership in the Eurozone everyone else knew the numbers were fabricated but said nothing...

In just a few years, Greece seemed to get its house in order. The country made unbelievable strides in its financial data to meet the [Eurozone] criteria. Truly unbelievable. Did anybody believe them?

"No," says Jacob Kirkegaard with the Peterson Institute for International Economics.

But the euro was supposed to be about unity. Countries were saying we are mature, civilized European nations with strong financials. Kirkegaard says no one wanted to stand up and say to Greece -- "you are lying." Also, Kirkegaard says there were a lot of comparisons to Sweden popping up in those those meeting rooms -- as in, we're all Swedes now.

"When that is the self-identity, all of a sudden standing up to one member of this club that look actually, you're dirty, well they're gonna turn around and say what about yourself?" says Kirkegaard. "You know, who is to say if you can't trust the Greeks can you really trust the Italians?"

Because, really, once you asked that question, the answer was likely to be no.

Spoiler: The Way to Avoid Failure is...

2011-12-01T01:50:37Z

...something I'm seeing in research on how children learn and how adults learn and in people whose work we celebrate. It is this:

Say something. Out loud.

I was reminded of this today when a friend told me a story. He was consulting a company that wanted to offer people the ability to select some songs and then buy a custom made CD of those songs. The company was essentially re-inventing the mixtape, but since this was during the early days of mp3 audio files my friend knew everyone would soon have this function right on their computers. But his job was to try to make them a success, and that's what he did. But they weren't.

Sometimes we need to lean across the conference room table and say, This idea isn't going to work, and here's why.

Sometimes we know when something won't work or we lack confidence in the idea, but we don't raise objections because of relationships or politics or fear. These are good reasons sometimes, but people who achieve the highest levels of performance provide feedback in spite of these reasons.

Note that this is different than being persistently negative. The goal should not be to shoot down ideas but to improve ideas by calling out bad ones and then focusing on solutions.

Keep Baby / Discard Water

2011-11-29T01:34:40Z

I have a new article at Fast Company's Design site about babies and bathwater, about keeping the parts of a design that work and getting rid of the parts that don't. It may sound like common sense, but more often than not, we deny the flaws or scrap the project altogether. I use the example of BMW's iDrive, their biggest customer experience failure to date, to make the point, and will have a longer case study on this topic in the book.