Domino’s Brutally Honest Re-branding Effort

Their rebranding effort has brought up quite a few memories for me, because I’ve seen the pain on the faces of designers and engineers when users eviscerate the products they’ve spent months or years developing. I also know convincing stakeholders they need to rethink a product can be a painful and costly endeavor. However, one of my most salient memories involves interacting with research participants who already had their minds made up about the quality of a product before they ever laid eyes on it. People’s preconceived notions can be another elephant in the room—a barrier to achieving accurate and actionable feedback on a concept or design.

The Angry Elephant
“To show these participants we were truly ready to make serious changes, we had to … be brutally honest about the failings of the existing design.”

A short time ago, my business partner, Bryan McClain, and I were giving a talk about communication and research at an ecommerce company, and someone asked a very interesting question: What was our most difficult interaction with a research participant? The question instantly brought to mind this memory: Some time ago, we were asked to conduct a focus-group session with a specialized group of participants. In this particular case, the focus-group participants had previously tested builds of the product, and their feedback had been overwhelmingly negative. Our client, to their credit, had already decided to redesign the product from the ground up and realized they needed direction. The company had chosen to bring in a fresh research team, so they hired our company to engage users, find out exactly what was wrong with the current design, and provide guidance on how they could redesign the product.

I’ll never forget the experience of walking into the room where the focus group was to take place to greet this group of participants and being met with glares bordering on open hostility. We explained that we were performing research the company intended to use to improve the design of the product, but that wasn’t anything they hadn’t heard before, and they were understandably skeptical. Our initial attempts to extract meaningful information fell flat as the participants just spat out short comments, then quickly clammed up. It was obvious we would have to win them over and show them we were on their side. Vague phrases such as we recognize that there are problems with the design were not enough to assuage their doubts. To show these participants we were truly ready to make serious changes, we had to do what Domino’s has done in their recent advertisements: be brutally honest about the failings of the existing design. So, after going through an extemporaneous list of the product’s design shortcomings, we were finally able to show the participants that we understood their pain, cared about their feedback, and were ready to make changes.

This kind of experience interacting with user research participants emphasizes two essential parts of successfully communicating with participants: establishing objectivity and building alliances.

Establishing Objectivity
“When engaging with user research participants, it’s essential that you put some distance between yourself and the product concept or design.”

When engaging with user research participants, it’s essential that you put some distance between yourself and the product concept or design. If you have any biases or a personal connection with the product design, participants may feel uncomfortable giving you negative feedback, because they unconsciously sense your disappointment. For this reason, we commonly advise that designers not lead user research for products they’ve designed. If a dedicated user researcher is not an option, we typically advise companies to have a different designer do the user research—one who has no personal attachment to the project. When we are interacting with participants, very early on in our conversations with them, we make a point of letting them know that we did not design the product, so anything they might say will not hurt our feelings. This gives participants the freedom to offer negative feedback without the social stigma of being mean or hurtful.

Building Alliances
“A user researcher is a user advocate. It is his or her job to prioritize the needs of users, even if it creates additional difficulties for the product team.”

When taking on the role of user researcher—whether you are actually a user researcher or usability professional or are a designer, engineer, product manager, or in an another profession—it is important to take to heart that, for that period of time, you are on the side of the users. A user researcher is a user advocate. It is his or her job to prioritize the needs of users, even if it creates additional difficulties for the product team. In doing this, a researcher builds alliances with participants, so participants feel that the researcher—and thus, the company—values their feedback and will use it to influence the design of the product. While establishing objectivity can help participants feel free to speak openly, building alliances can really motivate them to participate actively in a user research session. It’s another strategy you can use to maximize the amount of actionable user feedback you’ll get from a research session.

The Happy Elephant
“A company that enjoys a strongly positive brand perception may have difficulty getting objective feedback about a product….”

Of course, preconceived notions about the quality of a product can also swing in the other direction. A company that enjoys a strongly positive brand perception may have difficulty getting objective feedback about a product, because participants have already made up their minds to love the product before they even lay eyes on it. We’ve tested products with participants who were brand loyalists and overlooked numerous faults in a product’s design. In one particular instance, a participant had difficulty performing a majority of the tasks during a usability test, but reported that he loved the design, so we shouldn’t change a thing. It was only through our pointing out and inquiring about specific areas of difficulty he had encountered that he finally admitted that there were aspects of the user interface that should change. There are two methods that come to mind for dealing with this type of issue: anonymous testing and prioritizing objective data.

Anonymous Testing
“If it is possible to conceal the identity of the company developing the product during user research, it can be useful to do so.”

The easiest way to overcome such brand bias—whether user research participants have positive or negative perceptions of a company’s brands—is to simply avoid it entirely. If it is possible to conceal the identity of the company developing the product during user research, it can be useful to do so. Often, by not associating a brand with a product or concept, we can acquire a more accurate understanding of users’ reactions to the product itself.

As consultants, we can accomplish this by performing user research under our own company name. We can bring participants to our offices and use prototypes and builds that show no company logos. It can be much more difficult to establish anonymity if we are testing at our clients’ offices, but, of course, it isn’t always necessary to do so. We usually do suggest some type of anonymous testing when we’re working with companies that are trying to overcome a strong negative or positive brand perception. However, anonymous testing isn’t necessary for companies that are less well-known or have a more neutral brand perception.

Prioritizing Objective Data
“In cases where you do encounter real brand advocates, it is important to incorporate objective data into your research.”

Objective data refers to the information user researchers collect through direct observation of participants, while subjective data refers to data researchers collect through participants’ verbal reports. In cases where you do encounter real brand advocates, it is important to incorporate objective data into your research. For example, during a usability test, we note specific instances in which a participant has difficulty with a device or user interface. Then, when later interviewing the participant, we typically make statements like this: It’s great that you loved the Web site, but you seemed to have difficulty doing XYZ. A participant usually responds by acknowledging the difficulty, then making comments about their expectations or suggesting improvements to the user interface.

There are also ways to objectively document emotional responses such as frustration, disgust, and happiness. Learn to recognize such emotional cues, document them, and inquire about them when doing any type of product research, including generative user research such as ethnography. Typically, objective data is more accurate than subjective data, so it is important to prioritize data accordingly when findings are inconsistent.

Conclusion
“Show participants that you are on their side and invested in developing a product that will be both useful and enjoyable for them to use.”

Participant biases, both positive and negative, can get in the way of obtaining useful feedback when performing concept or usability testing. There are a number of ways to work around these biases, as follows:
* Establish objectivity. Show participants that you are not personally attached to a concept or design, allowing them to provide feedback without hurting your feelings.
* Build alliances with participants. Show participants that you are on their side and invested in developing a product that will be both useful and enjoyable for them.
* Use anonymous testing. Filter usability testing through a subsidiary or a contracted agency to entirely avoid any brand bias.
* Prioritize objective data. What users actually do is more important that what they say they do.

If you take these lessons to heart and apply them appropriately, they can be enormously helpful in making some of your more difficult interactions with users much more productive.

“To have any hope of satisfying customers’ needs, a company needs to have a deep understanding of their customers and their needs. Through user research, a company can reach out to their customers and gain that understanding.”

In actuality, the reason so many small software businesses fail so quickly is because they don’t understand that their knowledge of how to build a product is not sufficient to build success. The true vision of a multidisciplinary team of engineers, designers, product people, marketers, business people, and researchers is foreign to them. If some of the people creating software don’t understand our role, it’s not surprising consumers don’t. Over time, we’ve learned to liken the user research process to communication between a company and its customers. To have any hope of satisfying customers’ needs, a company needs to have a deep understanding of their customers and their needs. Through user research, a company can reach out to their customers and gain that understanding.

Consumers have two ways in which they can communicate with companies: through their purchasing behavior and through user research. Purchasing behavior is the most important and most definitive of the two. If a company can develop a product that is easy to use and provides value to consumers, consumers tend to demonstrate their approval by purchasing that product. On the other hand, if a product fails to tap into the needs of customers or has a user interface that is difficult to use, sales are likely to be disappointing. For example, over the past few years, American consumers have sent strong messages to American automotive manufacturers through their purchasing behavior, by largely avoiding large trucks and SUVs. Instead, they’ve opted to purchase smaller, more reliable, and more fuel-efficient cars. Detroit’s big three were slow to respond to this message and survived only with the aid of massive government assistance.

When Apple introduced the Newton Message Pad in 1993, the device had difficulty finding a place in the market. Shortly afterward, Palm introduced a line of PDAs (personal digital assistants) that enjoyed huge success, making the brand Palm Pilot synonymous with the term PDA. Palm was able to succeed where Apple had floundered—thanks to their ability to tap into and understand what consumers wanted. Apple was eventually able to learn from their foray into PDA territory and introduce their revolutionary iPhone, but the lessons that brought them to that point were rather costly.
“A product’s ability to meet the needs of consumers and fit into their lives is key to its success.”

Palm, on the other hand, introduced the Palm Pilot without a similarly costly learning process. They were able to accomplish this through user research, as Dev Patnaik and Robert Becker documented in their classic article “Needfinding: The Why and How of Uncovering People’s Needs.” Toyota learned similar lessons through user research. When developing a new vehicle to appeal to an aging Baby Boomer market segment, they turned to IDEO to guide the design process. IDEO engaged in a program of design research that established the needs of the demographic, letting Toyota develop a vehicle that adapted to the lives of adults who no longer need to drive their children from place to place. A product’s ability to meet the needs of consumers and fit into their lives is key to its success.

If a decision to purchase is the final word in a conversation between a company and consumers, user research makes the first impression. Just as communication is an interaction between people, user research is an interaction between a company and its market. It lets a company get to know the people who are its potential customers before making a decision about establishing a relationship with them. Just like in relationships between people, it’s important to assess compatibility before making a commitment. Similarly, it’s very common for consumers to do their own product research before committing to a purchase. They may accomplish this by reading product descriptions, specifications, and reviews and talking to their friends and neighbors who might have prior experience with a product or brand.
“While observing behavior can tell you a great deal about what people do, it’s only through talking them that you can learn why they do it. The why behind behavior indicates what drives it.”

As a form of communication, user research is subject to many of the same rules as interpersonal communication. This is the reason we’ve studied communication to inform our research methods. Good communication helps ensure good data. And good data helps ensure a successful product.

Throughout a user research process, a researcher is a company’s representative and participants represent the market. As the two engage in a conversation about a collection of features that might eventually coalesce into a single product, their ability to understand one another is vital to ensuring the product meets the needs of its intended users. Though there are various ways in which a user researcher can objectively measure behavior, the best way to get a complete understanding of people’s experience with a product is through communication. While observing behavior can tell you a great deal about what people do, it’s only through talking them that you can learn why they do it. The why behind behavior indicates what drives it.
“Concern for customers is essential, and the communication a user researcher has with customers reflects that.”

It’s important for a user researcher to be a good representative for the company. As the company’s representative, the researcher represents the brand. Participants’ first impressions of the company depend on the researcher. He should always demonstrate a real concern for the well-being of the company’s customers. Only through this deep and legitimate concern for customers can the researcher develop the kind of understanding of customer needs that can inform the design of a product or service and ensure it fits into people’s lives. It is also the only way to convince participants that the company cares about their input and really intends to use it to guide design—so expressing their experience is worthwhile. If a researcher fails to establish this trust, participants might not feel comfortable opening up about their experience.

For example, we’ve seen someone absolutely scowl as soon as she picked up a product and started looking it over, our immediate assumption was that she hated it. When we inquired, we learned that she actually liked the product, but she’d just had an argument with her sister before coming to the research session. Armed with this knowledge, we could pause the session for a few minutes, talking with her to get her mind off her sister. Then, when we brought her focus back to the product, we witnessed an entirely different display of emotion. If we had not taken the time to talk with the participant before diving into the study, she might not have felt comfortable revealing her argument with her sister. She might have responded, Oh, it’s nothing, the product is fine, and left it at that.
“Deeply understanding their customers is what allows successful companies to think five years ahead of the market and develop products and services that revolutionize the way we live our lives.”

Concern for customers is essential, and the communication a user researcher has with customers reflects that—for example, in a desire to know why a customer has a hard time sleeping at night or just can’t find the time to get to the gym, in curiosity about why a frown just crept across a person’s face or why he cracked a smile, or in thoughtful questions about a person’s life.

Deeply understanding their customers is what allows successful companies to think five years ahead of the market and develop products and services that revolutionize the way we live our lives. Rather than just responding to what’s already on the market with derivative knockoffs and copies of what was at one time disruptive technology, companies that listen to their customers can truly innovate, without shooting in the dark like so many startups that just take a chance on an idea. By understanding the market and the needs of their customers, these companies can develop products customers want and put themselves in the best position to achieve success.

This article will also outline what to expect from researchers and some ways to recognize when you’re working with a good one. These are indicators, not standards, based on what we’ve found to be effective. There are many ways to do research and every research study is different so it doesn’t mean that a researcher is incompetent if he or she doesn’t conform to these indicators. One sign of a strong researcher is that he or she will educate you throughout the process so that you know what to expect. With that in mind this article is ultimately intended to provide a useful starting point.

Recruiting

One of the most critical and time-consuming elements of test preparation is defining the right target audience and recruiting participants. Participant recruiting is usually conducted by professional recruiters who typically consult databases of potential participants. Sometimes researchers will do the recruiting themselves, but it’s usually more cost effective to use a specialist.

Recruiting will almost always take two weeks or more depending on the number of participants and the type of research, so make sure that you get started early enough for the recruiter to have enough time to find the appropriate participants for the study. Recruiting for phone interviews may take slightly less time and any kind of home visit will likely take longer (ethnography or contextual interview). Your researcher should be able to provide you with an estimate at the time of initial engagement.

A week for recruiting tends to be difficult and any less than that is pretty much unthinkable. Short-changing the recruiting could result in participants that don’t properly fit the target market segment, don’t provide quality feedback, or just don’t show up at all. All of these can have a negative impact on the data. Even if it is possible to get participants faster, it’s usually better to take the time to ensure that you are getting the right people. Your researcher should know all of this and recruiting participants is where he or she will start after getting a basic understanding of your product and schedule.

A recruiter will need a screener to get started. A screener is a description of the target user with open and close-ended questions about the participant that will help the recruiter to select the right people. What you can do to smooth the process along is to have a prepared concept of your target user. This does not need to be a full market research report—just an outline of the types of users that will use your product.

Your researcher should dig deep with questions that include more than demographic information by asking behavioral questions. Behavioral questions can include such topics as TV watching behavior, purchasing behavior, internet use, etc. Typically behavioral questions will give you a stronger understanding of those who are being recruited than demographics alone. These are important elements of market segmentation that are sometimes organized into profiles called personas.

Personas are useful because they create a consistent concept of the intended market segment that can guide the design process through multiple iterations. Personas can also be adjusted following deeper discovery research, such as in-depth interviews, as more information about the intended user comes to light. Within a few days, the researcher should present a screener that includes behavioral questions as well as demographics.

Scheduling

When creating a schedule for data collection, the researcher should know that you cannot run participants back to back. It’s generally not feasible to squeeze in 8 one-hour sessions in a single day, because of all of the activity that must occur between sessions. In an eight hour day, a researcher can perform four (maybe five) one-hour sessions but any more than that will take more time. Here are the reasons why:

One-hour sessions rarely go exactly one hour, some are shorter and quite a few will run longer. This can be due to a variety of reasons such as the product malfunctioning, the participant arriving late, or the participant providing lots of feedback. My rule of thumb is to allocate 50% of the session length as a buffer between sessions to allow for overrun, not including time needed to set up for the next session.

For sessions at an office or lab, some participants will arrive 10-20 minutes early, at which time they will need to use the restroom, sign NDAs and consent forms, and generally get comfortable. Comfortable participants give useful feedback, while uncomfortable participants tend to clam up and provide short, unemotional responses.
The researcher needs to set up and get ready. For usability or experience testing, the test will need to be reset, notes and documents need to be filed and new ones prepared. For any kind of home or location visit, the researcher will need to pack up all equipment and travel to the new location and set up equipment again.

Thus for every one-hour usability or experience testing session, there’s forty-five minutes to an hour of buffer and setup time. Home visits can take much longer.

Test Plan

A test plan should take no more than a week to develop and the researcher should give it to you for review and approval before being finalized. The test plan should specify the research and business goals associated with the project. During this period the researcher will need a significant amount of time with the product, either with a prototype or available concepts, while writing and checking the test plan. The better the researcher understands the intended final product, the more valuable the information he or she can get from the participant.

For usability or experience testing, the researcher will test the tasks with the product prior to a pilot test. He or she will need to make sure that there are no glitches, no unexpected areas under construction, and nothing giving away future tasks when performing each of the tasks with the product. With that in mind, it’s important to give the researcher a stable product or prototype and avoid drastic changes to the product prior to the test.

You should receive a well-written and organized test plan that details each research question and how it will be addressed. For usability testing this will include a list of tasks, what each task is intended to examine, approximate wording for the task (avoiding leading language), and detail on how each task will be scored or evaluated. For discovery research, it will include a list of topics to be addressed such as processes, environment and context, and expected pain points and needs.

Data Collection

When the data collection starts, it’s important to let the moderator work. During this time, the participant should feel comfortable enough to open up and provide honest feedback. In order to do this, it’s important to try to minimize observer impact during the testing session.

If you don’t have a separate place to watch the session (e.g. behind a two-way mirror or through a video feed), don’t make it obvious that you are paying close attention. Think about bringing in a laptop during the session to make it look like you’re doing other work. One way of doing this is telling the participant that you are also a researcher but you’re just going to be taking notes.

When you’re observing, remain objective and don’t make judgments based on one or two participants. It’s not uncommon to see a couple participants have a completely opposite reaction to a product compared to ten other participants. The researcher’s job is to sort through all the noise and report the real trends in the research. Take what you see with a grain of salt and listen to your researcher.

At the same time, it’s important to try to observe as many sessions as possible and give your researcher feedback between sessions if there are certain aspects of the user experience you want to know more about. The researcher should put the participant at ease and extract a great deal of information, including details that might have been overlooked or emotions that the person experiences. Different researchers will tend to achieve this in different ways as everyone has their own style, but you’ll notice by paying attention to the participant and seeing if they feel relaxed or nervous throughout testing.

Findings

Frequently, stakeholders will want to make immediate changes to a design, product, or prototype and won’t have the time to wait for the researcher’s final report. People have schedules that need to be met so it’s understandable that a project can’t always wait for the final report but the researcher should be able to provide you with quick findings within 24 hours of the last session.

For usability research, these quick findings should consist of a couple of short paragraphs including problems in the interface, possible solutions to these problems, and participants’ general reactions to the product, its look and feel, and expected usage. For ethnography or other forms of discovery research quick findings will tend to consist of expected usage of the product, expected value, high and low value features, and general trends about the intended user. Quick findings aren’t comprehensive and come before the researcher can get a complete look at the data, but it will provide you with the overall themes from the study.

When you do get the final report, make sure you take a look at it. It will tell you two things:

• Detailed findings regarding the interface, product, features, and intended user
• The quality and clarity of the report will tell you quite a bit about the quality of your researcher.

There’s one other thing to keep in mind when you are processing the findings from a usability test. The participants will tend to focus on the more obvious problems with a product or interface. There could be other, smaller or more abstract problems that are not identified in the first pass of usability testing. It’s usually a good idea to perform another test on the product after making changes to ensure that the changes you made were effective and identify any additional issues.

Summary

In summary, here are the most important points for non-researchers to know about the research process:

1. Recruiting will almost always take two weeks or more.
2. For every one-hour usability or experience testing session, there’s forty-five minutes to an hour of buffer and setup time, home visits can take much longer.
3. The researcher will need a significant amount of time with the product (prototypes or concepts) while writing and checking the test plan.
4. Try to minimize your impact during the testing session.
5. Remain objective and don’t make judgments based on one or two participants.
6. Ask your researcher to provide you with quick findings within 24 hours of the last session.

What would you do? How would you handle this situation?

Throughout the past six years, the founders of ActiveComm Labs have not only been performing design research but also assisting the law enforcement community by conducting research on the communication patterns of hostage negotiators. Specifically, we have been analyzing the communication between the hostage negotiator and hostage taker to locate patterns that could introduce new strategies to help resolve crisis situations peacefully.

We’ve come to realize that the techniques used by hostage negotiators to resolve crises are also extremely valuable to user experience researchers. In essence, both parties are attempting to establish a relationship, both are trying to keep the communication flowing, and most importantly, both are trying to extract valuable data.

There are certain myths about hostage takers. Most of them are not bank robbers or terrorists demanding millions of dollars and a plane to Cuba. The vast majority of hostage situations are a result of domestic violence, psychological disorder, or barricade situations in which a person is threatening to commit suicide, possibly with a child in the next room. Hostage takers are usually confused, upset, and very scared. It’s also pretty rare for them to be outright hostile toward the negotiator. Hostage negotiators are trained to gather important data about the situation. Who’s in there? Is anyone hurt? What kind of weapons does the hostage taker have? How much ammo does he have? To do this, negotiators have to master a variety of communication techniques.

Have you ever worked with a research participant who will only give you “yes and no” answers? How about a participant who tells you exactly what you want to hear? These situations can be frustrating, especially when you invest so much time and energy in recruiting candidates. But these experiences don’t necessarily mean that these people can’t provide valuable data, it means that you need a different approach to extract that information.

A research session isn’t usually an emotionally charged situation and research participants aren’t typically in crisis, but the fundamentals of communication tend to hold true across different types of people and contexts. Our negotiation instructor told us that we should approach a hostage negotiation in much the same way as going on a first date; it’s important to bring a certain level of calm into the situation and put the hostage taker at ease. It is also extremely important to connect with the hostage taker on a personal level. Negotiation provides a great example of how to perform this kind of communication because it demonstrates these fundamental communication elements under the most difficult of conditions. Ultimately, negotiation is about two strangers coming together to work toward a common goal built on an understanding of each other, much like design research.

Application to Design Research

There are two types of behavior that we try to extract when conducting research:

1. What the participant does (physical interaction with product)

2. What he or she says (communication about the product)

Our goal is then to study the interaction between these behaviors in order to tell a story about the user’s experience of the product.

One of the most difficult parts of research is getting the participants to tell us their story about the product. Some researchers only focus on physical interaction data, but we think too much valuable content is lost. We’ve found that the communication piece of the equation provides the emotional and logical connection that participants make with products and how it relates to their lives.

With that said, one of the most common issues with communication-related data is how to gather accurate information. What a participant says is not always what he or she believes, and what a participant does is not always what the participant reports.

Much like a hostage negotiator, who must build trust in order to successfully resolve the crisis, a user experience researcher must establish a relationship with the participant in order to extract useful and accurate information. So, the fundamental element of becoming a better communicator, and also researcher, is to establish a relationship. Hostage negotiators focus on establishing relationships in order to save lives, there’s much we can learn from the methods that they have established.

Learning from Negotiators

Dominick J. Misino is a retired NYPD crisis negotiator who has been involved in more than 200 hostage and barricade incidents. He is recognized for his successful resolution of the Lufthansa hijacking in 1993 and numerous other successful negotiations. When it comes to communicating, Dominick knows what he’s doing.

Since retiring, Dominick began training other hostage negotiators. To date, he’s trained thousands of negotiators across the country and around the world. A few years ago, we had the privilege to attend all three phases of Dominick’s negotiator training and certification program, which included hands-on practice as a negotiator and hostage taker. We learned communication techniques that we currently employ when interacting with research participants. These techniques include building rapport, building alliances, and using a team approach.

Building Rapport

Rapport is established through trust, open communication and empathy. Negotiators know that rapport is essential in their job. They use rapport to influence the hostage taker and gather information. If you can effectively build rapport with the participant, there is a higher likelihood he or she will trust you and disclose more information.

The following techniques used by hostage negotiators can help you build rapport with research participants:

1. Go slow – Engage in small talk at first. If you dive right into business, the situation can become uncomfortable.

2. Communicate openly – While you can’t disclose everything, it’s important to encourage an atmosphere of open communication. Tell the participant that there are certain aspects of the study that you can’t reveal, but he or she shouldn’t feel that you’re hiding something.

3. Actively listen – When you are listening to a participant’s story, listen for the emotions behind the words. Ask open-ended questions that dig for the source of those emotions.

4. Discuss personal topics – In a hostage situation, some of the most valuable topics that lead to a peaceful resolution are personal ones. The more a person feels that you accept them, the more comfortable they will feel with you.

5. Share your experiences – Building rapport is as much about sharing your experiences as it is about listening to the other person’s. Negotiators know that the more you reveal about yourself, the more the participant feels like he or she knows you and therefore trusts you.

6. Show you care – Hostage negotiators build rapport through empathy. Empathy is extremely important because it shows that you care about the other person and that you have their best interests in mind. As a researcher, you should do this also. If you show that you care, the participant will appreciate it and respond with more openness.

Alliances

In a hostage situation, the negotiator works for the police department but he has to show the hostage taker that he’s on his side. In order to do this, the negotiator can never be the one in charge; it cripples his or her ability to negotiate. Anytime the negotiator has to tell the hostage taker “no” it’s because his boss is being a jerk. Anytime the negotiator says “yes,” whether it’s a pack of cigarettes or just some extra time, it’s because the negotiator fought hard to get it for him. The negotiator intentionally shifts the blame for anything negative and takes credit for anything positive. It convinces the hostage taker that the negotiator is on his side.

In user experience research, the researcher is on the side of the user. In our work, we establish this by telling the participant that we are not the people who designed the product and that their comments, whether good or bad, will not offend us. This establishes objectivity and allows a certain freedom in the research session. In most cases, participants open up when they hear that you have nothing at stake. Also, if the participant can see that you share his or her common goal of improving the product, the participant is more likely to be truthful in his or her evaluation.

Team Approach

Hostage negotiators always work in teams, and so should you. In the event of a hostage situation, a negotiation team is called to manage the situation. Each person in that team holds a different but critical role in the event. One of the most important positions on that team is the coach. As the negotiator acts as the primary point of contact with the hostage taker, the coach sits with the negotiator and functions as another pair of ears. The communication can move very quickly during a negotiation, and the negotiator can have a hard time catching all of it. The coach specializes in listening, controlling access to the negotiator, generating questions and helping guide the communication process by passing notes to the hostage negotiator.

Through crisis negotiation training, my partner and I have learned that the ability to gather useful and accurate information dramatically increases when you work in teams. For example, when conducting an expert interview we have one person ask the questions and another person as a secondary moderator. Like the coach in negotiation, the secondary moderator listens closely, takes detailed notes and chimes in when he feels that something is being missed. This type of setup will reap maximum data in the shortest amount of time. We can’t always work in teams for logistic or financial reasons, but it is our preferred method.

Training

For hostage negotiators, training is a crucial part of the job and they understand that the more you train, the more comfortable you will feel in the situation and in turn the better the outcome.

From what I have seen in the user experience community, little or no time is spent training on the best practices of communicating with participants. Every so often a workshop is attended but that only happens a few times a year. If we take a page from the book of negotiating, we would learn that just a little bit of training on a regular basis will take us to a whole new level of success. Here are some modified exercises that we can use to polish our communication skills as researchers:

1. Communicate with new people all the time – every time you have a chance to meet someone new and learn about their life, do it. Ask questions about who they are, where they come from and what they’re like. When you start to feel more comfortable doing this, start pushing yourself and asking more intimate questions about their life such as “What keeps you up at night?” Play a little game with yourself where you try to learn as much as you can about a person in a short amount of time. Three things will happen when you do this. First, you will learn about boundaries and what you can ask and you should not ask. This doesn’t mean that you’ll necessarily insult people and then learn not to do it; it means that you will see what topics are challenging to people and how to pull that information out of them without upsetting them. You’ll also learn more about people in general and how they function in the world. You’ll get a better sense of behavior and start to see trends in people. Finally, you will make more friends in the process and that is always a nice outcome.

2. Learn to listen – There is a famous question that asks, “When someone is speaking to you, do you think about what they are saying or do you think about what you are going to say next?” This is a very important question when learning to communicate more effectively with others. When you are communicating with participants or friends, really listen to what they are saying. Listen to the emotions behind the words, look at body language, and ask questions about their responses if they are unclear. There are sometimes differences between what a user will answer and what they believe. Ask yourself what those statements say about the speaker as a person; it will enable you to discover the areas where people are most passionate.

3. Learn how to disclose – When talking with friends and new people, start disclosing about your life and observe how people respond. Most people will feel more comfortable sharing information if you share information also. You’ll be amazed at how quickly people will open up.

4. Learn to trust your gut – When working with a participant or talking with a friend, learn to listen to your instincts. There are times when you need to speak up, times when you need to bring the communication back on track and times when you need to let the other person just open up and talk about whatever they feel is important. Different types of communication are needed for different situations. If you train frequently, you will notice that your gut tells you what you need to do.

Conclusion

This article began with a scenario of a hostage situation. After the first 30 minutes, the hostage taker and negotiator were talking like old friends about food, sports, and pets. At two hours, the hostage taker confided in the negotiator about his relationship problems and issues that led to him losing his job. At two and a half hours, the young man sent his mother out of the house, despite her protests that she wanted to stay with her son. At four hours, the young man placed his empty gun in a bucket attached to a rope outside an open window, where it was retrieved by the police tactical team. At 7:50am, after five hours and fifty minutes of negotiation, the young man peacefully exited his home and surrendered to police. The negotiator followed up on all of his promises. He rode with the young man to the police station, allowed his mother to visit so that he could apologize and even made a statement on the young man’s behalf at his trial, resulting in a reduced sentence.

This process should be mirrored in a research session in a condensed period of time. After the first ten or fifteen minutes, the participant should feel like you know each other and feel pretty comfortable talking with you about your product or service. After about twenty minutes, the participant should understand the product and its goals. After thirty minutes, the participant should be discussing how the product would fit into his or her life. In order to achieve this, some form of communication training should be implemented. Researchers typically receive a great deal of training in research methods, statistics, human factors and elements of design, but little training on advanced communication. Researchers who really want to invest into their skills as a researcher should think about spending time and energy to learn effective communication methods.

I always make a point of clarifying what they mean as the difference in price and approach between the two can be dramatic. Understanding the differences between these two types of methodologies can help you to more effectively plan your user research, maximizing the value of the research while keeping cost in line. To cut to the chase, here are the different research approaches and their strengths and weaknesses.

Contextual Interviews
Contextual interviews are interviews that are conducted in the context in which the behavior of interest occurs. For example, if someone were trying to understand the needs of doctors, one would interview a doctor in his office or operating room or other relevant location. A typical contextual interview consists of a visit to a home or place of business, usually for an hour or two. The participant demonstrates certain processes of interest for the researcher, such as posting an item for sale on an ecommerce website or sitting down to watch a movie. The researcher asks questions about the process to get a clear understanding of it and identify pain points for the participant. In this way, the researcher can get an inventory of activities in which the participant engages when going through a process. In formal human factors research, this is referred to as a task analysis.

Contextual Interviewing is a superior form of interviewing as it allows the researcher to understand the person’s environment and get actual demonstrations of behaviors of interest. It also has another advantage; it helps the interviewee remember specific details about performing actions. A contextual interview is not perfect. It is a type of self-report research and is subject to the same weaknesses as all other forms of self-report research. People often distort, misremember, or overlook important facts when providing information. These inaccuracies in data are usually unconscious and extremely difficult to eliminate or control. In addition, people can get very accustomed to their pain points, adapting to them and working around them to such a point that they become practically unaware of them. It takes someone objective to identify these kinds of needs.

Thus, while contextual interviews are very useful for identifying more obvious needs, they don’t provide an objective and in-depth understanding of consumers or their needs. There’s always a difference between what people do and what they say the do.

Ethnography
The ethnography that is used in design research was adapted from the field of anthropology in which anthropologists would study different cultures by immersing themselves in the culture for months or years at a time. Design research cannot accommodate the same kind of budgets and schedules as traditional ethnography; however, the principle of in-person observation of behavior remains at the core of ethnography as a research method. In true ethnography, a researcher will spend one or more entire days with a research participant, profiling the person’s life from waking up in the morning to going to bed at night. An ethnographer might travel with the participant to work, riding together on the train and noting behavior in transit. He might sit with him in the office, eat with him during lunch, or observe him at the gym after work. Rather than relying on self-report data, the goal of ethnography is to directly observe and document the actual behavior of the participant and search for patterns and the needs underlying those patterns, rather than relying on the participant to recall them.

In other forms of true ethnography, the researcher may setup a camera and record the participant’s behavior for extended periods of time rather than being in the room with him or her. For example, if you wanted to understand how a surgeon functions with certain instruments when performing a procedure, a camera may be installed above the operating table to record the operation. This allows the researcher to review the recording over and over until behaviors of interest are uncovered.

Like contextual interviews, ethnography has its strengths and weaknesses. Not the least among these weaknesses is the high cost and schedule requirements. Ethnography is definitely one of the more expensive and time consuming forms of research, as a result it is not regularly used, nor is it appropriate for every project. On the other hand, there are researchers like us who are developing new ethnographic methodologies that would reduce costs.

From a methodological perspective, ethnography is subject to participant reactivity sometimes called the “Observer Effect.” The Observer Effect occurs when a research participant alters his or her behavior in response to the knowledge of being observed. A classic example is when a participant performs a thorough cleaning of his home rather than allowing the researcher to see how his environment exists normally. This is one of the reasons why ethnography requires a significant amount of time with a participant, the reason being that the participant’s behavior will gradually return to normal as he or she becomes accustomed to the researcher’s presence. In traditional ethnography, the participants get so accustomed to the researcher’s presence that the observer effect is believed to be almost eliminated. Unfortunately, design cycles and budgets don’t allow for these kinds of commitments, which can stretch out months or even years. Despite these shortcomings, ethnography is currently the most in-depth form of need-finding user research. It allows the researcher and design team to have a thorough understanding of the consumer and identify needs that the consumer might not be aware of. It is best used when making a major investment into a product that is positioned to revolutionize the market such as email or the automobile.

Conclusion
All research has its place and all research provides value when applied properly. The key is to understand each methodology and where they can be most effectively used. Contextual interviews are an excellent way of understanding consumers and identifying their needs and ethnography is an excellent way to dig deep to identify needs that are more difficult to find. Using ethnography when iterating a current product or technology probably wouldn’t result in much added value beyond performing a contextual interview, but it is essential when creating something entirely new and different.

Mobile Upload by: Bryan McClain

Thoughts?

Mobile post by Bryan McClain

The key to understanding when and how to use focus groups is to understand its underlying strengths and weaknesses. Once we understand these principles, we can design our research around them and take advantage of a means to include the user in the product design process.

Focus Group Prerequisites
Focus groups are commonly used to try to answer certain questions about the market including identifying the intended market segment and user needs. The problem being that focus groups are ill-suited to provide this information, rather consumer identity and consumer needs should be thought of as prerequisites for getting useful data from focus groups.

Focus groups cannot identify your consumer. You must know in advance what market segment you are addressing and make sure that your participants reflect that segment. It’s essential to have the right participants in the room, if you don’t, you can easily design to meet needs of an unintended market or no real market at all.

Focus groups cannot identify consumer needs. More often than not, people are so accustomed to their needs that they don’t even recognize them. They’ll find ways to work around their need rather than addressing them and definitely won’t be able to fully describe their experiences in a room full of people talking. It takes an objective person that challenges assumptions and thinks outside the box to recognize and truly address their own needs.

Focus groups need effective mediation. People do not naturally collaborate. In any group setting, some voices will naturally be louder and more outspoken. It’s the mediator’s job to not allow some personalities to dominate the group and allow other voices to go unheard. (http://en.wikipedia.org/wiki/Groupthink).

Strengths
As mentioned above, focus groups cannot identify the consumer or locate needs. However, if you have the right people in the room, you understand their needs, and you are realistic about the kind of data that you can get, then you can acquire quite a bit of value from a focus group.

What focus groups can tell you is an initial reaction to the question of HOW to address your consumer’s needs. Specifically, a good focus group can give you clues regarding how to design an interaction based on a consumer’s expectations and the context in which the product will be used. For example, a focus group can tell you how much time and effort a consumer says they are willing to commit to satisfying a need; it can tell you how quickly a product has to work, whether it should require single-handed operation, or whether it should be a tap, swipe, or shake form of interaction. A good focus group should feel more similar to an ideation session with the consumer involved. Different consumers should be thinking about the way in which they would like a product to work, adding and removing features and defining the interactions that would work and wouldn’t work. In this kind of situation you can get a verification of needs, validation of design concepts, and discover additional functional requirements for the interface, product or service.

Focus groups also provide very important pricing information. A price point can be very important before engaging in design. A great design that the consumer can’t afford or has a price that is significantly higher than perceived value will fail in the market. This is one of the reasons why it’s important to have a diverse and collaborative team. Once research has established an expected price point, a designer can begin to create concepts while engineers provide feedback on expected manufacturing costs. Focus groups allow you to have a conversation with the consumer around pricing, where more traditional methods of pricing analysis, such as surveys, don’t.

Conclusion
Focus groups can be useful when applied appropriately. Problems occur when they are used to replace other forms of discovery and market research in an attempt to save money. If one can recognize their two fundamental failings, their inability to identify consumers and their needs, then they can be very useful. Of course, there are other ways to address the same research questions, but there’s nothing more efficient than having a group of users in the room engaging in the design process. The concepts and designs that come out of direct interaction between designers and consumers can lead to real magic, plus they can be cost and schedule effective.

I was out for walk during lunch in downtown SJ and I was happy to see families enjoying the water fountain area setup for their enjoyment on these hot afternoons. It makes me wonder when more cities will design for the people and not just for aesthetics. Thoughts?

Mobile post by: Bryan McClain