The Art and Craft of User Research Interviewing

Share Your Fieldwork War Story

2012-05-10T19:02:51Z

I've started a new series of fieldwork war stories, and I'd love to get your examples as well...(if you are reading this in Google Reader, etc. you may simply see a link to an online form, but if you open the page itself, you will see the form and a bit more about what I'm asking for!)

Fill out my online form.

Nick Bowmast's design researcher toolkit

2012-05-08T19:44:55Z

Nick Bowmast has thought in-depth about what to take with him when he goes into the field

With design research (contextual inquiry or ethnography, if you like), there's an amount of planning and kit required too - I've been running in-home interviews, playing both manager and roadie roles, but isolating these activities as much as possible from my role as researcher.

Every minute spent with a customer is valuable, so I can't afford to be distracted by practicalities like recording equipment and timings.

After a few years experimenting with these practicalities I've arrived at a 'toolkit' of things in my backpack, so when I pull up at the customer's house the 'roadie' can take a back-seat and let me get on with capturing the insights.

Check out his 'toolkit' here.

Documentaries as interviewing exemplars

2012-04-07T18:15:23Z

In the book I describe a scene from Werner Herzog's Grizzly Man; but great (and not-so-great) examples of interviewing best practices can easily be found in documentaries. A nice little blurb from the New York Times review, 'Player Hating,' Maggie Hadleigh-West's Documentary

In contrast to the approach she took in her confrontational 1998 documentary, "War Zone," she stays mostly in the background, making no apologies for her subjects' off-screen crimes. Rather, like the best interviewers, she holds her shots for several seconds after someone has ceased talking, knowing that silence, like an empty belly, demands to be filled.

Meet Google's search anthropologist

2012-03-30T16:04:26Z

Some nice bits about the process (including a lovely mention of "listening") in this article today

But about four years after forming, Google came to realize it needed human insights to infuse that information with context and meaning.
The company began conducting user research studies and hiring human-computer interactions experts, eventually snagging Russell from IBM in 2005. His main role is studying Web searchers in their natural environment, at home or work, picking up the human scent where the data trail goes cold.

"One of the things we can get from data is the behaviors," he said. "But in many cases, we don't know why the behaviors are the way they are."

For instance, just knowing that people were fleeing from advanced search didn't begin to tell the company how to fix it.
...
But at least one change made around the end of last year seems to have had a lasting impact. Google peeled the "advanced search" button off the main page to make it clean and simple, qualities users always request. Usage numbers declined, however, and stayed down.

During a trip to the San Francisco Public Library this month, it was clear why. Russell spent about an hour observing and talking to Patrick Shea, a librarian working on the ground floor.

He asked about typical patron inquiries and the search tools Shea employs to help them. Then he ran him through some tests, asking how he'd use Google to find vague queries like: "A book about oranges by a Scottish author."

At one point when Shea was stumped, Russell suggested he try advanced search. But Shea couldn't find it. The company hadn't just moved his cheese, they'd hidden it.

Russell has heard from a number of other librarians who can't seem to find it anymore. But when asked, he said that doesn't necessarily mean Google is going to put it back.

Google has hundreds of millions of users, each with different needs, working styles and levels of search competence. Every change for one subset - like those who occasionally use advanced search - comes at a cost for others - like the vast majority of people who never use it and don't want it cluttering up the main page.

Striking the right balance for the greater good requires listening to the data - and, of course, to the users themselves.

"That particular interview didn't finish off the painting," Russell said. "But every interview helps fill in a little bit more of the canvas."

Sketchnotes resources

2012-02-24T20:18:42Z

In the chapter on documenting interviews, I devote a little bit of ink to visual form (which include a variety of techniques, such as visual recording, visual note-taking/sketchnotes, and mind-mapping). While I can only focus on a limited aspect of these tools, I wanted to offer a bit more info. Adam Menter has generously given me permission to post a PowerPoint presentation and the associated PDF handout from a talk he gave at CCA in 2008.

Visual Recording Presentation (3.3M PPT)

Visual Recording Handout (2.3M PDF)

Tips to Improve Your Interviewing Skills (and a request for more!)

2012-01-10T23:19:56Z

I'm working on some of the final chapters and am interested in the ways that people have developed their own skills as an interviewer. I'll list a few but this list can only get better with your input.

Practice, man, practice. It's how you get to Carnegie Hall and it's how you get better at interviewing.
Create your own practice occasions: that chatty seat mate on an airplane, the extroverted cashier - ask them a question and then ask them a follow up questions!
Reflect, just like a football coach who reviews the game films; watch your videos, read your transcripts, and look at what worked well and what you might have improved
Be interviewed whether it's for a survey or a usability study or a poll, experience the interview from the other side of the lens
Critique the interviews of others (without resorting to your just-got-your-drivers'-license-know-it-all we all were at 16)
Observe others at work including great interviewers and poor interviewers - this can be in your work context, or in the media (Marc Maron, Charlie Rose, Terry Gross, and others)
Collect war stories (more on this coming very soon)
Try improv

That's my starter list, but what have you done to get better as an interviewer?

Multi-platform rapport

2012-01-10T21:11:51Z

I encountered an amusingly challenging interview situation earlier this week. My team and I were at a client's site to kick off a project, including some interviews with stakeholders. For a variety of reasons (access and scheduling being the most obvious) we had agreed to interview the different stakeholders in pairs. Our meetings were spread throughout the day so in between sessions we were hanging out of the corporate lobby, working on our laptops, etc.

Our main client burst into the lobby about two minutes before our interview, startling us from our LCD-screen-reverie. "Let's go!" he urged us, as we grabbed up our gear and headed into the elevator. We arrived at an upper floor and followed him into a conference room, where he began dialing the speakerphone on the table. I just wasn't sure what was happening, so we all dumped our stuff and took seats. Our client got the stakeholder on the phone (we had an inkling it might be on the phone but we weren't entirely sure), threw the call to us to introduce ourselves and then disappeared to find the other stakeholder.

There was a slightly awkward moment as we took the reins of the call, beginning by asking our interviewee if he knew much about why we were in this phone call. As it often happens with stakeholders - or even users - they've agreed but aren't sure what the dealio really is. So we clarified the plan and then began asking him questions. As he regaled us with really interesting, the other stakeholder walked into the room, unescorted by our main client. Meanwhile, our telephone stakeholder is continuing with a relevant and interesting story.

What to do? This was a crisis of manners, exacerbated by the interviewing context. We acknowledged our newcomer with our eyes, with gestures, and a few whispered names/intros (which I'm sure the sensitive phone mic picked up). When the story finished, we acknowledged our newcomer, who fortunately knew the telephone stakeholder. We went back and restated the goals of the conversation, which of course our friend on the phone had already been through.

Anyway, this new stakeholder was equally fascinating and voluble, and quickly offered up a number of detailed examples and stories for us. I'm sure you've been in a group conference call when everyone is in a room - except for you. It's so easy to end up completely marginalized. I did not want to do this to our telephone stakeholder, especially with the momentum we'd already begun to build with him. But our other stakeholder is telling some long stories.

And this is where I caught myself flicking my eye contact between the two, as a way to (I guess - it was an automatic gesture) demonstrate interest and maintain engagement. Except one person was on the phone. Yes, I was looking back and forth between the guy in the room and the phone. I was projecting all of my rapport building onto a device, using eye contact only. Needless to say this wasn't very effective!

I think we muddled through, directing questions to our telephone interviewee, using his name, and otherwise trying to maintain our connection across all platforms. But the special case of interviewing more than one person and interviewing over the phone are challenging enough, finding ourselves trying to manage the combination of the two was an extra level of complexity. When I caught myself trying to make eye contact with the phone was when I knew that were in a very small pickle, but that my instincts were carrying me.

Crafting the Participant Agreement

2011-10-31T18:59:56Z

I'm so pleased that Harry Boadwee, a Silicon Valley lawyer, has contributed the following, which will be included in the book as a sidebar. It's great to get a legal perspective on the need, intent, and process for the document we create to manage our agreement with participants.

In the meantime, I'm looking for some stellar examples, especially from corporations who have had to go through their own legal department and still come out the other end with something vaguely consumer-ready. If you've got something you'd like to share for possible inclusion in the book, please get in touch!

Participant agreements for user research are simple but necessary in order to protect the study sponsors. Although a contract can be formed by spoken promises, or by a loose exchange of emails, the best and common practice is to sign a contract. Participant agreements must be short, often no longer than a page, so that participants can read and sign them quickly, without negotiation. Even a document with a short-form title such as "Permission" or "Release" can be a contract.

A typical participant agreement covers two main concerns.

First, the participant will agree to keep confidential the information disclosed in the study and make no use of the information beyond participation in the study. Such information can include, for example, the questions raised in the study as well as the details of the product or service being studied. For example, many studies cover early stage concepts or unreleased products/services. Study sponsors obviously don't want participants to disclose this information to competitors or (even worse) to the public by blogging or tweeting about it.

Second, the participant will grant permission to the sponsor to record, reproduce, display and distribute the participant's responses, voice and likeness without any additional compensation or royalty. A "likeness" can encompass video, photographs or even drawings. The study may include some small consideration, such as a T-shirt or gift card for a token amount. The release is needed in order to prevent legal claims for compensation under the privacy and publicity laws of many states. These releases often include an explicit waiver of any right to inspect or approve the materials created in the study.

These types of releases most often cover the sponsor's internal use only. Because the concepts and products/services in the study often are in an early stage, the sponsor probably wouldn't want to use a participant's statements as a public testimonial. If it did, the sponsor would need to obtain a separate testimonial release permitting public display and distribution of the participant's responses, voice and likeness, and if desired the participant's name and address.

Similarly, if the study sponsor wants detailed testing and feedback concerning a product as used by numerous participants in their homes or offices, the sponsor would use a "beta evaluation agreement" instead.

Participants less than eighteen years old generally do not have the legal right in most states to form a contract. Instead, a parent or legal guardian should enter into the contract on their behalf. Participant agreements and the issues that they cover are governed by state law, which might create different or additional requirements depending on the state.

Harry Boadwee is the founder of the Boadwee Law Office in Cupertino, California. He focuses on technology transactions, software and internet law. This article is for general information purposes only and is not legal or other professional advice.
Copyright © 2011 Boadwee Law Office.

Slides from today's workshop - Interviewing Users: Spinning Data Into Gold

2011-09-29T04:41:53Z

Here are the slides from today's workshop in Seattle.

What you need to survive going into the field...it's not just your research plan.

2011-09-13T21:16:15Z

Shelley-Ann J. West posts her mindmap of things to bring into the field.
You can see the full map here.

His Girl Friday and rapid exchange

2011-08-30T20:39:49Z

I've been working out some thoughts on turn-taking in the interview. Sometimes I'm challenged to figure out my respondent's rhythm so that I can adjust my own asking-pausing rhythm appropriately. I can't help but think of a wonderful example of extremely compressed turn-taking, the fast-talking dialog in Howard Hawks' 1940 His Girl Friday.

In the US, at least, you can watch the entire film here. Imagine trying to interview Burns or Hildy and managing your own pace in response!

Listen to Steve speak with Jared Spool about "Immersive Field Research"

2011-08-25T23:48:24Z

I was recently interviewed by Jared Spool in anticipation of November's User Interface 16 conference (where I'll be leading a full-day workshop). You can listen to the interview here (MP3), and read the transcript here.

There's a great sort of, what I think of as a great myth around that that maybe we can talk about for a sec. Just the oft repeated idea that you can't ask people what they want because they can't tell you so that if you're in the kind of business and design challenge that we're talking about where you want to break through and innovate and reinvent something you shouldn't ask people what they want because they can only talk about what is going on today.

I love hearing that because I feel like I have a good response to that. It's a conflation of a few things. One is, let's just say, looking more largely, doing field research to learn about people and asking people what they want. I think if this is not an area that you're experienced in you think those are the same thing. You think the only thing you can do in field work is to say, "well what do you want?" and then go off and build it.

And most people would say that's not an effective technique for learning new things. I agree with that on the face of it. If you, you know, are trying to change the game in a certain space that's well entrenched you'd better have a more interesting approach to the field than to say, "well what would you want to see different?"

You have to be looking more broadly at people's behaviors and their needs and, you know, what are kind of educated people trying to do and how are people solving problems? What are the entrenched kind of challenges there? And so you need to use techniques to gather that information and make sense of that information.

It's not a ladle that you dip into the soup, right? Scoop. Oh, here's what people said they want. We're going to go off and do it. That's never a way to do breakthrough stuff. So yeah, I agree when people say don't ask what people want because they can't tell you but I don't agree with the implication of that which is don't do research to try to innovate.

Steve leading Immersive Field Research Techniques workshop at UI16

2011-08-19T18:10:47Z

In addition to my upcoming Rosenfeld Media workshop, I'll also be presenting a full-day workshop on Immersive Field Research Techniques at User Interface 16 this November in Boston. Registration gives you

Two full-day workshops: The UI16 experts will dive deep and get to the nitty-gritty details that make any designer into a pro.
One day of short talks: This is where you'll discover the latest UX ideas and techniques from each of our expert speakers. Don't forget Jared Spool's entertaining and educational keynote.
Complete conference materials: We'll send you the PDFs of every session and workshop just before you leave for the conference. Then you can focus on insights and not note-taking.
Recordings of the short talks: The benefits keeping coming after the conference. Through the recordings, you can relive every short talk at your office with your entire team.

I hope to see you there!

Flow in the interview

2011-08-18T19:26:08Z

Earlier this week the San Francisco IxDa hosted a talk by Peter Stahl about The Rhythm of Interaction. As part of his presentation, Peter talked about Mihály Csíkszentmihályi's notion of Flow - "the mental state of operation in which a person in an activity is fully immersed in a feeling of energized focus, full involvement, and success in the process of the activity." Yesterday I came across a podcast I did a few years ago with the folks from Lunar where we talked about how speed, creativity and innovation intertwine in the design process and about getting results through design research. You can listen to the podcast here but meanwhile here's a snippet where I describe entering a flow state when interviewing users.

And all the power of noticing and stepping back and slowing yourself down and just disengaging yourself from the need to be making things happen, is just sort of creating that space and t hat's where insights happen. That's where creativity can happen. And I'm sure you guys have seen that moment when you're in the field, where you have all this responsibility to be managing a session and managing the other people in the session and making sure you stick to your time, and it's a lot of, lot of work. Your brain is just firing on all its cylinders. And then sometimes for me there's that moment where you kind of - it's almost like a hyperspace moment where the starts start to just stretch out. Things just get really, really quiet in my head and suddenly, I'm just riding it. Things are sort of happening and I'm riding it, and that can be - it's, I guess, a flow moment, right? Things can be really insightful at that moment. I don't know that I'm bored, but if I had to contrast that to the stimulation of trying to run everything and run everybody, that seems to be a really kind of creative moment for me when that happens.

Listen to Steve on the User Experience podcast

2011-08-16T17:19:22Z

I was interviewed by Gerry Gaffney for his User Experience podcast. The topic of the interview was, recursively, interviewing. You can listen to the interview here (MP3), and read the transcript here.

Steve: Yeah there's something about interviewing. It is such an individual and it's such a human activity that we can talk best practices, you know, all day. I think there's something really great that happens when people make it their own. I think this is one of those "find your own style" things. I like to be dictatorial about best practices but I also have to acknowledge very strongly that what people bring is very interesting and different. Along those lines think about introverts versus extroverts and what's easier or different for introverts or extroverts in these kinds of situations. Extroverts of course get energy from other people, introverts get energy kind of on their own and so that starts to manifest itself in interesting ways or in silence. But also just how much of yourself do you bring to it? And so I've seen extroverts be very successful at establishing rapport by talking about themselves, by being very open and genuine and giving.

My tactic as an introvert is to remove a lot of myself from it and really focus on them, express my interest in them, ask questions, ask questions, ask questions, ask follow-up questions, really drive everything towards my focus on them. So my long answer there is I think there's a personal style thing that kind of comes out. I think if you reveal things about yourself, regardless of your style, I think it needs to be very deliberate. It's a great tactic to give somebody permission.