Design Staff

How we built a research lab for mobile app testing in just a few hours

2013-04-17T00:00:00-07:00

OK, you’re ready to run a user study. You’ve learned how to recruit participants, write an interview guide, interview people, and summarize results. But there’s just one problem: you don’t have access to a research lab. Don’t worry! Read on to learn how Pocket built a lightweight research lab for mobile app testing in their office. —John Zeratsky

Last month at Pocket we ran three design sprints in a row, with help from Google Ventures. Three sprints means three user studies — we brought in 4–6 real people each Friday for three weeks.

This was our first time conducting user research in the office, so we only had a couple days to put together an A/V setup for user testing. Here’s how we hacked together a professional user research lab for mobile app testing with a few simple tools.

The prototype

We were testing prototypes of an iPhone app we built in Keynote. We copied the prototypes onto an iPad mini and ran them in Keynote for iOS.

The prototypes were designed in portrait mode, but Keynote for iOS only shows presentations in landscape mode. This was a happy accident — it turns out an iPhone-sized prototype fits quite nicely on a landscape-mode iPad mini. It’s a bit less realistic than a handheld iPhone, but there’s an upside: The iPad mini remains stationary on the table, making life easier for the researcher and observers.

The lab

We set up our research “lab” in a quiet conference room with a table and two chairs. (And yes, we gave the participant the slightly taller chair.)

Here’s the equipment we used:

iPad mini for running the prototypes
USB document camera for capturing the iPad screen (we used the Ipevo Ziggi for $94)
USB webcam for capturing the participant’s face — clipped to the top of a desk lamp so it was at the proper height (we used the Microsoft LifeCam HD-6000 for $35)
MacBook Air for displaying video from the document camera and webcam, streaming to the observation room, and recording the session

It was important that observers could see the device screen and the participant’s face, so we connected both cameras to the MacBook Air and opened the video feeds side-by-side. We opened the document camera feed in a simple viewer app included with the Ziggi, and routed the webcam video through Photo Booth (part of Mac OS X).

The observation room

We set up a second conference room as our observation room, where members of the sprint team and the rest of the company could watch the user interviews.

To stream video from the lab, we used AirPlay Mirroring on the MacBook Air to stream the desktop (which included both camera feeds; see above) to an Apple TV, which was connected to a large monitor in the observation room.

On a local network, AirPlay Mirroring provides very high-quality video. If we had observers outside the office, we could run Google Hangouts on the MacBook Air and share our screen to anyone connected.

Just before starting our first interviews, we discovered that AirPlay Mirroring does not include audio. We substituted two standard office speakerphones, calling the lab from the observation room (and muting the observation room so the participant couldn’t hear anyone on the other end).

Here’s the equipment we used:

Apple TV for streaming the MacBook Air’s desktop
Large monitor for showing Apple TV output in the observation room
Two standard office speakerphones

Easy, high-quality, and affordable

We were very happy with this lightweight setup: it was easy to assemble, provided high-quality video and audio, and didn’t cost much. (We already had the computer, monitor, iPad, and telephones in the office.) It was particularly valuable seeing the device screen and participant’s face side-by-side.

We’re always looking for ways to improve convenience or quality, so let us know if you’ve discovered other good tricks for user testing. And if you try this setup in your office, please tell us how it goes!

Jonathan Bruck does a little bit of everything (except write code) at Pocket.

View comments

#UXmarch recap: 1,000 pledges

2013-04-03T00:00:00-07:00

This article was originally published on the Google Ventures Blog.

Last month we launched #UXmarch to help companies build better products. I asked people to pledge to conduct a scrappy UX study in March. The reaction was amazing: 1,002 people took the #UXmarch pledge! That’s about 48 UX studies per weekday in March.

Most of these people work for startups or other small firms. But the list of companies also includes a surprising number of household names, as well as universities, nonprofits, and companies from around the world.

The personal notes I’ve received from pledgers further confirms my belief that user research is a key ingredient to great design:

Aigerim Shorman, Triptrotting — “We pledged to #UXmarch and it has completely transformed our view on how we make product related decisions. Our team conducted multiple weekly interviews for our weekly releases and got incredible insight from our users. We were able to move quickly, iterate fast and make right decisions.”

Jerome Tave, UberConference — “#UXmarch happened at a perfect time, as we were beginning to think about making some changes to UberConference. After attending Michael’s workshop we had all of the tools to go conduct interview sessions, in person and over the phone. The feedback we received was amazing, and the outcomes of this research have put us on the right path.”

Margaret Grobler, Gyft — “I could kick myself for putting off formal usability testing for so long! We’ve watched friends and investors use the app and tested vigilantly for bugs, but the insight (and reward!) you get from running a real user test is unparalleled.”

Ben Hadden, Shotgun Software — “Conducting our first usability study as part of UXMarch felt like shining a giant light on our design process. We now have much more clarity on what’s left to build for this release, which is going to help us ship better, more usable software to our users in less time.”

March is over, but it’s never too late to build better products by adding a little user research to your design process. Use these materials to start reducing your risk, accelerating your progress, and validating your priorities.

Video of my User Research, Quick ‘n’ Dirty workshop
Complete worksheets and slides from the workshop
Design Staff guide to research
Promo code (3forGV) for three free tests on usertesting.com

And remember: Friends don’t let friends launch before testing.

Michael Margolis is a user researcher and a UX Research Partner at Google Ventures.

View comments

The product design sprint: validate (day 5)

2013-03-07T00:00:00-08:00

At the Google Ventures Design Studio, we have a five-day process for taking a product or feature from design through prototyping and testing. We call it a product design sprint. This is the final in a series of seven posts on running your own design sprint.

As day 5 of the product design sprint dawns and the team files into the war room, there’s a certain something in the air. Is it the chemical scent of the whiteboard markers? Maybe. Is it coffee breath? Yeah, almost for sure — in fact, you might want to hand out some gum.

But there’s also something else: anticipation. Today, we test our prototype and learn which ideas worked, which didn’t, and what to do next.

Today you’re going to be running a user study, showing your prototype to 4–6 real humans. By “real humans,” I mean people who don’t work at your company — more specifically, people who you’d like to have as users. Learn how to recruit great participants for your study here.

Wait, what if I don’t know how to run a user study?

Bad news: This post won’t teach you how to talk to users. This post is for the people watching the study, not for the poor soul conducting the interviews.

Good news: If you’re the interviewer, you can get some great advice from our research guide. Be sure to check out Michael Margolis’ articles on interviewing tips and body-language hacks, and his video about running user studies from the Google Ventures Startup Lab.

And you might as well read this post, too, so you know what’s going on in that observation room. I went to all the trouble of typing the whole thing up, so it’s kind of the least you could do.

Before the first session: List your key questions

Start at the end. When the day is over, you’ll want a concrete list of ways to improve your prototype. Talking about your game plan before the study starts will help you get there.

The interviewer and the observers should make a list of the key questions for the day. Here are a few tips:

Review your conflicts and assumptions. Those assumptions you flagged as testable with a user study? Now’s the time!
Are you testing multiple prototypes in a battle royale? If so, be sure the interviewer understands the differences between the two versions, so they can ask the right questions.
Consider showing participants some real products for comparison — they’re like free prototypes!
What else do you want to see through your users’ eyes? Today’s study is not a usability test — you have a chance to find out how your users understand your product, what competitors or substitutes they use, and more. Think beyond the prototype and you are guaranteed to get some unexpected insights.

Set up the observation room

Everybody who participated in the sprint should be in the room. There’s no substitute for watching real humans use your product, and this is a golden opportunity to do it!

I like to reserve two rooms for the day: one for the user interviews, and another where the sprint team can watch live video of the sessions and take notes together. It’s kind of like a Battlestar Galactica marathon, only instead of comparing guesses on who’s a Cylon, you’ll be comparing guesses on which parts of your design are going to go up in flames. It’s actually pretty fun.

Test the A/V ahead of time

Setting up live video for observation shouldn’t be too hard — at the Google Ventures Design Studio, we’ve had success with WebEx, GoToMeeting, Apple Airplay, and Google Hangouts.

Just be sure to test your video before you start the study — ideally the night before and the morning of — because nothing’s worse than putting in all this work and not getting to watch the outcome. OK, there are a few worse things. But you see what I mean.

A few more A/V tips:

A conferencing mic can provide a huge improvement in sound quality.
Don’t forget to mute the audio in the observation room!
Quickly re-test the A/V between each session. Seriously, stuff gets wacky.

Don’t diss the user

If you see people struggle to understand the prototype, keep in mind that we’re testing your design — not the participant. If they don’t get it, it’s not because they’re dumb. It’s because you haven’t nailed the design yet.

Make it clear in the observation room that it’s not OK to diss the participant. It’s tough to wade through a prototype while people are watching. You owe the participants your respect.

Every observer takes notes

Everybody should take notes on things they see during the interviews: good, bad, and other. Insist on paper note-taking — it’s best to keep laptops closed, lest you lose your fellow observers to email.

Designate a court reporter

One person can use a laptop in the observation room, but they have a tough job: the court reporter. They’re responsible for typing a word-for-word transcript of the interview in real time. Assign a different court reporter for each session — it’s cruel and unusual to make the same person do it all day.

It’s kind of a pain, but these notes become an incredibly valuable reference after the study, and a text document is a heck of a lot faster to scan for quotes and reactions than a video recording. Often times, we don’t record the study and rely on these notes instead.

Make a scoreboard

Clear one big whiteboard to collect the group’s notes. Make a column for each participant and a row for each part of the interview (e.g. background, first prototype, second prototype, etc).

At the end of each session, write down the highlights from everybody’s notes — you can double check any questions against the transcript. I like to color code: green for things that went well, red for problems, and black for everything else. It makes it easier to find patterns at the end of the day.

Observing humans: the emotional roller coaster

I’ve never been in mission control when one of those rover things lands on Mars (NASA isn’t returning my calls). But I imagine it’s pretty similar to the atmosphere in the observation room of a user study. There’s tension and excitement and nobody wants to be the one who designed the doohickey that blows up. Here’s what you might expect to feel throughout the day.

First session: “We’re geniuses!” or “We’re idiots!”

The first user may love your prototype. Or they may hate it. Either way, take a deep breath. People are different, and I offer you an ironclad guarantee that not everyone will react in exactly the same way.

Regardless of how that first interview goes, resist the urge to make changes to your prototype. Unless it’s something simple like a typo or broken link, you risk “fixing” something that the next four users would have actually liked.

Sessions 2–4: “Oh, this is complicated…”

It’s not uncommon for the second and third participants to have dramatically different feedback, which means you’re going to feel a little confused. Just sit tight and keep taking notes. It’s still too soon to tell.

Actually, that’s not entirely true. Sometimes you know for sure that part of your prototype is rotten after two or three interviews, and watching more people suffer through it is punishment for all involved. If everyone agrees that something is way off, talk to the interviewer in between sessions and ask them to guide people around that part of the prototype.

Studies 5-6: “There’s a pattern!”

After the final interviews it’s easy to see the big patterns, but it’s worth double checking the notes. Go to the scoreboard and look for things that showed up two or more times. Mark good stuff with a big green dot, and bad stuff with red.

Now make two lists on the whiteboard: “things that work” and “problems to solve.” These are your top-line findings. The CEO or decider for the project should bless that list before you leave the room.

The sprint is complete. Take a deep breath.

How to start your next sprint

The vast majority of these studies ends with mixed results. Some of your solutions work, and some don’t. The outcome usually falls in one of three buckets:

A. Most stuff worked

This is pretty uncommon for the first sprint on a project, but if it happens to you, everyone on the team is probably on the same page about the fixes and tweaks you need to make.

What to do next: Tune your existing prototype and keep going. Try starting your next sprint at step 3 (decide).

B. Some big questions

The most common outcome after a user study is a mixed bag: a few hits, a few tweaks, and a couple of real head-scratchers. Fortunately Keynote prototypes are easy to change, and as long as some parts of your design are solid, you can probably build on what you have.

What to do next: You can move fast on the tweaks, but you’ll want to come up with multiple solutions for the bigger problems. Start your next sprint at step 2 (diverge).

C. Everything exploded

I’ve seen a lot of of my designs go up in flames. It’s OK. You learned that something didn’t work, and it only took you a few hours to build it in Keynote. This is great progress and — relative to building and launching for real — very cheap progress at that. Think what would have happened if you’d spent weeks or months implementing this solution!

What to do next: Start your next sprint back at the drawing board with step 1 (understand). (Hint: the results of this study are perfect fodder for reviewing and building understanding as a group.)

The next sprint will be easier

You may feel tired. Actually, if you don’t feel tired, you probably didn’t do the sprint properly. It’s an intense week. But you’ve built up a tremendous head of steam — even if everything exploded, you now have a much better understanding of the problem and possible solutions.

At the end of a sprint, CEOs often tell us they appreciate that they have a clear list of what to do next. Now that you’ve built up all that knowledge and momentum — not to mention a prototype — you’ll find you can do the next sprint more quickly, or with less effort, as long as you do it right away. Don’t let more than another workday go by before you jump back into it. If this problem is important, you’ve got to bear down and finish it off before people get distracted.

Thanks for reading through this series! We keep experimenting and we keep learning new things about our sprint process at the Google Ventures Design Studio. We’ll continue to share what we learn here at Design Staff.

I’d love to hear about your own experiences with product design sprints, or other processes that work well for you. How do you hack your process to get better results? What’s working? What still keeps you up at night?

Product design sprint series
Previous: Prototype (day 4)

Jake Knapp is a process geek and design partner at Google Ventures.

View comments

#UXmarch: better products for everyone

2013-02-26T00:00:00-08:00

This article was originally published on the Google Ventures Blog.

Blue Bottle Coffee does it. Nest does it. RetailMeNot, 23andMe, and Google do it. It’s UX research, a secret weapon used by successful companies to design better products.

As a UX Research Partner at Google Ventures, I’ve conducted hundreds of customer interviews with startups in our portfolio. Time and again we see that user studies are a key ingredient for great design. Through quick, scrappy user studies we’ve discovered ways to boost conversion, found usability issues before launch, figured out why users were behaving in certain ways, and resolved nagging internal product debates.

User research is like the flossing of product design — we all want to do it, we believe we should do it, but it’s hard to get started and make it a habit. To help with this hurdle, join me in #UXmarch, a commitment to rally your troops and run at least one user study this March. I’ll help guide you through the basic steps to getting it done.

Take the #UXmarch pledge now.

I’ll send you some materials to get started:

Templates and worksheets:
- Checklist for how to prepare in days leading up to interviews
- How to write a survey to quickly select the best people to interview
- Questions that we’ve found to reveal great info from customers
- Instructions for creating an interview guide to structure a one-hour interview
- Non-disclosure and consent agreement for your participants to sign.
An invitation to a limited number of office hours offered throughout March for personal help
A promo code for 3 free tests on UserTesting.com

For the first time I also want to share video of a class I’ve taught repeatedly for the Google Ventures portfolio at our Startup Lab: User Research, Quick ‘n’ Dirty. The two-hour class includes everything you need to know to plan and conduct a study right now.

Take the #UXmarch pledge now.

And remember: Friends don’t let friends launch before testing. Share the #UXmarch pledge with a buddy to help make sure you follow through. Let us know how it goes! We’re here to help.

Michael Margolis is a user researcher and a UX Research Partner at Google Ventures.

View comments

A tip for effective meetings: Always be capturing

2013-02-22T00:00:00-08:00

At HubSpot, we recently did a product design sprint with the Google Ventures Design Studio. It was a great experience, not only to work alongside a team of highly accomplished designers, but to observe their design process and how they proceed through a project.

Our design team learned a lot from Google Ventures. We learned about designing quickly. We learned about keeping laser focus on the goals of a project. We learned about keeping the scope as small as possible (but no smaller). But one of the most powerful things we learned was a simple lesson that applies to way more than design: Always be capturing.

“Always be capturing” is about the habit of continuously recording the value from your conversation. For example: If you’re talking about a new concept, you should be sketching it as you talk so your team has a shared understanding and an artifact of the conversation.

How to capture

Whiteboards

Fill your workspace with whiteboards so you can easily take shared notes. Fixed whiteboards (attached to the wall) are OK but movable whiteboards (on wheels) are better.

Post-it Notes

Write down individual ideas, questions, or UI elements on Post-its. Then you can easily post, cluster, and organize your notes later.

Smartphones

As you work, take photos of notes and drawings with your smartphone. The camera on your iPhone or Android is good enough to capture high-quality images of your work. You can email, print, upload, or manipulate these later.

Dropbox

At the end of the meeting, upload the photos from your smartphone directly to a shared Dropbox folder. It’s fast and easy, and gives everyone on your team immediate access to the output of the meeting.

Some principles to keep in mind

Don’t repeat yourself

When you capture every important idea or concept, you’ll find it easier to avoid repetitive discussions. During the sprint, if we started down a path that seemed familiar, we would look around the room at everything we’d captured to see if we were repeating ourselves. If so, we could say, “You know what, we already captured that,” and move on.

If you can’t capture it, stop talking

If you find yourself unable to capture your conversation, it may not be that valuable. Seriously. During the sprint, Jake (who was facilitating) would often say: “OK, you have been talking for a few minutes now without capturing anything. I want to help you do that.” Either the people talking would start capturing, or they’d stop talking because they realized they weren’t really pushing anything forward. This was a really valuable lesson!

Write down or sketch everything important

This is easier than it sounds. For example:

If you’re comparing two things, just make a two-column table and write out the differences.
If you’re talking about a bunch of features, write them down on Post-its and sort them on the wall.
If you’re brainstorming a concept, sketch it out. This immediately shows you whether everyone is thinking of the same thing. You might find that other people jump in to help complete the sketch.

Appoint a facilitator

Jake Knapp facilitated our design sprint. In this role he served as an objective manager of the discussion. He made sure that everything got captured and that everyone stayed on point.

By the end of the sprint we generated an enormous collection of shared artifacts that we could all access, and that we had all seen before. Reviewing that collection afterward shows us how much work we actually did — and it’s an invaluable foundation for future design work that the sprint kick-started.

Now we’re putting the practice of “always be capturing” to work at HubSpot, and our design sessions have gotten twice as efficient. We quickly move from project to project secure with the knowledge that everything we’ve discussed has been captured somewhere. Just knowing that we have a record of all the design work we’ve done makes us more confident, effective designers. And best of all, we’ve shared this technique outside the design team — it easily applies to all meetings, not just design-related ones!

Joshua Porter leads the UX team at HubSpot and writes about product design on his blog, Bokardo.com.

View comments

How to hack your body language for better interviews

2013-01-24T00:00:00-08:00

Research interviews go best when participants feel comfortable and confident — they’re more verbal, more willing to explore, and more willing to play along. So when an interview isn’t going well, I check for signs of low status in the person I’m interviewing and adjust my body language to make them feel more in charge.

(These techniques come from improv theater classes I took many years ago in San Francisco. In addition to learning about “yes and,” listening, and teamwork, I was introduced to the important concept of status — and how body language communicates high and low status.)

How to spot signs of low status

If you’re interviewing someone (for research purposes or otherwise) and see these symptoms, the other person is probably feeling low status and not giving you their best possible participation. I try to notice when they are:

Avoiding eye contact
Angling their body away from me
Holding something (computer, chair, backpack) between us
Fidgeting
Touching their face or hair
Crossing their arms or legs in a defensive or protective posture
Asking permission or waiting for an invitation before using the mouse or trying something
Speaking softly
Laughing nervously
Apologizing
Sitting or standing in a protective “fig leaf” pose

Individually, these behaviors don’t necessarily indicate discomfort or low status (people cross their arms when they’re cold, too), but when I notice several of them I make an effort to boost the participant’s status.

Some techniques to boost status and confidence

There are plenty of ways to improve research interviews (read 16 tips here) but these are some of my time-tested techniques for helping participants feel higher status, more comfortable, and more confident during interviews.

Lower your own status

If you come across as a confident expert, some people are less willing to offer up their opinions. These tricks can help you seem lower status, and help participants feel comfortable and confident responding.

Smile!
Look interested and attentive. Lean in. Make sure your arms and legs aren’t crossed.
Ask permission before moving to the next task or taking control of the mouse.
Sit lower: If you’re sitting on an adjustable office chair, drop it down until you’re sitting slightly lower than the other person.
Back up: Respect the participant’s space. If the participant is seated, don’t stand right beside or behind her.
Move back slightly to avoid invading personal space.
Avoid intense, dominant eye contact.
Dress similarly to the people you’re interviewing.

Lower the status of the product or design you’re testing

People are more willing to criticize something that’s “just a prototype.”

“Since this is a prototype [even if it’s not], some of it isn’t working quite right yet. It still needs a lot of work.”
“These are just some ideas we’re playing with. Looks like they still need a lot of work.”

Reassure and encourage

But try to keep your comments neutral to avoid “leading the witness.” It’s also a good way to show that you’re on the the same team as the participant — not the designs you’re testing.

“You’re doing great. This kind of feedback is really helpful.”
“It’s really helpful to watch you try to figure this out.”
“It’s so valuable to see this through fresh eyes!”
“Since I didn’t design this, there’s no risk of hurting my feelings or flattering me. Your frank feedback is really valuable to me.”
“You’re discovering a lot of important problems we missed.”

Be grateful

Make participants feel like they’re doing you a huge favor instead of trying to perform as your “test subject.”

“I really appreciate your helping me with this today.”
“Thanks so much for helping me figure out how to improve this.”

Monitor for results

As you use these techniques to boost participants’ status and build rapport, you should see their body language “melt” and the conversation improve. Try a couple tricks at a time — you don’t want to suddenly change your behavior in a way that seems awkward or artificial. And remember to continue monitoring their behavior for signs of low status throughout the interview.

Examples

Avoid hovering, invading personal space, and taking control of the computer.

Be sure to sit lower, avoid high-status body language, and show the participant you’re curious and interested in what she is saying.

Be sure to smile and build rapport with the person you’re interviewing.

Examples of high-status (left) and low-status (right) body language.

Thank you to Aideen Stronge and Lane Kuhlman.

As social animals, we’re sensitive to a wide range of verbal and nonverbal cues — often unconsciously. A raised voice, a raised eyebrow, or a raised finger can say a lot. With a heightened awareness of basic status cues, you can monitor and use body language to improve the quality of your interviews and the research data you collect.

Michael Margolis is a user researcher and a UX Research Partner at Google Ventures.

View comments

The product design sprint: prototype (day 4)

2013-01-07T00:00:00-08:00

At the Google Ventures Design Studio, we have a five-day process for taking a product or feature from design through prototyping and testing. We call it a product design sprint. This is the sixth in a series of seven posts on running your own design sprint.

On day 2 you drew concept sketches. On day 3, you made a plan and a storyboard for your prototype. Now it’s day 4 and the clock is ticking. You’re going to create a real-looking version of yesterday’s storyboard and show it to users tomorrow.

This part of the sprint is super exciting for me as a designer. Thanks to the storyboard, I know exactly what to do, and I also have a crazy deadline to get it done. It’s finally time to to open my laptop, put on my headphones, and start moving pixels.

But wait a second… what should this prototype look like? Are we going to have to code anything?

What your prototype should look like

Quite simply, a prototype is anything a person can look at and respond to. A prototype doesn’t usually have to be very complex in order to learn what you need to know.

Make it minimally real
You’ll probably be amazed at how much real feedback a user can give you on a slide deck of mockups that aren’t even pixel-perfect.

They can tell you what they understand about your product — and what they don’t. They’ll tell you what they expect things to do (“I’d click here because I’d want to see a list of your customers…”), and when they get confused.

You’ll also learn things that metrics alone can’t tell you, in particular why users do the things they do, rather than just what they do. And you’ll learn much faster than if you had waited to build something.

Write real text
While it can be tempting to use “lorem ipsum” when you’re building a quick prototype, don’t do it — always write real text for your prototype.

Why? First, user interfaces are mostly text. Punting on the text might save you time, but it won’t get you closer to solving your problem. Plus, in design sprints we’re often figuring out how to explain things and testing whether users understand. Lorem ipsum skips all of that. When you use dummy text, you avoid tough decisions and limit how much you can learn. (37signals has a great essay on using real words in their book Getting Real.)

Keynote versus code
Occasionally you’ll need to write some code for your prototype. This can be really valuable when you’re testing assumptions relating to data quality or personal content like email. But nine times out of ten we find that Keynote is sufficient. More on this later.

Here are some examples
We have created dozens of minimally real prototypes for the startups in our portfolio. Here are a few examples, covering the spectrum from grayscale and low-fidelity to nearly pixel-perfect.

Keynote is the world’s best prototyping tool

If you’re building a product for desktop, mobile, or iPad, the fastest and easiest prototyping tool you can use is Keynote. I’ve spent my career working with fancy tools like Photoshop and Illustrator and Fireworks. (OK, Fireworks might not be fancy, but it’s almost certainly a tool.)

Here’s why Keynote wins for prototyping:

It’s fast.
It’s easy to make things look pretty good…
But it’s impossible to make things look perfect, so you don’t get too precious.
Anybody can quickly learn to use it; not just designers.
The slideshow format is a natural fit for story-based design.
It only costs $20.
The animated transitions are a lightning-fast way to make your prototype look way more real than it really is.
If it’s a mobile prototype, you can export a PDF and open it on your device with reasonable results.
When you’re done, you not only have a prototype, you have a presentation, like, for free!
After your user study, when you learn all of the problems in your design, it’ll take minutes to make changes instead of hours or days.

(“Wait,” you say, “I use a PC.” Wow, you just blew my mind. Almost everybody I meet who works at a startup has a Mac, but true, there are exceptions. If I stereotyped you, I am deeply sorry. Let me make it up to you by pointing you to Henry Tsai’s excellent post on building PowerPoint prototypes.)

The final Keynote (or PowerPoint) prototype might be linear or it might have a few links that allow the user to click around. It might be a grayscale wireframe or it might be somewhat polished. Either way, these make for good prototypes. They’re going to be realistic enough that your user study participants will forget they aren’t clicking on a real product.

Keynotopia templates make it even faster
Here’s our favorite secret prototyping weapon: Keynotopia. It’s a template with buttons, menus, and all kinds of plug-and-play elements that you can just drop into your prototype without any design skills at all. This makes it easier for CEOs and marketing directors to dive in and make something to communicate what they’re thinking. And it’s available for PowerPoint too, if that’s how you roll.

All right already, I bought Keynote, now what?
You made a wise choice, my friend, and the prototypes are going to start flowing like milk and honey.

What to do in day 4

Here’s a rough schedule for your day of prototyping. Expect to spend about an hour in the morning to review and make your plan.

Divide and conquer
What if you have too much to design and not enough time? Work together, baby.

Take a look at your team. How many people can help in Keynote? Chances are good that almost everyone can — in our sprints, we frequently have engineers, PMs, and CEOs cranking out good work in Keynote. Remember that everybody’s work doesn’t have to be genius level. A designer can always clean it up afterwards, but it’s usually faster to clean up a slide after someone else has put the building blocks in place.

Now take a look at yesterday’s storyboard — or, if you’re doing a “battle royale,” storyboards plural. Break it into chunks and assign them out. It’s probably obvious, but it’s helpful to think about where different people have expertise, or where it’s best to spend your designer’s time, or who moves the fastest.

You probably want to divvy up the best of the storyboard sketches on paper, too. Those are often the blueprint for many of the mockups you’ll have to make, and they can save you a bunch of time.

Assign one person to be the stitcher. The stitcher’s job is to take everybody else’s work and put it into one cohesive flow. If you only have one designer, or if one person is really fast at Keynote, she might be a good candidate. If you’re doing a “battle royale,” you may decide to have one stitcher for each competing version.

Breaking up the design work and stitching it back together probably doesn’t sound like the way great design is done. And it’s not. Realistically you’ll get better results if you’ve focused enough that your designers can do all the work here. But if you need it, dividing and conquering is an incredibly fast way to build a good-enough-to-learn-from prototype.

Build an asset library
Another way to increase efficiency is to take a few moments at the beginning of the day to build a template slide deck. Include anything that everyone will need — screenshots, user avatars, logos, formatted text; whatever you think might help. And don’t forget to include a browser bar at the top for realism. You don’t want to go through 99 slides at the end, scooting each one down to make room.

Use a timer to maintain focus
Once again, the Time Timer is your friend — although if you don’t have one, I guess some other kind of clock might work. We’ll often say something like: “Let’s work heads-down for the next hour and then we’ll take a break.” Timeboxing can make the project feel less daunting, and reduce the impulse to check your email and burn up valuable make time.

Appoint an email sheriff
It’s sometimes helpful to appoint someone — usually the facilitator — as the “email sheriff.” This person’s job is to publicly shame anyone who is checking their email (or surfing the web or whatever time-wasting tactic you prefer). It’s usually enough to just make a big deal about saying that someone is going to do this… the threat alone practically makes the job unnecessary.

Lightning critique
If you have a bunch of people working in parallel, it might be useful to do a quick critique midday. It’s useful to ensure consistency and get outside eyes on your design. But critiques can eat up a lot of time if you aren’t careful. Just like on day 2, you’re going to want to limit the time spent talking about each design. I recommend 5 minutes.

You’re also going to want to prevent people from trying to redesign somebody else’s work — they might have a great idea, but there just isn’t time in this part of the sprint. Instead, use the lightning critique as an opportunity to raise questions and criticisms, but make it clear that the person who designed those components is going to be responsible for figuring out the solution, not the group.

Review with an outsider
Schedule 30 minutes with someone who is not doing design work today. It might be your user researcher if you’re lucky enough to have one, or it might be an executive — seems like everybody’s got one or two of those hanging around. The outside eyes will help prevent you from going too far down any groupthink rabbit holes. Just make sure to do this early enough in the day that you have plenty of time to respond to the feedback afterward!

Pointers, text, and other final touches
If you’re creating a linear prototype with no links, which is the very fastest thing to do, it’s a good idea to draw a mouse pointer and add extra slides showing where the user would click, and extra slides when text is entered. It takes only a little extra time but makes the user study much more realistic.

Other crucial details:

Check for consistency and typos, especially if you’ve got some made up user data. Don’t let it start out as Sally and end up as Suzy. That stuff is distracting in a user study.
Try to make any content current and relevant. If you’re running the study in Seattle, and you need to show a newspaper, show the Seattle Times, not the Milwaukee Journal Sentinel.
If you get stuck, remember what you’re trying to learn — don’t waste 30 minutes tweaking a button style if you’re doing a study about whether people understand the value proposition.

When Keynote isn’t enough

Nine times out of ten, you can learn everything you need to know in a user study with a click-through of mockups.

But sometimes you can’t avoid it. When you’re testing big assumptions involving data quality, or when users need to interact with their own stuff (email, docs, contacts, etc) to really understand the product, there’s no substitute for a code prototype. In this case, get someone with engineering chops to help.

Just make sure everybody, especially the engineer, knows that this code will be throwaway. That’s really important: The. Code. Is. THROWAWAY. We gotta move fast, so don’t get attached. Like the Keynote prototype, it just has to look sort of real, it doesn’t have to be anywhere near perfect.

It’s OK if you’re not satisfied, Michelangelo

It’s better to be done with something good enough than to be half-finished with a masterpiece. Remember that the goal is to learn from the user study tomorrow, not to have everything perfectly figured out and finished.

Stay tuned for the next — and final! — installment of the series, Day 5: Validate.

And in the meantime, what tricks do you use for super rapid prototyping? Do you have a tool that you swear is better than Keynote? I’d love to argue with you about it (respectfully, of course) in the comments!

Product design sprint series
Previous: Decide (day 3)
Next: Validate (day 5)

Jake Knapp is a process geek and design partner at Google Ventures.

View comments

Free prototypes

2012-11-29T00:00:00-08:00

By now we all know that successful startups focus on learning about their users — figuring out what features, branding, and messaging resonate and perform best. Smart teams invest valuable time designing, building, testing, and iterating their own designs and prototypes based on learning from analytics and user studies.

But don’t forget to test your competitors’ products too! Think of them as free, fully functional prototypes. By testing the “prototypes” your competitors have already built, you can learn a ton about users’ reactions to different designs, business models, and messaging. Free!

A few tips:

Compare and contrast
Whenever possible, let users test more than one design at a time. You can show people your prototype and a competitor’s product, or multiple competitors together. Feedback from users is much richer when they have different examples to compare and weigh against each other.

Focus on “why”
The goal is to learn which parts of each product are more and less successful — not to pick a winner. It’s much more valuable to uncover why users like or dislike aspects of the prototypes you’re testing.

Call them all “prototypes”
Even when I’m testing a competitor’s launched product, I tell test participants it’s a prototype. When people think a product is still in the workshop, they’re more likely to think about it as something that can be changed and improved.

Learn about branding (or not)
You can learn a lot by seeing how users perceive the branding of different products. However, if you want to even the playing field, you can obscure competitors’ brands or replace them with your own. Just take screenshots of the flows you want to test, edit them, and assemble them in order in a slide deck.

Learn how you can differentiate
Don’t worry — testing competitors’ products doesn’t lead to imitation. In our experience, it usually helps companies crystallize what makes them different from their competitors.

Any time a startup can find low-cost, fast ways to learn about their users and market, it’s going to help them make better decisions more quickly. So next time you’re considering a prototype to test your assumptions, don’t forget to test the free prototypes from your competitors. And please, let us know how it goes.

Michael Margolis is a user researcher and a UX Research Partner at Google Ventures.

View comments

User testing in the wild: research at conferences and other events

2012-11-27T00:00:00-08:00

Sometimes the hardest thing about user testing is simply making time for the logistics. At Twilio, we’ve found that events we’re already going to — like conferences and industry meetups — can be a great way to get easy access to a big pool of potential users. User testing at events will not give you perfect metrics, ideal user profiles, or a deep understanding of interaction details. But it’s a great technique for getting lots of broad feedback to guide detailed research you’ll want to do next.

Here are some tips to help you conduct great interviews in busy and unpredictable event settings.

1. Target and schedule

Begin by defining what you want to learn, your ideal users, and the kinds of metrics you will gather. And remember: Conferences are highly photographed, so don’t test anything that can’t show up on Twitter.

Identify events that attract the kinds of users you want to learn about (your sales and marketing teams can probably help). Schedule 30-minute sessions that align with session times, workshop hours, talks, panel discussions, lunches, and happy hours. Avoid keynotes and product announcement times — people will be too focused on the actual conference to pay attention.

2. Seek permission

Ask the conference organizers for permission to run user testing sessions during their event. Be precise about how, when, and why you want to do it at their conference — even if your own company is putting on the event. Ask where you can set up for your user interviews. If you get pushback, find the root of the hesitation and politely assure the organizers that your user testing sessions will not detract from the conference.

3. Prepare your setup

Here’s a checklist of things you’ll need to prepare in order to conduct an interview at an event:

Pens and paper for taking notes
Consent forms (for recorded sessions that will be shared)
A space where you’ll do interviews. Will it be quiet enough? Will people need to sit down? Will you be interrupted by other people who want to talk to your participants? Make sure you’ve got a plan that covers all of your contingencies.
A script for the questions you want to ask. Check beforehand to make sure you can easily read it while talking to your tester. Especially when at a busy event and short session, make sure to have your participants think aloud.
Reliable access to power for your laptop.
A wireless card for Internet access. If you cannot bring a wireless card, have a local build of whatever you want to test as a backup. At conferences, a good rule of thumb is: The Internet will go down.

4. Recruit

Getting potential testers to commit to an interview before the event can be difficult — they often don’t check the agenda or plan their own schedule until they arrive — so you’ll need to do most of your recruiting on the fly. When recruiting at the event, you’ll need to be a bit of an extrovert and talk to conference attendees throughout the day. Think of these conversations like a pre-interview that will help you figure out if they are the kind of customer you want to interview.

When you find someone to participate in your research, ask them to fill out a small form that asks for:

Name
Mobile number
Email address
What sessions they do not want to miss
Screening questions to determine if they’re a good fit for the study

Open a Google form on your phone, fill out as many fields for the participant as possible and then ask them to list which sessions they do not want to miss. At Twiliocon, we recruited users in the morning during breakfast and lunch and ran our research sessions in the afternoon.

If your company is hosting the conference, it gets a lot easier to recruit. You can use marketing channels like email, Twitter, and Facebook to call for interested participants. I link to the same Google form I use at the event in the call-to-action button.

5. Send reminders

Once you’ve recruited users, follow up with an email about what time they’re scheduled and specifically where to meet you (conferences can be hard to navigate). Include your mobile phone number. (I give out a Twilio phone number that forwards to my cell phone to keep my personal number private.) I remind people to call, email, or text me if they cannot make the interview. Finally, include a calendar invitation for their exact session time.

6. Ask for critical feedback

To get critical feedback, we let participants know we’re testing parts of Twilio that people often struggle with or that we have found not to work well. We also emphasize our dedication to improving our products, and tell them the more they find wrong, broken, or confusing, the better.

7. Say thank you

You and your participants will be swimming in new contacts after the event, so follow up with a “thank you” afterward. We send along a gift card as an extra thank you — it’s a memorable way to let users know that their time and feedback were genuinely valuable.

Hopefully these tips will help you take advantage of events (like conferences and meetups) to easily recruit and interview potential users of your product. Have you done user research at events? Please share your experiences in the comments!

Nina Mehta is a designer at Twilio and a music-visualization artist. She was formerly a designer at RockMelt, Gengo, and several newspapers. She has a master's in Human-Computer Interaction.

View comments

The product design sprint: decide (day 3)

2012-11-20T00:00:00-08:00

At the Google Ventures Design Studio, we have a five-day process for taking a product or feature from design through prototyping and testing. We call it a product design sprint. This is the fifth in a series of seven posts on running your own design sprint.

At this point in a design sprint, you’ve got a lot of ideas down on paper. You’ve explored the problem, generated a ton of solutions, and looked around at how other companies are solving similar problems.

It’s awesome to have a lot of ideas. It’s a great feeling. But I’ve got bad news: you can’t build and test everything. And even if you could, it wouldn’t be very useful, because you’d have too much information to sift through. So you’ve got tough decisions to make: which solutions will you pursue and which will you put on ice?

Today we’ll look at how to decide which solutions to flesh out, and how you’ll fit them together into something you can rapidly test with users to learn what’s working and what isn’t.

Combat the group effect

The decision-making process is hard, and this is one place where working as a group can become a liability. Companies and teams have a natural way that they make decisions — but in a sprint, the group effect can cause decision-makers to behave more democratically than they do in real life. Once the sprint is over and that rosy democratic feeling wears off, you can be left with something that doesn’t have true support from the deciders.

To combat this effect, the facilitator often has to draw out the decision-maker to give their honest, true opinion. You’d be surprised at how often this reluctant decision-maker is the CEO. No way, really? Yes way. In the sprint, people are out of their comfort zones, and even CEOs can begin to behave in non-standard ways.

One method is giving “super votes” to the deciders during design critiques, which you did in day 2. But most of the time, there are no special techniques. You just have to be blunt.

As facilitator, you should be upfront with the team if you sense you’ve got a case of groupthink. Let everyone know that you need more assertive participation from the deciders. Additionally, you should have the words “make the call, Sally” on the tip of your tongue throughout day 3. (I’m assuming your CEO’s name is Sally.) Don’t worry about being a sycophant. If you aren’t conscientious about bringing the decider in now, you’ll have a problem later.

Search for conflicts

The first thing I like to do in this phase of a sprint is comb through storyboards from the previous day looking for conflicts. A conflict is a place where there are two or more different approaches to solving the same problem. Conflicting approaches are super helpful because they illuminate the choices for your product.

For example, let’s say you’re designing your home page, where you explain your product to potential customers. Maybe one person’s storyboard uses a video, and another uses diagrams on a long scrolling page, and a third uses a single image of the product. Great, you’ve found a conflict! Every time you find one, write it down. I like to put the topic and solutions on sticky notes, like this:

Each conflict is like a little gold mine. In business-as-usual design, designers often end up picking one approach and going straight to high resolution. When I was working on products in-house, I’d often get so caught up in that one solution that I wouldn’t even have time to think about how else it could be done.

So one of the best things about the product design sprint is that it allows you to map out those decision points, and perhaps even to explore a few conflicting ideas in parallel instead of immediately committing to a safe choice.

Best shot or battle royale?

You have two basic options for what kind of user study you’re going to run at the end of your sprint. You can prototype several different approaches and test them against each other (the “battle royale”) or you can go with a single prototype (the “best shot”).

The advantage of the “best shot” approach is that you can put a lot more work into that one prototype, or just get it done faster. If you’re testing only one solution, the user study is less complex, and it gives you more time to see what the users say about your competitors’ products (or just interview users, which is always surprisingly valuable for teams).

The “battle royale” works well for newer spaces where there really aren’t many conventions, and you need to figure which one is going to work best for the user. The disadvantage is that it takes more time, and your testers may run out of patience before you get all of the information you’d like to have from them. You may have to bring in more participants and run more studies.

On the upside, the results of a “battle royale” can be very surprising — when working with startups, I’ve often seen a dark-horse design turn out to be the strongest in user studies. When that happens, we thank our lucky stars we didn’t “best shot” it, or we never would have known.

You may also do some kind of hybrid. Occasionally, if you choose the “best shot” approach, you’ll get into testing and find that something’s really not working in your prototype, and you need to go back and have a “battle royale” over that specific feature.

So how do you know which to pick? Start with a gut check: if everyone is excited about one option, you may be ready for a “best shot”. But if it feels more like you’re sitting there and scratching your heads about what to do next — or else you want to throttle each other because those fools just won’t agree with you — well, you may need a “battle royale.”

Test your assumptions

What else should you test in your user study? Listing out your underlying assumptions is a good way to revisit the big picture, especially when you’ve been heads down in a sprint for a few days.

Some of those assumptions might be about the users (example: “users are willing to upload a profile photo”), some about the business (“the designer-with-glasses-and-beard market is large enough to support our product”), some about technology (“we can automatically cluster profile photos by beard shape”), and maybe even some about messaging (“people will still find beard jokes to be amusing, even for the third time in a single paragraph”).

I can tell you that last assumption is false right now, but for most others, you’re going to need some kind of research. Guess what? You can test a bunch of them by showing a prototype to users.

For example, if you have a big assumption that users will be comfortable sharing private data in your product, you may want to pick the most aggressive sharing defaults you can think of to prototype. When you show it to users, you’ll find out pretty quickly whether your assumption was correct.

Try to come up with a way to test all your assumptions, either in the user study or in some other parallel task that can start right away (e.g. ask the engineers to spend a few hours hacking at that beard-clustering algorithm). If you can’t test every assumption now, keep a list for next time. Untested assumptions are like takeout containers in your fridge: if you leave them for very long, things get nasty.

OK, you’ve picked which conflicts to explore and you’ve decided which assumptions to test. Congratulations — you’re ready to script your prototype.

Whiteboard the user story

Now we’re going to make a storyboard that shows exactly how the user will step through your prototype, click by click. This storyboard will become the spec for building the prototype. This is an activity that the group does together — it’s actually the last group step before you break for prototyping.

Start by drawing a big grid on the whiteboard — each cell should be about as large as two sheets of copy paper, and for most sprints, you’ll cover one or two whole whiteboards with your grid. The idea is to draw a comic book that tells a story starting when the user opens the prototype and ending when they complete all necessary tasks.

In each comic book frame, you’ll draw a single action — whether it’s a pointer clicking on a button, text being entered, or a stick figure user doing something in real life. You don’t have to worry about layout or design in great detail, but you do have to think through every action that takes place in the story.

Drawing the storyboard is hard work, and you’ll want to facilitate carefully. Get one person to draw, but don’t make them figure everything out on their own. The group should be engaged and discussing what happens next and giving that brave soul holding the whiteboard marker as much help as possible.

When you begin drawing, imagine you’re framing the prototype for your user study participant. How will they get to your product? What will they be trying to do when they get there? That’ll help you figure out whether the first frame of your comic book is an email or a Google search or an advertisement or the App Store or whatever — and hopefully the story will flow easily from there, following the outline you laid out in day 1.

Keep the gloves off

As you storyboard, there will be lots of small decisions to make that didn’t come up earlier in the day. That’s expected, since you’re working at a finer level of detail now. The facilitator has to work hard here to not let people be too nice. You don’t want design by committee. If there’s a good argument going, don’t try to find middle ground or make people agree. Help the team place a bet on one of the opposing solutions and keep the other in your back pocket if it fails. Call on the CEO to make a tough call when needed. If both solutions are viable, you may want to opt for a “battle royale” — just don’t use it as an excuse to avoid decisions.

When you’re finished with the user story, take a moment to pat yourselves on the back and eat some chocolate because it’s probably been a pretty epic task. You’ve given form to everything you want your user study participant to experience, and you’re ready to turn that story into a higher-resolution mockup.

In the next post, we’ll move on to prototyping. It’s time for the Fellowship of the Sprint to break up, at least temporarily, as everybody puts on their headphones and cranks out a crafty imitation of a real product. Stay tuned.

Product design sprint series
Previous: Diverge (day 2)
Next: Prototype (day 4)

Jake Knapp is a process geek and design partner at Google Ventures.

View comments

How startups can learn more while building less

2012-11-20T00:00:00-08:00

This article was originally published in GigaOm.

If you’re running a lean startup, “launch and learn” is undoubtedly a familiar mantra. But launching a new feature can take weeks or even months, and for a scrappy startup that’s a potentially make-or-break issue. Our design studio works with dozens of startups each year to help teams define their products and features. Through the process of doing this over and over again, we’ve collected a time-tested toolkit of methods for learning that are cheap, fast, and perfect for startups to find those crucial mistakes earlier and then adapt their plans more nimbly. The result is almost always that they ship better products and do so even faster.

Clickable mockups

Most teams think they need to build an interface that functions and looks real before showing it to customers to get feedback. Nope. It turns out that if you string together a few simple mockups with clickable hot-spots, you can still get great feedback in a fraction of the time. We’ve done this with companies like HomeAway, AVOS, and Duo Security by designing a few screens in a flow and then building a clickable version, using basic consumer software tools like InVision or Apple’s Keynote.

At first I thought these prototypes would be too rough to be useful. But time after time I’ve seen customers engage with click-throughs like they’re real products, and that helps you learn if the designs are working. It’s a great method to use before engineering starts to build a design.

Customer interviews

Instead of working in a vacuum, gather data to use as fuel for designing your product. Specifically, go out and find the people who you think will use your product and talk with them about the problem(s) you’re aiming to solve. I know you’ve heard this a hundred times. Customer interviews are like flossing — everyone agrees it’s good for you, but it’s hard to build the habit.

It’s easy to get hung up on the details: How do you find people who will talk with you? What do you talk about? Relax. User researchers have been doing this stuff for decades, and there’s a wealth of knowledge about how to do it quickly and accurately. For starters, you can write a short survey called a screener to help you recruit the right people to talk to. Then, create an interview script to help guide the conversation.

If you want to know more, we created a research guide with plenty of tactical tips for finding and interviewing customers. Now you have no excuse. Get out of the building! (Then come back — there’s more good stuff below.)

Fake doors

You can quickly see whether customers will engage with a new feature by launching just the first part of it. We did this with CustomMade, a startup that lets people order custom-built products. Our idea was to let visitors save others’ projects for inspiration. But instead of laboriously building the whole feature, we just launched the first button. When we observed a huge number of visitors clicking the button to access that function, we new were onto something and built the rest of the feature. After a few changes like that, we saw a 3x increase in engagement. For more on fake doors, see Jess Lee’s excellent talk.

Recon

When teams design a new product, they come to the table with all sorts of assumptions about the competition. It’s easy to look at another product and have an opinion about which parts are valuable and which parts are broken. But if you guess wrong, you might just copy a bunch of functionality that your customers don’t actually need.

So we like to think of competing products as free prototypes. We watch customers use these products and learn very quickly which features are loved, unusable, ignored, or hated. With this knowledge, we can make better decisions in product design, marketing, and sales.

Micro-surveys

Surveys are a tempting way to learn from the comfort and safety of your office chair. But designing a good survey is surprisingly tough. Whenever I talk with survey scientists, I’m overwhelmed by all the ways you can screw up a survey design and unknowingly get bad (read: useless) data. So when we run surveys, we stick to a pattern we know works well.

We put the survey as close as possible to the behavior we’re trying to study. For instance, if we’re interested in why a customer picked one of our pricing plans, we’ll ask them with a small pop-up survey in the moment, not an email that might get read days later.

And we rely on open-response questions that let us hear directly from customers. You’ll learn more from reading 100 short responses than knowing that 32 percent of users chose option B in your survey. Here’s more on micro-surveys.

Prototype with real data

Clickable mockups are a good first step, but you can learn even more when you build a prototype that integrates real data. You might be tempted to start building the actual product at this point. You might even call that work-in-progress a prototype. But it’s not. Building a real product always takes longer than you think. If you really want to learn fast, build a true prototype – one that you’re not afraid to throw away.

When we were designing coupon pages with RetailMeNot, we needed real coupon data in order to evaluate our designs. So we built a prototype in two days. It was buggy and didn’t have many features, but it was just enough to get useful feedback from customers. And it was good we did, because it turned out that half our ideas weren’t working.

We iterated three more times, building prototypes and showing to customers, and were able to get to a design that improved both usability and click-through rates. Few startups build true prototypes, but it’s an immensely useful way to learn fast.

Site visits

Go to wherever your customers are, and watch them actually use your product. I know that sounds like common sense (or it should). But it’s too easy to think we know our customers from all the meetings, phone calls, and reports we’ve read about them. To deeply understand how people actually use our products we need to go to where they work, where they play, and where they live.

Recently, we were working with Foundation Medicine to improve their clinical cancer genomics reports. So we decided to visit oncology centers, watch how doctors used the reports, and see what we could learn. We were surprised to discover that the reports we’d worked so hard to design were often received by fax. Tiny text was hard to read and all color information was lost. It was an easy problem to fix, but we only noticed it through a site visit.

Learn more, build less

Being a lean startup means that we should first consider all these ways to learn, and then pick the fastest, cheapest method. I’ve listed seven methods that we’ve found work well at startups, but there are plenty more out there. Once you start looking, you’ll be surprised at the variety of ways you can learn incredibly fast, saving you and your team precious time and money (and heartache).

Braden Kowitz is a product designer, storyteller, prototyper, and partner at Google Ventures.

View comments

Seven tips for lean market research

2012-11-13T00:00:00-08:00

Want fast answers to big questions about your market, competitive products, or customers’ behaviors and attitudes? First look for existing research.

When companies start exploring new features, new products, or new markets, they often need to answer big research questions. For example: How has Internet radio changed the way people listen to FM radio? When and why do consumers rent cars vs. use other transportation options? What are people’s use and attitudes towards online security? Or cloud services? Or online coupons?

As a user researcher, I’m often asked to answer these kind of tough questions, which can require time-consuming and expensive market research. Since I’m naturally lazy and startups are impatient, I first look for research that someone else has already published (and often conducted with more academic rigor than I would have time to do myself). Depending on the topic and my luck, I can usually track down some helpful articles in less than 30 minutes, and a pile of them in 1–2 hours. Here’s how:

Start with Google

Sure, it’s obvious, but Google can give you a great lay of the land. Try both a regular Google search and Google Scholar. Don’t hesitate to just type in the question you’re wondering about. At first, don’t be too picky. Glance briefly at any results and articles that seem even slightly relevant. As I review search results, I look for different terms, phrases, ideas, jargon, authors, and names of research companies I can try as keywords for additional searches.

As I find interesting articles and quotes and charts, I copy them into a Google document (along with titles and links to the articles). I don’t worry too much about organizing it at first. It’s easier to do that later after I’ve seen and learned more.

Skim abstracts, conclusions, and bibliographies

Don’t avoid academic articles — but don’t waste your time reading them, either. If the abstract or intro look interesting, then scan the conclusions or discussion section at the end. The goal is to quickly triage articles to find the ones that are most relevant and useful. If the article is interesting, Google the author’s name or visit her site to see if she’s written anything else useful. And don’t forget to scan the bibliographies of good articles for interesting references.

Search research centers and organizations

Two of my favorite sources are Pew Internet and ACM Digital Library. Even if I find an interesting article locked behind a registration wall, Googling the exact title will often reveal the full text elsewhere, such as on the author’s personal site.

Search for syndicated research

Depending on the topic, I try searching sites like Forrester, Gartner, and Jupiter, as well as any domain-specific research companies I’ve uncovered in my Google searches. If you don’t want to pay for the companies’ expensive reports, search the Web to find newspaper or journal articles that summarize key parts of the reports.

Take advantage of less-obvious sources

Look for product reviews and comparisons in industry and consumer publications (including Consumer Reports). Reviewers often do a good job of summarizing key differences between products and highlighting features that are important to customers.
Sites for industry and professional associations often include links to research, white papers, and various publications. And associations exist for just about anything you can imagine, as you can see from the Yahoo! Directory of Trade Associations and Weddle’s Association Directory.
I’ve also found great statistics and infographics on Pinterest. Just try searching for your topic + [stats] or [statistics] or [infographic], etc. For example, look at these results from searching Pinterest for “Facebook stats.”

Reach out to experts

If your searching turns up anyone who seems like an expert in whatever you’re investigating, you have nothing to lose by trying to contact them directly. A brief conversation with an expert can usually give you a good overview of the area you’re exploring, highlighting key issues to consider.

Summarize and synthesize

After an hour or two of hunting and gathering relevant quotes and charts and data, I find it useful to stop and review it all, organizing my clippings into some kind of meaningful subtopics. I try to draw some conclusions and implications about what I’ve learned so far. Then I assess which areas — if any — require additional searching, or if it’s time to turn to other research methods.

I’ve used this process many times to quickly investigate a market, a product landscape, and customers’ attitudes and behaviors. Unfortunately, many companies don’t think to take advantage of all the existing research that’s out there. I hope this will make it easier for you to find those hidden treasures.

Michael Margolis is a user researcher and a UX Research Partner at Google Ventures.

View comments

Story-centered design: how to make a prototype in PowerPoint

2012-11-06T00:00:00-08:00

When I started my career as a management consultant, I was surprised at how versatile PowerPoint was. We used it for everything from seating charts to market-entry strategies for Fortune 500 clients.

At Astrid, I found a new use for PowerPoint: building prototypes. These clickable mockups are great for rapidly testing with users and applying what you learn right away. They simulate the experience of using a website or app without having to write any code. Braden Kowitz described how the Google Ventures Design Studio uses clickable mockups in a previous Design Staff article.

Here’s an example of a prototype I created using the process below.

1. Prepare your slides

Start by creating a PowerPoint file and setting the appropriate size in page setup. The actual size doesn’t matter as much as the aspect ratio — you’ll want to match the aspect ratio of the platform you’re designing for (desktop, tablet, or phone).

Create one slide for each state of your prototype. For example, a product setting with on and off states requires two slides. You’ll end up with quite a few slides, so it’s helpful to organize by adding sections in the thumbnail view on the left side of your screen.

2. Create the UI

Now it’s time to create the UI. The most straightforward approach is to create your UI directly in PowerPoint — use templates like Keynotopia’s or just build it from scratch.

If you’re really handy in Photoshop, Illustrator, or another app, you can create UI mockups there and make it clickable in PowerPoint. Here are some tips:

Export common UI elements (like backgrounds, buttons, etc) separately and use them to build the necessary states in PowerPoint.
Leave text out of your mockups and instead use PowerPoint’s text boxes — this will help you quickly create different states and make changes after usability tests.
Use transparent PNGs to simulate effects like Lightbox.
To save time, you can export mockups as complete screens (instead of separate pieces), but it’ll be much harder to make changes later.

3. Make it interactive

Now the fun part: making your mockups come alive.

First, select all slides in the thumbnails pane and disable “Advance slide on mouse click” (under “Transitions”). Web pages and apps don’t advance when you click just anywhere, so you’ll want to emulate this behavior in your prototype.

Next, link clickable elements like buttons and menus to their appropriate destination slides by right clicking and selecting “Hyperlink.” For example, if you have on and off states for a button, have the button link to the slide that shows the opposite state.

Link to specific slide numbers (for example, “Slide 5” instead of “Next slide”) and the links will automatically update even if you move slides around or insert new slides.

4. Use these advanced techniques to make your prototype really shine

Invisible boxes

These inconspicuous heroes are great if you need to add a clickable area where there is no separate element to select. Use the rectangle shape tool to draw a box, double click, select no fill and no line, then link away.

Invisible boxes are also useful when a tappable area in an app needs to be bigger than the actual UI element. (For reference, Apple suggests tappable areas should be at least 44 points by 44 points.)

If you copy invisible boxes and paste them in another slide, the boxes will appear in the exact same position on the new slide. This can be handy when working on a series of slides that are variations of the same state.

Decoy screens

Certain situations call for special transitions. For example, a welcome walkthrough for an app might have screens that swipe left as the user taps a “next” button.

In these cases, you can’t simply apply animation to a slide — transitions are tied to the introduction of the second slide (rather than the exit of the first), so you’ll trigger the same animation every time the slide loads. That’ll mess up your hyperlinks and ruin the illusion.

The solution is to create a decoy slide. When you need special transitions, create two identical slides — the first is a decoy with the sole job of displaying the animation. Set the decoy slide to automatically advance to the second, identical slide. Then hyperlink to the second slide to avoid the transition when needed.

Learn before you launch with a quick study

When your prototype is finished, conducting user research can be as simple as firing up presentation mode and watching someone use it.

You can even use a prototype like this to simulate interactivity on a mobile phone. The Google Ventures Design Studio uses Keynote to build prototypes, then exports a hyperlinked PDF and opens it in GoodReader on an iPhone. Research participants can click through the prototype like it’s the real thing!

To learn more about user research, check out the Design Staff guide to research.

Have you ever tested with clickable mockups? I’d love to hear about your experiences in the comments.

Henry Tsai leads UX at Astrid. Prior to joining Astrid, he was a strategy consultant at Bain & Company, where he advised leading technology companies.

View comments

The product design sprint: diverge (day 2)

2012-10-26T00:00:00-07:00

At the Google Ventures Design Studio, we have a five-day process for taking a product or feature from design through prototyping and testing. We call it a product design sprint. This is the fourth in a series of seven posts on running your own design sprint.

In the first two days of the sprint, we’ve learned about the problem, shared a lot of knowledge, and chosen the challenge we want to tackle in this sprint. It’s time to start cranking out solutions. Expect this step to take between two hours and all day.

I call this step “diverge” because when everyone (from the CEO to the marketing manager) is cranking out quick sketches, we tend to get a lot of ideas — and different kinds of ideas. Remember in the Legend of Zelda how the map would light up rooms you had visited as you explored the dungeon? That’s what you’re doing on Day 2: illuminating all of the possible paths.

Although you’re going to be generating ideas, don’t think of this as brainstorming — at least not the everybody-is-shouting kind of brainstorming. Instead, everyone in the sprint will be working quietly and individually, often around the same table. The exercises outlined below force you to get ideas out of your head and onto paper, without getting stuck feeling like they have to be finished or perfect.

Dust off those old ideas

In my experience, some of the best ideas that come out of sprints were usually around before the sprint started. It’s not that they were bad ideas; they just hadn’t gotten enough love yet. The sprint gives you a chance to put all solutions on a level playing field. If you don’t bring out your pre-existing ideas, you do yourself a disservice.

Because new ideas are so shiny and fresh, the facilitator needs to remind everyone to think old first. There’s no need to be embarrassed of that solution you thought of five months ago while eating a burrito or taking a shower.

Paper first

One problem with business-as-usual-design is that companies get in the habit of going straight to high fidelity mockups. In a design sprint, we start designing on paper for a number of reasons:

It’s faster
Everyone can contribute (not just designers)
Nobody gets too attached to the ideas that are generated because they’re so quick and rough. We purposefully use thick markers to make sure nothing gets too precious.
Did I mention it’s faster?

Run the series of exercises below to guide everyone from note-taking through sketching and sharing. See my earlier post for an exact list of the materials you need. I use my trusty Time Timer so everyone can see how much time is left in each exercise.

When I’m facilitating a sprint, I like to remind everybody that we’re not going to share any of the materials until we make storyboards — that’s step five of the cycle — and they’ll have plenty of time to polish those up. I want to make sure everybody feels loose and knows they’re actually going to have plenty of time to work, even though we’re keeping time as we go.

1. Choose part of the problem

In Day 1 you drew a user story diagram. Look at it together as a team. If the user story is more than two steps long (and it probably is) you’re going to need to divide it up before you start sketching. This is as simple as finding natural chunks in the story and drawing a box around them, like this:

Now decide which part to focus on first. It usually makes sense to have everybody in the sprint focus on the same part of the problem at once. If you take that approach, you’ll do one cycle for each part of the problem, with everybody collaborating on each part as you go.

You can also divide and conquer — everybody picks a piece of the story they’re interested in and works on that. This way is usually faster, although it introduces the risk that people don’t think about the user story holistically. If you have more than two or three chunks in your story, you might have to divide and conquer, or perhaps decide you’re going to focus on a smaller part of the problem for this sprint.

Either way, the facilitator ensures that everybody knows which piece of the user story they’re focusing on before you continue.

2. Take notes (5 minutes)

At this point in the sprint, the whiteboards and walls are probably covered in diagrams, notes, and “how might we” sticky notes. This is your chance to reload that stuff into your brain. Everyone takes a piece of paper and jots down anything they think is useful.

3. Mind map (10–15 minutes)

Now you’re going to add all the other ideas that are in your head, mix them with the notes you just took, and loosely organize them on paper. The mind map is going to be your “cheat sheet” you can use when you’re sketching UI ideas.

If you’re not familiar with mind mapping already, I often describe it as writing down everything in your head with no specific formatting; or quiet individual brainstorming. You can write words and connect them or not, you can draw pictures or not — you basically can’t do it wrong. The important thing is that everyone is getting every solution, old and new, out of their head and onto paper at very low fidelity.

Here’s an example:

4. Crazy Eights (5 minutes)

Everybody folds a blank sheet of paper in half four times, then unfolds it, so they get eight panels. Then you have 5 minutes total to draw eight sketches, one in each panel. Yes, you did the math correctly, that’s about 40 seconds per sketch, which is crazy… but it’s a great way to crank out variations of ideas quickly. And since these aren’t shared with the group, there’s no need to worry about making them pretty.

Since you only have 40 seconds for each drawing, you’ll need to turn off the self-editing and just get your ideas on paper. Crazy Eights will also help loosen up your creative muscles and make you more productive in subsequent sketching exercises. If you get stuck, try repeating an earlier sketch with a small variation — this type of exploration is useful and it keeps you moving.

For best results, do two rounds of Crazy Eights. On the second round, everyone will have the hang of it. You’re scraping the bottom of the barrel, which makes it more painful to come up with new ideas, but often this is where the most interesting solutions come from.

Now you may be thinking I’m a bit of a hypocrite: earlier in this post I said old ideas are best, and now I’m asking you to come up with new ideas. Don’t get me wrong, it’s OK to fill out your Crazy Eights sheets with old ideas. But new ones are good too — just because old ideas tend to be stronger doesn’t mean they always win.

Pro tip: Get the Bit Timer app for your iPhone and set it to 30-second work periods and 10-second rest periods for eight reps so you don’t have to time it yourself. The rest period alarm acts as your 10-second warning to wind down your current sketch.

(Crazy Eights are based on the 685 exercise introduced to us by Brynn Evans.)

5. Storyboard (10–20 minutes)

Now we’re going to make that user story diagram more concrete, and we’re going to make something that will be shared anonymously and critiqued by the group. The goal is to take the ideas we’ve generated so far and sketch an actual UI showing how a user would move through this part of the story — where they click, what info they enter, what they think, etc.

Start with a blank sheet of paper, and put 3 sticky notes on it. Each sticky note is one frame in the storyboard. It’s kind of like a comic book that you’re going to fill in.

Look back at your mind map and your Crazy Eights and find the best ideas. Chances are, you’re itching to illustrate at least one of them in more detail. Now you’re ready to rock. I ask everybody to draw UI in the three frames of their storyboard showing a progression: first this, then that, then that.

There are three important storyboard rules:

Make it stand alone — Just like a real product, your drawing has to make sense by itself, without you there to pitch it. In the next steps, people will be looking at these, but you won’t have a chance to talk about your idea until the end.
Keep it anonymous — Don’t write your name on your drawing. You’ll want all ideas to start on a level playing field and it can be distracting to know which one was drawn by the CEO.
Give it a name — Come up with a catchy title for your idea. That makes it easier to discuss and compare later.

When you finish the storyboards, hang them on the wall with some sticky stuff. Pro tip: Hang them side by side (like an art museum) so people won’t have to crowd in too tight to see them.

6. Silent critique (5–10 minutes)

Give everybody a bunch of dot stickers. Then, without speaking, everybody looks at the different storyboards and puts a sticker on every idea or part of an idea they like. There are no limits to how many stickers you can use, and I don’t even prevent people who want to brazenly vote for their own ideas. By the end, you’ve got a kind of heat map, and some ideas are already standing out.

7. Three-minute critiques (3 minutes per idea)

Next, everybody gathers around the storyboards one at a time. First, people talk about what they liked, then we ask the person who drew it if we missed anything important. Usually the best, most popular ideas are the ones people can understand without an explanation, so the author of the storyboard often doesn’t have anything else to add. This process works far better than letting people explain their ideas first — which almost always uses up a lot of time.

Sometimes I like to do this step on a projector, especially if there are a lot of ideas to get through. I’ll take photos of each storyboard on my phone, upload them to Dropbox, put them in a Keynote file, then make notes about parts we like with outlines and text labels as we go through on the projector. This is easier for everyone to see, and you have a digital artifact of the ideas for later. The downside is the setup: count on 15 extra minutes to capture and upload photos.

8. Super vote (5 minutes)

Once we’ve looked at all the ideas, everybody gets one or two “special” stickers (which can be the same dot stickers from before with a pen mark on them). These are “super votes” for the ideas you think are the very best. Between the original heat map and these super votes, it’s very easy to see which are the strongest concepts.

The super votes offer a unique way to tweak the process to reflect the decision-making structure of your team or company. Does your CEO make all final decisions about the product? If that’s the case, be honest about it, and give her three super votes and everybody else one. Or maybe it’s a UX director or maybe a tandem of product and design who call the shots. The simple rule is to give the deciders extra votes.

By default, this process will be a meritocracy, but that’s not always the way companies work and, frankly, consensus can lead to poor design decisions. The last thing you want are decisions that the deciders don’t truly support. On some teams, these may be unwritten rules, so don’t be surprised if it feels a bit awkward to bring it up — in the long run, you’ll be glad you did.

Repeat

Now it’s back to the first step to start the whole cycle over again. (Don’t worry, it gets easier with every repetition.) If you split up the user story last time, it may be time to move on to another chunk. Often when I’m running a sprint I’ll realize at this point that our scope was too large, and we should just double down and keep working on the same section. Either way, the end of a cycle is a good time to take a few minutes and carefully decide where to focus next.

Expect a team to be able to do this cycle two or three times in a day before getting burned out. Throw in plenty of breaks and snacks to keep the troops moving.

Stay tuned for the next episode, where we’ll talk about how to decide which pieces of these storyboards go into your prototype.

Product design sprint series
Previous: Understand (day 1)
Next: Decide (day 3)

Jake Knapp is a process geek and design partner at Google Ventures.

View comments

The product design sprint: understand (day 1)

2012-10-16T00:00:00-07:00

At the Google Ventures Design Studio, we have a five-day process for taking a product or feature from design through prototyping and testing. We call it a product design sprint. This is the third in a series of seven posts on running your own design sprint.

Now that you know when to get your team together for a sprint and how to set one up, it’s time to tackle the first day of the sprint: understanding.

Chances are that everyone involved in the sprint has different perspectives on the problem — and different information that might be helpful. The goal of the first day is to encourage everyone to share what they already know and develop a common understanding with the rest of the group. By starting at the beginning (even if some people are already familiar with the problem), it nudges the group into a beginner’s mindset and leads to fresh solutions. (This is where an outside facilitator can come in handy: since they’re truly new to the problem, their questions can keep the group in a beginner’s mindset.)

Use these exercises to help build understanding

We use the exercises below to get a ton of information on the table and quickly build understanding. You can do them in any order you want — and even omit the ones you don’t find valuable. Try capping each presentation or discussion at 10 minutes. This will keep the day moving and help everyone pay attention.

During the exercises, everyone in the sprint should be jotting down questions on sticky notes. We use the “how might we” format to capture opportunities that might be interesting to explore. For example, “How might we build trust?” or “How might we figure out the user’s style?” Often, these end up being extremely useful in the next steps of the sprint.

Business opportunity — The CEO or product leader should walk the sprint team through the business opportunity and market.

Lightning demos — Look at competitors’ products. It can also be helpful to look at non-competitive products that solve a similar kind of problem in a different market.

Lay it out — Print out all the important screens in your product, lay it out, and walk through it as a user would.

Success metrics — How will you measure the success of this design? Now’s a great time to talk about success metrics. We like to use Kerry Rodden’s HEART framework.

Existing research — If you have user research for your product, that’s awesome, and you should be sure to go over it. If not, you should talk about whatever data you do know about your customers.

Team interviews — Knowledge about the problem is usually distributed across the company. We’ve found it very useful to go around interviewing people at the company who have specific expertise, whether that’s engineering or sales or customer service. (Customer service people often have incredibly valuable information about the problem.)

Analytics — Look at any data you have on feature usage, where customers drop off your site, conversion rates, etc.

Feeling overwhelmed? Gather data before the sprint

If you’re reading this and feeling like the exercises will be tough because you don’t have enough data, don’t worry! That just means you should do some work before the sprint to quickly gather fresh data.

It could be as simple as scheduling a few user studies or deploying a very short survey with questions related to the problem you’re going to tackle in the sprint. You can also do team interviews ahead of time — this is especially useful if you’ve got a lot of people to talk to.

Sketch the most important user story

As a group, use your common understanding to collaboratively map out the user story that’s important in this sprint. The facilitator (or another volunteer) should stand at the whiteboard and sketch the flow. This doesn’t have to be fancy to be useful. Here’s an example:

How do you know which user story is most important? It depends on the problem you are solving in the sprint. For example:

Helping people understand and get started with your product — you probably want to focus on the experience of a user encountering your product for the first time.
Creating a new product concept — you probably want to look into the future and imagine the value proposition and core features for an engaged user.
Improving conversion rate from a landing page — you probably want to understand why people land on your page and what their goals are.

This step can be difficult and time-consuming, but it’s critical! Getting a visual map on the wall (like the one above) is invaluable for grounding the discussion and keeping everyone on the same page.

Now it’s time to focus

You’ve got a rapidly approaching user study and there’s a good chance you can’t prototype everything you uncovered, so you’ll need to choose which ideas to move forward. Use your story diagram on the whiteboard to frame this discussion.

At the end of the sprint, when you show your prototype to users, what do you hope to learn? What do you need to design and prototype in order to learn those things? What you decide here will set your course for the rest of the sprint.

The facilitator is responsible for keeping these discussions moving quickly (remember the timer). It’s also important to ask seemingly obvious questions because the answers usually further everybody’s understanding.

Now that you’ve fleshed out a common understanding of the problem and started to define which part of it you’re going to tackle in this sprint, it’s time to move on to the next step: rapidly developing as many solutions as possible.

We’ll cover that in the next post, so stay tuned!

Product design sprint series
Previous: Setting the stage
Next: Diverge (day 2)

Jake Knapp is a process geek and design partner at Google Ventures.

View comments

The product design sprint: setting the stage

2012-10-09T00:00:00-07:00

At the Google Ventures Design Studio, we have a five-day process for taking a product or feature from design through prototyping and testing. We call it a product design sprint. This is the second in a series of seven posts on running your own design sprint.

Now that you know what design sprints are good for, you’ll need a few important ingredients to make yours successful. Start with a big, important problem; pitch it to your team; and schedule a user study before you even start. Get the right people and the right supplies in a room and you’re on your way to a successful design sprint.

Pick a big fight

The first thing you need is an important design problem, and if you work at a startup, chances are good you probably have one lying around the office. Maybe more than one. It might be something big, like defining your product for the first time, or a big redesign or new feature. Or it might be something detailed, like improving conversion on a single user action. It just has to be really important to the company, and it has to be something you’re struggling to start or to make progress on — otherwise it can be difficult to get the other people you’ll need involved.

As long as it’s an important problem, it’s perfect for a design sprint. It’s OK if you don’t feel ready to start on it yet. No matter how overwhelming or ambiguous, you’ll be able to cut a big swath through the jungle of possible solutions.

Get the right people

The ideal sprint team is between four and eight people, but you can get by with more or fewer than that. Just make sure you have at least one:

Designer – If your startup doesn’t have a designer yet, try to bring in a ringer.
CEO – At a small startup, the CEO is the key decision-maker and needs to participate in order to get an actionable solution out of the sprint. At a bigger company, you’ll still need buy-in and it’s best to include the CEO, but if they can’t be there the whole time you can bring them in at key decision-making moments.
Product manager – The PM (or whoever is filling this role) will likely need to implement the solution that comes out of the sprint.
User expert – The person on the team who has the most direct contact with customers often has great input, and can be the lead on user testing.

It’s also great to include:

Engineer
Marketer
Anybody else who’s interested

Schedule the user study before you have anything to test

Once you know when you’re going to do the sprint, recruit users and schedule the user studies for the last day of the sprint. This is a bit terrifying: you haven’t even started to talk about the problem, let alone design solutions, and people — outsiders! — are going to come in and need to be shown something. This hard deadline, even though it’s artificial, will help you move faster and make tough decisions to focus your work throughout the sprint.

Find a facilitator

Pick someone to be the facilitator of the sprint. The facilitator is going to be responsible for managing the sprint and moving things along. They need to be confident leading a meeting, including synthesizing discussions and telling people it’s time to stop talking. They also need to be comfortable with not getting to participate as much in the actual design work, because facilitating is a lot of work. Since you’re the one reading this, you may be a good candidate — but it’s always easiest if the facilitator is an outsider. See if you can get a friend from another company to help out.

Put it on the calendar

Clear everybody’s schedule for five consecutive days. It’s also very important to have a dedicated room for the duration of the sprint, usually a conference room with lots of whiteboards.

Much of the magic in design sprints comes from the sense of urgency. By their very nature, startups always feel time-constrained; the short, focused time of the sprint adds another constraint.

Gather the ingredients

Luckily, you don’t need anything fancy to run a sprint. Here’s everything I use:

Sticky notes — I like the yellow 3×5 size.
Drawing pens — Any standard black or blue pen is probably fine. I like these or you can get these for extra credit.
Whiteboards — If your war room doesn’t have a lot of whiteboard space in it, find another war room or some rolling whiteboards or, heck, get some IdeaPaint and get busy.
Whiteboard markers — I like to use these instead of Sharpies because they’re so versatile. Buy some good ones and be sure to have enough black markers for each person in the sprint.
Dot stickers — for voting. You want something small with uniform color. Post-It brand dots are great.
8.5 × 11 blank copy paper — Nothing special, just have a pack of this on hand.
Time Timer Clock — Optional, but totally awesome, see here. I guarantee you’ll find it useful during the sprint, and probably during regular meetings afterward.
Snacks — You’ll need caffeine and food handy. Trail mix, bananas, and dark chocolate covered raisins have proved especially popular at our sprints, although it is possible that it’s just me eating all of it.
Sticky stuff — You’ll need to stick your drawings and storyboards on the wall. This removable gummy material is inexpensive and works great, with less fuss than tape.

Okay, the stage is set. Now it’s time to start the sprint.

Stay tuned for the next post, where we’ll explore the exercises we use on the first day of a sprint.

Product design sprint series
Previous: Overview
Next: Understand (day 1)

Jake Knapp is a process geek and design partner at Google Ventures.

View comments

The product design sprint: a five-day recipe for startups

2012-10-02T00:00:00-07:00

At Google Ventures, we do product design work with startups all the time. Since we want to move fast and they want to move fast, we’ve optimized a process that gets us predictably good results in five days or less. We call it a product design sprint, and it’s great for getting unstuck or accelerating projects that are already in motion.

I’ve planned and run over 40 of these sprints, first with teams at Google and now with startups in the Google Ventures portfolio. To give you an idea of what one looks like, here’s a project we did with CustomMade:

Over the next several posts, I’ll be sharing a DIY guide for running your own design sprint.

Before the sprint: Prepare
Get the people and things you need.

Day 1: Understand
Dig into the design problem through research, competitive review, and strategy exercises.

Day 2: Diverge
Rapidly develop as many solutions as possible.

Day 3: Decide
Choose the best ideas and hammer out a user story.

Day 4: Prototype
Build something quick and dirty that can be shown to users.

Day 5: Validate
Show the prototype to real humans (in other words, people outside your company) and learn what works and what doesn’t work.

If you think you’ve heard of this model before, well, you’re right. It’s based on the design thinking structure championed by IDEO and Stanford’s d.school. However, I’ve experimented and tweaked the process a bunch over the last few years. The version I’m going to share works especially well for startups.

What doesn’t work about brainstorming

I’m a big-time process nerd. Several years ago I started experimenting with product design processes at Google. At first I ran group brainstorming workshops inspired by the IDEO approach.

Group brainstorming, where everyone shouts out ideas, is a lot of fun. At the end of a workshop we’d be tired, in good spirits, and the proud owners of a big pile of sticky notes. But the new ideas we came up with didn’t go anywhere. It’s not that we were coming up with dumb ideas — most of the ideas were actually pretty good. Yet still, better ideas were coming from somewhere else. But where?

In my experience, the most successful ideas tended to come from individuals, not groups. The ideas took some individual heads-down work time to develop, too. I ran a lot of workshops before I realized this, so if it seemed obvious to you from the beginning, I hope you’ll cut me some slack.

To make matters worse, my workshops were choosing the winning ideas by consensus. But consensus doesn’t always pick the bold ideas, the unique ideas, or the ideas with design integrity. Consensus tends to compromise.

There were a lot of good things going on in the workshops: focusing a team on one project, considering a range of ideas instead of just one, working on paper, and more. But I decided my method was fundamentally flawed. I was getting good-but-not-great ideas through group brainstorming, then choosing winners by consensus. I knew that wasn’t working, but at first, I wasn’t sure what to do about it.

The magic of constraints

One day I noticed something about my own design projects. The best work happened in short bursts, when I was under a deadline.

One example was Gmail Priority Inbox, where we spent four weeks experimenting with different design prototypes. There were hundreds of internal dogfood users signed up to try a new experiment each week, so I had to move fast. By the end of the four weeks, I’d figured out which things worked, and saved months of noodling.

Another example was the project that became Google+ Hangouts. It started as a side project with two Googlers in the Stockholm office. I was only visiting for two days, so I designed as fast as I could. By the end, we had a working prototype that we could start using for our team’s meetings.

In both of these cases, I worked far more efficiently than I ever did in my normal day-to-day routine or in any of my brainstorm workshops. So what was different? I had time to focus and develop ideas on my own, not shouting and pitching the way I would in a group brainstorm.

But I also didn’t have too much time. I couldn’t afford to overthink things or get caught up in urgent but less important issues, the way I often did on normal workdays. And the people I needed to help me — engineers and product managers — were also focused on the project.

There was something magical about a tight time constraint combined with individual work, prototyping, and quick user feedback.

Adapting IDEO-style workshops to work at Google

I decided to try linking an IDEO-style “how might we” workshop to a few days of uninterrupted design time to execute on the best ideas. In that very first sprint, designer Jason Cornwell roughed out the idea for the Gmail people widget. I knew I was on the right track.

I focused full-time on running design sprints with various teams at Google. I switched from group ideas to individual ideas and gave people more time to develop those ideas before getting feedback. I tried a bunch of critique and decision-making exercises that didn’t rely on consensus and chose a handful that worked best.

I got a lot of practice: I’d jump from team to team within Google, for a few days or a week at a time, leading sprints for projects like Chrome, Ads, Commerce, Apps, Search, and Google X. It was exciting. The designs were launching, and lots of teams started to run the process on their own.

10x faster: Running design sprints at startups

When I joined Google Ventures, I thought I had sprints all figured out. But I quickly realized I had a lot to learn. The process had to be changed to reflect the differences between a large company like Google and the startups in our portfolio. For example, at Google, it was easy to get three or four designers together for a few days. At a startup, they might be lucky to have one. So we need design and critique processes that engineers and CEOs could do as easily as designers.

Startups want to get their product out there quickly and learn what’s working, but it’s costly to launch — you have to write more code, fix more bugs, and handle more issues than with a prototype. So we compressed the sprint cycle even further to get companies faster feedback. I ditched polished Photoshop mockups in favor of quick-and-dirty Keynote prototypes. Michael Margolis tied in his lightning-fast research techniques to deliver us next-day feedback.

We’re still learning, but we’ve run enough sprints to be confident the process works well.

Stay tuned to this series, and please give me your thoughts along the way — I’m always looking for more tricks to improve what we do. What processes do you use to get good design results? What helps your company move faster?

Product design sprint series
Next: Setting the stage

Jake Knapp is a process geek and design partner at Google Ventures.

View comments

Rapid user research: how to survey 400 users and interview 10 in three days

2012-09-19T00:00:00-07:00

I want to share an effective quick-and-dirty research approach I stumbled upon recently.

Not long ago, I got an urgent request for data about people’s habits and needs related to watching online video. (I’ve changed the real topic to protect the innocent.) How, when, and where do people watch online video? What do they watch? How are people using Product X? How has the launch of Product Z affected use and perceptions of Product X?

These are big discovery-research questions. Quickly finding users of Product X wasn’t going to be easy. And I needed results in just a few days.

As a shortcut, I first looked for relevant research that had already been done. Then I created a screener questionnaire to recruit Product X users and collect interesting survey data from a very large sample. Here’s how I did it:

Step 1: Quick review of existing research

I first spent a few hours scouring the web for relevant data and research that already exists. You can usually find loads of helpful articles and reports (and articles about the reports) by firms like Forrester, Pew, Gartner, and Nielsen. Google Scholar is another great source for relevant academic research. I also like to search for product reviews and comparisons. I collected and summarized key data and charts I found about usage patterns and industry trends.

(Update: I wrote a post describing how I review existing research.)

My quick review provided valuable data and prognostications about patterns and trends, but it didn’t answer our detailed questions about people’s viewing habits, especially for users of Product X. I needed to find some Product X users to talk to right away.

Step 2: Draft a recruiting questionnaire that’s also a survey

To find users of Product X I could interview over the phone, I drafted a recruiting questionnaire. (See “How to find great participants for your user study.”) In addition to standard questions about demographics and product use, I included a few more exploratory questions. For example:

When watching video over the Internet in the last week, what types of programs did you watch? (multiple choice)
Which one video app or website could you not live without? Why? (open ended)
In the past week, where have you watched video over the Internet? (multiple choice)

Step 3: Advertise widely

For just $25, I posted an ad (“$50 usability interview”) with a link to my recruiting questionnaire to Craigslist’s “et cetera jobs” category in a big city.

The response was overwhelming! In two days almost 400 people — including dozens of Product X users — completed my screener questionnaire and answered my additional survey questions. I had quickly flushed out the Product X users I needed and collected really interesting survey data from a large sample. I quickly completed ten 30-minute interviews and reported all of my results to a very appreciative team. Grand total? Three days and $525.

Next time you need to quickly gather information about users and products (especially products that aren’t your own), try using or adapting this scrappy hybrid approach to get the answers you need.

Michael Margolis is a user researcher and a UX Research Partner at Google Ventures.

View comments

Does your startup need a product overview video?

2012-08-22T00:00:00-07:00

Video is seductive. When done well, it is emotional, evocative, and revealing. It is one of the best ways to show the human side of a story. But it is almost never the right way for early-stage startups to communicate what their product does. Despite their strengths, product overview videos are expensive to produce, difficult to change, and present a large mode-switching barrier to the user.

Why do so many startups choose to explain their products with video?

Startups often emulate established, successful companies. For example, Apple has mastered communicating with video — they create both moving human stories about how to use their products and clear how-to videos that accompany new-product launches. And well-produced videos can make a tiny startup look larger than life. When you’re trying to build trust in a new service, looking big can be a big help — especially in sectors like finance or cloud storage.

When I worked at YouTube, I saw the power of great video every day — whether it was groggy kids, extreme sports in HD, or a dying professor’s last lecture.

But when I started working with startups, I saw small teams work hard on videos that didn’t meet their goals. They invested a lot of time and money in the video, so they felt obligated to use it. It took me a while to realize that video is rarely the right way for early-stage startups to communicate what their product does. These days, I advise startups to consider their communication goals and decide whether a video is worth the cost — monetary and otherwise.

Why you don’t need a product overview video

Videos are expensive to produce
Producing high-quality video is hard. You can produce a video yourself, but it will be incredibly time-consuming. Or you can hire an agency to do it, which is expensive and still requires a lot of your time. One challenge working with an agency is that they don’t know as much about your business and your users as you do, so they have a hard time making good editing decisions.

It’s hard to iterate with video
The high cost (both in money and time) of video production means iteration is expensive and time-consuming. Every change you make takes longer and costs more than making an equivalent change to text and images.

It’s particularly painful when you find a small error that needs correcting — you have to open a huge file, make the change, wait for it to re-render, upload, etc. (Plus, YouTube won’t let you replace an existing video. You have to upload a new one.)

Video presents a mode-switching barrier to your users
With video, the biggest obstacle to communication is getting people to click “play.” Images and text on a page present no such barrier — people can jump right in.

When I was a designer at YouTube, I watched dozens of people encounter videos in research studies. Even then (on a video website!) I often saw a barrier to clicking “play” — users need to adjust the volume, make sure they won’t disturb anyone (perhaps putting on headphones), consider the bandwidth, and decide whether this video might be worth the wait.

When is video a good idea?

When you know the best way to explain your product
The best early-stage startups do everything they can to learn and improve their product quickly. This process applies to messaging too — we all have assumptions about the best way to explain our products, and we learn quickly whether these assumptions are correct.

Video is expensive and hard to change, so it’s best to wait until you’re confident about your message before committing to a video. For example, this video explaining Gmail Priority Inbox was created after the team tested a series of cheap screencasts and learned what was working.

When you want to show, not tell
Video is more effective than text and images at showing how a product works — people can actually see it in action! Particularly with new or novel types of products (e.g. Dropbox), showing — not telling — people can make a big difference.

These types of videos are often cheaper to make and easier to change, too. You can use the techniques described by Jake in his post about making screencasts.

However, these videos quickly get out of date when the UI changes. When I worked at FeedBurner, I was constantly realizing (too late) that the new design I launched required me to go back and update a bunch of videos in our help content. If you decide to use video tutorials or demos, remember they can create extra work every time you update your product.

When your content is video
Some products are all about video: YouTube, Udemy, Funny or Die, and many more. These websites either pass along the cost and ease-of-iteration challenges to their users (YouTube) or build an internal competency at dealing with those issues (Udemy, Funny or Die).

Getting people to click “play” is an even more important challenge for these products — their growth and success depends on it. YouTube addresses this by automatically playing the video when the page loads and nudging people into watching playlists and other strings of videos.

What are your experiences using video to communicate important messages? Do you have examples of great videos created by early-stage startups? Have you found techniques or tips for avoiding the challenges I described?

I’d love to hear your thoughts in the comments!

John Zeratsky is a design partner at Google Ventures and the editor of Design Staff.

View comments

Design Staff guide to research

2012-07-17T00:00:00-07:00

Research provides the inspiration, guidance, and validation we need to design great products. From the personal (like interviewing users) to the analytical (like metrics) there’s a continuum of research skills that are essential for design teams. The most effective companies integrate this discipline into their culture and make research a habit.

This is our guide to design research, featuring Design Staff articles and other posts we think are important and influential.

The big picture

Questions to ask before starting user research
Michael Margolis, Google Ventures
Start with research questions — what are you trying to learn? — before selecting research methods.

Meetup’s dead simple user testing
Clay Shirky, Boing Boing
Make research a habit at your company: talk to users every week.

How to choose the right UX metrics for your product
Kerry Rodden, YouTube
Data-informed design starts with choosing the right metrics. Don’t settle for basic measures like traffic or unique users.

Seven tips for lean market research
Michael Margolis, Google Ventures
Want fast answers to big questions about your market, competitors, or customers? First look for existing research.

Talking to users

Why you only need to test with five users
Jakob Nielsen, Nielsen Norman Group
Optimize your research for speed and value by running small tests more often. Five users is a sweet spot.

How to find great participants for your user study
Michael Margolis, Google Ventures
Learn to recruit the right users for your studies. It’s simple once you know how.

Super-fast research recruiting with TaskRabbit
Henry Tsai, Astrid
Speed up research recruiting by hiring TaskRabbits as participants.

Thinking aloud: The #1 usability tool
Jakob Nielsen, Nielsen Norman Group
Simple usability tests where users think aloud are highly effective. They are also cheap, flexible, and easy to learn.

Get better data from user studies: 16 interviewing tips
Michael Margolis, Google Ventures
Great research interviews should be like Terry Gross on Fresh Air — engaging and insightful.

How to hack your body language for better interviews
Michael Margolis, Google Ventures
Learn how to monitor and adjust body language to help research participants feel comfortable and confident.

Remote research

Tips for testing your designs with UserTesting.com
Michael Margolis, Google Ventures
UserTesting.com is a great, low-cost way to run basic usability studies.

Improve your surveys and get even better data
Elizabeth Ferrall-Nunge, Twitter
Surveys are useful for gathering feedback, but they’re often misunderstood and misused.

Micro-surveys: a faster way to learn about your users
Sasha Lubomirsky, Airbnb
Micro-surveys are short and highly targeted. They’re not only easy to answer, but easy to create, implement, and analyze.

Rapid user research: how to survey 400 users and interview 10 in three days
Michael Margolis, Google Ventures
Speed up user research by writing a recruiting questionnaire that’s also a survey.

Tools and resources

UserTesting.com — easy remote usability testing
WebEx — conduct and record remote interviews
Ethnio — intercept website visitors for research
Rocket Surgery Made Easy — Steve Krug’s guide to finding and fixing usability problems
KISSmetrics — track user-centered metrics
ScreenFlow — record and edit video from in-person user studies (Mac only)
Silverback — record and edit video from in-person user studies (Mac only)
Verify — collect and analyze user feedback on visual concepts

John Zeratsky is a design partner at Google Ventures and the editor of Design Staff.