You will (probably) learn:

• Is it usable? Can people figure out how to navigate through your product?
• Are your calls-to-action visible? Do people notice the buttons or links that you want them to click on?
• Are the steps to use a feature clear? Can someone figure out how to complete a task that you set them?
• Is your copy clear? As people scan the words in your product, are they comprehending and moving forward smoothly? Or are they frowning and hesitating?
• Is your product intimidating / jarring / confusing / threatening? As people are going through testing, is their body language indicating discomfort? (do they have furrowed brows, tense shoulders, are they fidgeting or hesitating, are they frowning)

Your job as a designer is to solve problems, not to make things look pretty.   To do your job, you need to understand the who, why, when, what, where, and how.

People will try to hand you a spec or a list of requirements and say, “I’ve already thought about this a lot, just design it.”  Don’t accept that.  Insist, politely but firmly, that they tell you what the main goals of this project are.   For any project, one can say, “if it doesn’t achieve X and Y, then we’ve failed”.  Are you trying to sell or educate?  Reassure or challenge?  Are you encouraging exploration or optimizing for speed?   Is this a one-time signup or an everyday task?  Is the audience skeptical, or already enthusiastic?

People will suggest that you just use a lightbox, or just use the same styling that we used for feature Q, or just copy what Company Z is doing.  These might end up being the best available solution, but you won’t know unless you push back to the defined objectives.

“OK, I like how Facebook uses that design element to solve X problem.  Are we solving X problem, or is our situation different?” or “Yes, using that green button style would be consistent with what we’re doing on screen Y.  But on screen Y the user is completing a one-time configuration, and in our case we’re trying to make a common task as fast as possible.   Does it make sense to force consistency for completely different types of behavior?”

Shine some light on the design process.

I also call this, “don’t be magic.”

If you’re a skilled designer and an intuitive listener, you can probably combine what people are saying and not saying, deduce what they’re hoping to end up with, and make it magically appear on your screen. Voila!   People will love you for this.  It’s a critical skill for early-stage startups and design emergencies.

It’s also terribly non-scalable.  It allows people to believe that they are communicating clearly when they’re not.  It allows people to believe that design is “just drawing”, and that the thousands of implicit decisions you’re making about visual priority and color and scale and ordering are arbitrary.

You need to translate out loud: “So it sounds like you’re asking for X and Y, and you like the way that Company Z solves this problem because they have these similarities to us.  And you’re looking to solve problem Q.  Is that right?”

You need to explain your decisions: “I’m using this style because it emphasizes element A, which is the single most important action a user has to take.  I’m deliberately not copying what we do for feature B, because the target user is completely different.”

Fight for what you believe in (pragmatically).

In past jobs, I’ve worked with designers who were at opposite ends of this continuum:

[tell me what to design and I'll crank it out] <———————> [change 1 pixel and you'll destroy my masterpiece]

Neither is productive.  And it’s incredibly hard to learn where on that spectrum is the most effective for you, your personality, and the organization you’re in.  But you’ve got to try.

If you really believe that adding that fourth link will clutter up the UI, speak up and explain why.   Feel free to express your doubts and the risks.  And then, if your stakeholders disagree, pick your battles.  Sometimes it’s worth it to fight to the death.  Usually it’s not.

Be clear on what you will deliver, and when.

These are the questions that people will have but usually not ask:

• How likely are these designs to change?
• How final are the details like fonts, icons, and images?
• Are you going to illustrate the interaction or will this be static?
• Are you illustrating just the core use cases or multiple edge cases and usage scenarios?
• Will everything be done, or ‘enough to get started’?

You’re better off listing out what you’re going to deliver in writing, with the above questions answered, and a date.  It will feel like overkill.  It will prevent a lot of misunderstandings.  Use it as a checklist.

Recover and compensate.

If something comes up — and it will — and you are not able to deliver what you promised, immediately reach out and offer a plan for getting back on track.

Ask to make sure that’s the most convenient/effective plan for the people on your project.

Do not let a deadline slip without a word.  Do not go off without a word, work in silence, and re-emerge 3 days later with all the work done.   Speak up immediately so no one has to wonder or go looking for you.

Always be thinking about how we could be doing things better or smarter.

Our process is not there to constrain you, it’s there to help the team work more effectively.  If it’s not working, you chafing at it and making yourself miserable will not help.  Trying to sneak around the rules won’t help either.  Complain constructively so we can fix it.

If you find yourself doing the same task repeatedly, stop and ask if there is a way to automate it or simplify it.  If you feel like work you do is wasted, stop and ask why.

Or — if you did something awesome, stop and teach your peers what you did.  Share successes.  Rehash good meetings and projects, not just bad ones.  Analyze why things went well, and try to reproduce.

But fear not, because:

1. It’s not too late. Promptness is for after one-off coffee meetings and job interviews.  Conferences mean the recipient spent a couple days ignoring their inbox while they traveled and schmoozed — they’re probably a lot more receptive to your note if comes several days or a week later once they’ve caught up on sleep.
2. It’s OK if they don’t remember you. That’s what “reminding people” is for.  If you send a message specifying where the two of you met, what you chatted about, and 1-2 salient facts about yourself, the recipient will either remember you or they’ll take 30 seconds to Google you / check your LinkedIn.  This is good enough.
3. …Unless you sent a generic LinkedIn connection invite. I’ve started ignoring these on principle, unless they’re from a coworker or a customer.  Please customize the text so I remember who you are — those little avatars are not clear enough to jog anyone’s memory.
4. Connecting with people is why people attend events. If all I wanted to do was hear what was going on at any particular conference, it would be a lot faster and cheaper to just follow the tweet-stream.  It’s the connections to people (whether it’s me helping them this time or them helping me) that make conferences worthwhile.
5. One reminder is okay. Didn’t hear back?  Try again.  Personally, I’ve never been annoyed by a single followup reminder — if anything, it relieves me of the residual guilt of knowing there is some email, somewhere, that I haven’t gotten back to.  I send tons of them, too, and the response rate is incredibly high.  Fundamentally, people like helping you, and one reminder helps us do that.

oh, and uh, hi everyone I met.  Your email is coming soon.  Hope you’ve recovered!

But what do you DO about it?   Power users are your evangelists, your backup QA testers, and your visionaries.  They’re also by far the easiest to get in touch with (just try to avoid them — they’ll find you.)   It would be stupid to blindly ignore their feedback.   It’s also really hard to tell sometimes: is this an issue that only affects the power users?  Or is it a universal thing, and they were just the first to detect it?

Here’s how I try to evaluate feedback, bug reports, suggestions:

Understand When, How Often, Why

• When is X happening? (in what circumstances? if there was anything numerical involved like # of messages, file size, # of users, what was the number?)
• How often is X happening?  (every single time/occasionally/just once but it was really catastrophic?)
• Why is X happening? (what were you doing right before it happened?)

The fastest way to ask these questions is just to shoot the person an email asking followup questions — most people will be able to answer.  I often also add “if it’s easier to explain over the phone or chat, here’s my contact info”).

You’ll hear things like “…it happens when I upload a file.  What?  Oh, the file is 700MB, does that make a difference?”  or “well, I wanted to match our obscure data format, so I changed every single one of the settings…”

Look at Non-Average Data

Averages are really easy to measure, and lots of tools default to “average” values.  But the “average” value hides a ton of variability.   Let’s say you have 10 users who sent 1, 1, 1, 1, 2, 4, 4, 4, 800 messages.  The average number of messages sent is 81.8.  The median number of messages sent is 2.

That’s a 40x jump.  That’s a good multiplier to keep in your head when you read feature requests.  Just by asking yourself, “is this a situation that only affects the 40x user?”, you’ll more effectively prioritize feature requests, complaints (and compliments).

Go Find Some Normal Users

Want to know if this problem is affecting mainstream users?  Ask them.  You don’t need a lot of data – just a couple of anecdotes can provide a very useful sanity check on your assumptions.

I find mainstream users to talk to by:

• Responding to people who submitted support tickets.  (Even if it was an unrelated issue, if you email them personally, and it’s clearly not an automated/marketing email, and you’re nice, a decent percentage of them will answer a question or two for you.   Surprising but true!)
• Using KISSinsights to pop up a survey question while people are on-site, then follow up to their response via email.
• Asking your power users if they have “less tech-savvy coworkers” who also use your product, and can you get their email address?  You’ve got to be tactful with this one: you don’t want the coworker to feel dumb.

Usually the “conversation” can be as simple as: “One of our other customers described this problem.  Does this sound familiar to you? / Has this ever happened to you?”

If they have no idea what you’re talking about, this may be an ignorable power-user problem.

If their response is a “yes”, a “well, sorta…”, or a related followup question — you probably have an issue that affects a significant percentage of users.  Now it’s time to dig deeper into the when/how often/why.

Think about None, A Few, and A Lot Use Cases

Settings, features, preferences, limits, and displays are often very different based on ‘amount’ use cases.   We tend to spec out  or mock up “optimal” use cases — here’s how it’ll work with an average number of users, an average number of items, an average file size.  Too bad our users aren’t so average.

Instead, think about how this change/solution will work:

• When there is 0 activity
• When there are very few users
• When there are no messages/projects/photos/items to act upon
• When the display looks sparse
• When the files are enormous
• When there are 1000s of users
• When there are 1000s of messages
• When the display is overwhelmingly cluttered

If you’re lucky, you have an admin tool where you can emulate these situations.  If not, it’s worthwhile to take some extra time and mock up options.

And finally:  be prepared for complaints

The corollary of “if you’re not embarrassed by the MVP of your product, you waited too long to launch it” is “if your power users aren’t angry at you, you’re not going to succeed in the mainstream.”

Sorry everyone: I wish we could build that crazy feature too.  But we’re not going to.

Yes, you “know” that your power users are different (more vocal, more active, more tech-savvy, more exploratory) than your mainstream users.

But do you know HOW MUCH different they are?

Chances are, they’re not 10% more active.  They’re not 20% more expert.

It’s probably more like 100%.  200%.  500%.

That translates into realities like this:

• Mainstream users send an average of 0.8 messages per day.
• Power users send an average of 27 messages per day.

or

• Mainstream users have completed an average of 7 different tasks in your web app.
• Power users have completed an average of 492 different tasks in your web app.

or

• Mainstream users are following an average of 2 users.
• Power users are following an average of 1,100 users.

And then situations like this:

Power user: “I’m getting too many messages – it’s annoying and I feel overwhelmed.”

You (a person who is naturally an expert on your product AND a power user) identify with the customer.  You think, “hey, she’s right – I’m getting too many messages too!”

You make a change to your product to reduce the number of messages or display fewer messages.   Now you enjoy your product more.  The vocal power user enjoys your product more and tweets about how psyched they are about the fix.  But a month later, when you check your stats, you notice that usage is down.  Logins are down.  Activity is down.

How is this possible?  You listened to customer feedback!  You even focused on the problem rather than the solution — what went wrong?

Where you went wrong:

You listened to a non-representative user.

You applied a universal solution to a variable problem.

For your product to succeed, it needs to solve a problem for a lot of different people who will use it in a lot of different ways.  If this were easy, more people would be running billion-dollar companies.

Next week: how to think about power users vs. mainstream users.

Yes, you should start a blog.

• It’s the best way to brand yourself the way you want to be seen. Your job title may not currently be what you’re striving for. Even if it is, there are decisions and priorities beyond your control. With a blog, you have the control to write in-depth on the things you care most about and give people insight into how you think and tackle problems.
• It forces you to reflect on what you’re doing well and poorly. Most of us are super day-to-day busy, and it’s easy to be relieved that a task is just over, without thinking about why it worked or didn’t.
• Write for short attention spans. People skim blogs. They also skim everything else you write, dork. Bring some of this bullet-point, short-paragraph format back to your specs and one-pagers.
• Show consistency. When I see a blog with months or years of consistent entries, I feel a lot more confident that this person can shepherd through a product or long project.
• Random opportunities (eventually). After about a year of writing this blog, it helped me land a job. After about two years, I started meeting people at conferences who’d read and used my blog. Now (after about 4 years), I get about one speaking engagement or interview per month.

As you can see, my “No” list is much longer than my “Yes” list. Which should be sufficient proof that this is not a decision based on rationality.

It’s not hard to identify things that should not happen when people use the software we build:

• We should not lose customer data.
• We should not lose customer effort (i.e. half-written emails, file drafts).
• We should not share customer data in ways other than what the customer agreed was okay.
• We should not require customers to go through extensive learning to complete a task.
• We should not unexpectedly take away feature functionality or service levels.
• We should not allow customers to accidentally make decisions that will be irreversible and permanently impact their data/assets/experience.

You might think, if your software does not violate any of these guidelines, that you’re okay.

Unfortunately, it’s not enough to play by the letter of the law.  You will be judged by your customers’ perceptions of their experience.

For example…

Let’s say your web application’s main screen was starting to look very cluttered.  You’re worried that this will make it harder to use, so you decide that to build a feature that will archive all data that the customer hasn’t used in a month.  The data is still there — you’ve just hidden it behind a link-click. Great idea, right?  In alignment with principle #1 – you are not losing customer data.

You release — and customer A logs in to your app.  Things have changed… wait, where is my data from the last year?!  Customer A is freaked out.  He’s not sure he has that info backed up anywhere else.

Is he going to calmly and methodically check the screen for a link that says “Archived Data”?  No.  He’s going to panic, send customer support an anxious email, and send out angry Tweets.  Is he going to feel reassured when customer support tells him his data isn’t lost, it’s just behind the Archived Data link?  No.  Now he feels a little dumb, still frustrated, and residually angry at you.

Think of your “don’ts” as TWO requirements

1) Don’t do it.

2) Make sure your customers KNOW you aren’t doing it.

Some of these you probably already do:

• Asking for a customer’s email address?  Explicitly state that “we will never share your email address with third-parties or marketers.”
• Removing a feature but planning to let customers export their old data first?  Explicitly state that, in bold, with a clear link to export.
• Need customers to add more data?  Explicitly state how much they have to type and estimate how long it will take.
• Changing the UI to hide less-commonly-used items?  Explicitly add a “some features have moved, read more here” banner in the place where those features used to be.
• Allowing users to make an irreversible change?  Explicitly state the consequences AND add friction so that it’s harder for someone to make a mistake with one click*.

* When I worked on financial apps, “power user” types would frequently complain about the need to re-enter their routing number on a payment or transfer screen.  It’s a 9-digit number, and not one that anyone has memorized, so it IS a pain to type in.  It is ALSO a situation where an error can lead to very bad consequences (if you are transferring money to make sure your mortgage payment doesn’t bounce, and you enter the wrong routing number, the transfer will fail, your check will bounce…)  The two re-entry boxes are annoying; they are also a signal that “hey, you need to slow down, read these numbers carefully, and try hard to not make a mistake.”

This is all a very long-winded way of saying that you should think of customer-facing changes as Murphy’s Law.  If anything CAN be perceived poorly, IT WILL.  So get a second (or third) pair of eyes, and put on your pessimist hat, and try to catch as many of those bad-perception possibilities as you can — it will save you time and goodwill in the future.

You’re going to have to communicate those changes to your customers.

How you do this can make the difference between “a few angry Tweets” and “death threats from your community”

That last point may bear repeating — 9 times out of 10, it’s not what you changed that makes customers angry.  It’s how you communicated it.

You need to communicate these 5 things within 10 seconds / 1 screen of text:

• When the change is coming
• How this affects you (or “This does not affect you unless X”)
• What action you need to take (or “You do not need to do anything”)
• Why this decision was made (can be high-level/’spun’)
• You can complain here

Use bullet points or bold to facilitate scanning.  Resist the temptation to hide information.  If you’re taking something away, say so.  If you’re changing rules or charging, say so.

Here’s an example:

On Friday, February 10, WidgetCo will be switching to all-digital widget delivery.

What will change: Widgets will now only be delivered digitally, not by postal mail.

What you need to do:

• If you are currently subscribed to the Digital-Only plan, this change will have no effect on you.
• If you are currently subscribed to the Combination plan, you will receive your February widget shipment via postal mail.  Your February shipment was already charged.  Your credit card will not be charged again and you will not receive any additional widgets until you confirm that you wish to switch to digital delivery.  (You can confirm that here: [URL])
• If you are not sure which plan you are on, you can check by clicking this link: [URL]

Why we’re making the switch: Due to rising costs of postal delivery, WidgetCo is no longer able to provide the level of service that our customers demand without dramatically raising prices. In a survey of our customers, we discovered that our digital-delivery customers reported higher satisfaction rates — so we are confident that this move is the right one, that will help us to better serve our customers.

If you have any questions or comments, we’d love to hear your feedback: [EMAIL ADDRESS]

You shouldn’t use a survey if:

You aren’t sure which type of people you should ask to take your survey.

There are almost zero contexts where you’ll get useful data out of having “anyone” take your survey.   (And thinking, “I’ll just get thousands of responses, and then filter by some criteria later” is both a copout and unlikely.)   Do you want to hear from existing customers?  People from a certain region? People who share a common activity?  People with a specific job title?

Your “type of people” can be subjective, too — in a recent survey I conducted, I wanted to hear from was “smart people that we’d want to hire if we had the chance”.  So I distributed the survey through coworkers’ personal networks, letting them make that subjective determination.

You know which type of people you want to take your survey but have no idea how to find them.

Surveys are not an “if you build it, they will come” exercise.  Don’t waste your time on a survey if you don’t have a ready bank of people to send it to or a distribution strategy.   It’s a waste of time and social capital to send out a survey and get only 4 or 5 responses.

You’re better off conducting freeform interviews first so you can increase the “learning density” you get from each person.   For the long term, you’ll need to invest time in figuring out how to find more people.  This usually means “find where these people already are, and put yourself there” – participate in forums, join clubs, build up your network.

All of the questions you want to ask require a freeform response.

This is a sign that you don’t really know what the questions are yet, that you aren’t really sure what the problem is yet.

If you have an existing product or customer base, look at usage patterns and past feedback first.  Then start with freeform interviews, so you can tease out information in a back-and-forth context.  The survey format is bad for this type of learning.   People don’t like to write a lot, and even if they do, their first response is usually not where all the ‘meat’ is.

You don’t have a clear plan of action for how you’re going to use the results.

If you’re thinking “we might learn something” or “it would be nice to know…”, then save yourself the time (and save the time of the people who would fill out your survey).   Data without action is meaningless.

Is this information going to help you make a better decision on a specific feature or project?  Will it help you choose better wording or smarter defaults in your application?  Will it validate a specific hypothesis so that you can continue or pivot?

You can’t prioritize your list of questions down to fewer than 10.

This is another sign that ‘you aren’t sure what the problem is yet’.   Of course you have more than 10 things that you’d like to know — you probably have hundreds of things you’d like to learn — but you can’t possibly act on more than 10 at a time anyways.

I often see long surveys where the first 3-4 questions are asking about age, gender, zip code, income level.  Those are lazy questions.  If you need to segment by demographic, that should be part of your distribution plan or you should “buy an audience” from a market research firm.

You aren’t willing to write a draft, have another set of eyes review it, and then revise.

The first survey draft you write will suffer from at least one of these flaws:

• Unclear language (“wait, what are they asking?”)
• Biased language (“how much do you like feature X?”)
• Stilted language (technically makes sense but just sounds awkward)
• Stupid question (“do you want X?”  of COURSE they will say ‘yes’ – that doesn’t reveal anything about actual intent)
• Too many freeform questions (no more than 1 freeform for every 3 click-to-answer)
• Mistakenly using single-choice instead of multiple-choice, or vice-versa
• Doesn’t actually ask the question that you wanted answered
• Inadvertent rudeness (use of words with negative connotations, or a phrasing that sounds brusque or judgemental)

It is very difficult for you to catch these on your own.  I’ve been writing surveys for years, and I still always have at least one other person read it and comment on anything that is weird or confusing, and I still always have to change at least one thing.

So how can you apply customer development techniques to large enterprise customers and existing customers?    They’re still people, so fundamentally they also have problems and pain points and constraints.  But there are some things to keep in mind if you want to maximize how productive your conversations are.

Large enterprise customers

• Expect anything they see to look ‘good’. This doesn’t mean that you need to show a working app — trust me, big company folks are plenty accustomed to being sold products by PowerPoint deck — but it does mean that you should spend a few hours or days turning your Balsamiq wireframes into visuals that are simple, specific, and polished-looking.   Don’t use lorem ipsum, take the time to write the actual text that a customer might see.  Don’t use clip art.  Correct your typos.   If you don’t, you’ll immediately take a credibility hit.
• Need a memorable narrative. These folks are constantly being pitched.  All the features and benefits and overblown language like “best-of-breed” and “cutting-edge” blend together into a haze.   What they’ll remember is a story: “Let’s say you have an employee, Jill, and here’s what Jill does each day when she…”
• Are accustomed to hearing “yes”. Sales tends to say “yes” to everything in order to close a deal; this means that saying “no” may end the conversation abruptly.  (This doesn’t mean you have to agree, it does mean you have to be very creative in how you proceed when the customer is asking for something that you have no intention of providing.)
• Are often pleasantly surprised by being asked “why”. All enterprise customers have been burned by hearing “yes” and then finding out that that “yes” has a lot of exceptions or additional cost.  Having someone who actually listens, and asks thoughtful questions, may shock them into revealing a lot more about how their business works.
• Offended by being told what to do. A multimillion dollar widget-making company does not want to hear from you that they’re making widgets wrong.  If you try to sound credible by flaunting your widget-making expertise, that is unlikely to go well.   Be humble: acknowledge that they’re the expert, and you’re the one trying to understand.  (Note: this is always good advice.)
• Have a lot of stakeholders. The end-user is probably not the buyer.  The decision-maker is probably not the implementer.  You will need to talk to a lot more people in order to validate your assumptions.   You will also need to approach them in different ways (the way marketers perform due diligence or assess your credibility is usually very different from how the IT department does it.)

Existing customers

• Are really easy to get in touch with. There’s a perception that customers “don’t want to be bothered”, and I don’t know where this comes from.  This has never been my experience!  As long as you’re clear that you’re not trying to sell something, current customers are usually eager to talk to you.  Remember, they have already invested a lot of effort in learning your product – it’s in their best interest for you to thrive as a company.
• Their top priority is their current product/service. Have you ever been to a concert where the artist kept playing all the songs from his new album instead of the greatest hits that you wanted to sing along with?   Start by answering their existing questions, and make sure you learn as much as possible about how they’re using your current product or service.  Once you’ve covered that, they will be far more receptive to answering questions about potential new use cases or products.
• Hate change. Hold off on the mockups for as long as possible.  You want the customer to recognize that they have a problem first, before you threaten them with something new and different.
• Are biased by what they’re already using. They have strong opinions on the solutions they want, and are highly motivated to push those solutions instead of talking about the problems.  It will require a lot of conscious effort to keep directing the conversation back to why/how questions.
• Need reassurance that you’re not going to de-prioritize or drop their current product. If they think that answering your questions might cause you to stop working on your current product (remember, the one they have already invested effort in learning), it’s going to be a short and pointless conversation.  You can cut this off before it starts by immediately reassuring them that their current product is safe, that this is early/exploratory research.

Hope this is useful in encouraging you to extend customer development to some new audiences.  If there are additional concerns or pratfalls you’ve encountered, I’d be happy to address them in a followup post.