Why Research?

Entrepreneurs regularly receive contradictory advice. And some is downright false advice. It’s a complicated problem, but the key to changing this is providing evidence (data) to back our opinions and opening ourselves to scrutiny.

In addition, entrepreneurship research over the last three decades has been focused on economic policy. Perhaps it’s because that’s what grant funding will support, but this is of little use to founders, investors, or startup employees. We’re going to change this; it’s not difficult to design research, it’s just takes time and conscious thought. Together we can do it.

Our Focus

Our content will be crowd-sourced from the entrepreneurship community. You can pitch us your post or idea at anytime using the link at the top of our blog “Pitch Us Your Post.” Our writers will give you direct feedback and help with both writing and editing if your post is selected.

The types of posts we’re focused on fall under three specific categories.

1. Examples with Real Data

The main focus of our blog will be to publish case studies and in-depth examples backed by data. Everything will be audited. Our editors will do the diligence to make sure that what is published is authentic, high quality, and actionable.

2. Controversial Opinion Pieces

We will post evidence-backed arguments that are in direct contradiction to opinions of established thought leaders. Including investors, entrepreneurs, academics, economists, and public officials.

3. Controlled Research

Perhaps what I’m most excited about is conducting controlled studies on entrepreneurial management. There’s so much potential in this space that is yet to be uncovered.

I’m excited to present this case study on Chef’s Table, the winner of Lean Startup Machine in San Francisco for Dec. 2012.

This is Mark Abramson, the team leader who went through 5 pivots before getting to his final idea which attracted $4,500 in revenue in under 90 minutes.

18x Growth in 2 Years

We’re just at the beginning of the hockey-stick curve. With Harvard Business School, the United State White House, and General Electric all advocating for Lean Startup, the movement is trending towards something incredible.

Today is the 3rd annual Lean Startup Conference run by Eric Ries & Neo. New management methodologies usually mature by the 5th year, so we have at least 2 years of accelerated growth left. This means the movement will be four to ten times it’s current size and adoption.

How to Avoid Becoming a Fad

Many people wonder if Lean Startup is just another business fad. In order for it to stick, the concepts need to produce measurable results. Otherwise the growth and adoption we see now will be followed by rejection and decline.

At the core, Lean Startup is proposing a management science for creating new products. The opportunity here is best expressed by the fact that Apple gets 50% of its revenue from products created in the last three years. Furthermore, Apple spends less than 1/4 what Nokia does on Research & Development. There’s no correlation between spending and success; it’s the underlying process that matters.

The first risk is maintaining an open ecosystem where new entrants can learn to apply the techniques correctly. It takes a lot of work to do these processes correctly, and even more work to change the internal operations of a big organization so that they can work.

Next, the ecosystem needs to be sustainable. It needs to be a level playing field where everyone can compete and big companies can’t just buy their way to the top. It also helps to have success stories. IE. we all took notice when we heard about developers making a million dollars in a week from an iPhone App and nine-year-olds teaching themselves to program on the iPhone.

The second risk is in companies that try and fail to implement Lean Startup processes. As consultants, trainers, and internal champions, we need to pick partners that aren’t looking for a quick fix. We’ve been lucky to work with TLC Labs, who in under a year has used Lean Startup methodologies to incubate three new business, with one already at a $1M+ annual run rate. This is because their executive team made significant efforts and sacrifices.

The Physics Argument (Update)

At the Lean Startup Conference I had the opportunity to get feedback from many people who were part of the Agile movement on the risks of Lean Startup becoming a fad. Ryan Martens, Founder of Rally, mentioned how both Yahoo! and MySpace tried and failed to adopt Agile practices. While it hurt the growth of the movement, we know now that it just delayed the inevitable. What was key was that other companies in the ecosystem were successful at adopting the practices.

The first thing you should know is that there are Three Methods of Customer Validation that increase in the cost of execution. None of them involve writing code, which is even more expensive to execute.

We developed these methods at Lean Startup Machine by observing techniques from hundreds of startup teams. Here’s how to do them so you get real results.

Required Reading:

10 Things I’ve Learned by Cindy Alvarez

1. Exploration Method

Exploration is based on ethnographic research techniques. The goal of exploration is to collect a lot of data on potential problem sets within a given industry or customer type, and to collect detailed stories on customer behavior.

Techniques (decrease in cost and quality of data):

• Observation: seeing the problem actually happen
• Re-enactment: attempting to simulate or recreate the problem
• *Interviews: listening to the customer tell a story about the problem

Required Reading:

12 Tips for Customer Development Interviews by Giff Constable

5 Bonus Tips for Customer Development Interviews:

• Always qualify the person you’re talking to and make sure it fits your customer hypothesis
• The best question to start is always, “Tell me the story of the time you did…”
• Never ask a question that starts with the word “would,” ie. “Would you do X?” or “Would you pay for this?” These are hypothetical questions that produce inaccurate data
• Avoid Confirmation Bias by asking your team, “Could anyone disagree with this question?” The answer must always be yes.
• Avoid Observer Bias by with-holding the facts of what you’re testing or building (otherwise customers will reverse engineer the questions). Even better is if customers don’t even know it’s an interview.

Bad Questions

• “Do you like [something everyone likes, ie music]?”
• “Do you have a problem with [insert too broad or too obvious problem]?”
• “Do you want more time and money?”
• “Do you wish you could have a [insert something awesome like a Porsche]?”
• Anything that has to do with your solution

Good Questions

• “Tell me about the last time you…”
• “How are you currently solving [insert problem]?”
• “How/where did you search for a solution?”
• “If you could wave a magic wand and have anything you wanted, what would it look like?”

Recording Data & Scoring:

WARNING: If you don’t record and score your data it is as good as useless. You need to have a record for your team to access at any time when making a critical pivot. I recommend using Google Docs and sharing with everyone on the team. Unstructured data (notes) should be converted to a spreadsheet with structured data. My method is to list the different problem areas and rate the intensity of the problem from each customer’s story from 0-3 stars. I also like Ben Yoskovitz’s method for scoring problem interviews.

Avoid a false negative/positive:

Build rapport with your customers and dig deep into their story and emotion. Keep asking why. Look for inconsistencies incase you’re not being told the truth. Exclude any bad data from your sample.

2. Pitch Method

The Pitch method is about getting customers to give you a form of currency. This involves “pitching” them a specific solution (idea or low fidelity mock-up) or benefit (what without the how) before the product has been built. This is also known as smoke testing, a practice used in retail and direct marketing for years.

Forms of Pitch

• Landing Page
• Cold Email
• Cold Call
• Gimmick Sign
• Fake Ad
• Walking into a store and pitching the owner
• Kickstarter

Forms of Currency

• Cash $$$
• Letter of Intent
• Email Addresses
• Pay With a Tweet
• Taking a Meeting
• Time
• Building something on your API

What people say and what they do is usually different, so collecting currency actually proves intent. By separating a customer from their wallet you know that they are willing to change behavior and/or pay for a solution.

3. Concierge Method

Concierge is delivering your product through a physical service, hand-holding the customer. This method has also been referred to as Wizard of Oz, ie. faking the product with as little technology as possible. (In the movie the real Wizard is just a guy behind a curtain, pressing a button to turn on the flames.)

Concierge Tools

• Email
• Physical Labor
• Spreadsheets
• Open Source Software
• Cell Phone

The goal of concierge is to make a small set of customers ecstatic with the proposed product before building it and marketing to others. Food on the Table scaled from 1 to 200,000 users this way. The first 20 customers they added one at a time, using email and a clipboard to deliver the value prop of the product.

We’ve seen teams validate in the Pitch stage who skip Concierge and end up building the wrong thing. This is because doing Concierge is a critical step in understanding the “how” of a customer’s problem.

1. To Make Entrepreneurship Scientific

Entrepreneurship is the means to solve this century’s greatest problems, yet we know little about how to quantify or teach it. Lean Startup created a language for the process of entrepreneurship. Terms like Minimum Viable Product and Pivot paved the way for entrepreneurs to debate and explain patterns in entrepreneurship.

Now, to go from a language to a science, the process of entrepreneurship needs a common framework of documentation and story-telling. The Validation Board is the missing piece of this puzzle. It’s a way to compare the actual process of two or more startups head-to-head, and it’s the beginning of making entrepreneurship measurable.

2. To Make it Easier to Pivot

Pivot is an overused word, but an underused practice. In a startup, there are only two activities involved in searching for a business model: pivoting and validating. Every product’s reality is determined before it’s built, it’ll either be a huge success or it won’t.

A pivot is defined as a change in strategy without a change in vision. There are big pivots, like Burbn’s pivot to Instagram, and small pivots, which could be just redefining the customer. Many entrepreneurs ignore their pivots, because they equate the term to failure. In reality, entrepreneurs are missing the opportunity to internalize and communicate to the team what they have learned.

The Validation Board gives teams a clear signal when their current path is uphill and they should leverage their learnings to pivot into a better opportunity.

3. To Make Startups Test-Driven*

I dislike the term “metrics-driven” because people confuse metrics for being all about the numbers. In fact, metrics are all about testing hypotheses and assumptions.

Too many startups track metrics without going through a process to test their assumptions. Just tracking metrics adds no accountability and it doesn’t help you learn. As Eric Ries loves to say, “When you ship a product to see what happens, you are guaranteed to succeed—at seeing what happens.”

On the other hand, documenting your assumptions ahead of time makes it impossible to ignore reality. I’ve seen this with hundreds of startups, and I even know this to be the case from personal experience.

In my scooter business, I tracked early sales as a metric. I had one sale, and I kept the business going for six months trying to increase that number to no avail. Cold calling dealers, posting flyers all over town, posting on forums, etc.

When I restarted the business and focused on my assumptions of why the scooters would sell, I spent just twenty-four hours interviewing 20 customers. I found out that 0 in 20 cared about the environment (my Riskiest Assumption), and it was as clear as day I needed to pivot.

Using the Validation Board will force you to test your assumptions, and you will build better products because of it. Don’t wait another minute, download the Validation Board here.