We’d managed to stay mostly poker faced about getting an interview this go around up until decision day, when there was a little optimism that crept in, evidenced by the fact that I thought it opportune to pick up a bottle of startup-priced champagne during the day’s shopping.

I’d been up most of the night working when the mail came at 7:00 a.m. Berlin time. We were naturally pretty excited.

If you’re going to San Francisco, be sure to wear some flowers in your hair:

We made some quick decisions: it was to be our first trip as a company to the valley. Accepted or otherwise, we wanted to get as plugged in as possible. We booked a week long trip and wanted to get our schedule as full as possible.

Connected to that we also took one of the first interview slots — 10:00 a.m., first day. YC interviews span three days (in our case Friday to Sunday) and you get an answer at the end of the day of interviews. Picking one on the first day meant we’d know 24 hours after landing if our primary objective was building our network or finding a place to live for the summer. We also wanted to get our pitch in before pitch-fatigue set in. If you’ve ever been to an event where you’ve heard startup after startup pitch, you know what I’m talking about. Strangely, to us, anyway, most of the initial sign-ups were clustered at the end of the third day. I can only presume that this was so that teams would have an extra bit of time to prepare.

For our week there, Valentin started making a list of companies to cold-call and try to set up meetings with and we got a handful — some with former YC companies and some not. They weren’t actively sales meetings — we just wanted to try to get a better handle on how recommendations could fit into the startup landscape. We figured meeting some YC companies would also give us an in to the YC mafia even if we weren’t accepted and we’d arranged to crash on the couches of various startups over the week.

Preparing for the interview, or “So you think you’re pretty smart, eh?”

We’d talked to investors in the past, so we thought we knew what was going to be important. I just found the stack of printouts that we prepared for the interview:

• Mails from prominent pilot customers (with praiseful sections highlighted)
• Feedback that we’d gotten on social media (Twitter, mostly)
• Profiling information showing the speed of our engine
• Full-color printouts of our demo-applications
• Dashboard showing total number of qualified leads we’d generated

We brainstormed a big list of questions that I can’t find anymore that we thought might come up and talked through answers to all of them. We came up with a list of points that we wanted to be sure to mention and even practiced transitions from other topics to those.

All of that, however, turned out to be useless.

About 14 hours after landing we were at the YC office waiting to get started. On the whole we were pretty relaxed, since we felt like we had our pitch pretty together.

The interview that’s not an interview:

The setup is the team on one side of a table, the YC partners on the other side. We walked in, I opened up my laptop and set it on the table and put the folder of printouts next to it. I figured we’d start off introducing ourselves and saying something about our backgrounds, you know, the usual.

Paul started, “So, you guys are making a recommendations engine. Do you have a demo?”

30 seconds in Paul, Trevor and Robert were standing behind us while I cycled through the various demos. It played out well, they’d ask if we could do X, and I’d pull out an example of us doing X. They asked a little about our algorithms, I remember Robert’s (only?) question being, “So where’s the rocket science here?” Questions were being asked more quickly than we could answer them and I remember at one point Trevor and Paul standing behind us asking each other questions and Jessica having to reel them in to point out that I was trying to show them something.

“Interview” is really the wrong word for what happens. You might expect something that’s in the genre of a job interview or an investor pitch, but it’s nothing like that. It’s more like a brainstorming session with the volume turned up to 11.

I don’t know if we hit any of the points we wanted to be sure to talk about. Certainly not most of them, and I remember that there were some notable ones missing — like that we already had pilot customers.

There were no questions about the business model, or size of the market, or distribution strategies or any of that jazz. That said, sometimes it does go that way. I’ve talked to a few teams about their interviews, and heard mixed things.

The hard thing about this rapid-fire sort of setup is that it’s really hard to give advice on what to do to prepare for a YC interview, but here are a few general things:

• Have a demo and know your way around it. From what I’ve heard, demo-centric interviews are the ones that go the best.
• Talk to as many people as possible about your startup and try to get them to ask as many questions as possible. Show them the demo and see what they think.
• Make your demo look nice. Paul especially digs that and the first impressions help.

So don’t expect to go in and present. That doesn’t happen as far as I can tell. I think all of the preparation we did though, including talking both online and in person with YC alums helped us be mentally prepared.

Post interview:

One quirky note: if you put in your home phone number in the YC application, that’s the number that you’ll get a call on if you’re accepted. We’d given Jessica cards with our cell-phone numbers on them during the interview, but that didn’t seem to have registered. We waited, and waited and waited. Finally around 11:00 p.m. I sent a mail to Paul and Jessica asking if decisions had come down. No answer.

The next morning we had a mail saying yes, they’d tried to call us. Valentin fumbled around with figuring out how to get a message off of our answering machine in Germany and once we’d sorted it out, we basically heard, “Yes, please call us back.” So we did. Of course, they were interviewing people then, so a while until the next break for them to call us back, we accepted the offer and that was that.

We found an apartment in the next couple of days via an apartment-wanted post on Craigslist and spent the rest of the week doing meetings, which turns out to have been a really good idea — and that’s the thought I’ll leave you with for this installment — optimize so that whatever you hear from YC that you’re on a positive trajectory. We’ll be back with the next chapter soon at the same bat-time and same bat-place.

“They’re making a search engine.”

A bunch of my friends think that. It happens every week or so that I’ll get introduced as “a search engine guy”. And maybe there exists a definition of “search engine” which includes recommendations. But when people think about Google Search vs. Amazon’s recommendations, the difference is between finding and discovering.

Search is about finding. You start with a topic you know exists and you want to find information about it. Recommendations are about discovering things you didn’t know about.

See, there we’ve got a bunch of info about Miles Davis. But that’s the thing — it’s all about Miles Davis. If you already know Miles Davis exists, search is a great way to find out more about him and his music, but it’s awkward for discovering things you don’t know about.  For comparison:

When looking things related to Miles Davis we get a list of the giants of jazz — most of whom played with Miles Davis at some point in their career. No prior knowledge of Charlie Parker is required in this context. If you know about Miles Davis and want to discover things which are like Miles Davis you need a recommendations engine.

The genesis of graph-based web search was in Jon Kleinberg’s seminal paper Authoritative Sources in a Hyperlinked Environment.  Kleinberg’s paper predated Brin and Page’s by a few months and was cited in the original PageRank paper.  From Kleinberg’s abstract:

The central issue we address within our framework is the distillation of broad search topics, through the discovery of “authoritative” information sources on such topics. We propose and test an algorithmic formulation of the notion of authority, based on the relationship between a set of relevant authoritative pages and the set of “hub pages” that join them together in the link structure.

Note the recurring keyword: authoritative.

It turns out that the difference between search and discovery is not just the presentational difference between them — it is also algorithmic. When finding related rather than authoritative source in a graph we massage the data in fairly different ways. In fact, it turns out that authoritative sources are often simply noise in a search for related items. Let’s examine this visually again.

Here we have a mocked up subgraph of Twitter — a few people that are following Barack Obama. When you start a search with Kleinberg’s algorithm (HITS), it begins by extracting a starting set of nodes based on a text search. Let’s imagine here that we’d searched for people mentioning Barack Obama and this was the set of nodes that were returned. Kleinberg’s algorithm attempts to determine the authoritative source in the set, and it’s pretty clear on visual inspection from this set that it’s the node called “Barack Obama”. The algorithm in the paper is naturally a bit more involved — it also incorporates the notion of “hubs”, but we’ll ignore those for now for simplicity. (Incidentally, Kleinberg’s paper is a rare combination of disruptive and accessible and well worth the time to read.)

Now if we were looking through that same subgraph and trying to find related users we’d need to use different logic. That someone is following Barack Obama says very little about them; certainly it doesn’t go far in determining what they’re likely to be interested in. If we were recommending a friend for Matt to follow, visually it’s clear that Jim would be a better recommendation than Bob.

As it turns out, Barack Obama, the “authoritative” node in this graph is in fact just noise when trying to deliver a set of recommendations and it’s best if we ignore it altogether.

Again, as visually confirmed, removing the “authoritative source” from the subgraph makes finding related users for e.g. Matt or Dave much easier.

This problem surfaces all of the time in recommender systems. If we were applying it to finding related artists to Miles Davis, it would be that the terms “jazz” or “music” are far too often linked to Miles Davis and his ilk. On Twitter’s graph it’s people with so many followers that following them says little about a person. In a book store it’s that having bought Harry Potter says little about one’s more specific tastes.

In the early days of Directed Edge, we called this the “tell me something I don’t know” problem. That is, after all, what recommender systems are for. If you recognize all of the results in a set of personalized recommendations, they’re not doing their job of helping you discover things. If something in a set of search results seems unrecognizable, it’s probably just a bad result.

Today Facebook moved from displaying live streams of friend updates to “news” by default. There’s more to this than meets the eye at first. What’s actually happened is Facebook has placed recommendations front and center on the third most popular site on the web.

This is big stuff.

Now instead of showing every update by default — Facebook is picking the things you’re likely to be interested in based on feedback from your friends:  recommendations, essentially.

One of the things that first got us excited about the recommendations space was seeing the merging of the social graph and traditional recommendations applications on the horizon. Friend finders and the like are the most obvious (and boring) applications of recommendations applied to the social graph, but the potential goes far beyond that.

Information overload has always been the driver of major innovation on the web. Social applications and information overload have been on a crash course for a while.  The intersection where they collide says much about the shape of things to come. In these days where followers / friends / fans are quickly outpacing the ability for one to consume the “real time stream” as such, it’s interesting to think of how you get out of that conundrum.

So, let’s step back a little bit and talk about recommendations.  I wrote an introduction to recommendations a while back that talked some about traditional recommendations based on things like purchase histories, ratings and whatnot and compared that to graph-based recommendations.  In the graph-based example I mentioned the possibility of using friends within a social network to drive the recommendations.

The thing is that these data sets need not be separate.  The social graph isn’t just friends; it’s a broad model for interactions between people and content on the web — there’s no reason not to consider products and news articles as elements of the social graph, and once they’re in that grand unified model of interaction on the web to harvest that data to figure out what people are likely to be interested in.

But that’s not just possible — it’s absolutely necessary.  Recommendations present pretty much the only way forward for handling the explosion of real-time data on the web.

So, let’s start at the beginning:  Applying to YC.

This story starts a year earlier than one would assume.  Summer of 2009 wasn’t the first time that we’d applied to Y Combinator.

Valentin and I decided to start a startup together and gave notice at our jobs in March of 2008.  I’d been tinkering in the area of graph-based information organization for about 4 years already and had built a quick proof-of-concept of a recommender system based on some of those ideas.

Since I’d been reading Paul Graham’s essays since before the dawn of Y Combinator, applying seemed a natural step.  We labored over the application — finishing it a full month before the deadline; we had friends proof-read it; we went through several iterations.

I mean, we had to get accepted.  I’d previously been a fairly well-known open source developer, had been an invited speaker all over the world, had spent several years in R&D at Europe’s largest software company, I knew the problem space well and we were convinced that we’d just seen the tip of the iceberg in the shape of things to come in content discovery.  Valentin had studied business and communications and worked in the web world for over 10 years and been the webmaster for some huge German sites.  What could possibly go wrong?

We didn’t even get invited for an interview.

The really interesting part is what happened next.  I started coding furiously.  Paul Graham could go to hell.  I’d show him.

That was our first taste of rejection.  It’s a taste you get used to when starting a startup.  You have to learn to thrive on it.  We were starting a startup, dammit, and nobody telling us “no” was going to change that.

We’d already been going to some of the local founders meetups which were a great chance to get a taste for the scene.  About a month after getting rejected from YC we got invited to the mini-Seedcamp in Berlin and met some of the people that continue to advise us to this day.  We found our first pilot customer the next month after that — who subsequently died before they could go live — but it was the validation that “somebody wants our stuff” that was really important.  By three months after YC we’d had an investor approach us for the first time.  We pulled in more pilot customers, kept building out our demos, got our first press.

Even though I was certain that we were going to get into YC, I’d set aside enough money to live for a year on to get Directed Edge up and going.  That was essential.

There are thousands of details that could be dropped in there — I mean, it was a year in the life of an early-phase startup.  It was grueling.

We learned enormous amounts.  The money that I’d set aside was the best spent money on education in my life.  About midway through the year, Oliver Beste, one of the local startup luminaries, said something to us that’s stuck with me:

“It’s unclear in your first startup what the worth of your company will be, but the worth of your person is guaranteed to increase.”

The year flew by.  We’d thought we wouldn’t apply to Y Combinator again, but at that point we knew a number of Y Combinator alumni that encouraged us to give it another swing.  Like every point in that year, we were crazy busy.

Unlike the first time around, we did our application in a couple of hours, scarcely proof-read it, did a quick video and sent it off.  I mean, we already knew they didn’t like our idea, so it was just something we knocked out and moved on.

And of course, that time around, we got an interview, and were eventually accepted to Y Combinator.

What was the difference?  Team was the same.  Idea was basically the same.  I’m sure the application was better — I mean, we’d pitched 1000 times at that point, so it was a lot easier to just ad lib in talking about our company.  Plus we had a better demo and pilot customers.

Paul later told me that the big difference was that he recognized me from Hacker News and that I said smart things there.  So if that was the critical difference, the question is, “Why was I saying smart things?”

When we started with Directed Edge we were as ignorant about what starting a startup was about as the next set of fresh-off-the-employed-world-boat founders.  Maybe worse.

It’d be flattering to think that maybe it was sheer intelligence that got us over the hump, but, uhh, no.  It was that we kept going.  You’d have to try really hard to run your first startup full-time for a year and not learn a huge amount.

So, now I’ll finally get around to the point of this entry:  People now often ask me what they should do on their YC application to increase their chances of getting in. There are the usual things like concision, clarity of thought, quality of idea — those all help certainly.  But I’m convinced that the single most important thing that you can do to increase your chances of getting into Y-Combinator is to do what you should be doing anyway:  going full-speed ahead on your startup. That’s what teaches you about your company, your market, the real composition of your team.

It’s easy to look at some of the teams coming out of Y Combinator and assume that Y Combinator has been some great leveler making quality startups out of the raw founders, but the truth is that the teams that were moving the fastest at the end were the ones that were moving the fastest at the beginning (with a couple exceptions).

The term “accelerator” has been applied to the swath of Y Combinator-esque programs out there.  I never really liked that term, but now I realize that it is in fact descriptive:  Y Combinator operates on the first derivative of your speed — it doesn’t carry everyone a uniform distance; it speeds teams up.

In that sense, we’re lucky that we didn’t get into YC when we first applied.  We’d not have gotten nearly as much out of it.

And contrary to a bit of the mythology that surrounds YC, a pretty big chunk of the teams had working products developed on day one.  They weren’t teams that had decided to apply to YC — they had decided to start startups, and YC was a step along that path.

And that’s the second big point:  Decide you want to start a startup, not that you want to be in Y Combinator.

The single biggest fear that Y-Combinator has for the teams it accepts is that they’ll give up.  So if there’s something that you can have on your application that shows that you won’t I think that’s probably a lot more important than how you frame your idea.

So go build your company.

One final note:  Most of the people reading this and planning on applying to YC will get rejected.  Even if you’ve got a great company, they’re evaluating several hundred teams in a tiny span of time, so just bad luck might be enough for you to not get an interview.  I’d advise you to take on the “To hell with Paul Graham” attitude I mentioned.  He won’t hold it against you.  I told him about it and he said he’d probably have reacted the same way.  Rejection is part of starting a startup.  Just in the last two weeks I’ve seen companies on TechCrunch that interviewed with our batch and were rejected.

Good luck, and stay tuned for the next installment.

Soon we’ll get around to giving some more love to the developer site based on feedback we’ve been collecting in the last few weeks and round out the section of tutorials to cover all of the languages that we currently have bindings for.

Here are the total counts — the results were a little surprising:

• Media – 15
• Service – 12
• Search – 10
• Adult – 10
• Content – 7
• Blogging – 6
• Social – 6
• Images – 5
• Gaming – 4
• E-Commerce – 3
• Brick & Mortar Store – 3
• Advertising – 3
• File-sharing – 3
• Video – 2
• Auctions – 2
• Hardware – 2
• Maps – 2
• Software – 2
• Jobs – 1
• Music – 1
• Weather – 1

The top-100 list came from Alexa, and some of the categories are a wee  bit ad-hoc, but it does give an interesting idea of what the break-downs are.  I wouldn’t have guessed, for example, that adult sites beat e-commerce sites by a factor of more than 3-to-1 in the US top 100.

The full data is here in a PDF.  Holler if you want the spreadsheet version to bang on.

• TechCrunch: Silicon Valley Elite Flock To Y Combinator Demo Day
• Venture Beat: The YCombinator list: Bump, Mixpanel, JobPic take off with newest class

Today we were last in rotation, which means showing up last in the live blogging as well.  On the plus side, tomorrow, with more investor-like folks around, we’ve got the first slot of the day.

On Finding Reinforcements

I asked around. There were two suggestions that came my way multiple time: 99designs and David Pache.  David’s portfolio is stunningly good, so the decision wasn’t hard, and personally I’ve always been a bit squeamish about 99designs:  lots of people competing for a chance to get one sub-market price.  It just doesn’t seem like a system that would be biased towards high quality work.  Certainly the best designers would avoid it like the plague.

But there’s another more subtle reason that I’m glad we went with a great logo designer:  they’re better than me at picking quality logos.  If I were deciding from 20 logo designs on 99designs, I’d have been hard pressed to pick the best one.  In fact, David sent us 4 mockups, and we didn’t know which one to pick.  I’m glad this one was his favorite as we’ve since warmed to it.  We were using the current one as a stand-in when a bit of serendipity made things more interesting.

The Other Whiteboard Artifact

Yesterday, Paul Graham posted “A Whiteboard Artifact” about us mapping out some of our company is about — but that wasn’t the whole whiteboard.  When he was drawing out how to explain what we do, here’s what he drew on the right half of the board:

Notice anything?  Yeah, we did too.  Here’s our new logo in full-ginormus-glory:

Previously, amusingly, we weren’t sure if the new logo quite captured what we do.  Having an advisor duplicate it on the whiteboard, never having seen it — or even knowing that we were evaluating logos — once again made the decision easy.

And that’s the story of our new logo.  For a little more visual goodness, check out the corporate identity page that David put together, including some insight on the iterations in the design process.  We’re glad to have worked with him.

Amazon, of course does product comparisons, but there’s no reason recommendations shouldn’t be a part of news consumption, music consumption, social networking, basically everything we do on the web. And while there are no shortage of companies out there that focus on some of these different fields specifically, Directed Edge has developed a system that can be plugged into all kinds of different sites.

There will be some more technical news that we’ll drop in in the next couple of days — there are a couple of features that we’ve been chomping at the bit to get out to users, plus we’ll be putting together even more content for our developer site.

We’ve also, for the first time in a few months, updated the dump of wikipedia that our related pages are served from, which are also served up through our wikipedia related-pages API.