Jeff Miller's Blog

Algorithmic Trading is Not High Frequency Trading

2011-09-13T00:00:00-07:00

It’s not every day I come across a widely-shared article whose very first sentence is factually incorrect, but a 9/9/11 story in Computerworld UK, Algorithmic stock trading rapidly replacing humans, warns government paper, is just such a creature.

“Algorithmic trading, also known as high frequency trading (HFT), is rapidly replacing human decision making, according to a government panel…”

Problems

“Algorithmic trading, also known as high frequency trading (HFT)” — no, algorithmic trading and HFT are not the same.
“is rapidly replacing human decision making” — no, algorithmic trading is not replacing human decision making, and never will.
“according to a government panel” — self explanatory.

The Difference Between Algorithmic Trading and HFT

Algorithmic trading is like the GPS navigation system in your car: you tell the car your destination, and the GPS picks the route. The car generally doesn’t drive for you, and it doesn’t replace your decision of where to drive. Similarly, traders employing algorithmic trading systems pre-decide which stocks to buy or sell; they’re simply handing over the actual execution of the orders (when to trade during the day, and at what price) to the algorithm.

High frequency trading is a different game. True, an HFT algorithm actually does choose which stocks to buy and sell, but the program is not making decisions on the same scale as a pro fund manager. A fund manager (what bombastic writers would refer to as a “human”) says “I’m bullish on Amazon.com this year, so I’ll buy $100M of AMZN over the next week and hold the position until my view changes.” A high frequency trading system says “I estimate a high probability of AMZN going up within the next 30 seconds, so I’ll buy $10,000 worth” (and sell it 30 seconds later).

In short, algorithmic trading is used to implement large scale, economic views on asset prices, whereas HFT systems place automated, small scale, probabilistic bets on structural inefficiencies.

Algorithmic trading does not replace human decision making, and it never will.

HFT is a form of computerized decision making, but it doesn’t replace humans because humans don’t make decisions on millisecond scale to begin with.

Richard Bookstaber wrote an in-depth article two years ago about the difference between algorithmic trading and HFT, Risk from High Frequency and Algorithmic Trading Not as Big as Many Think.

Scary HFT

Ever since high frequency trading came onto the scene around 2007, there has been an unfortunate trend in the press of using HFT as a platform for linkbait stories. It’s fun to make computerized trading sound spooky, and it gets pageviews.

Common signs of HFT article linkbait:

Referring to traders/people as “humans”.
Calling computer programs “robots”, and saying they can “adapt” or will become “intelligent”.
Citing the pseudo-statistic “X % of trading is now computer driven”. So? Eventually, 99% of all trading will be computer driven, in the sense of orders being executed by computers. But “humans” will still be making the underlying, economic decisions.
Fake omens, like people being “replaced”.
Jokey headlines like “Rise of the machines”.
Pictures of the Terminator.

If you retweet or share such an article, congratulations. You’ve been linkbaited.

Addendum

9/13/11 5:00pm: noted that the author of the Computerworld UK source article modified the first sentence to read, “Algorithmic trading, including high frequency trading (HFT)”.

» Photo credit: Max Sang / Flickr

Baby Hacks

2011-05-06T00:00:00-07:00

Lesser known technology hacks pertaining to life with a baby or toddler:

OhLife

OhLife is an online journal for your inbox. How it works is simplicity itself. Every day OhLife sends you an email. You reply to that email with your thoughts for the day. That’s it. The site keeps a record of everything you write and you can go back at any time and read past entries. All of this makes OhLife perfectly suited to functioning as a low-maintenance baby journal.

I have OhLife configured to email me each night around the time my daughter has gone to bed. I make it a ritual to spend five minutes typing quick notes about what activities she did that day, new words she’s learned, and anything else I want to remind my future self about. Even if I’m tired and I just write one sentence, I do it. I don’t bother polishing the grammar or style because that doesn’t matter in a private setting. As with photo sharing, low friction is the key.

Every parent wants to capture in-the-moment thoughts about the adventure of raising a child, but the time to sit down and handwrite witty observations about one’s baby often never presents itself. The bar is too high. OhLife makes it do-able. Just remember not to break the chain.

Picasa Dropbox

If you’re a new parent, forget about putting together snazzy vacation Flickr albums (you won’t be taking any vacations for a while) or posting artistic photos of your beer with Instagram (you’ll be too focused on the drinking). Your highest purpose is now keeping your family and friends supplied with a continuous and never-ending stream of baby pictures.

Over time I’ve learned how to make this super easy:

I keep one monolithic “dropbox” album of baby photos on Google Picasa Web Albums.
I’m always snapping pictures of my daughter with my cell phone, an Android Nexus One. On average I probably capture two or three photos every day (sort of like DailyBooth, except I think the results are more interesting when the subject is someone other than one’s self).
Good photos are uploaded straight into the dropbox album. The Nexus One has a built-in uploader for Picasaweb, so this happens right away with no cables involved.
I never waste time composing clever captions or color-correcting the photos. People generally only care about one thing: the baby in the picture. Everything else is overlooked. If a photo isn’t any good, I delete it and take another.
The URL of the dropbox album is shared with my entire family. Then they just check it on their own schedule, whenever they want a fix. (Privacy note: Picasa can have URLs that are shareable but not visible by the public.)

The beauty of this setup is that it’s low-friction. Snap a photo on my cell phone; click Share; click Upload; done. Fast. Easy. Everyone’s happy. I no longer have to email out photos, hunt down JPEG files to attach, and so on. On the receiving end, friends and family don’t have to bug us to post new pictures. The entire problem is changed from push to pull.

I’ve also found that simplifying the sharing process encourages taking more photos, especially in normal day-to-day situations, which is really fun when kids are the subject.

By the way, this setup has nothing to do with Dropbox the file sync tool. “Dropbox” is simply the default album name chosen by my cell phone, and using the default choice means one fewer step in the process.

If your cell phone doesn’t have a built-in uploader for Google Picasa (like the Android Nexus One does) then unfortunately you might need to add extra steps to the process.

YouTube Playlists

For toddlers (probably not so much for infants) there always comes a time when you need to bust out a video or two to keep the kid occupied and inert while you handle some other critical task. In such situations, YouTube is indispensable.

Of course, YouTube itself is no revelation when it comes to kids; it’s about as secret as Skype and iPad games. Playlists, however, seem to be a little-known feature of YouTube. Most people use YouTube in surf mode—click on one video, and once it’s finished, click on the next—but a playlist allows you to queue up multiple videos and play them all automatically in sequence. Best of all, playlists are saved to your account, so you can rerun the same ones over and over again. Toddlers don’t mind this. In fact, experience shows that they love it.

Bottom line, YouTube Playlists are clutch like Robert Horry. They keep a toddler’s attention riveted and their hands off the keyboard. This is simply huge when you’re desparate for an extra 30 minutes of hands-free time during a busy day.

Check out the complete guide to YouTube Playlists.

Twitter Users Most Followed by Investors on AngelList

2011-03-01T00:00:00-08:00

Updated December 29, 2011

Here is a list of Twitter accounts heavily followed by the angel investor & VC community.

To create this list, I used the same methodology as an earlier post (Twitter Users Most Followed by Readers of Hacker News) but remixed with different data. This time I’ve looked at investors on AngelList instead of users on Hacker News.

Many thanks to the fine fellows at AngelList for making their data available.

By the way, if you’re an angel or VC investor with an account on AngelList, check out AngelFlow. AngelFlow is a social dashboard that makes it easy to connect your Twitter graph with your AngelList graph.

This list is available as a CSV file.

Twitter Users Sorted by % of All AngelList Angels Following

Rank	Twitter	Name	AngelList	% following
1.	@fredwilson	Fred Wilson	fredwilson	63.6%
2.	@TechCrunch	TechCrunch		52.9%
3.	@davemcclure	Dave McClure	davemcclure	50.3%
4.	@bfeld	Brad Feld		43.7%
5.	@msuster	Mark Suster		43.6%
6.	@joshk	Josh Kopelman	joshk	42.9%
7.	@sacca	Chris Sacca	sacca	42.6%
8.	@cdixon	chris dixon	cdixon	42.3%
9.	@venturehacks	Venture Hacks	nivi	41.7%
10.	@quixotic	Reid Hoffman	reid-hoffman	41.6%
11.	@om	Om Malik		40.5%
12.	@arrington	Michael Arrington	michael-arrington	39.3%
13.	@angellist	AngelList		38.2%
14.	@timoreilly	Tim O’Reilly		37.5%
15.	@paulg	Paul Graham		37.1%
16.	@jeff	Jeff Clavier	jeff	35.7%
17.	@ev	Evan Williams	ev	35.3%
18.	@BillGates	Bill Gates		35.1%
19.	@vkhosla	Vinod Khosla		34.7%
20.	@Scobleizer	Robert Scoble		34.4%
21.	@bgurley	Bill Gurley		34.3%
22.	@ericschmidt	Eric Schmidt		33.6%
23.	@jack	Jack Dorsey		33.5%
24.	@Jason	Jason Calacanis		33.1%
25.	@karaswisher	Kara Swisher		31.7%
26.	@naval	Naval Ravikant	naval	31.7%
27.	@mkapor	Mitch Kapor	mkapor	31.2%
28.	@davidhornik	David Hornik	davidhornik	30.2%
29.	@marissamayer	marissamayer	marissa-mayer	28.3%
30.	@johndoerr	John Doerr		27.7%
31.	@kevinrose	Kevin Rose	kevin	27.4%
32.	@biz	Biz Stone		26.9%
33.	@ericries	Eric Ries	ericries	26.7%
34.	@mashable	Pete Cashmore		26.7%
35.	@Caterina	Caterina Fake		26.4%
36.	@bijan	Bijan Sabet	bijan	26.0%
37.	@ycombinator	Y Combinator		25.9%
38.	@waltmossberg	Walt Mossberg		25.3%
39.	@rabois	Keith Rabois		25.1%
40.	@johnbattelle	John Battelle		24.7%
41.	@dickc	dick costolo		24.6%
42.	@bhorowitz	bhorowitz		24.6%
43.	@asenkut	asenkut	asenkut	24.4%
44.	@davemorin	Dave Morin	davemorin	24.1%
45.	@hnshah	Hiten Shah	hnshah	24.0%
46.	@SteveCase	Steve Case		24.0%
47.	@m2jr	Mike Maples		23.7%
48.	@pkedrosky	Paul Kedrosky	pkedrosky	23.4%
49.	@sgblank	steve blank	sgblank	23.3%
50.	@BarackObama	Barack Obama		23.2%
51.	@VentureBeat	VentureBeat		22.9%
52.	@Bill_Gross	Bill Gross		22.7%
53.	@Sequoia_Capital	Sequoia Capital		22.2%
54.	@mlevchin	Max Levchin	mlevchin	22.0%
55.	@TEDchris	Chris Anderson		21.7%
56.	@markpinc	mark pincus		21.6%
57.	@SAI	SAI		21.2%
58.	@erickschonfeld	Erick Schonfeld	erick-schonfeld	21.0%
59.	@zappos	Zappos.com CEO -Tony		20.7%
60.	@shervin	Shervin Pishevar	shervin	20.6%
61.	@hblodget	Henry Blodget		20.4%
62.	@pierre	Pierre Omidyar		20.2%
63.	@newsycombinator	news.yc Popular		19.9%
64.	@Borthwick	John Borthwick		19.8%
65.	@edyson	Esther Dyson	esther-dyson	19.6%
66.	@tferriss	Tim Ferriss		19.6%
67.	@bryce	Bryce Roberts		19.6%
68.	@firstround	First Round Capital		19.3%
69.	@sarahcuda	Sarah Lacy		19.2%
70.	@andrewchen	Andrew Chen	andrewchen	19.1%
71.	@bussgang	Jeff Bussgang	jeff-bussgang	18.5%
72.	@danprimack	danprimack		18.4%
73.	@peterfenton	Peter Fenton		18.4%
74.	@Chad_Hurley	Chad Hurley		18.4%
75.	@robhayes	Rob Hayes	robhayes	18.3%
76.	@joshu	joshua schachter	joshu	18.2%
77.	@dens	Dennis Crowley		18.1%
78.	@richardbranson	richardbranson		18.0%
79.	@daslee	David Lee	daslee	17.9%
80.	@parislemon	MG Siegler		17.8%
81.	@cshirky	Clay Shirky		17.8%
82.	@loic	Loic Le Meur	loic	17.7%
83.	@infoarbitrage	Roger Ehrenberg	infoarbitrage	17.7%
84.	@dharmesh	Dharmesh Shah	dharmesh	17.7%
85.	@mattcohler	Matt Cohler	mattcohler	17.6%
86.	@500startups	500 Startups	500startups	17.5%
87.	@mcuban	Mark Cuban		17.4%
88.	@ceonyc	Charlie O’Donnell	ceonyc	17.4%
89.	@tonysphere	Tony Conrad	tonysphere	17.1%
90.	@google	A Googler		17.1%
91.	@shitmydadsays	Justin		16.8%
92.	@VCMike	Mike Hirshland		16.8%
93.	@davidcohen	David Cohen		16.7%
94.	@howardlindzon	howardlindzon	howardlindzon	16.5%
95.	@RWW	ReadWriteWeb		16.5%
96.	@peHUB	pehub		16.0%
97.	@RonConway	Ron Conway		15.7%
98.	@garyvee	Gary Vaynerchuk	gary	15.7%
99.	@gigaom	GigaOM		15.6%
100.	@roelofbotha	roelof	roelofbotha	15.5%

Twitter Users Sorted by % of Their Followers on AngelList

Rank	Twitter	Name	AngelList	% followers
1.	@PitchaStartup	Pitch Startup		18.3%
2.	@chrisfralic	Chris Fralic		16.8%
3.	@cdouvos	Chris Douvos		14.4%
4.	@VCdelta	Neubot		11.2%
5.	@deekay	David King	deekay	9.5%
6.	@Sigalow	Ian Sigalow		9.5%
7.	@mkjung	Mike Jung		9.2%
8.	@EdGrapeNutZimm	Ed Zimmerman		8.9%
9.	@marquez140	Michael Marquez	marquez	8.7%
10.	@mcgd	Michael Dearing	mcgd	8.6%
11.	@yanda	Yan-David Erlich	yanda	8.6%
12.	@jamescham	James Cham	jamescham	8.3%
13.	@alanjpatricof	alan patricof		8.3%
14.	@sraney	Scott Raney		7.9%
15.	@dafrankel	David Frankel		7.8%
16.	@anthonysoohoo	Anthony Soohoo	anthony-soohoo	7.7%
17.	@nist	Scott Banister		7.6%
18.	@howardh	Howard Hartenbaum		7.6%
19.	@robstavis	Robert Stavis		7.6%
20.	@kefrusy	Kevin Efrusy	kefrusy	7.5%
21.	@finkelstein1	Alex Finkelstein	finkelstein1	7.4%
22.	@munjalshah	Munjal		7.1%
23.	@beninato	beninato		6.8%
24.	@yurimilner	Yuri		6.7%
25.	@niravtolia	Nirav Tolia		6.7%
26.	@jeremyl	Jeremy Levine	jeremyl	6.5%
27.	@tydanco	Ty Danco	tydanco	6.4%
28.	@dcstalder	Dana Stalder	dcstalder	6.3%
29.	@LewisGersh	LewisGersh		6.2%
30.	@standers	standers	standers	6.2%
31.	@todfrancis	Tod Francis		6.2%
32.	@zamland	zamland	zamland	6.1%
33.	@bsiscovick	Ben Siscovick		6.1%
34.	@j_seelig	j_seelig	j_seelig	6.1%
35.	@rochoy	Ro Choy		6.0%
36.	@leonardspeiser	Leonard Speiser		6.0%
37.	@mattcharris	Matt Harris	matt-harris	6.0%
38.	@ping_accel	ping li	ping_accel	5.9%
39.	@chazard	Chip Hazard	chazard	5.7%
40.	@gharik	Georges Harik	gharik	5.7%
41.	@robconeybeer	Rob Coneybeer	robconeybeer	5.6%
42.	@1CharlesH	Charles Huang		5.6%
43.	@ganeumann	Jerry Neumann	ganeumann	5.6%
44.	@JulesMaltz	Jules Maltz		5.5%
45.	@tgmtgm	Thomas McInerney	tgmtgm	5.5%
46.	@chamath	Chamath Palihapitiya		5.5%
47.	@deblanda	Debbie Landa	debbie-landa	5.5%
48.	@timc	Tim Connors		5.3%
49.	@dannyrimer	danny rimer	dannyrimer	5.3%
50.	@topherc	Topher Conway	topherc	5.3%
51.	@epaley	Eric Paley	epaley	5.2%
52.	@michaelpcassidy	Mike Cassidy		5.1%
53.	@BadVC	The Bad VC		5.1%
54.	@ttunguz	Tom Tunguz	ttunguz	5.1%
55.	@andreazurek	Andrea Zurek	andreazurek	5.1%
56.	@joevc	Joe Medved	joevc	5.0%
57.	@ventilla	ventilla		5.0%
58.	@jeff_jordan	Jeff Jordan		5.0%
59.	@jordancooper	Jordan Cooper	jordancooper	5.0%
60.	@mikeisbrown	Michael Brown		5.0%
61.	@carterkev	Kevin Carter	carterkev	4.9%
62.	@ChrisFRC	Chris Fralic	chrisfralic	4.9%
63.	@timechange	tim chang	timechange	4.9%
64.	@MikeBrownJr	MikeBrownJr		4.9%
65.	@m2jr	Mike Maples		4.9%
66.	@fcollective	founder collective		4.8%
67.	@sarahtavel	Sarah Tavel	sarahtavel	4.8%
68.	@blader	Siqi Chen	blader	4.8%
69.	@roelofbotha	roelof	roelofbotha	4.7%
70.	@BradUSV	Brad Burnham		4.7%
71.	@EghosaO	Eghosa Omoigui		4.7%
72.	@ethankurz	Ethan Kurzweil	ethankurz	4.7%
73.	@phineasb	Phineas Barnes	phineasb	4.7%
74.	@kteare	Keith Teare	Kteare	4.6%
75.	@VICTORIAeSONG	Victoria Song	victoria-song	4.6%
76.	@angelpadorg	AngelPad	angelpad	4.6%
77.	@mattcoffin	mattcoffin	mattcoffin	4.6%
78.	@daslee	David Lee	daslee	4.5%
79.	@witheiler	Matthew Witheiler	witheiler	4.5%
80.	@c_sheehan	Chris Sheehan	c_sheehan	4.5%
81.	@leehower	Lee Hower	leehower	4.5%
82.	@svangel	SV Angel		4.5%
83.	@rfradin	Russell Fradin	rfradin	4.5%
84.	@shaig	Shai Goldman		4.5%
85.	@mitchlasky	Mitch Lasky		4.5%
86.	@adaugelli	Adam D’Augelli	adaugelli	4.4%
87.	@davidcowan	davidcowan	davidcowan	4.4%
88.	@frenchie_vc	Geraldine Alias		4.4%
89.	@ktrae	Katie Rae	ktrae	4.4%
90.	@mattcohler	Matt Cohler	mattcohler	4.4%
91.	@ewiesen	Eric Wiesen	ewiesen	4.4%
92.	@christine_tsai	Christine Tsai	christinetsai	4.4%
93.	@karl	Karl Jacob	Karl	4.4%
94.	@jamesjoaquin	James Joaquin	james-joaquin	4.4%
95.	@davidblumberg	David Blumberg	davidblumberg	4.3%
96.	@bgarlinghouse	Brad Garlinghouse		4.3%
97.	@dshen	David Shen	dshen	4.3%
98.	@kevinhartz	Kevin Hartz	kevin-hartz	4.3%
99.	@venturedan	Dan Rosen	venturedan	4.3%
100.	@dfjjosh	Josh Stein	dfjjosh	4.3%

Production is Red, Development is Blue

2011-01-10T00:00:00-08:00

Paul Oswald tweeted a great idea for a productivity hack based on sensory feedback:

Anyone know of an OS X terminal that can be configured to change background color based on what machine you ssh to? #killerfeature9:35 PM Sat Jan 8, 2011 via Twitter for MacPaul Oswald
poswald

Example scenario: ssh into your production machine, and your terminal automatically sets its background color to red. Exit back to your development machine, and the background color automatically reverts to blue.

Jed Schmidt chimed in with a solution:

@poswald @JeffMiller iTerm is scriptable. could you alias ssh to an applescript that changes the active terminal and then runs the command?9:30 AM Sun Jan 9, 2011 via EchofonJed Schmidt
jedschmidt

I hacked up a bash script to implement their idea.

This script only works with the Mac OS X Terminal app—not iTerm as Jed suggested—but it could be modified to work with any terminal app that supports AppleScript.

The concept of sensory feedback could be extended further, too. One idea is to play a specific sound clip when you ssh into your production machine. Sort of like Happy Git Commits.

#!/bin/bash
#
# ssh into a machine and automatically set the background
# color of Mac OS X Terminal depending on the hostname.
#
# Installation:
# 1. Save this script to /some/bin/ssh-host-color
# 2. chmod 755 /some/bin/ssh-host-color
# 3. alias ssh=/some/bin/ssh-host-color
# 4. Configure your host colors below.

set_term_bgcolor() {
   local R=$1
   local G=$2
   local B=$3
   /usr/bin/osascript <<EOF
tell application "Terminal"
   tell window 0
      set the background color to {$(($R*65535/255)), $(($G*65535/255)), $(($B*65535/255))}
   end tell
end tell
EOF
}

# Host-specific background colors.
if [[ "$@" =~ production1.com ]]; then
   set_term_bgcolor 127 0 0
elif [[ "$@" =~ production2.com ]]; then
   set_term_bgcolor 0 127 0
fi

ssh $@

# Default background color.
set_term_bgcolor 34 79 188

The Three People to Follow for Tech News on Twitter

2010-12-31T00:00:00-08:00

2010 is the year I stopped going to blogs or newspapers directly for technology news. Now, I get it all through Twitter. Important stories naturally bubble up through my Twitter stream because so many people retweet them. There’s rarely a need to go out and manually check for news.

But the sources of high quality news tweets are surprisingly concentrated. I’ve found that probably 90% of the good stuff comes from just three accounts. These are the three people you need to follow for tech news on Twitter:

1. @atul. Described as a “super news-hunter” by ReadWriteWeb, and name-checked by the New York Times, Atul Arora tweets breaking news about Silicon Valley, startups and technology in general. He’s mysteriously fast, he doesn’t tweet too often (very important), and he never goes off-topic. Atul’s signal-to-noise ratio is among the highest I’ve seen on Twitter.

Follow @atul

2. @hkanji. A one-man news aggregator, Hussein Kanji is an amazing source of not only tech news but articles about markets and venture capital as well. He functions like a cross between The Economist and Hacker News. A master retweeter, it’s not uncommon to see 10 high quality links in a row from hkanji’s account.

Follow @hkanji

3. @newsyc100. Not technically a person, but a bot I made that tweets Hacker News stories scoring 100 points or higher. If major news in the startup world occurs, you’ll see it here, along with whatever else is deemed particularly interesting by the HN crowd. The score threshold is the key to avoiding the noise on HN. And if you want more stories, you can lower the threshold and follow @newsyc50 or @newsyc20 instead.

Follow @newsyc100

For more stuff like this, follow me on Twitter.

AngelFlow, a Social Dashboard for Angel Investors

2010-12-14T00:00:00-08:00

AngelFlow is a social dashboard for angel investors, and those who love them.

It brings together all the top angel investors from AngelList and Twitter into a single, unified interface.

You can sign in with your Twitter account and follow angels directly with one click. Hover your mouse over an angel’s Twitter name to see their bio and latest tweet.

If you’re an angel investor with an account on AngelList, AngelFlow makes it easy to connect your Twitter graph with your AngelList graph. You can see who you’re already following on Twitter, then follow them back on AngelList with a single click. (Be sure to sign in with your AngelList account first.)

AngelFlow was built with data from the AngelList API. Thanks to Nivi and Naval for providing the data, and to Brandon Leonardo for the API work.

» Continue on to AngelFlow

Bitly News

2010-12-09T00:00:00-08:00

I’m a believer in doing side projects as a way to avoid burnout. I think of them as being like Google’s 20 percent time, but at an individual scale.

Last month I was spending some of my side project time tinkering with news.arc, the open source software by Paul Graham that powers Hacker News. I’m not a Lisp hacker and I don’t know the Arc language, so I wasn’t coding new features or doing anything novel—just trying to get my own hosted news.arc instance up and running on Amazon EC2. (Which I did eventually, after tracking down the right version of mzScheme and getting a refresher on the Unix screen commmand.)

Around the same time, bit.ly announced it was running another bit.ly API contest. The contest was open to any submission that made use of the bit.ly API. 1st Prize: a Makerbot Thing-O-Matic 3-D Printer.

Sounded to me like a prime opportunity to hack up something fun.

Bitly News

The outcome was a mashup of bit.ly and news.arc: Bitly News.

Bitly News is a news site powered by bit.ly clicks. It works by automatically discovering bit.ly links on Twitter in real time, then posting stories with high click activity on bit.ly. Each story’s score is periodically updated with recent stats from bit.ly, so the most clicked-on stories bubble to the top.

As I see it, the primary value of Bitly News is its acute real time nature. Rankings are derived from trailing 30-minute click counts, so the top stories on the front page should be breaking news. Not breaking news in the New York Times sense, but whatever the masses of users clicking on bit.ly links collectively deem to be “breaking”. Seeing what everyone is clicking on right now is just inherently interesting.

This idea is similar but not the same as community-voting news sites like Reddit, Digg, and Hacker News. This article in TechCrunch about Bitly News draws a direct parallel between Bitly News and Hacker News, but I don’t think the comparison is quite apt. (I definitely invited the comparison by using news.arc as the platform for Bitly News, though.) Whereas Hacker News is intensely community driven, my goal with Bitly News was to remove the influence of a specific community from the equation entirely. Instead I wanted a parallel site where news discovery was more decentralized and where the community could be the entire web—or if not the whole web then its viceroy, Twitter.

As it stands, Bitly News is a decent MVP but it has its flaws. One is the lolcats effect. Looking on the front page right now I see celebrity interviews, shopping promos, sports news and the occasional viral photo—exactly the kinds of links that zillions of average Twitter users out there are sharing, but not quite a “Hacker News for the rest of us.” In retrospect, this pop-culture focus was the inevitable result of honing in on popular, trending links, I just didn’t expect the content to be so all over the place. WikiLeaks next to the Red Sox. Tony Hawk side by side with CSI Miami.

Improvements

How can Bitly News be improved?

Categories. Separate news stories into topic verticals (Entertainment, Technology, etc). This should help mitigate the lolcats effect. The challenge will be automating the categorization with high accuracy.
Other sources of real-time engagement. Expand the story ranking metric to include data points beyond just bit.ly clicks.
Social graph weighting. Allow users to login with their Twitter accounts, then overweight stories that have been retweeted by people by they follow.

If you have other suggestions for Bitly News 2.0, please leave them in the comments below, email me or reach out to me on Twitter.

Thanks

My sincerest thanks to bit.ly for running the API contest. Bitly News was lucky enough to win the robot.

» See the front page of Bitly News

Hacker News Twitter Feeds with Hashtag Metadata

2010-11-08T00:00:00-08:00

Certain types of Hacker News stories always make me pay extra attention:

Submissions by users with a lot of karma points. High-karma users are mostly HN gurus who have been on the site for a long time. They know the tastes of the community and have built a track record of quality posts. While not a perfect indicator, these superkarmic users tend to post better stories than users with more earthly karma.
An unusually high number of comments. Average HN stories always attract a handful of comments no matter what, but the juiciest submissions spark active discussions that go on for pages and pages. This is often a tell that there’s something interesting going on with the underlying story. And the comments make for great reading by themselves.
Trending, high velocity stories. Story A scores 100 points over 4 hours; story B scores 100 points in 30 minutes. Which one’s likely to be more intriguing? Story B. When a post piles up points quickly, it’s clearly grabbed the attention of HN readers. Of course the slow-burn stories can be great, too—especially in more esoteric topics—but items that rocket to the top of HN are usually the hacker-ish articles that I enjoy.

My Twitter feeds for Hacker News now contain hashtags that identify stories with these special attributes.

#guru

The #guru tag is applied to stories submitted by users with karma scores in the top quartile of all Hacker News users (a karma of ~ 1,000 as of Nov 2010).

Keep Tabs On Your Favorite Startups With StartupFollower http://tcrn.ch/9UzobE (http://bit.ly/a0ci2x) via @Stephen #guru8:05 PM Sun Nov 7, 2010 via newsycHacker News 20
newsyc20

#active

The #active tag is for stories whose comment-to-score ratio is above 0.5. For example, a story with 50 points and 30 comments would be tagged as #active. This applies at the time the story is tweeted.

Things in MySQL that won’t work as expected http://bit.ly/9mdlaP (http://bit.ly/bbBWcJ) #active10:25 PM Sun Nov 7, 2010 via newsycHacker News 20
newsyc20

#trending

The #trending tag identifies stories that have scored more than 20 points per hour. This velocity is measured at the time the story is tweeted.

Microsoft releases an incredible product http://nyti.ms/addunX (http://bit.ly/aGnQKS) via @carterac #trending8:30 PM Thu Nov 4, 2010 via newsycHacker News 20
newsyc20

Combinations

Sometimes a story hits a combination of hashtags, or even all three:

Zero to a Million Users in 15 Days http://bit.ly/alzAJ3 (http://bit.ly/bSb7Rg) #trending #active #guru8:55 AM Sat Nov 6, 2010 via newsycHacker News 50
newsyc50

Follow

Follow the Hacker News Twitter feeds here:

Hacker News stories reaching 20 points	@newsyc20	RSS feed
Hacker News stories reaching 50 points	@newsyc50	RSS feed
Hacker News stories reaching 100 points	@newsyc100	RSS feed
Hacker News stories reaching 150 points	@newsyc150	RSS feed

Or read my original post A Cure for Hacker News Overload.

Twitter Users Most Followed by Readers of Hacker News

2010-07-28T00:00:00-07:00

Updated December 29, 2011

One of the best ways to find interesting people to follow on Twitter is to look and see who other smart people are already following.

With that goal in mind, I’ve created a list of Twitter accounts heavily followed by the Hacker News community. The list is ranked by the percent of HN readers following each person. Many of the people uncovered by this list are themselves active users on Hacker News, but many are not, and overall I think it’s a fun way to get a glimpse of who the HN community finds it valuable to listen to (on Twitter, at least).

Data

One of the byproducts of my Hacker News Twitter feeds project was a data set of Twitter names for all Hacker News users.

This data set is not perfect. First, it only covers HN users who have listed their Twitter accounts in their profile (plus a handful of users I entered in manually, like pg). This set is only a fraction of overall Hacker News readership. Second, the set may be missing some users whose profiles I haven’t yet scraped. In the end, the total data set only amounts to 400-500 Twitter users, although I think that’s probably big enough to be representative of the population of Hacker News readers as a whole.

Once I had this HN user data, generating the list was done using Python and Tweepy.

This list is available as a CSV file.

Twitter Users Sorted by % of All HN Readers Following

Rank	Twitter	Name	Hacker News	% following
1.	@paulg	Paul Graham	pg	58.7%
2.	@fredwilson	Fred Wilson	fredwilson	37.1%
3.	@jeresig	John Resig	jeresig	29.1%
4.	@dhh	DHH	dhh	28.3%
5.	@cdixon	chris dixon	cdixon	28.3%
6.	@ycombinator	Y Combinator		27.2%
7.	@jasonfried	Jason Fried		27.0%
8.	@davemcclure	Dave McClure		26.3%
9.	@timoreilly	Tim O’Reilly		25.6%
10.	@ev	Evan Williams		24.2%
11.	@al3x	Alex Payne	al3x	24.0%
12.	@BillGates	Bill Gates		23.6%
13.	@hnshah	Hiten Shah		23.4%
14.	@github	GitHub	martius	23.3%
15.	@TechCrunch	TechCrunch		23.2%
16.	@jack	Jack Dorsey		22.5%
17.	@sacca	Chris Sacca		22.4%
18.	@Scobleizer	Robert Scoble		22.3%
19.	@msuster	Mark Suster	msuster	22.3%
20.	@kevinrose	Kevin Rose		22.2%
21.	@joehewitt	Joe Hewitt		22.1%
22.	@dcurtis	dustin curtis	dcurtis	21.8%
23.	@ericries	Eric Ries		21.7%
24.	@venturehacks	Venture Hacks	nivi	21.6%
25.	@arrington	Michael Arrington		21.0%
26.	@Jason	Jason Calacanis	jasonmcalacanis	20.3%
27.	@defunkt	Chris Wanstrath	defunkt	20.2%
28.	@paultoo	Paul Buchheit	paul	20.2%
29.	@gruber	John Gruber		19.9%
30.	@spolsky	Joel Spolsky	spolsky	19.8%
31.	@kn0thing	Alexis Ohanian	kn0thing	19.8%
32.	@yegg	Gabriel Weinberg	epi0Bauqu	19.6%
33.	@newsycombinator	news.yc Popular		19.3%
34.	@Dropbox	Dropbox		19.2%
35.	@ConanOBrien	Conan O’Brien		19.0%
36.	@shitmydadsays	Justin		19.0%
37.	@mattcutts	Matt Cutts		18.4%
38.	@google	A Googler		17.8%
39.	@garrytan	Garry Tan	rantfoil	17.8%
40.	@codinghorror	Jeff Atwood	codinghorror	17.7%
41.	@joshu	joshua schachter	joshu	16.9%
42.	@patio11	Patrick McKenzie	patio11	16.9%
43.	@bfeld	Brad Feld		16.8%
44.	@sgblank	steve blank		16.8%
45.	@twitter	Twitter		16.6%
46.	@sivers	Derek Sivers	sivers	16.5%
47.	@tferriss	Tim Ferriss		15.8%
48.	@rands	rands		15.5%
49.	@marcoarment	Marco Arment		15.4%
50.	@zedshaw	zedshaw	zedshaw	15.4%
51.	@om	Om Malik		15.1%
52.	@biz	Biz Stone		14.9%
53.	@dharmesh	Dharmesh Shah	dshah	14.9%
54.	@leahculver	Leah Culver		14.9%
55.	@Caterina	Caterina Fake		14.7%
56.	@AndrewWarner	Andrew Warner	AndrewWarner	14.7%
57.	@mojombo	Tom Preston-Werner		14.7%
58.	@ericschmidt	Eric Schmidt		14.6%
59.	@twitterapi	Twitter API		14.4%
60.	@Harjeet	Harjeet	Harj	14.2%
61.	@garyvee	Gary Vaynerchuk		14.1%
62.	@andrewchen	Andrew Chen		14.1%
63.	@marissamayer	marissamayer		14.1%
64.	@mkapor	Mitch Kapor		13.9%
65.	@mashable	Pete Cashmore		13.8%
66.	@naval	Naval Ravikant		13.6%
67.	@quixotic	Reid Hoffman		13.5%
68.	@foundersatwork	Jessica Livingston		13.5%
69.	@BarackObama	Barack Obama		13.2%
70.	@parislemon	MG Siegler		13.0%
71.	@jmattheij	jmattheij	jacquesm	12.9%
72.	@Stammy	Paul Stamatiou	PStamatiou	12.9%
73.	@Werner	Werner Vogels		12.8%
74.	@joshk	Josh Kopelman		12.7%
75.	@gvanrossum	Guido van Rossum		12.6%
76.	@swombat	Daniel Tenner	swombat	12.3%
77.	@anildash	Anil Dash		12.2%
78.	@chrismessina	Chris Messina		12.1%
79.	@azaaza	Aza Raskin		12.0%
80.	@cshirky	Clay Shirky		12.0%
81.	@37signals	37signals		11.8%
82.	@wycats	wycats		11.8%
83.	@mlevchin	Max Levchin		11.7%
84.	@antirez	Salvatore Sanfilippo	antirez	11.6%
85.	@rabois	Keith Rabois		11.6%
86.	@davemorin	Dave Morin		11.5%
87.	@KISSmetrics	KISSmetrics.com		11.4%
88.	@ryah	Ryan Dahl		11.4%
89.	@paul_irish	Paul Irish		11.3%
90.	@mcuban	Mark Cuban		11.3%
91.	@daringfireball	Daring Fireball		11.1%
92.	@marshallk	Marshall Kirkpatrick		11.0%
93.	@jeff	Jeff Clavier		11.0%
94.	@simonw	Simon Willison		10.9%
95.	@zappos	Zappos.com CEO -Tony		10.9%
96.	@jkottke	Jason Kottke		10.9%
97.	@sparker	Sean Parker		10.8%
98.	@bhorowitz	bhorowitz		10.8%
99.	@joestump	Joe Stump		10.6%
100.	@hipsterhacker	Hipster Hacker		10.6%
101.	@richardbranson	richardbranson		10.6%
102.	@holman	Zach Holman	holman	10.6%
103.	@danmartell	Dan Martell		10.4%
104.	@vkhosla	Vinod Khosla		10.4%
105.	@bijan	Bijan Sabet		10.3%
106.	@AppSumo	AppSumo.com		10.3%
107.	@igrigorik	Ilya Grigorik		10.1%
108.	@hotdogsladies	Merlin Mann		10.1%
109.	@ID_AA_Carmack	John Carmack		10.1%
110.	@noahkagan	noah kagan	noahkagan	10.0%
111.	@hmason	Hilary Mason	hmason	9.9%
112.	@drewhouston	Drew Houston	dhouston	9.8%
113.	@dickc	dick costolo		9.8%
114.	@dens	Dennis Crowley		9.7%
115.	@photomatt	Matt Mullenweg	photomatt	9.7%
116.	@RWW	ReadWriteWeb		9.7%
117.	@leolaporte	Leo Laporte		9.6%
118.	@timbray	Tim Bray		9.6%
119.	@spam	Spam Watch		9.6%
120.	@neilpatel	Neil Patel		9.4%
121.	@pud	Philip Kaplan		9.3%
122.	@markbao	Mark Bao	markbao	9.3%
123.	@ginatrapani	Gina Trapani		9.3%
124.	@alexia	Alexia Tsotsis		9.3%
125.	@heroku	heroku		9.1%
126.	@tqbf	Thomas H. Ptacek	tptacek	9.1%
127.	@ryancarson	Ryan Carson		9.0%
128.	@smashingmag	Smashing Magazine		9.0%
129.	@500startups	500 Startups		8.9%
130.	@slavingia	Sahil Lavingia	sahillavingia	8.9%
131.	@stephenfry	Stephen Fry		8.9%
132.	@SeanEllis	Sean Ellis		8.9%
133.	@wikileaks	WikiLeaks		8.9%
134.	@zeldman	Jeffrey Zeldman		8.8%
135.	@bryce	Bryce Roberts		8.8%
136.	@davewiner	Dave Winer	davewiner	8.7%
137.	@gaberivera	Gabe Rivera		8.7%
138.	@rtomayko	Ryan Tomayko		8.7%
139.	@dcancel	David Cancel	dcancel	8.6%
140.	@JasonLBaptiste	Jason L. Baptiste		8.6%
141.	@zefrank	zefrank		8.6%
142.	@peterc	Peter Cooper	petercooper	8.5%
143.	@TEDchris	Chris Anderson		8.5%
144.	@pkedrosky	Paul Kedrosky		8.5%
145.	@aplusk	ashton kutcher		8.5%
146.	@randfish	Rand Fishkin		8.5%
147.	@Oatmeal	Matthew Inman		8.5%
148.	@timberners_lee	Tim Berners-Lee		8.5%
149.	@asmartbear	Jason Cohen		8.4%
150.	@davidcohen	David Cohen		8.4%
151.	@TheOnion	The Onion		8.3%
152.	@Zee	Zee M Kane		8.3%
153.	@loic	Loic Le Meur		8.3%
154.	@stop	Doug Bowman		8.3%
155.	@dalmaer	Dion Almaer		8.1%
156.	@angellist	AngelList		8.1%
157.	@patrickc	Patrick Collison		8.1%
158.	@LaughingSquid	Laughing Squid		8.0%
159.	@karaswisher	Kara Swisher		8.0%
160.	@simplebits	Dan Cederholm		7.9%
161.	@jolieodell	Jolie O’Dell		7.9%
162.	@notch	Markus Persson		7.9%
163.	@tlrobinson	tom robinson	tlrobinson	7.8%
164.	@ericflo	Eric Florenzano		7.8%
165.	@startupfoundry	Paul Hontz		7.8%
166.	@rsarver	Ryan Sarver		7.8%
167.	@lessig	Lessig		7.8%
168.	@googleio	Google I/O		7.7%
169.	@albertwenger	Albert Wenger		7.7%
170.	@jasonkincaid	Jason Kincaid		7.7%
171.	@DEVOPS_BORAT	DevOps Borat		7.7%
172.	@danielbru	Daniel Brusilovsky		7.6%
173.	@nivi	Nivi		7.6%
174.	@atebits	Loren Brichter		7.6%
175.	@avibryant	Avi Bryant		7.5%
176.	@StephenAtHome	Stephen Colbert		7.5%
177.	@kneath	Kyle Neath		7.5%
178.	@techstars	techstars		7.5%
179.	@technoweenie	risk danger olson		7.5%
180.	@jashkenas	Jeremy Ashkenas		7.4%
181.	@pjhyett	PJ Hyett	pjhyett	7.4%
182.	@davetisch	David Tisch		7.3%
183.	@finkd	Mark Zuckerberg		7.3%
184.	@linode	Linode		7.3%
185.	@mingyeow	Ming Yeow Ng		7.3%
186.	@ezmobius	Ezra Zygmuntowicz		7.3%
187.	@cnnbrk	CNN Breaking News		7.3%
188.	@rbates	Ryan Bates		7.2%
189.	@BreakingNews	Breaking News		7.2%
190.	@rails	Ruby on Rails		7.2%
191.	@mikebutcher	Mike Butcher		7.0%
192.	@lonelysandwich	Adam Lisagor		7.0%
193.	@chacon	Scott Chacon		7.0%
194.	@ceonyc	Charlie O’Donnell		7.0%
195.	@tedtalks	TEDTalks Updates		7.0%
196.	@Techmeme	Techmeme		7.0%
197.	@andrewhyde	Andrew Hyde		7.0%
198.	@HackerNewsTips	Hacker News Tips		7.0%
199.	@kylebragger	Kyle Bragger	kylebragger	7.0%
200.	@zephoria	danah boyd		6.9%

Twitter Users Sorted by % of Their Followers from HN

Rank	Twitter	Name	Hacker News	% followers
1.	@sgrove	Sean Grove		9.4%
2.	@grellas	George Grellas	grellas	9.2%
3.	@icey	Paul Nakata	icey	9.1%
4.	@__dave	Dave Fowler		6.5%
5.	@scotchi	Scott Wheeler	wheels	6.0%
6.	@patio11	Patrick McKenzie	patio11	5.9%
7.	@dmitrigrabov	Dmitri Grabov		5.9%
8.	@tlbtlbtlb	Trevor Blackwell	tlb	5.4%
9.	@solomonstre	Solomon Hykes		5.2%
10.	@JeffMiller	Jeff Miller	jeffmiller	5.0%
11.	@YC_Y_U_NO	YC Y U NO		4.9%
12.	@drusenko	drusenko		4.2%
13.	@immad	immad	immad	4.1%
14.	@foundersatwork	Jessica Livingston		4.1%
15.	@davidbalbert	David Albert		4.1%
16.	@jmattheij	jmattheij	jacquesm	4.0%
17.	@samstokes	Sam Stokes	samstokes	3.9%
18.	@Lately	Luke Groesbeck		3.8%
19.	@yuris	Yuri Sagalov		3.8%
20.	@angryaustin	austin chang		3.7%
21.	@robfitz	Rob Fitzpatrick		3.6%
22.	@jamiequint	Jamie Quint		3.5%
23.	@antoniogm	Antonio Garcia		3.5%
24.	@sodio	Sam Odio	Sam_Odio	3.5%
25.	@niallsmart	niallsmart		3.5%
26.	@samuelclay	Samuel Clay	conesus	3.5%
27.	@newsyc50	Hacker News 50		3.5%
28.	@sama	Sam Altman		3.4%
29.	@euwyn	Euwyn Poon		3.3%
30.	@daleharvey	Dale Harvey	daleharvey	3.2%
31.	@HackerNewsTips	Hacker News Tips		3.1%
32.	@slava_pestov	Slava Pestov		3.1%
33.	@patrickc	Patrick Collison		3.1%
34.	@mrm	Matthew McEachen		3.1%
35.	@yegg	Gabriel Weinberg	epi0Bauqu	3.0%
36.	@Harjeet	Harjeet	Harj	3.0%
37.	@shazow	Andrey Petrov		3.0%
38.	@raygrieselhuber	Ray Grieselhuber	rgrieselhuber	3.0%
39.	@judegomila	judegomila		2.9%
40.	@tlrobinson	tom robinson	tlrobinson	2.9%
41.	@tikhon	Tikhon		2.9%
42.	@KirinDave	Dave Fayram		2.9%
43.	@swombat	Daniel Tenner	swombat	2.8%
44.	@cperciva	Colin Percival	cperciva	2.8%
45.	@stevedekorte	Steve Dekorte		2.8%
46.	@daniel_levine	Daniel Levine	daniel_levine	2.8%
47.	@michaelmontano	Michael Montano		2.8%
48.	@plc	peter clark	pclark	2.7%
49.	@RichAberman	Rich Aberman	aberman	2.7%
50.	@martinkl	Martin Kleppmann		2.7%
51.	@yurimilner	Yuri		2.7%
52.	@mhartl	Michael Hartl		2.7%
53.	@sebastmarsh	Sebastian Marshall	lionhearted	2.6%
54.	@substack	James Halliday	substack	2.6%
55.	@jf	Joel Franusic	jf	2.5%
56.	@kul	Kulveer Taggar		2.5%
57.	@mmcgrana	Mark McGranaghan	mmcgrana	2.5%
58.	@JasonShen	Jason Shen		2.5%
59.	@joedamato	Joe Damato		2.5%
60.	@hackermonthly	Hacker Monthly		2.4%
61.	@tqbf	Thomas H. Ptacek	tptacek	2.4%
62.	@limedaring	Tracy Osborn	limedaring	2.4%
63.	@nathanmarz	Nathan Marz	nathanmarz	2.4%
64.	@Tawheed	TK	Tawheed	2.3%
65.	@boucher	Ross Boucher		2.3%
66.	@ikirigin	Ivan Kirigin	ivankirigin	2.3%
67.	@rcrowley	Richard Crowley		2.3%
68.	@coderholic	Ben Dowling		2.2%
69.	@billclerico	Bill Clerico		2.2%
70.	@jgrahamc	John Graham-Cumming	jgrahamc	2.2%
71.	@argv0	Andy Gross		2.2%
72.	@ericflo	Eric Florenzano		2.2%
73.	@zenmatt	Matt Tanase	howradical	2.2%
74.	@3en	Ben Reyes	benreyes	2.1%
75.	@jazzychad	Chad	jazzychad	2.1%
76.	@davidwurtz	David Wurtz		2.1%
77.	@stripe	Stripe		2.1%
78.	@raganwald	Reg Braithwaite	raganwald	2.1%
79.	@progrium	Jeff Lindsay		2.1%
80.	@enkido	Paul Bohm		2.1%
81.	@amandapey	amanda peyton		2.1%
82.	@golda	Christopher Golda		2.0%
83.	@carterkev	Kevin Carter		2.0%
84.	@rahulvohra	Rahul Vohra	rahulvohra	2.0%
85.	@strlen	Alex Feinberg	strlen	2.0%
86.	@christine_tsai	Christine Tsai		2.0%
87.	@8en	Ben Silbermann		2.0%
88.	@tnm	Ted Nyman		2.0%
89.	@markessien	Mark Essien	markessien	1.9%
90.	@fogus	fogus	fogus	1.9%
91.	@pm	Paul		1.9%
92.	@newsyc20	Hacker News 20		1.9%
93.	@garrytan	Garry Tan	rantfoil	1.8%
94.	@bmizerany	Blake Mizerany		1.8%
95.	@brezina	Matt Brezina	brezina	1.8%
96.	@webwright	Tony Wright	webwright	1.8%
97.	@asmith	Adam Smith	adamsmith	1.8%
98.	@royrod	Roy Rodenstein		1.8%
99.	@paulsingh	Paul Singh		1.8%
100.	@jeffiel	jeffiel		1.8%
101.	@startupfoundry	Paul Hontz		1.8%
102.	@indexzero	Charlie Robbins		1.8%
103.	@jashkenas	Jeremy Ashkenas		1.8%
104.	@newsyc100	Hacker News 100		1.8%
105.	@kneath	Kyle Neath		1.8%
106.	@jedschmidt	Jed Schmidt		1.8%
107.	@dtran320	David Tran	dtran	1.8%
108.	@technomancy	Phil Hagelberg		1.7%
109.	@mojombo	Tom Preston-Werner		1.7%
110.	@justinkan	Justin Kan		1.7%
111.	@dodeja	Akshay Dodeja	lyime	1.7%
112.	@aq	Aaron Quint		1.7%
113.	@pjhyett	PJ Hyett	pjhyett	1.7%
114.	@paulca	Paul Campbell		1.7%
115.	@bragiel	paul bragiel		1.7%
116.	@ryanjunee	Ryan Junee		1.7%
117.	@jamesjyu	James Yu		1.7%
118.	@metabrew	Richard Jones		1.7%
119.	@mattmireles	Matt Mireles		1.7%
120.	@DanielZarick	Daniel Zarick		1.7%
121.	@startupschool	Startup School		1.7%
122.	@rickyyean	Ricky Yean		1.7%
123.	@sachinag	Sachin Agarwal		1.7%
124.	@jessenoller	jessenoller		1.7%
125.	@tastylabs	Tasty Labs		1.7%
126.	@jasonfreedman	Jason Freedman		1.7%
127.	@danielha	Daniel Ha	danielha	1.7%
128.	@rtomayko	Ryan Tomayko		1.7%
129.	@chrysb	Chrys Bader		1.6%
130.	@stevelosh	Steve Losh		1.6%
131.	@roidrage	Mathias Meyer		1.6%
132.	@jwegener	Jonathan Wegener		1.6%
133.	@voodootikigod	Chris Williams		1.6%
134.	@node_knockout	node knockout		1.6%
135.	@antirez	Salvatore Sanfilippo	antirez	1.6%
136.	@garethr	Gareth Rushgrove	garethr	1.6%
137.	@julien51	Julien Genestoux		1.5%
138.	@adaugelli	Adam D’Augelli		1.5%
139.	@jgebbia	Joe Gebbia		1.5%
140.	@noahkagan	noah kagan	noahkagan	1.5%
141.	@justinsheehy	Justin Sheehy		1.5%
142.	@tmm1	Aman Gupta		1.5%
143.	@coda	Coda Hale	codahale	1.5%
144.	@damienkatz	damien katz		1.5%
145.	@zedshaw	zedshaw	zedshaw	1.5%
146.	@bradfordcross	bradford cross		1.5%
147.	@jamesgolick	James Golick		1.5%
148.	@giffconstable	Giff Constable		1.5%
149.	@macournoyer	Marc-Andr Cournoyer		1.5%
150.	@alex_gaynor	Alex Gaynor	kingkilr	1.5%
151.	@dot_cloud	DotCloud		1.5%
152.	@defunkt	Chris Wanstrath	defunkt	1.4%
153.	@jezdez	Jannis Leidel		1.4%
154.	@tjholowaychuk	TJ Holowaychuk		1.4%
155.	@jacobian	jacobian		1.4%
156.	@atmos	Corey Donohoe		1.4%
157.	@jonpierce	Jon Pierce		1.4%
158.	@sh1mmer	Tom Hughes-Croucher	sh1mmer	1.4%
159.	@pkrumins	Peteris Krumins	pkrumins	1.4%
160.	@felixge	Felix Geisendrfer		1.4%
161.	@kn0thing	Alexis Ohanian	kn0thing	1.4%
162.	@rob_pike	Rob Pike		1.4%
163.	@ezmobius	Ezra Zygmuntowicz		1.4%
164.	@paultoo	Paul Buchheit	paul	1.3%
165.	@mxcl	Max Howell		1.3%
166.	@holman	Zach Holman	holman	1.3%
167.	@brynary	Bryan Helmkamp		1.3%
168.	@jm	Jeremy McAnally		1.3%
169.	@maccman	Alex MacCaw		1.3%
170.	@rauchg	Guillermo Rauch		1.3%
171.	@technoweenie	risk danger olson		1.3%
172.	@cramforce	Malte Ubl		1.3%
173.	@JessicaMah	Jessica Mah		1.3%
174.	@daveambrose	Dave Ambrose		1.3%
175.	@iamcal	Cal Henderson		1.3%
176.	@Chargify	Chargify		1.3%
177.	@dozba	Ted Dziuba		1.3%
178.	@gabor	Gabor Cselle		1.3%
179.	@avibryant	Avi Bryant		1.3%
180.	@slavingia	Sahil Lavingia	sahillavingia	1.3%
181.	@kadavy	David Kadavy	kadavy	1.3%
182.	@cloudhead	Alexis Sellier		1.3%
183.	@robwalling	Rob Walling		1.3%
184.	@subelsky	Mike Subelsky		1.3%
185.	@markimbriaco	Mark Imbriaco		1.3%
186.	@izs	isaacs	IsaacSchlueter	1.3%
187.	@fcollective	founder collective		1.2%
188.	@Pv	Patrick Vlaskovits		1.2%
189.	@spyced	Jonathan Ellis	jbellis	1.2%
190.	@peterc	Peter Cooper	petercooper	1.2%
191.	@Scott_Allison	Scott Allison		1.2%
192.	@etrepum	Bob Ippolito		1.2%
193.	@bchesky	Brian Chesky		1.2%
194.	@EnriqueAllen	Enrique Allen		1.2%
195.	@richhickey	Rich Hickey		1.2%
196.	@qrush	Nick Quaranto		1.2%
197.	@mitsuhiko	Armin Ronacher		1.2%
198.	@paulg	Paul Graham	pg	1.2%
199.	@tolmasky	Francisco Tolmasky	tolmasky	1.2%
200.	@evanreas	Evan Reas		1.2%

A Cure for Hacker News Overload

2010-07-23T00:00:00-07:00

Hacker News is the unofficial home page for the tech startup/entrepreneur community. It’s what most of us check first thing in the morning before the NYT, the WSJ, TechCrunch or any other news source. Like many other popular news feeds, however, the sheer volume of stories makes it difficult to keep up.

One strategy for consuming Hacker News is to periodically check the top stories on the home page since that’s where most of the high-scoring articles reside. This works okay if you’re always in front of the computer, but if you’re away, you miss out. It’s also terribly inefficient to repeatedly check HN, because you have to rescan stories you’ve already seen.

A different approach is to get Hacker News via RSS. This is nice because it frees you from having to check the HN site all the time, and you can catch up on articles whenever you have time. But there’s a fatal problem with the default Hacker News RSS feed: it contains way too many low-scoring stories–hundreds each day, it seems like. This renders it a huge time waster.

Score thresholds

Here’s a robust cure for Hacker News overload. Simply use score thresholds. This solves the problem of too many stories by cutting out low-scoring noise while still leaving behind those links interesting enough to have been upvoted by the HN community.

I’ve created a handful of Twitter and RSS feeds that apply score thresholds to Hacker News. Stories enter the feeds in near real-time as soon as they reach X points on the main HN site.

Hacker News stories reaching 20 points	@newsyc20	RSS feed
Hacker News stories reaching 50 points	@newsyc50	RSS feed
Hacker News stories reaching 100 points	@newsyc100	RSS feed
Hacker News stories reaching 150 points	@newsyc150	RSS feed

Here’s an example tweet from @newsyc20:

Peter Norvig’s Library http://books.google.com/books?uid=8640673873589796416 (http://bit.ly/9yVIjg) via @fogus9:05 PM Thu Jul 22, 2010 via newsycHacker News 20
newsyc20

Advantages over existing feeds

How does @newsyc20 (for example) differ from established Twitter bots like @newsycombinator and @hackernewsbot?

@newsyc20 includes links to the story itself and the Hacker News comment page for that story. Other feeds only link to the story. Having a link to the comments page is crucial because reading the HN discussion thread about an article is sometimes better than the article itself.

@newsyc20 preserves the actual story URL if it fits inside a tweet. This makes it easier to decide if a story is click-worthy. Compare with @newsycombinator, which applies bit.ly shortening to all URLs regardless of length.

@newsyc20 has a well-defined selection criteria: 20 points or above. It’s unclear how the other bots do their filtering.

@newsyc20 attributes stories to the original submitter on Hacker News if that user has a Twitter account listed in their HN profile. (Just for fun.)

The newsycNN feeds have different score thresholds to choose from, so if you feel like @newsyc20 has too much volume, you can switch to @newsyc50, @newsyc100 or @newsyc150.

Implementation

Hacker note: these feeds are generated by a Python script run every 5 minutes with cron. The program uses BeautifulSoup to scrape the Hacker News front page, Tweepy to send out out tweets, and sqlite3 to keep track of which stories have already been tweeted.

Feedback

I welcome any feedback or suggestions for ways to improve the Hacker News feeds. Please leave them in the comments below, email me or reach out to me on Twitter.

Solo Founders

2010-07-06T00:00:00-07:00

Startups with multiple founders have a greater chance of success than single-founder startups. This is received wisdom in the startup world. Paul Graham’s essay The 18 Mistakes That Kill Startups is most often cited together with this idea.

Strictly speaking, I don’t doubt that this has been true, and still is true.

However, there has been a tendency to interpret this idea as single founders have a near-zero chance of success, therefore founding a company alone is a fool’s errand. That is demonstrably not true. And I believe the odds in favor of solo founders are getting better and better everyday.

First, there are technology trends leveling the playing field. Starting a web company these days is unbelievably frictionless. Open source software has matured to the point where all the tools needed to make a web app are available for free. Cloud computing means CPU, network and storage resources can be paid for as-you-go instead of as fixed up-front costs, which in turn drives down the amount of capital needed to get a business off the ground. Even design services have seen some penetration by online pay-as-you-go models.

Second, culturally, more and more solo founders are going public with their experiences. Yesterday Ray Grieselhuber posted a thoughtful piece on his company blog, The Royal We: Single Founder Startups, where he shares his strategies for mitigating the problems inherent in being a solo founder. Hackers like Patrick McKenzie and Gabriel Weinberg have been documenting their efforts for a while now at building successful solo web businesses. There are many people flying a bit under the radar, like Tawheed Kader, and probably a great number more I’m not mentioning or not aware of. I’ve also become a solo founder myself. Inspiration is a vital part of running a startup, so the more solo founders who put themselves out there, the more there will be in the future.

One reason single-founder startups are sometimes viewed negatively is because the scope of opportunities is different than it is for multiple-founder startups. Many business ideas simply can’t be executed by a single person, no matter how much of a rockstar programmer + ninja bizdev person they are (and you generally have to be both to go solo). It’s difficult to imagine anyone building out a competitor to, say, Zynga all by themselves. These are the attention-getting businesses, the big winners on the road to IPO, so naturally it seems crazy that a single person would try and found a similar company.

The secret is that there is a growing universe of moderately-sized opportunities out there that are perfect for highly skilled solo founders, especially in the consumer web app space. These are often apps where the revenue path is clearly defined, the code can be written and maintained by one person, and the UI is not overly complex. A perfect example is PicClick, a visual search for eBay that entrepreneur Ryan Sit has seen recent success with. So to judge the odds of success for a single-founder startup, it is critical to take into account the scope of the opportunity.

All things being equal, multiple-founder startups still have the advantage. They are probably more resilient and almost always able to cover more ground quicker. The catch is that it all hinges on having the right co-founders. Finding that special someone to launch a startup with is way, way harder than most people imagine. Although having the right partner who complements your abilities can be a huge force multiplier, it cuts both ways–the wrong person can hinder or even destroy your startup. If you press most solo founders, my guess is that you’ll find a story about a partnership gone awry.

In the end, both single-founder and multiple-founder startups have their advantages and disadvantages. Solo operations do come with some pretty difficult challenges, but gone are the days when “Starting a startup is too hard for one person”.

This Hacker News thread from January 2010 contains some great discussions about being a solo founder.

Blackbirdpy Embedded Tweets in Python

2010-06-17T00:00:00-07:00

Update (December 2011): see Twitter’s own Embedded Tweets for a JavaScript solution to the same problem.

Announcing Blackbirdpy, a Python implementation of Blackbird Pie, the tool Robin Sloan created to generate embedded HTML tweets for blog posts.

Blackbirdpy automatically generates nicely formatted embedded tweets like this:

Just released some new code: Blackbirdpy embedded HTML tweets in Python http://github.com/jmillerinc/blackbirdpy11:26 AM Thu Jun 17, 2010 via webJeff Miller
jeffmiller

This tweet is rendered with pure HTML and CSS — no image capture necessary.

While Robin’s JavaScript version and my Python version primarily do the same thing–generate HTML snippets for embedding tweets in web pages–the difference with Blackbirdpy is that it can easily be scripted server-side, eliminating the need for copy-and-paste into static files.

Source Code

The source code for Blackbirdpy is posted on GitHub.

Thanks

All credit goes to Robin for creating the original Blackbird Pie concept. He generously allowed me to reuse the HTML and CSS from his implementation.

Hacker Angels

2010-06-14T00:00:00-07:00

Hacker Angels is a new group of hackers who are also angel investors.

That’s hackers in the good sense.

The group was formed by Gabriel Weinberg of Duck Duck Go.

Also on board are Joshua Schachter, Roy Rodenstein, Jim Young and myself.

Gabriel’s announcement is here. In his own words,

I believe that there are many good hacker entrepreneurs who are not getting the funding or advice they need to get traction for their startups. I also believe hackers relate well with other hackers, which is why I think the concept of hacker angels is so compelling.

Hacker angels can provide startups with targeted advice on technical questions, implementation issues, perhaps even do some coding here and there.

I’m especially interested in data-driven companies and concepts with a strong algorithms component.

If you’re a hacker entrepreneur, or a fellow angel, reach out to us at ha@hackerangels.com.

All of us are also on AngelList.

Twitter From the Command Line in Python Using OAuth

2010-05-31T00:00:00-07:00

Issuing Twitter updates from the command line used to be a simple affair: just use curl or its equivalent. Very soon, that will no longer be the case because basic authentication is scheduled to be removed from the Twitter API in June 2010 [edit: August 16th, 2010]. Instead, all Twitter clients will need to use more secure authentication based on OAuth.

Configuring an app to use OAuth properly requires a bit of legwork. My goal with this post is to save people some time by showing the complete step-by-step process of building a Python script that can tweet from the command line using OAuth. I’ve intentionally skipped over the details of how OAuth works and what all the different authentication tokens mean. This post is just about getting things done.

Step 1: Download Tweepy

Tweepy is an awesome Twitter library for Python. Much of this post is based on information I found in the documentation for Tweepy.

Download Tweepy from GitHub and install it on your system.

Step 2: Register a new client app with Twitter

Navigate to http://twitter.com/oauth_clients and click on Register a new application. You might have to log in to the Twitter site first, if you’re not already.

Fill in the registration fields as follows:

Note: whatever you specify for Application Name will be the “via” name your followers see in the details of tweets issued from your command line app:

When finished on the registration page, click Save.

Keep this browser window open. We’ll need the information in the next step.

Step 3: Connect the app to your Twitter account

Next, the app needs to be authorized to connect to your account so it can send tweets under your name.

We’ll create a one-off utility script to do this. Save the following Python code as a script on your local system.

#!/usr/bin/env python

import tweepy

CONSUMER_KEY = 'paste your Consumer Key here'
CONSUMER_SECRET = 'paste your Consumer Secret here'

auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth_url = auth.get_authorization_url()
print 'Please authorize: ' + auth_url
verifier = raw_input('PIN: ').strip()
auth.get_access_token(verifier)
print "ACCESS_KEY = '%s'" % auth.access_token.key
print "ACCESS_SECRET = '%s'" % auth.access_token.secret

Paste the Consumer Key and Consumer Secret from the end of step 2 into this script, replacing the CONSUMER_KEY and CONSUMER_SECRET constants. Then save and run on your system.

You should see a prompt like this:

Please authorize: <URL>
PIN:

Open that URL in your browser. You should see the standard OAuth Twitter connection screen:

Click Allow.

Twitter will then provide you with a PIN code that authenticates the connection between the client app and your Twitter account.

Enter this PIN into the prompt from the Python registration script:

PIN: 2781961

The script will then print out another key/secret pair:

ACCESS_KEY = '124242RCyi3g0cZ4r5BWL047rsh0S0yv5VxAGwTKCOsHAb'
ACCESS_SECRET = 'kaTXiC489qo8y6haTBSlwOqR1syG83tzPG2StdQ'

But the values will be different each time.

Keep this information on your screen because we’ll need it in the next step.

Step 4: Create the command line script

Save the following Python code as a script on your local system.

#!/usr/bin/env python

import sys
import tweepy

CONSUMER_KEY = 'paste your Consumer Key here'
CONSUMER_SECRET = 'paste your Consumer Secret here'
ACCESS_KEY = 'paste your Access Key here'
ACCESS_SECRET = 'paste your Access Secret here'

auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_KEY, ACCESS_SECRET)
api = tweepy.API(auth)
api.update_status(sys.argv[1])

Paste the Access Key and Access Secret from the end of step 3 into this script, replacing the ACCESS_KEY and ACCESS_SECRET constants.

You’ll also need to paste in the CONSUMER_KEY and CONSUMER_VALUES from step 2, just as we did in the registration script.

Save the final script as mycommandlineapp or whatever you want to name it.

Step 5: Send a test tweet from the command line

Finally, we’re all set up. Our command line app is registered as a Twitter client and the app is connected to our Twitter user account.

Try sending a test tweet from the command line. Don’t forget to enclose the text in quotes.

$ ./mycommandlineapp 'Hello from the command line'

If this is successful, there won’t be any output on the command line, but there will be a new tweet in your timeline:

Done.

How to Backup Google Calendar and Google Docs in 60 Seconds

2010-05-24T00:00:00-07:00

A few years ago, when I started using Google Calendar so heavily that I more or less ran my entire life off of it (and still do), I wrote a Python script to automatically backup all of my Gcal data to local disk. Same for Google Documents–I had one script that would download all of my Gdocs documents and zip them up into a nice package for safe storage on my local computer (which was in turn backed-up offsite).

This worked well for a long time, but in the intervening years Google has streamlined the export capabilities of Gcal and Gdocs to the point where automated scripts are not really worth the trouble to maintain anymore. It’s dead-easy to manually download a zip archive of all of your calendars and documents directly from the Google web apps.

Here’s how to backup Google Calendar and Google Docs in 60 seconds.

Backing up Google Calendar

Navigate to https://www.google.com/calendar/render
In the left-hand sidebar, at the bottom of the My calendars list, click the Settings link. This will take you to a Calendar Settings page.
On the settings page, click the Export calendars link at the bottom of your list of calendars. This will cause your browser to download a zip file containing each of your calendars in .ics format.
Backup that zip file somewhere safe.

Backing up Google Docs

Navigate to https://docs.google.com
In the left-hand navigation sidebar, select All Items.
In the documents list, hover your mouse cursor over the name of the first document. A link to an Actions menu will appear to the right of the document’s name. Click on it.
Select Download… This will bring up a dialog box.
Click the All items tab in the dialog box.
From there, Google lets you choose which file format to download for each document type. Once you’re finished choosing, click the Download button. This will cause your browser to download a zip file containing each of your documents in the formats you selected. It may take a while for the download to begin.
Backup that zip file somewhere safe.

Don’t forget

I’ve also found it helpful to create a recurring event in Google Calendar that will send an email once a month with a reminder to backup your data.

Twitter's Garbage Problem

2010-05-16T00:00:00-07:00

One person’s garbage is another person’s gold. Twitter has no shortage of both.

“A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty.” http://klck.me/Alo1:05 PM May 10th via ShareFeedHiten Shah
hnshah

I just ousted Owen S. as the mayor of Samovar Tea Lounge on @foursquare! http://4sq.com/2YiXec10:12 AM May 5th via foursquareHiten Shah
hnshah

But which is which? Information content is subjective. When someone I follow tweets about what they ate for breakfast, it’s just noise to me. However, that person might have other followers who find breakfast updates genuinely interesting. Who knows? It’s not for me to say.

Unfortunately, there’s not much I can do with my preferences on Twitter since it’s an all-or-nothing platform. I follow person X, or I don’t. I get all of X’s tweets, or none. There’s no easy way to follow X and only receive the tweets that have value to me without also plowing through tons of noise.

Human flexibility. We rock the flock & swarm like birds & bees… but a solo human is also v. formidable. We’re ants AND we’re lions.9:20 AM May 15th via TweetieRobin Sloan
robinsloan

Here in Seattle they put cream cheese on their hot dogs. W-E-I-R-D. Alaska’s got nothing on this.4:08 PM May 13th via TweetieRobin Sloan
robinsloan

A core problem with Twitter is that signal and noise are inseparable in a single person’s tweet stream.

Being an entrepreneur is not the same as starting a company. http://bit.ly/ddpRHV1:06 PM May 13th via TweetDeckMatt Mireles
mattmireles

Run: 3mi down, 3mi to go. Feeling #awesome!4:42 PM May 14th via TweetieMatt Mireles
mattmireles

(Disclaimer: all of the above Twitterers are awesome and you should follow them.)

Anti-search

The solution to this noise problem is to filter out tweets on the client side.

A first approach is to simply filter out tweets by keyword. I think of this as anti-search: specify a keyword, and never see any updates containing that word.

Keyword filtering alone could probably solve 1/3 of the Twitter garbage problem by tossing out tweets containing location check-in URLs like 4sq.com or gowal.la. Filter out “lunch”, “beer”, and other daily-trivia keywords (along with “iPad” or other topics that become oversaturated) and the signal-to-noise ratio could likely be raised even higher. Hashtags could be useful as filtering keywords, too.

A second approach is to filter out tweets by regular expression. This is basically the same idea as keyword filtering, but a regular expression would allow more subtle patterns to be recognized.

Adding a layer of boolean logic on top of these filtering operations would make them even more useful by allowing constructs like “remove tweets containing A but not B”.

Filtering with annotations

A third approach is to filter out tweets by metadata. Twitter’s Annotations will allow extra information (metadata) to be embedded inside a tweet without modifying the contents of the tweet itself.

Imagine if authors annotated each tweet with a subject category. Then, a perfect filter would be as simple as saying “don’t show me tweets with category = lunch”. Or “don’t show me tweets with category = checkin”. And so on.

The actual process of annotating a tweet would have to be painless and require zero extra work. Otherwise, no one will do it. I hope some clever client app will solve this annotation problem.

More information with less noise

If, say, 50% of my Twitter stream is noise, then filtering all of it out means I could follow 2x as many people and get 2x as much information for the same time spent. That’s a great ROI in my book.

The ability to automatically filter individual tweet streams is a killer feature that would make Twitter many times more useful.

It’s surprising to me that this feature is not commonplace.

In a future post I’ll review the few Twitter clients that have filtering-out capabilities. Among the options I’ve found so far are Mixero and Filttr. If you have other suggestions, please leave them in the comments.

A Simulation of Angel Investing, Part 2

2010-04-28T00:00:00-07:00

This is a follow-up to my post A simulation of angel investing.

Several readers commented on Hacker News that my first stab at a simulation was misleading because it showed negative average returns for low deal sizes, when in fact expected returns should be not only positive but constant regardless of deal size.

They are right.

I had been using payoff as the random variable, but rate of return as the measured variable. The formula for rate of return (x^(1/t)-1) places the most weight on the zero-payoff case (where return = -1.0), so the simulation results were skewed towards negative expectations, especially for low values of D.

In this new post, I present a simplified and more accurate simulation of angel investing.

As before, my goal is to shed some light on this question:

How many angel investments are needed to make the combined payoff look attractive from an investment standpoint?

Revised simulation

I coded the following simulation in Python. View the revised source code here.

1. Create a pool of 10,000 different investors, each investing in D deals, with a fixed distribution of payoffs per deal. Randomly simulate each investor’s combined payoff, then compute the mean and standard deviation of all payoffs in the overall pool.

Note that this simulation just uses the straight payoff, not a calculated rate of return.

2. Assume all D deals are made at the same time and that the payoff occurs at the same time.

3. For each angel investment, assume the following distribution of payoffs. Note that this is the same distribution as in the previous post.

Prob.	Payoff
50%	0x	lose entire investment
20%	1x	get investment back
15%	3x
13%	10x
2%	20x

(source: Gabriel Weinberg’s angel investing scenario spreadsheet)

4. New in the revised model: Calculate the median of all payoffs for each deal size. This gives an idea of what kind of results were achieved by the typical (not average, but typical) investor.

5. New in the revised model: Calculate the probability that none of the D deals were hits, which I define as investments with >= 10x payoff. Since big hits are what angel investors are after, this no-hit probability measures the odds of striking out across the board. It is calculated as (probability of < 10x payoff)^D = 0.85^D.

Simulation results

Observations

- The expected, a.k.a. mean, payoff is roughly constant for all deal sizes.

- The typical, a.k.a. median, payoff is low for deal sizes less than 5, but stabilizes quickly after 5 deals. After 20 deals, the median payoff is nearly identical to the mean.

- An angel investor would have to do about 13 deals before the risk of total loss is reduced to a > 2 std dev event.

- After 10 deals there’s still a 20% chance of never getting a “hit” investment. To push that chance to below 5%, an angel investor would have to do about 20 deals.

My takeaway from this revised experiment:

Angel investors can expect favorable payoffs with only 10 deals, but it takes at least 20 investments to truly be safe.

Some ideas for future enhancements to the model:

- Add true Google-like payoffs to the distribution. Very low probability with extremely high payoff.

- Measure percentiles other than median. 25% and 75% percentiles, for example.

- Don’t assume deals are all made at the same time.

- Don’t assume payoffs are uncorrelated.

If you have any suggestions, please leave them in the comments below.

A Simulation of Angel Investing

2010-04-27T00:00:00-07:00

In angel investing, it’s the extreme distribution of payoffs that keeps things interesting.

If anything, it resembles buying a deep out-of-the-money call option, but with nonlinearity. If you win big you might find yourself in on the ground floor of the next Google or Facebook. That’s incredibly unlikely, but still possible.

More likely, you’ll end up with a solid 2x-5x return from a startup that grows into a viable long-term business. But most likely of all–by a long shot–you’ll lose your entire investment in another failed startup.

The most successful angels invest in a long series of deals over many years. They know that any one startup in isolation is a gamble, and to eventually hit a big return, an investor needs to draw repeatedly from the payoff distribution.

How many deals?

A discussion with Gabriel Weinberg on this topic piqued my curiosity about the relationship between the number of deals an angel invests in, and the shape of the payoff he or she can expect from that specific number of deals.

It’s clear that a single investment would have a terrible expectation and huge variance, but how about five deals? 20? 100?

How many angel investments are needed to make the combined payoff look attractive from an investment standpoint?

Monte Carlo simulation of angel investing

I coded the following simulation in Python. View the source code here.

1. Create a pool of 10,000 different investors, each investing in D deals, with a fixed time horizon and a fixed distribution of payoffs. Randomly simulate each investor’s total payoff, then compute the mean and standard deviation of the returns in the overall pool.

2. Assume all D deals are made at the same time and that the payoff occurs in 5 years. When computing returns, use the Internal Rate of Return (IRR) over those 5 years.

3. For each angel investment, assume the following distribution of payoffs:

Prob.	Payoff
50%	0x	lose entire investment
20%	1x	get investment back
15%	3x
13%	10x
2%	20x

(source: Gabriel Weinberg’s angel investing scenario spreadsheet)

Simulation results

- After the 4th deal, the expectation turns from negative to positive, but the variance is still huge.

- An investor would have to do 10 deals minimum to just have the expectation of breaking even. Meaning: losing money would be a > 1 standard deviation event. That’s still fairly bleak.

- After 20 deals, an angel investor cuts their risk to almost half of the 10-deal scenario while achieving an expectation that is very near to the best they can get. This looks a candidate for the sweet spot.

- Beyond 20 deals, the payoff statistics improve at a very slow rate. To reduce the risk to half of the 20-deal scenario, an investor would have to do about 80 deals.

- At 100 deals, the mean return reaches its unconditional value of 18.3% with a standard deviation of only 4.21%.

My takeaway from this experiment:

New angel investors should have the capital, time, and deal flow to support making at least 20 investments. Anything less, and the odds are against them.

» Continue reading: A simulation of angel investing, part 2