Basecamp is used a lot in small and medium businesses, and I’ve heard that it’s becoming more common in niches of larger businesses. (It’s easy to put the relatively small monthly bills on corporate purchase cards.) It’s popularity means there’s a great population of potential customers that can start taking advantage of this new source of information.

(Inkling’s full post on how to implement prediction markets within Basecamp can be seen by clicking here.)

I’ve written before that I think there is a LOT of value in using prediction markets to forecast project milestones. (It’s generally done poorly, if it’s done at all.) One of my more popular posts according to Google Analytics is “RAG status – people lie, prediction markets don’t“. I dreamt of a day where prediction markets would be run on all project milestones, and the results of those markets would drive a project’s “RAG status”. That is now easily done with Inkling’s Basecamp widget.

There are probably still a few details to be ironed out with the actual implementation, but this is a great step forward for businesses that want better intelligence from their employees and partners. I look forward to hearing more from Inkling about how customers have taken advantage of this and if/how it’s worked for them.

From their blog post, here are two screenshots of what the screens look like before and after you trade:

The “trading” screen:

The results screen:

a

I’m a little partial to these guys, mainly because they come from my alma mater, the University of Michigan. The key people look to be a mix of entrepreneurial students and professionals. The company itself was started a year and a half ago, but has had success with early pilot projects.

In fact, three slides included in their “Learn more” online slideshow are quite powerful statements as to why prediction markets can be useful. They were predicting car sales in the winter of 2008/2009, during what was one of the most volatile months the industry has seen in many, many years. And the prediction market beat the internal forecast made at the beginning of the month, and the expert forecast made at the end of the month.

Here are the slides:

To recap, the prediction market beat the official GM forecast (made at the beginning of the month) easily, which isn’t hugely surprising considering the myopic nature of internal forecasting. But the prediction market also beat the Edmunds.com forecast. This is particularly interesting, as Edmunds would have had the opportunity to review almost the entire month’s news and data before making their forecast at the end of the month.

Examining the numbers

Let’s quickly quantify this error. Assume an average Chevrolet sells for $18,000. After dealer markup, assume that GM/Chevrolet receives $16,000 per vehicle.

Within the first week of November 2008, the prediction market would have warned Chevrolet that they were going to miss their revenue targets by $800 million in the Chevrolet division alone. And depending upon the exact product mix, this could have easily exceeded $1billion.

Now Chris can blather on about corporate prediction markets, but he’s simply wrong. Assume that even with three weeks’ early warning Chevrolet was only able to save 10% of that gap, it’s still $80million in savings. Even if a corporate prediction market for a giant company like GM cost $200,000 a year, that would still be a return on investment of 40,000 %. And again, that’s in the Chevrolet division alone. (Note: It would be a rare prediction market that cost $200k/year to run.)

Now not every problem should be solved by a prediction market. This is where management expertise comes in: are the errors large enough to warrant the cost of reducing those errors? But big problems with big numbers are often very suitable to address with a prediction market.

Summary

I’d like to wish good luck to the CrowdClarity team. It’s great to see Wolverine entrepreneurs working on prediction markets. There are more and more players each year in the corporate prediction market scene, but with case studies like this behind their belts, they’ll be well-placed to pick up some business.

a

Y Combinator is a very unique seed accelerator program. Startup companies get ~$15-20k in funding in return for ~6% equity, three months of intensive business and product advice, connections to mentors and potential advisors. Inkling Markets was in the second cohort of Y Combinator funded companies, for example.

My hypothesis was that a lot of people/organizations are starting seed accelerators without really examining the full scope of innovations they need to think about in order to achieve their goals. I wanted to take the opportunity to look into why entrepreneurs choose to go into a seed accelerator, why individuals choose to start a seed accelerator, and then propose a framework for designing new programs.

You can click here to view the post on my personal blog that has a bit longer executive summary. But I’ve embedded the paper and data sources below. I hope you find it interesting.

Copying Y Combinator

Appendix A – List of Seed Accelerators

Click here to view the list of seed accelerators. Only seed accelerator programs are listed; see main paper for details.

Appendix B – Example Seed Accelerator financial model

Appendix C – List of all companies founded by Seed Accelerators

OR

Click here to view the list via Google Docs.

a

Chris Masse over at Midas Oracle has recently generated a fair number comments on his post “Are IBM Smarter Cities prediction markets too smart for people?” As the commenters rightly point out, what IBM was doing was NOT a prediction market, but instead a polling system.

This is something I’ve been talking about since May 2008. When you try to use a prediction market to forecast the “best” ideas, you get a “Keynesian beauty contest.” To quote (via wikipedia):

It is not a case of choosing those [faces] that, to the best of one’s judgment, are really the prettiest, nor even those that average opinion genuinely thinks the prettiest. We have reached the third degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be. (Keynes, General Theory of Employment Interest and Money, 1936).

Starting from the wrong metaphor

Ideas in an organization are a long-term investment, where significant work and development is required before you can even get to the prototype stage of development. This is a very different concept than regular, known markets, such as the number of customers in the next quarter or “will me meet sales targets next year” which are asked in prediction markets. Prediction markets work well for the regular, known markets, because prediction markets involve liquid, easily-traded contracts.

Idea markets should be seen through the metaphor of venture capital. An individual sees lots of good ideas, but none of them are well developed. Just as a venture capitalist puts resources into ideas and those resources are locked-up and illiquid until the idea has proven its success or died, an individual in an idea market should see their votes as an illiquid, sunk cost.

The organization running the idea market then concentrates their resources where individuals concentrate their votes. But just like venture capital, this cannot be a one-time market. The organization needs to regularly run similar markets, say every quarter or every six months. If an idea gets resources but doesn’t make enough progress (or doesn’t look like it will be as effective as originally thought) the market will stop voting for it. As an idea makes good progress, it continues to receive resources for development. Finally, when an idea is implemented and made successful, all of the people that voted for it during its development will be rewarded. And just like venture capital, the earlier you contribute the more you are rewarded.

The key is that idea selection and development is long-term work, and thus when markets are used to forecast and help allocate resources, the market structure must match that long-term approach.

a

#1 – Trading UK Housing Prices

CityOdds, based in London, has recently opened a prediction market (currently marketed as a competition) to predict a UK housing market index. While this is a play-money market, there is a £10,000 first-place prize.

CityOdds runs both fantasy markets, as well as real-money markets on currencies and other commodities. Mike Chadney, the founder, is a good guy and experienced “City” man. Check out the housing market here: http://www.cityodds.com/hpitrading.html

#2 – Predictify shuts down

Predictify made a bit of a splash when Scott Adams used them to forecast how many sales of his (then) new book were going to be sold.

It seemed a bit of an odd company; venture-funded, but with people who had no background in prediction markets at all. They offered a small number of markets where accurate forecasting would win cash, while most markets were just for leaderboard position. (My trading was perhaps typical: I’d log it and only scan the markets where I could win cash and ignore the rest.)

Well, they’ve died. According to an announcement on their website:

Due to the tough economic climate, we are planning to cease operations and shut down the company in the near future. If you have an account balance of $20 or more, please visit your account page and enter your withdrawal information by 11:59pm on August 31, 2009 to receive payment.

We sincerely enjoyed building and operating Predictify, and we’re glad that you could be a part of it.

The Predictify Team

My question is this: what does this mean for Crowdcast?

Crowdcast has an absolutely fantastic team who have great experience in prediction markets. But can they thrive as a venture-funded company? I’m hopeful, but perhaps am more skeptical that there are simply enough customers truly interested in their level of solution. Particularly as so many companies need hand-holding and thus a lot of expensive people-time through the early stages of implementation, are the profits sufficient to satisfy investors?

I don’t know, but I do believe they’ve at least got the people to give it a go. (Strangely enough, it was Crowdcast’s founder Mat Fogarty who originally told me about Predictify well before their launched.)

a

From Emile of NewsFutures, here are three links and a quote from each:

The imagination market, Information Systems Frontiers (July 2007)

Participants were able to trade shares of technology ideas over the course of 3 weeks, resulting in the market identifying the “best” idea as the highest priced security. Our findings suggest that information markets for idea generation result in more ideas and more participants than traditional idea generation techniques; however, using markets to rank ideas may be no better than other methods of idea ranking.

Examining Trader Behavior in Idea Markets: An Implementation of GE’s Imagination Markets , Journal of Prediction Markets (April 2009)

In this experiment, we examine the behavior of traders that have submitted the ideas on the market and their influence on the market’s outcome. An idea’s submitter is clearly motivated to have his idea valued highly by the market, both by the funding given to the top idea as well as smaller prizes given to the top three ideas. In general, founders tended to buy their suggested ideas at prices above the volume-weighted-average-price (VWAP) in significant volumes. We discuss the implications and mitigation strategies. A survey of market participants yielded mixed results regarding the market’s effectiveness at ranking ideas but very positive results regarding the quality of ideas proposed.

GE Global Research blog (link here)

From the comments,
“GE Healthcare IT attempted an imagination market a few months ago to bring forth some new ideas for the company’s future. It left me with very strong mixed impressions: on one hand, it’s wonderful that we’re leveraging the power of technology for mass collaboration and idea sharing. On the other hand, I felt that the tool obfuscates the very opinions it seeks to gather by due to the inherent complexities of market behavior.

My primary objection is the use of the stock market paradigm to evaluate these ideas. Simply, I find it too abstract to be useful in gathering feedback about the quality of an idea. Stock investment is done by trying to predict *the change in collectively perceived value of something over time*. However, when dealing with ideas, neither the collectively perceived value, nor the change in this value over time are valuable metrics; you want people evaluating ideas based on *their opinions*, not based on their attempts to predict changes in the investment decisions of others over the course of a few weeks. These are static ideas isolated from one another, not evolving companies that interact. I think a stock market is an unnecessarily abstract, and distracting way to retrieve simple information: what do people think of these ideas?”

I’d be interested in hearing from people that disagree with this…

a

I’ve been blogging here about prediction markets for two and a half years. In that time, a lot of discussion has taken place both here and elsewhere about specific prediction market contracts; what happened on those markets and when.

One of the problems with prediction markets is that the information from them is not just valuable when it’s “live” but also after the market has finished and been closed out. It would be useful to go back and observe changes in markets over time, and even more valuable to compare markets to each other. Unfortunately, the data is largely useless to the various vendors and typically gets deleted after a certain period of time. (Though the real-money markets like InTrade do tend to protect their data as a valuable asset.)

I’ve recently found a solution that could prove quite useful to the prediction market industry: Timetric.

The Future

Timetric is essentially YouTube for time-series data. Their current library of charts (nearly 100k time-series) includes everything from exchange rates to crime data to Twitter replies. These can be manipulated, compared, layered on top of each other, and more. Their standard charts can be easily dropped into blog posts, and the raw data easily exported for manipulation elsewhere. (Full disclosure: I’m friends with the founders, three very smart PhD’s from Cambridge University.)

In my ideal future Timetric would be the home for all prediction market data. Users could compare how InTrade, NewsFutures, and Inkling performed over time on the 2008 election. Those political charts could be compared to economic data or virtually anything else to draw further conclusions. In fact, Timetric is now being used in Guardian blogs (a UK newspaper) to enhance coverage; an example is here.

This is an example of an embedded chart. (It doesn’t display correctly in Google Reader, so please click to see the original post.) Be sure to tick the “Multiple Axes” box for the full effect.

A year and a half ago at the London Prediction Markets conference a number of people talked about what the prediction market industry should do; one of the big ideas was a warehouse for old prediction market data. Now that warehouse doesn’t need to be built; Timetric has done it. All we need to do is fill it with data.

The Challenge for Prediction Market software vendors

My challenge to software vendors is to create feeds that can be easily imported into Timetric. Don’t throw away your data… give it away to someone who values it!

While it’s not explicity stated on their site, they can easily take RSS feeds (specifically Atom) of data. This means that as people trade on contracts the Timetric will be able to update data for the contract nearly automatically. Doing so will make prediction market data permanent, and widely available to academics and the public. It will enable individuals to do their own experimentation and potentially be a great tool for prediction market enthusiasts to debate the merits of methods and approaches for different markets.

Software vendors can also provide a simple CSV or XLS file with datetime information in the first column and values in the second column. While it won’t update like Atom feeds, it does provide the same data for easy import into Timetric.

So what prediction market software companies are open to creating an Atom feed of contract data?

What companies will provide data for a permanent archive?

a

“In God We Trust; all others must bring data.”

– W. Edwards Deming

There has been more discussion recently on Midas Oracle, here and other blogs about the value of enterprise prediction markets. Part of this is because we don’t have a good measurement of value, and partly because a lack of information.

When it comes to value, there’s one great way to determine if a prediction market is valued by a company: they keep using it! No matter what people “think” the value of an enterprise prediction market may be, if the actual customer is willing to pay for it, the tool is valued.

But another big part of why we’re even having this discussion is because there’s no clear perspective on what’s going on with the industry. Why? Think of the parable of the blind men and the elephant. (Quote from Wikipedia).

A group of blind men (or men in the dark) touch an elephant to learn what it is like. Each one touches a different part, but only one part, such as the side or the tusk. They then compare notes on what they felt, and learn they are in complete disagreement. The story is used to indicate that reality may be viewed differently depending upon one’s perspective, suggesting that what seems an absolute truth may be relative due to the deceptive nature of half-truths.

So the problem is, we’re all talking our of our collective a**es.

There’s no way we can talk intelligently as a community unless we have a shared understanding of what’s actually going on in the industry.

The measurement

I propose that we start by looking at two measurements: retention rate and customer growth rate. Retention rate measures how long a customer stays active. This should be a relatively good indication of the value a prediction market provides to a client; the longer they pay for it the more valuable it is to them!

Customer growth rate is exactly that, how quickly the industry is growing and finding new clients.

While rates won’t provide a total magnitude on the size of the industry, it should provide a good proxy of value to customers and growth of the customer base. It’s not the sum-total of what can be measured right now, but I think it’s a solid first step.

A proposal

I hereby make a public proposal. In order that the entire industry can talk about a common set of data, I volunteer to act as the point of contact for data aggregation. Enterprise prediction market vendors would only need to provide limited data, and would get in return a comparison of their own company’s statistics to the industry at large. The public would get the aggregated statistics (only, no company-specific details) on the retention rate and growth rate of the industry.

The information that would be required of the Enterprise Prediction Markets software vendors is:

A list of clients (who don’t have to be named, code names/numbers okay) and for each of those:

• Date (by quarter) when the client was first invoiced
• Date (by quarter) when the client was last invoiced

I believe that invoicing is a good proxy for measuring the start and end of a prediction market, but if a software vendor would like to use other measures that’s fine, as long as they fairly represent the start and end of a paid prediction marketplace.

Thoughts?

I’d be interested in your thoughts on this. So far I’ve gotten some solid interest from some of the software vendors, but am posting this here to gather some additional interest. (Yes, a little public pressure.)

I think that if we’re confident enough in the prediction market industry, we shouldn’t be afraid of the data!

PS- Because of a recent spate of comment spammers, I’m moderating all comments so you see a bit of a delay before your comment shows up here. But once I approve it, you won’t need to wait if/when you comment again.

a

Comments and Criticism

[Chris Masse]: “Number one, I don’t understand why information aggregation would be a “bottom-up” approach (as opposed to “top-down”). Our traders bring bits of information to the market —but these bits of information were originally produced by the traditional sources (news, political polls, political forecasters, opinion leaders, etc.). I don’t understand why this “bottom-up” metaphor would apply to the prediction markets.”

I have a few comments for this. First, I don’t know where Chris gets the idea that “bits of information were originally produced by the traditional sources.” Corporations aren’t trading on political markets where there are polls and expert opinions. They’re trading on things that matter to their company and to their industry. While what he mentions certainly applies to public prediction markets, it’s virtually irrelevant to corporate markets. Individual traders bring their judgement and the perspective from their place in the company and their personal history, which when combined with other employees in the company is very valuable indeed.

With the “bottom-up” metaphor, I was trying to show that forecasts are built from the views and opinions of individual employees… from the bottom-up. “Top-down” is how much of traditional forecasting is done: put data through a model at corporate HQ and generate a forecast which is then distributed throughout the company… from the top-down.

[Chris Masse]: “[trying to paraphrase me] EPMs are such a novelty, and the corporate forecasters such a bunch of retarded people, that it will take decades before commercial organizations get to adopt the prediction market tool.”

It’s not that corporate forecasters are retarded, just that prediction markets are completely different to anything they’ve ever encountered for forecasting. And like anything different, they’re generally going to be ignored. Note that virtually all of the corporate prediction market trials are NOT initiated by forecasters, they’re initiated by general managers who aren’t so directly tied into a specific forecasting tool world-view.

[Chris Masse]: “If enterprise prediction markets were such a revolutionary and powerful forecasting tool, it would have found a market already —just like the iPod, the iPhone, FaceBook or Twitter did.”

This goes to the heart of my post. The iPod became incredibly popular because people clearly understood what it did: it served the same purpose as a portable CD/tape player, but carried the equivalent of hundreds of CD’s instead! The iPhone is still just a smartphone; it’s just got a significantly better interface. All of these technologies became popular because they did the same things their predecessors did, but better. Prediction markets haven’t become as popular as they could have been- because they do the same thing (forecasting) differently.

[Chris Masse]: “The added accuracy of the enterprise prediction markets is marginal —and anyway does not fill the gap with omniscience (contrary to people’s expectations).”

This is where I think Chris unnecessarily limits himself to examples where there is “added accuracy.” There are a LOT of applications for prediction markets where little or no forecasting is currently done; the example I commonly use is forecasting project management milestones. I think it would be ideal if a management dashboard (RAG status) was created using only the inputs of prediction markets on the probability that a project would meet its next milestones!

Sure, in cases where prediction markets are “competing” with other forecasting mechanisms, such as for demand for products down to the individual SKU level, prediction markets may not be the best tool. The power of prediction markets really comes into play in situations that are difficult to forecast without a market.

[Chris Masse]: “In the context of a Fortune-500 company, which is of course much smaller than a country, the pool of potential active participants whose trading activity is sustained over time is quite tiny.”

I just want to point out again that in my research I found that a group as small as 16 people could generate calibrated forecasts.

[Medemi]: “In order to make good predictions one needs both approaches, bottom-up as well as top-down.”

I completely agree with this. Companies can’t live on prediction markets alone, but neither should they do all of their forecasting without prediction markets!

[Medemi]: “The problem is, this valuable information (from the experts in the field) and how the problems can be solved was not passed on the management. Why? Because they are not interested. The bottom-up approach is simply NON-EXISTENT.”

I disagree slightly. I agree that certain people in an organization are dis-interested, because they’re close enough to a problem that they think they can solve it and don’t want to hear any bad news. But I have also talked to senior managers that are very interested in using prediction markets for exactly this reason; they want to know if their project managers are telling them the truth! Unfortunately, these tend to be fairly senior people in a company, and it’s tough to get in contact with them.

Summary

I hope this clarifies my position; that prediction markets are a completely different way of approaching the problem of business forecasting, and should not be pitched or considered as a replacement technology, but as a powerful but complementary technology.

a

Core Issue

Traditional forecasting is done through highly analytical techniques using past data. Statistical measures are used to generate forecasts, with probability ranges. This industry is quite large, and is highly exacting.

Prediction markets take an orthogonal approach to traditional forecasting. Instead of a “top-down” approach where huge data sets are analyzed, prediction markets use a “bottom-up” approach that combine individuals’ forecasts.

The reason prediction markets haven’t been adopted widely is because they are a tool that approaches the forecasting problem from a completely different perspective.

An example – Enterprise Business Intelligence

I’ve recently been looking into the Enterprise Business Intelligence/Business Management industry, and came across what I think is a similar phenomenon. The vast majority of the industry is composed of massive analytical solutions from the likes of SAP, Oracle, IBM, etc. They are massive companies, and implementing a “solution” can easily take a year or more. Their clients design the system from a “top-down” perspective, determining from the outset what the processes and procedures are going to be.

But then there is software like Thingamy. Thingamy is the creation of Sig Rinde, a Norwegian living in the south of France. Instead of looking at enterprise business intelligence from the top-down, he has created software that approaches the problem from the bottom-up. Instead of establishing pre-defined processes (that may not even work or will be changed by the time the software is configured), Thingamy tracks emergent processes as they happen. It can start with a very small, hard-to-define process and then scales up as the business needs it.

While Thingamy has gotten some good press and attention over the years, it’s still a fairly small company. Again, I believe this is because it takes a fundamentally different approach to the problem compared to the rest of the current industry. Hugely different approaches cause cognitive dissonance, which slow adoption.

What does this mean?

There are new types of technologies that approach business problems from entirely different directions. Prediction markets is one of these technologies. Using PM’s means companies have to upset some of their current notions about how power and influence flow in a company, relying on “soft” information from lower-level employees. A different approach also means that in certain situations they’ll be clearly superior, but also that in other situations they won’t be. Traditional methods and thumb-rules for situations just don’t automatically work.

For example, prediction markets where there is a lot of public information (like election markets) may prove to integrate new news and information more quickly, but may not be quite as accurate as other methods in the final analysis. But where information is scarce (like some internal corporate forecasts), a prediction market may be ideal. In general, new ways of thinking have to be established to know when and where to use this new tool effectively. That’s why I believe prediction markets will take quite some time to see any sort of a spike in growth; expect a slow burn for a long time.

Quick note

Just a quick note for you all. I’m curious about how well Google’s AdSense can be used to monetize a blog, so I’m going to be running AdSense on this blog on a one-month trial. If you have any opinions on this, please feel free to e-mail me or comment below.

a