They’ve kindly given me permission to post it online and share it so go ahead and check it out here. (PDF Link) They give a history of the topic and discuss what many companies are doing now. In the conclusion they suggest that news sites should adopt hybrid subscription models.

The paper is a good qualitative treatment of the subject and a fresh take from some people not personally invested in the subject. This was a final paper for the class, and from what I know, none of the five team members have ties to or have worked in the industry.

I am officially a graduate of the University of Pennsylvania.

This infographic I made for the DP does a fair job of summing it up.

Abstract: This paper outlines a method for finding revenue maximizing mixed bundling prices for news websites. This can help better understand paid content strategies for online news content. Drawing on work in the field of bundling information goods, I apply a two-parameter model of consumer preferences to web site traffic data and a roughly estimated willingness-to-pay curve. We can then calculate revenues for different price points and find the optimal one for any given site. This method is applied to a sample of ten sites. At revenue maximizing prices, the majority of paid revenue for these sites comes from the sale of individual articles, rather than subscriptions. Site traffic showing highly loyal consumers is found to correlate with higher subscription prices. This model suggests that while it is possible for overall revenue to be higher with a paid content plan, total traffic will certainly fall.

It can be found online here in PDF form.

I’m mostly happy with the way it turned out, though there were a lot of compromises and broad assumptions needed to bring it to a finished product. There’s so much interesting material in this field, I wish I could spend a few more years studying it. I guess that’s what graduate school would be, if I ever decide to attend.

Special thanks go out to Aleks Jakulin for supporting and encouraging me in this work.

But meanwhile, I will blog in short-form, and a little more personally, at http://albertsun.posterous.com/ to keep things flowing.

This stuff is right up my alley, and I’m currently working on a senior thesis in the field and so I’ll try to extend Salmon’s analysis a little bit. Later on, I’ll take on Stray’s model as well.

Salmon’s Analysis

Let’s say a reader in a given period reads articles from the New York Times. Then suppose the New York Times sets the paywall after a consumer has read some articles. In order to read the article, the reader must pay a fee of . If is the value the reader gets from each article, then he will only pay the fee if . This is a good simple model synopsis.

Article Values are Different

Let be as before. The first issue that jumps out is that the value of any given article is not constant. The value of articles over a period varies, so let’s arrange them in order of value from highest to lowest.

Let be a monotonically decreasing sequence of article values for our reader, with . Then the reader gets value,

The reader would clearly choose to read the articles he values most first, and after that only pay the subscription if the rest of the articles he has yet to read are still valuable enough. Only if will the reader pay the fee.

But this is not quite right either. There’s no way for a reader to know ahead of time which articles are most valuable to him.

Predicting future value

Now, instead of ordering the values of articles from highest to lowest, let’s say that the value of articles our reader reads are drawn independently from a probability distribution. Let the value of articles be a random variable with a normal distribution and the average value of an article. are the value of the first article read, second article read, etc.

Let the period of time for which the reader pays be represented as , and the moment when the reader has read free articles and must choose whether or not to pay the fee be at time . Assume the reader reads articles at some constant rate throughout the entire period. Then .

Now the reader must predict what the value of articles he will read will be to determine whether or not he should pay the fee. Up to point , he has gotten value and average value per article of . is also the sample mean of the distribution.

Result

Our reader will choose to pay the fee if . As goes up, so does and as goes up, goes down.

There are some interesting suggestions from this. When the New York Times imposes the paywall, they should carefully monitor the rate at which people read its articles. Those that have a low rate would be ideally suited for targeted discounts. Also, since readers make their predictions based on past articles they’ve read, the ideal time to convert non-paying readers is right after a reader reads a series of good articles. If the Times can be subtle about dialing up and down , then they can exploit variance in article value to increase sales.

Further work

This analysis is of course still incomplete. Problems I still see with it.

• Knowing that you’ll only get a limited amount of articles for free will change a reader’s behavior. If they’re still uncertain about whether or not paying the fee will be worth it, they will more carefully pick which articles they read before time t. This will bias upwards, but push downwards. At time , there will also be a back-log of articles that would have been read but weren’t influencing the decision of whether to pay or not.
• How will the reader decide whether or not to read an article before time ? He’ll have to depend on the headline and a summary if available to make a prediction. Before actually reading the article, the reader will predict some value and after reading the article realize some value . This average spread will likely affect predictions of future value.
• As is, the model says decreasing and increasing leaves the reader’s decision of whether to buy unchanged. But as this becomes a strict paywall, which the gut says people would be less willing to pay for. Another factor in the reader’s decision of whether or not to pay is their confidence about their decision. The larger is the more confident they will be about their value prediction since the sample mean’s standard deviation will fall, as .
• Paywalls, as described by the New York Times and as currently implemented by the Financial Times and WSJ, are easily bypassed. This can be done either by spoofing the referrer header, or by clearing cookies. This avoidance could also be modeled in in some way.
• Letting people in for free if they come via social media or links from other sites screws everything up. I think this may turn out to be such a huge gaping hole in the paywall that they severely restrict it, but if they don’t there are several ways it can be modeled.
  You could divide articles between different distributions of those that are primarily found through social media and those that aren’t. The reader would choose whether or not to pay based on the value of those that aren’t. Alternately, an article’s ability to be found through social media could just affect its .
• Print subscribers get free access as well. In Salmon’s post he looks at , the difference between print subscriber’s fee and online subscribers. If this is less than the value of getting the print paper then the reader will choose the print subscription.
• What if users can choose between a short period, and a longer period with a discount? What does the renewal decision look like?

There are undoubtedly more things that can be done with this model. One of the most obvious is to try and figure out what and should be set to.

Finding good values for F and n

Since it’s reader’s will not have the same distribution for it would be theoretically ideal to pick values for and individually for every reader. Realistically, the New York Times probably shouldn’t be that opaque about their pricing as it would cause confusion and a negative reaction among readers.

If forced to pick a single price, it would be necessary to find the average value of articles for all readers. That’s what Stray did with his paywall simulation. However, part of the reason that simulation has such wild swings in revenue from relatively small changes is because many of the variables are dependent on each other. For example, the percentage of people who pay for a subscription does not stay constant when or change.

I’ll tackle this issue more in my next post.

Special Bonus! A pricing algorithm for the FT

This part might still be a bit half baked, but working backwards from the consumer’s decision, it seems possible to figure out a demand curve for each individual piece of content if enough data is available. Since the Financial Times already has a metered subscription plan, if they’ve been good about collecting user data they should have what’s necessary to do this. Here’s an outline of the method.

It requires some change of notation from the above.

Let be an article, and be a reader. We will now represent the value of an article to a reader as a mapping with representing to the value of article to reader . The functions and replace and as the fee and rate for reader . is as before.

Define the set such that reads before deciding whether or not to buy.

So our former equation becomes .

Rearranging, we get .

The left side of the above equation is the average value of an article that a reader reads before making the buying decision. So if does buy a subscription, we then know that the average value was at least the right side.

Now that we have an estimate of a given readers average value for content we want to estimate that value across all readers. For any given piece of content, some fixed , to determine its value we sum the average value for content of all readers who read before purchasing, and then divide by the total number of readers (who aren’t already subscribers) who’ve read .

Define, .

Equivalently, .

This function is an estimator of the average has for an article.

Now define the set such that reads before deciding whether or not to buy a subscription. This set is all non-subscribing readers that read article in the current period, whether or not they’ve ultimately paid for a subscription by the end of the period or not.

If we take for each in the set , we have a distribution of estimated values for article . That might look something like this.

Finally, to come up with a set value for a specific piece of content, we sum over the entire set and divide by the number of readers.

With this value, you can now derive a demand curve for the entire site. Or you can dynamically set prices based on what articles a reader has viewed before hitting the paywall.

Exciting stuff, if actually implemented.

If you think I’ve screwed up the math in some way, or if anything isn’t clear, please please let me know. The thoughts in this post are still very much a work in progress.

Unfortunately this is empirically very unpopular (in the US at least). As Greg Mankiw explains, it’s about signaling.
Social interactions aren’t just about exchanges of economic value, they’re an intricate dance of signaling to others, posturing to others, projecting an image, fitting in with the cool crowd, and all manner of things a high-schooler would be embarrassed to admit to.

The content we consume becomes part of that intricate social game of posturing and positioning that we all play to navigate our social spheres. And luckily for content publishers, this all can be exploited.

Social Content

2009 may be remembered as the year that Facebook and Twitter topped the search engines in sites’ referrer logs. Increasingly, people are turning back to their friends and acquaintances to point them to accurate information, interesting news and entertainment. Search has become the victim of spam and gaming of algorithms, leading to online social spaces becoming a more reliable way of finding what is good online. Fred Wilson gives a clear summary.

For me, social has become the definitive content discovery mechanism. As this happens, content begins to look less and less like a commodity. In search engine land, content is ruthlessly and algorithmically ranked and rated in value and relevance. No one has a personal relationship with a search engine.

Information with social context becomes part of our relationships with other people. And no one is better at persuading us to do things we might otherwise not do than our friends.

For more evidence that social interactions destroy rational economic thinking, one need look no further than luxury goods that people consume and enjoy more simply because they are expensive and can signal to others that they are of high status. Demand for these goods actually rises as they become more expensive. Or take the vicissitudes of fashion, which require people to swap out their wardrobes on a continuous basis in order to convey their social status to others.

Applications

How to tap into this economic irrationality? Get people to consume goods in a social space that displays their consumption to others. People do this on the subway by holding open the covers of their magazine or book. People do it online by sharing links to their favorite sources and stories.

More specifically. Publishers could erect a paywall on a site that takes advantage of these social tendencies by giving paying subscribers the ability to share content by whisking their friends and followers past the paywall. With customized links, or a custom link shortening tool that meters the number who use the link it would be possible to set up multiple tiers of sharing and charge more to let someone share with more people.

This also lets people band together and buy subscriptions. It sounds bad for the bottom line, but in the economic literature, its been shown that allowing this kind of sharing, can actually improve profits for a content distributor. What causes this counter intuitive result? In a paper by Bakos, Brynjolfsson and Lichtman, they explain that this kind of social sharing works to aggregate widely distributed consumer valuations for content and present a more favorably shaped demand curve to the producer.

Content producers continue to struggle with monetizing their products online, and it looks increasingly unlikely that advertising will be the solution because content sites operate at the level of “intent generation” not “intent harvesting”. Perhaps by making the consumption of content more of a social experience, producers will have more success.

Whether you like or dislike him, it’s time to stand up and recognize that Murdoch’s threat to pull News Corp sites from Google’s index has worked brilliantly.

Publisher unrest and the threat of a Bing deal and serious search engine competition on site indexing have pushed Google into a major concession. Google’s change to its First Click Free guideline is a bigger deal than many people realize. What appears to be a simple change in degree is actually a change in kind.

Google has now said that its okay with sites showing different content to its crawler than to a human following a search results link. There is no longer a guarantee that what shows on a search results page will actually be on the destination page. The Google search user experience will suffer slightly and publishers will now find it much easier to run a pay site.

A Little Background on First Click Free

I’ve seen First Click Free described by some bloggers as a Google “program” or “service”. It’s neither. It’s more accurate to call it a guideline or policy. Google has always taken a strict stance against “cloaking”, or showing different content to its crawler than to a human visitor. What First Click Free said to publishers of paysites was in effect that they had three options. (1) Opt out of indexing at all. (2) Let the crawler index all content, but direct a human reader to a sign-up page, and risk the wrath of Google, which could include de-indexing or ranking penalties. (3) Implement First Click Free, and check all incoming requests to see if they’re the Googlebot or have a Google referrer and show them the content for free. (This is what WSJ.com implemented)

Now with “First Five Clicks Free”, Google has given sites permission to not show a user the same content as the Googlebot sees after their fifth click.

First Click Free Created the Leaky Paywall

I had never understood the complaints about search engines “stealing” content that emanated from the top of News Corp. If anything, search engines were providing free advertising and new visitors to convert to paying subscribers. Pulling their sites from Google’s index wouldn’t hurt Google, and it wouldn’t help the site either. I thought that the Journal was allowing visitors from Google past the paywall voluntarily to increase traffic. Now its clear that the Journal was choosing between maintaining the loophole and violating Google’s rule against cloaking and risking losing Google derived traffic. In that context, the ire directed at Google makes much more sense.

Editors and staff at the WSJ are well aware of both the power of Google to drive traffic and visitors to the site, and the degree to which people were using it to circumvent their paywall. Every morning, an email report goes out to editors and staff detailing what search keywords were driving traffic to the site and what stories and trends are hot online. During my internship, compiling and writing this email report was one of my responsibilities. Visitors searching for the exact headlines of Journal stories often ranked among the top sources of Google referrer traffic.

That Google has so clearly and quickly reacted, means that some negotiating power is returning to the big publishers. Five free clicks per day is still probably too many to make them happy, though. But the more search share Bing gains, the more leverage publishers will have.

Predictions for the Future

I predict that we will soon see a future where major publishers will let search engines see and index the full text of a story, but show just a teaser and a “Purchase” button to users. In fact, paywalled sites could try it now, if they feel like playing chicken with Google. Would they actually follow through and penalize a sites ranking or de-index it? Especially for a site like the WSJ.com, it’s plausible that doing so would noticeably hurt the quality of web and news search results. If Bing doesn’t penalize a site for doing so, will their results look better in comparison?

By explicitly ignoring Google’s guidelines, publishers would throw the ball back into Google’s court to see how they’ll respond. “First Five Clicks” is a sign that Google may cave on this. My advice to Rupert Murdoch would be to patch that hole in the WSJ.com paywall (give away maybe one free click per day) and see what Google does.

Many of those papers remain surprisingly relevant today and anyone interested in the field would do well to go back and read them. I’ve been doing just that recently in preparation for my senior economics thesis.

One particularly good paper was written in 1999 by John Chung-I Chuang and Marvin A. Sirbu, two professors of Engineering and Public Policy. The paper, “Optimal Bundling Strategy for Digital Information Goods: Network Delivery of Articles and Subscriptions”, provides many interesting insights into recent discussions of the topic. In it, the authors come up with an optimal pricing model for access to academic journals online. Their method applies just as easily to news articles and websites.

Results

The first conclusion of the paper is that the optimal strategy is always to offer both site wide subscriptions AND a micropayments plan for sales of individual articles. This pricing strategy will (with reasonable assumptions) always be more profitable than solely offering one or the other.

A second indirect point is that visitor loyalty will determine not only how many visitors can be converted into paying visitors, but also what proportion of revenue will come from subscriptions versus individual sales. The more loyal visitors are, the greater the fraction of revenue that will come from subscriptions.

Sidenote: This paper assumes that the publisher is acting as a monopolist. The publisher’s offerings must be sufficiently differentiated from competitor’s products that consumers will not switch, and any switching that does occur is not taken into account. If switching does occur, then the assumption is that a competitor offers a similar mix of products. Thus, the proportion of subscription versus individual sales revenue does not change.

Theory

The theoretical underpinnings of this paper are in bundling theory. In bundling, we examine the problem of a publisher offering multiple goods (articles). A consumer places some value on each article. If the price of the article is below the value they place on it then they will purchase it. Likewise, for some bundle of articles, if the value the consumer places on the sum of their individual valuations is less than the price of the bundle, they will purchase it.

Over a subscription period where a publisher produces N-articles, then there are 2^N different sub-bundles to sell. These sub-bundles could include content grouped by category, author or any other distinguishing feature. However, it is extremely computationally difficult as N gets large. To simplify things, only the entire bundle, and sales of individual articles are considered.

Customer Preferences

In addition to these two conclusions, the paper also illustrates how little hard data is available in this field with which to do research. Their model describes the preferences of consumers as a distribution on two factors. Willingness-to-pay and percentage of articles valued. Their willingness-to-pay for their most valued article and the percentage of articles with non-zero value. Without hard data on the actual value readers place on articles in journals or on news websites, the study assumes a uniform distribution. Data on percentage of articles with non-zero value comes from a study by researchers King and Griffiths showing the distribution of the number of articles read in a Journal.

A good analogue to this survey data for content websites would be to use traffic data on visitor loyalty. How many pageviews per unique visitor does a site have? What’s the distribution of this statistic? Nielsen Online has begun to put together a new statistic for newspaper websites, session per user per month.

Pricing Strategy

Based on the data for academic journal readers, the authors calculate that the optimal price for a subscription should be approximately 10 times greater than that of an individual article. With this pricing strategy, the content producer’s revenue stream is well balanced with 56% from sale of individual articles and 44% from that of subscriptions.

Some publishers have started to speculate that their best hope of monetization may be with their most loyal visitors and not with ever higher traffic numbers. Still, much is up in the air.

A recent BCG survey begins the task of gathering the necessary data to make intelligent decisions about whether or not to charge for news. Still plenty more to do though.

More Data Needed

To apply a model similar to Chuang and Sirbu’s to news websites two datasets are required. One, is the survey or experimental data needed to find the distribution of consumers’ willingness-to-pay for a specific article. The other is a data set on visitor loyalty. If anyone knows of an existing data set for either of these, please let me know. I’m also in the process of gathering these data sets. If you want to share analytics from your news website to help my research, please let me know too.

I’ll be summarizing a few more of the papers and studies in the field and looking at other theoretical pricing models for digital content.

Both are stupidly easy to circumvent.

For WSJ.com, simply copy the headline of the paywalled article you want to read into Google and hit search. It’ll pop up as the first one and by following a search engine link, you skip over the paywall. For ft.com, once you’ve hit your limit, just clear your cookie from the site and keep on reading.

But despite how easy it is to get free access, lots of people pay anyways. They’re paying for the convenience of not having to use these work arounds. Where else are people willing to pay for convenience? Look at the proliferation of paid iPhone apps. Many (especially Twitter apps) just provide a nicer interface. Convenient.

When creating a premium product, make it more conveient and easier to use than the free one.

What would make a site more convenient? Alternately, what irritates you about a website currently that you might pay to avoid?

And since the marginal cost of distributing one more unit of news through the internet is essentially zero, news should be free.

In my last post I looked almost exclusively at the demand side of the equation. This time I’ll look at the supply side a little bit.

Marginal cost pricing is not a trivial objection to charging for online news, and I used to be firmly in the “information wants to be free” camp. But something clearly seems wrong about that. Information is so valuable I just can’t imagine that it could all be free.

1. Infrastructure Costs

Marginal cost isn’t quite zero. Most estimates say that Google is losing hundreds of millions of dollars a year running YouTube. The IT infrastructure to run a complex or high traffic site costs a lot of money.

2. Pricing Power

Profit maximization only makes price equal marginal cost if firm’s don’t have pricing power. (i.e. they are in perfect competition). And while a lot of news content might be in perfect competition, most people still have strong preferences about which publications they do and do not like. By widening the gap in perceived quality and value, publications can make it easier and easier for themselves to charge for content.

3. Measuring the Wrong Thing

Most significantly, number of pageviews isn’t the right quantity to consider. The right quantity should be some abstract measure of how many pageviews can be attracted to the site. What’s the difference?

For most goods economists look at, the company can produce and sell identical copies and as long as they keep reducing the price, people will keep buying more to more and more people. With news articles, that’s not true. No matter how cheap an article about something someone don’t care about is, they won’t buy it. And no matter how cheap a second copy of a news article someone has already read is, they won’t buy it.

When someone makes the argument that the marginal cost of digital content is zero, they are thinking of the marginal cost of one more person reading the content, one more pageview of a website, or one more copy of a single piece of content being distributed.

But that’s not at all the supply side decision being made by media companies. A media company will decide how large a staff to hire, and that in turn will determine how much content is produced each day. Cost has labor as an input, and the marginal cost curve is the standard J-shaped curve.

In turn, the more content produced and the better that content is, the larger an audience can be attracted that will want to view it. The first pageview is easy to get, the millionth or the ten millionth is much harder and more expensive to get.

Instead of looking at the price of distributing one more copy of an article, we look at the price of creating content that will attract one more pageview. (Either more content, or better content — so that either more people will read, or the same people will read more)

All of a sudden marginal cost isn’t zero. But what significance does this have?

Marginal cost above zero means there’s hope for paid content online! Since content has value to consumers AND the cost of supplying content is not zero, the equilibrium price of content need not be zero either!

It also means that content companies need to lose their single-minded focus on expanding audience. To make money from paid content, growing an audience can’t come at the expense of the ability to price the product.

I’ve been kicking this last idea around in my head for quite a while and I’m still not sure that I’m thinking about it the right way. It’s certainly not as rigorous as it could be, and I haven’t tried to draw up some mock numbers with it to see if modelling things this way will work. In the fall, I’ll be starting to research this topic more rigorously for my senior thesis.