For example, here is a post worth reading on this topic by Rok Hrastnik.  There are some very good points in this post on why using a funnel to attribute media value is really a troubled idea.  I was flagged on this post because it has a quote from me that seems to support Rok’s thesis about the death of the funnel model and the related idea, “Direct Response Measurement is a Wet Dream”.   The quote is from a comment I made on a post by Avinash where we were discussing the value of sequential attribution models:

There are simply limits on what can be “proven” given various constraints, and that’s where experience and a certain amount of gut feel based on knowledge of customer kick in.  If you can’t measure it properly, just say so. So much damage has been done in this area by creating false confidence, especially around the value of sequential attribution models where people sit around and assign gut values to the steps.  Acting on faulty models is worse than having no information at all.

But none of this means the Funnel Model is dead, or that Direct Response Measurement overall is a Wet Dream.  What’s (hopefully) dead is  people using the funnel model inappropriately for tasks it was never designed for, in this case multi-step attribution of media value to goal achievement.  On the other hand, if this specific funnel use case is what Rok was coming after, I agree, because it didn’t make any sense to use a funnel model for this idea in the first place.

Let’s unpack these ideas

Funnel thinking is based on a relatively reliable model of human behavior, AIDA.  This model from human psychology does not specify tools, channels, or media.  It simply says that there is a path to purchase most humans follow.  That is:

A – Attention: (Awareness): attract the attention of the customer
I – Interest:  (Intent) promote advantages and benefits
D – Desire: convince customers the product will satisfy their needs
A – Action: lead customers towards taking action / purchace

Example:  I’m Aware of tons of products I would never buy.  There are lots of products I think are Interesting but I have no Desire for.  There’s a short list of products I Desire but have not Acted on.  The list of products in my head worthy of purchase consideration gets smaller and smaller at each stage of the AIDA model.  This is the funnel.

The AIDA funnel has not changed and it’s not dead.

It’s a model of human behavior, not media consumption.

Said another way, it’s not the tools, channels, or media that are a funnel, it’s the way humans process buying decisions that is a funnel, left side of chart below.  Media touches and impact, relative to the AIDA path, typically do not happen in a linear or funnel fashion, see right side of chart below, click to enlarge:

In a chaotic environment such as the one above, it’s near impossible to determine the contribution to end goal conversion of any one media touch or step in a multiple-step the sequence; there’s just too many uncontrolled variables.  I guess Rok knows this, because in a follow-up post, he presents his alternative view he calls the purchase cycle:

Yea, I know, looks like a funnel to me too.  In fact, it’s the AIDA model, right?  So, I think what Rok was saying in his original post is the using these funnel ideas for media attribution is screwed up, not the funnel idea itself.  And I would totally agree with that notion.

What Went Wrong

Where I think using a funnel model for attribution purposes starts to go off the rails is confusing measurement of  funnel state / position during  the sales cycle and attributing state to particular media.

These are two completely different ideas.  The first idea is simply a position, state or status with no value implied; it does not really matter “how” someone got to a state, just that they are there.  State tells you about what message might be most effective; where someone is in the process.   The second idea is brave but lacking data; given nothing but  a media touch in a particular sequence, let’s determine what value a touch contributed to end goal.  This idea really has nothing to do with funnel states at all.  Stir in an incomplete and inaccurate picture of all the media touches happening during the sales process (see Rok’s first post), and you have a real mess.

If what an analyst wants to achieve is similar to this second idea, funnel thinking is probably not a great model to use.  The best way to attack this idea is through media mix modeling.  Analysts still won’t be able to see the contribution of specific touches to value creation, but will be able to accurately measure the overall contribution of the different media types used in the mix to outcome.  The actual sequence or level of media exposures at an individual level is still not knowable  but a mix model can determine, in the aggregate, which media types and mixes drive optimal sales or other actions.

The fact many orgs don’t have the tools or institutional stamina to do media mix measurement properly should not result in people declaring they have measurement of these media step effects.  In fact, all they really have is a list of media exposures that is clearly an incomplete and broken picture, and guesses about the value creation at any step.

Hopefully, this approach is dead (or more likely dying) as well.

In fact, the whole topic of  attribution needs  a more complete discussion so analysts and marketers fully understand the benefits and pitfalls of various approaches to attribution and set themselves up for success in the future.  Those interested in this topic from a media / goal conversion perspective should check out my recent post for Econsultancy: Online attribution models: getting close.

By the way, I think this sequential touch data is fabulous for gaining a greater understanding of media interactions, it’s a great step forward in the evolution of digital analytics.  The vendors can’t keep people from abusing this data, and it can be used for some very creative testing ideas, which I cover in the Econsultancy post.

If analysts can’t measure these media contribution to goal effects properly, they should just say so and ask for the resources to measure them correctly.  There’s no shame in saying you just do not have the right tools or enough resources.  If you have to provide metrics, just be honest about what they do and don’t mean.  But please, don’t sit around and assign arbitrary values to exposures based on budgets or whatever else.  A list of events is not a media value measurement, and you hurt the marketing measurement cause when you just make stuff up.  And people wonder why analysts don’t get respect…

What You Can Do

If you have the capability to get a bit more sophisticated but are not ready for mix modeling or controlled testing, here’s a framework for approaching this media value attribution issue.

For Awareness, secure Awareness-generating media and measure the effectiveness of those media at generating Awareness using Awareness metrics.  Forget the rest of the funnel for these Awareness media, the next steps in the AIDA funnel are not the primary success metric of Awareness media.  If you are successful here, some percentage of the audience will move to the Interest stage.

Secure Interest-generating media and measure the effectiveness of those media at generating Interest using Interest metrics.  Forget the rest of the funnel for these Interest media, the next steps in the AIDA funnel are not the primary success metric of Interest media.

And so forth.  Those interested in more on this idea check out the Marketing Bands series here.   A similar approach can be taken for the entire Media / Marketing / Service effort, one that after initial goal achievement adds customer centricity as part of the marketing mission.  The financial results of such an approach are here.

I’m sure some people will say, “But Jim, one piece of media can have effects in different AIDA layers.  For example, a campaign designed to generate Awareness might also create some Interest, Desire, or Action (see the chart above).  How do you account for or measure these spillover effects using your approach?”

The correct answer is a media mix model.  Do you have a media mix model?  No?  Then I’m sorry, you’re just going to have to deal with an imperfect marketing measurement system, and realize there will be beneficial effects you cannot measure.  If what you can measure properly is successful, you can take comfort in knowing the actual results are probably even better!

Yes, the Marketing Bands framework is an imperfect approach, but without a mix model, it’s at least a realistic way to approach this marketing measurement challenge.  And it’s a whole lot better then fabricating attribution out of thin air using a list of sequential exposures, a model that is broken in so many obvious ways if you want to measure campaign effectiveness.

Direct response measurement is not a Dream or Wet for that matter, but it’s important to use it when it makes sense. Direct response metrics don’t take into account  all the vagaries of  media touches, real or imagined.  That’s true, but who said they were supposed to?  The advantage of direct response measurement is consistency and predictability, not accuracy.  If you want to measure Awareness, don’t use a direct response metric, use an Awareness metric.  Problem is, measuring Awareness properly is not easy and it’s expensive.  So sure, conversion has always been a shallow metric, but if it’s all you can afford, it’s pretty important.  Cost per order or campaign, preferred.  Better still is to use profit not cost, and where you really want to go is value created over time because profitability can change dramatically, see this example:  Freemium Customer Conversion.

Media Not the Only “Attribution” Play

The discussion above is just the beginning of the attribution journey if your definition of “Marketing” is larger than “Advertising”.  All of the above is just about getting the customer to one action or purchase.  For some companies, that is all they care about.  But what happens once a person is a “customer”, however your company defines it?  How do you measure and optimize the downstream relationship?

There are Marketing Bands for these downstream stages of the relationship too, as well as when customers begin the dis-engagement process.  As with acquisition, different approaches are most effective for each stage of the dis-engagement process, and those should also each be measured on their ability to do a specific job.

As the LifeCycle of the customer plays out across time, as the funnel continues to narrow down to the final defection of the customer and resulting terminal LifeTime Value, there are optimal contact approaches for each stage or Marketing Band that you will need to discover for your business (click to enlarge chart).

It all starts at Satisfaction with the first goal achievement.   Those not satisfied defect (left side of chart).  Those who are Satisfied usually end up in a General Information communication cycle, e.g. weekly newsletter.  This is pretty much where current customer marketing practices end; people remain satisfied for some time then quietly move down through the At Risk and Dormant stages to Defection without Marketers noticing and are purged from the list (small black arrows).

However, those looking to maximize the profitability of the business  ask a further attribution question: what is the source / cause of At Risk and Dormant stage customers?  Is it the acquisition method?  Initial customer experience?  Product quality relative to expectation?  Service / experience after initial goal achievement?

What is learned from attribution analysis of At Risk and Dormant customers is then used to:

1. Correct problems in the front end of the business that cause customers to defect in the first place
2. Develop customized communications (Behaviorally Targeted / Last Chance) to drive customers back up into a state of  Satisfaction where they enter the Lifecycle anew, e.g. they start opening weekly emails again (blue arrows)

The right message, to the right person, at the right time.

For examples of  a measurement and action framework for driving increased business value from these “lower funnel” customers, see the Measuring Engagement Series.  Those not having the required customer analysis capabilities to attack this area could upgrade their digital analytics platform or use a tag-based tool optimized for this purpose like the LifeCycle Grid feature offered by Listrak.

Folks, the Marketing Funnel is not dead, for most it’s yet to be discovered.  What’s dead (or should be) is the way people currently think about the Marketing Funnel and measure the implications of it.

For more on the future of attribution and how you can leverage it, come to my eMetrics Boston presentation or grab me at the show, I’d be glad to wrestle the topic to the floor with you!

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Marketing Funnel Not Dead,
Using Funnel Model for Attribution Is

Is someone who visits or purchases or comments one time really a customer, for the purposes of analyzing “centricity” ideas and concepts?  I think not.  Visit metrics simply don’t work for understanding these customer concepts, because by definition they unfold over time, not as single events.   Add in the fact most web activity is 1x in nature – even buyers – and you begin to realize that analyzing “traffic” yields very little in the way of “customer” insight.

From a Marketing perspective, hey, happy to have the 1x revenue, but these are interactions I’m not really excited about increasing spend on, knowing they will be a one-night stands.  This is especially true when you also know re-allocating some of the funds spent on the 90% 1x-ers to the other 10% could double company profits!

If you have followed my writings over the past 12 years, none of the above perspective is new.  What might be changing is this: more people in the online world are beginning to think the same way.

Now comes eConsultancy with a review of “major trends in marketing and technology” that lead to some key takeaways, which they outline here.  Marketers, they say, are moving towards customer-based metrics, and the organization has to be prepared for change.

I’ve been through this change several times before.   My experience is this: as an organization moves from campaign or funnel-based metrics to metrics based on customer value over time, old beliefs are shattered and new ones need to be accepted.

Take,  for example, the eConsultancy comment on attribution:

“Organizations have to be prepared for the changes that attribution should cause to media mix and budget.  If compensation packages are still tied to siloed spending, there will be resistance to adopting an attribution-based model.”

Wow, resistance?  Maybe full out internal warfare, if you ask me.  When people have paychecks tied to metrics, things can get ugly in a hurry if these same metrics turn out not to be in the best interest of the company (metrics are not authentic KPI’s).

Implications of this change

What could this change mean for web analysts?  In fact, much of your approach to the work, the thought process, the general concepts, do not change.  But as you might imagine, changing how success is defined from a visit-based value model to a customer-based value model can radically impact perception of “what works”.

Beneath what eConsultancy is saying, the driving principle for the tension, is that customer value metrics are the universal yardstick of success, because they can be used across any platform, media, and channel, allowing direct comparison of performance across programs in any silo.  Think about what that means.

Customer metrics are the end of silos hiding behind metrics customized to prove their own success and grab budget.  If the silo can’t move the needle at the customer metrics level, well, it doesn’t really matter what the metric championed by the silo says.  That silo-based, tortured to make us look good metric is now  obsolete.

As you might guess, this kind of thing creates considerable organizational tension.  When there is a move from “campaign” analytics to customer analytics, winning ideas can become losers, and quite often, losers become winners.

Take conversion, for example.  Did you know it’s common for campaigns with both high volume and high conversion rates to create the lowest value customers?  Are web analysts and surrounding silos ready for that, to accept the campaigns they are most proud of are in fact the poorest performing when the “value” goal posts are moved?

As the marketers move towards managing by customer metrics, conflicts like this will definitely surface.  Does management want short-term conversion / sales or long-term customer value / profit?  Who decides?  How will people be compensated?

This won’t happen overnight or next year or perhaps in 5 years time for many companies.  Leading companies with a proper analytical culture in place, especially those that face extreme competition, are already managing by customer value, because they must to survive.  Lots of others will follow slowly.  This will be especially true if Gartner is correct in the analysis mentioned here:

“In fact, industry watchers say today’s CMO must become the de-facto chief customer officer — or lose out.  CMOs have historically been the brand steward. This is an opportunity to be a customer steward”.  Love the subtitle on that article:  And If They Don’t, They’ll Be Relegated to Overseeing Promotions While Someone Else Takes Chief Customer Officer Role.

I’d guess “Relegated” is not a scenario most CMO’s are interested in…but is a correct assessment.  Why?

The nature of customer analysis is that it often ties very closely to company finances and quarterly reporting, meaning  natural alignment with strategic issues and on up to the C-Level folks.   A universal success metric like customer value that is valid and actionable across any program or platform creates the ability for the org to manage all aspects of the business using analytics, because apples can be  apples no matter where they are grown – Marketing, Customer Service, Merchandising, etc.  Also true:  it’s much easier to determine the source of bad apples, no matter which silo is generating them.

This enables the C-Level to finally start managing using analytics, rather than just nodding and saying the reports are interesting.  Since customer analysis often allows C-Level folks to act on problems before they become income and profit issues, customer analysis ideas and metrics are often quickly embraced once proven to be valid.

This is what many web analysts have wanted, right?  Respect and buy-in for analytics from the C-Level?

On a practical level, using the same yardstick to measure success across any silo means budgets can be reallocated not just within a silo, but across silos. For example, seeking the highest value, Marketing budget could be reallocated to Customer Service, if certain programs in Service are found to generate much higher customer value than some in Marketing.

What’s a (web) analyst to do?

The first question for web analysts is how they will handle the transition.  Here’s what you do not want to happen.  You are sitting in a conference with high level execs talking about how great a certain campaign is performing, and the customer analyst says,   “Actually, that campaign the web analytics folks are telling you is a success is one of the worst performing campaigns we have, in terms of customer value created”.

Ouch.  This  happened to me early in my Marketing career, when I concentrated on the results of campaigns as opposed to impact on customer value, which translates to profits and stock prices.  I know what it feels like, and it’s not fun.

So, if you’re not doing the customer analysis yourself, make sure you are in the loop with those folks.  Help them figure out how to properly integrate web data into the customer analysis stream, and get yourself an inside look at how it all works.   Don’t make this transition a battle between the web analysts and the customer analysts.

The second question is this: as marketers move towards customer analysis, who will be doing this customer analysis?  Are current web analysts interested in doing it?  Or will someone else – maybe in Finance, maybe a BI unit, maybe an outside agency – be providing this service to the org?

I’ll go out on a limb here and say most web analysts with more than 5 years experience are not only capable of doing customer analysis, they’d be really good at it.  Segments, experience paths, value creation – you got a handle on that?  Same general idea.

The biggest difference is simply the time frame of the analysis –  analysis is not over when the initial goal is achieved; the org wants to know what value is created downstream by the customer – 3 months later, 6 months later, a year later.  Does the customer come back and take action again?  What is the downstream value created by this campaign versus that campaign, this content versus that content, this product versus that product?

Same general ideas a web analyst works with all the time, with a longer “tail” on the measurement of success or failure.

Sure, at the tool level there can be a difference between web analytics and customer analysis , depending on which WA tool you are using – more advanced tools often have native capabilities for customer analysis.  But the hardest part, the analytical mindset, most web analysts have got that covered.  And on the tools, if your WA tool lacks the chops, we’re talking about simple customer database queries and spreadsheets to start – is that so hard?

I doubt it, for the WA folks I know.  If the analyst doesn’t have database query skills (as I do not, ’cause I’m a Marketing guy), they know people who do, either within the company or as contractors.  Customer analysis is not nearly as difficult as many people would like you to think it is, there is a middle ground between doing nothing on customer analysis and working with regression models, neural networks, and other “Big Data” promised lands.

Then, once you define the impact of using customer value as a universal yardstick for success across the silos, you have made the investment case for moving up to specialized tools that allow automation and discovery, then on to modeling and prediction.

So, I’m just checking – isn’t this a path web analysts have been asking for?  C-Level attention and follow-through, an org that respects analytical work and will “do something” based on the analysis?

Are you up for measuring, understanding, and driving business action using models and concepts that can make an even greater impact on the business than what you do now with web analytics?

I hope so.  We’re going to need you…

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Marketing to Focus on Customer. Analytics?

Has to be There

I find it really interesting that whenever there is a discussion of measuring the value of social media, there’s such a bias towards believing there is value in social beyond what can be properly measured.  See the comments following this post by Avinash for a good example.  Speculation is fine, but the confidence being expressed that a new tool or method will uncover a treasure trove of social media value seems un-scientific (as in scientific method) at best.

I don’t doubt there is some value in social media beyond what can be measured, as this has been the case for all marketing since marketing measurement began.  These measurement problems are not new to social either:  Marketing value created is often situational, it depends on the business model and environment.  What works in one situation may not work in another.

For example:

To small local businesses who do no other form of advertising, there is a huge amount of relative value to using social media, versus no advertising at all.  Social advertising is much better than none, and since it’s free, the incremental value created by (properly) using social is huge.  It’s also really easy to measure the impact and true value, since the baseline control is “no advertising”.  Lift, or actual net marketing performance, can be pretty obvious in his case.

On the other hand, many companies are running lots of advertising designed to create awareness, and the incremental value of social as a “media” may be close to zero for these companies, or at least less than the cost to analyze the true value of it.  Possible explanation:  Social events such as “Likes” or comments are simply representations or affirmations of awareness already created by other media, so by themselves, create little value.  In other words, events such as Likes might track the value of other media spending, but may not create much additional marketing value.

Why is this plausible?  It seems unlikely a person would “Like” a brand or product if they have not already experienced it, and are already a fan.  This means in the vast majority of cases, little incremental awareness / acquisition is created.  If this case is not true, if people “Like” a company even though they have no reason to (paid to Like?), then the problems with social marketing analysis go way beyond tools – the concept and data driving the analysis itself is flawed.

But if Like really means Like, the number of Likes or any other similar social events do not have as much to do with awareness as they do with the size of a loyal customer base, and are much more aligned with tracking the success of other awareness / acquisition campaigns.

Looking for Love in All the Wrong Places?

That all said, I believe there is some value being created in the acquisition / awareness area from social.  The problem seems to be this value, when measured, is quite a bit less than everyone expects.  So “the hunt for social value” seems never ending, with speculation and measurements contrived from thin air immensely  popular.  This missing value just has to be there, right?

The core problem is an old one: online value measurement definitions are all over the map, so it’s easy to claim value was created by simply inventing a new way to measure success.  I can’t wait for the day when established test and measurement standards (like using control groups) are adopted in the online space.

Meanwhile, I think it’s quite possible if people keep looking in the awareness / acquisition area, the value of social “beyond what can now be measured”, in many cases, is probably less than the cost of actually measuring it.

Why this approach?  Based on my experience, People are Social, Media are not.  So if you want to derive social value, you use people metrics, not media metrics.

Using this approach, I have unbridled optimism for the value of social.

But I won’t go as far as insisting value is there without measuring it properly first.  Because that’s not how science works.

See ya at eMetrics NYC!

P.S.  There’s lots of real experimental science out there on the effects of social media in the marketing space, have you reviewed it?

You will find this material to be a treasure trove of new ideas and proper methods worth pursuing in the social measurement space, examples here, here, here, here.  Get yourself a subscription to Marketing Science or if you are a WAA member, you can request copies of these fully documented social measurement experiments.

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

“Missing” Social Media Value

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: I purchased your book and have a few questions you can hopefully help me out with.

A: Thanks for that, and sure!

Q: We have 4 product lines and 2 of them are seasonal. i.e we have customers that year in year out purchase these items consistently but seasonally, for example, every spring and summer.  Then they are dormant for Fall and Winter.  Should I include these customers along with everyone else when doing an RFM segmentation?

A: Well, it kind of depends what you will using the RF(M) model for, what kinds of marketing programs will be activated by using the scores. If you know you have seasonal customers and their habit is to buy each year, AND you wish to aim retention or reactivation programs at them, I would be tempted to divide the customer base so that seasonal customers are their own segment.  Then run two RF(M)  models – one for the seasonals, and one for everyone else.

Q: If I include seasonal customers, and I run RFM say on a monthly basis, these seasonal customers will climb / fall drastically with time depending on the season, so it seems like it may complicate the scoring process.

A: Sure, and you could segment as I said above.  Or, you could run across a longer time frame, say across 2 – 3 years worth of data. This would “normalize” the two segments into one and take account of the seasonality in the scoring – perhaps be more representative of the business model.  However, the scores would become less sensitive due to the long time frame so the actions of customers less accurately predicted by the model.

Q: Can you provide me with some examples as to how segmentation is carried out?  Let’s say I being with RFM and all my customers are rated 5-5, 5-4, 4-5 etc.  What are the next steps, do we overlay with other characteristics like age, gender, etc?  Or are the 5-3 etc. our actual segments?

A: This goes back to what you want to use the RF(M) model for.  In the standard usage, each score will have roughly the same number of customers in it, those with higher scores will be more likely to respond to marketing and purchase, lower scores less likely.

Another way to say this is: since the lower scores are less responsive, they require higher value discounts or promotions to activate them.  So you can customize offers based on score, which will provide the maximum response for the minimum discount exposure.

Try testing different discount levels with different scores to see where you get maximum profit; an example of that kind of testing and math is here:

Using Recency to Drive Promotional Profit

The above assumes you are focused on response and profit, two of the more common objectives in data-based marketing ;)

However, other valuable information can be discovered using this framework, see chapter 20 about adding customer characteristics to RF(M) scores.  This is not needed if you are just concerned about response / profit for campaigns – the segments are the segments.

But if you’d like to know, for example, what kind of merchandise is appealing to 5-5 (ultra-best, highest responding) customers, you could run the scoring, take the 5-5’s, and then cross-tab with whatever else you have – find out what kind of merchandise they buy, what their sex is, etc. – any kind of data you have or can get on them.  This kind of work can help develop creative, decide what to feature on the home page, etc.

Q: Should we provide them a segment name like High Recent / Low Frequents, Low Frequents / Low Recents etc?  How do the other characteristics come into play with respect to the naming conventions, segmentation etc.  I keep reading of different segments like ‘Loyalists’, ‘Laggers’, are these defined through our subjectivity?

A: Generally yes, subjective labels; you may  find it is easier to communicate using labels rather than scores, so you could suggest that certain ranges of scores be called something like ‘Loyalists’ or ‘Laggers’.

The High Recent / Low Frequents segmentation is the quadrant approach from LifeCycle mapping, see the chapter 22 in the book.  This is an alternative way to use the RF(M) variables to map your customer base in a more Strategic way than RF(M) affords.

In other words, you could use RF(M) to score for Campaigns and the LifeCycle maps to report on your progress over time to management, since both approaches are based on the same variables – great way to “connect the dots” for manic-ment.  Also, RF(M) is a bit hard to visualize just using the scores; the LifeCycle maps allow you to plot campaign results in a very visual, easy to understand display of the data.

See these newsletters for a deeper explanation of Strategic versus Tactical use of RF scoring:

RFM versus LifeCycle Grids

LTV, RFM, LifeCycles – the Framework

Hope that helps!

Jim

===================

Q: How do I establish the optimal number of segments / buckets I should have when analyzing a customer database?  I was using a method of dividing them into deciles and assigning Decile 1 to Segment A, Decile 2 & 3 to Segment B, Deciles 4 – 6 to Segment c and Deciles 7 – 10 to Segment D.  However, the absolute number in each is not practical as I will not be able to manage that number of customers in the way I need to.

A: Well, it would help to know what Kind of business this is, but in general, the most effective place to break behavioral segments is where behavior changes.  You can force people into Deciles, which often does not create meaningful segments, or you can look at the data and see where there are “bulges” or changes that seem significant.

So, for example, look at this LifeCycle Grid:

Current / Potential Value Matrix

Note the low Current Value breaks of 1, 2, 3 because there are large groups of customers there.  Then, 4 – 9, because each of those has many fewer customers and their behavior is similar.  Same with 10 – 24, that’s a pretty good customer comparatively, and after 25 purchases, well, they’re crazy-good customers and behave similarly.

These cutoffs start out as somewhat arbitrary, but over time and testing you find out, for example, that a 9X buyer behaves more like a 5X buyer and not so much like a 10 – 24X buyer, so the cutoff is 9X.

Here’s another example:

Freemium Customer Conversion

The segments graphed in this post are actually deciles of 1 year customer value; but the data plot is of the first 14 weeks of weekly spending.  Note how there are clear inflection points where the  behavior changes, and this is where I would generally create the “boundaries” for a segment.  Doing it this way means the segmentation already has some type of powerful behavioral trait, making it by definition significant and “actionable”.

Hope that helps!

Jim

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Defining Behavioral Segments

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: We’ve been playing around with Recency / Frequency scoring in our customer email campaigns as described in your book.  To start, we’re targeting best customers who have stopped interacting with us.  I have just completed a piece of analysis that shows after one of these targeted emails:

1. Purchasers increased 22.9%
2. Transactions increased 69%
3. Revenue increased 71%

A: There you go!

Q: My concern is that what I am seeing is merely a seasonal effect – our revenue peaks in July and August.  So what I should have done is use a control group as you described in the book – which is what I am doing for the October Email.

A: Yep, that’s exactly what control groups are for – to strain out the noise of seasonality, other promotions, etc.  But don’t beat yourself up over it, nothing wrong with poking around and trying to figure out where the levers are first.

Q: Two questions:

1.  What statistical test do I use to demonstrate that the observed changes are not down to chance

2.  How big should my control group be – typically our cohort is 500-800 individuals

A: Good questions…

On a group that small, you are probably not going to get anything “statistically significant” without ruining your total profit, e.g. might have to use 50% in control.   If you have the leeway to do it, that’s what I would do.

On the other hand, in some cultures people will go bonkers over giving up sales to learn something really important.  OK, so take 10% as control and repeat it 3 times; if the results are stable then you have your proof.  Do another control every once and a while (every 6 months?) just to make sure it tracks.

Either way, you don’t really need statistics.

Practically, confidence is the likelihood a sample represents the population.  This can be a really useful idea when you are forced into very small sample sizes or the event is highly risky to repeat.  But here, if you are testing a really large slug of the population, confidence is less useful.  Or if you can repeat the event (because essentially, you are in control of it and it’s low risk), do you really need to force yourself through the screw of complying with the statistical math?  It’s like using a 727 for crop dusting, overkill for the situation, methinks.

If you were running a drug manufacturing line, statistical concepts like confidence and significance are absolutely valuable.  But for a marketing program?

That’s why I love the idea of “beefy controls” in start-up projects because I *do not* have to rely on statistics that the audience likely does not understand and provide room to question the results, e.g. “Yea, but what if the result is an outlier?”  Very appropriate in high risk situations, with giant populations and a lot of money on the line.  For this situation, perhaps not.  But, if you’d like to go that way, there’s lots of calculators on the web that let you play with some of the numbers anyway.

Here’s one, make sure to read the descriptions of the variables underneath the calculator:

http://www.surveysystem.com/sscalc.htm

Nice work on the core campaign idea, by the way!  Now we just have to tighten it up a bit…

(3 months later)

Q: We decided to tighten the targets and do a “best customer defection” email program. Basically, we look at customers who has an RFM score of 555 in the previous scoring period who have dropped out of that score.

A: Interesting!  So instead of targeting by  guessing the current score of a defecting best customer (say 355), you are looking for all customers who were formally best customers, regardless of current score.  This is a subtle difference, but much more of a LifeCycle approach and frankly why I prefer these kinds of ideas over “straight” RFM.

An example might be helpful.  Let’s say the acquisition folks run a huge new customer campaign in between the prior RFM scoring and the scoring done before your campaign drop.  A big inflow of new customers can artificially “force” certain groups of customers down in score – even though their own behavior has *not changed*.  In this case, the new score is not reflective of actual behavior, so increases noise in the system.

That’s the problem with the “Snapshot” or date-specific view of Customer State – it’s a single point without reference.  By using prior score, you are acknowledging behavior over time and the primary importance of the former State, as opposed to the current State – a Movie as opposed to a Snapshot.

In other words, from a  Marketing perspective, I’m more interested in the path they are taking through the LifeCycle than any particular point in time during the LifeCycle represented by a single RFM score.

Q: Good news on your advice.  We ran a 50% control (500 purchasers in each group) and the results really nailed the issue for us. The actual number of purchasers remained unchanged at 20% but Total Revenue and Average Spend increased by 40% compared to control.

(Jim’s Note: for those not following, a very precise target group of 1000 was split into 2 groups of 500.  One group received this  campaign, the other did not.  People who did not receive the campaign purchased at the same rate as people who did receive the campaign, but the people who received the campaign averaged 40% higher spend).

A: Awesome.  So what you are seeing is Customer State makes a huge difference in terms of what offers / timing can be most effective for this “Recently defecting best customers” cohort.  If I’m reading your numbers correctly, no lift in response versus control but a huge lift in revenue.

To me, that means these customers are early in the process of defection – still buying, but without a special treatment, slowing down the monthly spend.  After all, they are very Recent (former 5XX), so highly likely to purchase again, which is why lift in response was flat – they likely would have purchased anyway.

Not a bad time to hit them.  Offers to a very Recent State should focus on increasing order value, not generating response – you don’t want to spit into the wind, but go with the natural flow of the behavior.

In other words, these customers likely would have purchased anyway, but at lower price points if they had not received the campaign.  The common way this is addressed is with  “threshold” discounts – if average order is $50, then something like “$10 off any purchase over $50″ – test different thresholds to maximize profitability.

Looks like you gave them the right offer ;)

On the other hand, a straight discount to this specific best customer group – $10 off anything, and especially when their normal category of purchase is promoted to them – almost ensures that you will lose money.  Why?  Most of these customers would have bought at full price anyway, as demonstrated by equal buying activity whether the customer received the campaign or not.  So the discount turns into a loss versus no campaign at all.

Unfortunately, I see a lot of this exact type of campaign delivered to best customers because all customers get some version of the same offer.  “Hey Jim, we’re not sending the same message to every customer, we send different messages by segment”.  Sure, the copy and art are customized for different segments, but the segmentation is not by Customer State, so the offers are mismatched and suboptimal.

This is the value of using control groups; they drive understanding of Marketing concepts like opportunity costs and subsidy costs.  These two concepts are the reasons why ignoring Customer State is suboptimal: by not segmenting using State, you will get lower than possible profit or sales at most customers, depending on Customer State.

Had you not delivered a campaign tailored for prior Customer State, money would have been left on the table by way of lower order size.  And 40% Revenue lift sounds like it might have covered the cost of the campaign ;)

Q: We tried to run a Student T test on the results but our new statistician informed me that the distributions were not normal – so on her advice we ran a Wilcoxan Test which gave us a highly significantly p = 0.016

A: Oh, so you still went the stats route?  Well, the fact you HAVE a statistician tells me the culture there is more familiar with interpreting these ideas, so more power to you.

Glad it worked out and keep me informed on how things go downstream.

Jim

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Increase Profit Using Customer State

Before every eMetrics (I’ll be in San Fran teaching Basecamp, at the Gala, etc.), I try to ask myself, what is the most critical issue facing the web analyst community right now?  Then, at the show, I ask everyone I run into what they think about this issue.

There’s lots of issues to choose from.  Career path I think is a big area of discussion, given the mergers in the space and trend towards outsourcing.  Then there’s the “we don’t get no respect” thing; senior management doesn’t seem to listen / understand / act on the information provided.  And one of my favorites from the past is still out there, data torture - people being pressured to manipulate data to reach a predetermined analytical outcome.

But seems to me, more important at this juncture is trying to resolve why there is so much written about the importance of “the customer” but very little measurement at the customer level.  Think about it.  Customer experience, customer centricity, the entire social thing, it’s all about customers.

But when folks wants to trot out “proof” that this or that approach is the road to the promised land, they analyze impressions, visits, clicks, etc.  Visitor-level stuff.  Does that seem like the correct approach to you?  Seems to me, if you want to provide knowledge about customers, you should measure customers.

One thing we know is customers do express behaviors through a web interface that are not relevant to the future behavior and value of the customer.  One of the earliest and most widely publicized incidents of this was with Amazon gift purchases.  People went on and on about buying a gift from Amazon unrelated to their interests yet having that category Marketed to them relentlessly over time, even though they never purchased from the category again.

This problem was eventually solved by Amazon using Recency, a classic customer behavior metric – only more Recent behavior was used to make suggestions.  Recency is predictive; and lack of behavior is often just as important, if not  more important, than expressed behavior when trying to understand customers.  Unfortunately, most web analysts are trained to look for expressed behavior, not the lack of behavior.

Further, just because an event of some kind happens in the stream of web activity does not mean the event had any affect on the behavior of the customer.  Display impressions, searches, social interactions, all of it – how can you tell whether the event had any effect on the customer at all?  The only way is to measure at the customer level, for example, comparing the behavior of customers who were exposed to the events with customers who were not exposed.  Or, modeling different mixes of events against customer behavior over time, a “marketing mix” model of sorts, to stretch the idea.

Now some people are going to say. “But Jim, we don’t have web tool access to this data!” or “We don’t pass web data to the back end” and all manner of other related excuses, to which I would say,

“Where is your curiosity?”

Clearly, a unified database is best.  But just because your company can’t afford an advanced WA tool doesn’t mean you can’t do this.

I mean seriously, get a dump from the order management system into a spreadsheet.  Run a query against the CRM database.  Look up individual cases in the customer service or lead management systems.  This the way analysts make breakthroughs, how  business cases are built.  If key web data (campaign codes, logins, etc.) doesn’t make it into the back end, why?  If form data crosses over, how hard could it be to send a campaign code, login, or other critical data?  With proof, then pitch the advanced WA tool, or systems, processes, people, whatever you need to make it easier to analyze customer level data.

OK, so let’s hear all the reasons why it’s fine to draw customer-level conclusions using visit-level data, or why you can’t do the above, which I’m sure will include some of the following:

1.  My boss doesn’t care about customer-level data, ignorance is bliss, pseudo-analysis is OK

2.  I’m too busy learning very little about a lot of things instead of going deep on the most important stuff

3.  Shiny objects rule, so see #2 above

4.  I’m a web analyst, back-end data is not my thing

Other reasons?  What do you think?

Do you see the hypocrisy in claiming to understand customer behavior based on visit behavior?

Let’s talk about this at eMetrics San Fran…and Toronto too.

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

All Talk, No #Measure

Just because you pay $5 / CPM for impressions, does not mean they are worth $5 / CPM, does it?  Do people really still have that kind of mentality?  Is the price of the media equivalent to its value?

For example, I’m sure you have heard of multi-million dollar campaigns that generate very little lift in sales.  Happens frequently in fast food, for example.  What is the value of that media?  Is it the millions paid?

What really blows my mind about this approach is it’s so offline, so old school PR. Do the folks who put forth these kinds of models believe nothing has changed in 50 years?  What happened to the whole rap of online being “different”, that you can’t measure it like offline, blah blah.

Except when it’s convenient to do so?

If you want to know the value of a Facebook fan, why not measure the value of a Facebook fan?  Because it’s hard, and would require organizational discipline?  Too bad.   Substituting the kind of models used in the example above for actually measuring the value of a Facebook fan is misleading at the very best.

Make sense?  If you’re with me on this line of thought, let’s not stop here.  We should go ahead and question the value of awareness.

Now comes a better view, but likewise,  just because an event happens does not mean it has value or contributes value.  Looking at the recent post Social media and SEO massively undervalued: study we see a great data collection effort through TagMan but a similar premature jump as above:  that because an event occurs, it somehow must contribute value to the final outcome.  Again, this is a very old-school idea being applied to an environment where there really is no need to guess; set up a test and measure it.

I realize people get excited by the potential of new applications and tools, but have to wonder why folks are so willing to throw logic out the window and “find an answer” even if they have to torture the data to do so.  In many cases the reason is promotional, to sell a product or service, and hopefully this is pretty transparent to the reader.

One of the big problems at the root of all this is the lack of a common value reference point.  In other words, a standard that can be applied to compare the relative value of impressions, events, touches, opens, clicks, and so forth.

This standard exists, it’s called a controlled test.  In academic environments, where all the studies, results, and conclusions are peer-reviewed before they are published, it’s the gold standard for determining “the value of”.  This is a particularly important concept when you are dealing with interactivity; the results of controlled tests can be surprisingly different from common perceptions.

Perhaps it’s time for this community to require (OK, at least ask for?) the same level of transparency.  Count me in.

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

But What is an Impression Worth?

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: Thank you for creating this useful website!

A: You’re welcome!

Q: When figuring out retention rate for an annual or a 8 months life time cycle period, how do I pick the starting period?  Do I look at their first orders on a date?  Or I pick a time frame such as one month?

A: It depends on:

1. What kind of “retention” you are talking about, the definition, which is probably impacted by the audience for the data

2.  What you will do with the retention data, what kind of decisions will be made and actions be taken because of the data

You should always ask these questions above  when someone requests “retention data” – or any other kind of analysis, for that matter!  For example, there probably is a huge difference in what you would provide to the Board of Directors for an annual benchmark and what you would provide to Marketing people for executing campaigns.

In the first case, the data would probably be used to inform Strategic decision making, for example, should we change our product mix or approach to pricing given the market?  In the second case, the data would probably be used in a Tactical way, for example, to target new customers who are predicted to defect because of the campaign they responded to or the product they bought.

If providing data to the Board, “annual retention rate” would probably make the most sense (again, you should ask, what’s it for?). If that’s what they want, you would pick a starting period, probably aligned with the fiscal year (Jan – Dec?), and find out what percent of people who purchased Jan – Dec 2009 also purchased Jan – Dec 2010.

That’s the annual retention rate.  Useful information, perhaps leading to the Board requesting action of some kind.  But by itself, you really can’t “do” anything with this data, there’s no source or targeting information, there’s no customer value information.

However, if you segment by campaigns, product of initial purchase, price points, offers, or other actionable variables, the retention rate could be just about any formula, e.g. what is the retention rate:

a. Today, of people who made their first purchase in 2005?
b. End of 2009, of people who made their first purchase in 2005?
c. Today, of people who ever bought Product X as their first purchase?
d. Today, of people who bought Product X as their first purchase in 2009?
e. Today, of people who had at least 2 service calls in 2010, who became new customers in 2009, who used a 50% off promotion?

and so on.  Retention rate for anything tactical almost always requires and audience and time frame to be defined.

Q: You mention in your article, “Total number of customers” as the denominator for calculating the customer retention rate, do you mean the total customers at the end of the period?  Or those total customers came in on the first date of a fixed period?  Or the first fixed period that I’m observing?

A: Whatever definition is the correct definition depending on the need of the audience.  There is no standard, other than perhaps the very first one, the Strategic “reporting” idea of year over year retention.  This is commonly used in reporting to Wall Street, for example.

While discussing this particular idea of “customers”, one might encounter the common problem of not knowing the definition of a customer, at least in terms of retention.

When does the company declare a customer is no longer a customer?  Is a customer  “everyone” who has ever purchased?  If the company has been around 10 years, and you are calculating retention rate “today”, as in how many of these total customers purchased in the last year, you may find you have a very low number, one that won’t mean much to anybody, and is not actionable.

On the other hand, if your definition of “customer” includes a level of activity, for example, “must purchase at least twice, one of those purchases in the past 3 years”, now you are talking about a highly actionable kind of retention definition.  Why?

Because there is some hope that people who have purchased at least 2x (Frequency), at least once in the past 3 years (Recency) could actually still be customers, as opposed to defected customers.  If I am calculating a “serious” retention rate, something to be used to take Marketing action, or pay out bonuses, or revise policies, I want to measure against people who actually have some Potential Value, some Value to the company in the future.  That’s how I define a customer.  To me, there isn’t any point in calling someone a customer who is unlikely to contribute to profits in the future.

If you define as a customer “anyone who purchased over the past 10 years”, you just have a dead metric that really does not reflect the reality of what taking action might produce.  In other words, you are including people who are extremely unlikely to still be customers, so what’s the point of the “customer retention metric” you created?

Does the above help answer your question?

Q: I wasn’t expecting you to reply me so fast and in such detail!!!  Thank you so much!  I’m calculating this retention rate for marketing and your answer is very helpful for me!!!

A: Great!  So maybe ask them specifically how they want to look at it, and if they seem puzzled, suggest to them various options.

I can tell you from experience with businesses like yours is the buying behavior tends to peak early and you have to act quickly if you want to extend the lifecycle.  Perhaps not quite as time-critical given your “triple bottom line”, but probably not too different.

This argues for a tighter leash on the definition of a customer, perhaps purchased at least twice, one of those past 6 months.  You could also do 2x purchase, at least once in past 3 years, and compare, it will give them a feel for customer defection trend / rate.

The next step would be the Lifecycle map, which uses Recency and Frequency in a more actionable way, like this example.

Marketing people should be able to use this map to target specific groups of customers, e.g. purchased 4 – 9 times, but not in the past 90 days.  These are good customers who are in the process of defecting, and require special attention to keep them on board.

After all, the point of measuring retention is not retention rate itself, it’s about increasing the productivity and profitability of the business system.  Just as you can optimize for conversion, you can optimize for retention, and sometimes you discover they conflict.

For example, one company I worked with featured certain products on their home page because those products had a high conversion rate on visits to the home page; they had “optimized” the home page for this scenario.

However, a very quick and simple calculation showed these products generated customers  with terrible repeat purchase rates relative to just about every other product with volume.   A quick survey of these customers found out why the repeat purchase rates were so low – almost all customers disliked the product and thought the company deceived them.  Turns out the company “over-sold” the product – and that’s why the high conversion rates.

In another case, PPC campaigns had been optimized for conversion without regard to customer retention.  Under a budget crunch, the lowest converting campaigns were killed, but overall sales volume over the next 3 months dropped much more than the sales generated by these campaigns.

Reason?  These low converting campaigns generated the company’s very best customers in terms of 30-day, 90-day, 180-day value, while most of the highest converting campaigns generated low value, single purchase customers on the same time frames.

This kind of analysis is simply not that difficult to set up and execute, relative to the extreme amounts of value that can be created:

1.  Pass campaign codes / info with the order to the backend order processing.  If you are not doing this yet, start right now!

2.  Select a campaign, choose a time frame.  If you want to match up to financial statements (a good idea if you will be talking to C-Level folks), say January 2010.

3.  Grab all new customers who came in on Campaign X during Month Y – what is their average value 1, 3, 6, 12 months later?  This is a Lifecycle by Campaign analysis, similar to the LifeCycle map example mentioned above.

The new customer experience (channel, offer, product) is one of the most powerful predictors of future customer value, and the value of these new customers relative to each other tends to remain stable regardless of how many other generic campaigns (weekly email) you throw at the customer over time.

Across all campaigns, about 60 – 80% of these new customers will have the same value at 12 months they had at 1 month.  The question to answer, as with any optimization, is this: knowing the customer value created by these campaigns varies widely, are we allocating the acquisition spend optimally?  For example, are we spending 70% of the budget to generate  20% of the annual customer value?  Are we willing to pay more for clicks that generate new customers with 10X higher annual value?

Retention rate isn’t just some mystical number, retention rate quickly turns into profit dollars and can have incredible financial impact!

Jim

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Optimizing for Customer Value

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: First off, I very much appreciate you sharing all this wonderful content on your blog and conferences such as eMetrics.

A: Thanks for that!

Q: My question is a simple one, but I think the answer may be hard: When does a visitor “need” a coupon?  *Need* defined as: visitor would not have placed an order unless presented with the coupon.

A: Hmmm…methinks we’re going to have to define a few concepts and be clear on the goals to make sure we are nailing this down… visitor versus customer, sales versus profit, etc.  In other words, answer is not hard, but could be complex without defining context.

Q: It’s still a mystery to me why so many retailers seem more than willing to hand over all their margins to Groupon or give coupons to basically all visitors.  I am curious whether you would approach this question using  observational data (eg web analytics) or experiments (eg AB testing), or both.

A: Right – is a mystery to me too!

There are certain situations where this approach might be appropriate, but the problem with much web “marketing” (which often is really just advertising without much thought about marketing) is often there is success in a narrow or special situation.  Then the pundits jump on and say “if you’re not doing this you are stupid”, regardless of the business situation and / or without recognizing the special circumstances that are driving success.  This is all the real Marketing stuff people leave out; understanding why it works, under what circumstances, for which segments, involving which products.

When you don’t really understand what is happening and why, no learning takes place.  When no learning takes place, everything is a “new idea” and people are surprised when the outcome is different.

So, for example, if I was launching a brand new service business (restaurant) or a new product that is complex and won’t sell without trial (like yogurt that “naturally regulates your digestive system”), Groupon might be a slam dunk for a product launch.  No surprise here; coupons are often used to drive trial in product launches because the need is to reduce price resistance and drive sampling.

On the other hand, if my product or company is well known and I have tons of loyal customers, Groupon could generally be a financial disaster if you care about profits.  But if all you care about is response to the coupon, it could be a great success!  Because tons of people who would have bought at full price anyway get a huge discount and you get to sell the product below cost.  Awesome!

What do I mean by this, how can you have high response and low or negative profits?

Here is what I have seen over the years: whenever response rates are abnormally high, it means you have a high percentage of responders who would have bought anyway without the coupon.  This is seen over and over in database marketing, online and offline.  From a financial perspective, it means you have probably given up the coupon value with no benefit, a so called “subsidy cost”.

How do you prove this is happening in a promotion?  If you want to really look into and prove these effects, first examine the percentage of response that is from current customers.  If it’s high, that’s the first clue the discount is cannibalistic, not incremental.

If you want to quantify this subsidy cost a bit more is a relatively simple way, take the customer redeemers as a group and look at their average sales for a few months prior to the coupon promotion, during the promotion, and a few months after the promotion.  Often what  you will see is their spending behavior changed very little during the promotion.

For example, let’s say the coupon is 50% off.  The monthly net spending sequence over time might look similar to this:

2 months prior: $100
1 month prior: $100
Promotion month: $50
1 month after: $100
2 months after: $100

This shows the customers redeeming the 50% off coupon did not change their behavior at all; they simply took the discount and bought what they would buy anyway.  Meaning, the coupon cost is a real cost to the bottom line with no offsetting incremental profit.

Bottom line, for every response you lost $50 in sales plus the cost of the campaign, even though you had tons of responses and sales from those responses.  If it was a large campaign, your overall sales for the month net of discounts probably dropped.  Financially, if your cost of goods is 50%, you gave up $25 in profit for every response, minus the per response cost of the campaign.

And, the above behavior is most likely to occur with best, most active customers!  Across all redeemers, you might get $60 or so instead of $50 during the promotion month, but you are still losing money on every redemption – the higher the response, the more money is lost!

This is one big difference between Advertising and Marketing.  Marketing goes beyond Advertising, wants to understand the relationship of specific products to segments of customers, how pricing and modes of distribution affect this relationship, and the profitability of the relationship.

So, with that backdrop, let’s try the question:

Q: When does a visitor “need” a coupon?

A: If I take your question literally, there is a concept in Marketing called coupon proneness, and it’s the classic definition of “needs a coupon”.  Essentially, it means the more coupons you give people the less likely they are to buy without one. If you can imagine what this looks like over time, it’s margin erosion hell.  It’s taking the example above, where no incremental profits were generated, and ensuring it will happen time and time again.

From a Brand perspective, always offering coupons means you are teaching people your prices are too high, or there is a tangible reason your company has to “beg” for sales (implies poor service or quality).  Either way, the outcome is not so good for Brand trust and any evangelism that might result.

The exception to the above is among the “never pay full price” segment, who don’t buy anything without a discount / coupon.  From this segment, you get the benefit (?) of your coupon offers being spread all over the web, attracting many other “never pay full price” customers who generally have negative net values to the company.  Great, huh?

The end result of this pattern is horrible customer loyalty, margin erosion among current customers, and lots of new customers that are 1x buyers.  This means you have to spend *even more* on Advertising to constantly chase new 1x customers, while at the same time your margins in the current customer base are being consistently eroded.

Certainly not an optimized system!

Some people will argue the “extra sales” they get are worth the price of encouraging the above behavior.  But there are not too many businesses that put sales in the bank, what they put in the bank are profits.  So this is very short-term thinking and in fact, you find a lot of businesses that follow this model perform very poorly financially.

So, you ask, when would a visitor “not have placed an order unless presented with the coupon”?  The answer is this: when you have “presented” a coupon before, and the more often you have done this, the less likely they are to buy *until* you present one.

Sure, you could use A/B testing, but it’s not hard to guess what you will find – when you present a coupon, more people buy.  Duh, that’s Advertising, right?  But that’s optimizing for conversion, not for profits, and conversion can’t be deposited in the bank any more than Sales can be.  You have to go further.

For example, if you had the capability to recognize purchase or visit patterns among visitors, you could segment by these behaviors and present coupons on the site only when they were likely to have an incremental rather than cannibalistic outcome.  For example:

1. A new visitor who becomes a repeat visitor X times but does not buy

2. A current customer who has not purchased in over X weeks

and so forth.  You could test for “X” and optimize for highest profitability if you also ran a “null” control group  – where if A = coupon, B = no coupon.  Then look for incremental sales  behavior or “lift” from those offered the coupon versus those not offered a coupon, and run out the profit and loss.

Of course there are other scenarios, mainly current customer comes to your site because  you sent them a coupon, as opposed to presented one on the site.  Not sure if you were including that in your question, but I took your meaning literally.

The scenario with subsidy costs when sending customers a coupon is basically the same as the example above, except you control which customers get what coupon values or if they get a coupon at all.  More info on executing and measuring in that scenario for customers here, and for an example of a company putting this approach into practice, see here.  You can really drive higher profits by doing this correctly.

Jim

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

When Does a Visitor Need a Coupon?

Want to see the answers to previous questions?  Here’s the blog archive; the pre-blog newsletter archives are here.

Q: I was wondering if you’ve done any work with, or given thought to, companies who have a cloud based Freemium business model?

Should they be tracking usage (or anything) at the free level?  Should they be tracking usage at the paid level?  I’m sure defection rates are a big problem, but I’m wondering how many focus on engagement thru mass marketing versus trying to keep what they’ve got, or influence the free users to make the leap to paid.  Any thoughts on this?  Maybe you could do a blog post on it.  It seems like a good fit with your brand of analysis but I’m just starting to think it through…

A: I just finished an analysis that’s a good example of this problem.  Behavior during the Freemium period can predict who is highly likely to become a paying customer, who will need marketing efforts like additional sampling / package discounts, and who will not become a customer no matter what you do.

So the answer is you need both, analysis of paid and free.  But in particular, what you need to do is understand the transition from free to paid by comparing the behavior of known converters versus non-converters over time, preferably using events that create value for customers.

Typically the differences will be volume / persistence related, generically, low Recency high Frequency.  Also, likely converters to paid will tend to use a wider variety of features consistently.  In the analysis, the question to answer is which of these value-creating behaviors is predictive of becoming paid?

Said another way, you tend to see a fairly fast drop-off among *all* new Freemium customers after the initial burst of activity, but the ones that are not going to stick tend to drop off even faster in their activity.  Then, there is a “bounce” in activity where the ones who are most likely to end up as paid begin to cycle behavior more quickly and begin to use more features, and the others simply drift off the map, with no “bounce” as Recency becomes extended.

Classic LifeCycle analysis – customers tell you what they will become in the future by what they do today.  Having the very detailed behavioral information typically seen with interactivity just multiplies the ability to do this kind of prediction.  More on the Freemium model, including determining appropriate cost to acquire, is here.

Q: Do the standard analytics packages allow a business to look back at the “free” behavior of paid subscribers?  I’m thinking of Freemium cloud based solutions and how they would track this.  Do products like Crazy Egg get you there or do you really need something more sophisticated to do this kind of analysis?

A: I’ve never used Crazy Egg so I don’t know about that one specifically, but in general you can do quite a lot with the basic tools that support customizable segmentation.  The challenge with going that way is you have to be super-technical with the implementation to capture important event data points, you have to create many different segments, and then the killer problem – you can’t “re-analyze” a different approach with these tools, for the most part.  If that’s what you mean by “look back”, it’s highly unlikely you could accomplish what you need to do.

So it’s possible, but these tools are not really designed for “behavior over time” work and certainly don’t allow for much “exploration” of the data – any change in analytical approach is likely to be a “going forward” type of measurement, not looking back.  So there would be lots of iteration before you even knew if the analytics set-up was correct or what events are meaningful.  In other words, it’s possible but could waste a LOT of time.

I’d much rather find the system that contains the key elements of activity – when did they sign up, what features are they signed up for, when did they add other features.  This data probably resides in whatever system manages the account.  Dump that data off into a spreadsheet or database, try to figure out what’s meaningful, look for correlations.

Then, once you have a grip on some solid ideas, then maybe you go into the front end and try to align traffic and behavior with known “events” that seem to predict upgrade to pay, if that’s what the mission is.

Otherwise, you will be setting up a ton of tracking on all kinds of events not knowing what is meaningful, and then dealing with a really poor interface for the analysis of those events.

The other way to go, of course, is to use one of the advanced web analytics tools, which sit on real databases and can be queried.  But assuming that’s not an option, I would try to look for hard data points in the backend first, then knowing key behaviors, look for what might cause those behaviors in the traffic side.

Example

Let’s say you have a project management application that has a 60-day free trial then converts to paid.  Value is created for the customer when they use the functionality of the app – say create project, comment on project, upload file, or any other actions you deem critical.  “Traffic” in a situation like this may be only marginally indicative of value creation; rising activity could even be a negative indicator (frustration with application).

So, you want to create a situation where you can analyze these important behavioral events by account, and (ideally) you want to know the source of the account creation – campaign code, referrer, etc. That’s all you need for data, simple table, maybe a dozen columns.

Then, compare average account that converts to paid with average account opened at the same time as the converters but does not convert, over the 60-days before trial end.  For each of converting and non-converting, aggregate each of key events by week, divide by number of accounts to get average behavior per account, and you would have 8 weekly average data points for each of the events, both for non-converting and the converting accounts.  Maybe a dozen simple line graphs with 8 weekly data points, one set for accounts that paid, one set for accounts that did not.

Converting and non-converting graphs should look different for some variables.  Both will typically start out with high levels of activity, then for some variables you will see them diverge.  This not only predicts which variables affect conversion, but reveals to you the best time during the 60-days to intervene with surveys, help, or other marketing programs to re-engage the accounts that appear to be headed for defection.  If you have campaign data, also which campaigns tend to create accounts that convert and which don’t.

One of the event graphs may look to be more predictive than the others, with abrupt changes in direction going into the conversion event.  For example, perhaps it will look like this:

(Click pic for larger image)

This is the behavior of 10 different 1st year spend levels (deciles) over the first 14 weeks of their Life, engaging in an event that creates value for them.  The dark blue line represents average top spender.  Note how for top spenders, the profile is quite different.  The graph tells you that by week 4 or so, you can probably predict who will become a best customer and who will need intervention based on this activity.

You can run this kind of event profile for any variable – events, campaigns, etc. as long as you know complete / non-complete goal or end value of the customer.  In your case, since the goal outcome is binary, there would be 2 lines instead of the 10 spending deciles above: converters versus non-converters.  Create a converter versus non-converter chart for each key activity variable (create project, comment on project, upload file, or any other actions you deem critical), and look for this kind of divergence.

Drilling down more deeply by excluding all but 3 lines so we can see the behavior “in the middle”, we find some interesting patterns:

(Click pic for larger image)

Here, we see patterns that provide clues to the testing targets one might want to address to see if “middle” customers could be turned into better customers.  The blue segment, showing a series of higher highs and higher lows after it “bottoms” for this behavior, is most likely to benefit from intervention of some kind.  The pink segment looked promising, but then put in lower highs and lower lows – these customers lose momentum quickly and have trouble self-sustaining.  The yellow segment was never really in the game at all.

Yes, the comparison to stock market charting is intentional!  It’s an expression of group behavior.

If I had to pick the segment with the best potential, I would try the blue segment first, and the data points could be used for automated triggering of different types of campaigns. For example, “If by week 4 activity for Variable X  falls below 60, trigger Campaign A.  Then if by week 11 activity for Variable X is not above 40, trigger Campaign B.”  Remember, these are averages, so not all customers in the segment are below threshold.  The idea is to target a specific behavior with a specific message.

Just to be clear, you don’t need the goal value of the customer to put a model into practice, only to prove the initial model – certain patterns in behavior predict high value customers.  Once you know the end value of  customers – convert or not, monetary value, any goal – you can run the LifeCycle movie “backwards” like the charts above and find out which early  behaviors are predictive of high and low value customers.

If you want to go further, you could show these graphs and data to a modeler and see if they can create a more precise mathematical model, which can be developed much more quickly with this kind of evidence to review.

Once you fully understand what this LifeCycle landscape looks like, THEN you could go back and instrument the web site and analytical tool to monitor some version of this data in a more automated way.  But trying to guess what’s going to be important beforehand and work through a study like this using a vanilla web analytics tool is the very, very long way to get where you need to go!

Jim

Have a question on Customer Valuation, Retention, Loyalty, or Defection? Go ahead and send it to me here. If on the topic above, you can leave a comment on the post:

Freemium Customer Conversion