Comparing Analytics Programs - Ask & Ye Shall Receive

Way back in November of last year, I asked if there was some worthy soul out amongst the Internet Marketing world who would take on the monstrous task of gathering truly comparable data from the major analytics providers simultaneously, side-by-side, on several websites and publishing the findings.

Well... Ask & Ye Shall Receive.

Stone Temple Consulting's brilliant Eric Enge took up the mantle and with some help from industry brethren, has succeeded in delivering what is certainly the most comprehensive, impressive peek under the hoods of the analytics providers to date. From their interim report, released early this morning, comes:

1. An analysis of how the user deletion / non acceptance rates of third party cookies and first party cookies differ.
2. Comparative data showing:
   • Visitors
   • Unique Visitors
   • Page Views
   • Specific segments as defined per site for 2 sites
3. An analysis of the comparative data and discussion of the following topics:
   • What the numbers tell us
   • Range of results
   • Does one package always report lower numbers than the others?
   • Does one package always report higher numbers than the others?

I'm more than a little impressed, and we should all be very thankful for this information. My big takeaways:

• WebSideStory's HBX Analytics severely under-reports visitors, uniques and page views when compared to all the other major analytics packages (using either 1st or 3rd party cookies)
• The third party cookie deletion / non-acceptance rate exceeds the first party cookie deletion / non-acceptance rate by a little more than 13%
• Google, Indextools & Unica provided the most "average" counts of visitors, uniques & page views (with Clicktracks close behind)
• The final report is going to rock.

This is the kind of data you can use to make a smart decision not only about which analytics providers to choose, but about how to interpret data when switching from one to another and how to interpret competitive data or data provided by a firm before a sale or partnership offer. It's simply invaluable.

Go read the report, then salivate with me for the final version. It feels so good not to be the only one obsessed with statistics and accuracy :)