When I think about these changes, I find myself coming to the undeniable conclusion that our industry is undergoing a radical transformation. More companies than ever are paying attention to digital measurement, and despite my disbelief in Forrester’s numbers, an increasing number of these companies are forging a smart, focused digital measurement strategy. At the X Change, at Emetrics, and at Web Analytics Wednesday events around the world there is more and more evidence that this wonderful sector I call “home” is really starting to grow up.

And we’re just getting started.

If you pay close attention to the marketing you see from Omniture, WebTrends, Unica, Coremetrics, and the other “for fee” vendors you’ve surely noticed a dramatic change recently. Nobody is talking about web analytics anymore; the entire focus has become one of systems integration, multichannel data analysis, and cross-channel analytics.

All the sudden web analytics is starting to sound like, gasp, business and customer intelligence.

Eek.

Since it’s late and since this post will be over-shadowed by the hype around Google Analytics releasing more “stuff” on Tuesday I’ll cut right to the chase: I believe that we are (finally) on the cusp of a profound revolution in web analytics and that the availability of third-generation web analytics technologies will finally get digital measurement the seat at the table we’ve been fighting to get for years.

Statistics, people … statistics and modeling, predictive analytics based on web data, true forecasting, and true analytical competition for the online channel. Yahoo’s use of confidence intervals when presenting demographic data and the application of statistical models in Google’s new “Analytics Intelligence” feature are just the beginning. As an industry it’s time to stop fearing math and embrace analytical sciences that have been around for longer than many of us have been alive. It’s time to stop grousing about how bad the data is and actually do something about it.

Do I have your attention? Good.

Thanks to the generosity of the kind folks at SAS I have a nicely formatted white paper that is now available for download titled “The Coming Revolution in Web Analytics.” Just so you can see if you might be interested here is the Executive Summary from the document:

“Forrester Research estimates the market for web analytics will be roughly US $431 million in the U.S. in 2009, growing at a rate of 17% between now and 2014.  Gartner reports that the global market for analytics applications, performance management, and business intelligence solutions was US $8.7 billion in 2008—roughly 20 times the global investment in web analytics.  Among their three top corporate initiatives, most companies are focusing their efforts online, expanding their digital efforts Internet to increase the organization’s presence in the least expensive, fastest growing channel.

Today, a majority of companies are dramatically under-invested in analyzing data flowing from digital channels.  Even when business managers have committed money to measurement technology, they usually fail to apply commensurate resources and effort to make the technology work for their business.  Instead, most organizations focus too much on generating reports and too little on producing true insights and recommendations, opting for what is easy, not for what is valuable to the business.

Web Analytics Demystified believes this situation is exacerbated by the inherent limitations found in first- and second-generation digital measurement and optimization solutions.  Provided by a host of companies primarily focused on short-term gains in the digital realm, not long-term opportunities for the whole business and their customers.  Historically these companies worked to differentiate themselves from traditional business and customer intelligence, focusing on the needs of digital marketers.  Unfortunately, as the need for whole business analysis increases, many of these vendors are playing catch-up and forced to bolt-on data collection and processing technology as an afterthought.

The current state of digital analytics is untenable over time, and Web Analytics Demystified believes that companies that persist in treating online and offline as “separate and different” will begin to cede ground to competitors who are willing to invest in the creation and use of a strategic, whole-business data asset.  These organizations are using third-generation digital analytics tools to effectively blur the lines between online and offline data—tools that bridge the gap between historical direct marketing and market research techniques and Internet generated data, affording their users unprecedented visibility into insights and opportunities.

This white paper describes the impending revolution in digital analytics, one that has the potential to change both the web analytics and business intelligence fields forever.  We make the case for a new approach towards customer intelligence that leverages all available data, not just that data which is most convenient given the available tools.  We make this case not because we believe there is anything wrong with today’s tools when used appropriately, but because we believe digital analytics should take a greater role in business decision making in the future.”

Since I pride myself on the quality of my readership I sincerely hope that each of you will download this document and  take the time to read it. More importantly I’d love you to share it with your co-workers, friends, and followers on Twitter. I believe we are at a critical juncture in our practice’s history where the skills that have served us all along are not going to serve us for much longer, but I am always willing to admit that I’m wrong and more than anything I love a spirited debate.

Are you ready for the revolution?

It still leaves me scratching my head about the deal since the synergies are less obvious to me than they clearly are to Adobe and Omniture’s management and shareholders, but hey, with the sheer number of changes occurring in the industry right now who knows what might actually work. Hell, based on what I’m hearing about the Google Analytics announcement next Tuesday, it’s going to look like a great time to be focusing on something other than competing with Google Analytics …

I’m going to get to spend time with many of their largest customers next week so I suspect I’ll hear a great deal more about how this sale is being met by HBX customers, Visual Sciences customers, and those folks who have a tremendous amount invested in the SiteCatalyst line of products. If you’re an Omniture customer going to Emetrics next week and have an opinion you’d like to share please reach out to me directly and we’ll arrange some time to chat.

Again, congratulations to Josh James and all of the OMTR shareholders on what is increasingly looking like a great deal for all involved.

I really did enjoy the Forrester report, primarily because the author (John Lovett) managed to surface totally new data.  When he first told me that over half of Enterprise businesses were using free solutions I have to admit I didn’t believe him.  In a way I still don’t, but perhaps that’s only because I work with a slightly different sample than he presents.  Regardless, John’s report paints a picture of an increasingly challenging market for companies selling web analytics and a new sophistication among end users.

Speaking of sophistication, there are a few points in the report that I question, and since I have pretty good luck getting feedback from readers on big picture stories I figured I’d bring them up here in the blog.  Before I do I want to emphasize that I am not questioning Forrester or John’s work—I am merely trying to explore some data that I find contrary to my own experience in this public forum.  To this end I pose a handful of questions that I would love to discuss either openly in comments or via email.

The first point I question is the observation in Figure 3 that 70% of companies report having a “well-defined analytics strategy.”  Two years ago my own research found that fewer than 10% of companies worldwide had a well-defined strategy for web analytics.  Last year Econsultancy reported that only 18% of the companies in their sample had a strategy for analytics.  To jump from these low numbers to the majority of Enterprises just doesn’t square with my general experience in the industry.

.

Remember, the implication of this data point is that 70% of all companies having more than 1,000 employees have a “well-defined analytics strategy.”  According to a 2004 report from the U.S. Census Bureau there were just over 12,000 companies in the U.S. with more than 1,000 employees.  Without assuming any growth between 2004 and 2009, Forrester’s 70% figure would result in over 8,500 companies in the U.S. that have a “well-defined” strategy for web analytics. Does that sound right to you?

Consider that the combined customer count for Omniture, WebTrends, Coremetrics, and Unica combined in the U.S. doesn’t even add up to 8,500 companies.  Even if you use the more conservative 13% who “strongly agree” with Forrester’s statement you end up with over 1,500 U.S. companies.  I may suffer from sample bias, but personally I can barely think of 150 companies that I would identify as having any strategy for web analytics, much less a “well-defined” one.

Most companies I talk to have the beginnings of an over-arching strategy—they’ve realized the need for people and are beginning to reduce their general reliance on click-stream data alone.  But given that I think about this topic from time to time, I think a “well-defined” strategy for web analytics takes into account multiple integrated technologies, appropriate staffing, and well thought-out business and knowledge processes for putting their technology and staff to work.  What does the phrase “well-defined strategy” imply to you?

Similarly, if 60% of companies truly believed that “investments in Web analytics people are more valuable than investments in Web analytics technology” there would be THOUSANDS of practitioners employed in the U.S. alone.  But again, every conference, every meeting, every conference call, and every other data point suggests that the need for people in web analytics is still an emerging need.  Hell, Emetrics in San Jose earlier this year barely drew 200 actual practitioners by my count.  How many web analytics practitioners do you think there are in the United States?

Same problem with the rest of the responses to Figure 3 on web analytics as a “technology we cannot do without” (75%) and the significance of the role web analytics plays in driving decisions (71%).  Perhaps I’m talking to entirely the wrong people, perhaps I’m interpreting these data wrong, and perhaps I’ve gone flat-out crazy, but these responses just don’t match my personal understanding and experience in the web analytics industry.

This issue of data that simply does not make sense, while not universally manifest in the report, manifests elsewhere as well. For example, Figure 8 reports on the percentage of application used segmented by fee and free tools:

.

When I look at these responses and see that 63 percent of respondents using fee-based tools and 50 percent of respondents using free tools claim to be effectively using more than half the available functionality, again I find myself scratching my head. As this data appears to speak to the general sophistication of use of analytics I went back and looked at Dennis Mortensen’s quantitative study of how IndexTools was being used around the world.

Dennis reports that fewer than 10% of his customers were using even the most basic “advanced” features in web analytics (report customization) and that fewer that 4% of his customers (on average) are making any “advanced” use of the IndexTools application. While this dataset is somewhat biased towards European companies who I believe, on average, to be somewhat behind their U.S. counterparts it does provide an objective view in how web analytics are used that seems to directly contradict the self-reported responses in Forrester’s figure 8.

Clearly there is a gap between the responses John collected and the current state of the web analytics market.  Since John is a very smart guy I know part of his rebuttal will include the observation that he surveyed people directly responsible for web analytics (see Forrester’s methodology) and that people in general have a tendency towards positivism. Trust me, my son is the most handsome little boy ever born and my daughter’s beauty is only matched by that of Aphrodite … same for your kids, right?

Given the difficulty associated with gathering truly objective data regarding the use of web analytics, this type of self-reported data is usually what we have to go on.  While Omniture, WebTrends, Coremetrics, and Unica all have the fundamental capability to report data similar to that provided by Mr. Mortensen, it may not be in their best interests to expose underwhelming adoption and unsophisticated use (if that is what the analysis uncovered.)  Ultimately we’re forced to accept these self-reported responses and  then reconcile them against our own views, which is why I’m asking my readers what they think about the data Forrester is reporting!

Regarding these self-reported attitudinal responses on how web analytics is used strategically, perhaps the truth is found in the companies who “strongly agree” with John’s statements.  If we apply this lens, as opposed to the more optimistic view, we get the following:

• 17% of companies recognize that web analytics is a technology they cannot live without;
• Web analytics plays a significant role in driving decisions at 12% of companies;
• 13% of companies have a well-defined web analytics strategy;
• 9% of companies recognize that investments in people are more valuable than investments in technology

These numbers start to make a lot more sense to me.  Likely the truth, as with so much in our industry, lies somewhere in between, but I would love to hear what you think about these adjusted numbers.  Do the lower numbers make more sense to you, or do you agree with John’s more optimistic assessment?

Unfortunately if the lower numbers are correct the implication is that despite the incredibly hard work that companies, consultants, and industry thought-leaders around the world have done for years we still have an incredibly long way to go before web analytics is recognized as the valuable business practice that you all know it can be!

Regardless I want to state that I do not disagree at all with the fundamental thesis in this report, that “free” is creating a whole new level of interest in web analytics and that, given proper consideration, free is an excellent alternative to paid solutions.  Lacking clear strategy and resources, too many companies have wasted too much money on paid solutions for free to not be compelling.  Thanks to the dedication of the Google and Yahoo teams, the world now has access to great applications that are in some regards more compelling than fee-based alternatives.

While I may not have said this a few years ago, today I honestly do believe that “free” is a viable and appropriate alternative to fee-based solutions. While not appropriate in every situation, it is irresponsible to suggest that any company not willing to fully engage in web analytics should pay for ongoing services and support. Given advances from Google and the availability of Yahoo Web Analytics, any motivated company large or small now has access to a wealth of data that can be translated into information, insights, and recommendations.

Conversely I agree with John (and Jim, and almost ever thought leader I respect) who states that you need to “prioritize your business needs and culture for analytics first and then evaluate the tools.”  This goes back to the fundamental value proposition at Web Analytics Demystified: It’s not the tools you use but how you use them. If you’re not invested in developing and executing a clearly defined strategy for digital measurement, you may as well be grepping your log files.

I would love your feedback on this post, either directly in comments or via email. Thanks again to the folks at Google for making this awesome research freely available and to John Lovett for shedding light on this incredibly important aspect of our sector.  Remember: we are analysts—our jobs are to ask hard questions and then ask even harder ones!

• The Online Performance Management seminars, hosted by Creuna, in Copenhagen on Thursday, October 8th and in Oslo, Norway on Friday, October 9th. More information about our hosts and registration is available from Creuna.
• While we’re in Copenhagen we will be having a Web Analytics Wednesday on Wednesday, October 7th. I will be giving a short presentation on testing and if you’re in Copenhagen please join us at this FREE EVENT sponsored by IIH Nordic and Webtrekk
• Over the weekend Aurelie and I will be hanging out in Stockholm, Sweden. If you’re in Stockholm and want to meet-up please either shoot me an email or Twitter me and we’ll make plans!
• On Monday, October 12th and Tuesday, October 13th Aurelie and I will be joining the excellent Emetrics crew at Emetrics Stockholm. I will be giving the keynote on Tuesday morning and Aurelie and I will both be participating on a series of panels and shorter presentations. Those of you keeping score will note that I have attended EVERY SINGLE Emetrics ever held in the United States but this is my FIRST EVER event in Europe. Yahoo!
• On Wednesday, October 14th, I will be hanging out in Amsterdam with the Nedstat crew but have a fair amount of downtime during the day. I’m staying near Vondelpark and if you’d like to meet and get a cup of coffee (seriously, I mean coffee, I’m too old for the other stuff) Twitter me and we’ll make plans!

Since I usually do three European cities in three or four days this trip is a lazy walkabout for me (four cities, seven days) but Aurelie and I have planning to do and, of course, we’ll spend a little time enjoying the local culture.

If you live in any of these cities, or if you plan to come to Emetrics, please join us and come say hello!

Other folks appear to be worried that Adobe will be integrating Omniture into Flash and this raises privacy concerns. While certainly folks have concerns about tracking and the possibility of embedding tracking into Flash Local Shared Objects (LSO) I just have to believe that management at Adobe is smart enough not to risk Flash’s dominance by subjecting the technology to the scrutiny, navel-gazing, and paranoia of the “privacy police.”

Their customers, at least the ones I am talking to, are more or less 2 to 1 against the acquisition at this point citing a variety of concerns (transition, failure to execute on stated product plans, talent flight, Adobe is not adept at services, etc.) Far be it from me to tell anyone’s customers they are wrong when expressing concerns, especially since this is an out-of-sector acquisition and Omniture is now more or less a medium-sized cog in a very big machine. Arguments for include loving Adobe (I love Adobe!), being relieved that Adobe is a big, grown-up company, and hopes that Adobe will focus on fundamentals like customer support, product execution, and global expansion.

Another customer complaint is that Omniture is now losing the (thin, pasty) veneer of third-party objectivity and that some companies may not actually want Adobe to have access to their site’s data.  I think this may be the same boondoggle that Omniture (and others) have used to explain why “the Enterprise wouldn’t use Google Analytics” — except there is more and more (and more) evidence that the Enterprise does use Google Analytics — but it will be interesting to see how the “free-standing” analytics vendors work to make Omniture eat their own words now that they too are part of something larger.

The comment that has me most concerned is one best detailed by Carter Malloy from Stephens, Inc. Research Analyst who I have known for years and who I know to be pretty level headed regarding the sector.  Carter sent me this, which I am simply repeating with his permission:

“I don’t understand the strategic rationale on adobe’s part. Different end market buyers. Very different products. No real cost savings or integration between the two products. OMTR is very capital intensive vs. adobe not much at all. Seems like Adobe is buying growth with hopes for cross sells. I would be surprised to find out that OMTR did not shop the business around before accepting the bid from Adobe – we should find out soon in public filings required by the SEC. Omniture will still have to report 3Q09 earnings in October, but I think the deal will get closed before Q4 in Jan/Feb. I also think Adobe will show Omniture’s revenue performance on an informal basis going forward. It will be <10% of Adobe’s total revs, but I still think they will give analysts at least some idea of what growth looks like.”

This was in response to my comment detailing a thesis that I have heard from several of Carter’s peers: that Omniture was about to blow Q3 earnings and that the result would be a dramatic dip in OMTR share price as investors head for the exit. The rationale is, apparently, that the company has over-promised and under-delivered for too long, both to investors and customers, and the economy has been the “last straw” for many who have opted to look elsewhere for web analytics technology. This, combined with slower-than-hoped adoption of non-core solutions (data warehouse, Test & Target, Search Center, Survey, etc.) resulted in a “company who’s greatest days are behind them” (direct quote, and I begged to attribute but was told “no” due to company policy.)

Don’t get me wrong: This is not my thesis, at least not yet.

While I have seen evidence of larger Omniture customers switching, increasingly to Unica, I have not seen enough evidence of the kind of massive shift away from SiteCatalyst that would warrant a sudden exit. The good news is that Carter’s thesis can easily be tested: Either Omniture will make expectations for Q3 or they won’t. I’m sure this will make for an interesting Q3 call, at least for those investors who are taking a bath on the acquisition price.

My concern is this: If the investment banker thesis is correct, if Omniture was about to report a second quarter of, um, disappointing results, then what does that mean for the larger industry? Is Adobe really evidence that the larger market is taking an interest in digital analytics? Or was the company thrashing about looking for something new to cover for recent declines and this really isn’t about Omniture or web analytics at all?

Again, I don’t know, at least not yet, and I don’t think any of us do. But given the very mixed reviews about the acquisition I think we as an industry should take a step back and consider the larger ramifications. Personally I don’t think web analytics is going ANYWHERE — hell, I’m recruiting at Web Analytics Demystified — but we can all admit we collectively haven’t done the best job explaining what we do and what the data we live and die by means.

This interesting acquisition will certainly get more interesting as the days pass. Congrats again to all involved.

Wow.

I didn’t get to talk to the press (John got the honors, congrats) and am just not getting a chance to cogitate a little on what Adobe’s entrance to the web analytics market means after non-stop phone calls for the past five hours.  A lot of interesting comments have already been published so I will try and reference the stuff I think is insightful in an effort to avoid repetition.

• In general, the more I think about the deal the more it makes sense, at least for Omniture. Given increasing pressure from lower-cost (and free) solutions, the economy, and a customer base that is more and more prone to complain about service issues and the high cost of doing business with the company, exiting now makes good sense.  Why fight the sea change in the analytics market when you can saddle someone else with the responsibility?
• Like others, I don’t really see the synergy in the deal, but I admit that I love Adobe and so I’m willing to be surprised. I think of Adobe as a software company for creative types; Omniture sells software-as-a-service to analytical types; these are different business models and very different customers. The idea that somehow this acquisition bolsters Adobe’s position in content management or as a global delivery platform just doesn’t resonate with me.
• Similarly, I don’t see this acquisition as creating anything new regarding measurement being embedded into rich media applications. Thanks, perhaps ironically, to Macromedia (owned by Adobe) we have been embedding tracking codes into Flash, Flex, Silverlight, AJAX, etc. for years … and while the integration is botched as often as not, I don’t see how adding a “Click here to Omniture-ize” button into Dreamweaver and Adobe’s RIA development suite will solve that problem.
• I do agree with Alex Yoder’s general thesis that this acquisition increases the overall visibility of the sector and that this is a good thing. I also agree that this acquisition is likely not the last — both WebTrends and Coremetrics are owned by investors and you know how those guys are. His citation of Microsoft and Oracle is interesting given both companies historical interest in the sector (although neither has had the chutzpa to actually pull the trigger — at least in a substantial way.)
• I also agree with Gary who is somewhat skeptical about acquisitions, especially out-of-sector ones like this (anyone remember NetIQ? How about you Deepmetrics customers?) and since the Instadia, HBX, and Visual Sciences acquisitions that Omniture made didn’t really generate the benefits promised. However, where Gary favored IBM (who I didn’t realize wanted back into the sector after selling SurfAid to Coremetrics) I liked the idea of WPP increasing their $25M investment by, well, I guess about $1775M or so. Given my position on how companies will deploy web analytics in the future, WPP adding a premium measurement brand to their analytics tools and giving them the ability to pass world-class analytics along to their best customers made sense to me. Oh well.

Regarding Omniture customers … I am getting feedback from across the spectrum. Some customers are encouraged by the news, largely because they believe that Adobe will bring a new level of rigor to product development, integration, and customer support.  Others (including those customers still on HBX) are somewhat discouraged by the news, given that they’ve been hearing a lot of promises lately and they’re not sure what a new owner will mean.  Still others have expressed that they really liked what the company had been doing this past year and so are bummed that things might slow down while the deal and integration are completed.

Prospects are a different question. Since I am working with a number of companies currently evaluating Omniture products … the best guidance I can give is “wait and see.” Again, I think Adobe is an awesome company and every interaction I have ever had with them has been positive. Hopefully this acquisition will be mostly painless and largely transparent to outsiders. We’ll know soon enough if Omniture’s recent aggressive pricing and willingness to cut deals to close business differs from Adobe’s business practices. And while competitors will almost certainly claim “Omniture is out of the game,” I am personally encouraging my clients to think carefully about what Omniture and Adobe have been able to do independently before writing the combined company off.

At the end of the day I’m really happy for the bright folks I know who have been plugging away at Omniture all these years in a variety of their companies. The teams at Omniture, HBX and Visual Sciences, Offermatica, and hell even Matt Belkin (remember that guy!) who hopefully get to participate in the largess that Omniture has created should all get credit for the thousands of hours they spent on the road, fighting for the big green machine, never willing to concede until they’d finally lost (and sometimes even after they’d been asked to go home!)

Congratulations to Josh, Chris, Brett, John, Kristi, and the entire senior management team at Omniture! Also, best of luck to the management team at Adobe with your new acquisition; your new customers are among the best in the business and will look to you to make a good thing even better.

I am certainly excited about the group we have coming next week: some great vendors, some awesome consultants, an incredible keynote event, and many of the best and brightest practitioners in the digital measurement industry. Excellent!

Also, as Gary pointed out in his blog post today, if you really, really, really need to join us in San Francisco and have already gone to bat for the budget, just let me know. We can always squeeze one more in, but we’ll probably make you attend Gary’s Think Tank training session just so he feels a little better (see his blog post for the back-story …)

If you can’t make it and you’re on Twitter please watch for conversation and insights on the #xchange hashtag in Twitter. Like Gary, I’m not foolish enough to promise to blog from the conference (hell, I barely blog as is anymore … too busy with work I guess!) but I will definitely try and push up 140 characters here and there, or slightly less when I co-tag with #xchange and #measure LOL!

If you’re coming to San Francisco, I look forward to seeing you next week!

P.S. piggybacking on Gary’s comments about WebTrends … I’m with Gary and Phil on the whole “9″ release. I’m encouraged by the company making a move in the right direction, but I feel that the release was dramatically over-marketed and set a new, all-time high for “hype over substance” in this industry making even the great green machine look conservative. When your own staff are forced to admit the release is not “the new UI” despite marketing’s claim that “[9's] clean, professional interface lets you creatively explore your data like never before,” well something has gone wrong somewhere.

That said, it’s great to see Alex committing to the product and, at the end of the day, it’s not what Gary, Phil, or I think … it’s what their customers and prospects think that counts. I know a handful will be at the X Change, along with someone from WebTrends marketing, so that interplay will be … ummm … interesting to watch!

Hard, but not impossible.

In the past few years we have definitely started seeing an increase in the number of companies that “get it” — so much so that we’re able to program an entire conference built around the superstars of web analytics. More and more I am talking to, working with, and hearing about companies who have leveraged the “web analytics is hard” mindset to properly set expectations regarding their use of technology, their deployment of people, and their use of business process to really excel at web data analysis and turn this analysis into tangible improvement for both the business and their customers.

As I collect more information about these web analytical competitors one thing that nearly always emerges as a hallmark of their success is some type of structured testing program. Of course this makes perfect sense because without testing the analyst is never really sure about the impact of their recommendations, so much so that I have often said “if you’re not testing, you’re not really doing web analytics!”

But the increase in testing has raised an interesting corollary to my “web analytics is hard” manifesto … testing is hard too!

Fortunately we’re all a little bit older and a little bit wiser this time around and we recognize that testing requires more than just throwing code on the page and clicking the “Optimize” button. Testing is a process that requires people and technology … sound familiar?

I’m bringing this up for two reasons:

1. At the X Change conference in San Francisco on September 9, 10, and 11 Matthew Wright from HP will be leading a conversation titled “Testing, Testing, Testing: Building Consensus and Evaluating Results” to discuss the nuances behind testing, things like getting stakeholder approval, planning, and clearly defining measures of success;
2. This morning the nice folks at SiteSpect published a white paper I wrote that details ten “best practices” for testing that I think a lot of folks new to testing often forget. Titled Successful Web Site Testing Strategies: Ten Best Practices for Building a World-Class Testing and Optimization Program, this white paper is freely available (requires registration) and covers nuances like testing teams, stakeholder involvement, test plans, timelines, and, of course, measurement.

If you’re working to become an analytical competitor and join the ranks of the kinds of companies who get invited to lead a conversation at X Change I highly recommend either grabbing the testing white paper, coming to the X Change, or BOTH! Especially if you’re serious about testing I think you’ll find this free white paper useful when you work to set expectations, and trust me, testing is as much about expectations as it is execution!

Register to download the free white paper on successful multivariate testing strategies!

I am personally honored that John accepted my invitation to return to the X Change this year and both lead the huddle on “Industry Standards (or a lack thereof)” and co-lead a huddle on technology with Bill Gassman. If you haven’t met John personally, and if you are able to join us at the X Change, I strongly recommend you make a point of introducing yourself to him.

Finally, before my questions and John’s answers, I wanted to point out how incredibly deft Mr. Lovett really is: in response to a high-and-hard fastball question about “which vendor is really the best,” John knocked the ball clear out of the park with his answer: none of them. I’ll let you read the rest for yourself …

Well Eric, I like to say that surprises are for birthdays and not for business. So in terms of actual surprises, there weren’t any big bombshells for me. I was however pleased that the vendors demonstrated innovation in a number of areas (like social media measurement) and that despite my attempts to develop extremely challenging criteria, the vendors continue to improve year over year.

One comment people have made to me is that they question the validity of comparing fee and free solutions in a single matrix due to the fundamental differences in their business model.  How would (or do) you respond to that challenge?

That’s preposterous! I respond by saying that it’s negligent not to compare free vs. fee based solutions. In today’s economic environment if you’re not watching expenses by understanding the cost to benefit ratio of your Web analytics solution, you are acting irresponsibly. Free tools have merit for many organizations as both primary and secondary tools, while fee based solutions are more appropriate for others based on their capabilities. Organizations must do their diligence to understand what they need in a Web analytics solution to decide what’s right for them, which is really the insight the Wave attempts to provide.

I asked Bill Gassman from Gartner a variation on this question recently, but do you now or see in the near future a situation where you as a Forrester analyst are advising your clients to actively consider these free solutions in addition to “traditional” web analytics solutions from Omniture, Coremetrics, and Unica?  As a follow-up, how do you see free tools impacting the market in the next 12 to 24 months?

I advocate that a single system for measurement is always the best way to go, yet recognize that this isn’t always feasible. Duality of Web analytics tools is a reality for myriad reasons. Thus, company’s need to manage their data dissemination practices to ensure comprehension and mitigate doubt. This is tricky, but certainly possible. I often help clients determine which solution is best suited to meet their needs and financial implications are always a part of that discussion.

With regard to how free tools will impact the market: we are just witnessing the beginning of the incoming tide on this one. By this I mean that “free” will continue to disrupt the market by placing pressure for improvement on all vendors. Just look at the recent Webtrends product upgrade announcement – the majority of press around it cited a “look out Google Analytics” slant. Why the comparison…they’re worried! Fee-based vendors have even more to fear now that Yahoo! Web Analytics opened up its partner program.

Another comment I hear about the Wave results, and forgive me this, is that they’re lame because they do nothing to differentiate the “market leaders” who appear as a tight cluster.  The evidence cited is that all four vendors issued press releases declaring their “market leadership” which appears technically correct based on the Wave but as the Highlander said, “There can be only one.”  First, how do you respond to this and second, who is the real market leader in web analytics?

Here’s the dirty little secret – the real market leader is the wildly talented Web analytics practitioner. It’s not the tools that differentiate it’s the craftsman. Any company that believes the Web analytic technology alone will make them incredibly successful is delusional or just plain out of touch. There is no get rich quick scheme here. Each of the leading vendors on the Wave offers a highly customized solution that can be tricked-out to meet nearly anyone’s individual needs. But this takes a great deal of work. For those organizations that are looking for the far-and-away winner in this technology category, guess what: the tools will only get you so far – you need talented people to really make it happen.

Rumors are that Omniture has a bunch of “800 lb gorillas” hanging in their offices right now.  Clearly they’re proud of their position, but last quarters results highlighted that there are clear risks to their business that are beginning to manifest.  What do you think are the greatest risks to Omniture’s business over the next 18 months?

Well, I don’t buy into rumors and sure don’t know where I left my crystal ball. But things are tough all over. As I stated earlier, free solutions are threatening all fee-based vendors and forcing them to work harder. I can tell you that measurement technologies are an imperative for executing on digital marketing endeavors. Solutions like Omniture’s, Webtrends’, Coremetrics’, Unica’s and everyone else’s will continue to play an important role in the evolution of organizations conducting business online. I believe that Web analytics is increasingly becoming an integrated service and expect to see things evolve to easier access to data through new and alternative means. The leading vendors, including Omniture, will play a role in this evolution.

What’s your taken on the current hype cycle around “open”?  Omniture bangs the Genesis drum, Coremetrics connects, and now WebTrends appears to have decided that “open” will be the foundation of their future success (or lack thereof) … but some people think that “open” is a check-box requirement, not a competitive differentiator.  What do you think?

Open is not a feature, it’s a philosophy. The ability to get data into and out of a Web analytics solution is the crux of the issue and leading vendors facilitate this through bi-directional API’s, other import and export functions and data dissemination capabilities. Webtrends is currently doing this as well as anyone, but “open” also means talking to your customers about development plans, listening to criticism and demonstrating a willingness to change. These qualities aren’t unique to Webtrends, they’re characteristics that all vendors should exhibit. Webtrends is just marketing around them and if that’s causing people to want open, then it appears to be working.

As a previous attendee to the X Change what do you like best about the conference and what would you like to see us change this year or next?

I appreciate the intimate conversational format of X Change. The huddles really facilitate deep thought, controversial leeway and provocative discussion. As someone who attends a number of conferences, it is refreshing to engage in dialogue with individuals who are passionate about what they do and to initiate a true collaborative thinking environment. As far as change goes, I really hope to be able to guide the huddles that I’m leading toward resolution. Within our industry, all too often we surface problems and issues without identifying solutions. I’ve taken your challenge to heart and hope to walk away with some tangible results from my huddles.

http://www.xchangeconference.com

Still, it’s nice to see WebTrends making some moves so congratulations to Jascha, Casey and the entire Portland crew for getting the update out the door!

Anyway …

I said I had been thinking about the evolution of the web analytics vendor market. A lot of my thinking this past week has been colored, well, purple, thanks to the announcement of Yahoo’s Web Analytics Consulting Network (the YWACN or, as I think about it “the Yack’n!”.)  On July 30th Yahoo announced that they were making Yahoo! Web Analytics much easier to get through 48 partners around the globe.

Now, when you look at the partner list you might not recognize a lot of the names — I sure don’t — but a few should stand out. Specifically Stratigent, Semphonic, Sapient, and my own company Web Analytics Demystified. While I can’t speak directly for any of these companies, all are run by very smart people, and I have to wonder if they’re not thinking about YWA much the same way I have been.

I mean, if you think about it, Yahoo! has basically come to us and said “Go sell excellent implementations of YWA and provide awesome ongoing support” for an application that, according to Forrester Reseach, has 77% of the core functionality of Omniture SiteCatalyst. Or, put another way, “Find companies that are struggling to get value from their existing investment in {pick a vendor}, kick that vendor out, and then make money helping them be successful for less then they spend today.”

Sweet, thanks Yahoo!

Not to brag (since it was pretty obvious) but I did say this would happen back in April 2008 given the hard work Google Analytics (who is ironically NOT a YWA competitor) had done with their similarly badly acronym’d GAAC. Yahoo wisely avoids having to support customers directly, leverages some incredibly smart folks, and lets companies reduce their annual analytics spend without having to forgo core functionality like multiple custom variables and visitor-level segmentation.

Hell, we’re not even talking about real-time updates and demographic reporting and segmentation, which while the former often has more value ascribed than necessary the latter, if I can say so based on my own usage, is pretty fantastic and not available in any other web analytics application in the market today. I mean, who would have guessed that so many mature, responsible adults love to Twitalyze themselves!

Now I sincerely doubt that any of the YWACN members are going to suddenly stop supporting the big for-fee applications out there … I know I’m not! And I fully expect the adoption of YWA to be slow and methodical (mostly because of existing contracts, Yahoo’s terms of service, and the fact that Yahoo is somewhat limiting YWACN access to new accounts although I think their strategy is fair and makes perfect sense.) But at the end of the day Yahoo has made quite possible the single best move they could have if their goal was to provide an awesome service with excellent third-party support at the best possible price.

Now if you were paying attention you may have noticed I commented that Google Analytics and Yahoo Web Analytics are not competitive. Crazy, huh? But they’re not. Google Analytics (as it exists today) and Yahoo Web Analytics (as it exists today) serve two near completely distinct target markets.

Now I know I’ll get heat for saying this (again) but I just don’t think Google Analytics is appropriate for “free standing” use within the true Enterprise. I’ll point again to Bill Gassman’s recent note on the service (which I thought was excellent) and will obviously concede that it is well within Google’s power to make GA the “bestest, most Enterprisey” web analytics application the world has ever seen … but it isn’t today. More importantly when I go looking for companies mature in their use of web analytics who rely exclusively on Google Analytics and have chosen to do so explicitly, I simply don’t find them.

I could be wrong — if you’re an analytics samurai using nothing but GA please let me know –  but what I see a lot of is mature businesses using Google Analytics to back-fill some limitation in their for-fee vendor’s service. For example, up until today it was amazingly difficult to get WebTrends to calculate a bounce rate and some people think setting up visitor segments in SiteCatalyst is a lot of work. More importantly, while lots and lots of people complain about how difficult their analytics application is to use, the team at Google has done a freaking brilliant job with the GA user interface and in my humble opinion it sets the bar for ease-of-use in web analytics.

Yahoo Web Analytics in an Enterprise context, and hopefully Dennis will forgive me this since he’s tanned and rested after a week or two in the islands, is not really that easy to use, not that easy to set up, and not that easy to configure — remember it’s 77% of Omniture SiteCatalyst which nobody ever describes as “easy to implement and easy to use” (except for Adam Greco, but he’s clearly an exception!)

But here’s the secret: Yahoo Web Analytics is not supposed to be easy to use, it’s supposed to be really, really powerful! Yahoo Web Analytics is an Enterprise-class web analytics application out of the box designed to support businesses with custom data collection needs, custom reporting needs, custom segmentation needs, and the challenges typically found within any company of size.

More importantly, because of this functionality I believe that Yahoo Web Analytics will be a gateway to a much deeper relationship between the YWACN and their customers than the GAAC have found for the most part.

Yes, Yahoo’s APIs are tightly held and thusly YWA is not as “open” as WebTrends or as “integrated” as Omniture. Yes, Yahoo is keeping Rubix under wraps so it is not as flexible as Affinium NetInsight or Coremetrics Explore. Yes the interface is kinda clunky and the terms of use were written by lawyers … I get the complaints and hear the FUD loud and clear. But given the massive adoption of Google Analytics I think that coupling exceptional services and support with a free Enterprise-class application has a lot of potential to be the permanent game changer that I first described last year.

What do you think? Is purple the new green? Is “9″ too little, too late? Does Yahoo! have a chance to focus now that they have outsourced their search business? Or am I missing the point and despite two great free solutions will the world continue to pay for web analytics the same way we always have? I’m totally willing to be wrong about this one … but if you don’t believe me about how powerful Yahoo Web Analytics is either read this book or contact me directly and I’ll see about getting you your own YWA account.

As always your comments are welcome.