SemAngel

Aggregation and Detail in the Big Data World

2013-05-12T12:14:36-07:00

In last week's post, I took another crack at defining what makes "big data" real and not just more of the same with an extra helping of hype. Creating a sound definition of big data may not have a huge amount of practical significance - but it isn't without importance. Knowing what you're dealing with and why/whether it's really different from what you've done before has a host of implications for questions around resourcing, process and technology.

But finding a good definition of big data isn’t what I thought most compelling or important in the discussions that triggered these posts. It was the discussion around the nature and role of detail-level data in the big data world. If the theme of last week's post was that big data skeptics are somewhat right to question the hype but very much wrong to question the reality of big data, today's post addresses the most common piece of bad advice you'll get from big data experts.

The Many Advantages of Detail Data

If there’s one thing that virtually every big data vendor and consultancy will tell you, it’s that the key to big data systems is leaving your data at the detail level. Indeed, it’s precisely the ability to do this that makes big data technologies unique. They process enormous amounts of detail data (whether structured or unstructured) fast enough to allow for the data to be kept in its native form. What do you gain from this?

Independence from structure – removing layers of modeling and indexing
Ability to re-define the data model for every analytic problem – critical since most anslysis projects require unique aggregations
The ability to join multiple data sources at the detail level as necessary

These are all legitimately important things – and are all pretty much true. That being said, there’s no bigger mistake you can make in a big-data system than to believe that your data must always live at the lowest level of detail.

Taking a Rule One Step Too Far

The problem, as I see it, is that just as the big data vendors have largely failed to understand what makes big data problems unique, they’ve often failed to grasp what’s involved in the solution to those problems. They understand the technology, but not the analytics.

Just as with my last post on what makes big data truly different, it’s the stream nature of the data that drives my thesis – and though I’m going to use digital data as my example, what I’m going to argue applies equally well to sensor data, meter data, and a host of other important big data use-cases.

The problem starts with the lack of meaning at the event level. In digital, the analysis of page views isn’t interesting. The page is not a significant entity in marketing. Rather, the analysis is all about sequences of pages views – those sequences being a visit or a related part of the visit – in which the visitor is trying to accomplish something. If I’m building a model of the customer journey, I don’t want to capture every page. I want to capture what the sequence of page views was all about and how successful it was.

One of our main practice focus areas here at E&Y is how to use segmentation techniques to identify visit intent – what a sequence of pages tells us about the customer’s intent and interests. Over the past few years (as Semphonic), we’ve developed an entire analytics methodology around this. But whether you use those methods or not, the key point is that almost any analysis you do is going to have to create some level of meaning around the sequence of touches that constitute a visit. If you take the time and trouble to build a complex decision-tree or cluster analysis of visits, that’s an incredibly valuable foundation for nearly every subsequent analysis.

But if you listen to your big data vendor and insist on having nothing but the lowest level of detail data on your box, you’ll have to re-create that full segmentation EVERY single time you want to use it. That’s preposterous.

Think about this. It's the key point in this post. If you are always constrained to leave all of your analytics data at it's lowest level, you're force to re-create EVERY analytics step EVERY single time you want to re-use it. This might be reasonable when it comes to fairly simple techniques like sessionization, but it's madness when it comes to complex steps like segmentation. What's more, leaving the data in it's native detail state puts dramatic limitations on the number of analysts who can productively use the data. Big data environments are difficult enough without adding silly rules to make them harder.

Creating new and permanent levels of data that include not cubes but data aggregated into levels of meaning above the detail level (like a single row per visit based on segmentation) doesn’t violate any aspect of the big data paradigm - it’s essential to it.

This doesn’t mean you don’t need the detail. You do. Not every analytics problem will be captured within or will take advantage of a visit or task-based segmentation. You still have to have the ability to start over, and you'll use that ability all the time. But a large number of subsequent analytics tasks (including customer journey models) will benefit from having that visit-level segmentation-based aggregation and will be nearly impossible without it.

It’s just a case of people taking something true (don’t create complex data models, cubes, or fixed structures on your big data system) and taking it to a level where it no longer make sense – never have anything but the lowest level of detail on your big data system.

Summing Up

When you have visit-level, cluster coded data, you still have something that is, for all practical purposes, flat. It's no different than the call-center, call-level data your big data consultant will load onto your system without blinking an eye. But that call-center data isn't the lowest level of detail possible (unless it comes from call digitalization). It's data aggregated by the call-center system. If you'll load aggregations onto your system, why shouldn't you create them on your system too?

Whether the aggregation happens on your big data box or elsewhere is completely irrelevant. This visit-level segmentation coded digital data is just the kind of data your big data systems will chew up and digest wonderfully.

It's particularly useful as an integration layer between various types of customer touchpoints.

When you need to join call-center, Web, mobile, and bricks-and-mortar touchpoints, you simply can't do it at the detailed stream level. Data at that level is too disjoint - completely different for every type of stream.By using segmentation techniques to aggregate streams up to visit intents and success measures, you create a level of data perfect for customer journey modeling.

It's not detail level data, but don't let that worry you. That it happens to be easily used, meaningful, and valuable should not be taken as three strikes against it!

If you think your big data strategy needs a re-think, drop me a line. And if you're in Las Vegas this week for VoC Fusion stop by my presentation on Thursday to chat!

The Four V's, Big Data, and Big Data Skeptics: Half Right and Wholly Wrong

2013-05-05T10:16:33-07:00

This past week I did an internal webinar for a client on big data systems. It was a good session with lots of back-and-forth and questions – which not only makes for a more interesting and relaxed presentation, it’s more likely to stimulate my thinking. I hear more than enough of myself talking!

So even though this is a digression (from my posts on Forecasting and Dashboards) inside a digression (my larger series on re-engineering Voice of Customer at the enterprise level), I wanted to recap and expand on a couple of the central themes of that presentation.

First, a quick table-set. I’ve taken a crack in the past at defining what “big data” is all about. It’s always hard when a concept takes off to keep control over it – the Hype Cycle (Gartner's lovely concept) can take anything – from Presidential candidates to epistemology to piano-playing cats and so over-expose them as to make the underlying reality nearly impossible to discern.

Naturally, this creates a kind of backlash. Plenty of web analytics pundits are more than willing to describe big data as just hype. Not only is there a certain cachet in running counter to a popular trend, there’s a certain self-interest here too. Web analytics companies nearly all come from Web analytics tool backgrounds – said tools being, in most respects, the antithesis of big data tools. I imagine that it’s always safest to assume that anything you don’t understand must be unimportant!

I’m not a fan of the industry standard definition of big data (probably best exemplified by the four Vs: Volume, Variety, Velocity, Veracity) in part because I think it’s vulnerable to the criticism of skeptics that we've always had these exact same factors.

The four Vs do describe most big data situations, they just don’t get to the heart of what big data is all about. Back in the early ‘90s when I was doing credit-card work, we had volume most of today’s big data companies would still consider massive. We had plenty of velocity too, and veracity was pretty darn important when clearing card transactions. We didn’t have variety, but it’s implausible to argue that variety is essential to every big data application. There are plenty of big data applications that are single source. If I’m trying to mine CNN’s digital data stream, I don’t need variety to be in the big data universe.

So were all of us in credit-card working on big data in the early ‘90s? Some might say yes, but I don’t think so.

Instead, I’ve proposed a simpler, more basic, and more fundamental definition of what uniquely defines big data. Big data happens when you drive your data capture and analysis down from the traditional levels of analysis (like customer or transaction) to a level where the meaning of each event can only be interpreted in relationship to the stream of events. Digital is a paradigm case for this. Web site page events are not, in and of themselves, meaningful. The meaningful level of aggregation is somewhere in the sequence of events and that's what you have to interpret.

It’s not too different in utilities. When you move from the once a period reading of a meter per customer to constant collection, you change the nature of the analysis and data capture problem. No single meter reading is, in and of itself, important. It’s in the flow and pattern of the readings that meaning emerges. This is a different type of analysis.

It should also be clear from this why the Four V’s look like a reasonable definition of big data to those in the field. When you drive your unit of analysis down a level, you increase by one or more orders of magnitude the volume of your data capture and the velocity of your data. You place additional demands on data collection that can result in poor data quality. And while variety isn’t necessarily wrapped up in the concept, you have created a whole new set of challenges around joining data that lives at the stream level – making multiple sources (variety) far more difficult to handle.

But the beauty of the definition I’ve provided is that it makes it clear why my ‘90s credit card work – despite hitting the V’s pretty well – wasn’t necessarily big data. No amount of the four V's make for big data if you're just scaling up the same exact types of data and analysis as you've always done. It also explains much of why today’s generation of big data technologies are built the way they are and why they provide unique advantages that traditional transactional systems don’t. Those traditional systems sure-enough handled lots of volume - just not in the ways we need it handled now.

It also explains another aspect of big data that is particularly important and represents one of the biggest risks if you’re building a big data system. The nature of the analysis and the methods necessary to join, process, and understand the data all change at the stream level.

Traditional analysis techniques, from joining methods to sql queries to aggregatations to traditional statistical techniques like correlation, regression and clustering all work differently - if they work at all – when applied to this type of detail, stream data.

I’ve seen this cast as a debate between machine learning and traditional analysis; it isn’t.

Machine learning may (though I think it’s debatable) be particularly useful in big data situations because of the symptoms (the four Vs) that spring from detailed stream-level analysis. As far as I can tell, there is nothing about detailed stream-level analysis itself that makes machine-learning particularly suitable. The really important point isn't about machine learning - it's that your standard analytics toolbox is mostly out the window.

So while defining big data by the four V's may miss the mark, it's far, far more misleading to suggest that big-data is just "more of the same" - perhaps with a bit of an emphasis on the "more". If proponents of the more of the same view are claiming that we still need to decide how to structure data, how to join data, how to query data, and how to analyze data then their claim is merely empty. Of course we do. But if they mean to claim that we should use the same methods to join, structure, query and analyze the data as we always have in traditional transactional or BI systems, then they are flat-out wrong.

It's nearly always a safe bet that opposing the hype cycle will make you at least half-right. But if the big data skeptics are half-right about what's wrong with the hype, they are wholly wrong about the alternative.

[I should mention that I'm going to be speaking on a Big Data panel at the DAA Symposium in Washington DC on June 4th. The Symposiums are the probably the single best thing the DAA has created - I've attended a fair number and been consistently impressed. If you're in the area, do come out!]

The Role of Simulation in Building Powerful Enterprise Dashboards and Reporting Tools

2013-04-28T11:18:23-07:00

The vast majority of enterprise reporting and dashboarding is confined to “showing the current state”. We spend a huge amount of time showing people what happened – virtually none showing what’s likely to happen or what’s driving those numbers. We’re like weathermen who only report the temperature – never the forecast or the factors explaining that forecast. It just isn’t that useful. By embedding models into reporting, we have the opportunity to fundamentally transform them: from static snapshots that serve as little more than “Warning” signs to real business tools that help users understand what levers they have to drive performance and the implications of adjusting those levers. Embedding models inside our dashboards also creates a powerful discipline – connecting analysis and reporting – wherein every time our forecast is wrong we’re driven to go back and figure out why, continually refining our understanding of what drives the business.

It sounds great (I hope), but leaves open the question – how do you do it? How do you build models of complex digital marketing or operational systems?

There are actually quite a few different approaches – and it’s not as hard as it might seem – particularly to create an initial forecast that can be your entry-way into that virtuous cycle of continuous forecasting, analyzing, and tuning.

One technique that we’ve been experimenting with is simulation.

Why Simulation?

We’ve built models for many aspects of digital marketing: attribution, mix, internal optimization, use-cases, conversion funnels, onboarding and more.

Historically, none of these models have been simulation models. We’ve used a variety of techniques for these models – from multivariate regression to factor analysis to cluster analysis (at least to drive underlying segmentations).
So why add simulation to the mix?
Well, here are some examples from a simulation design program that we’ve created in a couple different problem spaces.

Here’s a high-level piece of one of our early, exploratory simulation models for a Product Launch:

And an even smaller piece of a broader simulation of a digital marketing system:

These aren't just diagrams. The simulation software they are built in allows you to define the stocks, flows, and parameters that make up the system and then use the simulation engine to see how changes will impact the whole system.

What do you gain by using simulation around these problems as opposed to our traditional models? The beauty of simulation is that it allows you to combine two or more very different models into a single system. For example, both our Product Launch model and our Digital Marketing System model require elements of campaign optimization – particularly mix modeling. Those elements are built using traditional techniques. But there's more to a Product Launch than optimizing your media spend and there is more to a digital marketing system than campaigns. In both cases, representing the Site System is also critical.

Now here’s the really interesting point – we use completely different techniques (segmentation and use-cases) to analyze the Site System. Those techniques are NOTHING like the techniques we use for campaign optimization. What’s more, there’s no way (at least as far as we’ve ever been able to figure out) to combine the two analytic techniques into a single method.

If you want to create a comprehensive dashboard of the digital system, however, you can’t just create models of campaigns and site and the leave them as independent entities. If you do, then you’ll miss the deep connections between them. As you adjust your marketing mix between online and offline or between display and PPC, you don’t just change the amount of traffic to the site; you change the mix of traffic to the site and the distribution of visit types. This means that campaigns and site systems need to be related to create an accurate view of the larger system.

Simulation is the only way I know of to accomplish that.

Challenges

It isn’t all hunky-dory though. First, simulation is a completely new discipline for us. We’re working to build out this capability, but I won’t pretend it’s easy. Building simulations isn’t much like building regression models (though there are some similarities), and the results of some of my initial simulations were downright puzzling.

Maybe that’s a good thing, though. Simulations create a real discipline in that, if you don’t have a reasonable working model of the world, the chances are high that your simulation won’t produce results anything like the real-world. Building a simulation forces you to tackle reality head-on and make sure you’re actually capturing most of the important elements of the system.

There’s a second problem with simulation that I’m less sanguine about. The images from above are directly from simulation software. They have a gee-whiz quality to them that I must admit I like. Particularly becuase they help capture something I think is vitally important in reporting - showing the interconnections between factors. But they most certainly are NOT the stuff of an executive dashboard. When you’ve created and tested your simulation, you need to instantiate it in a real tool. There’s no obvious tool for doing that - so grafting your simulation into a dashboard is very much a technical exercise.

Summing Up

Simulation combines traditional analysis (to drive the parameters of key systems) and the inter-relationships of complex independent systems, to create models that transform dashboards into working tools. They provide a way to forecast the future, identify the drivers of performance, track how actual performance deviated from expected, and give decision-makers the ability to try what-if scenarios with much higher real-world fidelity. It's cool stuff. They change your whole approach to reporting and dashboarding and replace static views with powerful tools for running the business.

Afterword

I’m speaking at the AIM Conference on Monday in Los Angeles – on how to measure beauty (or the Science of Aesthetics as my PPT is titled). I won’t pretend to have an answer, but it's certainly an interesting topic. I’m also very excited to be presenting out at VoC Fusion in a few weeks. It looks like a terrific conference. I’ve been building that presentation and it’s a really nice overview of my recent posts and work on creating a comprehensive Customer Intelligence System as well an early distillation of our integration with E&Y’s existing Customer Experience practice. It represents the first fruits of what I hope will soon be a very rich garden. Not only am I much pleased with the presentation, it’s great to be presenting outside the traditional analytics community. Really looking forward to it!

eMetrics: Reporting, Forecasting, Simulation and Social Media

2013-04-21T18:29:33-07:00

You might be tempted to think that my title refers to “big picture” trends taken from this past week’s eMetrics. That could be true, but in fact it’s pretty much the Table of Contents from my presentation in the Social Media Track this past Wednesday. If that seems like too much, and too disparate, ground to cover in any forty minute presentation, you’d be right. But in my defense, I was really addressing a single topic – building good enterprise dashboards – that just happens to integrate all four of these things into a single, unified approach.
Reporting often seems like the simplest part of digital measurement. Building reports is the first job we give to new analysts here at (my fingers were half-way through typing Semphonic before my mind caught up) E&Y. But my experience is that it’s actually one of the hardest jobs to get right. So hard, in fact, that I’ve never been entirely satisfied with any of our efforts. That despite re-inventing our approach repeatedely over the last four to five years.

During that time, the state-of-the-art in enterprise dashboarding has swung from walls of data to reports so pretty and information sparse they feel more like efforts in abstraction than analysis. None of it has ever felt like the definitive approach I was looking for. So not surprisingly, I’m at it again.

I started my eMetrics presentation with a brief introduction into what I find to be the biggest problem with nearly all the reports being built for enterprise digital measurement. I call this the problem of the “Current State”, and I like to illustrate it using weather reporting.

When we watch a typical weather report, we get three different types of information. In the first, we get information about what’s happening right now – as in the temperature map in the upper-left corner. Next, we hear the forecast: what weather is actually expected tomorrow or for the weekend. Finally, we may also get a model that tells us why we can expect a certain type of weather (a low pressure system is in place). Because we have no control over the weather, by far the most valuable part of a Weather report is the Forecast. We don’t really need to be told the current temperature. We can just open a window. Seeing the model gives us a deeper kind of knowledge, but it’s not something we can usually act on.

The vast majority of digital reporting in today’s enterprise is about what’s happening now (or this past week or past month). It’s all about the current state. This isn’t completely useless. Unlike the weather, we can’t simply open the window on digital and see what’s happening. So reporting on the current state has a real function.

But if that’s all we do with reporting, how valuable is it? I think this is an especially trenchant question when we have non Web-analytics means of knowing the current state. For an eCommerce site, we know sales, revenue and profitability without recourse to Web analytics. So if you’re primarily interested in taking the temperature, what else do you need?

Here’s another example from my eMetrics presentation borrowed from real-life.

We’ve all seen and built exactly this kind of report. But what good is it really? If you’re on the business end of this report, it’s telling you that you’re short of plan and the shortfall is growing. But isn’t that more of an alert than a report?

Wouldn’t something like this:

Work just about as well?

In fact, it might work rather better since you aren’t sending the same report out every week and people are more likely to pay attention when they get an alert than when they get the umpteenth copy of report that, heretofore, has never had any useful information.

The simple truth is that the Be Worried Alert and the Video Consumption report are pretty much the same thing. They tell you whether you have a problem, but nothing more. Here’s one simple question to ask yourself about your reporting – if the news isn’t bad, does it have any function? If the answer is no, then perhaps you should be generating alerts not reports.

But my aim here isn’t to argue for the superiority of alerts over traditional reporting; it’s to point out that good reporting should do more than simply describe the current state.

For a decision-maker, the ideal would be to have a tool that described the current state, provided a real forecast of what to expect, and provided a means of testing different strategies to improve future outcomes. Such a tool would be useful no matter what the current state of the system was. It would provide a means to optimize and act no matter whether the current state was bad, good or about what we expected.

Even better, such a tool would create understanding. That’s something our current generation of reports simply doesn’t do. Temperature and precipitation may be the key metrics when it comes to weather, but neither give us any basis for understanding why the current state is the way it is or how we might predict the future more accurately. KPIs are, almost by definition, NOT predictive and NOT explanatory.

To create an accurate forecast, you have to build your understanding of the underlying system. You can use a barometer without understanding why it works. But you wouldn’t create a barometer unless you understood that atmospheric pressure is an integral part of understanding weather systems.

In building a forecast, we’re forced to test our understanding of what matters in a system and how those factors are related against the real-world history of the system. That’s a powerful discipline. If we then embody in our reporting those models (as weather forecasters often do), we’ve helped our consumers not only understand what the current and likely future state are, we’ve helped understand what levers of change actually exist.

Because unlike weather, in digital, we usually have significant control over many aspects of the systems we’re reporting on. Nevertheless, I can count on two hands the reports I’ve seen that actually help decision-makers understand the levers of change and how they inter-related.

So the first half of my eMetrics presentation described why so much current reporting suffers from rapid fatigue – with users quickly tiring of and ignoring even beautiful, KPI-filled reports. These reports do nothing more than report on the current state and all too often, tell us nothing we didn't already know. It demonstrated how that focus on showing the current state leads us to reports that would be better cast as simple alerts and causes us to ignore the things that really matter to decision-makers. It introduced the idea that reporting as a tool would embody current state, forecasting, and predictive modeling. And finally, it argued that this concept of reporting would drive a virtuous cycle of analytics improvement where deviations from forecast would necessarily drive ongoing analysis to understand why things didn’t work out as expected and that this, in turn, would deepen decision maker’s understanding of the levers of change and the options available to them to optimize their business.

Which brings me to the half-way house of my presentation and the end of this blog; here, we jump from the abstract presentation of the problem to the more concrete discussion of how to build the solution. There are many, many different ways to create predictive models that can be embedded in this type of reporting tool. Indeed, it would be a considerable mistake to think that one approach is always correct. However, one of the most interesting avenues we’ve been trying uses simulation techniques to create models of moderately complex digital situations (like video consumption, product launch, or digital marketing).

In my next post, I’ll show examples from my presentation of simulations that we’ve explored and I’ll also try to lay out some tentative guidelines for when simulation is an appropriate (or even necessary) technique and when different kinds of modeling techniques might be better.

After that I’ll walk through how Social Media can play a significant role in tuning both simulation and non-simulation models in digital marketing and lay out some new techniques we’ve been using to help explore Social Media segmentation (along with the role of segmentation in modeling) to analyze demand signals. This was, after all, a presentation in the Social Media track…though I must admit that the connection felt, particularly in the early going, a bit tenuous.

So that’s how you get from Reporting to Forecasting to Simulation to Social Media in forty minutes or, in this case, a mere 3-4 blog postings.

X Change Berlin Keynote – A Look back and to the Future

2013-04-18T22:21:45-07:00

With all the activity and inevitable disruption surrounding our acquisition, I haven’t had as much chance as usual to talk about and work on X Change. We‘re a couple of months out from the Berlin event, however, and I wanted to spend a little bit of time covering some of the highlights of this year’s event – starting with the Keynote.

Several years back at the X Change U.S. we did a Four Founders Keynote that was, and has remained, my all time favorite X Change opening. That session featured four pioneers in the industry all of whom had started Web analytics technology companies and had helped shape and mature the industry. Every one of them had spent years growing a business, struggling, succeeding, adapting to, and shaping our small digital analytics world. They weren’t there to sell anything. Most of them were on to 2nd or 3rd generation startups and, in any case, they weren’t sales-people. Their talk (ably facilitated by Eric) was relaxed, personal, and reflective. It felt as if we were listening to a particularly interesting after-dinner, at the bar, conversation between four uniquely experienced and talented old-hands discussing a game we were still learning. It remains, in my mind, the quintessential X Change experience. Relaxed, honest, personal, and fascinating.

When Michael, Matthias and I were kicking around concepts for this year’s Berlin X Change, we all agreed that something similar might work very well in Europe. There are, after all, many respects in which the European market is more challenging than the U.S. – it’s certainly more fractured – and there are a host of EU companies that have pushed the boundaries of digital analytics and helped shape the market there. The hard part, was narrowing down who to ask (and some of that just came down to who we knew well).

In the end, we settled on Mathieu Liorens (CEO of AT Internet), Simon Burton (CEO of Celebrus), John Woods (Founder and CTO at iJento) and Christain Sauer (CEO at WebTrekk). Not only does this present a cross-section of key European markets, it reflects the diversity and breadth of the digital analytics space. And if you know any of these guys, you know it should be a pretty special conversation.

AT Internet and WebTrekk are both classic digital analytics solutions that have carved out significant national markets and deliver distinctive capabilities. iJento has built an open platform for data warehousing analytics and has been aggresively working to make the very challenging leap to the U.S. market. Celebrus provides a real-time data collection infrastructure for the warehouse. Classic Web analytics, open digital data warehousing, real-time data collection (and personalization)...a pretty good picture of the world of digital analytics.

So all of these companies have an interesting story to tell (and long-time readers will know that I’ve written about Celebrus, iJento and WebTrekk in the past). But more than the technologies, it’s the people that I think are fascinating. To be the Founder or CEO of a startup digital analytics company is, almost without exception in my experience, to be an interesting person. A blend of the evangelist, the technologist, the dreamer, and the pragmatist. To be good in that role requires a seemingly impossible combination of personality traits. As someone who loves technology, who started working life as a programmer, who once tried to build Web analytics software company (with dismal success), I find myself deeply resonant with their world-view and with the things they care to talk about. I suspect most of us in the digital analytics community feel the same.

When I sit in meetings with or tip a pint (as I do with Simon whenever I get the rare opportunity) with these guys, I find myself in conversations that make sense, mean more, and cut deeper than in 99% of what I do.

I hope to bring that same magic to X Change and catch, one more time, lightning in a bottle.

We’re also bringing more of the classic X Change experience to Berlin. Last year, we concentrated on just getting the basic Conference together and making sure it worked in Europe. This year, we’ve expanded the concept and added some of the longstanding features of the U.S. version. Probably the biggest addition is the day of Think Tank training prior to the actual class. Think Tank Training sessions are designed to be deep-dive, small group sessions led by top consultants. In a world where training seems almost universally focused on basic point-and-click tool usage, Think Tank is designed to something completely different.

I’m teaching two sessions there (Attribution and my Analyst's Toolkit course - which covers a full range of methods for digital analytics). Matthias Bettag is deep-diving into metrics and Michael Feiner is covering another topic dear to my heart - Voice of Customer. Maybe I can drop in for a minute and we can argue about it! It's a full day of training - you pick the classes you want - and if you are planning to come out to the Conference, it’s more than worth it to add the Think Tank day. Heck, if you can’t make the Conference, it’s worth it to make the one-day hop to Berlin and just sign-up for the training!

We’ve also, I think, found a very X Change-like venue. In a lovely neighborhood I explored last year (while taking my daughters to the nearby zoo – highly recommended by the way), the InterContinental is very, very nice and has the sophisticated but relaxed feel I consistently strive for (don't get me started on X Change's U.S. venue - the Ritz Carlton at Laguna Niguel).

Add in, of course, the two-day, all conversation, peer-to-peer Conference itself and it should be a wonderful, valuable, deeply worthwhile experience.

Hope to see you there!
Register for X Change Berlin

Mining Your Business - Fox News Features Semphonic

2013-04-14T20:49:10-07:00

Early last summer, the Semphonic offices were briefly torn apart, thoroughly disrupted for a day, and then (rather like Thing 1 & Thing 2's work in the The Cat in the Hat) carefully restored to normalcy by a Fox News Documentary crew.

Many months later, we finally got to see the fruits of that labor when Fox News aired the documentary "Your Secret's Out" this Sunday evening.

It's a hoot. Check out the old Semphonic offices, a bunch of Semphonic folks busily analyzing data, and yours truly on what "everyone should know about data analysis."

Here's the link: http://www.youtube.com/watch?v=nzJrevDHuGg

Given that they filmed us for hours, it's a little humbling. But at least we made it on air!

Social Media and the Role of Functional Classification

2013-04-07T15:01:39-07:00

I certainly intend to write more about our new home at E&Y and our strategy going forward. However, with all the transition and onboarding and setup I've been dealing with (not to mention things like Income Taxes) I have an irresistible impulse to talk about real work!
I’ve been working through a series on the creation of a Customer Intelligence System to track and report on customer attitudes. It’s a sprawling subject that touches on quite a range of topics – from online survey research to unstructured data to social media – each of which could consume a book-length treatment. For much of the series, I’ve focused on the power of online survey research and how under-utilitized it is in the enterprise. But in my last post in the series, I described a set of research questions that really couldn’t be addressed with online survey research and for a fair number of those problems, Social Media research was a likely alternative.

The advantages/disadvantages of Social Media Research vs. Site Intercept Surveys are pretty easily understood. Here’s a little cheat sheet I’ve put together that captures the key advantages of each:

Many of the key differences can be summed up in two simple differences. Social Media lets you sample against the entire universe of prospects and customers while site intercept surveys are limited to the population of your site visitors. Social media also provides a less guided, more open research experience. Conversation in Social Media is open-ended and can (and will) cover topics you’ve never considered. With survey research, you can frame very precise questions that allow you to explore differences that would be almost impossible to capture in social chatter, but you have to know what you want to investigate.

These two differences tend to combine and reinforce the advantages of each channel for different kinds of research.

All this tends to assume, however, that HOW to do Social Media research is fairly obvious. There’s no mystery, certainly, about how to do online survey research. I’ve been extremely critical of the way most enterprises conduct online survey research and the uses they make of it. Still, the basic methods of doing survey research are understood by almost everyone.

With Social Media I don’t think that’s true. It’s such a new research channel that I’m not sure anyone really understands how it can be used or what exactly are the best ways to do that research.

At a very basic level, Social Media Research involves four steps:

Sampling: identifying conversations from the population of interest

Classification: grouping those conversations into meaningful bucket

Interpretation: building models or analyzing relationships between classifications

Presentation: contextualizing the data for decision-makers

Each of these is rather unique in social. Sampling, for example, involves some steps that simply have no analog in traditional research. You have to figure out where to listen, you have to figure out your high-level gating keywords (that demarcate potentially interesting conversations from the firehose), and you have to figure out how to separate your target population from the rest of the conversations in the resulting stream. It’s hard.

It’s in the classification step, however, that most of the rare magic of Social Media Research has to happen. Classification is the step that changes unstructured text data into analytically useful, structured data. It’s only when you know what a post was about that you have a means of analyzing it.

So how do you classify social media data?

From a technology standpoint, this is a job for text analytics systems. Most listening tools have built-in capabilities for doing this and those capabilities range from machine-learning systems to simple keyword classifications. Regardless of the method or technology however, it’s the analyst’s job to decide on what type of classification is appropriate.

The most obvious answer (and the one we’ve used the most) is by topic. It just seems perfectly natural. You write (or read) a post about Product X. That’s how you classify it. At a high-level, topic classifications might be things like:

About our Advertising

About our Brand

About our Products

About our Service

About our Competitors

About our Industry

Depending on your research needs, a topic taxonomy can be endlessly extended and broadened. About our Products can be expanded into About Product X, About Product Y and About Product Z. This can, in turn, be further deepened into a set of feature classifications: About Product Speed, About Product Form Factor, About Product Reliability, etc.

Topic classifications obviously add tremendous value. They take unstructured data and make it far more meaningful. But (and there’s always a “but“ isn’t there?), it doesn’t seem to me that topic classifications meet the need of every Social Media analysis.

If you run a Facebook community, one of the basic measurement needs is to understand the popularity and impact of various types to postings. We call this editorial support – and it has a direct counterpart in traditional media.

Suppose you’ve posted a picture contest, a vote on a video, a push to a mobile app, a sweepstakes drive, a reference to National Pi day, and a discount offer. A topic classification of these posts isn’t likely to be particularly revealing. Nor will it help to establish comparability or to build segmentations. Knowing that a visitor re-tweeted National Pi day doesn’t seem all that useful unless you have a steady stream of Math-oriented postings. You might classify this as a holiday posting, but does it really suggest that a Valentine’s post would perform similarly or would appeal to the same audience? I think not.

The fact is that while a small percentage of social media content does classify along interesting topic lines, a huge part of what gets communicated isn’t necessarily topic oriented. It’s not unreasonable to suggest that the topic-oriented content is the more important stuff, but it would be wrong to suggest that the rest of the communication is just noise.

Particularly for a community manager creating non-topical posts, that doesn’t seem like a useful answer.

To handle this type of classification, we’ve borrowed one of our old Web analytics tricks – Functional classification. In Functional analysis, you classify pages not by their content but by their function on the Website. So Web pages have functional categories like Engager, Router, Convincer, Informer, and Closer. These Functional categories turn out to be very useful for establishing the proper measurement of pages (if a page is a Router, you measure how well it moves visitors to the appropriate sections of the Website) and for establishing comparability of Web pages. Pages that perform similar functions can and should be compared. Pages that perform different functions shouldn’t be compared.

It’s a simple, elegant solution to a significant categorization problem around Web pages and it seems to apply equally well to Social Media.

Community posts, after all, are designed with similar functions in mind. Some are supposed to attract new members. Some are supposed to drive to products. Some are supposed to put a human face on the brand. Some are supposed to give community members a bit of a laugh and keep them engaged.

Just as you wouldn’t expect a Router page on a Website to perform (or be measured) in the same way as a Product Detail Page, you wouldn’t expect a post designed to give folks a bit of a laugh to perform (or be measured) in the same way as a post designed to drive to a mobile app download.‘

Keep in mind that a Functional and Topic Categorization aren’t mutually exclusive. They are complementary and can exist simultaneously. A post can be About Product X and be Informational. Or it can be about Hurricane Sandy and designed to put a human face on a brand.

For most community managers (and for a great deal of Social segmentation), I think the Functional Taxonomy is probably more interesting than a topic-based grouping. It creates a categorization that can be used to compare posts that, from a topical perspective, seem very distinct. Even better, just as with Websites, it creates a natural path to measurement. When you know what something is for, you are half-way to understanding how to measure it!

In my next post, I’m going to show a sample Functional Taxonomy for Social Media and then describe how we’re using that classification to tackle one of the holy of holiest questions in marketing – how to measure the impact of brand advertising.