I’m a bad golfer and a worse baseball player, but I found myself wanting to combine the two sports by calculating Woods’s “batting average” for PGA tour events. This required two major definitional leaps:

• An “at bat” was a tournament
• A “hit” was a win

This is a whopper of a stretch, I realize, but stick with me, anyway.

The batting average math is now simply: with Woods’s win, his career batting average in tour events was 78/300, or .260! In baseball, a “good” hitter bats over .300. Of course, for my definitions to hold up, in real baseball, a player would only get credited with a hit if he hit a game-winning walkoff home run every time he got a hit!

This led me to wonder what Woods’s batting average over his career to date has been. So, using data from Woods’ profile on pgatour.com, I plotted it out (even though Woods was an amateur until 1996, the tournaments he played in before that still counted as PGA tour starts):

His batting average peaked in 2009, just a couple of months before he had his worst Thanksgiving ever.

As the end of the chart shows, it does look like he is on his way back. Keep in mind that, like a real batting average, the fewer tournaments he’d played in, the more a win would increase his cumulative average and the less a non-win would drop it. That’s one reason that, in baseball, there is more focus on the batting average for the season than on the career batting average.

So, that got me wondering how this tour season compares to Woods’s past seasons. The gray in the chart below shows his average as of the end of each season:

To date, this is his highest win percentage of any year other than 2008, which was severely shortened by a knee injury. In 2008, he won 4 out of 6 PGA events before his season ended. In 2013, he has won 4 out of 7 so far!

Idle fun with Excel and online data!

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | No comment | Add to del.icio.us
Post tags: golf, Tiger Woods

Feed enhanced by Better Feed from Ozh

I’ve attended the “privacy” session at a number of recent eMetrics, and the San Francisco one represented a big step forward in terms of specificity. “Privacy” seems to be a powerful word in the #measure industry — it’s a single word that seems to magically turn many people and companies into ostriches! It’s not that we want to avoid the topic, but there is so much complexity and uncertainty that putting our heads in the sand and kicking the can down the road (everyone loves a good mixed metaphor, right?) seems to be the default course of action.

In the session sardonically titled “Attend this Session or Pay €1 Million,” René Dechamps Otamendi of Mind Your Privacy covered European privacy regulations and Joanne McNabb of the California Department of Justice covered California and US privacy regulations.

When Pop Culture Picks It Up…

I was a West Wing fan, but had no memory of this clip that René shared:

When you’ve got mainstream network television referencing a topic, it’s a topic that is at least on the periphery of the mainstream.

“Fundamental Right” vs. “Business/Consumer Negotiation”

René pointed out that many Americans miss the point when it comes to the European privacy regulations — in typical America-centric fashion, we ignore history. We see privacy as a topic that is up for debate — how do we protect consumers with minimal regulation so that businesses can capitalize on as much personal data as possible.

In Europe…there was the Holocaust. René described how, in The Netherlands prior to WWII, the  government maintained detailed and accurate records on every citizen. When the Nazis invaded, this data made it very easy for them to identify and persecute Jews. Of the 140,000 Jews who lived in The Netherlands prior to 1940, only 30,000 survived the war, and historians point to the availability of this data as one of the main reasons for this. Yikes! For many Europeans, this sort of history is both deeply embedded and strongly linked to the topic of personal and online privacy.

Thinking of privacy as an undisputed as a fundamental right is somewhat eye-opening.

It Doesn’t Matter Where Your Company Is Based

This isn’t exactly news, but it seems to be one of the excuses marketers use for burying their heads in the sand: “We’re based in Ohio — not California or Europe. So, how much do we have to worry about privacy regulations there?”

The answer comes down to where your customers are. The European Directive, as well as California regulations, do not care where a company is based. They’re focused on where the consumers interacting with those companies are. Pull up your visitor geography reports in your web analytics platform and look at where your traffic is coming from — anywhere that has a non-miniscule percentage of traffic is likely somewhere that you need to understand privacy-regulation-wise.

Why California instead of “the U.S.?”

Joanne pointed out that California is clearly in the forefront when it comes to developing, implementing, and enforcing privacy regulations in the U.S. The California Online Protection and Privacy Act (CalOPPA) has been in effect since 2004 (although not widely understood for the first few years). That’s closing in on a decade!

To me, this sounded a lot like fuel economy standards in the auto industry — California is a large enough market that businesses can’t afford to ignore the state’s residents. At the same time, other states, and the federal government (because the U.S. has a long — and checkered — history of using the states as laboratories for testing ideas), are watching California to see what they figure out. There is a very good chance that what works for California will be a basis for other states and for federal regulations.

Is California the Same As Europe?

Yes and no. They’re the same in that they have a similar orientation towards “individuals’ rights.” They’re the same in that they are increasingly starting to enforce their regulations (with very real fines levied on companies).

They’re different…in that the U.S. and Europe are different — both culturally and structurally.

They follow developments in each others’ worlds, but they’re not actively marching towards a single, unified regulation.

So, Where Should Companies Start?

Step 1: Check your privacy policy. Really. Read it. Read it for your country-specific sites (simply translating your U.S. privacy policy into German doesn’t work!). If you give it a really close read, are you even complying with what you say you are?

Step 2: Learn some details. For Europe, reach out to René at the email address in the image below. He’s got a document that explains the ins and outs of EU privacy regulations (if the number “27″ doesn’t mean anything to you, you haven’t learned enough):

For California, one resource is the California Attorney General’s site for online privacy. Unfortunately, it is a bureaucratically built site, so be ready for some heavy document-wading.

Step 3: Educate your company. This one is no small task, because, when asked who to include in that discussion, it seemed like a simpler answer would have come if the question was who not to include. The web team, marketing, legal, and IT are a good start. The best hook is “We could be fined 1,000,000 euros…”

In Short: It’s Still Messy, but Things Are Getting Clearer

The heading says it all. “We” all need to take our heads out of the sand and get smarter on this. If a regulatory agency comes calling, the worst response is, “Tell me who you are again?” The best (but not currently possible) response is, “We’re totally compliant.” A good response is, “We’re working on it, here’s what we’ve done, and here’s our roadmap to do more.”

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | No comment | Add to del.icio.us
Post tags: eMetrics, privacy

Feed enhanced by Better Feed from Ozh

On Tuesday, I attended Ian Lurie’s presentation: “Data That Persuades: How to Prove Your Point.” This session was a “fist pumper” for me, as Ian is as frustrated by crappy data visualization as I am (he led off the presentation by showing a mouth guard, sharing that he wears one at night because he grinds his teeth, and then noting that the stress of seeing data poorly presented was a big source of the stress driving that grinding!).

One of the ways Ian illustrated the importance of putting care into the way data gets presented was with this image:

I think it’s fair to say this a representation of the three types of memory:

• The “lizard brain” represents iconic memory — the “visual sensory register.” It’s where preattentive cognitive processing occurs. If we don’t put something forth that is clear and instantaneously perceptible, then the information won’t get past the lizard brain.
• The “ape brain” represents short-term memory — where conscious thought and basic processing occurs. The initial, “Do I care?” question gets asked and answered.
• The “human brain” represents longer-term memory — where we actually need to digest the information and develop and implement a response.

Ian also spent a lot of time on Tufte’s data-ink ratio — imploring the audience to be heavily reductionist in the visualization of data by removing extraneous words, lines, tick marks, etc. so that “the data” really comes through.

Otherwise, the recipients of the data will be like screaming goats:

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | One comment | Add to del.icio.us
Post tags: eMetrics

Feed enhanced by Better Feed from Ozh

The closing keynote at eMetrics was Matt Wilson and Andrew Janis talking about how they’ve been evolving the role of digital (including social) analytics at General Mills.

Almost as a throwaway aside, Matt noted that one of the ways he has gone about increasing the use of their web analytics platform by internal users is with video:

1. He keeps a running list of common use cases (types of data requests)
2. He periodically makes 2-minute (or less) videos of how to complete these use cases

Specifically:

• He uses Snagit Pro to do a video capture of his screen while he records a voiceover
• If a video lasts more than 120 seconds, he scraps it and starts over

Outside of basic screen caps with annotations, the “video with a voiceover” is my favorite use of Snagit. When I need to “show several people what is happening,” it’s a lot more efficient than trying to find a time for everyone to jump into GoToMeeting or a Google Hangout. I just record my screen with my voiceover, push the resulting video to YouTube (in a non-public way — usually “anyone with the link” mode), and shoot off an email.

I’ve never tried this with analytics demos — as a way to efficiently build a catalog of accessible tutorials — but I suspect I’m going to start!Similar Posts:

• Analysts as Community Managers’ Best Friends
• eMetrics Washington, D.C. 2010 — Fun with Twitter
• #eMetrics Reflection: Visits / Visitors / Cohorts / Lifetime Value
• Gilligan’s eMetrics Recap — Washington, D.C. 2010

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | No comment | Add to del.icio.us
Post tags: eMetrics, General Mills

Feed enhanced by Better Feed from Ozh

One of the first sessions I attended at last week’s eMetrics was Jim Novo’s session titled “The Evolution of an Attribution Resolution.” We’ll (maybe) get to the “attribution” piece in a separate post (because Jim turned on a light bulb for me there), but, for now, we’ll set that aside and focus on a sub-theme of his talk.

Later at the conference, Jennifer Veesenmeyer from Merkle hooked me up with a teaser copy of an upcoming book that she co-authored with others at Merkle called It Only Looks Like Magic: The Power of Big Data and Customer-Centric Digital Analytics. (It wasn’t like I got some sort of super-special hookup. They had a table set up in the exhibit hall and were handing copies out to anyone who was interested. But I still made Jennifer sign my copy!) Due to timing and (lack of) internet availability on one of the legs of my trip, I managed to read the book before landing back in Columbus.

A Long-Coming Shift Is About to Hit

We’ve been talking about being “customer-centric” for years. It seems like eons, really. But, almost always, when I’ve hear marketers bandy about the phrase, they mean, “We need to stop thinking about ‘our campaigns’ and ‘our site’ and ‘our content’ and, instead, start focusing on the customer’s needs, interests, and experiences.” That’s all well and good. Lots of marketers still struggle to actually do this, but it’s a good start.

What I took away from Jim’s points, the book, and a number of experiences with clients over the past couple of years is this:

Customer-centricity can be made much more tangible…and much more tactically applicable when it comes to effective and business-impacting analytics.

This post covers a lot of concepts that, I think, are all different sides of the same coin.

Visitors Trump Visits

Cross-session tracking matters. A visitor who did nothing of apparent importance on their first visit to the site may do nothing of apparent importance across multiple visits over multiple weeks or months. But…that doesn’t mean what they do and when they do it isn’t leading to something of high value to the company.

Caveat (defended) to that:

Does this means visits are dead? No. Really, unless you’re prepared to answer every new analytics question with, “I’ll have an answer in 3-6 months once I see how visitors play out,” you still need to look at intra-session results.

When I asked Jim about this, his response totally made sense. Paraphrasing heavily: “Answering a question with a visit-driven response is fine. But, if there’s a chance that things may play out differently from a visitor view, make sure you check back in later and see if your analysis still holds over the longer term.”

Cohort Analysis

Cohort analysis is nothing more than a visitor-based segment. Now, a crap-ton of marketers have been smoking the Lean Startup Hookah Pipe, and, in the feel-good haze that filled the room, have gotten pretty enamored with the concept. Many analysts, myself included, have asked, “Isn’t that just a cross-session segment?” But “cross-session segment” isn’t nearly as fun to say.

Here’s the deal with cohort analysis:

• It is nothing more than an analysis based around segments that span multiple sessions
• It’s a visitor-based concept
• It’s something that we should be doing more (because it’s more customer-centric!)

The problem? Mainstream web analytics tools capture visitors cross-session, and they report cross-session “unique visitors,” but this is only in aggregate. You can dig into Adobe Discover to get cross-session detail, or, I imagine, into Adobe Insight, but that is unsatisfactory. Google has been hinting that this is a fundamental pivot they’re making — to get more foundationally visitor-based in their interface. But, Jim asked the same question many analysts are:

Having started using and recommending visitor-scope custom variables more and more often, I’m starting to salivate at the prospect of “visitor” criteria coming to GA segments!

Surely, You’ve Heard of “Customer Lifetime Value?”

“Customer Lifetime Value” is another topic that gets tossed around with reckless abandon. Successful retailers, actually, have tackled the data challenges behind this for years. Both Jim and the Merkle book brought the concept back to the forefront of my brain.

It’s part and parcel to everything else in this post: getting beyond, “What value did you (the customer) deliver to me today?” to “What value have you (or will you) deliver to me over the entire duration of our relationship” (with an eye to the time value of money so that we’re not just “hoping for a payoff wayyyy down the road” and congratulating ourselves on a win every time we get an eyeball).

Digital data is actually becoming more “lifetime-capable:”

• Web traffic — web analytics platforms are evolving to be more visitor-based than visit-based, enabling cross-session tracking and analysis
• Social media — we may not know much about a user (see the next section), but, on Twitter, we can watch a username’s activity over time, and even the most locked down Facebook account still exposes a Facebook ID (and, I think, a name)…which also allows tracking (available/public) behavior over time
• Mobile — mobile devices have a fixed ID. There are privacy concerns (and regulations) with using this to actually track a user over time, but the data is there. So, with appropriate permissions, the trick is just handling the handoff when a user replaces their device

Intriguing, no?

And…Finally…Customer Data Integration

Another “something old is new again” is customer data integration — the “customer” angle of of the world of Master Data Management. In the Merkle book, the authors pointed out that the illusive “master key” that is the Achilles heel of many customer data integration efforts is getting both easier and more complicated to work around.

One obvious-once-I-read-it concept was that there are fundamentally two different classes of “user IDs:”

• A strong identifier is “specifically identifiable to a customer and is easily available for matching within the marketing database.”
• A weak identifier is “critical in linking online activity to the same user, although they cannot be used to directly identify the user.”

Cookie IDs are a great example of a weak identifier. As is a Twitter username. And a Facebook user ID.

The idea here is that a sophisticated map of IDs — strong identifiers augmented with a slew of weak identifiers — starts to get us to a much richer view of “the customer.” It holds the promise of enabling us to be more customer-centric. As an example:

• An email or marketing automation system has a strong identifier for each user
• Those platforms can attach a subscriber ID to every link back to the site in the emails they send
• That subscriber ID can be picked up by the web analytics platform (as a weak identifier) and linked to the visitor ID (cookie-based — also a weak identifier)
• Now, you have the ability to link the email database to on-site visitor behavior

This example is not a new concept by any means. But, in  my experience, the way each of the platforms involved in a scenario like this has preferred to work is that they set their own strong and weak identifiers. What I took away from the Merkle book is that we’re getting a lot closer to being able to have those identifiers flow between systems.

Again…privacy concerns cannot be ignored. They have to be faced head on, and permission has to be granted where permission would be expected.

Lotta’ Buzzwords…All the Same Thing?

Nothing in this post is really “new.” They’re not even “new to me.” The dots I hadn’t connected was that they are all largely the same thing.

That, I think, is exciting!

 Similar Posts:

• eMetrics Washington, D.C. 2010 — Fun with Twitter
• Gilligan’s eMetrics Recap — Washington, D.C. 2010
• Analysts as Community Managers’ Best Friends
• #eMetrics Reflection: Self-Service Analysis in 2 Minutes or Less

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | 2 comments | Add to del.icio.us
Post tags: eMetrics, Jim Novo, Merkle

Feed enhanced by Better Feed from Ozh

Last fall, I pitched a session topic to Jim Sterne for the eMetrics conference that occurred last week. At the time, I was just a few weeks into my job at Clearhead, and I figured that, by April 2013, I’d easily have a fully baked, deliverable-supporting process that I could use as the basis for the session.

You’re expecting this sentence — the one following that last paragraph — to say, “Boy…was I wrong!” The fact is…I was mostly right!

A handful of articles, posts, and content all came out of my effort to get spit and polish on the material in time for the session:

• The eMetrics session itself, as well as the various downloadable templates that accompanied it, are posted at clearhead.me/emetrics
• I did a high-level summary of the content and approach in a Practical eCommerce article that was published last week
• My unified theory of analytics (requests) was an operational umbrella for the ADAPT to Act and Learn thinking
• Thanks to Avinash, I sorta’ rediscovered Lean Analytics right as I was wrapping up the presentation

Lots of content. You be the judge if it’s good content. Or, if you’re reading this shortly after it got posted and you’re in central Ohio, come get an abbreviated version at this month’s Columbus Web Analytics Wednesday.

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | No comment | Add to del.icio.us
Post tags:

Feed enhanced by Better Feed from Ozh

“I don’t know what I don’t know, so can you just analyze the traffic to the site and summarize your insights?”

“Can I get a weekly report showing top pages?”

“I need a report from Google Analytics that tells me the gender breakdown for the site.”

“Can you break down all of our metrics by: new vs. returning visitors, weekend vs. weekday visitors, working hours vs. non-working hours visitors, and affiliate vs. display vs. paid search vs. organic search vs. email visitors? I think there might be something interesting there.”

“Can you do an analysis that tells me why the numbers I looked at were worse this month than last?”

“Can you pull some data to prove that we need to add cross-selling to our cart?”

“We rolled out a new campaign last week. Can you do some analysis to show the ROI we delivered with it?”

“What was traffic last month?”

“I need to get a weekly report with all of the data so I can do an analysis each week to find insights.”

The list goes on and on. And, in various ways, they’re all examples of well-intended requests that lead us down the Nefarious Path to Reporting Monkeydom. It’s not that the requests are inherently bad. The issue is that, while they are simple to state, they often lack context and lack focus as to what value fulfilling the request will deliver. That leads to the analyst spending time on requests that never should have been worked on at all, making risky assumptions as to the underlying need, and over-analyzing in an effort to cover all possible bases.

I’ve given this a lot of thought for a lot of years (I’m not exaggerating — see the first real post I wrote on this blog almost six years ago…and then look at the number of navel-gazing pingbacks to it in the comments). And, I’ve become increasingly convinced that there are two root causes for not-good requests being lobbed to the analytics team:

• A misperception that “getting the data” is the first step in any analysis — a belief that surprising and actionable insights will pretty much emerge automagically once the raw data is obtained.
• A lack of clarity on the different types and purposes of analytics requests — this is an education issue (and an education that has to be 80% “show” and 20% “tell”)

I think I’m getting close to some useful ways to address both of these issues in a consistent, process-driven way (meaning analysts spend more time applying their brainpower to delivering business value!).

Before You Say I’m Missing the Point Entirely…

The content in this post is, I hope, what this blog has apparently gotten a reputation for — it’s aimed at articulating ideas and thoughts that are directly applicable in practice. So, I’m not going to touch on any of the truths (which are true!) that are more philosophical than directly actionable:

• Analysts need to build strong partnerships with their business stakeholders
• Analysts have to focus on delivering business value rather than just delivering analysis
  
• Analysts have to stop “presenting data” and, instead “effectively communicate actionable data-informed stories.”

All of these are 100% true! But, that’s a focus on how the analyst should develop their own skills, and this post is more of a process-oriented one.

With that, I’ll move on to the three types of analytics requests.

Hypothesis Testing: High Value and SEXY!

Hands-down, testing and validation of hypotheses is the sexiest and, if done well, highest value way for an analyst to contribute to their organization. Any analysis — regardless of whether it uses A/B or multivariate testing, web analytics, voice of the customer data, or even secondary research — is most effective when it is framed as an effort to disprove or fail to disprove a specific hypothesis. This is actually a topic I’m going to go into a lot of detail (with templates and tools) on during one of the eMetrics San Francisco sessions I’m presenting in a couple of weeks.

The bitch when it comes to getting really good hypotheses is that “hypothesis” is not a word that marketers jump up and down with excitement over. Here’s how I’m starting to work around that: by asking business users to frame their testing and analysis requests in two parts:

Part 1: “I believe…[some idea]”

Part 2: “If I am right, we will…[take some action]“

This construct does a couple of things:

• It forces some clarity around the idea or question. Even if the requestor says, “Look. I really have NO IDEA if it’s ‘A’ or ‘B’!” you can respond with, “It doesn’t really matter. Pick one and articulate what you will do if that one is true. If you wouldn’t do anything different if that one is true, then pick the other one.”
• It forces a little bit of thought on the part of the requestor as to the actionability of the analysis.

And…it does this in plain, non-scary English.

So, great. It’s a hypothesis. But, how do you decide which hypotheses to tackle first? Prioritization is messy. It always is and it always will be. Rather than falling back on the simplistic theory of “effort and expected impact” for the analysis, how about tackling it with a bit more sophistication:

• What is the best approach to testing this hypothesis (web analytics, social media analysis, A/B testing, site survey data analysis, usability testing, …)? That will inform who in your organization would be best suited to conduct the analysis, and it will inform the level of effort required. 
• What is the likelihood that the hypothesis will be shown to be true? Frankly, if someone is on a fishing expedition and has a hypothesis that making the background of the home page flash in contrasting colors…common sense would say, “That’s a dumb idea. Maybe we don’t need to prove it if we have hypotheses that our experience says are probably better ones to validate.”
  
• What is the likelihood that we actually will take action if we validate the hypothesis? You’ve got a great hypothesis about shortening the length of your registration form…but the registration system is so ancient and fragile that any time a developer even tries to check the code out to work on it, the production code breaks. Or…political winds are blowing such that, even if you prove that always having an intrusive splash page pop up when someone comes to your home page is hurting the site…it’s not going to change.
• What will be the effort (time and resources) to validate the hypothesis? Now, you damn well better have nailed down a basic approach before answering this. But, if it’s going to take an hour to test the hypothesis, even if it’s a bit of a flier, it may be worth doing. If it’s going to take 40 hours, it might not be.
• What is the business value if this hypothesis gets validated (and acted upon)? This is the “impact” one, but I like “value” over “impact” because it’s a little looser.

I’ve had good results when taking criteria along these lines and building a simple scoring system — assigning High, Medium, Low, or Unknown for each one, and then plugging in some weighted scores for each value for each criteria. The formula won’t automatically prioritize the hypotheses, but it does give you a list that is sortable in a logical way, It, at least, reveals the “top candidates” and the “stinkers.”

Performance Measurement (think “Reporting”)

Analysts can provide a lot of value by setting up automated (or near-automated) performance measurement dashboards and reports. These are recurring (hypothesis testing is not — once you test a hypothesis, you don’t need to keep retesting it unless you make some change that makes sense to do so).

Any recurring report* should be goal- and KPI-oriented. KPIs and some basic contextual/supporting metrics should go on the dashboard, targets need to be set (and set up such that alerts are triggered when a KPI slips). Figuring out what should go on a well-designed dashboard comes down to answering two questions:

1. What are we trying to achieve? (What are our business goals for this thing we will be reporting on?)
2. How will we know that we’re doing that? (What are our KPIs?)

They need to get asked and answered in order, and that’s a messier exercise oftentimes than we’d like it to be. Analysts can play a strong role in getting these questions appropriately answered…but that’s a topic for another time.

Every other recurring report that is requested should be linkable back to a dashboard (“I have KPIs for my paid search performance, so I’d like to always get a list of the keywords and their individual performance so I have that as a quick reference if a KPI changes drastically.”)

Having said that, a lot of tools can be set up to automatically spit out all sorts of data on a recurring basis. I resist the temptation to say, “Hey…if it’s only going to take me 5 minutes to set it up, I shouldn’t waste my time trying to validate its value.” But, it can be hard to not appear obstructionist in those situations, so, sometimes, the fastest route is the best. Even if, deep down, you know you’re delivering something that will get looked at the first 2-3 times it goes out…and will never be viewed again.

Quick Data Requests — Very Risky Territory (but needed)

So, what’s left? That would be requests of the,. “What was traffic to the site last month?” ilk. There’s a gross misperception when it comes to “quick” requests that there is a strong correlation between the amount of time required to make the request and the amount of time required to fulfill the request. Whenever someone tells me they have a “quick question,” I playfully warn them that the length of the question tends to be inversely correlated to the time and effort required to provide an answer.

Here’s something I’ve only loosely tested when it comes to these sorts of requests. But, I’ve got evidence that I’m going to be embarking on a journey to formalize the intake and management of these in the very near future, so I’m going to go ahead and write them down here (please leave a comment with feedback!).

First, there is how the request should be structured — the information I try to grab as the request comes in:

• The basics – who is making the request and when the data is needed; you can even include a “priority” field…the rest of the request info should help vet out if that priority is accurate.
• A brief (255 characters or so) articulation of the request — if it can’t be articulated briefly, it probably falls into one of the other two categories above. OR…it’s actually a dozen “quick requests” trying to be lumped together into a single one. (Wag your finger. Say “Tsk, tsk!”
• An identification of what the request will be used for — there are basically three options, and, behind the scenes, those options are an indication as to the value and priority of the request:
  • General information — Low Value (“I’m curious,” “It would be be interesting — but not necessarily actionable — to know…”)
  • To aid with hypothesis development — Medium Value (“I have an idea about SEO-driven visitors who reach our shopping cart, but I want to know how many visits fall into that segment before I flesh it out.”)
  • To make a specific decision — High Value
• The timeframe to be included in the data — it’s funny how often requests come in that want some simple metric…but don’t say for when!
• The actual data details — this can be a longer field; ideally, it would be in “dimensions and metrics” terminology…but that’s a bit much to ask for many requestors to understand.
• Desired delivery format — a multi-select with several options:
  • Raw data in Excel
  • Visualized summary in Excel
  • Presentation-ready slides
  • Documentation on how to self-service similar data pulls in the future

The more options selected for the delivery format, obviously, the higher the effort required to fulfill the request.

All of this information can be collected with a pretty simple, clean, non-intimidating intake form. The goal isn’t to make it hard to make requests, but there is some value in forcing a little bit of thought rather than the requestor being able to simply dash off a quickly-written email and then wait for the analyst to fill in the many blanks in the request.

But that’s just the first step.

The next step is to actually assess the request. This is the sort of thing, generally, an analyst needs to do, and it covers two main areas:

• Is the request clear? If not, then some follow-up with the requestor is required (ideally, a system that allows this to happen as comments or a discussion linked to the original request is ideal — Jira, Sharepoint, Lotus Notes, etc.)
• What will the effort be to pull the data? This can be a simple High/Medium/Low with hours ranges assigned as they make sense to each classification.

At that point, there is still some level of traffic management. SLAs based on the priority and effort, perhaps, and a part of the organization oriented to cranking out those requests as efficiently as possible.

The key here is to be pretty clear that these are not analysis requests. Generally speaking, it’s a request for data for a valid reason, but, in order to conduct an analysis, a hypothesis is required, and that doesn’t fit in this bucket.

So, THEN…Your Analytics Program Investment

If the analytics and optimization organization is framed across these three main types of services, then conscious investment decisions can be made:

• What is the maximum % of the analytics program cost that should be devoted to Quick Data Requests? Hopefully, not much (20-25%?).
• How much to performance measurement? Also, hopefully, not much — this may require some investment in automation tools, but once smart analysts are involved in defining and designing the main dashboards and reports, that is work that should be automated. Analysts are too scarce for them to be doing weekly or monthly data exports and formatting.
• How much investment will be made in hypothesis testing? This is the highest value

With a process in place to capture all three types of efforts in a discrete and trackable way enables reporting back out on the value delivered by the organization:

• Hypothesis testing — reporting is the number of hypotheses tested and the business value delivered from what was learned
• Performance measurement — reporting is the level of investment; this needs to be done…and it needs to be done efficiently
• Quick data requests — reporting is output-based: number of requests received, average turnaround time. In a way, this reporting is highlighting that this work is “just pulling data” — accountability for that data delivering business value really falls to the requestors. Of course, you have to gently communicate that or you won’t look like much of a team player, now, will you?

Over time, shifting an organization to think it terms of actionable and testable hypotheses is the goal — more hypotheses, fewer quick data requests!

And, of course, this approach sets up the potentially to truly close the loop and follow through on any analysis/report/request delivered through a Digital Insight Management program (and, possibly, platform — like Sweetspot, which I haven’t used, personally, but which I love the concept of).

What Do You Think?

Does this make sense? It’s not exactly my opus, but, as I’ve hastily banged it out this evening, I realize that it includes many of the ways that I’ve had the most success in my analytics career, and it includes many of the structures that have helped me head off the many ways I’ve screwed up and had failures in my analytics career.

I’d love your thoughts!

*Of course, there are always valid exceptions.

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | 9 comments | Add to del.icio.us
Post tags:

Feed enhanced by Better Feed from Ozh

That soapbox (although hopefully presented in a much less preachy way than “soapbox” connotes) is one of the core elements of one of the eMetrics sessions I’ll be leading next month. And, I also got to try to capture those thoughts in a recent Practical eCommerce article. The premise for the article comes from the cliché that, when all you have is a hammer, all the world looks like a nail. Not a week goes by when I don’t have a co-worker or client view their “main” analytics or optimization platform as a universal tool.

Web analytics tell you what visitors did. Site surveys tell you what they wanted to do and, to a certain extent, who they are. Testing platforms let you construct a parallel universe. You get the idea.

Read more in the article itself.

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | No comment | Add to del.icio.us
Post tags:

Feed enhanced by Better Feed from Ozh

Just like the old adage that, if a vacation doesn’t end before you wish it did, then you stayed too long, one of my measures for a conference is how many thinks I didn’t get to do that I wish I had.

In the case of Summit, I had a pretty healthy list:

• I didn’t get to see more of Adobe Social – Adobe has been all sorts of crazy hard at work on the product, and the glimpses I caught in keynotes show that there’s a lot going on with it.
• I missed Unsummit — the unaffiliated, peer-driven conference on Tuesday. I didn’t actually know about Unsummit, which, I think, is pretty common with first-timers.
• Microsoft Surface — Tuesday night, I had a conversation with some guys from MSN who indicated they all had Surfaces. I’ve never actually seen one up close, so I was fully expecting that I’d bump into one of those guys later in the conference and get a look. That’s not really related to analytics, but it’s a gadgethead’s regret.

Then, there was a list of people I regret not getting to hang out with or not getting to hang out with more:

• Carmen Sutter – Carmen is one of the Adobe Social product managers who I met last fall shortly before she dived into that role. I got to see and meet a lot of people, but I really racked up the near misses with Carmen. I’m pretty sure she wasn’t actively avoiding me.
• Ben Gaines – Ben’s an Adobe Analytics product manager, and I did manage to chat with him on Tuesday night for a bit, attend his “Sitecatalyst Tips” breakout, and swap a number of tweets. But, still, you really can’t get enough of Ben, and we didn’t get enough time to solve the world’s problems. I’ll just have to lobby to get him to Columbus for our April Web Analytics Wednesday. Cross your fingers if you’re in Columbus.
• Gregory Ng — Chief Strategy Officer for Brooks Bell and guy-who-never-sleeps-as-he-pursues-a-gazillion-quirky-side-interests. We chatted for a bit at the welcoming reception and then failed to connect again. That’s one of the things about Summit — you get 10 minutes with a person and say, “Let’s catch up later,”…and then the conference is over!
• Jason Thompson — even worse than Greg, I saw Jason right as I arrived at the hotel on Wednesday evening…and never saw him again (excluding tweets). Curses!

The list of things I don’t regret is wayyyyy longer — I saw some neat things, learned some good stuff, and got to hang out with some great people!Similar Posts:

• #AdobeSummit Is Next Week

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | One comment | Add to del.icio.us
Post tags: Adobe, Adobe Social, Adobe Summit

Feed enhanced by Better Feed from Ozh

This post isn’t long, but I picked up a few real nuggets of brilliance that were very tactical tips that I’ll be exploring further in my day job in the next week or two.

Finding Questions in Site Search

Nancy Koons might be the nicest person on the planet (feel free to leave a comment if you think you know someone nicer) and also is the source of two of my favorite tips (neither of which is at all Adobe-specific).

I’m a fan of site search data (I even wrote a Practical eCommerce article on the subject last year). Nancy set up the tip by explaining why site search analytics makes sense, but then she gave this tip:

“Filter your site search terms report by the words: who, what, why, where, and how.”

Literally. Filter for those 5 words. What this will give you a list of results that are full questions people typed into your search box. These are all going to be unique — they’ll be wayyyy out on the long tail of the report. But, they’re also context-rich. They tell you exactly what the visitor was trying to do.

Cool, huh?

A Poster of Insights

This next tip is also completely to Nancy’s credit. The entire panel touched on the need to not just do analysis, but to effectively communicate their results. Nancy shared a situation where her team was doing a “year in review” and had a number of useful insights that they had turned up over the course of the year. The challenge they had was, “How to actually communicate them in a way that they wouldn’t be forgotten at the point when they would be most useful to apply in the coming year?”

The solution: a printed poster that captured the insights that would most be able to be applied in the coming year. The poster was heavily designed — almost infographic-level detail. The posters were good-sized — they looked to be 24-30″ wide and maybe 15″ tall — and were distributed to the marketers to put up in their offices. Brilliant! A constant reminder/reference of the most useful learning from the prior year!

Report Builder…

There were several tips that were geared towards “don’t present the data directly from within SiteCatalyst,” which meant Report Builder and Excel got some real love. Report Builder is a great way to get automated data updates into Excel, where the richer visualization options for the platform can be put to full use.

If you want to hone your Report Builder and Excel chops, consider Kevin Willeitner’s class this fall in Columbus (and stick around for #ACCELERATE).

Context Variables in SiteCatalyst

I’m not proud. I’ll admit that I totally missed context variables in the v15 release…until Ben Gaines explained them in his “10 tips” session. Basically, remove developer confusion over the difference between props, eVars, and event.

Did You Pick Up a Favorite Tip?

I got a number of other little nuggets and ideas, but these were the ones I most felt like I’d be putting to use almost immediately. What did you take back from Salt Lake City that you’ll be putting into action soon?

 Similar Posts:

• #AdobeSummit Takeaways: Adobe Marketing Cloud
• #AdobeSummit Takeaways: Adobe Puts on a GREAT Event
• Web Analytics Tools Comparison — Columbus WAW Recap Part 2
• Columbus Web Analytics Wednesday Recap: Don’t “Antisappoint” Visitors

© Tim for Gilligan on Data by Tim Wilson, 2013. | Permalink | One comment | Add to del.icio.us
Post tags: Adobe, Adobe Summit

Feed enhanced by Better Feed from Ozh