Gilligan on Data by Tim Wilson

The Three-Legged Stool of Effective Analytics: Plan, Measure, Analyze

Tim Wilson — Wed, 22 Feb 2012 11:00:16 +0000

Several weeks ago, Stéphane Hamel wrote a post that got me all re-smitten with his thought process. In the post, he postulated that there are three heads of online analytics. He covered three different skillsets needed to effectively conduct online analytics: business acumen, technical (tools) knowledge, and analysis. And, he made the claim that no one person will ever excel at all three, which led to his case for building out teams of “analysts” who have complementary strengths.

I’ve had several unrelated experiences with different clients and internal teams of late that have led me to try to capture, in a similar fashion, the three-legged stool of an online analytics program. Just as others have started tacking on additional components to Stéphane’s three skillsets, I’m sure my three-legged stool will quickly become a traditional chair…then some sort of six-legged oddity. But, I’d be thrilled if I could consistently communicate the basics to my non-analyst co-workers and clients:

I hold to a pretty strict distinction between “measurement and reporting” and “analysis,” and I firmly believe there is value in “reporting,” as long as that reporting is appropriately set up and applied.

Just as I believe that reporting should generally occur either as a one-time event (campaign wrap-up, for instance) or at regular intervals, I firmly believe that testing and analysis should not be forced into a recurring schedule. It’s fine (desirable) to be always conducting analysis, but the world of “present the results of your analysis — and your insights and recommendations therein — once/month on the first Wednesday of the month” is utterly asinine. Yet…it’s a mindset with which a depressing majority of companies operate.

Reporting Done Poorly…Which Is an Unfortunately Ubiquitous Habit

I’ve been client side. I’ve been agency side. I’ve done a decent amount of reading on human nature as it relates to organizational change. My sad conclusion:

The business world has conditioned itself to confuse “cumbersome decks of data” with “reporting done well.”

It happens again and again. And again. And…again! It goes like this:

Someone asks for some data in a report
Someone else pulls the data
The data raises some additional questions, so the first person asks for more data.
The analyst pulls more data
The initial requestor finds this data useful, so he/she requests that the same data be pulled on a recurring schedule
The analyst starts pulling and compiling the data on a regular schedule
The requestor starts sharing the report with colleagues. The colleagues see that the report certainly should be useful, but they’re not quite sure that it’s telling them anything they can act on. They assume that it’s because there is not enough data, so they ask the analyst to add in yet more data to the report
The report begins to grow.
The recipients now have a very large report to flip through, and, frankly, they don’t have time month in and month out to go through it. They assume their colleagues are, though, so they keep their mouths shut so as to not advertise that the report isn’t actually helping them make decisions. Occasionally, they leaf through it until they see something that spikes or dips, and they casually comment on it. It shows that they’re reading the report!
No one tells the analyst that the report has grown too cumbersome, because they all assume that the report must be driving action somewhere. After all, it takes two weeks of every month to produce, and no one else is speaking up that it is too much to manage or act on!
The analyst (now a team of analysts) and the recipients gradually move on to other jobs at other companies. At this point, they’re conditioned that part of their job is to produce or receive cumbersome piles of data on a regular basis. Over time, it actually seems odd to not be receiving a large report. So, if someone steps up and asks the naked emperor question: “How are you using this report to actually make decisions and drive the business?”…well…that’s a threatening question indeed!

In the services industry, there is the concept of a “facilitated good.” If you’re selling brainpower and thought, the theory goes, and you’re billing out smart people at a hefty rate, then you damn well better leave behind a thick binder of something to demonstrate that all of that knowledge and consultation was more than mere ephemera!

And, on the client side, if the last 6 consultancies and agencies that you worked with all diligently delivered 40-slide PowerPoint decks or 80-page reports, then, by golly, you’re going to look askance at the consultant who shows up and aims for actionable concision!

Nonetheless, I will continue my quixotic quest to bring sanity to the world. So, onto the three legs of my analytics stool…

First, Plan (Dammit!!!)

Get a room full of experienced analysts together and ask them where any good analytics program or initiative starts, and you’ll get a unanimous response that it starts: 1) at the beginning of the initiative, and 2) with some form of rigorous planning.

The most critical question to answer during analytics planning is: “How are we going to know if we’re successful?” Of course, you can’t answer that question if you haven’t also answered the question: “What are we trying to accomplish?” Those are the two questions that I wrote about in this Getting to Great KPIs post.

Of course, there are other components of analytics planning:

Where will the data come from that we’ll use?
What other metrics — beyond the KPIs — will we need to capture?
What additional data considerations need to be factored into the effort to ensure that we are positioned for effective analysis and optimization down the road?
What (if any) special tagging, tracking, or monitoring do we need to put into place (and who/how will that happen)?
What are the known limitations of the data?
What are our assumptions about the effort?
…and more

In my experience both agency-side and client-side, this step regularly gets skipped like it’s a smooth round, rock in the hand of an adolescent male standing on the shore of a lake on a windless day.

An offshoot of the planning is the actual tagging/tracking/monitoring configuration…but I consider that an extension of the planning, as it may or may not be required, depending on the nature of the initiative.

Next, Measure and Report

Yup. Measurement’s important. That’s how you know if you’re performing at, above, or below your KPIs:

Here’s where I start to get into debates, both inside the analytics industry and outside. I strongly believe that it is perfectly acceptable to deliver reports without accompanying insights and analysis. Ideally, reports are automated. If they’re not automated, they’re produced damn quickly and efficiently.

Dashboards — the most popular form of reports — have a pretty simple purpose: provide an at-a-glance view of what has happened since the last update, and ensure that, at a glance, any anomalies jump out. More often than not, there won’t be anomalies, so there is nothing that needs to be analyzed based on the report! That’s okay!

I was discussing this concept with a co-worker recently, and, in response to my claim that reports should simply get delivered with minimal latency and, at best, a note that says, “Hey, I noticed this apparent anomaly that might be important. I’m going to look into it, but if you (recipient) have any ideas as to what might be going on, I’d love to get your thoughts,” she responded:

I think this makes sense, but wouldn’t we provide some analysis as to the “why” on the monthly reports?

I immediately went to the “dashboard in your car” analogy (I know — it breaks down on a lot of fronts, but it works here) with my response:

You don’t look at your fuel gauge when you get in the car every day and ask, “Why is the needle pointing where it is?” You take a quick look, make sure it’s not pegged on empty, and then go about your day.

That’s measurement. It may spawn analysis, but, often, it does not. And that’s to be expected!

Which Brings Us to Testing and Analysis

Analysis requires (or, at least, is much more likely to yield value in an efficient manner) having conducted some solid planning and having KPI-centric measurement in place. But, the timing of analysis shouldn’t be forced into a fixed schedule.

The bottom part of the figure above gets to the crux of the biscuit when it comes to timing: sometimes, the best way to answer a business question is through analyzing historical data. Sometimes, the best way to answer a question is through go-forward testing. Sometimes, it’s a combination of the two (develop a theory based on the historical data, but then test it by making a change in the future and monitoring the results). Sometimes the analysis can be conducted very quickly. Other times, the analysis requires a large chunk of analyst time and may take days or weeks to complete.

Facilitating the collaboration with the various stakeholders, managing the analysis projects (multiple analyses in flight at once — starting and concluding asynchronously based on each effort’s unique nature), can absolutely fall under the purview of the analyst (again referencing Stéphane’s post, this should be an analyst with a strong “head” for business acumen).

In Conclusion…(I promise!)

There is a fundamental flaw in any approach to using data that attempts to bundle scheduled reporting with analysis. It forces efforts to find “actionable insights” in a context where there may very well be none. And, it perpetuates an assumption that it’s simply a matter of pointing an analyst at data and waiting for him/her to find insights and make recommendations.

I’ve certainly run into business users who flee from any effort to engage directly when it comes to analytics. They hide behind their inboxes lobbing notes like, “You’re the analyst. YOU tell me what my business problem is and make recommendations from your analysis!” I’m sure some of these users had one too many (and one is “too many”) interactions with an analyst who wanted to explain the difference between a page view and a visit, or who wanted to collaboratively sift through a 50-page deck of charts and tables. That’s not good, and that analyst should be flogged (unless he/she is less than two years out of college and can claim to have not known any better). But, using data to effectively inform decisions is a collaborative effort. It needs to start early (planning), it needs to have clear, concise performance measurement (KPI-driven dashboards), and it needs to have flexibility to drive the timing and approach of analyses that deliver meaningful results.

Feed enhanced by Better Feed from Ozh

Facebook Status Updates: Exploring Optimal Time of Day/Day of Week

Tim Wilson — Tue, 14 Feb 2012 11:00:36 +0000

Although Facebook has unofficially admitted that there seems to be little rhyme or reason these days when it comes to the time of day or day of week when a brand posts content on their page, it’s still worth doing a quick analysis to see if this is, indeed, the case for your page.

The challenge, it turns out, is that there are multiple aspects of what sounds like a pretty straightforward assessment:

What metric(s) actually make for a “successful” post?
How do you effectively consider time of day and day of week?
Have you actually posted on a sufficient variety of dates and times to have the data to do a meaningful analysis?

After scraping together some hasty cuts at this, I thought it would be worthwhile to try to knock out something that was easily shareable and reusable. The result is the downloadable spreadsheet at the end of this post.

What It Looks Like

The spreadsheet takes a simple export of post-level data from Facebook Insights (the .csv format) and generates three basic charts.

The first chart simply shows the number of posts in each time slot and each day of week — this answers the question of, “What spots have I not even really tried posting in?”

In this example, there have not been any posts from 9:00 PM to 6:00 AM, only one post between 6:00 AM and 9:00 AM, and only four posts on a Friday. Don’t worry if you don’t like the time windows — we’ll get to that in a bit.

The next two charts are crude heatmaps of a couple of metrics, but they both use the same grid as above, and they use a pretty simple green-to-red spectrum to show which spots performed best/worst relative to the other slots:

(I know, I know: red/green is not a colorblind-friendly palette selection. I’ll keep working on the visualization technique!)

The first of these charts looks at the average total reach of the updates that were posted in each time slot — the number of unique users of Facebook who were exposed to the post:

In the example above, Wednesdays looked to perform pretty well reach-wise, as did Saturday afternoon. If you have Facebook paid media running, these results may get skewed. It’s easy enough to update this chart to use Organic Reach rather than Total Reach, or, you can simply factor an awareness of what was running and when into your assessment of the results. Also, keep in mind that Facebook continues to fiddle with the EdgeRank/GraphRank algorithm, so there are aspects of a post’s reach that are beyond your control.

The next chart shows the average engagement rate of the posts, defined as the number of users who engaged with the post in some way (clicked on a link, posted a comment, liked the post, viewed a photo, etc.) divided by the total reach of the post. This is a pretty solid measure of the content quality — did the post drive the users who saw it to take some action to engage with the content? Now, arguably, the propensity for a user to engage is less impacted by the time of day and day of week, but, who knows?

In this example, Sundays and Thursdays were the days when posts appeared to get more engagement (although be leery of that Sunday, 6:00 PM to 9:00 PM, block — there was only a single post in the data set).

Timeframe Flexibility

Picking a set of timeframes is the most subjective aspect of this whole analysis, and it may be worth iterating through a few times to get to timeframes that are likely to be meaningful for the page given the target consumer. So, I’ve set up the worksheet to make it easy to customize these timeframes. For, instance, below is the same data set used above, but with only four windows of time:

The change look less than 60 seconds to implement (it’s all about VLOOKUPS, pivot tables, and conditional formatting!).

How to Use This for Your Own Page

If you want to try this out for your page(s), simply download the Excel file (this was created using Excel 2007, so it should work fine in both 2007 and 2010) and follow the instructions embedded in the worksheet. You will need to export post-level Facebook Insights data for your page, which may require several iterations (we’ve found that Facebook Insights is prone to hanging up if you try to export more than a couple of months of data at once):

Then, just follow the instructions in the spreadsheet and drop me a note if you run into any issues!

Some Notes on the Shortcomings

This approach isn’t perfect, and, if you have ideas for improving it, please leave a comment and I’ll be happy to iterate on the tool. Specifically:

This approach measures all updates against the other posts for the same page — there is no external benchmarking. This doesn’t bother me, as I’m a proponent of focusing on driving continuous improvement in your performance by starting where you are. Certainly, this analysis should be complemented by performance measurement that tracks the actual values of these metrics over time.
The overall visualization could be better. It’s not ideal that you need to jump back and forth between three different visualizations to draw conclusions about what days/times are really “good” or “bad”…including factoring in the sample size. I’ve toyed with making more of a weighted score and then doing the same color grid, but, then, you’d be looking at a true abstraction of the performance, so I didn’t go that route. Suggestions?
A red–>yellow–>green scale just isn’t good when it comes to supporting: 1) black-and-white printouts, and 2) certain forms of color blindness. A more iconographic approach might make more sense.

Please do weigh in with how you would change this. I’m happy to rev it based on input!

Feed enhanced by Better Feed from Ozh

Working Around Sampled Search Data in Google Analytics

Tim Wilson — Thu, 09 Feb 2012 11:00:34 +0000

I got into a discussion of sampling in Google Analytics with SEO expert and Web PieRat Jill Kocher earlier this year, which led to some profile/filter noodling that seemed worth sharing. Specifically, Jill and I were discussing how, in the world of search engine optimization — where the long tail can be a handy thing to analyze — sampling in Google Analytics can be a real nuisance.

That got me thinking that a partial solution would be to have a Google Analytics profile that only includes organic search traffic. This isn’t a profile that you would use for cross-session analytics, but it’s one that would allow simplified segmentation, reduced cases of sampling, and, perhaps, a more complete data set.

As it turns out, it was pretty simple to set up, and it seems to do the trick.

Step 1: Make a New Profile

Create a new profile under the same web property that you’re using for your site and name it Organic Search Traffic Only:

There’s nothing magic about this. The key is that this is a profile that uses the same web property ID as the profile where you’re running into sampling issues with your SEO analysis. We’re just going to take that same feed of data coming in as visitors visit your site and carve out the subset of that data that is traffic from organic search referrals.

Step 2: Apply an Organic Search Filter

The next (and final) step is to create a filter and apply it to the profile such that only organic search traffic is included.

In the new profile you just created, select the Filters tab and then click New Filter:

From there:

Give the filter a name like “Organic Search Referrals”
Select Custom Filter as the Filter Type
Set the filter as an Include filter
Set the Filter Field to Campaign Medium
Set the Filter Pattern to “organic”
Save the filter

The screen below shows the filter settings:

Step 3: Sit Back and Let the Data Roll In

The profile is only going to include data from the point you set it up going forward. But, it will accurately reflect (to the extent that any web analytics package can accurately reflect this) new versus returning visitors for all time (well, since you initially implemented Google Analytics), because it’s getting that data from the cookie that already exists on users’ machines.

Initially, I saw some odd data on the unique visitors front, which I can semi-intuitively understand…but not quite explain.

Suffice it to say that, once you have the profile up and running for a week or so, you can select the Non-paid Search Traffic segment in your main profile and compare it to the All Visits segment in your new profile, and the numbers will be virtually identical. But, you can now do SEO analysis with a base set of data that only includes search traffic.

Is that handy?Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2012. | Permalink | No comment | Add to del.icio.us
Post tags: Google Analytics, SEO

Feed enhanced by Better Feed from Ozh

Facebook Engagement (aka, Facebook Rhetoric Facebook Reality)

Tim Wilson — Thu, 26 Jan 2012 13:44:54 +0000

Oh, Facebook.

Facebook, Facebook, Facebook.

Ours is a tumultuous relationship of unrequited frustration, is it not? I am an analyst, therefore (apparently), you scorn me. And, by “scorn,” I mean “ignore.”

You never responded to my letter last year. You don’t return my calls. (Well, that’s not entirely true: you put salespeople on my calls whose general response to any question is, “Buy Facebook media.” I get it. That’s their job, but they act like they’ve parachuted straight out of Mad Men and are pushing traditional mass-blast advertising. Ironic, no?)

Facebook, I’ve dug into the data. Your own documentation states:

Posting regularly with engaging content gets more people to talk about your business with their friends. As a result, you end up reaching more people overall.

Yet, the data you provide us tells a very different story. We debunked this particular claim — that getting people to talk about your content leads to greater reach – a month ago.

So, What Can We Debunk This Month?

Lately, I’ve been digging into a more basic mystery: you claim that, the more someone engages with a page’s content, the more likely that person is to get presented with more of that page’s content in the future. That seems pretty reasonable. Of course, you hedge at the same time:

No matter how engaging your Page posts are, not all of your fans will see them in their News Feed. In order to make sure that more of your fans see your posts, you should create a Page Post Ad

Can we quantify that “not all of your fans…” statement? AllFacebook.com did just that when they published a pretty alarming article last week based on Edgerank Checker data. Their study showed that, on average, across 4,000 pages, only 17% of total fans were being reached per individual post by the brand. “Zoiks!” were the cries that echoed through the halls of community managers the world over!

To be fair, not everyone is on Facebook all the time, and, while that number matches data we’re seeing overall, it also leaves out the fact that these don’t appear to be the same 17% day in and day out. When it comes to looking at the 28-Day Total Reach from Page Posts measure you provide, we see numbers that are more in the neighborhood of half of a page’s Lifetime Total Likes (when there is no Facebook media running — it’s much higher than that if that exposure is being purchased from Facebook).

Is 17% really all brands can expect, or is it all they can expect if they’re doing a lousy job posting content?

Are Brands Simply Not Publishing Engaging Content?

We’ve been working pretty hard to learn what kind of content our clients’ fans like, as well as how often and when to post. That put us in a good position to dig into the data to see how we were doing, especially in light of the drop we felt we were seeing in the Reach of posts across a range of our clients’ pages.

We looked at data from a half-dozen pages. These pages were all devoted to major consumer brands, had Lifetime Total Likes ranging from the low 100,000s to multiple millions, and cut across a range of different verticals. Is “6 pages” on the order of the “4,000 pages” from the Allfacebook.com study? Well, no, but we were working with over 600 status updates, and it quickly became apparent that we’d dug in enough to draw some pretty sound conclusions..

For the chart below, we removed the handful of posts that were clearly data anomalies (skewing both wildly high and wildly low) and then, for each post, took the Lifetime Engaged Users for the post (the number of unique people who clicked anywhere in the post within 28 days of it being posted, regardless of whether the click generated a story or not) and divided it by the Total Reach for the post.

It’s not the cleanest of graphs, but it seems pretty clear that, if anything, these pages are, overall, making some headway when it comes to producing more engaging content.

The idea here is that the only people a post has a chance of engaging are people that it reaches. So, we have Total Reach as the denominator. This is similar to the Post Virality calculation that you, Facebook, generate for me…but we’re looking at a lower level of engagement than “generated a story” — just looking to see if fans are interacting with the post in any way. Because, in theory, if they are, then you will be more likely to present them with subsequent posts from the same page.

So, Engagement Isn’t Dropping. Presumably, Reach Isn’t, Either?

In the post engagement chart, there’s nothing all that shocking. What does get alarming, though, is when we look at the average Organic Reach (unique users who saw the post directly as a result of the page posting it — not because a friend talked about it, and not because the brand ran paid media to extend the reach of the post). We divided that organic reach by the Lifetime Total Likes for the page to see what % of the total fans were reached by the post organically.

Again, outliers (high and low) were removed (this included locally-targeted posts, where the reach, obviously, was very low relative to the total likes for the page). Each point on the chart represents all of the status updates on that day from our sample:

Wow. I’m not a data scientist, so the above doesn’t have any true statistical rigor applied to it. Rather, it is an exercise in what a stats professor once preached to me: “Start off by plotting the data! That’s going to tell you a lot!”

It’s pretty conclusive, I think, that a Facebook algorithm change (and related UI changes — but the algorithms drove what content appears anywhere for a user, regardless of the UI) in late September gave brands a temporary ability to reach a higher proportion of their fans. That, undoubtedly, led to any number of community managers thinking they had been listening and learning and publishing more engaging content.

Then, (alas!) November arrived. And, suddenly, Reach plummeted.

WTF?

It’s not that I’m opposed to paying you for reach, Facebook. I’m totally okay with paid media being part of my social media mix. But, if I have to pay you each time I want to reach someone, the numbers start to get hard to justify. If someone likes my page, and then they engage with my content, why don’t they keep getting my content for some period of time?

Here’s what I think happened (and, frankly, I’d respect you a bit more in the morning if you just came out and admitted it):

You put some sharp people in a room and told them to come up with a good EdgeRank/GraphRank algorithm
While you have “a lot of data,” that algorithm still was largely driven by that team’s instincts around what weighting should be given to different factors
There was a fair amount of teeth-gnashing, and the team even tried to do some testing of the algorithm before rolling it out. But, that’s a taller order than it sounds.
The algorithm got rolled out.
You had no idea what was going to happen. What looked good on paper looked, well, different in practice.
For various reasons — none of which have been openly stated — the algorithm has been quietly tweaked a couple of times. In one case, it was related to the Timeline rollout, but, by this time, the algorithm had become the red-headed stepchild of Palo Alto. No one really wants to own it, because no one can really figure out what will make it “work.” After all…the algorithm-heads are all just down the street in Mountain View! (zing!)

How close am I with the above speculation? I don’t have inside knowledge (as noted earlier, you don’t call, you don’t write), but I’m not sure what other explanation makes sense.

Know that you’re killing us — the analysts who are trying to drive learning and optimization! At least set up some sort of open dialogue. We don’t need to see the full formula. But, we need to have useful information about how to do things better. And we need to know when you’re tinkering with the algorithm and what the likely result of that tinkering will be. Otherwise, we can’t trust the data, which means we can’t learn from it. Without data we can use, it’s hard to justify investment and action.

Feed enhanced by Better Feed from Ozh

New Blog Design –> Responsive Design and Web Analytics Musings

Tim Wilson — Tue, 24 Jan 2012 17:36:13 +0000

If you’re reading this post on the site itself (as opposed to via RSS or email), and if you’ve been to the site much in the past, then you’ll notice the design of the site has been completely overhauled. This was one of my goals for my weeklong holiday break…and it’s a goal I entirely missed! Luckily, though, I wound up with a kid-free/spouse-free weekend a week-and-a-half ago, so I got to tackle the project.

So, Why a New Design?

I updated the design for two reasons:

The old design was starting to wear on me. There were a number of little alignment/layout/wrapping issues that I had never quite managed to fix, even as I tinkered with the blog functionality (for instance, my social icons never quite lined up well). I also figured out last fall that the nested table structure pretty much precluded me from getting the mix I wanted of fixed and liquid elements. In short, a redesign just seemed in order.
Responsive web design is here. This was more of the direct-tie-to-my-day-job reason for the overhaul. Various sharp people at Resource Interactive have started pushing responsive web design as something that should be actively considered for our clients. As I dug into the topic, I realized that: 1) this blog is a good candidate for a responsive design, and 2) there are some analytics implications to a responsive design, and I needed somewhere to experiment with them.

So, this site is now using a fully responsive WordPress theme.

What Is Responsive Design, Exactly?

As I understand it, responsive design is an “Aha!” that grew out of the increasing need for web sites to function across a wide range of screen sizes and experiences and platforms: laptop monitors, desktop monitors, tablets (iOS and Android), and smartphones (also iOS and Android). The idea is that, rather than having a “desktop site” and a “mobile-optimized site,” you can have “a site” that works effectively on a wide range of devices.

There are two keys to this:

The site needs to be viewable in different devices – 3 columns that display on a desktop monitor may need to become a single set of stacked content on a smartphone. Or, a list of links in the sidebar on the desktop may need to become a dropdown box at the top of the page on an iPhone.
The site needs to support the most likely use cases in different devices – this is a stickier wicket, because it forces some strategic thought (and possibly research and testing) to think through what a visitor to your site who is using an iPhone (for instance) is likely looking to do and how that differs from a visitor to your site who is using a desktop.

Both of these are questions that have always been asked when it comes to developing a “mobile-optimized version of the site,” but they’re a bit more nuanced given that responsive design isn’t a “separate site.”

Wow, Tim, I’m Impressed with Your Coding Skills!

Don’t be impressed with my coding skills.

I did a little research and then shelled out $35 to buy the Rising theme. That doesn’t mean there wasn’t a fair amount of tinkering (and more tinkering yet to be done — I certainly have not fallen prey to a need to have the perfect site designed before pushing it live!), but the end result is an improved site. And, more importantly, having a site that actually works well across devices (Try it! Just resize your browser window and watch the sidebar at the right. Or, fire up the site on your smartphone and compare it to your desktop.)

Now, of the “two keys” above, I really focused on the first one. This is a blog, after all. Regardless of what device you’re on, presumably, you’re here to consume blog post content.

I’m still working with the palette (too little contrast between the hyperlink color and the plain text color), the font selection (I’m not in love with it), and the header logo (pulling what strings I can to get a professional to contribute on that front), but I’m reasonably content with the change. Let me know if you have any tips for improving the design (I’m not proud!).

Where Does Analytics Come into All of This?

While I have access to tons of different web analytics accounts across a range of platforms through our various clients, I don’t actually have a great sandbox for trying things out (you would think our company’s site would be a good testbed, but the reality is that there are so many competing agendas for competing resources there that it’s seldom worth the effort). Luckily, this site has built up enough content and enough of a presence to get a few hundred visits a day, which is enough to actually do some tinkering and get some real data as a result.

Here’s my list of what I’ll be toying with over the coming weeks:

Responsive design analytics — we’ve had “screen resolution” and “device” reporting for years, but responsive design introduces a whole new twist, because it’s truly experience-centric. I’ve done a little digging online and haven’t found much in the way of thinking on this. While I don’t think it’s possible to directly pull CSS media query data into the web analytics platform, it should be possible to use Javascript to detect which responsive layout is being used for any given visitor and then pass that information to the web analytics platform (as a custom variable or a non-interaction event in Google Analytics). And, it should be possible to record when an onresize event occurs. In both cases, using this data to segment traffic to determine if a particular layout is performing poorly or well, as well as how visitors move through the site in these different experiences, seems like a promising thought.
Facebook Insights for Websites — I’ve had this running for a while, but, as part of another experiment, I switched over from using my Facebook user ID in the meta data to authenticate my ownership of the site to using a Facebook app ID. That’s a better way to go when it comes to “real” sites, and I’m now actually doing some tinkering on some client sites to fully validate what happens, so look for some thoughts on that front in the future.
Detecting the Facebook login status of visitors to the site – this is some experimentation that is actively in work. It’s the implementation of some code that Dennis Paagman came up with to use Facebook Connect and Google Analytics non-interaction events to detect (and then — my thinking — segment) visitors based on whether they’re logged into Facebook or not at the time of their visit to the site. This seems like it has intriguing possibilities when it comes to determining what types of social interactions should be offered and how prominently. I’ve hit a minor snag on that front and am hoping Dennis will be able to help get to the bottom of it (see the comments on his blog post). But, if I get it figured out, I’ll share in a post down the road.
Site performance – anecdotally, it seems like this site is now loading more slowly than it did with the old design. The Google Analytics Site Speed report seems to indicate that is the case, but I don’t feel like I have enough data to be conclusive there just yet. I have signed up for a site24x7.com account, which is a platform we use with some of our clients for a couple of reasons: 1) to see what it reports relative to Google Analytics (it’s a fundamentally different data capture method, so I’m not going to be surprised if the results are wildly divergent), and 2) to get more reliable data if I start playing with changes to reduce the site load time. In hindsight, I wish I’d signed up a month or so ago so I had good pre- and post- data. If I had a nickel for every time I wanted to have had that, I’d be a wealthy man!

In a nutshell (a gargantuan, artificial nutshell, I’ll grant you), I’ve got a backlog of topics, some of which will require some additional experimentation. This blog post, I realize, is almost more of a “to do” list for me than it is a “how to” list for you! Oh, well. They can’t all be winners!

© Tim for Gilligan on Data by Tim Wilson, 2012. | Permalink | 2 comments | Add to del.icio.us
Post tags: responsive design

Feed enhanced by Better Feed from Ozh

10 Presentation Tips: Tip No. 10 — Respect the Audience

Tim Wilson — Mon, 16 Jan 2012 16:20:28 +0000

This is the last post in a 10-post series on tips for effective presentations. For an explanation as to why I’m adding this series to a data-oriented blog, see the intro to the first post in the series. To view other tips in the series, click here.

Tip No. 10: Respect the Audience

This last tip is more of a perspective than a tip.

It’s last because it’s the tip that drives the reason for paying attention to all of the other tips.

It’s last because it’s a tip that is all too often flagrantly ignored.

It’s last because it can be a little scary.

The experience that prompted me to write this series was my participation in the inaugural #ACCELERATE conference in San Francisco last fall. As it turned out, I was the last presenter of the day — one of the 5-minute Super #ACCELERATE presentations.

Here’s one way I could have viewed my presentation:

It’s only 5 minutes, so I should try to do something pretty solid, but, if it falls flat, it’s only a small fraction of the overall conference.

Here’s how I actually viewed the presentation:

It’s 5 minutes, but it’s 5 minutes in front of of 300 people, so that’s actually 1500 minutes, or 25 hours. If I swag that the fully loaded cost of the members of the audience is, on average, $50/hour, then I need to deliver a $1,250 presentation!

Okay, so it’s a little tough to really make this math work is a 5-minute presentation, but think about a 20-minute presentation ($5,000) or a 30-minute presentation ($7,500) or an hour-long presentation ($15,000). Change the hourly cost however you see fit, but do the mental exercise to consider the opportunity cost of the presentation — the total amount that is being invested by the audience members who could be doing something else rather than listening to you present. That is the amount of value you should fully commit to delivering with your presentation.

Each member of the audience is paying to watch your presentation, regardless of whether they had to pay a monetary fee to sit through it.

They’re paying with a finite and valuable commodity: their time.

Recognize that. Respect that. Do everything you can to make it a worthwhile investment on their part.

Photo by Eric T. Peterson

Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2012. | Permalink | No comment | Add to del.icio.us
Post tags: 10 Presentation Tips

Feed enhanced by Better Feed from Ozh

10 Presentation Tips: Tip No. 9 — Make it Personal, Descriptive, and Tangible

Tim Wilson — Fri, 13 Jan 2012 16:20:49 +0000

This is the ninth post in a 10-post series on tips for effective presentations. For an explanation as to why I’m adding this series to a data-oriented blog, see the intro to the first post in the series. To view other tips in the series, click here.

Tip No. 9: Make it Personal, Descriptive, and Tangible

Imagine someone you know giving a presentation about how to present effectively and saying the following:

“Studies have shown that the most effective presentations incorporate personal anecdotes and are descriptive and tangible. This increases the likelihood of the audience being engaged and, thus, actually paying attention to the content being presented. You should really try to come up with things that have happened to you or that you have done and relate those to the audience so that they are more interested in you, which means they are more likely to pay attention, which means they will be more likely to retain what you have presented. You should also avoid abstract examples — abstractions are harder for the brain to process, and it’s easy for the brain’s subconscious to simply give up and zone out.”

Now, imagine someone covering the same material, but doing it as follows:

“I once had to give a presentation to 300 co-workers at my company’s annual meeting. I had five minutes to talk about measurement and analytics, which I knew was a topic that wasn’t inherently of interest to the group. This was one of a series of five back-to-back presentations in a modified Pechu Kucha format — 15 slides, with the slides auto-advancing every 20 seconds. I came up with the idea to use my 5-month, 2,100-mile backpacking trip form Georgia to Maine on the Appalachian Trail as an underlying theme to stitch together the 2 points I was trying to drive home in my 5-minute talk. It turned out to be an incredibly effective presentation, which, almost 2 years later, people still remember and reference. You see, by incorporating a personal anecdote that I could relate to the topic I was covering, I actually made the content more engaging and, thus, more memorable.”

Which of the above presentations-about-presenting do you think would be more likely to “stick”?

In their book Made to Stick: Why Some Ideas Survive and Others Die, Chip and Dan Heath work through an acronym — S.U.C.C.E.S. — as to what it takes to effectively convey ideas. While the book goes well beyond presentations, their mnemonic nails this tip pretty well:

Simple
Unexpected
Concrete
Credible
Emotional
Stories

Really, this tip is about concrete, credible, emotional, and stories. It’s totally, totally, totally fine to start developing your presentation using abstractions. That’s probably what you’re going to have written down when you come up with your answer to the question: “What do I want the audience to take away from my presentation?” (Tip No. 7). The trick is to identify every generality and abstraction in the flow of your presentation and try to come up with a way to make each one more tangible, either by adding in specific examples or by introducing an analogy (personal or otherwise). Not only will this make your presentation more memorable, it’s fun (and it can really help when it comes to tracking down meaningful images — Tip No. 3!).

Three examples (yeah, I damn well better include tangible examples, right?) of this tip in practice from the three guys at Web Analytics Demystified:

Eric Peterson presents on how he works with Best Buy to re-tool their analytics program: he co-presents with Best Buy (tangible example), and he uses a “house” analogy to illustrate, with pictures of ways houses can evolve (additions) as well as be rebuilt (when needing a new foundation or entirely new floor plan)
John Lovett talks about his history as a licensed skipper (personal anecdote) and then uses naval navigation as an analogy for developing social media metrics programs
Adam Greco uses a chess analogy to describe some of the key aspects of implementing a successful web analytics program…and relates that his younger son beat him at the game (both a personal anecdote…and one that he then ties back to web analytics)

As with all of the other tips in this series, the key to this one is that the goal isn’t simply “entertainment,” but, rather, relating examples and anecdotes that reinforce your key message.

Picture by Steve Snodgrass (modified by me to put the circle-slashon it, and, to be
clear, it’s making a point — I actually think the original piece is pretty cool)

Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2012. | Permalink | No comment | Add to del.icio.us
Post tags: 10 Presentation Tips

Feed enhanced by Better Feed from Ozh

10 Presentation Tips: Tip No. 8 — We Have Five Senses. Use TWO!

Tim Wilson — Thu, 12 Jan 2012 16:20:21 +0000

This is the eighth post in a 10-post series on tips for effective presentations. For an explanation as to why I’m adding this series to a data-oriented blog, see the intro to the first post in the series. To view other tips in the series, click here.

Tip No. 8: We Have Five Senses. Use TWO!

One of the most interesting books I’ve read over the past few years is Brain Rules: 12 Principles for Surviving and Thriving at Work, Home, and School by John Medina. In easy-to-read prose, with lots of interesting examples, Medina lays out 12 “rules” of how the brain works — acknowledging up front that there is an infinite number of things we don’t yet understand about the brain, but that there actually are a number of things that we absolutely do know. The book focuses on the latter (for a slightly deeper read on my take on the book, jump over to this blog post from a couple of years ago).

Many of these the presentation tips in this series can be tied directly back to Medina’s brain rules, but this post is focused on three specific ones:

Rule #4: We don’t pay attention to boring things
Rule #9: Stimulate more of the senses
Rule #10: Vision trumps all other senses

Now, obviously, when it comes to presentations, you typically only have two senses to work with: sight and sound.

From Medina’s book:

We absorb information about an event through our senses, translate it into electrical signals (some for sight, others from sound, etc.), disperse those signals to separate parts of the brain, then reconstruct what happened, eventually perceiving the event as a whole.

What neuroscientists have figured out is that, by routing the same information through multiple senses, you have a better chance of making the information “stick.”

In a typical presentation environment, the senses of smell, taste, and touch are largely off the table, so you’re working with two senses. The good news is that sight is far and away the most dominant sense, but, “We learn and remember best through pictures, not written words.” (see Tip No. 3)

Here’s where presenters, even ones who intuitively know they need to be leveraging both sight and sound, often go awry. They approach their presentation with this mindset:

Sight = “what’s on my slides”
Hearing = “what I say”

This is a formula for under-utilizing these senses. In addition to the above, there are a number of other ways to play off these senses:

“Hearing” is not just what you say, but how you say it – changes in volume and tempo are a second layer of “hearing”; avoid the monotone (and know that, even when you feel like you are dramatically changing your pitch and tone…it’s probably not coming across as nearly that dramatic. This is one of the reasons it makes sense to video some of your rehearsals).
“Sight” is not just the content on your slides, it’s the sight of you – your facial expressions and movement. Can you think of a presentation you’ve seen where the presenter literally seemed to bounce around the stage and or gesture dramatically with his/her hands? Chances are, you can. Now, can you remember what the presenter was talking about? Again, you probably can. This actually dips into Medina’s Rule #4 (we don’t pay attention to boring things), but my point here is that your audience is looking at you as much as they are looking at your slides. So, you need to be cognizant of that and use “the sight of you” to reinforce your content and make it more memorable.

Two examples where this tip has been creatively applied to great effect:

At eMetrics in Washington, D.C., in 2010, Ensighten launched a campaign by starting a “tag revolution” — a “tagolution” — that included the distribution of colonial wigs to all of the conference attendees. When Josh Manion got on stage to talk about Ensighten for 5 minutes, he delivered the presentation with one such wig on his own head. I don’t remember any other vendor that presented in that session. And, because the wig wasn’t simply a “be goofy” gag — because it actually tied directly to the point Josh was trying to convey — his presentation “stuck.” In essence, Ensighten actually leveraged a third sense — touch — by distributing wigs to the conference attendees. I got to plop a wig on my head (in the privacy of my hotel room!), so the point really, really, really “stuck.”
As another example, I teach an internal class at Resource Interactive that is focused on how to go about establishing clear objectives and KPIs up front in any engagement. The material was co-developed with Matt Coen, and one of the points he introduced was the classic play on “Ready, Aim, Fire,” and how digital marketers have this ugly tendency to instead go with “Ready (‘I need to do social media!’),” “Fire (‘I’m throwing up a Facebook page!’)”, “Aim (‘Did the Facebook page deliver results?’).” As we worked through the content, I found an image of someone firing a gun, and then introduced a simple build of three words on top of the image: “Ready” then “Fire” then “Aim.” Simple enough. I had imagery, it was a valid analogy to the point we were discussing, and the slide only had 3 big words on it. Then, I had the idea to introduce a sound effect — right as the word “Fire” appeared, a gunshot sound effect went off. Without fail, everyone in the class jumps, then sits up straight, then chuckles. It works.

I’m not saying that you should always include props in your presentations, nor that you should drop gratuitous sound effects throughout your deck. But, if you consciously think, “How can I maximize the impact of the senses of sight and sound,” you have a better shot at making your presentation — and its content – more memorable.

Photo by gabriel amadeus

Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2012. | Permalink | 3 comments | Add to del.icio.us
Post tags: 10 Presentation Tips, Brain Rules, Ensighten, John Medina, Josh Manion

Feed enhanced by Better Feed from Ozh

10 Presentation Tips: Tip No. 7 — Be Memorable By Identifying the Memory

Tim Wilson — Wed, 11 Jan 2012 16:20:04 +0000

This is the seventh post in a 10-post series on tips for effective presentations. For an explanation as to why I’m adding this series to a data-oriented blog, see the intro to the first post in the series. To view other tips in the series, click here.

Tip No. 7: Be Memorable By Identifying the Memory

This tip is really about simplicity and clarity. Accept at the outset that only a fraction of what you present is going to be retained by the audience, so it’s much better to have a small handful of key takeaways and then spend your time reinforcing those points.

The earlier in the development of your presentation that you clearly articulate for yourself what it is you want your audience to take away, the better off the presentation will be.

This is such an easy point to skip that, well, most presenters do!

The process that is required in order for information to get from a presenter’s mouth all the way to an audience member’s long-term memory requires multiple steps:

The material first gets captured/absorbed by iconic memory, which has a sub-second retention time
If the person is “paying attention,” the information will then be transferred into short-term memory, which lasts only a few seconds, but is where it can be consciously considered
If the material that is in short-term memory is sufficiently repeated and reinforced by the audience member’s own cognitive processing, it will actually make it into long-term memory so that it can be recalled the next day, next week, or next month

Bringing focus to the presentation and not being overly ambitious about how much information you want to convey enables you to build a presentation that repeats and reinforces the key points sufficiently that they are more likely to make it to the long-term memory banks of your audience.

Over the past few years, almost every formal presentation I have developed has started with me jotting down in my notebook the question, “What do I want the audience to take away from the presentation?” I then take multiple stabs at answering the question clearly and succinctly in writing (often revisiting my answer over several days in brief spurts). It can be surprisingly difficult, but it’s an exercise well worth the effort!

The answer to this question becomes a recurring litmus test for everything that goes into the presentation:

Does content that is being considered speak directly to the desired takeaways?
If not, is the content critical supporting information for the takeaways?

I can point to cases where a picture, diagram, or point that was one of the first things I put into a slide for a presentation — and was an idea or concept that actually sparked the whole idea for the presentation — ultimately got dropped when I considered it against these questions. This can be really tough, as it can means dropping content that is clever or insightful…but that is ancillary and nonessential. Dropping this content is the right thing to do — otherwise, you risk having your audience completely miss (or fail to retain) the fundamental purpose of the presentation.

For an hour-long presentation, aiming for 2-3 key takeaways is about right. That may sound like an unduly small number, but it’s reality. Think about the last presentation you sat through and jot down the main points. How long is your list?

The more focused your presentation is, and the more clear you are on the key points that you want your audience to retain, the better your presentation will be.Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2012. | Permalink | No comment | Add to del.icio.us
Post tags: 10 Presentation Tips

Feed enhanced by Better Feed from Ozh

10 Presentation Tips: Tip No. 6 — Bring the Energy!

Tim Wilson — Tue, 10 Jan 2012 16:20:20 +0000

This is the sixth post in a 10-post series on tips for effective presentations. For an explanation as to why I’m adding this series to a data-oriented blog, see the intro to the first post in the series. To view other tips in the series, click here.

Tip No. 6: Bring the Energy of a Dinner or Bar Conversation

We’ve all seen it happen time and time again: someone who we personally know to be energetic, outspoken, and lively in 1-on-1 and small group conversations…speaks in the driest of monotones when delivering formally prepared presentations.

Few things kill a presentation’s impact more quickly than a nuclear blast of impassivity from the presenter

It’s understandable why this happens — it’s a chain reaction:

Anxiety about the importance of getting the presentation “right” ups our caution level
The natural way that humans react to caution is to be tentative
In a public speaking situation, tentativeness manifests itself as a low voice with limited modulation, as well as minimal physical movement

Our brains say, “Tread carefully! You’re walking a tightrope and don’t want to do anything risky! One misstep and you will catastrophically plummet into the Presentation Disaster Chasm!”

Unfortunately, this is one of those cases where our natural instincts as to how to be “safe” actually lead to disaster. Think about when you were a kid and first learning to ride a bike. Because the bike was wobbly, your instinct was to go slow…which made the bike more wobbly, because the gyroscopic action of the wheels needed faster rotation to kick in and provide stability. Presenting is similar — if you force yourself to be “the animated you,” you will quickly reap the benefits:

The energy you exhibit on stage will add energy to your audience
The audience will make eye contact and “lean forward” to see what you are so energized about
That energy from the audience will feed back to you, and you will be off and rolling!

I know this sounds a little hokey, but, if you take Tip No. 2 to heart and analyze presenters who are ineffective, consider them through the lens of this tip. How often is the person who is presenting noticeably less energized than you know that person to be?

The fact is, you are going to come across to your audience as being less energetic than you personally feel you are being. That’s because you are likely operating with a slight shot of adrenalin, so you feel more energy as you speak than you are necessarily showing.

There are several non-exclusive ways to apply this tip:

Be aware of it — most people don’t realize how passive and monotonal they are being when they are on stage
Rehearse, rehearse, rehearse! (see Tip No. 5) — as you gain confidence with the flow of your presentation and your content, it becomes infinitely easier to focus on your expressiveness
While you’re rehearsing, look for opportunities to use a hand gesture, a facial expression change, a change in the volume or tone of your voice, or other ways to alter your physical and audio presence to add emphasis
Video tape yourself rehearsing (I’ve never actually done that…but, as digital video becomes more and more accessible, I fully expect to start!)

This doesn’t mean go crazy and jump around all over the stage, nor does it mean to step wildly outside of your own natural character. But, a little bit of energy goes a long way, and, chances are, you’re not going to overdo it. Bring the energy!

Photo by Eustaquio Santimano

Similar Posts:

© Tim for Gilligan on Data by Tim Wilson, 2012. | Permalink | No comment | Add to del.icio.us
Post tags: 10 Presentation Tips

Feed enhanced by Better Feed from Ozh