The Analytics and Site Intelligence Blog @ MoreVisibility

“Big Data”: What is it? And, more importantly, what can I do with it?

Jocelyn Ward — Wed, 29 May 2013 12:30:29 +0000

These are just a couple of the questions that have been plaguing industries and enterprises worldwide since the “Big Data” phenomenon surfaced. By now, most of us have heard this buzzword/phrase that has been penetrating the minds of IT and analytics professionals alike. However, many organizations are still unsure how to effectively analyze and gain new insights from it. Luckily, there are expert specialists in this field who are eager to join and guide them through their journey.

What is “Big Data?”

I’ll spare you the formal definition and put it simply: “Big Data” is everything, and it’s everywhere. “Big Data” is defined by (at least) three ‘Vs’: Volume, Velocity and Variety. And you might even hear about a fourth ‘V’ depending on which “Big Data” solution provider you’re talking to.

Volume: Explosive data sets that can translate into terabytes; petabytes; exabytes; zettabytes;

zettabytes = as much information as there are grains of sands on all the world’s beaches

Velocity: Speed in data generation, processing and consumption; real-time where appropriate

Variety: Any type of data coming from a multitude of sources – structured, unstructured, semi-structured; numeric, non-numeric; traditional, non-traditional

Fourth ‘Vs’ you might come across:

Veracity (IBM) – Accurate, truthful and trustworthy data

Variability (SAS) – Data flows that may be unpredictable, inconsistent and anomalous

Now that we have a better grasp of what exactly “Big Data” is, I’d like to explore some of the complexities and challenges companies face because of it, as well, as the opportunities it presents.

Challenges & Complexities

The size, requirements, boundaries and resources of an organization, as well as the industry it’s in, can dictate the adoption of “Big Data” in addition to which obstacles will prevent them from extracting high-value impact and gaining new business insights that were previously unattainable.

However, there are a few common challenges despite the nature of the business:

Technology:

An abundance and variety of data sources and the information collected

Inherent complexity in processing, management and aggregation

I intentionally left out a fundamental part of the “Big Data” definition when I talked about the three or four ‘Vs’ of this concept, but this is a perfect place to sneak it in.

IDC’s definition of “Big Data” embraces the hardware, services and software that integrate, organize, manage, analyze and present the data that is characterized by the ‘Vs’ discussed at the beginning of this post.

This is why new technologies and architectures, advanced tools and platforms are needed and are continuing to be developed. These appliances will allow enterprises to leverage “Big Data” and (you guessed it) analytics.

Skills: Shortage of finding “expert” talent

Technical: Data scientists with an unparalleled level of skill to understand the interactions of a new class of technologies

Analytics: Data mining; statistics; business analytics; problem solving; creativity

Funds: Investing in new technologies & skill sets without a full awareness of what it takes

Although there are some hindrances to enterprises fully embracing this new era of “Big Data” and analytics, there are evolving approaches to conquer them. For example, the Google Analytics Premium and BigQuery integration that will be taking place toward the end of this year was just announced at the Google I/O a couple weeks ago. If you’re a GA Premium user, I’ll venture to guess that this made you smile – even if you’re not 100% sure what it’s going to mean for your business.

Check back next week when I’ll discuss what value, advantages, opportunities and possible use cases can arise from utilizing more advanced technologies, solutions, and analytics strategies such as the “Big Data” movement. Stay tuned!

Google Analytics is Lying to You!

Theo Bennett — Tue, 21 May 2013 13:05:49 +0000

Let’s cut right to the chase: Google Analytics (GA) is lying to you. Huh? How is GA lying to you, you ask?

Let me illustrate, via screenshot, my beef with my old friend Google Analytics:

Can you believe that? I know I can’t. What’s that you say? You don’t see the lie? Let’s try that again:

Yes that’s it! Goal Values are not “optional”!

We all know that Google Analytics is the best thing since sliced bread; what other product could give you so much business intelligence at its price point (free)? Unfortunately, to collect data with Google Analytics one must only copy and paste the Google Analytics tracking code (GATC) onto their site and that’s where most businesses stop. To get the most from Google Analytics, code customizations; including events and custom dimensions are probably required. But most importantly—your company must sit down and answer some tough questions. Some of these include:

Why do we have a website? (Really, you have to answer this.)
What do we want people to do when they get to our site? (Desired outcome.)
How do we measure these outcomes? (Goals in Google Analytics.)
How much are these desired outcomes worth to our business? (Goal Value.)

If you need help with the questions above, please visit Avinash Kaushik’s brilliant post on the Digital Marketing and Measurement Model.

Okay, so you’ve answered the questions above, your marketing efforts have clearly defined objectives and you now know your reason for being; it’s time to set up your goals WITH goal values.

Goal Values again? Yes! (That’s the theme of this post, remember?) Ninety percent of the new GA accounts we review as a Google Analytics Certified Partner either do not contain goals or goals are present but goal values are missing. You must commit to ALWAYS adding a value for your goals. With values added, two magical things happen.

1. Per visit goal value is calculated:

What’s Per Visit Goal Value tell me? Well, at a glance it can tell you:

a. How effective each of your campaigns, sources, affiliates, ads, ad groups, and more are in real dollars!

b. How much you should be bidding for each keyword. (Really. Look at per visit value in the goal set view of your keyword reports.)

2. Page Value (Content Reports) is calculated:

Page Value tells you how each of your pages perform in the context of helping people convert. Now this data requires more analysis than Per Visit Goal Value because checkout and home pages will always have a high value, but it can help you understand which middle of the funnel pages perform best and help your visitors to take your desired action.

In summary, Goal values are not optional. You should always endeavor to understand how much goals add to your bottom line and add the values into GA. That said; if for some reason you find it too difficult to determine the value of key interactions– any number is better than nothing. So start with $100 or $1 or some other arbitrary value. From there you can work toward some relative goal values (e.g. contact us submission = $100, newsletter subscription = $10) and then hopefully as you gather more data, real goal values for each outcome.

Finally, there is one exception to this rule. Goals for Ecommerce transactions are desirable for many reasons, but if you are using GA Ecommerce tracking, then your corresponding goal should never have a value as this will distort the calculated values we mentioned previously.

Google Analytics Premium Gets Interactive with BigQuery Integration

Jocelyn Ward — Mon, 20 May 2013 12:51:02 +0000

Google Analytics Premium customers smiled a little brighter yesterday after one specific announcement was made at the Google I/O. An additional feature, Google BigQuery, is planned to become available to Google Analytics Premium users later this year.

Talk about a game changer. This goes far beyond just Web Analytics and opens the doors to a tremendous scope of opportunities. But, the basics are just amazing enough to make me anxiously call out a couple of them.

Access and query your Google Analytics data alongside and in combination with any of the other data sets your company derives for its business intelligence needs; all in a matter of seconds. I’m talking about trillions of rows of data right at your fingertips. Really, you can export the data, print it out and touch it if you’d like.

As if Web Analytics data isn’t multifaceted enough, there is an abundance of other data sets companies collect that are just as intricate and essential to running their businesses. With more and more organizations aware of the “big data” phenomenon, key stakeholders are starting to ask more complex questions and are leaning on their analysts for the insights to make better-informed business decisions. And they should.

This exercise can be daunting; exhausting; frustrating and sometimes near impossible. And, chances are most time is spent trying to query, join, and report on the correct data which means less time for true analysis to provide actionable recommendations to decision-makers.

The Google Analytics Premium and BigQuery integration is planned to ease these pains. It will provide granular data access for you to:

Join and cross-tabulate multiple data sources

Understand complex queries

Create detailed custom analyses

Identify what’s really going to drive and improve the bottom line

With so many possibilities, you can get lost in exploring them all. But, before you meander off on your own, make sure you get back to the boss on that request from…

Mastering the Shortcut in Google Analytics

Theo Bennett — Wed, 27 Mar 2013 12:39:04 +0000

As an analyst, one of my favorite mantras is: “Keep Digging”. Actually it’s probably “Always be Segmenting” or “If you are not Segmenting, you are wrong!” but for the purpose of this post we’ll stick with “Keep Digging”. Whatever it is that drives you to keep drilling through reports and looking at every facet of your data is unimportant. What’s important is that you do it.

So how do Shortcuts help you as a user of Google Analytics (Ga)? Let’s walk through a scenario to illustrate when you may want to use GA Shortcuts. Let’s imagine that you are using GA to optimize the performance of a new campaign. Inevitably you’ll find yourself three levels deep in a report with an alternate view (something other than what GA presents for you), and a filter and segment applied — and there it is – the data nugget you’ve been looking for, the key to optimizing said campaigns! You share the data with the rest of your team and your boss is amazed by your ability to squeeze so much from Google Analytics. Now they ask you to show them how you got the data. You pour back into the reports and … well … you get lost and can’t recreate what you’ve reported. Don’t be embarrassed, it happens to everyone — until now.

Enter Google Analytics Shortcuts. Now as you drill deep into reports you can simply save a shortcut and once applied, it will take you right back to the view, segment and report as if you’d never left! It’s like Hansel and Gretel’s breadcrumbs, but faster.

So let’s use one of my favorite reports – Landing Pages – as an example.

This is how the landing page report is presented in GA (with page names redacted):

As you can see by the red box above, we are in the view that GA presented to us, which is known as Data (Table) view. By itself, a nice report with some useful data; however, with a little customization, this report is more powerful.

Let’s change views, add a filter and a segment for paid traffic:

As you can see in the red boxes above, we’ve filtered for pages containing “index”, changed our view to comparison and selected bounce rate as the metric we’d like to compare. Now we can see the index pages’ bounce rate as they compare to the site average for paid traffic (in this view you are not able to see that the paid traffic segment is applied).

Let’s just say that this is reporting nirvana for you and its how you always want to use the Landing Pages report. In the past, to recreate this you’d have to make a note of what you did and then follow these steps:

Go to Landing Pages Report
Change Views to comparison
Select Bounce Rate for your comparison metric
Add a filter for “index”
Add an advanced segment for Paid Traffic

Wow that’s a lot of steps! Google Analytics Shortcuts makes this so much easier. To save your Shortcut:

1. Click on Shortcuts

2. Name your Shortcut (something relevant)

And that’s it! Now when you want to apply those same settings, go to Shortcuts in the left-hand report navigation:

Click on the name of the Shortcut that you want to apply and GA will revert to the report, view, filter column selection and segment that you had applied when you created the Shortcut! (Date ranges do not change.)

I’m sure you can think of many more uses for Shortcuts. For now Shortcuts can not be shared with other users, but they are still one of my favorite new features in Google Analytics.

Google Images Updates Affect Analytics Referral Traffic

Kayla Kuruc — Tue, 19 Feb 2013 11:57:21 +0000

In January Google updated its image search algorithm to improve the user experience. The “new” Google Images now shows the selected picture in a black display pane.

By comparison the older version pulled up the website in the background. This seems like a straightforward enough change, certainly in line with Google’s many adjustments geared to a cleaner visual experience.

However, for websites that have many images in the Google index, there is a less than exciting result to this new and improved user experience. The newest update to the image search decreases the referral traffic from Google.com. Because the new display feature does not pull the website up behind the selected image, the Google Analytics code does not have a chance to fire, removing the page view credit and decreasing referral traffic.

Websites with substantial image libraries will see the decrease in Google Images traffic starting at the end of January when the switch was made. Traffic from Google Images can be found under Traffic Sources > Sources > Referrals> Google.com. To identify the Google Images traffic specifically, click Google.com from the referral sources listed and look for /imgres.

In order for the Google Analytics code to fire using the new Images features, the searcher must select to visit the page by clicking on the image a second time, or selecting one of the two buttons to the right of the image. Although the impact of the change may appear extreme for some sites initially, the new functionality will result in a truer site visit, rather than one indirectly generated by Google’s Image search.

Blocking traffic in Google Analytics

Theo Bennett — Thu, 17 Jan 2013 14:37:07 +0000

As I mentioned in my last posts, AdWords Bots in Google Analytics (GA), and Bot Traffic in Google Analytics we’ve seen more and more bot traffic registering as visitors in GA. (Bots are automated programs that normally don’t fire JavaScript or images, so this traffic shows up in web server logs, but not in Google Analytics.)

In this post we’re going to discuss how to block these bots from your GA reports so that you can have clean data from which to make smart business decisions and make more money!

First let’s understand that “well behaved” bots usually have one of two fingerprints:

Technology profile – this includes browser user agent, version, java profiles, etc.
ISP – Yes that’s right, Microsoft, Yahoo! and Inktomi bots come from Microsoft, Yahoo! and Inktomi ISP’s.

What kind of tracks does your bot traffic leave? Well it’s best to start with an Advanced Segment that begins to whittle away this bot traffic from the real traffic. Apply this segment to your Google Analytics data and you’ll be viewing only direct traffic that bounces.

https://www.google.com/analytics/web/permalink?uid=BH3Nk-LCSpmv7UQyPXgGDA

Do you see any patterns in the browser version reports? (Audience>>Technology>>Browser & OS)

In the screen shot above we’ve selected “Mozilla Compatible Agent” and version 5.0 with no Java Support looks like a likely bot suspect.

What about identifying bots by ISP? (Audience>>Technology>>Network)

So from the data above we can see some interesting sources of bouncing traffic. Next I created another segment to view each of these ISP’s (1. microsoft corp , 6. yahoo! inc. and 16. Inktomi Corporation).

https://www.google.com/analytics/web/permalink?uid=nt7BieC5RR24dRcai6Kb_g

As you can see from the data above, all of the traffic from these ISP’s bounced.

Your next action is to decide if you want to go a step further and actually filter out this traffic from your analysis profiles.

If you decide to block traffic based on browser profile, then you’ll need to construct a series of filters to do this.

The first two combine browser data together with Java Support (yes or no) to allow you to then filter only the offending browser profile (Third filter listed in the image above) that is utilized by the bots.

If you decide that the ISP route is the way to go, then you’ll have a much easier path:

Where the pattern is equal to yahoo|microsoft corp$|inktomi

Either way, you should remember to:

Always have an unfiltered profile that collects all data

When possible, test your new profile filter as a segment

Always apply a filter first to a test profile and when happy with the resulting data, to your analysis profile(s).

Consider creating a profile to capture the traffic you’re excluding so that you can monitor it more easily and ensure that you’re not excluding any “real” traffic.

In summary, it really does not matter why the bot is on your site, what’s important is that they are triggering nuisance pageviews that can skew your numbers and conversion rates. Are they affecting your GA data? Apply the segments above and find out for yourself!

Excellent Analytics for Google Analytics

Maeve Greenaway — Fri, 11 Jan 2013 13:11:09 +0000

Google Analytics API has multiple reporting tools, one of them being Excellent Analytics. Marketers did you know about the Excel plugin that pulls your data from Google Analytics into Excel sheets?

To start this integration you can use this free tool called Excellent Analytics. Excellent Analytics is easy to install by clicking the download button at the top of the page.

Use Excellent Analytics to automate reporting and spend more time exploring in Google Analytics.

Measurements within Excellent Analytics include:

Time span: Use to set up daily, weekly and monthly reports.

Dimensions: Are displayed in rows such as landing pages and time dimensions.

Metrics: Visits, visit bounce rate, goal completions and revenue. Metrics are shown as the values in these reports.

Filters: Exact match, regular expressions, phrase match and broad match.

Additional advanced segmentation can be applied as offered in Google Analytics.

Once you have customized measurements for your report, choose the profile(s) the report will be run for. If there is more than one profile, Excellent Analytics will display a report for each profile side by side on the same spreadsheet. The Execute button will trigger these reports to start collecting data and your automated reporting will begin. The data will be collected and exported into Excel. Excellent Analytics is convenient, because the range with headers can be used for pivot tables and other marketing charts.

Bot traffic in Google Analytics

Theo Bennett — Wed, 26 Dec 2012 14:11:06 +0000

As I mentioned in my last post, AdWords Bots in Google Analytics (GA), we’ve seen more and more bot traffic registering as visitors in GA. (Bots are automated programs that normally don’t fire JavaScript or images, so this traffic shows up in web server logs, but not in Google Analytics.)

In this post we’re going to discuss bots from Yahoo! and Microsoft, why it’s important and how to identify the traffic and see if it’s affecting your site.

Why is this important? Well as you’ll see, all of this bot traffic comes into your site as Direct traffic, has exactly one pageview and then does nothing, and that is the problem. We have to remember that a visit as just described equals a bounce – which is a bad thing. So as you look at your reports over time, you may wonder why your goal conversion rates or Ecommerce conversion rates from Direct traffic have plummeted while your bounce rates have increased. Part of the answer could very well be bots. And if you don’t account for this traffic in your quest for the analytics intelligence that will turn your site from a business cost to a profit center, you may never get there!

So how do you know if this traffic is affecting your site? Well by looking from 30,000 feet, you may never know — you have to dig deep. So if you haven’t already been digging for answers about your Direct traffic performance, let me walk you through how to identify these bots.

First of all we know that the focus area is Direct traffic that bounces; so the first step is to create an advanced segment to “filter” all of our reports for these visits in Google Analytics Reports.

If you’d like to “play along” as you read this post here is a link to the segment:

https://www.google.com/analytics/web/permalink?uid=BH3Nk-LCSpmv7UQyPXgGDA

This is a view of the service providers in the Network sub-section of the Technology Report that have been the source of our Direct, bouncing traffic.

Why look here? Well there isn’t much to glean from other reports. Content reports are varied. Traffic Sources and Conversions we know, so the best place to try to find some answers is in the Audience section of Google Analytics. Any report here is a good starting place and in this case we can see from the screen shot above that we’ve gotten a lot of traffic directly from Microsoft and yahoo! inc.

So let’s take a closer look at this bouncing, Microsoft and Yahoo! traffic by applying the segments below.

Microsoft:
https://www.google.com/analytics/web/permalink?uid=c8BzLRzjTnmR9JFa48cbsw

Yahoo!
https://www.google.com/analytics/web/permalink?uid=xXjmJTD8RuakRpcLeR9cQA

So the data that points to bots here is pretty straight forward:

100% Bounce Rate

Nearly 100% New Visits

No Java Support

So let’s look closer at the Internet Explorer Traffic:

We can see that most visits are from IE7 and again with no Java Support. (Java Support by itself isn’t necessarily a “bot indicator”, there are other supporting traits that we don’t have the space to include or address in this post.)

While Yahoo! ignores IE, both companies are leveraging Mozilla Agents.

While Microsoft eschews Firefox 3.5:

So what does this all mean? First, it’s highly likely that Google, Microsoft and Yahoo! are using automation to explore websites and on the highest level, that it’s no longer safe to say that bots are not tracked in Google Analytics. As Analysts, Marketers or business owners, we all need to make sure we’re accounting for their presence as we explore analytics data.

In my next post I’ll share some strategies to filter out this traffic and more segments to help you remove the unwanted effects of this traffic.

Bot traffic from AdWords in Google Analytics Data

Theo Bennett — Mon, 10 Dec 2012 18:58:42 +0000

In order for Google Analytics to track visitors; both javascript and images have to be enabled in the visitor’s browser. Most bots, or programs that are written to digest the coding of web pages and collect information, don’t fit that tracking criteria ; therefore, this traffic show ups in web server logs but not in Google Analytics.
Recently there has been an increase in the number of bots that are visiting via real browsers and are able to execute GA code and thereby pollute your Google Analytics data reports. For me, these bots fall into four categories:

Website Monitoring Services – These services continuously check your site to monitor uptime and other things like page load time. (Addressed in this post by a fellow GACP, Blast Media.)
Legitimate Bots other than Google: — This is unexplained bot traffic, but we’ve seen a lot of it recently from Yahoo! Microsoft and Inktomi.
Rogue Bots – Lets face it, any 12 year old can probably write a bot to send “visitors” to your site and wreak havoc on your GA data. (Numbers 2 & 3 here are to be addressed in a future two part post.)
Google AdWords – This is the biggest surprise and as I’ll demonstrate in this post, Google is clearly sending either multiple visitors to AdWords customer pages or they are leveraging a bot.

My interest in this topic began after a colleague inquired about seeing triple the volume of test data expected after creating staged campaigns in AdWords. This data was visible in the Advertising section of Analytics:

Another colleague experienced something very similar when creating new staged campaigns with a small about of URL testing:

After isolating the traffic via an advanced segment using the campaign name of these yet to be launched campaigns, we were able to view the unique characteristics of these visitors.

Most were from the same geographic area, the united states, but suspiciously, city locations were equal to (not set):

Most used the same browser, resolution, and flash version:

As you can see above, 100% of this non-testing traffic bounced.

And interestingly, all of these visits shared the same service provider: google inc..

While we’ve always seen some visitors from google inc.; the spikes in the last few months are different and concerning as to the number of visits and how they can affect data analysis if not accounted for. We’re still evaluating why only some clients see this spike of activity and there are some commonalities we noticed; however, it’s too early to say exactly why this is happening.

As we learn more, we intend to update this post and would like to know about your own experience. Have you seen AdWords bots in your own reports?

If you want to explore your own data, feel free to use the advanced segment below to identify your own traffic from Google. Disclaimer: This segment could also identify the traffic from real Googlers (people that work for Google) that share the same identified service provider.

Google Inc. segment:

https://www.google.com/analytics/web/permalink?uid=PoOkSXfSQuWLt6St9YH5gQ

And here is one to exclude traffic from google inc.

from your reports:

https://www.google.com/analytics/web/permalink?uid=6fLzxhGOTs6-g0e3Y8T7Qw

Create Shortcuts to See Your Wanted Data, Sooner

Anne Garcia — Wed, 17 Oct 2012 12:11:35 +0000

Google Analytics recently released Shortcuts to help you access your saved data reports more efficiently. Want to see a particular report with specific secondary dimensions for a set of campaigns? Instead of navigating to the Campaign report, adding said secondary dimensions and then layering the report with advanced filters, you can create the report once and save it as a Shortcut, so that it’s easily accessible every time you log into the Home tab of Google Analytics.

Once you’ve created your desired report and have customized it to your liking, select the Shortcut button in your utility bar. See screenshot below for Shortcut link.

A new window will appear prompting you to save your recently created Shortcut.

As soon as you save the Shortcut, you’ll be able to access it via the “Shortcuts” section in the Home tab. You can easily change the date range of your saved Shortcut to be able to look at data for different time periods.

Your saved Shortcuts can be emailed and exported, or even added to your Dashboard. Shortcuts are only applied at the profile level and must be reproduced to be viewed in other profiles you have access to.