When comparing search keyword referrals across our sites it is common, and obvious, to see the site ranked numero uno in the SERP to get the highest amount of referrals, followed by numero dos, etc, etc.

While checking stats today I noticed a keyword on Site A was generating a bunch of referrals so I clicked through and saw that it was ranked #2 with an indented listing while another site of ours was ranked #1.  I was curious to see which one generated more referrals.

The results are below:

It held true for most of the keywords I reviewed.  If the listings were normal the #1 ranking drove far more referrals than the #2 or #3 rankings.  But if the #2 ranking had an indented listing it rose above the #1 ranking.

Anyone else see similar results with their sites?

Here are a few posts about getting indented listings:

• http://andybeard.eu/2008/05/google-double-indented-listing.html
• http://www.problogger.net/archives/2006/07/09/gaining-the-second-indented-google-listing/

The folks at Blogstorm.co.uk reported that in April Google began testing their new AJAX SERPs, and then followed up the post at the end of April with a step-by-step guide to modifying your Google Analytics setup to track rankings with each referring keyword.  Sooooo sweet.

At SIM we use Omniture (not giving them a link here because they sneakily added a link to their site within our code…see this article) for the majority of our sites but decided to test out G Analytics recently on one of our smaller properties.  Just last week we followed Blogstorm’s instructions and got the keyword ranking tracking set up as well.   Extremely cool data.  Below is a snapshot for a relatively small time period and even though it is a small sample set I think the data is telling.

Here are a few nuggets:

• The #1 ranking accounts for 65% of keyword referrals and 66% of visits
• Top 3 rankings account for 81% of all keyword referrals and 82% of all visits
• Top 5 rankings account for 90% of keyword referrals and 90% of visits
• Top 10 rankings account for 98% of keyword referrals and 98% of visits
• Rankings 6-10 only accounted for 8% of keyword referrals and 8% of visits

The data isn’t perfect, but it is certainly interesting and opens the doors for more research.  We don’t know the total number of rankings for each position or as a whole and therefore there could be a higher percentage of #1 rankings which would explain the high number of referrals for that position.  The data above becomes valuable if we obtained those rankings or if we were to assume, yes I know what assuming does, that there were an even number of rankings at each SERP position.  This would confirm the power of obtaining that #1 spot in the search results.  It would also shed light on how useless it is to be on the second page and how little value there is in being listed anywhere from #6 to #10.

With this data we can:

1. Calculate the opportunity cost of being ranked in positions 6-10 or in positions 11-20.
2. Calculate how long it will take to see a ROI if we were to invest in pushing our listing higher for a specific keyword.

A similarly interesting study would be to track referrals from each SERP position for a given keyword.  This would give us insight into CTR per position, how brands play a role in CTR, and if the keyword type plays any role.  The challenge here is that it requires you to either a) have enough sites that you have complete coverage of the first page, or b) track the keyword long enough that it bounces around to each listing.

Oh wait, I do have this data….post coming soon.

When working with members of the editorial and production staff we need to incorporate content optimization and keyword research training into the mix.  We teach these employees that one of the first steps when dealing with a new piece of content is to turn to a keyword research tool to determine the best possible keywords to use in each of the page elements.  We then teach them how to optimize each element on the page that they are responsible for.  When they have questions regarding the topic of the page or the best keyword to use we ask them to turn to the keyword research tools.  Is it a 1967 Mustang or a ’67 Mustang?  Let the data be your guide.   Let our users decide how you tag the content.  While we have subscriptions to both KeywordDiscovery.com and WordTracker.com we suggest that our staff utilize the former due its larger database, user friendly interface and tools, and API availability. On any given day there may be 100 people in our company turning to KeywordDiscovery.com (KD) to tell them what keywords are best suited for the content they are producing.

As one would imagine, this heavy reliance on third party tools can cause problems. On numerous occasions we have come across data that makes you shake your head and say “WTF?”.  There will always be the occasional gibberish keyword like “pne qrnyre” showing up in your results but after using KD for awhile you get used to skimming those instances.  In some cases the data issues can’t be seen until you dig a bit deeper or compare against other datasets.  Stay with me…

Search Engine Market Share

As I mentioned before one of the positive aspects to KD is that it provides users with a number of different tools and ways to view the data.  One of them being the “Market Share” view which is available by clicking on the pie chart icon next to each listing.  In theory this is pretty cool data.  For example, I can look at the number of searches for the keyword “movies” and then find out the distribution of those queries across the major search engines.  The pie chart looks good right?

Movies Keyword SE Market Share

This is great…when it works.

As I mentioned before, most of our internal SEO projects start with keyword research. One of our SEO team members delivered a project to me that focused on the keyword “auto dealers” as opposed to “car dealers” or “car dealerships”.  To his credit he did everything right.  He researched all of the possible keyword variations and the data suggested that “auto dealers” was a far better keyword than any other alternative.  Currently KD shows “auto dealers” with 35,227 searches and “car dealers” with 5,668 searches.  No brainer, right?

Maybe because I am cynical that people really say “auto” or “automobile” instead of “car”, or for some other fortuitous reason, I clicked on the market share icon for this keyword. As you can see in the pie chart below KD suggests that almost 72% of the searches for this keyword are conducted on Ask.com. My apologies to the folks over at Ask.com, but this pie chart tells me the data is incorrect. Due to Ask.com’s non-existent traffic I have to take the original number of 35,227 and cut it back by 72% to account for the inflated numbers which leaves us with 10,568 searches for this keyword. Still “auto dealers” has double the amount of “car dealers” searches but we know from our own referral data that this isn’t correct as the latter produces far more referrals than the former.

Auto Dealers Keyword SE Market Share

Is the problem widespread or specific to this keyword? I pulled the market share graph for a few more keywords in various markets and got mixed results. For some keywords like “life insurance” and “car insurance” the Ask.com numbers are inflated but for others like “movies”, “Christmas”, and “auto insurance” the percentages are normal.

Christmas Keyword SE Market Share

Life Insurance Keyword SE Market Share

Auto Insurance Keyword SE Market Share

Car Insurance Keyword SE Market Share

Trending Data

Are the market share issues a concern or just bugs in the system? Does it point to bad data? Without KD giving me the keys to look under the hood I can’t know for sure, but another one of their cool views gives us some more insight into this. If you click the bar graph icon next to each keyword result in KD it displays a nice twelve month trend graph. For certain keywords like “Christmas” it can show valuable trending data that will help explain traffic patterns and help you plan out your marketing, PPC, and SEO campaigns.

Christmas Keyword Yearly Trend

But for others the results are odd. For example, when I pull the view for the keyword “car dealer” it shows me the graph below. I’m sure you notice the extreme spike between June and July. This to me seems like bad data that should’ve been omitted from their results.

Car Dealer Keyword Yearly Trend

Maybe there was a spike of activity due to something in the news? Or maybe seasonality factored in? If we trust Google Insight and Google Trends to provide accurate data than the answer would be no as you can see in the graph below and also here and here.

Car Dealer Keyword Google Trends

And when we run some more queries we see more of the same unnatural, irregular spikes in the trends. See below.

Health Insurance Keyword Yearly Trend

Auto Insurance Keyword Yearly Trend

Life Insurance Keyword Yearly Trend

KD claims that the Global Premium DB, the source of these graphs and aforementioned data, is impervious to bots. Is it?

Premium and Regional Keyword Databases

If you are looking for clean and pure search results without skews caused by automated web agents (rank checkers), Keyword Discovery offers a number of Global Premium Keyword Databases. The Global Premium database contains over 4.4 billion keyword searches, based purely on user panel data. The Global Premium database is also used to create the many regional and language specific keyword databases. (read more here)

Keywords Misrepresented

Another thing I’ve noticed is that keywords can often be under represented in their database. To KD’s credit they have a disclaimer in their FAQ section to cover this.

Why are there differences in the Keyword Discovery search results compared to actual organic traffic results?

Keyword Discovery’s search counts differ from actual search results as we collect only a percentage of all searches done on the Internet. The search counts that we provide are best used as a comparison to other search terms in our database. You should treat Keyword Discovery’s search counts as a rank, not actual total search volume.

The potential problem lies with how far off the numbers are and how it is interpreted by those using the data. Like I mentioned way back at the top of this article the people using KD on a daily basis at our company ranges from Senior SEO Specialist to the Assisted Content Product with minimal SEO and keyword research knowledge. The novice has been taught to seek the best possible keywords for the content they are producing. So if a keyword shows up in KD with 5 or 10 searches it may be dismissed.

Here is a small snapshot of KD data compared against actual organic search referrals. As you can see the numbers are quite different. Take the bottom result for example. You can see that KD has 97 searches for this keyword phrase in its DB over a 12 month period and they predict roughly 5 searches per day. With our #1 ranking in Google it produced 4,000 referrals in the month of March and roughly 100 referrals on one Monday in April.

Keyword Discovery & Referral Data Comparison

Equally impressive is the difference with the third from last keyword. KD shows it with only 146 searches over a 12 month time span and predicts 8 searches per day. Our data in Omniture shows us that this keyword phrase produced almost 10,000 referrals in March, the second highest out of this small group, and roughly 120 referrals for a single Monday in April.

So What’s the Point?

I’m not telling you to stop using Keyword Discovery, Word Tracker, or any other keyword research tool. I’m merely pointing out that there are flaws in the data and you need to dig deep in order to uncover some of them. In addition, when using these tools for keyword research it is important that you factor in your own common sense, competitor research, and knowledge in the space. If we had strictly used KD data to determine our keyword targeting we may have lost out on the thousands of users represented in the table above. Sure glad we didn’t.