What I want to talk about isn’t as complicated as it sounds. Basically, in collecting daily site specific metrics for the purposes of SEO, you sometimes get some bad data from your third-party sources.

These bad data points can skew your graphs and make it almost impossible to visually derive any useful information from them.

For example:

The graph above (click to expand) shows the number of pages for a 6 month date range. In early January the data spikes dramatically upwards. This is clearly an anomaly, as the day after, the data is back in its normal range. The trouble here is that this anomalous point skews the entire graph, making it impossible to derive any real insights from it. The above graph is rendered useless by that one piece of anomalous data.

Here is the same graph WITHOUT that one bad piece of data (click to expand)

It’s amazing how one bad data point can skew your entire graph, isn’t it? Now we have an image that is useful!

Here’s some (hackish & sketchy) code to identify these statistical anomalies and replace them:

Google Webmaster Tools has released some new data for all you data-junkies to get your fix on. Specifically, they’re now showing impression numbers and click-through percentages for organic search queries. Rad right?  It looks that way on the surface…but the good folks on Dave Naylor’s team have done a bit of research into the numbers and what they’re calling the ‘Google Gap’.

Read on http://www.davidnaylor.co.uk/google-webmaster-tools-2.html, and remember to always question the numbers.

Big thanks Dave’s team for writing this post…if you’re looking for a top notch UK SEO, look no further.

Alot of people recommend MaxMind or some other geoIP database, but really, thats a pain in the ass…there is a much easier way to do it.

On your landing page, just include the following in header.

<script type=”text/javascript” src=”http://www.google.com/jsapi”></script>

This script will calculate your visitors location for you….and all you have to do is pull the ClientLocation variable!
For example, if I wanted to display a popup with the users city:

alert(google.loader.ClientLocation['address']['city'])

Seriously, its THAT SIMPLE.

Have fun!

I’ve recently undertaken a personal project that I hope will make my day-to-day activities on a the web a bit more efficient. I’ve always dreamed of having the perfect home page for myself; one that had all my most crucial data at a glance. Unfortunately, all the services I’ve tried have fallen short (*cough* netvibes *cough*). So, I’ve decided that enough is enough; I am going to build my own dashboard.

The dashboard will take quite a few cues from the interface of The STAT Codex, in so far that it will be window/panel based and entirely modular, but it will be customized to my needs.

At the very least, I want the dashboard to pull the following data:

• email module (gmail and others, just reading, sending can open up  a new window in the email client.)
• twitter module (multiple accounts…send and receive tweets)
• a twitter favorites module that allows my to follow the favorited tweets of various twitter accounts
• instapaper module for items to be read
• remember the milk  module for to-do items
• major site statistics module (only for my most active web properties)
• new incoming backlinks module
• paypal module for showing paypal balance and recent transactions
• google calendar module
• log monitor module (would tail the server logs for most active web properties)

What else would you add to the list? What data would you love to have at a click on your home page?

Today is a very big day for me, because today is the day when I get to start telling everyone about our latest creation: STAT – Search Tracking & Analytics Tools.

STAT is a comprehensive suite of search analytics tools geared towards the needs of search marketers. Make no mistake: STAT is not another SEO tool, it’s an analytics tool, and it is truly one-of-a-kind.

STAT is ideal for researching competition, tracking organic rankings in the major three search engines (Google, Yahoo, Bing), and monitoring incoming backlinks and indexed pages. Further, with the ability to monitor changes to SERPs in real-time, you can identify algorithm changes when and where they happen. STAT archives past SERPs and audits them in the background to determine key search metrics that affect your rankings. Best of all, STAT does all of this in a drag-and-drop window based interface; it is a web application that looks and behaves like any of your run-of-the-mill desktop applications. Because of the intuitive nature of the platform, using STAT is easy and you can hit the ground running.

After a year of being available only to larger search agencies, I’m proud to announce that STAT is now open to the general public on an invite-only basis. If you want to signup, visit STAT – Search Tracking & Analytics Tools and I’ll send you an invite ASAP.

I could talk ad infinitum about how amazing this platform is, but really, it’s best if you experience it for yourselves. To take it for a test drive, visit STAT and click on the link for a demo.

So, today I learned that Amazon S3 limits the amount of buckets you are allowed to have at any one time. This was a HUGE pain in the ass for me, as I just rewrote my S3 caching libraries to separate my caching into daily buckets.

The idea I had was that I would keep 30 days worth of buckets before automatically shuffling all those buckets into long term archival at the end of each month.

You see, Amazon S3 doesn’t allow for folders within buckets (not technically, I’ll get to that later), and I think it’s bloody ridiculous for me to store the ~15 millions xml cache files I need daily access to in just one massive bucket. So, I thought my plan to create daily buckets was pretty damn good. Apparently Amazon disagrees, as hey have set a limit on the number of buckets I can have associated with my account (FYI, I have ~90 buckets right now and that’s where the limit is).

So, I’ve been looking into my options, and I have decided that I’m going to have to go back to my original set up of storing all 15 million (and growing) xml files in one bucket. As some of you may know, S3 GUIs like S3Fox are able to show sub-folders within buckets, so I figure I will go this route. My first step was to take a crack at the Ruby S3 library from Amazon (not the AWS gem, which is crap for multi threaded environments) to see how I can create folders within buckets. Turns out, you can’t. At least, not out of the box. You see, AmazonS3 doesn’t actually SUPPORT sub-folders within buckets.

Stupid, right? I know.

So, how does S3Fox do it? It turns out they create virtualized folders by creating a special object that acts as a folder, then you access your stored objects by appending that folder object name to the actual file key. For a directory named “/foo”, you would create an object with the key “foo_$folder$”. Then, to get a directory listing of all files stored under the foo path, you just query S3 for objects with keys that start with “/foo”, and you ignore any objects that end with “_$folder$”.

I’m about to waste my day setting this up, and I’m none to happy about it. It seems like a hackish and shitty work around for an obvious service flaw. I’m sure there is some technically-reasonable answer for why Amazon has set an arbitrary limit for buckets and also why they don’t allow you to create folders, but I don’t know what it is. Anyone have any answers?

Hat-Tip the Dead Programmer Society for the code to create virtualized folders in S3.

I wanted to share some interesting tid-bits with you all from a recent Ipsos Reid presentation (June 18th) on Social Network Marketing.

This presentation was made to BC Hydro, the major Power authority in British Columbia (that’s in Canada, dummy).

The average Canadian:

• 5.4 hrs/week spent on Social network sites, about 1/3 of total Online time.
• LinkedIn captures 70% of the share of Social Networking in the workplace.
• The stuff teens do online is very different from adults, and VERY limited: socializing, music, and gaming.
  • 88% of Adults use the internet to visit a new or informational website, versus only 44% of teens.
  • 68% of Adults have clicked a website advertisement versus 28% of teens.
  • 70% teens are weekly social network users, versus 36% of adults

Highlights:

• Advertising budgets are not aligned to where people are spending their time – online expenditure lags significantly behind traditional mediums
• 58% of Canadians receive 51+ unsolicited commercial emails a week (17% don’t even know)
  • 64% of people don’t open ANY unsolicited emails
  • of the people that do open unsolicited emails, 53% do so out of curiosity, 28% because they thought it was legitimate
• Permission-based (opt-in) emails are still one of the best ways to market to your websites audience. Be diligent about collecting them
• 77% of Canadians are registered to receive some kind of opt-in email (2008 average number of sites registered with: 15.3)
  • Top 3 reasons for opting-in: Personal Interest (42% ), Entertainment (38%), New and Information (32%)
  • It’s worth noting that E-Commerce and Retail is the 7th most common reason (27%)
• Activities resulting from opting-in to an email list:
  • 60% entered an advertiser’s contest
  • 52% visited the advertiser’s website
  • 17% purchase or received products at a later date
• 68% percent of Canadians are willing to provide their email address depending on reasoning

I’ve uploaded the full presentation for you all, which can be viewed here:

Frank Lee, Senior VP of Client Services, and his team found that their paid search clients are seeing dramatically improved results on Bing compared to Live Search.  Even though impressions in the first 3 weeks of Bing have dropped 22% compared to the last 3 weeks of Live Search, Microsoft now seems to be serving far more relevant ads on each keyword.

The result for The Search Agency’s advertisers:

Click Through Rate up 15%
Conversions up 6%
Conversion rate up 18%
Cost per acquisition down 3%

Until now, all the industry reports on Bing have covered the increase in search volume, share of search, or total clicks.  Their data is the first analysis of Bing’s impact on paid search advertising performance.  With this growth in search volume, along with these improved metrics, advertisers might want to consider shifting more of their paid search spend to Bing.

You can read Frank’s full post their their blog The Search Agents.

I suggest you subscribe to their blog, as these guys really know what they’re talking about.

While many agencies seem to base marketing strategy on the sole-opinion of some conference-hopping SEO ‘guru’, The Search Agency always surprises me with their dedication to gathering real, actionable & measurable market data. They are one of those rare Search Marketing agencies that build strategies based on real, hard-earned market research, not a magic eight-ball.

There are many of different reasons why this error can occur, and in my case, it was a reason that no one else has written about (that I could find), so I figured I’d post up the cause of my problem (and the solution).

This application was on a shared development server with a few other applications. They all used the same mySQL server. One of my other applications had a corruption in a few of its tables, and whenever this application was called, it would crash the mySQL server, triggering a “Mysql::Error: Lost connection” for all my other applications connected to the mySQL server at that same time.

I tracked down this issue by check the mysql logs, and then I simply reloaded last nights backup of the corrupt database, and everything was fine! It just goes to show you how important it is to always perform regular backups of your databases.

I hope this helps someone!

The code was poorly presented, and poorly formatted. It wasn’t at ALL my normal caliber of work, so I went ahead and rewrote the concept into a tighter little wrapper.

I am using the Gattica gem to interact with the Google Analytics API.

This little script provides a wrapper around the gem that easily allows you to extract your sites organic keywords and sort them based on the number of unique entrances to your site via each keyword.