You might compare it to an “No Trespassing – Armed Response” sign on a fence bordering private property. Is the sign saying that if you trespass you’re a criminal? No, but it’s saying if you do trespass, the owner reserves the right to shoot at you with a gun, and you’ve been duly warned. Can Google’s TOS be taken in a similar light? That is, I can send automatic queries with a clear conscience, I just need to be aware that Google reserves the right to combat my activities in any way they see fit, and I’ve been duly warned of that fact? Or, when I visit their site, am I formally entering into a binding contract and giving my word to follow their TOS, any violation of which makes me a liar?

All I was doing was manually typing in some keyword searches, and bam! I got the screenshot you see below. The restriction lasted for about 20 minutes and now it works again, but not after seriously interrupting my work and making me wonder if it was going to take hours, days, or weeks to sort out. I guess the way I was typing in my searches must have coincided perfectly with some algorithm they have for detecting automatic queries. Well Google, I guess you have a ways to go.

hiee dis is [name withheld to protect the silly] n i hav got over 4 years of experience in Internet mktg n in overall SEO n SEM operations? n wud like to serve u people with my efforts? M luking for any work at home opportunities Related with the SEO n SEM operations, Or anything related with an Internet Marketing, n when i works, u can c the results ASAP, I know exactly, how to improve the website’s Rank Rapidly n quickly on search engines, cos I hav got the Creative ideas n diff n innovative techniques by which We can deliver tons of traffic everyday to any website. so i can ensure the quality work for u people? So for further details u can hav a luk to my CV In Attachment. hope n wish to hear positive frm u people Thanx Regards [name]

I don’t like to make fun of people, so I held onto this email for a few days, but then I decided it was just too good to not share.

Yeah, stupid. I’ve found when you make a mistake the best thing to do is to own up to it, and then find something interesting you can learn from it. Aside from learning to check the stats on my blog more than once every three months, I’ve also learned that the robots.txt file actually works. Yeah, in fact it works really, really well. This graphic says it all:

That’s what happens when your robots.txt settings look like this:

User-agent: *
Disallow: /

Not all that different than:

User-agent: *
Disallow:

Wouldn’t you say? But it’s quite different. It’s the difference between having your site almost completely de-indexed or having your site indexing marvelously. So kids, learn from my mistake and don’t do this at home, at work, or anywhere else unless you actually want to avoid the search engines.

The first thought that occurred to me is that perhaps it’s some sort of automated spam that causes these types of results to appear in the analytics for countless websites, and then people like me actually start searching for “sedat yüksel”. The only flaw with this plan, if it is indeed the plan, is that when I searched for “sedat yüksel” it didn’t pull up anything meaningful. I can’t see why anyone would want to optimize for the keywords “sedat yüksel” or drive traffic to any of the sites that do come up for “sedat yüksel”.

I’ve had some other ideas as well, but they all break down when I try to figure out what the motivation would be unless it’s just good ol’ mischevious fun on behalf of some 14-year old hacker in Russia named Sedat Yüksel. Well, whatever the case, if you know what Sedat Yüksel is up to, if it’s a person, or if sedat yüksel is some sort of Slavic dish involving cabbage, let me know.

You see, I’m in a bit of a unique situation. I’ve got Domain A, and my associate has Domain B. My associate owns the content on Domain A, but wants to move it to Domain B, and then he wants me to 301 redirect Domain A to Domain B. But just temporarily. After two months, I’ll remove the 301 redirect from Domain A and will do with it as I please. However, my interest in Domain A is primarily based on its current value, which is substantial, and I’m concerned that after two months of 301 redirecting it elsewhere that the value will have disappaited like so many programmers around the continental breakfast table at a conference once the muffins run out.

So, if anyone has the definitive answer to my question, I’d love to hear it. If not, I’ll probably take the risk just to find out what happens, and then report back here upon completion of the experiment.

Take billboards, for example. Prices vary, but in a decent urban area you’ll pay anywhere from $2K to $5K per month or more for a billboard. That’s about the same as what you would pay for a decent SEO campaign, but the SEO campaign excels in two ways; 1) you actually know what you’re getting (with a billboard you can only guess at the results unless you’re driving traffic to a unique phone number or website or using a code or something for tracking purposes), and 2) the billboard is only affecting a local audience, whereas SEO is national/global. Now, for a business that only sells locally, the billboard might be the better route, or at least a good option, but for a national company there’s no comparison between the value of SEO and a single billboard.

If you want to go national with advertising, you can look to radio, TV, phone books, magazines, and newspapers. But here you’re looking at astronomical costs. If you want to take out a full page ad in a national magazine with 100,000 subscribers you might pay tens of thousands of dollars. And that’s just to run your ad once. Again, that might be a good option, but you might get 12 months of SEO for the same price, and you can track the results better and you stand a good chance of getting more results.

I’m not saying SEO is better than everything else out there no matter what–different situations require different solutions. But when companies are looking to cut costs and can’t afford expensive advertising with dubious results, SEO logically must begin to look more and more attractive because of it’s higher potential for a positive ROI and easy-to-track results. What are your thoughts?

Disclaimer: I’m not really calling you a liar, cheat, or dishonest person. But I had to get your attention somehow because I want answers to this question, and apparently the post title worked. And if you use one of the tools mentioned in the post title, there’s at least a chance that you are doing something dishonest. First, try out this simple self-test to determine whether you’re an honest person or not:

1. Do you steal things from stores?

2. Do you tell lies or misrepresent things?

3. Have you ever charged a client for SEO services you never provided?

4. Have you ever made outrageous claims you knew were false about your SEO services, just to get a client to sign up?

5. Have you ever signed a contract and then violated it because it was beneficial to you and you knew you could get away with it?

6. Have you ever used Web CEO, SpyFu, SEOmoz’s Rank Checker, or any other similar programs?

If you’re thinking that question #6 doesn’t seem to fit with the others, then you’re where I was a few months ago. But then I started doing some research and stumbled onto section 5.3 of Google’s Terms of Service, which reads:

5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.

Tools like Web CEO, SpyFu, and SEOmoz’s Rank Checker (as near I can tell) all perform automatic queries of Google. They also display data from Google within an interface that is not provided by Google. Therefore, these tools violate the Google Terms of Service, as do you when you use these tools.

There seem to be five primary arguments for ignoring section 5.3 of the Google Terms of Service:

1. I’ll never get caught or be punished.

2. Everyone else is doing it.

3. It doesn’t hurt anyone.

4. Google doesn’t care anyway.

5. The intent of section 5.3 isn’t to stop programs like SpyFu or SEOmoz’s Rank Checker.

Let’s take these one at a time.

1. I’ll never get caught or be punished. The same rationalization is used by many, if not most, criminals. Would you say that somebody who has a foolproof way to steal cars and never get caught is an honest person? Of course not. Whether or not someone ever gets caught or punished has nothing to do with whether they are honest or not.

2. Everyone else is doing it. Just because everyone else is doing it doesn’t make it an honest act. The same logic has been used for downloading pirated music and movies, doing drugs, and killing Jews.

3. It doesn’t hurt anyone. To say something doesn’t hurt anyone is speculative at best, and merely a way to make one feel better about actions they are uncomfortable with but want to justify somehow. People use the same logic to steal from Wal-Mart. After all, they’re just a big corporation, so how does me stealing some batteries hurt anyone? That argument can be made, but it is, admittedly, a more difficult argument to make when it comes to automatically querying Google. But it doesn’t matter. Whether or not it hurts Google or anyone else doesn’t change the fact that automatically querying Google is a violation of their terms of service.

4. Google doesn’t care anyway. Just as with #3, the same goes for claiming that Google doesn’t care. First of all, you’re making an assumption that may or may not be true, and second, it doesn’t matter. Violating an agreement is breaking your word and being dishonest, regardless of how the person or entity you’ve made that agreement with feels about it. If they don’t care, then they can let you out of the agreement, but for someone to make an agreement and then break it based on the assumption the other party doesn’t care is still dishonest.

5. The intent of section 5.3 isn’t to stop programs like SpyFu or SEOmoz’s Rank Checker. This is the only way out, but it’s a hard case to make. In order to say that you are not breaking your agreement with Google, you would have to somehow claim that Google does not mean what they seem to be saying. That the words they are using mean something other than what those words generally mean. You would have to be able to prove that when Google says they don’t want you to “access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google” that this does not apply to Web CEO. You would have to be able to read the words “You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers)” and interpret that as not applying to how SEOmoz’s Rank Checker gets its data. If you can do that, I’d like you to explain it to me, because I can’t see any way around it.

So what is the intent of section 5.3? My opinion is that it truly is not intended to target services like Web CEO or SpyFu. I believe Google included section 5.3 as a safety measure to give them the ability to prosecute anyone doing such things if Google deems it worth pursuing. I can imagine a meeting between Google and their lawyer going something like this:

Google: “Hey, what about this section 5.3? Why do we need that in there?”

Lawyer: “Because you might need to use it some day.”

Google: “What do you mean? I can’t see any reason why I’d need to stop anyone from automatically querying us? I mean, I know these tools like Web CEO and SEOmoz Rank Checker are doing it, but we don’t care about that stuff.”

Lawyer: “You might not see a reason today, and even I can’t think of a reason, but you should put it in there any way just in case. Something may happen that you can’t foresee that damages your business and that section may be the only way you can prosecute the offender and stop it.”

I’m 99.99% sure if Web CEO or any other company talked to Google and Google were completely transparent, Google would say “Go ahead, we don’t care.” But of course Google can’t say that publicly, because that could be used against them in a court of law if they ever did want to sue someone based on section 5.3.

If my opinion is correct, then that’s pretty annoying, isn’t it? After all, how are we supposed to track rankings for our client? By checking them manually? What if we’ve got 50 clients and each one has 200 keywords they’re tracking? Well, then you’ve got a problem. But people have problems all the time. What if everybody said “Hey, I believe in being honest when it’s in my favor, but if it benefits me to break my word then I don’t think that’s dishonest.” That’d be a nice world to live in, wouldn’t it?

Honest people are honest whether or not they’ve got problems and regardless of the money involved. Here’s what real honesty looks like.

Jon Huntsman owned the largest private business in the world–Huntsman Chemical. He’s mostly retired now and is funneling his billions of dollars into curing cancer. In 1986 Jon Huntsman agreed to sell 40% of his company to Emerson Camp, the chairman and CEO of Great Lakes Chemical for $54 million. They shook hands on the deal, but then Emerson lagged on getting the paperwork together. Six months later, nothing had been done on the deal, and Huntsman’s company had done spectacularly well and some things had happened with the economy so that in that short time period that 40%, instead of being worth $54 million, was now worth $225 million, or $171 million more than it was worth just six months earlier.

Emerson came to Huntsman and said that he knew the business was worth about $200 million. Huntsman told him no, it was worth about $225 million. Emerson suggested they split the difference, and that he would pay $125 million for the 40% of the business. Huntsman told him no, he wouldn’t do that deal. He told him he had shook his hand six months earlier and told him he would sell him that part of the business for $54 million, and that’s what he was going to stick to because he had given his word.

So are you an honest person? If you like to think that you are, and you use these SEO tools, then what decision are you going to make that will allow you to sleep at night?

Now, is there some way out of this? Perhaps. A company can try to make an agreement with Google so that they have permission to automatically query the search engine. I’ve tried that myself, but have been unable to get a response. And so far I haven’t found anyone else who has been successful either. If you know of someone who has made such an agreement, I’d love to talk to them. Or if you see some other way around this that allows a company to automatically query Google without violating the terms of service, I’d love to hear about it, because I like these tools and would like to be able to use them without feeling like I’m a dishonest person. I would love to find a way around section 5.3, I just don’t see it.