Yeah, stupid. I’ve found when you make a mistake the best thing to do is to own up to it, and then find something interesting you can learn from it. Aside from learning to check the stats on my blog more than once every three months, I’ve also learned that the robots.txt file actually works. Yeah, in fact it works really, really well. This graphic says it all:

That’s what happens when your robots.txt settings look like this:

User-agent: *
Disallow: /

Not all that different than:

User-agent: *
Disallow:

Wouldn’t you say? But it’s quite different. It’s the difference between having your site almost completely de-indexed or having your site indexing marvelously. So kids, learn from my mistake and don’t do this at home, at work, or anywhere else unless you actually want to avoid the search engines.

The first thought that occurred to me is that perhaps it’s some sort of automated spam that causes these types of results to appear in the analytics for countless websites, and then people like me actually start searching for “sedat yüksel”. The only flaw with this plan, if it is indeed the plan, is that when I searched for “sedat yüksel” it didn’t pull up anything meaningful. I can’t see why anyone would want to optimize for the keywords “sedat yüksel” or drive traffic to any of the sites that do come up for “sedat yüksel”.

I’ve had some other ideas as well, but they all break down when I try to figure out what the motivation would be unless it’s just good ol’ mischevious fun on behalf of some 14-year old hacker in Russia named Sedat Yüksel. Well, whatever the case, if you know what Sedat Yüksel is up to, if it’s a person, or if sedat yüksel is some sort of Slavic dish involving cabbage, let me know.

You see, I’m in a bit of a unique situation. I’ve got Domain A, and my associate has Domain B. My associate owns the content on Domain A, but wants to move it to Domain B, and then he wants me to 301 redirect Domain A to Domain B. But just temporarily. After two months, I’ll remove the 301 redirect from Domain A and will do with it as I please. However, my interest in Domain A is primarily based on its current value, which is substantial, and I’m concerned that after two months of 301 redirecting it elsewhere that the value will have disappaited like so many programmers around the continental breakfast table at a conference once the muffins run out.

So, if anyone has the definitive answer to my question, I’d love to hear it. If not, I’ll probably take the risk just to find out what happens, and then report back here upon completion of the experiment.

Take billboards, for example. Prices vary, but in a decent urban area you’ll pay anywhere from $2K to $5K per month or more for a billboard. That’s about the same as what you would pay for a decent SEO campaign, but the SEO campaign excels in two ways; 1) you actually know what you’re getting (with a billboard you can only guess at the results unless you’re driving traffic to a unique phone number or website or using a code or something for tracking purposes), and 2) the billboard is only affecting a local audience, whereas SEO is national/global. Now, for a business that only sells locally, the billboard might be the better route, or at least a good option, but for a national company there’s no comparison between the value of SEO and a single billboard.

If you want to go national with advertising, you can look to radio, TV, phone books, magazines, and newspapers. But here you’re looking at astronomical costs. If you want to take out a full page ad in a national magazine with 100,000 subscribers you might pay tens of thousands of dollars. And that’s just to run your ad once. Again, that might be a good option, but you might get 12 months of SEO for the same price, and you can track the results better and you stand a good chance of getting more results.

I’m not saying SEO is better than everything else out there no matter what–different situations require different solutions. But when companies are looking to cut costs and can’t afford expensive advertising with dubious results, SEO logically must begin to look more and more attractive because of it’s higher potential for a positive ROI and easy-to-track results. What are your thoughts?

I’m not necessarily accusing you of being a liar, unethical, or dishonest, but I had to get your attention somehow and apparently the post title worked. And if you use one of the tools mentioned in the post title, there’s at least a chance that you are doing something dishonest. First, try out this simple self-test to determine whether you’re an honest person or not:

1. Do you steal things from stores?

2. Do you tell lies or misrepresent things?

3. Have you ever charged a client for SEO services you never provided?

4. Have you ever made outrageous claims you knew were false about your SEO services, just to get a client to sign up?

5. Have you ever signed a contract and then violated it because it was beneficial to you and you knew you could get away with it?

6. Have you ever used Web CEO, SpyFu, SEOmoz’s Rank Checker, or any other similar programs?

If you’re thinking that question #5 doesn’t seem to fit with the others, then you’re where I was a few months ago. But then I started doing some research and stumbled onto section 5.3 of Google’s Terms of Service, which reads:

5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.

Tools like Web CEO, SpyFu, and SEOmoz’s Rank Checker (as near I can tell) all perform automatic queries of Google. They also display data from Google within an interface that is not provided by Google. Therefore, these tools violate the Google Terms of Service, as do you when you use these tools.

There seem to be three primary arguments for ignoring section 5.3 of the Google Terms of Service:

1. I’ll never get caught or be punished.

2. Everyone else is doing it.

3. It doesn’t hurt anyone.

4. Google doesn’t care anyway.

5. The intent of section 5.3 isn’t to stop programs like SpyFu or SEOmoz’s Rank Checker.

Let’s take these one at a time.

1. I’ll never get caught or be punished. The same rationalization is used by many, if not most, criminals. Would you say that somebody who has a foolproof way to steal cars and never get caught is an honest person? Of course not. Whether or not someone ever gets caught or punished has nothing to do with whether they are honest or not.

2. Everyone else is doing it. Just because everyone else is doing it doesn’t make it an honest act. The same logic has been used for downloading pirated music and movies, doing drugs, and killing Jews.

3. It doesn’t hurt anyone. To say something doesn’t hurt anyone is speculative at best, and merely a way to make one feel better about actions they are uncomfortable with but want to justify somehow. People use the same logic to steal from Wal-Mart. After all, they’re just a big corporation, so how does me stealing some batteries hurt anyone? That argument can be made, but it is, admittedly, a more difficult argument to make when it comes to automatically querying Google. But it doesn’t matter. Whether or not it hurts Google or anyone else doesn’t change the fact that automatically querying Google is a violation of their terms of service.

4. Google doesn’t care anyway. Just as with #3, the same goes for claiming that Google doesn’t care. First of all, you’re making an assumption that may or may not be true, and second, it doesn’t matter. Violating an agreement is breaking your word and being dishonest, regardless of how the person or entity you’ve made that agreement with feels about it. If they don’t care, then they can let you out of the agreement, but for someone to make an agreement and then break it based on the assumption the other party doesn’t care is still dishonest.

5. The intent of section 5.3 isn’t to stop programs like SpyFu or SEOmoz’s Rank Checker. This is the only way out, but it’s a hard case to make. In order to say that you are not breaking your agreement with Google, you would have to somehow claim that Google does not mean what they seem to be saying. That the words they are using mean something other than what those words generally mean. You would have to be able to prove that when Google says they don’t want you to “access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google” that this does not apply to Web CEO. You would have to be able to read the words “You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers)” and interpret that as not applying to how SEOmoz’s Rank Checker gets its data. If you can do that, I’d like you to explain it to me, because I can’t see any way around it.

So what is the intent of section 5.3? My opinion is that it truly is not intended to target services like Web CEO or SpyFu. I believe Google included section 5.3 as a safety measure to give them the ability to prosecute anyone doing such things if Google deems it worth pursuing. I can imagine a meeting between Google and their lawyer going something like this:

Google: “Hey, what about this section 5.3? Why do we need that in there?”

Lawyer: “Because you might need to use it some day.”

Google: “What do you mean? I can’t see any reason why I’d need to stop anyone from automatically querying us? I mean, I know these tools like Web CEO and SEOmoz Rank Checker are doing it, but we don’t care about that stuff.”

Lawyer: “You might not see a reason today, and even I can’t think of a reason, but you should put it in there any way just in case. Something may happen that you can’t foresee that damages your business and that section may be the only way you can prosecute the offender and stop it.”

I’m 99.99% sure if Web CEO or any other company talked to Google and Google were completely transparent, Google would say “Go ahead, we don’t care.” But of course Google can’t say that publicly, because that could be used against them in a court of law if they ever did want to sue someone based on section 5.3.

If my opinion is correct, then that’s pretty annoying, isn’t it? After all, how are we supposed to track rankings for our client? By checking them manually? What if we’ve got 50 clients and each one has 200 keywords they’re tracking? Well, then you’ve got a problem. But people have problems all the time. What if everybody said “Hey, I believe in being honest when it’s in my favor, but if it benefits me to break my word then I don’t think that’s dishonest.” That’d be a nice world to live in, wouldn’t it?

Honest people are honest whether or not they’ve got problems and regardless of the money involved. Here’s what real honesty looks like.

Jon Huntsman owned the largest private business in the world–Huntsman Chemical. He’s mostly retired now and is funneling his billions of dollars into curing cancer. In 1986 Jon Huntsman agreed to sell 40% of his company to Emerson Camp, the chairman and CEO of Great Lakes Chemical for $54 million. They shook hands on the deal, but then Emerson lagged on getting the paperwork together. Six months later, nothing had been done on the deal, and Huntsman’s company had done spectacularly well and some things had happened with the economy so that in that short time period that 40%, instead of being worth $54 million, was now worth $225 million, or $171 million more than it was worth just six months earlier.

Emerson came to Huntsman and said that he knew the business was worth about $200 million. Huntsman told him no, it was worth about $225 million. Emerson suggested they split the difference, and that he would pay $125 million for the 40% of the business. Huntsman told him no, he wouldn’t do that deal. He told him he had shook his hand six months earlier and told him he would sell him that part of the business for $54 million, and that’s what he was going to stick to because he had given his word.

So are you an honest person? If you like to think that you are, and you use these SEO tools, then what decision are you going to make that will allow you to sleep at night?

Now, is there some way out of this? Perhaps. A company can try to make an agreement with Google so that they have permission to automatically query the search engine. I’ve tried that myself, but have been unable to get a response. And so far I haven’t found anyone else who has been successful either. If you know of someone who has made such an agreement, I’d love to talk to them. Or if you see some other way around this that allows a company to automatically query Google without violating the terms of service, I’d love to hear about it, because I like these tools and would like to be able to use them without feeling like I’m a dishonest person.

1. Set up a blog, just like this one.

2. Spend 100+ hours writing posts.

3. Spend another 20+ hours building links to your posts.

4. Give up and don’t bother to update your blog for a year or so.

5. Start posting again, and huzzah! You’ll start getting links from SearchCap.

Well, I wish I could enlighten you more on what did the trick, but I have no clue. The steps I’ve outlined are the exact steps I took. I won’t count out other factors involved like charm, good looks, and charisma, but if you’re looking for anything more I’m afraid I can’t help you.

Update: Since posting this, it has come to my attention that there is no “About OrangeSoda” paragraph at the end of the press release, although there is one for REMAX, which might indicate that this press release wasn’t even issued by OrangeSoda, but perhaps by REMAX, which affects some of the comments I’ve made below about OrangeSoda.

Any SEO professional knows that it’s easy to get a company ranking well in Google for that company’s name, especially if that company’s name is present in the URL the company uses for its website. For example, if I have a company named Purple Monkey Dishwasher and I own the URL PurpleMonkeyDishwasher.com, then it’s going to be a snap to get the #1 spot in Google when people search for “purple monkey dishwasher”. But what if Purple Monkey Dishwasher is a bike shop located in Tampa, Florida? What good does it do me to rank well for “purple monkey dishwasher” in that case? The only benefit I get is that people who already know who I am will be able to find me. But they’re going to find me anyway if they’re looking for me. The problem is I’m not going to get any new customers.

That’s more or less what is being trumpeted as search engine optimization success in this press release from Utah-based SEO firm OrangeSoda, only the company is REMAX of New Jersey. As stated in the release “When people search Google or another search engine for homes for sale in new jersey, or on our brand name, they’ll see RE/MAX New Jersey at the top of the list. Thanks to our partnership with OrangeSoda we’re seeing top results on many real estate keywords,” said Jeanie Farrell of RE/MAX New Jersey.

Now there’s the thousand-dollar question; is the REMAX site really showing up in the top results for “many real estate keywords” or just the company name? According to the Google Keyword Tool the top five real estate/homes for sale related keywords with accompanying search volumes are:

new jersey real estate – 368,000
new jersey homes – 201,000
new jersey homes for sale – 110,000
new jersey homes sale – 110,000
homes in new jersey – 22,200

By comparison searches for “remax new jersey” total approximately 2,900 per month.

The New Jersey REMAX website doesn’t show up in the top 10 search results for any of these keywords, and doesn’t show up in the top 100 search results for the fifth. To be fair, the site is showing up in the 10-20 results for the first four keyword searches. While not exactly “top results” you could make the argument that the site is “on it’s way” to the top, assuming the current rankings are an improvement over what they were before REMAX New Jersey started working with OrangeSoda. Or I suppose you could even make the argument that those are “top rankings” if by “top rankings” you mean anything within the first 20 results.

What really matters, of course, is whether New Jersey REMAX is getting more business as a result of this campaign than they would otherwise. Judged from that perspective, it could be that coming up in the top results for searches for “REMAX” could be generating a net gain, and that being in the top 20 results for keywords that are heavily searched also generates results good enough to justify the expense of OrangeSoda’s SEO services.

Why bother posting about this at all? Primarily because I think it’s misleading. The release puts forth the idea that OrangeSoda is doing great things for REMAX New Jersey by getting them ranking well for searches for their company name, which isn’t much of a challenge, nor is it likely to generate much in the way of real improvement to the client’s revenues. Ranking for the other terms listed above is likely to provide real value
for the client, but rankings that aren’t in the top 10 aren’t exactly newsworthy.

The bottom line is that if you’re a client looking to hire an SEO company, make sure you’re getting a true positive return on your investment, based on a clear comparison of what results you would be getting without the SEO firm vs. with them.

Yesterday I was reading an article on CNN.com entitled Poll: GOP image goes from bad to worse. I then went to my politics blog and posted asking why are Republicans so unpopular these days? Within a few hours I noticed something happening that hadn’t ever happened before, which was that I was getting multiple comments on this blog post. That’s unusual for this blog, because it’s a new blog and nobody reads it other than one friend of mine. So I checked my stats and I had received about 60 visitors during the day to the page, and they were coming from the CNN page I had linked to.

Now it’s not there anymore, and I can’t link to another page as an example since that page will probably also change before you read this, but on many CNN stories there is a link at the bottom saying “From the Blogs”. If you expand the link, it will show you links to three blogs, along with snippets of what is on those blogs. You can then click through to the actual blog. These links are no-followed, but as I’ve found out, they can generate some traffic. About 100 unique visitors in my case. If you check the source of the page and search for “clearlydeparted” you’ll still find the link to my blog in the code.

However, time is of the essence. Apparently you have to post quickly after the original article is posted to have a shot at being one of the blogs that gets a link back. Which means you may want to sign up for the CNN feed so you can be Johnny-on-the-spot and post quickly.