Enterprise SEO RSS From re1y.com

The Risks Of Relying On Search Results

by Bob Sakayama

14 April 2011

(This is the longer version of the presentation made at the London seminar. Ray Snoddy's article on the topic is posted here)

Example Of Widespread, Unrecognized Risk:

I realize that risks of a search engine penalty, or a Google penalty, is something that is not very well understood, even by the community of website businesses. So I'd like to start by demonstrating how this risk may be present for your enterprise very early in the process, long before any commerce even takes place on the new website.

Imagine this scenario:

Your enterprise has acquired the worldwide rights to market a hugely successful new product called Super Magic Widgets. This product has such incredible potential that the decision is made to create a separate internet business to directly market it, rather than distributing it through existing venues.

Of course, the very first action is to search for a suitable domain to launch the internet commerce platform for the marketing effort. The selection of the domain is very important, because if it carries the correct semantics, it can make a huge difference in the rank performance of the website. Search engines respond to the relevancy of the domain semantic, so if you're selling widgets, it's a very smart move to make sure that the domain name includes the term "widgets." To your amazement, marketing team finds that SuperMagicWidgets.com is available and buys it (with the blessing of your seo team).

Right at this point, if you do not vet this domain for search viability before pouring resources into the implementation, you could create an existential risk to this new business. Because if this domain had a previous owner who nuked it with a toxic optimization strategy, it will never perform in the natural search. This happened to me in 2007, and I since then I have prevented many clients from wasting resources on a dead domain.

Domains have existed long enough now so that many that are in the pool of available domains have had previous lives. Their previous owners were either unsuccessful at building businesses on them, lost interest, or created such a mess that the domain was 'nuked' so it could never rank. So prior to purchase, the enterprise must be able to know if the domain is viable in the search. It is obvious that as more and more domains are abandoned, that this problem is growing with time. Also, when looking at available domains, the more generic and desirable the name, the more likely it was pre-owned.

In order to protect the enterprise, the domain name vet must proceed no matter where the domain is purchased - whether from the publicly available pool, or from an individual or business selling the domain. I am aware of one case where a domain that sold for $20,000 by the previous owner was penalized and not recoverable.

One interesting side note: Non-dot com domains may have a lower risk of previous ownership and provide the same semantic edge. For example, the .co domains became universal only in the past year, so there should be a low probability of a previous owner triggering penalties on them.

Most of the penalties that we work on are not related to domain purchases, but rather are applied to once thriving websites. While there may be many variations on exactly how these sites are penalized, the one thing they all have in common is that they are all financial emergencies - where at least one major revenue stream is lost. In some cases the penalty is across a network of sites. These are existential threats to the businesses involved - I have seen businesses forced to shut down as a result of the revenue loss. Given this fact, it is critical to begin to understand the risks taken on when businesses choose strategies that depend upon performance in the natural search.

A Devastating, Yet Largely Unknown & Misunderstood Risk:

The enterprise thrives by making smart decisions, especially in the area of risk management. Internet businesses must contend with a special set of risks that accompany their heavy reliance on technology. The reliability of the website's functionality and the security of these systems are 2 obvious examples of well known risks that have solutions in place.

But for internet businesses that rely on the natural search to provide sales and leads, there are significant risks that remain very poorly understood. These risks are present because the search ranks of your business are not within your control - you rely on a third party, the search engine, to provide the ranking mechanism. This dependency inserts a huge unknown into the pool of risks. One such risk is the possibility of devastating rank loss due to a search engine penalty.

According to a survey mentioned here, only 11% of the website owners who responded were aware of the possibility of a search engine penalty impacting their business. Given what we already know from our client work about the devastating consequences of Google penalties, this is shocking. We have seen $100 million businesses watch their revenue streams shut down overnight because all their previously productive sites could no longer be found in the search.

One of the explanations for the lack of knowledge on search penalties is that it would only impact an organization whose websites have already achieved productive, organic search ranks - in reality, that's #1-5 on page one of a Google search - but let's count any site on page 1. The businesses that hold those positions represent a small club compared to all others. Most businesses BUY their search engine traffic, via ads that appear along with the organic results. So the number of sites being harmed by search penalties is relatively small.

Another reason we don't know more: Most of our clients who have been penalized expect confidentiality, so the damage is essentially kept secret, and that just serves to insulate the larger community from awareness of the risks.

The small number of victims, and the privacy concerns of those compromised businesses keep us ignorant. This ignorance creates an environment in which most successful search dependent businesses do not have an understanding of the potential for catastrophic rank loss.

Now imagine the impact on a longstanding, search dependent business of a severe penalty that degrades all their productive ranks back to page 5 or worse. In many instances, that would be an existential threat to the enterprise. And it happens more often than most realize.

Why Websites Get Penalized

A large part of our work has been an attempt to understand why sites get penalized, and to gain insight into the restoration process. We have intentionally penalized sites, then unwound those penalties in order to discover where the red lines are. We also keep many domains in penalties as platforms for other experiments. This is really the only way to observe how penalties are handled by Google, and to gain an understanding of the impact on related sites. So, for example, we know that, for most compliance based penalties, links from penalized sites do not cause harm - we can see that from many perspectives on our own sites.

Google would like us to believe that sites get penalized for falling outside their published "guidelines" - their rule set for search behavior. And for the most part this is probably true - we see many penalties that get imposed for obvious breaches of the guidelines. And their ranks return when the breach is remedied - ie. when search compliance is restored.

Many retail SEOs believe that all rank issues can be remedied with optimization. We view the process of obtaining productive high ranks to be broken into 2 distinct parts: compliance & optimization. Search compliance means making sure the site is implemented within the guidelines and is indexable or viable for the bots. Optimization is the strategy or technique used to manipulate ranks higher. Attempting to optimize a non-compliant website will either fail (no rank improvement), or trigger an event (penalty). Maintaining search compliance is critical to avoiding penalties, and is something that automation can be applied to.

The world would be a much less risky place if the "published rules" compliance model were the only explanation for penalties. But it turns out that the published guidelines are only that - guidelines. The actual rules are secret. And constantly changing.

We know from first-hand experience that optimization strategies that performed fabulously in 2001 can now get you penalized. In fact, if you look at the list of "don'ts" from the guidelines (examples below) all of them were once effective strategies.

- Avoid hidden text or hidden links.
- Don't use cloaking or sneaky redirects.
- Don't send automated queries to Google.
- Don't create multiple pages, subdomains, or domains with substantially duplicate content.

Google updates their algorithm very frequently - evidence of reordering sometimes occurs daily. This constantly changing rules set, combined with zero transparency from the search engines, creates an environment where the seo strategies used by many seo agencies lag behind the current best practices. So much so that uninformed seos are one of the main causes of search penalties.

We also know that Google's infrastructure is vulnerable to attack. And that 3rd parties are able to take advantage of these vulnerabilities to hack their index, inserting information that leads to high ranks being hijacked. Sometimes the frailties of the algorithm create rank issues by themselves. Google is usually very quick to address these issues when they are pointed out, but the fact that the frailties exist at all creates a risk.

Another consideration is that most large sites are complex structures, and their complexity often explains how they might fall outside compliance inadvertently, and how that might escape detection until a penalty is imposed. Automation can be your best friend when it's working for you, and your worst nightmare when it triggers an unintended non-compliance. Most penalties are triggered by human error or oversight. And at the enterprise level, this includes compliance breaches caused by everything from poorly coded automation to mistakes made by a copywriter. Ignorance is almost always behind most penalties triggered by non-compliance.

Because businesses are harmed/destroyed by penalties and because of how these penalties are imposed, ethics enters the conversation about search. As the dominant player, we would hope that Google would be sensitive to the ethical quandaries they create as a matter of course, and in some way act to mitigate these perceived risks. But this is not happening - in fact, the ethical issues seem to grow larger with time. Consider the ethical appropriateness of:

-1- imposing harsh punishments based on secret rules
-2- an unwillingness to even acknowledge when a site is penalized
-3- being the sole arbiter of justice with a stake in the decision
-4- permitting no recourse
-5- creating victims through Google's own frailty
-6- penalizing sites in Google for the actions of 3rd parties
-7- rules that change during the game
-8- the absence of an effective warning mechanism

Managing the risks

So the risks of relying the search are very real, but how are they being addressed? This is the question that we all need to seriously address because the current solutions are not viable. For example, a check of your errors and omissions policies probably rules out an insurance claim. A Google penalty is more like an act of God, than an understood market risk.

Even our law enforcement agencies are unclear as to how sites harmed in the search can seek justice. I once was asked by a client to report an attack on their ranks to the FBI. This was an instance in which my client was losing half a million dollars in sales per day as a result of a Google penalty triggered by the actions of a third party. My clients sites were not hacked, Google's index was, so the attack on my client's revenue was indirect, but the impact was severe and instantaneous. After explaining several times what had taken place, and the losses caused by it, the FBI agent asked me, "What's the crime?" There was no way for them to take any kind of enforcement action because there was no obvious violation of law.

Even though our work is focused on performance in the search, we strongly recommend to clients that they move away from complete dependence on search ranks. This is the only way to mitigate this risk. Many companies create numerous websites and feel that this diversifies their exposure to a penalty, or any kind of rank loss. But we often find penalized environments in which all the websites are penalized simultaneously. Other online techniques that can act independently of the search include a robust social media presence, targeted email campaigns, other advertising, promotion.

Finally, there is one solution that is not within the grasp of us as individual businesses, but may be available if we act in concert. And that is to work to force the search engines to act more responsibly when it comes to their enforcement actions. Some form of regulation is clearly warranted, but exactly how this will come about is unclear. There are some tiny first steps in this direction that I would advocate. For example, simply notifying sites that they are penalized.

Every major industry that developed in the US, including railroads, telephone, petroleum, steel, etc. all evolved in such a way that eventually a monopoly was created. And regulation was required to force ethical standards and fair play into those markets. We are at such a point now with search, without the regulation.

Related notes - short term solutions for penalized sites:

AdWords
Affiliates
Email marketing
Social media
other search engines

Turn Old Product Pages Into Link Bait

12 August 2010 : Bob Sakayama

Every time the product line changes, in many cases every year, most businesses simply discard the old product pages and replace it with the new one. By "cleaning house" in this way, we keep the site neat and orderly - but should we rethink this process? Could there be some value in preserving the record of the product evolution as it changes over time? The answer to both questions is: absolutely!

Since you already know what this article is advocating from the titles, you probably wonder what good are those old product pages. You probably never thought it might actually be a candidate for link bait. But it is.

Who would want to link to your old product pages? Have you ever searched for an obsolete product and found it in an archive? If you're a fashion maven and like to compare old styles with new, you probably already link to the stores or manufacturers that keep active archives. The point is that archives of discontinued products are valuable - the bigger they are the more valuable. And valuable archives get links.

The primary caveat is that you organize your archive of discontinued products so that there is absolutely no chance any could be confused with available products. But that's easy. Just create a "discontinued" link and automate the display to move from the store to the archive when a product is discontinued. Make that area easily searchable, and keep it organized as you do the rest of the store - probably brand centric, or by product type. The goal is to have an archive where it's easy to find things.

Another caveat: make sure that the archive product descriptions are not exactly like the new product descriptions. But even if they are very close, you can add uniqueness by coding a "archive" or "discontinued" tag (or any other marker that is appropriate) whenever the discontinued product appears in text. Creating a special automated discontinued product script using a variable for the product may also contribute to uniqueness if your old/new descriptions appear too similar.

The beauty of this technique is that it can be automated, requires no ongoing work to manage, and can actually help the sale of new products. People very often search for the product they have, even if it's obsolete. If they find it on your site, even if it's in the archive, they are much more likely to at least look at your new offerings.

Since this area tends to grow very large, you should spend some time making sure it is manageable and displays well once the numbers get large. But a very simple data set up is all that's really needed to keep a monster archive functional and accessible.

If you're discarding old product pages every time the manufacturer discontinues a product, you're ignoring an opportunity to build the content critical mass of your site. You might be throwing away valuable stuff.

Keep in mind that value only comes with time, so start it NOW. Your archive's attractiveness increases if it's been around a while. Think about archive.org - they create an archive of the web that is only valuable because it has been archiving for so long. The early adopters will have the most valuable archives.

If you have a comment on this strategy, please post it here.

Google Is Once Again Withdrawing Access To Metrics - Now Only Showing Small Set Of Sample Inbound Links Instead Of All Indexed

1 July 2010 : Bob Sakayama

There's a disturbance in the force. And it ain't welcome.

We noticed that a lot of our clients had huge changes occurring in the metrics on external links within Google Webmaster Tools.

Clearly some very large change just occurred in the way links are reported within WMT, and it's not for the better. It was never a robust area, and the links G indexed never came close to the actual numbers, but at least we had some granular information we could act on.

One of the compliance checks we always perform when checking sites for ranking issues is a link evaluation - something that relies on access to this data. It also used to be one way to gauge the effectiveness of any link building campaign, or social marketing efforts. And it's the only way to identify a link attack by a 3rd party - although G says it can't happen, we know otherwise. And now we just lost the only avenue for discovery.

By moving to this new, truncated system, we once again lose something we once had (remember when the supplemental results used to be labeled in the results?). This is one that makes some very important forensic work now impossible. If you can't see the links that Google is using to evaluate you, you can't act to address issues caused by them.

This action should be interpreted an intentional effort to reduce Google's accountability to the sites that rely on it for ranks - you can't question something you can't prove. And website owners can't improve something they can't see.

We realize that whatever information Google does provide, we should be grateful for. And clearly by only showing a tiny fraction of what they used to show us in terms of link information, can save resources that can now be allocated for other purposes. But this change is drastic and very harmful for the management of large sites.

Here's just one example: There are 114,000+ links to a url according to WMT for one client's site. But instead of permitting access to that data, the heading on the page now reads, "This table lists a sample of 13 external pages that link to (url)." That's 0.01% of the actual number of links.

Google gets negative respect for this one, and we hope we see a reversion when they recognize the extent of the blinders they just put on us all. Haven't seen any major complaints yet, so make noise if you read this and agree.

13 July 2010 - I just posted this question to the Webmaster Central Help Forum - Bob Sakayama

Is the removal of inbound link metrics from WMT a permanent degradation of that tool?

Why have you removed the most valuable tool for link discovery used by large websites from Webmaster Tools? Until a couple of weeks ago we could download the entire list of indexed inbound links, but now, even though the link on that page says "Download all external links" we only get the samples. One client's site has over 117,000 links pointing to a url, but the "sample" is only 15. ONLY 15 out of 117,000!!!!

Anyone attempting to keep a large website search compliant is hugely and negatively impacted by this change.

A white hat link building campaign can no longer be effectively evaluated from within WMT. Since Google is going to penalize sites that link inappropriately or buy links, doesn't it make sense to maintain the ability to discover and remediate these kinds of issues? A site penalized because a former seo agency bought links can no longer do the forensics necessary to discover and remove them. The withdrawal of these valuable metrics is taking Google in the OPPOSITE direction from transparency and openness - where we hoped you were going. Clearly, forcing us to wear blinders is not welcome.

Is it really necessary to eliminate this most valuable resource?

To post a question in this forum, sign in and go here:

http://www.google.com/support/forum/p/Webmasters/ask

Posted as a comment to Matt Cutt's blog 11 July, but by 14 July it is still 'awaiting moderation' - comments by others, posted after this are now showing live. Why would this comment be restricted and not pushed live? 15 July - now live.

Bob Sakayama July 11, 2010 at 9:38 pm

Would really be useful to reveal more forensic information for sites having difficulty, especially for sites that are penalized. We help in the recovery process and know that many sites that get penalized are actually themselves victims - of other seos, their own ignorance, etc., and that no malfeasance is intended. I understand that revealing too much is not wise, and that there are bad players. But for those who actually intend to be good web citizens but inadvertently triggered a Google penalty, there must be a better way.

I'm particularly alarmed at the recent pullback on information available from within Webmaster Tools on inbound links. We used to be able to download the full list, and now we're restricted to a tiny sample.

Given Google's emphasis on relevant, organic links, and the fact that Google penalizes sites with inappropriate linking, why would you intentionally neuter one of the most valuable tools used for the discovery of rank issues triggered by links? When attempting to diagnose a site's ranking or penalty issues, we are no longer able to see the links that you are crediting the site with, only a minuscule sample, many of which are often multiple links from only a few domains, making this information useless.

I strongly encourage you to revert the discovery metrics within WMT regarding inbound links, plus 2 other useful discovery items:

Redirect Detector - sites get harmed by chaining redirects, but most don't even know they're doing it. Would be great to be able to see that from within WMT

Penalty Confirmation - I know this is asking a lot. But it shouldn't be. Can we at least know, from within WMT, are we penalized, yes or no?

Search ForensicsDiscovery Through Search

One of the common problems faced by enterprise seos is how to achieve situational awareness from the outside. That is, how do you discover the structure of the implementation if you don't have server access?

This is especially important when you work with very large sites. Often no one person even has all the details about all the subdomains. In fact, subdomains may be hosted in different server environments in different locations around the world.

If you're trying to understand how the enterprise is managing it all, there are some very simple tricks that can help you assemble the pieces. Here's a really smart technique using the search functions site: and inurl: The cool thing is that these use Google's own search results to reveal a site's hidden information. (Using these searches may trigger the Google warning on the right.)

Here's a simple iterative search routine that well uncover all the indexed subdomains of any given top level domain.

Start with this search:

site:domain.com -inurl:www

This will show you all the urls without www - usually will show you https pages, and subdomains (www is actually a subdomain of every top level domain). You can easily find all subdomains by iterating through them. Pick a subdomain revealed by this search and then search like this:

site:domain.com -inurl:www -inurl:subdomain1

This will filter out all of both www and subdomain1, revealing other subdomains. By itertaing through all that you find, you end up with a search that gives no results. That search will show all the subdomains:

site:domain.com -inurl:www -inurl:subdomain1 -inurl:subdomain2 -inurl:subdomain3 ...

The limitation is the current 32 word search limit.

Try It With Google.com

When we tried to discover all the subdomains of Google, we had to stop at 32 (plus we got the warning and could do no further searches for a while):

site:google.com -inurl:www -inurl:adwords -inurl:knol -inurl:ditu -inurl:maps -inurl:local -inurl:translate -inurl:books -inurl:picasa -inurl:video -inurl:code -inurl:picasaweb -inurl:mail -inurl:chrome -inurl:ejabat -inurl:investor -inurl:wifi -inurl:labs -inurl:checkout -inurl:images -inurl:docs -inurl:photos -inurl:gears -inurl:pack -inurl:sites -inurl:documents -inurl:wave -inurl:afp -inurl:canadianpress -inurl:blogsearch -inurl:earth -inurl:answers

(limit was reached but we could still see:)

-inurl:research -inurl:trends -inurl:sitescontent -inurl:scholar -inurl:trends -inurl:toolbar -inurl:services -inurl:sketchup

Remember the "supplemental results" and all the issues surrounding their revelation and then their disappearance from the Google's search results? They're still around, and you can find them still with this secret handshake:

Here's a hack we uncovered long ago, when supplemental results markers were removed from the search results. So in a sense, this is even more important, because it lets you know whether your urls are in the main index or in the now hidden supplemental results.

We discovered a long time ago that by simply adding /* or /# to a site: search, the result set was drastically changed. Back when the supplemental results were flagged, we found that this search:

site:domain.com/*

revealed all urls in the main or primary index. If you subtracted them from the results you get when you search:

site:domain.com

the remainder were the urls in the supplemental results.

The image on this page is one we see a lot when we do these kinds of searches. Check our blog post on this. We strongly suspect we're being profiled because we see this warning often. Do you?

How Do You Evaluate The Work Of Your SEO Agencies?

by: Bob Sakayama

7 December 2009:

The biggest unknown for many clients is, "What exactly did the SEO agency do last month?" Executives tend to give agencies wide latitude if they appear to be performing. But if that latitude is the result of ignorance and the hesitancy to ask questions, a very bad syndrome sets in where the vendor has no real incentive and no accountability. If that's the case for your enterprise, you need to act.

Or if you can't assign tasks to expenses. It's one thing to agree that you need a consultant and are willing to pay a monthly fee for access. But it's an entirely different thing to have an expectation of performance, but not be able to measure that performance.

Ask your SEO agency for some specifics. Depending on the tasks they are performing for you, these might be different things, but it's not enough to point to existing ranks as proof of recent work.

Be especially cautious of agencies that use charts and metrics that display indexed link numbers, or rank changes. These metrics do not prove labor.

Some of our clients have hired SEO agencies who ONLY use metrics to justify work done. We have issues with this for several reasons. First, the metrics don't necessarily show the impact of recent work. In fact, most stable structures will very likely retain their ranks without any additional work. If the expectation is that the agency is performing rank increasing tasks each month, these metrics don't cut it.

Also, if an agency points to a chart displaying indexed links a client has no idea what kind of value is represented. For example, the most often used metrics is the numbers of indexed external links. Clients like to see large numbers and the corresponding referrals. But the link side can be gamed quite easily.

All you have to do is throw several hundred social bookmarks or software links at the site. Something you can do for very little or no money. This will spike the charts, but provide very little push. In other words, this metric can be gamed.

But more than that, the enterprise must hold everyone that works for it to account. Knowledge is the key to getting control. Agencies hired for general optimization will usually provide a detailed report if asked. Never give an agency carte blanche and a rank based incentive. We have seen too many of these relationships explode when the paid links go toxic.

Always ask for details. Always provide the expectation and hold the agency to account.

The best management strategy is to UNDERSTAND the process, and to work with your agencies to clearly define the tasks they will perform, and insist on transparency and monthly reports. Having in-house SEO expertise is a requirement for large enterprises if only to be able to better ride herd on the agencies. Yes, you're buying expertise when you hire an SEO agency, but that expertise must be held accountable to a resource allocation, not only for performance, but also for general standards of work.

The Canonical Condom

by: Rev Sale

13 October 2009:

You read that right, condom. Because condom equals protection. And it's an unsafe web.

Now that I've provoked you, let's see how this new tag is being used.

The marketing team loves the new link canonical tag, once they discovered that they could use it to manipulate existing ranks - not to create high ranks, but to REPURPOSE existing high ranks. So we can now easily replace that old, ranking product page with the new product page, just in time for Christmas. But I'm getting way ahead of myself here.

The concept of the canonical url is as old as the internet. It's the notion that ONE url has the all the necessary ingredients to produce the page in question, even though there may be other urls that produce an equivalent page in a browser. It's that notion of oneness that's not only Zen, but also identifies the uniqueness quality that Google so dearly loves.

This could be the page that runs the script. Or a product page that paginates. Or a product url that all your affiliates use when they add their tags to your url.

The point I'm making with the "condom" nomenclature is that we're finding this tag protects you from some nasty consequences that could be plaguing your site right now. For one, it protects you from the actions of your affiliates by forcing the recognition of your canonical url, even when it gets slammed with parameters and tags.

If you have secure pages, the link canonical on every page can protect you from accidentally getting your entire website indexed twice - once as http and the other as https in Google's main index. Can't tell you how many Google penalties were remediated this way.

And just posting this tag on your homepage can have a beneficial consequence. There are many implementations where the cms automation is creating undiscovered redundancies. Whether it's index.html, default.asp, home.php, or any other, the link canonical tag can wipe away decades of non-compliance. We've got plenty of client experience to support this observation - sites whose homepage was suppressed suddenly starting to show strength after being tagged. It is now our recommended practice to use the link canonical tag on every page of the site.

So whether or not you understand any other strategy we're about to discuss here, USE this tag for protection.

Some Ways You Can Benefit From The Link Canonical Tag

- If you have one article about a product, but your store gradually grew so that those blue widgets now need 5 pages. The problem is that the same article is on all 5 pages. The link canonical defining the entry url for all these pages allows Google to ignore the redundancy, because all 5 pages are seen as the same page.

- If you've seen https pages in your index, you really need this tag. Just posting it on every page will wipe out all https urls in the index. We've pulled sites out of penalties with this technique.

- If you've ever seen urls created by affiliates in your index, use the link canonical to wipe those out. These unwanted urls getting indexed are never good for your ranks, and while they may not get your ranks suppressed, they can still harm your ranks INCREMENTALLY.

- Need to get your new product page to the top of the search? You can do it if you already have the old product page ranking. Clone the old page, add the new product content, add the new page canonical to the old page & the new one. Upon next indexing the ranks should swap. There are some caveats here - and we've seen some weird results sometimes. But the experiments are basically working, and the results are very consistent when your pages really are close. Once the rank swap occurs (you'll notice that the old page will fall out of the index), redirect (always do this) and delete the old url.

Firefox now indicates whether or not you're on the canonical url. Note the gray "C" in the url bar when you visit this page - that indicates that the page in your browser is the canonical url:

http://www.e-wigs.com/Jon-Renau-Wigs.html

But is blue on this page:

http://www.e-wigs.com/brands.php?page=2&brand=Jon-Renau

Go look at the source code and see the canonical is the same on both pages.

Paying attention to this indicator can help you implement properly. For example, if you leave off the trailing slash (eg http://www.yoursite.com - should be http://www.yoursite.com/) you'll see a blue "C" instead of the correct gray color.

The link canonical is a hybrid of a redirect (a mod rewrite) and a content/PR aggregator. I'm running experiments to see if the PR is aggregated along with content, and the consequences of chained canonicals - for example what happens when you swap out the ranks a number of times. Does the inbound PR suffer like in chained redirection? Should we count the number of swaps? So many questions!!

Send us your experiments, please.

SEO Recourse - How To Use Search To Fight Back

by: OneInAmelia

8 October 2009:

SEO is way more than just a killer marketing tool or traffic builder. The ability to insert pages into competitive searches suggests a much broader power. Because if you have the ability to focus an audience's attention onto the topic of your choosing, you hold the power of propaganda. Some of us, like Rev Sale, go so far as to say it's a potential weapon of mass destruction and that we should be concerned about how a terrorist could control the publicly posted information via the search. Bob Sakayama has even consulted with the FBI on this very matter. Our discussion today acknowledges this power and demonstrates that SEO can be a very effective enabler of justice - an equalizer bar none, because in our world, little fish can take down big dogs.

Sometimes this power is obtained serendipitously - unearned as it were. Perhaps you met that crazy person who demands an unreasonable refund and starts posting nasty reviews that start showing up for searches for your brand. If you didn't give her the money, then you will probably soon realize that those posts are hurting you, and that you're getting slammed by an SEO offensive - one that can have very bad consequences for your bottom line.

Or you may be aware of the success of businesses like reputationdefender.com - specialists who apply basic SEO to unwind bad publicity by getting better commentary ranked higher than existing bad news. Again an example of SEO justice. Or SEO justice undoing previous SEO.

Last year, our good buddy Tony paid a refundable deposit of $10,000 on a property he was considering buying. The short story is that although he was due a refund, the real estate entity refused to pay. We posted a site (no longer live) optimized for the perps trademark with the whole story spelled out in detail (vetted by our attorney). The site also solicited other victims to post their complaints. Tony got his money back within 2 weeks - just before the company declared bankruptcy.

Bob's 91 year old father in law walked into Citizens Bank in Wolfeboro NH with his nest egg, with the intent of buying very safe certificates of deposits (CDs), but was talked into a front loaded bond fund at a time of rising interest rates. Although an obvious case of unethical banking, restoring the funds was not forthcoming. After getting push back on calls to the bank, Bob posted a single page detailing the story. Money was returned within the week.

You can see where this is going. Don't let unscrupulous businesses or persons go without accountability. It's your duty as a powerful propagandist to warn the public at large of the bad citizens among us, whether you expose unethical institutions, political malfeasance, or clients who default on payments. Using a lawyer to vet your story can save you from being forced to change or remove it - you always want the law on YOUR side.

For those of us focused on enterprise optimization, this is a snap - very low level SEO skills are needed. And if you control an enterprise, you can probably push significant PR advantage to your posts and get them ranked very high, very fast. Use geo advantages and multiple documents to get as many instances into the results as high as possible. Since it's so easy I recommend you do at least 20 pages and spread it out around the names of the parties, corporations, and the significant terms surrounding the issue.

And we know this works. We have tons of great anecdotes where wrongs were righted simply because of push back from the search. Once you see how effective and easy it is to rank most of these kinds of posts, you'll never let the bad guys get away with anything.

Please send us your stories - we'll share them with our enterprise community. We know from our chats that this is a very popular topic - blog posts coming.

Everyone understands the need for recourse, sometimes known as revenge. We are often placed in an environment in which the only other real world alternative to your injustice is just not going to cut it. When that happens to you, I recommend that you use your optimization skills to set things right.

(added 2 February 2010)
Compelling, real world example of creating a consequence for a deadbeat is here in 3 parts: 1 | 2 | 3

Using Search For Recourse

- Pick your fights - Don't bother unless the issue is significant - a major ethical violation, a lot of money, etc.

- Name names - Let your target know exactly what terms you're optimizing for. As we all know you're much more likely to rank for a brand in combination with a person's name.

- Publicize all corporate entities. This is often the weakest link. Bring investor & client nomenclature into your target terms - gets you both rank & leverage.

- Use a good number of pages, across multiple sites. I suggest 20, targeted at all the iterations. If you do it right, all 20 documents will rank appropriately.

- Use services like RipOffReport.com, which will automatically rank for the offenders company and name, and will create a permanent record on the web.

- Stay legal - don't defame. As long as you stick with the facts, you're fine. Run your copy by your lawyer just to be safe.

- Don't take your posts down without compensation for the time it took to create, post, and link.

OneInAmelia specializes in search forensics & CSS.

Optimizing Media : Audio & Video SEO

Since all seo of content is semantically derived, any content that is not text needs special attention. This goes for both video and audio, so it usually makes sense to present media as one concept apart from the normal semantic focus.

Both video and audio content are completely in accessible to search engine spiders without a transcript. This forces us to require a textual version on the same page - can be hidden from view as long as indexed and compliant.

Attention to tags is really important, as are all all textual cues, including titles (h1) and alt tags. It's also important to use the other title attributes for the links and other style containers if possible. You can deliver additional title impressions even in div, span, even p tags.

In both audio and video content, because there are much fewer numbers of documents competing for rank, it's much easier for those that actually pay attention to the details. Also, because most posters are not aware of the SEO impact of added semantics in the markup, you can usually outperform most other players.

Maintaining Compliance

This is the really the most important topic for this organization, given what we've just been through. The only real solution to this issue starts with centralized accountability for all live posts, combined with automation to prevent the creation of the most common redundancies in content, tags, and filenames.

Blog Promotion & Subdomains

Why you want a blog directly on the top level domain, and NOT the subdomain:

- Blog in a directory contributes to tld critical mass, whereas blog on subdomain contributes only to the subdomain critical mass

- Blog in a directory can share site-wide navigation, whereas blog on subdomain must be treated as a separate entity, with it's own nav and content set

- Blogs that get constant updates get indexed daily. This enables resident blogs to pass links to content/products that are new or current and get them indexed very quickly.

see other notes on subdomains

----

On-Page SEO

This probably has the most meaning to your content creators, because this is where their contribution can really impact ranks. Communicating the important on-page factors is the best contribution you can make - awareness is everything.

Start with with a discussion of semantic targets. Just understanding the difference between category level and long tail targets from a productivity point of view will be very instructive.

The really important details here have to do with exactly matching the words used on the page with the targets revealed by the research. Singulars and plurals of the same targets are separate competitions. Other iterations of the same words (order of the words) are also separate competitions.

I've found that this is a world where appearances are misleading. Most marketing teams will benefit from this discussion because this is their turf - a place where conversion rates and traffic are directly impacted by.

If you use the example of a target set like that for your fax terms, you can quickly make a point about the importance of long tail targeting. When they see terms like "fax," "email fax"

Replacing Nofollow Strategies For Managing Internal PR Distribution

* by: Bob Sakayama *

15 July 2009: The recently announced changes in nofollow require us to rethink the strategies for handling internal PR. This is a big topic for enterprise SEO, because once you have significant PR on your site, how it is conserved and passed internally can make a huge difference both in the ranks and the PR of your pages. Basically, you want to be able to enable PR to flow to your important pages, even as you restrict what pages get indexed.

The Old Way

Once, we blocked all links, especially from the homepage, to pages that did not contribute to rank using the nofollow attribute on the links, assuming that the PR was conserved for the do follow pages. Now that we know this only causes the PR to be discarded, we should no longer use nofollow on link attributes except to protect the site from outbound links being seen as paid.

We have been working on higher level strategies to replace what we thought we were doing with the nofollow attribute on links. The tools at our disposal that can address how PR is handled include the robots.txt file, on-page robots noindex tags, and the new link canonical tag. A quick review of these tools, and their impact on PR is in order.

Robots.txt

Some relevant information about robots.txt in this regard:

- blocks the page from indexing based on internal links
- disallowed pages still accrue PR
- pages are not crawled and so do not pass PR
- but pages may appear in the search results if external links point to them

The simplest strategies of blocking pages with robots.txt disallows still work in keeping the site from internally indexing pages and directories that create conflicts or redundancies. And while these simple blocking actions can be effective, the limitations of robots.txt become apparent once you begin to examine its impact on PR.

Generally this protocol enables us to globally disallow specific directories and pages from being indexed via internal links. Pages that are disallowed in robots.txt are usually permitted by Google to be manually deleted from its index. So this is one technique we can use to clean the index should pages appear in there that we want to remove. But the fact that the page is not crawled means that even though the page may accrue PR, it is not passed to other pages. So this is NOT a good replacement for nofollow strategies.

On-Page Robots noindex Meta Tag

Some relevant information about an on-page robots noindex tag:

- blocks indexing - period - page will not appear in the search results
- still accrues PR
- still passes PR (unless nofollow is part of the instruction)

The last 2 are the critical points. Because pages blocked by on-page robots noindex tags still pass PR (provided you don't also use the nofollow instruction), this looks like a potentially useful tool for controlling internal PR distribution.

Canonical Link Tag

Some relevant information about the link canonical tag:

- permits redundancies to exist without causing conflict
- aggregates PR from multiple urls to one specified url
- protects against improper indexing of tagged pages, paginated pages, etc.

While this tag enables the aggregation of PR (like a weaker 301 redirect without leaving the page), it is intended for pages that are the same or very similar. We have experiments running to test the association of unrelated pages - and they appear to show that the tag is respected even when the content is not similar. But we don't yet have enough information to recommend using it for anything other than dealing with redundancies.

At this time this does not appear to be part of a viable replacement strategy for nofollow.

****************

The Specific Problem:

The Policies page has no rank value, don't want it indexed, and the links to it used to be nofollowed. How do we handle now?

The Easy Way

- Continue to nofollow the link. (PR discarded, page not indexed unless via external link)

- Do follow the link, disallow the page in robots.txt. (PR is trapped on the page, which will not be indexed unless via external link)

- Do follow the link, use on-page robots noindex tag. (PR is passed through the page, page will not be indexed)

The easy way usually involves loss of PR either discarded by the nofollow, trapped on a disallowed page (because it is not crawled, the links do not pass the PR on), or lost incrementally (15%) via passing through the page.

The last solution is our current recommendation of the easy fixes, because it has the smallest PR loss. This is also an easy fix to automate across all previously nofollowed links/pages. NB: the robots meta tag cannot also carry a nofollow instruction, only noindex.

The Optimized Solution

The optimized model will show no PR loss. That means that links that used to be nofollowed cannot be normal href links, because there is no way to prevent them from wasting or discarding PR.

One more time. In the highly optimized model, these can no longer be normal href links.

The concept is incredibly simple: use a link the bots can't spider, and use noindex on the page, just in case someone links to it. Then you can keep the primary navigation structure, and should it receive some external PR from a natural inbound link, it will flow back into the site, even though the page itself will never be indexed.

We're not advocating any particular kind of non-indexable link, just showing the principle we want to apply.

What kind of links work? ... more needed here...

If you're concerned about tactics that require using links that are not indexable, think back to the days of bad SEO when many dropdown navs relied on javascript that made the links invisible to search engines. Now we seen an ADVANTAGE for those kinds of links. Life is strange.

Just a side note: It's interesting that Google, by introducing us to the idea of PR sculpting, via their deceptive nofollow dead end, made some very savvy folks aware of the flaws in the PR model. And how these flaws make it obvious and necessary to hide stuff from Google in order to genuinely optimize for it.

Some of my peers are claiming that by just by using the nofollow attribute, your site is flagged as one influenced by an SEO, and will draw a higher level of scrutiny. Wow, that's profiling paranoia! But then again, ever since my experiments went up, my phone makes strange clicking sounds...

Resource Description Framework Is RDF A Game Changer?

by: Bob Sakayama

18 September 2009: If you're running a web commerce enterprise, you need to read a little about this development. This could really be a game changer if the experiment Google is running goes as planned. While technical sounding, what with the acronyms and all, this can seem intimidating and overly complex. But most webmasters and web marketers will already have familiarity with some of these techniques just by being around html, xml, or css. We're talking about optimization at the most granular level, providing more details after you've gotten someone's attention. This can be the difference between a conversion and a bounce.

Resource Description Framework (RDF and RDFa) is the term applied to the organization of data prepared specifically for the search. The standards set by RDF/RDFa enable the communication of granular, specific information, embedded within web pages and xml feeds to be extracted via search.

The terms "microformats" refers to the specific tagging protocols that can be used within html or xml that specify the searchable pieces. These tags specify a class that is used to communicate this information directly to the engine. This enables us to embed product information, like brand, category, description, price, etc. within the context of an html page, using existing css instructions like class. Similar standards apply to xml feeds.

One of the most important applications can be the use of these formats within web commerce, where traffic is directed to pages fulfilling a search imperative. And the document can be anything - a review, an article, a product page, an affiliate comparison, etc. The point being that html can now be used to contribute knowledge to the searcher via the microformat tags.

And the use of these tags can be easily brought onto the page via automation, since this already is supplying details as data points. So converting existing documents to microformat specs is not even difficult. And the impact could be huge - someone reading the review of your product could easily find any specific detail, like the price, or special purpose, or brand, or color, etc. because the information is flagged and easily recognized by the tags.

For web commerce, this is a huge leap forward if this becomes universally accepted. it means that the information most people make decisions on, are always available on any site using microformats. Google is currently experimenting with it - rather seriously, if you consider they've even built tools to let you test your rich snippets (snippets that have microtags), to see how your site may appear in their search.

It's worth experimenting with this right now, because if/when this goes mainstream, the early adopters will have the lead. We're seeing evidence that it already has traction, especially in getting information from product reviews and product descriptions.

Microformats in product info
http://www.google.com/support/webmasters/bin/answer.py?answer=146750

Microformats in reviews
http://www.google.com/support/webmasters/bin/answer.py?answer=146645

More on rich snippets
http://knol.google.com/k/google-rich-snippets/google-rich-snippets/32la2chf8l79m/1#
http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=99170

Rich snippets testing tool
http://www.google.com/webmasters/tools/richsnippets

Background on Microformats
http://microformats.org/about/

Technicals
http://www.w3.org/TR/xhtml-rdfa-primer/

Sign up form: Let Google know you're using rich snippets
http://www.google.com/support/webmasters/bin/request.py?contact_type=rich_snippets_feedback

Should We Continue To Attempt To Manage Internal PR Flow Using The Nofollow Attribute On Links?

* by: Bob Sakayama *

4 June 2009 : According to statements made by Matt Cutts yesterday at the SMX Conference, the world of high level seo may have just changed dramatically for some players. If you are using an aggressive nofollow approach to sculpt internal PR distribution, your site could be negatively impacted by this change.

The weeks leading up to this post were marked by observed changes in ranks impacting a large number of sites. Traffic losses on the order of 15%-30% were observed in some cases in addition to a major reordering on target terms, especially long tail terms in some markets. The widespread nature of these observations clearly indicate something major just happened. This usually suggests an algorithm change, or a change in the protocols governing how sites are getting indexed. It must also be said that we observed a similar traffic loss in the stats when Google was down for a few hours on 26 April 2009 (also on 6-8 Jan 2009), but we don't think that was in play here, since there were no other reports of an outage.

For the past couple of years, Google has been encouraging webmasters to use the nofollow attribute on links pointing to pages that do not contribute to the search. At the enterprise level, these suggestions were taken very seriously, because it seemed as if Google was providing a way for us to 'sculpt' the PR flow across our sites, enabling us to limit the wasteful passing of PR to pages that do not contribute to the search. Here's an early quote from Matt Cutts that clearly indicates how nofollow was being promoted:

"What are some appropriate ways to use the nofollow tag? One good example is the home page of expedia.com. If you visit that page, you'll see that the "Sign in" link is nofollow'ed. That's a great use of the tag: Googlebot isn't going to know how to sign into expedia.com, so why waste that PageRank on a page that wouldn't benefit users or convert any new visitors? Likewise, the "My itineraries" link on expedia.com is nofollow'ed as well. That's another page that wouldn't really convert well or have any use except for signed in users, so the nofollow on Expedia's home page means that Google won't crawl those specific links."

The idea that you could "save" your PR for more valuable pages is clearly implied in this statement, and this was taken as fact by the seo community: You should preserve or 'sculpt' your on page PR to advantage pages that contribute to rank, and nofollow all that do not serve this purpose.

But yesterday, at SMX, Cutts made a staggering claim - that this technique no longer passes MORE PR to pages not blocked by the nofollow attribute. We're reading this to mean that if you are blocking bots from accessing pages, the PR saved from that effort is NO LONGER passed to the other pages. Basically, if you block pages, you're just throwing that PR away. This is a complete reversal of the previous policy, and one that will have a much larger impact on sites that use higher level optimization tactics - in other words this could harm sites who have been following Google's suggestions.

To address the flood of questions that will likely flow from this announcement, and because Google has never posted information providing real guidance on nofollow, they just posted some basic information in their webmasters area to explain what nofollow means:

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=96569
http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=33582

Noticeably lacking from these documents is guidance on the specific issue of how PR is distributed when the nofollow attribute is used. So this is not contributing useful knowledge to all those businesses who implemented Matt's previous suggestions. And although he made the announcement at SMX, the WebProNews video just released at SMX includes a discussion by Cutts on nofollow but he makes no mention of the change. So as usual, we're left somewhat in limbo - no documentation, only verbal claims by Google's spokesperson that contradict a previous verbal claim.

Who Gets Hurt

We doubt that penalties will ensue from the overuse of nofollow, but if it is true that the PR is no longer "saved" for followed links, then those sites that are making heavy use of this technique may be harmed, as the PR connected to these nofollow'ed links is now discarded. And if your site was using this technique in the extreme - if all your links were blocked except one or two, you may be seeing a huge rank loss on your big terms, and because all your other links were intentionally blocked from indexing, you could be in a position where you have no productive ranks.

But most sites that have structural strength are not really dependent on PR sculpting and will probably not experience any issues with their ranks. Sites that are using semantically organized power centers, or are structured so that PR is groomed across related content, should not see any consequence of this new change unless the numbers of nofollow'ed links are very large with respect to the total number of links.

The real annoyance is that, once again, Google is changing the rules during the game, potentially harming sites whose intention was only to follow best practices. More information clearly needs to be forthcoming, especially from Google. This post is intended as a heads up for our clients and other power users.

And if you're using nofollow, a higher level strategy readjustment is in order. We had been advising clients to consider blocking all internal links, except for those that specifically contribute to rank, in an effort to conserve valuable PR. If this change is real, and we think it is, that strategy no longer applies. We're not recommending that everyone remove all their nofollows, but unless you're selling links, there is no longer any incentive to apply a nofollow strategy broadly across sites.

The real question for this moment is, "Why did this change occur?" We suspect 2 reasons:
1) Because of the ability to sculpt internal PR the nofollow attribute was being used (gamed) to advantage sites that were blocking all but their biggest targets. Many businesses could thrive by ranking one term in the top 5, even if it meant holding no other rank.
2) Widespread use of nofollow on all outbound links was becoming a default standard, and the resulting consequence on the natural search was that no one wanted to link out to anyone else for fear of either losing their PR, or appearing to be selling links. This is clearly not good for the links-as-votes model that Google uses. So by changing the consequences of using the nofollow attribute, Google may expect to see a greater willingness to share links, perhaps reversing the existing trend. We'll see how that plays out.

The "Xeeneexee" Link Authority Experiments

We have been enthusiastic adapters of the nofollow techniques starting in late 2007, as a result of several experiments that were designed to test the claims put forth by Google. The one below involves 3 PR3 sites with varying numbers of active links pointing to the target page. Based on Cutts' original claim we were able to see that the nofollow attribute did indeed preserve PR. The experiment is still live and we should expect to see some reordering in the ranks as a result of this recent change. We're not seeing any proof yet of this latest claim, but this experiment should either confirm or refute what was just made public once enough time passes:

http://www.bobseo.com/aaa/link-authority-071231.php?password=seo

Foreign Language Sites

When the enterprise grows to have a global reach, exactly how the overall web presence is managed becomes a concern for search compliance. Lack of attention to this can result in domain level redundancies triggered by multiple sites sharing common content, functionality, and other assets. Before you propagate foreign language sites make sure the consequences are understood from a compliance viewpoint. Basically you want to reduce the impact of those sites on the mother ship in the primary search, while enabling each site in each country to rank within their geo area, where it is much easier to attain high ranks.

One Site Per Language

You CANNOT clone your US site in another English speaking country, and expect to rank both. This can eventually result in a penalty on the most valuable property, since the second English language site will be seen as an attempt to spam the system with duplicate content for a ranking edge. Additional English language sites require UNIQUE content, AND no robot detectable connection to the main enterprise. Remember, Google has a guideline requiring each enterprise to be represented by only ONE website. Of course you can have more, but they must be implemented compliantly, meaning that ownership of both cannot be detected by a visiting robot.

Foreign Language Sites : Their Edge

Because of the semantic nature of the search, different languages keep sites with the same, but translated content, from being redundant with one another. Of course, all other compliance requirements are in effect, so this site can rank, on its own, in its geo segmented market. Ranking sites in foreign versions of Google is much easier, and large sites have a distinct advantage.

Foreign Language Sites : Common Problems

Assuming your translation is not an issue, the most common problems faced by foreign language sites relate to their autonomy.

It is not recommended that you share web assets with any other site in the enterprise, unless natural search ranks don't matter. This is a very common problem, because the developers of your sites are looking for efficiencies. They will often make one version, say a form, post it in once place, and the share it across all the sites by referencing the one file. Sharing assets does save work, and managers will always agree to the most efficient allocation of resources. But there are search consequences to implementations that reveal ownership relationships. Especially if the enterprise oversees multiple sites with valuable overlapping target terms. You can't be seen to have more than one horse in each horse race. And sharing a file from one location across all the foreign sites is telling Google something you want to keep to yourself. This is one very good example of how a compliance mandate can save the enterprise a huge rank loss headache.

Within one organization, the control of the foreign language sites may be handled locally or centrally. The latter is obviously easier to oversee for compliance. Locally managed sites that tap shared web assets need to be vetted for compliance from a central authority for reasons stated above. If the foreign sites are separate domains, or subdomains, make them as completely autonomous as possible. Let there be no robot detectable connection that reveals ownership.

And if you oversee an enterprise with many foreign language sites, you need to consider how the interconnectivity of your sites is impacting the ability of each site to rank. If you're looking at interlinked sites, and poor ranks, you might want to conduct a compliance review.