Live Search Webmaster Center Blog

New Webmaster Community in Bing

rickdej — Mon, 01 Jun 2009 15:01:25 GMT

As you may have noticed, Bing is live today as a preview. The site will be fully deployed by Wednesday. Be sure to go to www.bing.com and check out for yourself all of the new features and functionality of the new search engine. It’s pretty cool stuff.

We’re also excited to welcome you today to the new home of the Webmaster Center blog and forums, as part of the new Bing Community site. Welcome! The old URLs are now automatically redirecting visitors to this new site, but be sure to update your Favorites listing in your browser to the new site.

Also note that we’re working on a series of new blog posts that explains what the release of Bing means to webmasters and publishers, so stay tuned. As you can tell, we’re excited about Bing and ready to talk about it. Keep a look out for plenty of updates to come.

Bing!

-- Rick DeJarnette, Bing Webmaster Center

Announcing Bing, Microsoft’s new search engine

rickdej — Thu, 28 May 2009 15:51:35 GMT

As you may have seen, Steve Ballmer unveiled Bing, Microsoft’s new search product today. Bing will begin rolling out internally to Microsoft employees today and will be unveiled to the public in the coming days. For more information on what Bing is all about, take a look at today’s post on the main Bing blog.

So what does Bing mean to you as a webmaster? All good things; including an improved Webmaster Center with access to the tools you need to help drive more traffic and enhance your site. We have a few more updates up our sleeve including a brand new community site for all things Bing, which you will hear more about in the coming days. In the meantime, visit http://www.decisionengine.com and follow Bing on Twitter (@Bing) for all the latest news.

We’ll posting additional blog articles and other information for webmasters in the coming days and weeks detailing how to get the most out of this new search experience. We think you’ll like it!

-- Rick DeJarnette, Live Search Webmaster Center

Are you content with your content? (SEM 101)

rickdej — Wed, 27 May 2009 18:13:07 GMT

Let’s just say it. Content is king. There. It’s done. That’s everything you need to know about improving page rank, right? Well…

Content may be king, but as in modern monarchies, the king may not be the sole authority. Parliaments now rule, and they are made up of many constituent parts of society. And while the quality of your site’s content can have a leading role in helping you optimize your site’s organic ranking in the search engine results pages (SERPs), it’s a multitude of constituent parts of your webpages that constitute the whole process of search engine optimization (SEO). But in this post, we’re talking about the king. King Content.

If content is king, how do you optimize it? How do you make it better? Well, there are many ways, both easy and difficult, to do this.

Fix simple problems

First of all, clean up any errors. Run a spell checker. You’re not doing yourself any favors if you misspell your keywords in your headers and page links. (Yes, a strategic typo might helpfully anticipate an end user’s error and thus be a good keyword to target, but save that for use in the meta tag using the name="keyword" parameter.) You want your visible body text to be clean and free of errors.

If you’re simply not a good writer (and let me tell you, writin’ good ain’t easy!), look for the assistance of someone qualified to help. If you want your website to create conversions (where you “convert” visitors into customers), consider investing in the services of a professional editor for a thorough review of your site’s text. While you might get a 12-year-old kid to work for free to fix your iTunes connection, you’ll want more qualified help for this. Don’t pinch your pennies here -- this is valuable work.

Next, make sure the content on your pages include clear expressions of your site’s goals (what is the conversion desired?), thorough descriptions of the products or services you offer, and other pertinent and persuasive information your customers need to know. In fact, pretend you are a potential customer who’s a first time visitor to your site. Does the content you provide make a complete and compelling case for them to covert? If not, make it so.

Most importantly, show off your expertise in your subject area. Write interesting and informative content about what you know. This sort of content, especially if it is regularly refreshed, draws new and repeat visitors to your site, and often helps you attain a critical goal — the qualified, inbound link!

A sample scenario

So I can imagine what some folks are thinking. “What more can I say about my business?” Let’s look at a scenario where you expand your thinking on what you can do to boost your content, and thereby your site’s value.

Suppose I own a business near Redmond, WA, called Rick’s Pretend B&B (hey, a guy can dream, right?). It’s an old, converted mansion with eight bedrooms, an elegant dining room, and nice mountain views.

So what content does my site include? Well, what would potential customers want to know about my place? They’ll want detailed room descriptions, including photos. They’ll want descriptions of the breakfasts (It’s a bed AND breakfast, after all). They’ll want to know about the amenities of the facility, room rates and availability, and direction on how to get to my place. They’ll probably also want to be able to book a reservation online.

OK, so I have all that on my pretend website for Rick’s Pretend B&B in Redmond, WA. Have I covered all the angles? No, I’ve only just done the bare minimum. Consider this: Why would people want to come to my place in my town instead of going somewhere else? They might be thinking about a visit to somewhere in the Pacific Northwest, and are browsing for tempting places to stay, but haven’t decided on which area yet. I can do much more than talk about the basics of my B&B. I can add content that describes what activities visitors can do in my area and which attractions, both local and regional, they won’t want to miss. That might help those undecideds choose to visit Redmond and thus stay at my place.

What more can I do? That’s everything, right? Well, not so fast. Let’s say I offer concierge services to my guests. I need to write about it! I should dedicate a page to that, where I mention how I can arrange romantic and fun activities, such as a hot air balloon ride and visits to local day spas, upscale restaurants, fine wineries, and a nice, local outdoor concert venue. I could write about setting up guest participation in these activities, tips for getting the most out of these places, and how I can make it all happen. I can even promote those local businesses on my site with outbound links. Those local businesses, once informed of my link to them, might choose to reciprocate with a link back to my site.

What more can I say? Well, what alternative ways do I use my business facility? Let’s say the building is an old Edwardian mansion with a grand entrance and a couple of large, ground floor rooms not used as bedrooms. Perhaps I occasionally rent those Wi-Fi-enabled rooms as meeting rooms for day-long business retreats or to wedding parties. I can even arrange for catering and custom floral arrangements. Adding new content about those additional, secondary business services will enhance my site.

But what will really make a difference, content-wise, is creating original, expert content on my site relevant to my business. I can create a page on advice and tips for organizing and planning those events. I can post a separate article on how to set up a catering plan for successful, carefree receptions! And once I really become an expert in this field, I can start a blog on my website about those areas of my expertise. Or I can even host a forum to build an online community where folks come to get their questions answered. By building my own little hub of expertise centered around my business, I make my site a destination place on the Web.

Self promotion time!

To reap the benefits of all my content creation work, I’ll need to let folks know that it exists. I can get the word out, thereby promoting myself and my website, by participating in industry-related online forums, getting media coverage through press releases, and writing articles for publications. All of those should, when relevant, link back to the content on my site.

If you do all of this content work for your site, you’ll find that as your reputation grows in your industry and community, your site will be deemed an authority site for your specific subject matter. Your fellow, SEO-savvy webmasters, who want to link to authority sites to help their customers (and their own SEO efforts), will gladly link to pages on your site.

The goal for all of this is the organic, relevant, inbound link. Good content begets fans, and fans link to their favorite places on the Web from their websites. Those high-quality, inbound links to your site are vital for improving your site’s rank in the SERPs.

Now, I admit, many of these content ideas will take a lot of work to implement and even more work to sustain. If I am busy running my pretend B&B, where is the time I’ll need for creating all of this additional content? Well, good content is not easy to get. That’s part of what separates the good from the great in search engine indexes.

The greats pay attention to their website content. They build it over time, keeping it fresh and insightful. They know that new, expert content will draw new, and more importantly, repeat visitors to their authority sites. And they know that authority sites are rewarded with higher page ranking in the SERPs. If creating great content wasn’t hard, it wouldn’t be so valuable. Think of it as a vital part of building your business.

No cheating, please

Content is king when it is well-written, relevant, and expert in nature, thus worthy of an outbound link from another website. Start your website’s reign in your industry by thinking what how you can add more relevant content to your site to attract new visitors and inbound links. But one word of caution: just like in high school, plagiarism is still an honor code violation, and search engines are like your strict English Composition teacher. They know if you’re copying content from elsewhere, and there can be penalties for cheating. Either develop your own content or give credit where credit is due by linking back to the original (after all, you’d want that link back for your original content!).

If you have any questions, comments, or suggestions, feel free to post them in our SEM forum. See you soon…

-- Rick DeJarnette, Live Search Webmaster Center

Live Search is Listening

Brett Yount — Thu, 21 May 2009 16:59:00 GMT

In a blog post last month, we identified the preferred method for providing feedback on any Live Search crawler issues you may have experienced. We would like to thank everyone for taking the time to provide us with this feedback and let you know what we have done on our end. But first, a little background.

Our crawler, MSNBot, performs many different functions. We have blogged about our cloaking detection in the past, so I’m not giving away any secrets by saying that we are actively looking to identify and weed out spammers who are using cloaking in malicious ways. We’ve also introduced a new feed crawling function that will help provide fresh results. In addition to this, we are also introducing a new 2.0 version of our crawler, MSNBot, which has various other new functions as well as upgrades on some of the old functions.

So, what I’m getting at is, we are busy adding upgraded and updated technologies to provide better search results. That’s good news for you!

Unfortunately, things can and do go wrong from time to time. The initial complaints, that we were over-crawling some servers with our cloaking detector, was compounded by and also confused with the new release of our feed crawler that was also overzealous in its attempt to crawl and provide up-to-the-minute results. However, we have taken all of the feedback you have provided and made some improvements.

We have modified the cloaking detector. Using the valuable feedback we received regarding the feed crawling issues, we proactively released a patch late last week that should significantly reduce the number of requests to a more acceptable rate.

What you can do

Help us discover the content changes on your site. You can do this via sitemaps and various meta properties per link or via RSS link to notify us about very important content. To prevent us from having to monitor lots of feeds often, we recommend aggregating content change onto a few feeds; adding the name, “Aggregate” somewhere in the feed name. We also suggest referring to them in robots.txt and your sitemap—both of which will help us detect them and their use.

Despite releasing a patch for our feed crawler, not all sites are the same, so it is a challenge to gauge a feed crawl-rate that is considered reasonable for all sites. If you believe we are still crawling more than necessary, an alternative option would be setting a crawl-delay. We would urge caution while setting the crawl-delay times as they can severely hamper our ability to crawl your site. As an example:

A crawl-delay: 5 means we wait 5 seconds per page request. That means if you have a site with 100 pages, it will take us 8.3 minutes to crawl your entire site. A site with 1,000 pages will take 83 minutes. As you can see, this can severely hamper our ability to index fresh content from your site.

If you are using crawl-delays and we don’t seem to be honoring those directives, please do not block us. Instead, let us know about these issues and we will work together to find out why we aren’t honoring the crawl-delay directive.

For all of these issues, please send us feedback by using the webmaster feedback form or by posting a message in our crawling/indexing forum. Thank you once again. We look forward to working with you in the future!

--Brett Yount, Live Search Webmaster Center

Put your keywords where the emphasis is (SEM 101)

rickdej — Wed, 20 May 2009 19:09:23 GMT

In my previous post on developing your keyword list, we discussed techniques and considerations for developing a list of relevant keywords and key phrases for the pages on your website that are highly specific and (hopefully) not overly competitive. All of this is effort is geared toward making your website stand out from the competition in search engine results pages (SERPs).

But developing a list of great keywords is just the start. You now need to use them strategically in your web pages (especially the home page). The manner in which you implement your keywords can make the difference between mediocre success and optimal success. And I am all for optimal!

Not all words are equal

When search engine web crawlers (aka robots, or more simply, bots) index your site’s content, the words you use help the search engine identify the theme of your site on a per-page basis. But not all words are equal. There are ways to prioritize the importance of some words over others. And by knowing what a search engine does to identify that priority, you can “speak” to it so that the search engine reflects your priorities as its own.

First and foremost, keep in mind that the bot crawls your site in an effort to help classify its content from an end user perspective. After all, our efforts are aimed toward helping end users get highly relevant and comprehensive search results. If the content on your home page is written in an awkward, choppy, keyword-stuffed manner that would not read well for a human visitor, the bot will pick up on that and your efforts will not achieve what you intend. It’s like the difference between a beginner cook and a master chef. Your keywords are the spices of your content, not your main ingredient. A good beef barley soup has black pepper in it, but no one wants to eat black pepper soup!

So the goal for you, the webmaster, is to flavor the natural content you have in your site with the keywords and key phrases you developed earlier. Keep your site’s content readable, ensuring the use of language on the page sounds natural, and use the keywords and phrases when it makes sense naturally. That said, if rewriting some passages of text so that you can use more of your keywords in a natural sounding form in strategic areas of your pages is doable, then it is usually worthwhile to do so.

Keyword placement priority

Here’s the key to using keywords:

Search engines look for keywords and key phrases to be in spots where writers use them to emphasize key points. Put another way, words used in certain places automatically carry a higher prioritized value. For example, words used in page titles are considered definitive for assessing the contents of that page. So because the title defines the page content, those words carry more weight than do words found in body text. In fact, body text is the least definitive use of keywords, because it is the most common text. Words that describe the body text as a whole or even other pages as a whole naturally carry the weight of those entire pages. Writers of technical books know the power of words that define entire passages (such as page titles, chapter, and section headings), and that assumption is used by search engine bots as well.

So in a nutshell, written in the form of naturally spoken language, use your keywords in the following areas for the greatest impact:

Page title (inside the <title> tags)
Body text headers (as used within tags such as <h1> </h1>, <h2> </h2>, and so on)
Links to other pages (the link text between the <a> </a> tags)
Meta tags (specifically when used with the name="description" and the name="keyword" parameters)

Lastly, don’t mistakenly think using keywords in body text is a waste of time. Your body text content is indexed just like everything else, and in the effort to determine the site’s theme and the relevance of your content to keywords, your body text is definitely considered within that process. Including your keywords and key phrases in your body text simply reinforces your use of those terms used in the higher priority parts of the page. But without their use in those aforementioned areas, they won’t stand out as easily as definitive keywords for your site’s pages.

Too much of a good thing is not good

Remember the reference earlier to the black pepper soup? Well, like spices to a novice chef, keywords can definitely be overused by a webmaster. You need to use them strategically, not exclusively, for the most desirable effect. Some search engine optimization (SEO) practitioners have in the past, mistakenly thinking more of a good thing is always better, have tried keyword stuffing techniques, where keywords are repeated constantly, which is detrimental to the normal flow of the language. They have added very long meta tags with super long descriptions and laden with way, way too many keywords. Some have even resorted to adding hidden text and links (using fonts formatted with the same color as the background or using the smallest possible font size so as to render them ostensibly invisible to browsers, but still there for the bots to read in the code). That’s just not right.

Way back in the day when search engine technologies were just emerging, this might have been considered a viable, albeit simplistic, strategy. Today, however, such techniques are detected for what they are – attempts to fraudulently load up pages with keywords in an effort to artificially gain higher ranking in the SERPs. Just as black pepper soup would be unpalatable to a diner, so is keyword stuffing to a modern search engine. When the search engine index algorithms detect the use of these techniques, they will typically impose penalties on those sites to artificially lower their organic rank status in the SERPs or, if the fraudulent effort is egregious enough, purge the site from the search engine index altogether. And that unfortunate result is totally counter to the reason the effort was attempted, isn’t it?

As search engines work hard to serve up the most relevant listings in their organic SERPs, they look for legitimacy in the sites they rank most highly. The legitimate use of great keywords will go a very long way in getting where you want your site to be.

This is just the start of SEO techniques we’ll discuss in this column. All of the techniques we’ll discuss can be used on a site to optimize its organic rank position in the SERPs or to attempt to fraudulently increase its rank beyond what its content and design deserve. SEO often walks a fine line between what is acceptable and what is not. We’ll talk about penalties in more detail in later SEM 101 columns, but for those curious today, check out a past column called Getting out of the penalty box.

If you have any questions, comments, or suggestions, feel free to post them in our SEM forum. Later…

-- Rick DeJarnette, Live Search Webmaster Center

The key to picking the right keywords (SEM 101)

rickdej — Fri, 15 May 2009 18:31:50 GMT

Mark Twain once famously said, “The difference between the right word and the almost right word is the difference between lightning and a lightning bug.” Twain never saw a webpage or performed search engine optimization (SEO) in his life, but his assessment of the importance of using the right words is, shall we say, illuminated.

So why are words important in terms of SEO? Because users type them into search engines to try to find you! That’s what separates regular body text words from keywords. Focusing on the wrong keywords on your website may, at best, bring visitors who don’t see the relevance of and aren’t interested in what your site has to offer. At worst, they’ll bring no visitors at all because your site will rank so poorly in the search engine results pages (SERPs).

Words and phrases

Recall that in my last post, I discussed defining your intended primary audience. I encourage you to now think about the words those folks would use to describe the theme of your website. I want you to also think about your goals for your website. Most often, the goal is a conversion of some sort, be it a sale, a download, a subscription, or some action you want the visitor to take so they “convert” to being a customer.

The best words to use for keywords are often not individual words, but short phrases. Brand names as individual keywords are probably good bets if you own that brand. But if you don’t own such a property, then individual words are likely to be too generic. You need to distinguish your site from the masses. By combining generic but relevant keywords into key phrases, you add a significant level of specificity that will help users find you.

Let the brainstorming begin

So how do you begin? Simple. Just begin. Think, but don’t edit. Brainstorm. Write down all words and phrases you think of (sure, you can use your computer!). Include your brand, descriptions of your products, services, or features, and your URL. When you’ve exhausted all your ideas, get ready to continue. You’re not done yet. You’ve only just started.

Now think like your customers, not like you. What terms would they use to describe your business or service? Write them down. Raid the thesaurus for synonyms. Note that plural forms of words are counted separately in the eyes of search engine indexes, so include them, too. Remember that individual words can often be too generic, so think of both single words and short phrases for greater specificity.

Next, talk to friends and relatives. How would they describe the theme of your site’s content? Is your physical location relevant? If your primary audience is local, add references to your location to your growing list as well. Are there any online forums that pertain to your site’s product or service offering? If so, check there for more prospective keywords and phrases. Consider common acronyms and abbreviations. Even write down typical typographical or spelling errors for these words as well. Harvest all of that into your collection.

Tool time

Lastly, augment all of that good data with professional keyword research tools. Microsoft offers a tool called adCenter Excel Add-in Keyword Research Tool for versions 2003 and 2007. (Note: You’ll need to set up an adCenter account before you can use the tool. Luckily, unlike most other online ad vendors, adCenter offers customer support over the phone with a real person — at no cost to you! — to help you get your account set up and running.) Both Google and Yahoo! offer their own keyword research tools. In addition, there are many third-party keyword research tools available, some for free, others for a fee.

The adCenter Excel Add-in Keyword Research Tool can do the following:

Scan your current website and extract the keywords that offer the highest confidence levels based on their current usage
Suggest new keywords based on user behavior or your existing keyword list
Provide:

Research data on top performing keywords
Performance data on the keywords you specify
Information on keyword usage based on geographic and demographic data

Note that the keyword tool is primarily designed to help users figure out which keywords to use with their Pay-Per-Click (PPC) advertising campaigns. However, the tool’s output is also extremely relevant to developing or revising a keyword list for your website as part of an SEO update. We’ll talk about the process of creating a PPC campaign in later posts.

To use the tool, I recommend adding your keywords (one word per line) to an empty Excel spreadsheet, listing them in column A. Select the words for which you want to see adCenter’s confidence level rating, click the Ad Intelligence tab, and then click the lower half of the Keyword Suggestion button on the toolbar, using both the Contained and Similarity tasks. You’ll get a list of additional suggested keywords and phrases that correspond to each of the keywords you selected. Use the ones that are relevant to your site’s theme and content to augment your burgeoning keyword list. For detailed information on how to take full advantage of the adCenter Excel Add-in Keyword Research Tool, see the Step-by-Step webinar tutorials.

Separate the wheat from the chaff

Once you have your master list from the brainstorming sessions and the keyword research tools, you then need to pare it down. Sort the words for relevance and specificity to your site. Lean your preferences toward highly specific keyword phrases over generic individual words; there’s likely too much competition for single words. Look for keywords that identify relevant niches in which you can stand out. Weed out the jargon to that only your customers will use, not that used specifically by industry insiders (unless they are your primary audience!). Note that each page of your site should have one or more unique keywords for themselves, but a carry-over theme of certain keywords will span the page boundaries across your site. Those cross-boundary keywords and phrases will define the theme of your website for search engines.

As part of the winnowing down process, review the keyword tools again for data on how prevalent your emerging favorite keywords and phrases are with searchers and how many other sites key on them. If the competition is too great for those keywords, you’ll still be lost in the crowd. And if searchers never use the keywords you choose for searches, no one will ever find you, even if you grab the top rank.

In the end, the keywords you pick should identify both the content and the thematic goals of your website. A list of no more than 15 or so keywords or phrases is good (and not every page needs so many, but every page containing valuable content needs at least a few good keywords).

Once you have your keywords and key phrases selected, you’re ready to begin strategically implementing them in your website. This way the search engines will see their optimized use on your site and increase your site’s relevance to those words. And because you wisely selected the right words, as Twain encouraged, you’re site will be properly illuminated for all interested searchers to see.

If you have any questions, comments, or suggestions, feel free to post them in our SEM forum. We’ll talk about strategic keyword implementation techniques next time. Later…

-- Rick DeJarnette, Live Search Webmaster Center

Who’s looking for you? (SEM 101)

rickdej — Tue, 12 May 2009 18:51:31 GMT

Quick note from the author: The closing teaser from my last post said the next post would cover building a toolbox. I’ve decided to take another tack on this and make references to tools as we go along in the series to keep the unfolding story as interesting, active, and relevant as possible. Sorry for the misdirection! – Rick

---------------------------------------------------------------------------------------

Before you can begin to effectively optimize and market your website for your audience, you need to know who that audience is.

Who are your website’s customers today? From which group can you effectively draw the most response, whether that be an online sale, a visit to your brick-and-mortar business, a subscription to your newsletter, or whatever it is you’re offering? That really depends upon who you are. Think about it for a moment.

Who are you looking for?

I can hear some folks now. “I want everyone to know about my site/business/service/fill-in-the-blank.” Sure. And I want to find a million dollars in my piggy bank. Every day. But is that realistic? (It certainly isn’t for me!) But you will likely find great success if you define your primary audience as the one with the highest potential for conversions (where the website visitor “converts” to a customer by buying/subscribing/downloading something, which fulfills the goal of your website). Spending your limited search engine marketing (SEM) budget on ill-defined or poorly selected audiences will only waste your money, limiting your return on investment. Consider who is the best potential target for your type of business, but also be sure to consider the caveats of your particular business.

For example, if you are a restaurant owner, your most effective advertising will most likely be to your local residents. Unless you just hired a new, Michelin-rated chef and are located in Napa, CA, New York City, on the Las Vegas strip, or other such major culinary tourist destination spot, the highest percentage of your nightly customer base are not likely to be out-of-towners. Your best bets are with locals.

How about the B&B owner? I’d guess that very few of your customers drive a few blocks away from their own home to stay overnight at your B&B, even if they are die-hard B&B patrons when they travel. As such, casting a wider net is needed to capture this likely audience, such as regional and, as success grows, eventually national, in scope, but not necessarily with folks across the globe. It might be true, though, that locals might like to refer their visiting in-laws there, so locals could be a secondary audience. Then again, if your B&B is exceptionally attractive and can accommodate private events such as wedding parties, off-site business meetings, and the like, then the local audience comes back into the picture as a potential primary audience.

How about an online retailer? Since you likely offer shipping, then limiting the scope to local or even regional folks is not necessary (unless the product has primarily a local/regional appeal, such as local sports team paraphernalia). This audience would likely be nationwide.

And for a highly specialized service designed around an unusual resource, such as organizing and leading Alaskan wilderness salmon fishing trip adventures, then you might find there is a niche, but global audience for your site.

You’ve no doubt noticed that the caveats to this abound, but this is all good. Now you’re thinking about the specific audience for your site based on what it is you offer.

Confirm your suspicions

You certainly know your business better than anyone else, but double-check your assumptions. To discover who’s actually visiting your site today, turn to the power of web analytics. Web analytics tools collect, analyze, and report on the data received from the Internet about your web site.

With web analytics tools, you can learn a great deal of information about who is accessing your site. You can typically learn how visitors come to your site (be it from search engines referrals, inbound links from other sites, or users directly typing your URL in their browser), where they are physically located (broken out by country, region, state, and even city), how many pages they view when they get there, how many of your visitors converted, and much more. You can get reports on all sorts of data collected about the way visitors use your site and its visibility on the Web.

Web analytics is done in two ways. Some tools are considered to be off-site analytics, and they analyze your site’s potential and measure its current visibility on the Web. Most analytics tools, however, are called on-site, and they measure the actual traffic through your site. Some of these tools use your web server’s access logs for the visitor data, but other on-site tools require some type of tracking mechanism be added to each page of your website so that the analytics engine can collect visitor data.

Web analytics tools are available through a variety of sources. Some web analytics tool providers charge fees, while other tools can be used free of charge. Find one you like and set up an account for your site. Note that analytics tools using the tracking mechanism of data collection need time to gather their data; they won’t offer much meaningful reporting initially. But that’s all the reason more to set it up ASAP! Once you have visitor data to analyze, you can confirm whether your assessments about your primary audience are borne out. We’ll talk more about exploring the utility of web analytics in later posts.

Help them find you

Evaluating your business knowledge and using web analytics tools is the process of identifying who is truly your best-bet, primary audience. And once done, you’ll be better equipped to make your site more discoverable to those folks. If you manage an Italian restaurant, do you really think your goal is to get the top rank for the search phrase “Italian restaurants”? It’s not realistic (unless you run a huge, nationally known restaurant franchise!). It’s also not really desirable. Who wants to spend the time addressing e-mails from out-of-towners asking pointless questions when they will likely never spend a dime at your tables? But ranking high for a more targeted search, such as “Italian restaurants Redmond WA,” might be a very good goal to pursue for your local patrons. After all, those local patrons are the ones who, by percentage, do the most to cover your monthly bills and payroll with their spending.

I can’t tell you how often I’ve seen websites for small, local companies who seem to assume everyone seeing their site already knows who they are, where they are located, and how to contact them. These sites are typically for brick-and-mortar businesses that have all but hidden where they are physically located, yet for all intents and purposes, their intended audience is strictly local. It’s as if they used the text from their local print advertising for their website (in fact, they probably did). How is that going to help them increase business through web search? How will the search engines even know to rank them with high relevance when a searcher looks for their business in their locality? They won’t, and that’s a lost conversion opportunity.

Alternatively, online retailers who focus on local information are also missing the point. If the primary audience wants to buy from you, they will most likely want the item shipped. Emphasis on your office location is likely of little relevance to the vast majority of them. Focus on what your audience needs to know, not on what you want to tell them. But to do that, you need to first know who they are.

What we’ve been dancing around and leading up to here is the concept of developing keywords. But to make those keywords work, you have to know who will be using them. Who’s looking for you? Help them find you!

If you have any questions, comments, or suggestions, feel free to post them in our SEM forum. Until next time...

-- Rick DeJarnette, Live Search Webmaster Center

Jane & Robot Meetup Wednesday @ MS, Mountain View

rickdej — Tue, 05 May 2009 17:39:46 GMT

Are you going to be in the Bay area tomorrow? Do you have a vested interest in search marketing? Know any web developers? Well then, make sure drop in to the Microsoft technology center in Mountain View, CA for a hearty discussion of search marketing hosted Microsoft, with guest speakers: Vanessa Fox, famed creator of Google’s Webmaster Tools, and Sean Suchter, previous VP of Yahoo.

Microsoft Campus in Mountain View

Event: Jane & Robot SEO Meetup

When: May 6, 2009 from 7:00 pm - 10:00 pm

Where: 1065 La Avenida, Mountain View,
California 94043

Bring all your search questions, and of course there will be plenty of food, drinks and opportunities for networking. Follow @janeandrobot on Twitter for event updates and last real time logistical information. See you there!

-- Nathan Buggia, Live Search Webmaster Center

Does search marketing really matter? (SEM 101)

rickdej — Fri, 01 May 2009 18:39:39 GMT

I have a good friend who creates websites for personal users and small non-profits. When I told him about my new project, blogging about search engine marketing (SEM) and search engine optimization (SEO), he wondered aloud, “Who in the world cares about that stuff? What does it matter? There are only so many Amazons in the world, anyway.”

I was initially surprised by his reaction. But his comments got me thinking – what is the point of SEM? Does it only apply to the big players in the online business world? Does this stuff really matter to anyone?

Of course, the answer isn’t as clear as those of you reading this blog might initially think. In reality, the answer is (as with so many things in life): it depends. It all comes down to the intention of the website.

What is SEM?

Let’s first define our terms. SEM encompasses all of the various ways site owners can use search engines to attract more traffic to their websites. SEM is all about helping users of search engines find your website. SEM is comprised of two major parts: SEO and Pay-Per-Click (PPC) advertising.

SEO techniques help a website reach its optimal rank in the “organic” (naturally relevant) section of search engine results pages (SERPs), which increases its visibility to searchers. SEO is fundamentally all about making a website more discoverable, accessible, and usable for visitors. If you consider the needs of your users during the design phase of your site, then you will automatically go a long way toward optimizing your site’s code, structure, and content for search engines.

One of the most basic users your site will host is the search engine web crawler or robot (often referred to as simply a “bot”). It primarily reads text-based content, just as the most basic web browsers do. When you consider and accommodate the lowest level of text browser in your site design, search bots will be able to crawl more of your site, which helps the search engines index more of your site, which then allows us to evaluate more of your site’s content relevance. And once a searcher uses keyword queries that match your site’s content, your site will then be included in those SERPs (assuming you have compelling content to offer). This is the essence of SEO.

PPC, on the other hand, is paid advertising on SERPs related to the keywords the searcher used, so the ads are (hopefully) just as relevant to the searcher as are the organic results. (Note that some ad distributors can place PPC ads in other, affiliated websites beyond SERPs.) Thus PPC also increases a site’s visibility.

However, there are big differences between these techniques. PPC is a temporary listing, only lasting for the duration of the ad campaign, and costs the sponsoring advertiser money each time a user clicks the ad (aka impression). Alternatively, changes made in organic rank achieved by SEO techniques are done in the permanent search result listings, which are provided at no cost to either the listed website owners or the searcher. (Of course, the ranking order of organic listings is constantly evolving due to many factors, such as new, relevant sites coming online, others going offline, tweaks to the algorithms search engines use for determining relevance, and when sites with great content implement SEO improvements).

Another difference between PPC and SEO is time for efficacy. As soon as a PPC ad campaign starts, searchers will see ad impressions. The benefits of SEO on organic search results, on the other hand, take time to see. While your SEO-driven changes are immediately seen as improvements by visitors to your site, search engines need to recrawl your site, reevaluate the changes, and reassess your site’s relevance and value in the SERPs. That takes time, so patience is a virtue when practicing SEO.

Does it matter?

Whether SEM matters depends upon the intentions a webmaster has for his or her website. Some folks put up websites for personal reasons, such as for posting photos of their latest family vacation, writing personal movie reviews, or to simply blog about their daily life’s activities (Facebook, anyone?). Maybe they’re learning a new technical skill and they simply want to exercise their new wings on a largely meaningless, initial web project. But honestly, who cares about these sites? There’s nothing in those sites for a visitor to buy, subscribe to, or download. That’s the process when you “convert” a visitor into a customer – a key concept for SEM.

But if your website’s reason for being is to promote something, such as a business selling a product or service, subscriptions to an e-mail-based newsletter, a software download, or just advocating a political/social cause, then you want conversions. You get them by attracting qualified visitors, who are already interested in and actively seeking the information you provide, to your website. That’s the role of the search engine in this process. In that case, SEM (and especially SEO) is extremely relevant. The thoughtful use of SEM here can have a profoundly positive impact in promoting awareness of your site to a targeted set of users.

Best of all, the value of well-executed SEM applies equally to locally-based promotions (such as a new restaurant in town) as well as those whose target audience is regional, national, and even global in scope. Simply put, SEM brings interested eyeballs to your website.

So sure, SEM is important to companies like Amazon.com. Amazon’s online business model relies upon customers coming to their site to make purchases. SEM not only helps Amazon sell products, it also actively sells their brand, so customers learn to automatically visit their site again for their next purchase.

Of course, the business goals of getting conversions and selling a brand can apply to your business just as much as Amazon. You may run a great local bed & breakfast that accepts online reservations, sell products in a niche market (like Ford Model T restoration parts and information), or write detailed white papers on Internet security as a consultant. Until your own brand’s name recognition is strong enough to be your industry’s own version of Amazon.com, you need to be found by your target audience. This is most effectively handled through search engine results, which are optimized by SEM.

It’s your question to answer

In the end, the big question that each webmaster must answer is, “Does it matter to me that people find my website through search?” That’s the key question that only you can answer. And if your answer is, “Yes, it matters to me!”, then SEM matters. You don’t need to be as big as Amazon to benefit from SEM. In fact, it’s the smaller sites that benefit the most, as they will get a chance to stand out in the light of day. The Amazons of the web-based world can otherwise cast pretty big shadows!

New SEM forum online

The new Webmaster Center forum for all issues SEM- and SEO-related is up and running. Be sure to check out Search Engine Marketing and Optimization. That will be your chance to exchange SEM ideas and strategies with fellow forum readers and our forum moderator, Brett Yount (I’ll poke my head in there from time to time as well). I’ll even collect suggested topics for future SEM 101 columns.

Coming up next: Getting started in site optimization by building a toolbox. Until next week…

-- Rick DeJarnette, Live Search Webmaster Center

Search Engine Marketing 101, a new blog series

rickdej — Tue, 28 Apr 2009 19:44:13 GMT

As a VP candidate once famously said, “Who am I? Why am I here?” I’m Rick DeJarnette, and I’ve been with the Live Search Webmaster Center team since fall, 2008, learning the ropes, helping with writing and editing work on both the tools Help system and the blog. I have a long and advanced technical writing background, focused on both IT Pro and end user topics. I’ve been involved with web technologies since the mid-1990s, and have always had a particular passion for helping people get more out of technology to improve their lives. I was recently given the opportunity to develop a new series for the blog, oriented toward search engine marketing (SEM) for novices, with a particular focus on search engine optimization (SEO) topics, and I immediately ran with the idea!

Audience

My goal for this series is to help folks who suddenly find themselves responsible for promoting their organization’s website. I’ve seen many visitors in the forums who ask basic questions, so I know there is an audience for this line of articles. I personally know many folks in my life outside work (there is such a thing, right?) who manage websites for their businesses but know little-to-nothing of SEM. Here are a couple of examples of my intended audience:

Small business owner. Think of someone like a long-time bed and breakfast owner, who knows her business inside and out. She understands that her business today is greatly dependent upon a successful online presence. Perhaps she once had decent web development skills from a prior career, so she understands basic web technologies. Most of her successful, local competitors have websites, but she can’t understand why she is not getting better value, higher ranking, or more business from her own site.
Business professional. Then there’s the guy who was initially hired to be a marketer/writer, but was then asked to improve the visibility of their corporate website. This employee knows basic HTML coding, and certainly knows how to write a good press release. However, knowledge of SEM had never previously been a requirement, but now success in his job requires this skill.

These sample personas actually represent everyone who delves into SEM for the first time. The particulars of this audience will be diverse in origin, but they all share the same issues and questions. What do I do? How do I start? What’s helpful and what’s not?

When people first look into SEM, they are usually overwhelmed by new concepts, new jargon, new technologies, and the fact that there is usually an assumed level of technical knowledge that they don’t have and lack the time to learn (they’ll often tell you they’re too busy trying to run their business to devote time to the website!). As a result, they either hire a consultant to “make it work,” or if they are DYIers, they spit out some basic HTML code and call it a day. But unless the person doing the work knows the right strategies and tactics, these efforts are often only marginally successful. Let’s change that.

Topics to be covered

I want to orient these folks and others in similar situations (that would be you, my dear reader!) to what SEM is and why it is important. I will show you the particular steps you‘ll want to take to improve your website’s structure and content, and discuss other technical details to make your site more usable and more compelling to users. As a result of these optimizations, your site will also become more visible to users by showing up earlier on relevant search engine results pages (SERPs). This increased visibility will help it draw more qualified visitors, whom you can convert, with the aid of strong, compelling content, into new and repeat customers.

I’ll also go into successful search engine marketing strategies, such as relevant Pay-Per-Click (PPC) advertising campaigns, that can bring an even higher number of highly qualified visitors to your site, which usually translates into more customers. Lastly, I’ll touch on the concerns about over-zealous optimization techniques and why, in the long run, they will be more detrimental than helpful.

New forum for SEM

To enhance this new blog series, our forum is has added a new section dedicated to SEM issues. This new forum, called Search Engine Marketing and Optimization, is intended to be a place for a community of users to exchange ideas and strategies on what works, what doesn’t, and even suggest topics of interest for future SEM 101 columns (although I already have a long list of topics to cover!). Be sure to check it out!

There’s a lot to talk about. Look for new SEM 101 blog posts on a regular basis. Coming up next: What is SEM and why does it matter? Until next time…

-- Rick DeJarnette, Live Search Webmaster Center

MSNbot Complaint Escalation Path

Brett Yount — Thu, 23 Apr 2009 19:46:00 GMT

The Live Search development team is busy making significant improvement to the quality and depth of our search index, and these changes often affect how our crawler, MSNBot, collects data. As we test new algorithms, we invite webmasters to contact us to give us feedback on these changes, especially if you have any concerns.

Live Search always encourages webmasters to use our product support escalation paths to help identify and resolve issues as quickly as possible. We thought this would be a good time for a quick reminder on how to send us your feedback. For specific questions and concerns about our bot behavior and crawl frequency, please do one of the following:

· Check the Live Search Webmaster Center’s Crawling/Indexing Feedback and Discussion forum. Check to see if your feedback has already been addressed in an existing thread. If not, post your feedback in a new thread. Please be sure to include all of the pertinent details surrounding your incident. This will help us investigate the incident from our side and assist in the process of providing a response to you.

If you don’t want your details posted in the forum, please post the basic information and identify the issue as a private matter. The Webmaster Center forums moderator will contact you directly for more information.

· Contact Live Search Support directly. You can use the Live Search Support form to send us your feedback. Select option Other from the form’s drop-down list and complete the rest of the form. Include as much detail as you can in the comments box on specific events, dates, and other information that will assist us in tracking down the details from our side.

Of course, all feedback is welcome, not just bot-related issues. Once we have received your feedback, we will investigate and get back to you as soon as possible. Please allow time for the technical investigation.

Thank you for your assistance in enabling us to address your issues, questions, and concerns with Live Search.

-- Nathan Buggia and Brett Yount, Live Search Webmaster Center

Getting out of the penalty box

rickdej — Thu, 19 Mar 2009 21:42:39 GMT

Life can be cruel. You work hard to create new and compelling content for your site. You’ve studied legitimate SEO techniques. Everything is going well and your site is getting decent page rank scores across the board. Then it hits. A search engine penalty comes out of nowhere and knocks your site out of the index. Talk about a bad day.

To be honest, most webmasters who get penalized know why it happened. They’ve used a myriad of overly aggressive (and often outright malicious) SEO techniques to try to egregiously manipulate their website’s position in search engine rankings. They often go to extraordinary lengths in an effort to game the system rather than do the legitimate work necessary to compete fairly for their page rank. We’re not concerned with those folks in this post. Instead, we’ll talk to the those folks who were working hard and trying to do what was right for their businesses by innocently but actively optimizing their sites who got dinged. It is frustrating, but it happens.

Because there are a small few who will go to such extremes to cheat, search engines have to go to great lengths to protect the integrity of their SERPs. So what happens if you got caught in the effort to stamp out malicious SEO when you are sure you are innocent of any overt crimes? Well, let’s walk through this.

Am I really in the penalty box?

Let’s confirm it. Go to Webmaster Center and log on (or sign in with a Windows Live ID for the first time and then add your site. Note you’ll need to have your site authenticated before you can use Webmaster Center tools).

Once signed in, click the site link you want to check on, and then click the link for the Summary tool. In the Site status section, does it say Blocked: Yes? In this case, No is good. Yes is bad.

How did I get here?

Something on your website was flagged as a problem, either via an automated process, by a review from a person on the Live Search team, or from a report we received from a third party (which was then reviewed by a person on our support team). Perhaps the flagged issue was that you used an optimization technique that you didn’t realize was considered problematic. Perhaps your site was hacked by a malicious third party. Perhaps you were unfortunately corralled by a newly implemented search algorithm that caught something that had been previously undetected. Job #1 right now is to figure out what happened so it can be resolved.

Since the Summary tool confirmed that your site is blocked from the index, the first thing to do is to confirm the issue is not related to malware. In Webmaster Center, check out both the Crawl Issues tool (review the Malware infected issue type) and the Outbound Links tool (select the Show only outbound links to malware option) for possible malware issues. If any malware issues were detected, clean them up immediately.

If malware’s not the issue, then it’s time to carefully review the Live Search Webmaster Center Guidelines. While all are important, pay special attention to the bullet items listed in the section titled, Techniques that might prevent your website from appearing in Live Search results. Posted there are the basic rules that can cause your site to be penalized.

Keep in mind that the basic guidelines between the three major search engines are largely compatible. To get clarity on the issue of search engine guidelines, it would be wise to review the guidelines for both Yahoo! and Google as well.

How did this happen?

Good guys who inadvertently get caught in the penalty box often end up there by one of these three common scenarios:

You noticed that your competitor’s website is ranking well (just where you want your site to be!), so you copied what they were doing with their site. However, you might not realize that the site you copied could be using disreputable technologies and techniques that have yet to be discovered by the search engines. If this is the case, eventually that site’s handy work will be discovered and penalized, and your site may soon be penalized as well. It’s even possible that the site you copied from is already being penalized for some flagged issue and you didn’t recognize that.
You hired a new SEO consultant and they implemented changes to your site with the explicit goal of improving your ranking in search engines rather than helping customers find what they need on your site. Unfortunately, you didn’t realize that some of the tactics they used violated the Live Search Webmaster Center Guidelines. If these tactics are deemed too aggressive, search engine penalties will likely ensue. (A good tip for hiring an SEO is that they should give you a full report of the work they are doing, and they should certify that their work doesn’t violate search engine guidelines.)
Your amazingly clever web engineers devised a creative solution to a technical challenge on your site. Unfortunately, you don’t know what they did, and often times a clever solution can mimic (if not outright be) malicious search engine manipulation. Non-standard tactics, such as cloaking, can raise red flags in some of our automated quality tests. Once discovered, such techniques will often trigger penalties in search engines.

In each of these scenarios, it pays to understand what the “experts” are having you do. If you blindly put your web business in the hands of someone who does not know what they are doing (or worse yet, knows full well and chooses to use malicious technologies and strategies in an attempt to fraudulently manipulate your ranking), your site will be discovered for what it is by the search engines and be penalized.

Where did I cross the line?

SEO work is not easy. Live Search knows that. But there can be a fine line between site optimization and over-optimization (and malicious optimization is just a small stretch beyond that). Keep in mind the following thoughts as you consider what was done to optimize your site:

Having lots of valid, relevant links is good. But using hidden links is not. Having tons of artificial (and irrelevant) links gotten through paid link farms or exchanges is worse. Linking out extensively to known web spam sites is worst of all.
Using on-page keywords on the body text of your pages is good. Using keywords in your page titles and in links is great. But massive, misleading keyword stuffing in meta tags is not. Keyword stuffing in hidden text is even worse.
Providing lots of good, expert content is great. Ripping off someone else’s content or plagiarizing commercial content as your own is not.
Having lots of pages of content is good. Having one set of content for users and a completely separate set of content dedicated solely to search engine bots (aka cloaking) is not.
Providing valuable content to your users is good. Providing malware or linking out to malware-infected pages is not.

Fix it!

Now that you’ve identified what likely caused the problem, go into your site’s source code and remove the offending issues. Note that the issues mentioned earlier are not only considered to be problems with Live Search. When your site does something that gets you penalized with one search engine, it’s only a matter of time before the other two are likely to act in the same manner. As such, the investment of your time to correct these issues is well worth the effort.

OK, I’ve fixed the problems. Now what?

Once site has been fixed, you can go to the Live Search Support page and request reinclusion into the Live Search index. Fill out the form completely, including the security code from the presented image. Select the Content Inclusion Request option, and be sure to include a clear explanation of what you have done to fix all previous conflicts with the Live Search guidelines. Once the form is completed, click Submit. A member of the Live Search support team will quickly review your request and schedule your site to be recrawled. If the crawler determines that issues have indeed been resolved, then your site will be added back into the index.

Soon you’ll be back and doing better than ever. Talk about a good day!

-- Rick DeJarnette, Live Search Webmaster Center

Clarifying turnaround times for content URL removals

jandrick — Wed, 11 Mar 2009 22:33:29 GMT

A few weeks ago, we wrote a post addressing the often-asked question of how to remove a URL from the Live Search index. Since then, we’ve received many questions about the process and wanted to provide more details about the turn-around time for processing these requests..

The reason for removal can affect the time needed to do the work. We strive to process each removal request within 48 hours of acceptance. However, some requests require investigation to identify the scope and source of the problem, and as a result, additional time might be needed.

For malware reports, our support team will typically need additional time for its investigation. Instead of just blocking the individual pages reported, we investigate the situation so we can aggregate all the sources behind the malware and block them all together.
For illegal content, our support team acts as quickly as possible when we receive a such a request. And as with malware, we don’t want to block just the page URLs reported. Instead, we want to investigate and find all the illegal content sources related to the reported URL so that we can block them all in bulk.

Because of the additional overhead of an investigation, we may need extra time to appropriately deal with blocking malicious and/or illegal content. Thanks for your understanding.

-- Rick DeJarnette, Live Search Webmaster Center

Going international: considerations for your global website

jandrick — Sat, 21 Feb 2009 00:56:39 GMT

One of the best parts of publishing online is that, on the Web, anyone can have a world-wide reach. But while being global is made easy on the Internet, ensuring that the content you produce will be found by the right audience can be a real challenge. Search engines can have trouble understanding geotargeting because of a few technical limitations. These include:

Search engines may not be crawling your site from the location of your customers.
Search engines may not execute JavaScript, which can break some targeting methods that rely on self-selection or JavaScript-based redirection.
There’s no standardized way to tell a search engine which region or language your content’s targeted for.
Content language usage can be misleading. For example, code comments and URLs could be written in English and hosted in a data center in Washington State, USA, while the main content could be written in Spanish, and the user generated content (UGC) comments at the bottom of the page could in French, German, Portuguese, and Pig Latin.
Top level domains may not indicate the intended audience. For example, http://ma.tt/, an English-based personal site or Orange.com, a French Telecom site hosted in France.
Some sites use redirection techniques that are unfriendly to search.

At Live Search, we attempt to overcome these and other challenges by examining the contents of a site, looking for indications that help us determine its intended audience. Sometimes it’s easy, for instance, when a site has a country code top level domain that matches the body text language of its intended audience. Other times, it’s more difficult. Live Search looks at a number of indicators to ascertain a site’s intended audience. The following are a few of the indicators we look at:

Country code top-level domain (ccTLD). For example, aw.ca specifically targets users in Canada.
Host server location, particularly for .com, .net, and .org.
Language of the body text on the page.
Locale of pages that link to your site.

While we do our best to read the indicators you give us, none of them alone are crystal-clear gauges of a site’s intended geographic interest, so we take them all into consideration. So, how do you ensure that we’re able to determine the intended audience for your content?

Some best practices

There is no single, fully-effective approach to architecting and localizing websites. Books could and have been written on this subject, but there are ways to do it that are friendlier to both your customers and to us. Consider the following recommendations:

Target the location

When writing the content for your page, are you using keywords that tell the end user what location your content is relevant to? If you’re a local business, be sure to include text with the telephone number with the area code and country codes, the physical address (if applicable), the city, state, and country name where you’re located. This’ll help both search engines and customers find you.

If you’re already doing this, one thing you may not have considered is to target additional location keywords that may also represent the location. For example, if your business is located in the Capital Hill area of Seattle, Washington, you’ll want to list both “Seattle” and “Capital Hill” because your customers will likely be searching for both. However, in this case, if you list “Capitol Hill” (note the spelling difference, because your customers might spell it incorrectly) and “Washington,” you might be mistakenly considered to be back east in the District of Columbia!

Be consistent in language

One problem we see from time to time is inconsistent language usage, especially where user generated content is concerned. These pages can trip up our detection, especially when other best practices aren’t being followed.

For example, from time to time, content in MSDN is required for external markets like France, but the page content hasn’t been converted to French or perhaps the content was user-generated, such as in the following image.

Ensure, wherever possible, that you’re speaking the same language in the title tags, description tags, and rest of your page. Consistency is key.

Create a hierarchy of language

When you’re architecting your site, we recommend grouping your localized content by the TLD, subdomain, or subfolder. Keep all of the content from a region or language grouped together in a single structure. The following are all examples of the good organizational structures for languages using the sample URL www.domain.com:

www.domain.ca
ca.domain.com
www.domain.com/ca-fr/

Don’t mix content intended for one market with the content of another. This is bad for search and can be a bad experience for the customer.

Note: One thing to consider in planning the hierarchy of a global site is how many URLs you produce. Having too many URLs for each market may dilute the whole relevance of your pages and site. Furthermore, you may not get customers to link to the most important pages on your site. See our part 1 post on large site optimization for more thoughts on this topic.

Common mistakes

Sometimes search engines struggle to deliver your content to the right audience due to problems within your (the publisher’s) control. Some of these common mistakes include:

Using a cookie to store the language setting

Some sites store the language setting as a preference in a cookie, but provide no navigational method for seeing the content for other markets. The problem with this approach is that the search engines don’t support cookies when crawling, so we never see anything but the default language. This can also create a less-than-optimal user experience. For instance, my friend Mishka may be reading a site here in the US but then switches to the German version of the same site. The site drops a cookie on her computer to note the change. If Mishka then emails a link to the site to her mother, who doesn’t know English, the site won’t find the German language cookie on her mother’s machine, so she’ll see an English page that she can’t read.

JavaScript

Some website owners spread localized content around a site without a clear path to get to the content. For example, let’s say your site enables customers to use a JavaScript control, like the one shown in the image below, to load localized content within the page, but it doesn’t change the URL for the different language content. In this case, because the crawler can’t execute JavaScript, it’ll never be able to reach anything except the default language content. This scenario also prevents a user from linking to the localized version on the content.

Scripting against HTTP_ACCEPT_LANGUAGE

The browser’s HTTP_ACCEPT_LANGUAGE header is passed to a website when the page requests it and informs the site about which language you prefer to receive the content in. If you’re using a script to detect this setting and change the content based on that setting, it’s easy to load all the languages in the website under the same URL. However, if that’s the only way the localized content is accessible, the crawler, which doesn’t pass values to this header, will only see the default language. Having a contextual URL per language is necessary for ensuring the content is crawled.

Market as a parameter

Search engines may sometimes be able to recognize a market setting in a query parameter if it’s using a common nomenclature (such as EN-US), but this is still neither optimal nor friendly for search. It’s better to follow the pattern for URL hierarchy we described above. The following are examples of both standard and non-standard market nomenclature that are used in URLs:

Standard nomenclature:

www.domain.com/default.aspx?mkt=ca-fr

Non-standard nomenclature:

www.domain.com/default.aspx?market=fran
www.domain.com/default.aspx?market=89372

Wrapping up

Search engines are always looking to improve how we detect the location and language of the intended audience for your site. At Live Search, we’re considering several possibilities, from meta tag standards to tools in Webmaster Center. But until those tools become available, by following the recommended practices and avoiding the common mistakes listed above, you can help us make those determinations today. As we make improvements in this space, we’ll continue to make announcements as well as seek your feedback and questions in our forums. Some additional great reads on the subject include:

Jeremiah Andrick, Program Manager, Live Search Webmaster Center

Partnering to help solve duplicate content issues

Nathan_Buggia — Thu, 12 Feb 2009 16:06:00 GMT

One of the most common challenges search engines run into when indexing a website is identifying and consolidating duplicate pages. Duplicates can occur when any given webpage has multiple URLs that point to it. For example:

URL	Description
http://mysite.com	A webmaster may consider this their authoritative or canonical URL for their homepage.
http://www.mysite.com	However, you can add 'www' to most websites and still get the same home page.
http://mysite.com/default.aspx	You can also often add the specific filename of the homepage and get the same page
http://mysite.com/default.aspx?promo=ABC	Many times websites use parameters to track things like where customers are coming from (in this case an offline promotion), or parameters that determine how the content on the page is formatted.

These four cases are just a few of the many possibilities. When you consider all the combinations of these, you could have more than 10 clone URLs for every page on your site. That means if there are 1 million pages on your site, we could possibly find 10 million or more cloned URLs pointing to them. Determining your canonical URL amongst all the duplicate clutter has been an onerous challenge for search engines as we all work to reduce cost and improve relevance.

To help solve this issue, Live Search has partnered with Google and Yahoo to support a new tag attribute that will help webmasters identify the single authoritative (or canonical) URL for a given page. The link tag defines a relationship between a document and an external resource. In this case, that resource is the canonical URL. The following is an example of the new link tag attribute for canonicalization:

<link rel="canonical" href="http://mysite.com"/>

A few notes about the implementation of the new attribute:

This tag will be interpreted as a hint by Live Search, not as a command. We'll evaluate this in the context of all the other information we know about the website and try and make the best determination of the canonical URL. This will help us handle any potential implementation errors or abuse of this tag.
You can use relative or absolute URLs in the “href” attribute of the link tag.
The page and the URL in the “href” attribute must be on the same domain. For example, if the page is found on “http://mysite.com/default.aspx”, and the ”href” attribute in the link tag points to “http://mysite2.com”, the tag will be invalid and ignored.
- However, the “href” attribute can point to a different subdomain. For example, if the page is found on “http://mysite.com/default.aspx” and the “href” attribute in the link tag points to “http://www.mysite.com”, the tag will be considered valid.
Live Search expects to implement support for this feature sometime in the near future.

While we expect this command will help us solve many of the more complex duplicate content issues, we still highly recommend that webmasters follow the existing best practices for normalizing their URLs through domain canonicalization and normalization of URL parameters. We’ll provide more details on the link tag after we’ve implemented full support in one of our upcoming releases. In the meantime, we look forward to hearing your feedback on the new tag.

-- Nathan Buggia, Live Search Webmaster Team