Search Engine Optimization - SEO

How to Generate Traffic, Fast Indexed and Get Good PageRank

2008-03-30T13:00:00.000-07:00

How to Generate Traffic, Fast Indexed and Get Good PageRank

Generating website traffic and increasing website page rank can lead to big business. The uses of social bookmarking are becoming increasingly popular to aid in this task. The two web applications that are the most used at the present moment are digg.com and del.icio.us.com.

The key to generating more traffic from digg and del.icio.us is to concentrate on delivering truly useful resources to the readers. If the people value the information in your article, they will be more likely to bookmark or digg what you have to say.

Bagi anda yg memiliki blog/situs pasti yg kita harapkan adalah terindex oleh search engine, mendapatkan traffic dan meningkatkan pagerank dari situs kita.
Artikel ini akan memberikan sedikit penjelasan mengenai beberapa cara meningkatkan semua tujuan diatas.

Cara tsb antara lain:
- Melampirkan artikel yg mengarah pada situs/blog anda melalui Social Bookmarking seperti digg, del.icio.us, reddit, mixx, stumbleupon dan article directory seperti ezine dll. Konsep mereka adalah sama yaitu mereka dibentuk buat anda untuk melampirkan artikel yg berguna bagi pembaca disitus mereka dan menjadikannya bahan favorit mereka (seperti bookmarking) serta mengarah pada situs atau blog anda.

- Bagi anda yg memiliki situs berbasis bahasa inggris mungkin calon visitor anda akan familiar dengan fasilitas ini dikarenakan pengunjung anda mungkin berasal dari luar/mengerti bahasa inggris. Dan kebanyakan org yg mengenal bahasa inggris biasanya sudah faham akan fasilitas ini.
Tapi jika pengunjung anda berasal dari indonesia dan tidak mengenal bahasa inggris, sedikit sekali dari mereka yg tahu dan memahami fungsi fasilitas ini. Sehingga kalaupun mereka mengklik fasilitas tsb, mereka jarang/tidak tahu memanfaatkan fasilitas ini. Bisa dikatakan terbuang sia-sia.
Walaupun tidak semua orang kita yg tidak mengetahui fasilitas ini. Tapi jumlahnya tidak terlalu signifikan. Dan kalaupun mereka tahu, mereka tidak terlalu berminat dgn fasilitas ini. Sehingga maksud dan tujuan artikel kita kurang 'tersalurkan'.

Disini anda harus kreatif, membuat artikel yg benar2 berbobot dan menarik, dan bukan tiruan (murni dari anda) karena jika artikel anda berbau spam maka para pembaca bisa 'bury' membakar artikel anda.

Only a few years ago reciprocal linking schemes were highly effective in increasing search engine rankings for keywords. In recent years, Google has devalued reciprocal links to combat automated reciprocal linking scripts. Google considers links to outside sites as votes for that site's importance. Automated scripts that add reciprocal links without human intervention run contrary to this idea, as a result one way inbound links have become the industry standard when undertaking any linking strategy.

Unfortunately the reverse is also true, if you link out to sites you are draining away some of your site's PageRank as well. As a result, the usage of reciprocal links is fundamentally less effective than one way inbound links.
How do you get one way links?

- Mendapatkan/Membuat One Way Link.
Beberapa tahun kemarin reciprocal link menjadi populer untuk meningkatkan rangking situs kita, sehingga maraklah bisnis di internet yang menjual bidang ini hingga banyak terjadi cara kerja terlarang seperti script otomatis yg menambah reciprocal link tanpa melibatkan manusia itu sendiri. Dan Google tidak menginginkan sama sekali! juga Search Engine lainnya.

Kenapa hal ini begitu penting dalam meningkatkan rangking situs kita.
Di Google, project research yg dibangun oleh Larry Page & Sergey Brin mengenai ini memiliki tujuan untuk menilai seberapa 'terpercaya' situs anda berdasarkan dari link situs lain yg terhubung dengan situs anda.
Hal inilah yg meningkatkan rangking situs kita di mata Google.

Pertanyaanya bagaimana kita mendapatkan One Way Link tesebut?
Beberapa cara yg bisa anda kerjakan antara lain:
1. Submit situs anda kedalam Directory Situs. Misalnya seperti DMOZ, Addme dll
Power dari situs directory ini memang tidak terlalu besar tetapi mereka sudah 'approved'/ disetujui oleh Google. (Artinya kalau situs anda disetujui dalam directory tsb maka situs anda pun di approved oleh search engine juga).
Akan tetapi beberapa tahun ini tidaklah mudah untuk situs kita disetujui masuk dalam direktory mereka. Bahkan dalam beberapa forum luar, seorang webmaster mendaftarkan situs mereka ke DMOZ dan hingga selama 1 tahun lebih belum disetujui oleh direktori tsb.
Ya..seperti DMOZ, cara kerja mereka murni dari manusia. Tenaga mereka adalah tenaga sukarela yg sudah disetujui situs tsb untuk mereview situs yg mendaftar dalam direktori mereka. Dan Google sangat suka cara ini. Murni dari manusia. Direktori ini tersedia gratis bagi siapapun tapi tidak menjamin situs anda akan termasuk dalam direktori mereka.

2. Cara lain dengan menyewa tempat seperti TextLinkBrokers.com , Text-Link-Ads.com dan lainnya. Tentu saja cara ini perlu biaya dan mungkin juga besar.

Untuk Directory Site yg gratis, ada beberapa tersedia di internet yg menyediakan fasilitas ini kalau anda ingin melampirkan blog/situs anda. salah satunya yg menjamin situs anda disetujui dan gratis adalah Payifm
Disana selain anda masuk dalam directori mereka juga memiliki fasilitas promosi situs gratis yg peringkatnya berdasarkan seberapa besar blog anda mereferensikan situs mereka kepada org lain. Dan mereka murni tidak ada "Ujung2nya Minta Uang" agar segera termasuk ke Direktory Site mereka dengan instant dengan alasan keuntungan menjadi 'Premium Member'. Semua yg ada dalam fasilitas mereka tsb murni Gratis dan Dijamin approved.

Silahkan klik Disini jika anda ingin mensubmitkan situs/blog anda kedalam directory mereka.

Semoga artikel kecil ini bermanfaat buat anda dalam memaksimalkan indexisasi, traffic dan pagerank situs anda walaupun masih ada banyak cara lain selain ini.
Cara-cara lain nanti kita bahas di lain halaman ya..kalau ada yg mau sharing, kasih komentar silahkan...

Search Engine Optimization (SEO) - the basics

2008-02-19T13:28:00.000-08:00

How to improve your Search Engine Rankings

Most people think of search engine optimization to improve their search engine rankings as being such a skilled task that, without putting a great deal of time and effort into it, it is simply beyond their capabilities. Wrong!

Yes, improving search engine rankings in competitive topic areas does require a good deal of knowledge and expertise and search engine optimization experts are needed, but most websites aren't in very competitive areas.

Many of them can achieve top rankings by applying just the search engine optimization basics - which can be learned in less than 30 minutes.
This article lays out the basics of search engine optimization. It can be well worthwhile trying them before paying an expert as, oftentimes, the basics are all that's needed.

NOTE: the SEO copywriting method (a.k.a. search engine optimization copywriting) applies these basics to a site's existing pages. It doesn't go into more advanced search engine optimization techniques that require more knowledge and expertise.

What is Search Engine Optimization?
Search engine optimization is the process of achieving top rankings in the search engines for a website's most relevant search terms. The most relevant search terms are the phrases that people are most likely to type into a search engine when looking for what the website has to offer. These are the search terms that it is essential to rank highly for, and these are the search terms that search engine optimization targets.

The first step is to choose the most suitable search terms for your site. Then allocate one or two of them to each suitable page within the site. One search term per page is preferable, but two per page is not so bad. Sometimes it is useful to split a largish page, that covers several closely related topics or several aspects of a topic, into two or more smaller pages so that a different search term can be targeted on each of them. Matching search terms to a page's content is essential.
NOTE: smaller pages are better than larger ones because it is easier to target a search term when there is less text on the page to dilute the focus.

Search Engine Optimization - the basics Link structure within the site
An obvious, but sometimes overlooked, aspect of search engine optimization is to make sure that search engine spiders can actually find (crawl) all of the site's pages. If they can't find them, they sure as hell won't get spidered and indexed, and no amount of search engine optimization on them will help.

Some points to note
Spiders can't see links that are accomplished by Javascript so, as far as search engines are concerned, they don't exist. Don't use them if you want spiders to follow your links. Google won't spider any URL that looks like it has a Session ID in it, so URLs with longish numbers in them must be avoided. These are usually dynamic URLs. Make sure that all pages link to at least one other page. Links to pages that don't link out are called "dangling links", and the reason to avoid them can be found Geogle's PageRank Explained archieve. It is good to structure the internal links so that targeted search terms are reinforced. E.g. organize the links so that a topic's sub-topic pages link to the topic page with the right link text (see below), and vice-versa.

Off-page elements

Link text
This is one of the two most important elements for good rankings. The link text can be on pages within the site or on other sites' pages. Either way, it is important. The target page's main search term should be included in the link text. When possible, don't use identical link text for every link that links to a page, but do include the target page's main search term in the link text.
Google attributes link text to the target page - as actually being on the target page, and it treats it's pseudo-presence as being an important element of the target page. Links carry even more weight if the text around them is concerned with the target page's topic and search term(s).

On-page elements

The Title tag
This is second of the two most important elements for good rankings. Make sure that the page's search term is contained in this tag, and place it as near to the front as is reasonable, whilst ensuring that it reads well. There's nothing wrong with placing the search term up front on its own, followed by a period; e.g. "Pagerank. Google's PageRank and how to make the most of it". The target search term is, of course, "PageRank". Obviously each page's Title tag should be different to the Title tags on the site's other pages.

The Description tag
Some search engines, such as Google, don't display the Description like they used to do but, even so, it should still be included in each page for those engines that do, and for the odd times when even Google displays it. Write an appealing description for the page and incorporate the page's search term into it at least once and, preferably, twice. Place one instance of it at the start or as near to the start as is reasonably possible.

The Keywords tag
The words in the Keywords tag were never treated as keywords by the search engines; they were treated as text on the page. The tag isn't as effective as it used to be but there is no reason to leave it out. So put plenty of relevant keywords into the tag and include the search term once at the front, and a second time further along the line. There is no need to seperate keywords and keyphrases with commas, as is often done, since the engines ignore commas.

The H tag
"n" is a number from 1 to 6; the biggest heading size being 1. H tags are given more weight than ordinary text and, the bigger the H size, the more weight it receives. So include the target search term in H tags at least once on the page, and two or three times if possible. Also, place the first H tag as near to the top of the page as possible.

Bold text
Bold text is given more weight than ordinary text but not as much as H tags. As much as is reasonable, enclose the search term in bold tags when it appears on the page.

Text
Use the search term as often as you can on the page whilst not detracting from the page's readability. Make sure that you use the term once or twice very early in the page's body text and as often as possible throughout. Reword small parts, and even add sentences, to make sure that the search term is well represented in the text.
In all probability, each word in the search term will be found on the page seperate from the search term itself. This is good. In fact, if they are not there on their own, add a few of them through the page.

Alt text
Include the search term in the alt text of all images on the page. Keep in mind that some systems such as Braille readers and speach synthesisers use the alt text, so you might want to make them usable whilst including the search term.

Summary Select your main search terms:
- Allocate each search term to a suitable existing page. Split some pages if necessary.
- Organize the internal linkages and link text to suit the target search terms and their pages.
- If possible, organize links from other sites to suit the target search terms and their pages.
- Organize all the on-page elements to suit each page's target search term.
- Sit back and watch your rankings improve!

About SEO Copywriting

2008-02-19T13:20:00.000-08:00

What is SEO Copywriting?

SEO Copywriting, or to give it its full name, search engine optimization copywriting or search engine copywriting, is the technique of writing the viewable text on a web page in such a way that it reads well for the surfer, and also targets specific search terms.

Its purpose is to rank highly in the search engines for the targeted search terms. As well as the viewable text, SEO Copywriting usually optimizes other on-page elements for the targeted search terms.
These include the Title, Description and Keywords tags, headings and alt text. The idea behind SEO Copywriting is that search engines want genuine content pages and not additional pages (often called "doorway pages") that are created for the sole purpose of achieving high rankings. Therefore, the engines cannot possibly view SEO copywritten pages as undesirable, and the rankings they achieve tend to be as stable as those that are achieved by other search engine optimization techniques.
Practitioners of the search engine copywriting method recommend around 250 viewable words per page, with one, or at most two, targeted search terms strategically placed within the text and other on-page elements.

SEO Copywriting strengths One thing that can be said about search engine optimization copywriting is that works for suitable websites and for suitable search terms (see below).
SEO Copywriting can achieve rankings that tend to do well across the search engines, although no page can do equally well in all engines. It is sometimes said by practioners of search engine optimization copywriting, that the method tends to maintain its rankings as the engines tweak and change their algorithms, whereas other methods produce less stable rankings.
This can't be true. If 2 pages are in the top 10 search results; one getting there by the SEO copywriting method and the other by different search engine optimization techniques, they are both there because they match the engine's criteria (algorithm) quite well. When the criteria is changed, the match that each of them had is necessarily changed.
The matches could become closer to, or further from, the engine's criteria. Whether each page goes up or down in the results depends on what changes have been made to the engine's criteria. It is a matter of chance, and not a matter of whether SEO copywriting was used or not.

SEO Copywriting weaknesses Competitive search terms
The technique only works for search terms that are not particularly competitive. Competitive search terms are those where many people are trying very hard to gain the top rankings for their sites. Casino, sex, insurance, health and hotels sites are among the most competitive, and there are many other topics where people fight for rankings. For medium to highly competitive search terms, other, more vigorous, methods are needed.

Suitable sites
Not all websites are suitable for SEO Copywriting. Many simply don't have sufficient text on their pages, and adding text would spoil the design or nature of the sites. Also, some sites that do have sufficient text sometimes don't want to be forced into changing what is written on the pages, just for the sake of the search engines.

Cost, and the limitation of targeted search terms
SEO Copywriting is a time-consuming process, and professional SEO copywriters are not cheap, therefore the cost of each page is significant. Since each page can target only one or two search terms, it would usually require a good number of pages to be made-over in order to target all the required search terms.

Tied to a copywriter
What happens when a website owner finds it necessary to alter the text on a page that has been worked on by a professional SEO copywriter? It can't be done without either ruining the costly SEO work and, with it, the page's rankings, or re-hiring a professional copywriter to redo the work once the changes have been made.

Slipping in the rankings
If a page is successfully optimized by SEO Copywriting, and is ranked in the top 10 search results for its targeted search term, then the optimization was worth the cost. But what happens when someone else decides to optimize a page from a different website for the same search term? If their optimization technique is successful, and the page gets into the top 10, the #10 page will slip to #11 - and off the first page of results. Then suppose another website does the same thing...and another...and another. Sooner or later, the successful page will slip from the first page of search results. As soon as people decide to optimize their pages for the chosen search terms, existing top 10 pages are on the way down. Then what? If the sliding pages were professionally SEO copywritten, there is nothing else that the technique can do for them, or if it can, the whole costly copywriting process must be redone. Adding one or two instances of the target search terms isn't merely a case of typing them in somewhere, because the final text still needs to read well for the site visitors. Again, the website owner is tied to a copywriter.

Summary:
SEO Copywriting is good when:
- there are not many search terms to target the search terms are on the low to middle end of competitiveness money isn't a problem, or if it is your own website you don't mind the text on your pages being frozen (if money does matter) Otherwise, 'search engine friendly' techniques should always be done as a first measure, and real search engine optimization should be done for the search terms for which 'search engine friendly' techniques are unsuccessful.

NOTE:
Competitive search terms are not necessarily those where millions of results are returned. They are those where people are competing hard for the top positions. There's a big difference.

Copywriting Tutorial
This article is about SEO copywriting and not about copywriting in general.
Copywriting is a specialized field, which has the ability to make a website successful - or not. Those of us without that skill can flounder along or we can enlist the help of one of the best in the business.
This copywriting tutorial is the best that I've found on the web, and I thoroughly recommend reading it.
SEO copywriting and normal copywriting don't always mix very well, but it is worth taking the time to integrate them as far as is possible. After all, SEO copywriting will help to bring people from the search engines to the website but, if the website fails to sell itself or its products to those people, there isn't much point in them coming.

DMOZ

2008-02-19T13:17:00.000-08:00

Getting a website listed in DMOZ can be very frustrating. We know that being listed will probably help our Google rankings, but getting in can take a very long time. In this article I will explain why it often takes so long and why what you do when submitting your site is sometimes the cause of the delay. But first I will explain what DMOZ is and why it is worthwhile for websites to be listed in it.

DMOZ, also known as The Open Directory Project (ODP), is a large, categorized directory of websites and pages, which is staffed by volunteers. Every website and page that is added to the directory has to be manually reviewed before it is included. Being listed in the directory is free.

Not many people actually use DMOZ for searches in the same way that Yahoo! is used, so the directory itself is of little value in generating traffic. However, its data can be freely downloaded, and any website, however small, can use it. One not so small website that downloads and uses DMOZ's data is Google. In fact, Google's directory is nothing less than the downloaded DMOZ directory.

This has some significant effects for websites that are listed in DMOZ. PageRank is an integral part of Google's ranking algorithm, and higher PageRank helps towards higher rankings. The PageRank within a website is increased by pages from other sites linking to it, and the higher the PageRank of the pages that link to it, the better it is for the receiving site.

A listing in DMOZ creates two significant links into a website - one from DMOZ (Google spiders DMOZ just like any other site) and one from the Google directory. Both of these usually have decent PageRank. Then add the links from the thousands of small sites that have downloaded and use the DMOZ directory, and you can see why it is usually quite beneficial for a website to be listed in DMOZ. Simply being listed in DMOZ can take a website from a Toolbar PageRank value of 3 to 4, and even from 4 to 5.

Why does it take so long to get listed?

At the time of writing, the front page at DMOZ states "57,251 editors" (volunteers who review and add websites to the directory), but this is misleading. They don't in fact have that many editors, or anywhere near that many. That number is the total number of editors that they have had since the project started. Most of them are no longer editors. Of the ones that are still editors, a significant proportion of them are not actually active or are only slightly active. So the number of editors who are actively reviewing and adding websites is relatively small.

On the other side of the equation, there is a massive backlog of sites waiting to be reviewed. Each editor can only edit in his or her own categories. Some editors have small categories with very few submissions to deal with, and they can be dealt with very quickly. Others are simply overwhelmed by the mountain of unreviewed sites, and there is little chance of getting through them in the near future.

But the huge backlog, and the relatively low number of active editors, are not the only reasons why websites seem to wait forever to get listed. Many times, the delay is the fault of the person who submitted the website. Imagine that someone submits a site to a category that is reasonably close to what the site is about, but the site really belongs in a different category. What happens? The submission waits in the unreviewed queue of the category to which it was submitted. Sooner or later its turn comes and the editor reviews it, but finds that it belongs in a different category. That editor can't edit the other category, so the submission is passed along to the other category, where it is added to the unreviewed queue. It doesn't jump the queue just because it has already waited in a different queue. Eventually its turn will come again and it will be reviewed - again.

That's the simple course of events when a site is submitted to the wrong category. In practise, though, it is often significantly different. When the first editor reviews the site, often quite a long time after it was submitted, and finds that it doesn't belong in the category, what is s/he likely to think? "If you can't be bothered finding the right category for it, neither can I". And so the site is often sent to a category that is closer to where it belongs but not necessarily to the exact one. The editor there eventually gets to it, and sends it a bit further towards the right one - maybe to the right one this time, and maybe not - and the delays mount up just because the person who submitted the site didn't take enough time to make sure that it was submitted to the right category in the first place. If the submitter can't be bothered, why should anyone else be all that bothered?

So, when submitting a site, always take time to find the right category for it. Don't be tempted to submit it to a category that is higher up the tree than it belongs, because it won't be accepted there and, doing so, could cause unnecessary, self-induced delays.

Why are some sites rejected?

DMOZ's policy is to include sites that have unique content, which means that many sites don't qualify for inclusion. Among the sites that are likely to be rejected are those that have too much content of an affiliate nature. Some affiliate content is acceptable but when it occupies too much of a site, then the site will probably be rejected.

Another reason why a site may be rejected is because of the submission. If the Title and Description provided in the submission don't follow DMOZ's guidelines, then some editors will think, "If you can't be bothered to spend a little time on it, why should I bother rewriting it for you?", and reject the site. Personally, I find it hard to believe that editors would do that, but I've heard of it happening. So, when submitting a site, read and follow the guidelines. The description is intended to give people an objective statement of what can be found in the site, and not to promote it.

People are not informed that their site has been rejected, and there must be many people out there who think their submissions are still pending when, in fact, they've already been rejected. There's only one way to know the status of a submission and that's to get someone on the inside to tell you. Fortunately, there's a place where you can do that. It's the Open Directory Public Forum, which is run by some of the editors. They are very helpful in that they will find out the state of play for a website's submission. If it's been rejected, they will tell you, and they will usually give you the reason. Sometimes they will even review a long-delayed submission, but only if the particular category has no editor or the category's editor has shown no signs of doing any editing in quite some time.

[Note: since this article was written, the editors at the Open Directory Public Forum have stopped allowing people to enquire about their site's submission status. There is now no way to find out if a site has been reviewed and rejected, or if it is still waiting for a review.]

About the DMOZ editors

As I mentioned earlier, there are not many active editors when compared to the number shown on DMOZ's front page, but most of those that are active are keen. They are keen to add websites that have unique content, and keen to improve the directory; in fact, they remind me of a colony of ants busying themselves in and around their anthill (the directory), building it up, each tiny bit by each tiny bit. Contrary to what some people think, they do care about the directory and about adding new sites, but they have an uphill struggle because there aren't enough of them.

The Madness of King Google

2008-02-19T13:09:00.000-08:00

When Google arrived on the scene in the late 1990s, they came in with a new idea of how to rank pages. Until then, search engines had ranked each page according to what was in the page - it's content - but it was easy for people to manipulate a page's content and move it up the rankings. Google's new idea was to rank pages largely by what was in the links that pointed to them - the clickable link text - which made it a little more difficult for page owners to manipulate the page's rankings.

Changing the focus from what is in a page to what other websites and pages say about a page (the link text), produced much more relevant search results than the other engines were able to produce at the time.

The idea worked very well, but it could only work well as long as it was never actually used in the real world. As soon as people realised that Google were largely basing their rankings on link text, webmasters and search engine optimizers started to find ways of manipulating the links and link text, and therefore the rankings. From that point on, Google's results deteriorated, and their fight against link manipulations has continued. We've had link exchange schemes for a long time now, and they are all about improving the rankings in Google - and in the other engines that copied Google's idea.

In the first few months of this year (2006), Google rolled out a new infrastructure for their servers. The infrastructure update was called "Big Daddy". As the update was completed, people started to notice that Google was dropping their sites' pages from the index - their pages were being dumped. Many sites that had been fully indexed for a long time were having their pages removed from Google's index, which caused traffic to deteriorate, and business to be lost. It caused a great deal of frustration, because Google kept quiet about what was happening. Speculation about what was causing it was rife, but nobody outside Google knew exactly why the pages were being dropped.

Then on the 16th May 2006, Matt Cutts, a senior Google software engineer, finally explained something about what was going on. He said that the dropping of pages is caused by the improved crawling and indexing functions in the new Big Daddy infrastructure, and he gave some examples of sites that had had their pages dropped.

Here is what Matt said about one of the sites:

Some one sent in a health care directory domain. It seems like a fine site, and it’s not linking to anything junky. But it only has six links to the entire domain. With that few links, I can believe that out toward the edge of the crawl, we would index fewer pages.

And about the same site, he went on to say:

A few more relevant links would help us know to crawl more pages from your site.

Because the site hasn't attracted enough relevant links to it, it won't have all of its pages included in Google's index, in spite of the fact that, in Matt's words, "it seems like a fine site". He also said the same about another of the examples that he gave.

Let me repeat one of the things that he said about that site. "A few more relevant links would help us know to crawl more pages from your site." What??? They know that the site is there! They know that the site has more pages that they haven't crawled and indexed!
They don't need any additional help to know to crawl more pages from the site!

If the site has "fine" pages then index them, dammit. That's what a search engine is supposed to do. That's what Google's users expect them to do.

Google never did crawl all sites equally. The amount of PageRank in a site has always affected how often a site is crawled. But they've now added links to the criteria, and for the first time they are dumping a site's pages OUT of the index if it doesn't have a good enough score. What sense is there in dumping perfectly good and useful pages out of the index? If they are in, leave them in. Why remove them? What difference does it make if a site has only one link pointing to it or a thousand links pointing to it? Does having only one link make it a bad site that people would rather not see? If it does, why index ANY of it's pages? Nothing makes any sort of sense.

So we now have the situation where Google intentionally leaves "fine" and useful pages out of their index, simply because the sites haven't attracted enough links to them. It is grossly unfair to website owners, especially to the owners of small websites, most of whom won't even know that they are being treated so unfairly, and it short-changes Google's users, since they are being deprived of the opportunity to find many useful pages and resources.

So what now? Google has always talked against doing things to websites and pages, solely because search engines exist. But what can website owners do? Those who aren't aware of what's happening to their sites simply lose - end of story. Those who are aware of it are forced into doing something solely because search engines exist. They are forced to contrive unnatural links to their sites - something that Google is actually fighting against - just so that Google will treat them fairly.

Incidentally, link exchanges are no good, because Matt also said that too many reciprocal links causes the same negative effect. The effect being that the site isn't crawled as often, and fewer pages from the site are indexed. It's a penalty. There is no other way to see it. If a site is put on the Web, and the owner doesn't go in for search engine manipulation by doing unnatural link-building, the site gets penalised by not having all of its pages indexed. It can't be seen as anything other than a penalty.

Is that the way to run a decent search engine? Not in my opinion it isn't. Do Google's users want them to leave useful pages and resources out of the index, just because they haven't got enough links pointing to them? I don't think so. As a Google user, I certainly don't want to be short-changed like that. It is sheer madness to do it. The only winners are those who manipulate Google by contriving unnatural links to their sites. The filthy linking rich get richer, and the link-poor get poorer - and pushed by Google towards spam methods.

Google's new crawling/indexing system is lunacy. It is grossly unfair to many websites that have never even tried to manipulate the engine by building unnatural links to their sites, and it is very bad for Google's users, who are intentionally deprived of the opportunity to find many useful pages and resources. Google people always talk about improving the user's experience, but now they are intentionally depriving their users. It is sheer madness!

What's wrong with Google indexing decent pages, just because they are there? Doesn't Google want to index all the good pages for their users any more? It's what a search engine is supposed to do, it's what Google's users expect it to do, and it's what Google's users trust it to do, but it's not what Google is doing.

At the time of writing, the dropping of pages is continuing with a vengeance, and more and more perfectly good sites are being affected.

A word about Matt Cutts

Matt is a senior software engineer at Google, who currently works on the spam side of things. He is Google's main spam man. He communicates with the outside world through his blog, in which he is often very helpful and informative. Personally, I believe that he is an honest person. I have a great deal of respect for him, and I don't doubt anything that he says, but I accept that he frequently has to be economical with the truth. He may agree or disagree with some or all of the overwhelming outside opinion concerning Google's new crawl/index function, but if he agrees with any of it, he cannot voice it publically. This article isn't about Matt Cutts, or his views and opinions; it is about what Google is doing.

The thread in Matt's blog where all of this came to light is here.

Update:

Since writing this article, it has occured to me that I may have jumped to the wrong conclusion as to what Google is actually doing with the Big Daddy update. What I haven't been able to understand is the reason for attacking certain types of links at the point of indexing pages, instead of attacking them in the index itself, where they boost rankings. But attacking certain types of links may not be Big Daddy's primary purpose.

The growth of the Web continues at a great pace, and no search engine can possibly keep up with it. Index space has to be an issue for the engines sooner or later, and it may be that Big Daddy is Google's way of addressing the issue now. Search engines have normally tried to index as much of the Web as possible, but, since they can't keep pace with it, it may be that Google has made a fundamental change to the way they intend to index the Web.

Instead of trying to index all pages from as many websites as possible, they may have decided to allow all sites to be represented in the index, but not necessarily to be fully indexed. In that way, they can index pages from more sites, and their index could be said to be more comprehensive.

Matt Cutts has stated that, with Big Daddy, they are now indexing more sites than before, and also that the index is now more comprehensive than before. If that's what Big Daddy is about, then I would have to say that it is fair, because it may be that Google had to leave many sites out of the index due to space restrictions, and the new way would allow pages from more sites to be included in the index.

Google's "Big Daddy" Update

2008-02-19T12:52:00.000-08:00

In December 2005, Google began to roll out what they called the "Big Daddy" update, and by the end of March 2006 it had been fully deployed in all of their datacenters. It wasn't a normal update, which are often algorithm changes. Big Daddy was a software/infrastructure change, largely to the way that they crawl and index websites.

As the update spread across the datacenters, people started to notice that many pages from their sites had disappeared from the regular index. Matt Cutts, a senior software engineer at Google, put it down to "sites where our algorithms had very low trust in the inlinks or the outlinks of that site. Examples that might cause that include excessive reciprocal links, linking to spammy neighborhoods on the web, or link buying/selling."

That statement pretty much sums up the way that the Big Daddy update affects websites. Links into and out of a site are being used to determine how many of the site's pages to have in the index. Matt then went on to give a few examples of sites that had been hit, and what he thought might be their problems...

About a real estate site, he said, "Linking to a free ringtones site, an SEO contest, and an Omega 3 fish oil site? I think I’ve found your problem. I’d think about the quality of your links if you’d prefer to have more pages crawled. As these indexing changes have rolled out, we’ve improving how we handle reciprocal link exchanges and link buying/selling."

About another real estate site, he said, "This time, I’m seeing links to mortgages sites, credit card sites, and exercise equipment. I think this is covered by the same guidance as above; if you were getting crawled more before and you’re trading a bunch of reciprocal links, don’t be surprised if the new crawler has different crawl priorities and doesn’t crawl as much."

And about a health care directory site, he said, "your site also has very few links pointing to you. A few more relevant links would help us know to crawl more pages from your site."

The Big Daddy update is mainly a new crawl/index function that evaluates the trustability of links into and out of a site, to determine how many of the site's pages to have in the index. Not only does it evaluate the trustability of the links, but it takes account of the quantity of trustable links. As the health care site shows, if a site doesn't score well enough, it doesn't get all of its pages indexed, and if the site already had all of its pages indexed, many or most of them them are removed.

I've written about the gross unfairness of evaluating links for that purpose at The Madness of King Google archive, so I won't go over it again, but I want to suggest a reason why Google has done it.

Since Google came on the scene with their links-based rankings system, people have increasingly arranged links solely for ranking purposes. For instance, link exchange schemes are all over the Web, and link exchange requests plague our email inboxes. Over the years, such links have increased and, because of them, the quality of Google's index/rankings has deteriorated. Google's system relies on the natural linking of the Web, but in implementing the system, they ruined natural linking, which in turn has eroded the quality of Google's index and rankings. It's my belief that Big Daddy is Google's way of addressing the problem. They are evaluating the trustability of both inbound and outbound links to try and prevent unnatural links from benefiting websites.

They had to address the problem but, in my opinion, they've done it in the wrong way. Nevertheless, it's done, and we have to live with it. We still have a lot to learn about the Big Daddy update, but the way I see it is that reciprocal and off-topic links are not dead, but they won't help a site as they did before. Perhaps those links won't count against a site, but they won't count for it, and links are now needed that count for a site, if it is to be fully indexed.

The best links to have are one-way on-topic links into the site, but because of what Google did to natural linking, they aren't easily found. Google caused people to not link naturally, and most sites don't naturally attract links, but the links must be found. The most obvious places to get them are directories. DMOZ can take a very long time to review a site, and even then the site may not be included, but it's a very good directory to be listed in, so it's always worth submitting to it (read article about DMOZ before submitting ).

Other directories are well worth submitting to, and a good sized list of decent ones can be found at VileSilencer. Google may not credit all the links from all of them, but that doesn't matter as long as some of them are credited - and all of them may send some traffic.

Google isn't against link-building, and their own people suggest doing it. But it is ludicrous that we have now the situation where Google first destroyed the natural linking of the Web, and then turned around to suggest ways of unnaturally getting natural links, just so that a website can be treated fairly by them. It's a ludicrous situation, but that's the way it is. Some of the unnatural ways that Google suggests are, writing articles that people will link to, writing a blog that people will link to, and creating a buzz. But most people don't want to write articles and blogs, and would have nothing to write or blog about, and very few sites can create a buzz, so for most people, a buzz is a complete non-starter.

Update:

Since writing this article, it occured to me that I may have jumped to the wrong conclusion as to what Google is actually doing with the Big Daddy update. What I haven't been able to understand is the reason for attacking certain types of links at the point of indexing pages, instead of attacking them in the index itself, where they boost rankings. But attacking certain types of links may not be Big Daddy's primary purpose.

The growth of the Web continues at a great pace, and no search engine can possibly keep up with it. Index space has to be an issue for them sooner or later, and it may be that Big Daddy is Google's way of addressing the issue now. Search engines have normally tried to index as much of the Web as possible, but, since they can't keep pace with it, it may be that Google has made a fundamental change to the way they intend to index the Web. Instead of trying to index all pages from as many websites as possible, they may have decided to allow all sites to be represented in the index, but not necessarily to be fully indexed. In that way, they can index pages from more sites, and their index could be said to be more comprehensive.

Matt Cutts has stated that, with Big Daddy, they are now indexing more sites than before, and also that the index is now more comprehensive than before.

If that's what Big Daddy is about, then I can't find fault with it. But it doesn't make any difference to webmasters. We still need to find more of those one-way on-topic inbound links to get more of our pages in the index.

Google's PageRank Explained

2008-02-19T12:49:00.000-08:00

PageRank is a numeric value that represents how important a page is on the web. Google figures that when one page links to another page, it is effectively casting a vote for the other page. The more votes that are cast for a page, the more important the page must be. Also, the importance of the page that is casting the vote determines how important the vote itself is. Google calculates a page's importance from the votes cast for it. How important each vote is is taken into account

when a page's PageRank is calculated.
PageRank is Google's way of deciding a page's importance. It matters because it is one of the factors that determines a page's ranking in the search results. It isn't the only factor that Google uses to rank pages, but it is an important one.

From here on in, we'll occasionally refer to PageRank as "PR".

Notes:

Not all links are counted by Google. For instance, they filter out links from known link farms. Some links can cause a site to be penalized by Google. They rightly figure that webmasters cannot control which sites link to their sites, but they can control which sites they link out to. For this reason, links into a site cannot harm the site, but links from a site can be harmful if they link to penalized sites. So be careful which sites you link to. If a site has PR0, it is usually a penalty, and it would be unwise to link to it.

Inbound links, link exchanges

2008-02-19T09:54:00.000-08:00

Link text (the text that is clicked on when clicking on a link) is singly the most important factor when Google determines the rankings for any given searchterm. This article isn't about that aspect of inbound links, although it is briefly touched on. This article discusses inbound links for their link popularity/PageRank value, and provides ideas on how to acquire them.

Why do we need inbound links?

Even before Google came on the scene, link popularity (linkpop) was being used by one or two search engines as part of their algorithm when determining the rankings for any given searchterm. Then when Google arrived with their link-based PageRank, link popularity took off and became an absolute essential ingredient in achieving top rankings.

The idea behind linkpop is that the more pages that link to a page, the more important the page is and it, therefore, deserves a higher ranking than it would otherwise have.

Some engines simply counted the number of links coming into a page (inbound links), but Google took the idea a step further. Each inbound link comes comes from a page which itself has inbound links. The more inbound links on the linking page, the more important that page is and, therefore, the more important the link to our page is. So Google gives more weight to inbound links from important pages that it does to inbound links from lesser pages. They call the idea "PageRank".

Google is the world's number one search engine, and currently provides the results for around 80% of all the searches done in the world. Because of that, it is vitally important for any website that relies on search engine traffic to do well in Google. Doing well in Google means making the site 'important' in Google's eyes and, to do that, the site must have good inbound links - as many of them as possible, and preferably from important pages (pages with medium to high PageRank values).

How do we get inbound links?

There are a number of ways. Some of them are:

Directories

Directories usually provide one-way links to websites, although some require a reciprocal link. Personally, I have no time for those that require reciprocal links, because they aren't really trying to be useful directories. Submitting to directories is time-consuming and boring, but there are a number of cheap directory submitting services that do a very good job.

Forums
Join forums and place links to your site(s) in your signature line. Use your main searchterms as the link text - I'll come to why that is necessary later in this article. But before spending time writing lots of posts with your signature line in each post, make sure that the forum is spiderable by checking the robots.txt file, and make sure that non-members don't have session IDs in the URLs. Also make sure that links in signature lines are not hidden from spiders (view the source code to make sure that signature links are in plain HTML format and not in Javascript).

Link exchange centers
Find a join free link exchange center like LinkPartners.com There you can find a categorized directory of websites that also want to exchange links. Be careful not to sign up with FFA (Free For All) sites because they are mostly email address gatherers and you can expect a sudden increase in email spam soon after you sign up. Also, only sign up with centers where you can approach other sites personally, and where they can approach you personally.

Do not join any link farms!!!

Link farms, such as LinksToYou.com, sound excellent for building up linkpop and PageRank, but search engines (Google in particular) disapprove of them as blatant attempts to manipulate the rankings and they will penalize sites that use them. Once a site has been penalized, it is very difficult to get the penalty lifted, so avoid all link farms.

Email requests
(a) Search on Google for your main searchterms and find the websites that are competing with you. Then find which sites link to them by searching "link:www.cometitor-domain.com". Email them and ask for a link exchange.

(b) Search on Google for websites that are related to your site's topic, but not direct competitors, and ask them for a link exchange.

Buy them
There are websites that want to sell links. They are usually medium to high PageRank sites, and often the link will be placed on multiple pages, or all pages within the site. It's possible to approach individual sites where you would like your links to appear, but it is much quicker, easier and more reliable to use a middle-man service (or broker).

Link brokers offer links for sale on behalf of other websites (you could use the service to sell links on your site!). With these services, it is usual to be able to choose the type (topic) of the website(s) where you want to place your links. I am not aware of any disreputable practises with link brokering.

Finally, there are even links for sale by public auction, such as the one at LinkAdage Auctions.

What type of links?

Inbound links serve three purposes:
(1) they add linkpop and PageRank to the site, (2) they send traffic to the site, and (3) if the link text (the text that is clicked on) is good, then the page's rankings can be improved.

(1) for linkpop, either text or graphic links are fine, (2) for traffic, text links are known to be more effective than graphic links, and (3) to positively affect a page's rankings, text links are the best. So get text links and not graphic links.

(1) for linkpop, get links from anywhere, (2) for traffic, get links from related sites, and (3) to improve the page's rankings, get links from anywhere - the higher the PageRank of the linking page, the better and, whenever possible, make sure that the link text is one of your main searchterms.

Summary

Inbound links are important for websites that want to move up the rankings. Inbound links, with the right link text, are essential for achieving top rankings.

Most websites are not natural link magnets, and link acquisition can be time-consuming and frustrating. Many websites that are approached by email will say no, but some will say yes. For top rankings, it is essential to take the time and get many inbound links.

Update
Google's "Big Daddy" update which was rolled out during the first part of 2006, was largely concerned with their crawling and indexing functions. The update contained a new concept for them - making a mathematical calculation to determine how many of a site's pages to have in the index. The mathematical formula is concerned with the nature and trustability of links coming into a site, the nature of links going out of it, and the quantities of those.

The new crawl/index system doesn't specifically penalise sites for having too many links of the 'wrong kind', but it doesn't give them credit for crawl/index purposes, and many sites suffered by having large numbers of pages removed from the index.
This is important, because it means that exchanging links willy-nilly can actually cause some or many of a site's pages to be removed from the index, due to the lowering of the site's IBLs/OBLs score.
So it is better to exchange on-topic links than to exchange off-topic ones. The days of a link is a link is a link are over. The helpfulness of an over-abundance of reciprocal links are gone, and it can actually cause some of a site's pages to be dropped from the index.