Search Engine Showdown Blog

Google Search Focus

Wed, 21 May 2008 22:56:11 -0700

Sometimes I find the Google blog posts to be long winded, high on hype, and low on information value. Yesterday's post about Google Search Quality started out in a similar vein, but it quickly improved and contains a number of interesting points about how Google handles searches and ranking. And for all those who like to say, "Just make it more like Google" and expect that to be a simple fix, please note the way Google describes their hard work on search quality is that "more than one thousand programmer/scientist years have gone directly into their development."

Several extracts that I found of interest include:

Ranking algorithms include many aspects beyond PageRank:

language models (the ability to handle phrases, synonyms, diacritics, spelling mistakes)
query models (how people use language today)
time models (some queries are best answered with a 30-minutes old page, and some are better answered with a page that stood the test of time)
personalized models (not all people want the same thing)

Evaluation includes automated evaluations every minute (to make sure nothing goes wrong)
Change Frequency: "In 2007, we launched more than 450 new improvements"

While these do not, perhaps, have any direct bearing on how we can better use Google, it does help to inform us about the rationale for changing results and different processing from one day to the next.

Major Expansion at Google Translate

Sun, 18 May 2008 23:55:44 -0700

Earlier this month Google expanded the number of languages available in Google Translate. While the press release and most other coverage talked about ten new languages, the number of language pairs (from language X to language Y) increased far more substantially. Previously, Yahoo! Babel Fish had the most with 38 pairs. Google not only upped the number of possible languages, but every language listed can translate to the other. So depending on how you count, Google Translate now has over 500 language pairs available! That's a major increase. As Google Operating System notes, the counting varies depending on how you count Chinese. Only one choice is given for input of "Chinese," but Google Translate seems to accept both the Simplified or Traditional versions. Output can specify either Simplified or Traditional. So, if you count both versions of Chinese as one languages, this means Google Translate can machine translate 506 language pairs. If you consider that as two, it would be 552. And do note that you can input either version of Chinese characters and have it translated to the other.

Also note that Google has not only expanded its machine translation abilities but has augmented its Translated Search as well. Translated Search (also available on the Language Tools page as "Search Across Languages") will translate the query words and then display results in both the original language and in translation. Google translated search can machine translate query words and pages between the following languages. The following ten languages have been added along with the ability to translate between any of the possible language pairs.

Bulgarian
Croatian
Czech
Danish
Finnish
Hindi
Norwegian
Polish
Romanian
Swedish

Presumably, Google has been able to make such a major expansion of language translation pairs available by using statistical machine translation developed in house. This process is described in their FAQ: we feed the computer billions of words of text, both monolingual text in the target language, and aligned text consisting of examples of human translations between the languages. We then apply statistical learning techniques to build a translation model." Moving to this approach certainly seems to have allowed such a major expansion. Bear in mind that all of this automatic translation is prone to error, although it should give some rough sense of the underlying meaning. I've updated my Online Translation and Translated Search pages with the new languages.

Search Switching

Tue, 08 Apr 2008 18:48:55 -0700

For some time now I have been speaking and writing about ways of speed searching and search switching. Somehow, I've neglected to add the links to my site. So I'm fixing that tonight, before my presentation on Wed. at CIL 2008. The new Search Switching page includes sections for Search Switching Between Web Search Engines, Geographic Search Switching, Book Search Switching, and other options including another link to my Bookmarklets page (with its search transfer bookmarklets).

See also my article, "Speed Searching," in the March 2008 issue of Online (available for fee at ITI's InfoCentral or free from many library databases such as AccessMyLibrary) and my article, "Switching Your Search Engines," from the May 2007 issue of Online (available from many library databases including AccessMyLibrary).

Language Limit Showdown

Mon, 31 Mar 2008 23:04:38 -0700

In recent months I have been speaking and writing about some of the language search and translation features of the search engines. Which search engine has the most language limits? Which online translator has the most language pairs? And which ones offers translated search? (Exalead, Yahoo!, and Google, respectively). So I've added a new Language Search Tools page, with links to pages about Language Limits, Online Translation, and Translated Search. I have also finally made a major update to my Search Engines by Search Features page and linked the language page from there.

For many searchers, especially those of us in the middle of a fairly mono-lingual part of the U.S., the language tools may have little appeal, but even in the midst of Montana, I still find times that I come across a non-English site, email, or term that can benefit from the use of these tools.

Yahoo! Face Change

Wed, 26 Mar 2008 19:27:14 -0700

The main Yahoo! page has cleaned up and re-emphasized its search box. The whole top inch or so of the page has a new look.

Similar to the cleaner, pared down page at search.yahoo.com, and reminiscent of some other well-known search engine, it is a pleasing change. The main page still has plenty of portal content below the search box:

Contrast that with the old look that had the logo on the left and additional buttons and boxes. See the Google Blogoscoped post for a series of home page screenshots over time. I still wish that News was one of the choices above the box instead of having it hidden under "more." Otherwise I find the new look to be easier to use and cleaner than the old.

Updated Bookmarklets

Tue, 25 Mar 2008 08:49:20 -0700

I've updated my search bookmarklets page due to changes from some of the search engines including Gigablast's new interface, finally changing msn.com to live.com in the code, and the addition of several new links:

An animated .gif of the search transfer bookmarklets
Moving the search transfer bookmarklets to the top
A new bookmarklet for numbering Yahoo! results
Updating the bookmarklet for numbering Google results
Gigablast and Exalead search box links

The search transfer bookmarklets are the ones that I use the most to quickly switch from one search engine to another with the same search query. It has been interesting seeing how the bookmarklet functionality changes over time even when I do not change the underlying JavaScript of the bookmarklet. For the past several months, I could use the Yahoo! bookmarklet after a Google, Ask, or Live search, but I could not use any others after a Yahoo! search. Now those bookmarklets work again. So at the moment, I am only have a problem using them after a Gigablast or Exalead search.

Amazon Dropping Search Inside The Book?

Thu, 13 Mar 2008 09:52:55 -0700

I am very disappointed to see that Amazon seems to no longer have the Search Inside The Book feature. Clued into this loss by Marydee's post yesterday, I can no longer even get Marydee's workarounds of going to the Canadian or British versions of Amazon to work. So what's missing?

No more Search Inside the Book arrows on certain cover shots
None of the text statistics and ability to search within a book on those books
A search on a unique term like tilinca which only occurs in the contents of books now gets no results.
No Look Inside excerpts or any view of the contents of the book

I certainly hope that this is a temporary issue and not a permanent closing of the program. Some of the initial cover shots still have the Search Inside graphic on top as in the following example. But clicking on the link takes me to a page with no ability to view any of the contents.

Searching via the nearly-defunct A9, I was able to get a list of results for books including my search term, but I still could not get inside the books at Amazon. If this continues, it would be a significant and major loss of searchable content in books!

[Update: Just before clicking post, I am starting to see the Search Inside links again. Phew! I hope this was just a temporary glitch and not a major change in policy!]

More Caches: Japan and China

Sun, 27 Jan 2008 18:48:37 -0700

So after learning yesterday that the Russian search engine Yandex (Яндекс) cached pages, I started looking at a few other well-known, non-English search engines. Baidu, the Chinese search engine has just expanded into Japan with a Japanese Baidu. Both of these also have cached copies of pages. At Baidu, look for the 百度快照 links after the URL (similar to Google's placement). For the Japanese version, the cache linked is キャッシュ in a similar location.

For us non-Chinese and non-Japanese speakers, is there any use in these? Well, they are one more source for archived versions of pages, including English-language ones. For example, a search on library of congress (in English) finds hits at both. Here is a screenshot the Chinese version with the cache link in gray at the end.

And here's the Japanese version, with the cached link again in gray at the end. I'll be adding both of these to my Finding Old Web Pages page.

(Screenshots via kwout.)

A Russian Cache

Sat, 26 Jan 2008 14:34:59 -0700

I do not usually spend much time with country-specific search engines, especially those in languages I do not speak. Even with English-language country-specific search engines, the general search engines usually have more comprehensive results and better search functionality. So when Phil posted about the Russian search engine Yandex (Яндекс), I just thought I'd take a quick look. Something piqued my curiosity, and I tried a few of the links. Sure enough, Yandex caches copies of many of the pages that it indexes. Look for the Сохраненная копия link at the bottom left of a search result record as in the screen shot below.

Yandex's cache does not include a date, at least that I could identify, but from a few tests, it seems that the cached page may be quite recent (the day before) to several months old. I've added Yandex to my Finding Old Web Pages page.

Yahoo! Adds Del.icio.us in Search Results

Sat, 19 Jan 2008 21:09:08 -0700

TechCrunch discovered that for some Yahoo! searches, they have added in links to (Yahoo!-owned) Del.icio.us bookmarks. Yahoo! does not use the Del.icio.us name. Instead, the Del.icio.us logo is followed by number and "people bookmarked this page under" whatever tags they used. See the two lines outlined in red in the screen shot.

While not all Yahoo! results have been bookmarked in Del.icio.us, this is a great combination of information from the two services. Click on the number to see what comments and notes people have written about the site when they bookmarked it. While I wish the comments would appear when I mouse over the link, it is still an incredibly easy and quick way to see what others have said about a Web page before you visit it. So how do you get Yahoo! to show the Del.icio.us information?

Google Stop Word Message Gone?

Sat, 19 Jan 2008 20:33:20 -0700

At SearchEngineLand, Barry noticed that Google is no longer alerting searchers that stop words are not searched. Previously, stop words in a query that was not in phrase marks would usually find Google prompting searchers that the stop word in the query is "a very common word and was not included in your search." Does this mean that Google no longer has any stop words? Based on a few of my tests with a small retrieval set, comparing a search with a stop word and another search with a + in front of the stop word, it does seem that Google will on occassion still ignore some stop words.

AskEraser: Privacy Potential

Wed, 09 Jan 2008 09:36:02 -0700

Forgot to mention my Newsbreak article from last week, AskEraser: Privacy Potential. In working on that article and looking at AskEraser, one issue occurred to me that I did not cover in the piece itself.

I received email from Ixquick about AskEraser even before I saw anything from Ask. press release. In that email, Ixquick claims that

there is one crucial element Ask overlooked and did not address. It reduces their privacy offering to near zero: As can be read in their privacy policy under ''Third Party Providers'' they continue to send the ''users search activity data'' to these third party providers, one of which is .......Google! In other words: Even with Askeraser ON the user gets the same privacy as with .......a Google search!

OK, Ask uses ads from Google. To display context-sensitive ads, related to the actual query, they have to send at least the query string to the Google ads server. While I could not get a definitive answer from Ask about what specific data elements Google sees, given Ask's commitment to privacy with AskEraser it seems likely that not much more than the query is sent. Even so, I was curious to see how Ixquick funds its service, and sure enough, it uses Google AdWords as well (under Sponsored Listings). So how is their privacy better than AskEraser?

While neither Ask nor Ixquick gives complete privacy, nor do they claim to. I credit both of them with raising the search privacy issue and providing at least some tools for helping protect searcher privacy. Since the vast majority of searchers pay no attention to such issues, I'd rather see all the search engines providing better privacy options rather than just criticizing their competitor's attempts.

Scholar Down, Books Up

Thu, 03 Jan 2008 17:38:51 -0700

Dean posted a scathing review of Google Scholar's performance over that past year based on a 32% decline in unique visitors according to ComScore data. More data on the changes at various Google properties between Nov. 2006 and Nov. 2007 are available in a TechCrunch posting. While I am sure that this data does not fully reflect actual Google traffic (and at least one comment on Battelle's Searchblog post says "a staff member from Google . . . tells me that ComScore has some of their numbers wrong"), I still find it fascinating. To no one's surprise, Web search is by far the busiest Google property. Google Directory traffic went down, which is not surprising since Google has made it so much harder to find. But the huge declines in Product Search (down 73%), Scholar (down 32%), and the Video Search (12% decline) surprised me. Book Search on the other hand has grown significantly in visitors (up 55%).

The chart showing which Google properties get the most visits is interesting as well. Web and Image search dominate and are both growing. After those two comes Gmail and Google Maps, which both rank higher in visitors than Google News. Given its increased prominence on the Google News page, I was also surprised to see how few visitors ever went to Blog search.

For Google users who visit many of their services, this is a telling lesson about how others use or do not use so many of Google's search services and applications. I also agree with Dean that Google Scholar's drop in visitors (if that is indeed accurate) comes in part from their failure to improve the service. I have found general Web searches often more effective than Google Scholar searches for at least some scholarly documents.

Live Links Lost Again

Wed, 19 Dec 2007 19:46:47 -0700

At least it worked for awhile. The trick of using the plus for getting Live's link searching commands to work again (link:, linkdomain: and inurl:) that has worked since those commands were disable in March has stopped working. It looks like link searching is again unavailable at Live, it continues to be incomplete at Google, so Yahoo! is the best choice for link searching.

While I can understand that problems with "automated usage for data mining" caused Microsoft to disable the command in March 2007, it is really sloppy of Microsoft that all these months later, the advanced search still has a "links to" section that does not work, and their online help still has instructions on how to use this non-functioning "links to" section.