Custom Search Engine

Advanced Custom Search Configuration

2009-06-29T11:19:00.000-07:00

Posted by: Rajat Mukherjee, Group Product Manager

Here's the presentation on Advanced Custom Search Configuration from Google I/O. You can navigate it with the help of a convenient timeline at YouTube, or download the presentation in PDF format. The talk features Nick Weininger from the Custom Search team on how to create a contextual search experience in minutes, along with details of the Custom Search APIs, including:

Context and Annotations
Advanced features, e.g., Promotions, Synonyms and XML results
RESTful URLs for managing Custom Search engines
Provisioning API for partners
Linked Custom Search
Rich snippets and metadata - pagemaps, microformats and RDFa
The Blogger search gadget
The Custom Search Web Element

In addition, Mark Nichoson from Adobe talked about Adobe Community Help, a customized search application that's available within apps in Adobe's Creative Suite 4. Joel Brandt from Adobe followed up with a description of Blueprint, an innovative application that integrates Custom Search into the Flex development environment.

For more information, please check out the Custom Search developer guide.

Automatic transliteration for Custom Search

2009-06-16T10:38:00.000-07:00

Posted by: Mohamed Elfeky and Adel Youssef, Software Engineers

Typing is harder in some languages than others! When you're searching for content in a specific language, it is often convenient to think in that language, but type in another, e.g., English.

We've just made this easier to do in Custom Search. We've enabled transliteration in Custom Search for a set of languages, making it easier to find news in Arabic, Indian news in Hindi, your favorite Bollywood song lyrics, or local content in a bunch of other Indic languages - Kannada, Malayalam, Tamil and Telugu. Other languages will be enabled soon.

The Google Arabia blog recently posted information on Google Ta3reeb, enabling Arabic transliteration in various modes, including within Custom Search.

Here's a screenshot from oneindia.in, a website that has enabled transliteration in a set of Indian languages.

Users can type in English; automatic transliteration converts the query to Hindi, Kannada, Malayalam, Tamil, or Telugu, and helps find relevant content in that language.

Automatic transliteration can now be easily configured in the Custom Search control panel using the Language settings. It will enable conversion of Roman characters to the above languages, allowing users to search for content in these languages. After you turn transliteration on, you can select the specific transliteration languages your users can use.

After enabling transliteration, you can preview your search experience - a language button will show up before the search box, alerting users to the fact that transliteration is enabled, and that they can issue queries in that language while entering the queries in English. In fact, multiple languages can be selected for transliteration. As a query is entered, it is transliterated into the selected language in real-time.

Hit search, and results in the desired language are presented. Voila!

For more information on transliteration, you can check out our help center. As always, we're looking forward to your feedback on our user forum.

A search box for Blogger

2009-05-29T10:36:00.000-07:00

Posted by: Nicholas Weininger, Software Engineer

Custom Search enables anyone to create a tuned search experience that's contextually relevant. For example:

Individuals can create a personalized search experience around their bookmarks, blogs, and public web sites
Web site owners can provide Site Search
Publishers can provide search across multiple publications
Communities can collaborate to create topical search engines across thousands of web pages

For bloggers, blogging platforms typically provide in-built search tools that allow searching across published blog posts, or across tags and categories. With Custom Search, you can go one step further: you can define a search experience that evolves over time, and includes not just your blog posts, but links extracted from those posts, as well as links from your blog's link lists and blog lists - in short, all items of interest related to your blog.

If you author a blog on Blogger, we've built a search gadget for you that does this - the AJAX Custom Search gadget creates a Linked Custom Search engine that automatically updates to allow your readers to search your blog's entire neighbo(u)rhood. It is a uniquely flavo(u)red search experience that gets richer over time. Search results appear inline, so your users don't have to leave your blog. The results inherit the look and feel of the blog, as shown in the screenshot below.

On Blogger, you can add the gadget with a couple of clicks:

Edit your blog's layout (Page Elements tab)
Click on "Add a Gadget" and configure the new "Search Box" gadget.

You can configure tabs that will allow your blog's readers to restrict their search to specific link lists or blog lists; you decide which ones you want to configure.

If you are using this gadget, we'd love to hear your feedback in our discussion group.

The AJAX Search and the Custom Search APIs have also been combined to create the Custom Search element that we recently announced at Google I/O.

Introducing the Custom Search Web Element

2009-05-26T16:46:00.000-07:00

Posted by: Rajat Mukherjee, Group Product Manager and Nicholas Weininger, Software Engineer

Google Web Elements, unveiled today at Google I/O, build on your favorite Google products so you can easily add richness, interactivity and monetization to your website with the simplicity of copy and paste. We've packaged this simplicity into the new Custom Search element, making it even easier for you to add Google-quality search to your website. By helping your users find information faster, you can retain users on your website and monetize search with relevant Google ads. The element will allow you to create an interactive, inline search experience, so users don't need to leave the web page they are on in order to search.

You can create a Custom Search element in two convenient ways: via the Custom Search element wizard, or by selecting the appropriate options within the Custom Search control panel.

When you log on to the control panel, you'll see links to the Custom Search element wizard if you list your search engines (My search engines). If you click through to the wizard from the control panel, the appropriate search engine information is automatically extracted by the wizard -- all you need to do is preview your search results, and copy and paste the generated code into your website.

You can also generate the right code for embedding the Custom Search element into your website by going to the updated Get code tab in the control panel. A new option allows you to generate element code.

The Custom Search element supports Promotions, which let you highlight specific information for specific queries. Refinement Labels automatically appear as separate tabs in the element, so you can navigate quickly to the category of results that you're looking for. The screenshot below shows a preview of the search results in the element wizard.

While the Custom Search element is designed to help you get started quickly without spending time on the deep technical details, it is powered by Google's scalable and flexible developer APIs, including the AJAX Search and Custom Search APIs, offering a world of customization.

Let us know if there are further improvements we can make to this user experience. To learn more about Google Web Elements, check out the AJAX API blog.

Custom Search at Google I/O

2009-05-22T13:19:00.000-07:00

Posted by: Saksiri Tanphaichitr, Software Engineer

If you're attending the upcoming Google I/O Developer Conference (May 27th and 28th), make sure to attend the sessions where we plan to discuss the latest Custom Search developments.

There are 2 sessions that will highlight Custom Search:

These sessions, part of the AJAX + Data APIs track, will discuss ways in which you can easily build, embed, and customize a contextual search experience on your website. You will learn how to develop advanced applications using the AJAX Search and Custom Search APIs, create dynamic experiences with Linked Custom Search, and programmatically provision Custom Search for your users. We'll also talk about interesting developer applications, as well as advanced presentation of search results including features that we recently launched: Rich Snippets and Promotions.

Adobe, one of our partners, will join us in discussing their use of Custom Search, and will also show interesting developer applications built on top of the Custom Search platform at the Developer Sandbox.

Here's a sneak peek:

We look forward to seeing you at Google I/O.

Custom Search promotions made easier

2009-05-18T15:58:00.000-07:00

Posted by: Bartlomiej Niechwiej and Nicholas Weininger, Software Engineers

Last year, we made it easier to promote relevant information to the top of your search results for specific queries, but there were still a couple of steps you had to take to make this happen. So now, we've made it easier than ever to create promotions in Custom Search. You can quickly configure specific webpages or information to display above your search results for specific queries, allowing you to highlight key events, announcements, services or products.

We've added a new Promotions tab to the Custom Search control panel to help you manage all your promotions.

Creating a promotion is easy - decide on the queries that should trigger a specific promotion, select a title and URL for your promotion, and add optional information, such as description and image thumbnail. The promotion edit pane also shows a preview of your promotion.

When you're done, promotions will display in your search results for the queries that match your trigger list. You can test a promotion by issuing one of the trigger queries in the control panel results preview.

A few pointers:

you can control the look and feel of the promotions to distinguish them from regular search results (through Promotion Design Settings)
you may want to create promotions to highlight information for popular queries on your search engine (use search statistics to identify them)
you should delete promotions when they are no longer relevant
our updated developer documentation includes more information on promotions

We hope to see useful promotions popping up in custom search results all over the web.

Don't miss the promotions when you search on the Custom Search website! We'd love to get your feedback on how we can make this feature even more useful.

Enabling Rich Snippets in Custom Search

2009-05-12T10:40:00.000-07:00

Posted by: Rajat Mukherjee, Group Product Manager and Nicholas Weininger, Software Engineer

Today, we announced Rich Snippets for Google web search, a new presentation of snippets that highlight structured data embedded in web pages. Rich Snippets provide summary information, including important page-specific attributes, to help you quickly identify the relevance of your results. Experiments on Google have shown that people find this additional data valuable -- if they see useful and relevant information from a webpage, they are more likely to click through to it. Our web search team is currently experimenting with a limited set of attributes for reviews and user profiles that webmasters can provide through in-line markup in their web pages by using open standards such as microformats or RDFa.

Since Custom Search sits on top of the Google web search platform, we're enabling Rich Snippets for custom search engines too. In fact, Custom Search provides support for richer snippets; you can define your own custom attributes that we'll index and return with your custom search results. In addition to microformats and RDFa, you can also provide custom metadata within your web pages via special markup called page maps. A page map identifies specific attributes that are recognized and preserved by Google at index time, and returned along with search results for presentation.

So, if you're using Custom Search or Site Search on your website, you can now control further how your content appears in search results. You can showcase key information -- such as image thumbnails, summaries, ratings, and prices -- in your result snippets if you provide the appropriate markup on your pages. As in the example shown below, displaying specific attributes can make the snippet more useful for shoppers.

At this time, Rich Snippets attribute information for Custom Search is only returned in XML (via <PageMap> tags), so you can use your own customized presentation controls. We're looking to make this easier for you in the future. Indexing of the rich snippets information can have unspecified latency, as some pages are indexed and refreshed more frequently than others, and page map attributes may not be indexed from all web pages.

As an illustration, the web page featured in the following example provides custom information about an image thumbnail that is displayed in the rich snippet of the result along with date, author and category information.

A sample page map for the above example defines custom attributes that are encoded in DataObject tags via the following markup:


<!--
<pagemap>
<dataobject type="image">
<attribute name="image_src" value="http://www.nytimes.com/images/2009/03/14/business/14vend.751.jpg">
</dataobject>
<dataobject type="publication">
<attribute name="author" value="John Tagliabue">
<attribute name="date" value="March 14, 2009">
<attribute name="category" value="Business/World Business">
</dataobject>
</pagemap>
-->

If you are getting results back via XML, then the custom attributes are returned in the results within the PageMap tag, as shown below. You can parse the DataObjects within the PageMap tag and provide customized presentation of the relevant attributes.


<r n="1">
<u> http://www.xyz.com/business/vending_machine.html </u>
...
<t> In Italy, a Vending Machine Even Makes the <b>Pizza</b> </t>
...
<s>
The European vending machine industry has annual sales of about #33 billion, much of it coming from factories and offices.
</s>
...
<pagemap>
<dataobject type="image">
<attribute name="image_src" value="http://www.nytimes.com/images/2009/03/14/business/14vend.751.jpg">
</dataobject>
<dataobject type="publication">
<attribute name="author" value="John Tagliabue">
<attribute name="date" value="March 14, 2009">
<attribute name="category" value="Business/World Business">
</dataobject>
</pagemap>
...
</r>

Let us know if you have feedback about custom Rich Snippets. For more information, join us for a discussion on Advanced Custom Search Configuration at Google I/O on May 27.

Web Search APIs: The next generation

2009-04-17T11:01:00.000-07:00

Posted by: Rajat Mukherjee, Group Product Manager

If you are going to the World Wide Web conference in Madrid next week (April 20-24), make sure to attend the panel entitled "Web Search APIs: The Next Generation". Nick Weininger from the Custom Search team will be on the panel. He's going to talk about real-world needs from a Search API, and how the Custom Search API provides various customization options for indexing, ranking, navigation, presentation and monetization.

Don't miss the panel!

Check out details on the WWW 2009 conference program.

Google Services for Websites includes Custom Search

2009-03-25T16:51:00.000-07:00

Posted by: Radu Cornea, Software Engineer and Kevin Godzilla Lim, Technical Writer

As you might have read on the Official Google Blog today, Google is launching Google Services for Websites, a new program that lets partners offer Google services — including Custom Search, Site Search, Webmaster Tools, and AdSense — to their users.

Partners, such as hosting providers and blogging platforms, can integrate these services into their own management consoles, making the services easily accessible to their users. Google Services for Websites allows partners to create Google accounts, submit Sitemaps to Webmaster Tools, configure Custom Search, and provision new AdSense accounts on behalf of customers. Additionally, partners are eligible for referral fees for certain services.

Once Google Services for Websites has been integrated into the control panel, users can configure their Google services in minutes. These services can help increase website traffic, provide site search, and monetize website content. In fact, a number of web hosting providers have already integrated these services into their offerings.

Furthermore, all of these services are designed to complement and enhance one another. For example, Sitemaps submitted to Webmaster Tools not only improve the discoverability of websites on Google search, but also improve coverage and freshness in the Custom Search index. AdSense accounts can also be associated with search, which lets website owners earn revenues from relevant ads displayed alongside search results.

To learn more about integrating Google Services for Websites with your services, read the Google Services for Websites program overview.

Integrating Custom Search with your service

2009-02-03T10:47:00.000-08:00

Posted by: Kevin Gollum Lim, Technical Writer

Most people who need something just go to the store to get the item, but a number of people take the time and effort to handcraft their own stuff. They do that because they enjoy tinkering with things until they work just the way they like them. Well, Custom Search has a group of developers who are like that, and they want to create and share their tools.

For such developers, we have good news. We launched a new API that lets other services connect to Custom Search. You can programmatically create, update, and delete search engines without going through the Custom Search control panel. In fact, the API offers most of the functionality that the Custom Search platform provides, and we're working on supporting the rest.

If you provide a service such as hosting websites or blogs, you can partner with Google to create custom search engines for your users. You can use the API to integrate Custom Search into your management console. Your users can then create search engines using your service and will need to use the Custom Search control panel only if they want to be more involved in the customization of their search engines. In our previous blog entry, we talked about how several web hosting providers have already done this. (If you are a web hosting provider who wants to integrate search into the sites you host, please ping us.)

Even if you are not providing a service for a large set of users, you could still create a lot of cool things with the API. Here are a few ideas: If you have a list of websites that change frequently, you can automate the updating of your annotations file; if you have a content management system, you could create search for all your webpages and even create search labels based on your existing categories; if you are a blogging platform, you can enable search for all your blogs.

To learn more about the new API, read the programmer's guide.

Home delivery of Custom Search to webmasters

2009-01-20T09:48:00.000-08:00

Posted by: Radu Cornea, Software Engineer and Rajat Mukherjee, Group Product Manager

Until today, when you wanted to add Google Custom Search to your site, you had to head directly to our Custom Search Engine site. Although creating your own Google-powered search engine only takes a few minutes, we thought it might be easier if we delivered this functionality right to where many of you manage your sites online. Choice is a beautiful thing.

If you own a website hosted on IPOWER, StartLogic, PowWeb, FatCow, BizLand, EasyCGI and several other web hosting providers, you can now configure Custom Search right in your hoster's management console - these hosters now support both Webmaster Tools and Custom Search. It's like having pizza delivered to your door instead of having to drive over to the pizza parlour!

When you sign up for the service, you get the following benefits, all from the comfort of your hoster's control panel:

a Google account is automatically created for you - you don't have to remember additional passwords or login to use these Google services
you can generate search engines for your websites by simply identifying your domains
shortcuts are provided to manage your Webmaster Tools account and your Custom Search Engines
Sitemaps for URLs on your sites can be automatically submitted to Webmaster Tools for better indexing on Google.com
your Custom Search engines will automatically recognize and use the submitted Sitemaps for enhanced indexing coverage
you can make money by associating your Adsense publisher ID with your search engines and sharing advertising revenue with Google
you can upgrade your search engines to use Site Search for further control over branding, presentation (via XML results) and On-Demand indexing

If your site is hosted by one of the above providers, give it a try - go to your hoster's console and add a search box to your site if you don't already have one. High quality search improves user experience, and also directly increases user engagement and improves conversions on web sites.

Additionally, let us know if there are things about this integrated service offering that we can improve. Our programmer's guide provides more details about the new Custom Search APIs that allow programmatic creation and management of Custom Search engines. If you are a web site hoster interested in incorporating these Google webmaster services into your platform, please contact us.

Searching the Google Timeline

2008-12-11T13:58:00.000-08:00

Posted by: Tyler Morse, Software Engineer and Dylan Casey, Product Manager

When we set out to build a timeline to commemorate Google's 10th birthday, we wanted to add an experimental search feature. We may be a little biased, but because we needed to search a specific subset of the web, and needed customized presentation, we decided to use Google Site Search, which is powered by the Custom Search platform. Using Site Search, the team designed a customized user interface and user experience, tightly embedded within the timeline application, in a few days.

The Google Timeline data is stored and served up as static files of JavaScript objects and arrays. While this was very efficient for the Timeline JavaScript application, we needed to massage the data in order to get it indexed and searchable, as well as tie it back to the timeline objects.

We generated static HTML files for all timeline events. The filenames have identifying data in them that allow us to correlate a search result with the original event; this way, we can translate a user click on a search result to the appropriate timeline event. Since we wanted explicit control of event dates, we also encoded the event date in each HTML file.

We use Site Search's flexible XML interface for retrieving the results. When we get back search results, we parse the XML data, extract specifically the information we need, and render these results.

Since we did not want to take users away from the timeline application, we presented the results inline. As we had limited real estate available, we only displayed 3 results at a time. As shown in the screenshot above, we added pagination to the results section, and the ability to sort the events by event date.

We couldn't treat our search results like typical URLs, as on Google.com. We wanted to show users the event within the actual timeline when they clicked on a search result, instead of navigating them to the indexed static HTML file. Clicking on a result provides an interactive experience, and brings up the specific timeline event.

The end result: a Google search for Google's history, custom fit to meet the specific needs of our timeline, just before we blew out the candles on our cake.

On-Demand Indexing for a fast-moving web

2008-11-13T09:27:00.000-08:00

Posted by: Tom Duerig and Nicholas Weininger, Software Engineers

Earlier this year when we launched Google Site Search, and AdSense for Search started using the Custom Search platform, we created a special Custom Search Engine (CSE) index for enhanced indexing. Webmasters could submit Sitemaps in Webmaster Tools, and the Custom Search platform indexed URLs from these Sitemaps into a special CSE index for more comprehensive coverage.

This solved an important problem: the need for enhanced index coverage for site search. The special CSE index, in addition to the Google index, enabled us to search deeper on selected sites, providing higher search quality.

Today, with our launch of On-Demand Indexing, we're addressing the need to provide fast indexing for your important, frequently-changing content. On-Demand indexing allows anyone with a Google Custom Search Engine or Google Site Search to identify and tell us about new pages or recent site changes by submitting a Sitemap to our Webmaster Tools. You can then select this specific Sitemap in the Indexing tab of your CSE control panel and hit the new "Index Now" button. We will immediately schedule the relevant pages for crawling and indexing, and these pages will be included in your search results usually within 24 hours - often much faster.

We realize that new or frequently updated content is often crucial to a website. In order to give you more control over how you use the On-Demand Index resources allocated to you, we will honor the priority and lastmod attributes that you provide in your Sitemap - this means you can identify what URLs should be given preference. Please ensure that your Sitemap metadata is up-to-date.

On-Demand Indexing uses a special index that is designed just for Custom Search. The main Google search indexing process is separate and uses different selection criteria, so submitting pages for On-Demand Indexing will not make them appear any faster in the main Google index, or impact ranking on Google.com. For more information on On-Demand Indexing, as well as the allocation per CSE, please refer to our FAQ.

We are always listening at our Custom Search discussion group, so please continue to tell us what features you'd like to see added to Custom Search.

Knol - now with Custom Search

2008-10-30T09:59:00.000-07:00

Posted by: Cedric Dupont, Product Manager

Knol is a project aimed at helping people share their knowledge. Knols (units of knowledge) are authoritative articles written by people about a specific subject, ranging from tooth pain to solar energy to buttermilk pancakes. With all of these knols to browse, our readers have been begging us for a more powerful search tool. Well, today we have good news - we've added the power of Custom Search to Knol.

Custom Search gives us all of the speed and relevance of Google's search technology, but required none of the hard labor that went into making Google search what it is today. With Custom Search, Knol visitors will have a fast search experience that features all of the bells and whistles Google searchers have come to love (including our advanced spell checker). When you search within Knol, the search results look a little different from Web Search results (see this search for knee surgery, for instance). We were able to maintain that distinct feel for Knol search by taking advantage of the Custom Search XML results format (via Google Site Search) and creating the search result display that we wanted.

We want to hear from you if you have feedback on Knol search. Please leave your comment on our help page.

Synonyms for your Custom Search Engine

2008-10-03T11:26:00.000-07:00

Posted by: Victor Wang and Bartlomiej Niechwiej, Software Engineers

With our launch of Google Site Search, we added a new feature to the Custom Search platform: custom synonyms. Here's how this feature can be used to improve the quality of your Custom Search Engine (CSE).

How can custom synonyms help?
Synonyms help by finding documents with relevant related terms and ranking them higher. Synonyms can help to alleviate the mismatch that often occurs between the queries that users type, and the actual words, phrases and concepts used in the documents being searched. Custom Search, of course, automatically takes advantage of synonyms used in Google Web search. In addition, Custom Search goes one step further: we allow you to explicitly define custom synonyms that are specific to your web site, community or topic of interest.

To illustrate situations where CSE custom synonyms can help, we created 2 CSEs that both search content from the Palo Alto Medical Foundation (PAMF). The first CSE does not have custom synonyms enabled, while the second CSE has a few custom synonyms enabled:

Terminology: The queries people use sometimes don't match up with the words and phrases used in the content being searched. During allergy season, for example, many people look for information on "hayfever", but the results without synonyms aren't that great since the web pages we're searching across don't necessarily use this specific term. However, if the technical phrase "allergic rhinitis" is added as a synonym for "hayfever", the results with the synonym are far better.

Acronyms: Acronyms often stand for different terms in different contexts. This is especially true within organizations, where acronyms are used frequently. In such cases, it may be possible to improve retrieval via the use of synonyms. Thus, "PAMF" refers to "Palo Alto Medical Foundation", and adding this synonym improves search results: searching for "PAMF" without synonyms gives only a few relevant results, while the CSE with the synonym returns more relevant results.

Community: Within specific target user communities, words or phrases have different usage and significance. When a patient is looking for "hearing doctor", adding the synonym "audiologist" to the CSE provides much better results while the results in the CSE without synonyms are not optimal. Conversely, if a doctor is searching for "somnambulism", she finds no results at all in the CSE without synonyms, but much better results via addition of the synonym "sleepwalking" to the CSE. Synonyms can therefore be used to improve the experience of specific classes of users of a web site.

How can I add synonyms to my Custom Search Engine?
The CSE administrator can control the set of synonyms used by uploading a synonym dictionary that is specific to the domain and website. The synonym dictionary can include alternate words or phrases for common search queries. The following steps show how to add the synonyms for the Palo Alto Medical Foundation CSE.

Download the existing CSE context file through "Control panel"->Advanced->"Download context"

Add custom synonyms to your search engine. The synonym dictionary is uploaded as part of the context XML file. Here are the synonyms we added for the above examples:


<customsearchengine>
 <title>...</title>
 <description>...</description>
 <context>
  <backgroundlabels>...</backgroundlabels>

  <synonyms>
   <synonymentry word="hearing doctor">
    <synonym>audiologist</synonym>
   </synonymentry>
   <synonymentry word="hayfever">
    <synonym>allergic rhinitis</synonym>
   </synonymentry>
   <synonymentry word="somnambulism">
    <synonym>sleepwalking</synonym>
   </synonymentry>
   <synonymentry word="pamf">
    <synonym>Palo Alto Medical Foundation</synonym>
   </synonymentry>
  </synonyms>

 </context>
</customsearchengine>

Upload the context file through "Control panel"->Advanced->"Upload context"

A few notes:

CSE synonyms are unidirectional, not bidirectional. Thus, a context file with
```
<synonymentry word="migraine"><synonym>headache</synonym></synonymentry>
```
defines "headache" to be a synonym for "migraine". However, if you also want "migraine" to be considered as a synonym for "headache", you need to add a separate SynonymEntry to the XML, as follows:
```
<synonymentry word="migraine"><synonym>headache</synonym></synonymentry>
<synonymentry word="headache"><synonym>migraine</synonym></synonymentry>
```

In the current version, the synonym dictionary can only be uploaded/downloaded as a part of the context XML file. We hope to make this much easier in the future.

We allow up to 500 individual synonyms for a given CSE.

Each word can have no more than 10 synonyms. If there multiple synonyms, the query will be expanded to include all synonyms uniformly.

If you are using custom synonyms in your CSE, we hope to get feedback from you about what improvements we can make.

Connecting with the Adobe Community with Google Site Search

2008-09-24T14:05:00.000-07:00

Posted by: Tom Duerig, Software Engineer

We love Google Site Search. And when working with our customers, we often discover new and interesting ways to apply our technologies to their business needs.

With Adobe's latest launch of Creative Suite 4, (which includes some of their most popular desktop applications, such as Adobe InDesign, Illustrator, Photoshop, Dreamweaver and Flash), we had the chance to work with their team to integrate Google Site Search into their product in a way that truly blurs the line between off-line and online content.

Adobe is using Google Site Search as part of Creative Suite 4's Community Help to connect suite customers to Help content on and off Adobe.com, including tutorials, technical support, online product help, videos, articles, tips and techniques, blogs, examples and other resources.

In chatting with Adobe Product Manager, Mark Nichoson, we learned that Adobe chose Google Site Search for a variety of reasons. First, we learned that user experience and consistency are essential for Adobe customers. After several studies, the Adobe Creative Suite Team concluded that their customers would use Google Site Search to access information on their products.

At the same time, the nature of Adobe Community Help is to align, share and offer the best available information online. Google Site Search allowed this to happen by aggregating more than 4000 URLs coming from the active members of their customer base (bloggers, partners, developers, webmasters and others).

This is just one of the ways Google Site Search is connecting customers and their users with the information they need -- whether they're within their applications or on their websites. We can't wait to see what unique Site Search implementations come up next!

Beefed-up developer guide

2008-09-17T12:12:00.000-07:00

Posted by: Kevin Gargoyle Lim, Technical Writer

We've updated the developer guide to include the more advanced Custom Search features, such as synonym expansion and integration with Subscribed Links. These features let you trick out your search engine and give your users a richer search experience. Synonym expansion lets you expand a user's search term (such as "running") to include its variants (such as "jogging" and "sprinting"), so users will not need to search for each variant. Subscribed Links enables you to directly answer your users' questions by promoting a specific result at the top of the results page. You can create your own result text and define sets of queries that would trigger subscribed link results.

The developer guide may not be edge-of-your-seat material, but it does now come with a freebie glossary and a search gadget that you can add to your iGoogle page. We're still tinkering with the documentation and bulking it up with more juicy tips, so keep coming back to check our progress.

Happy reading!

Delivering Custom Search to your applications

2008-09-09T14:44:00.000-07:00

Posted by: Mark Lucovsky, Software Engineer

One of the most exciting things happening with our APIs is the clever use of Custom Search engines (CSEs), blended results, and innovative uses of UI components, resulting in rich user experiences. I'd like to highlight a few interesting applications that use the AJAX API with Custom Search, and then illustrate how Custom Search engines are accessed in the AJAX Search API. With a little effort, it is possible to get a very different search experience than the standard iframe version of CSE results to embed into your web site.

Over the last couple of weeks, users found interesting information about the U.S. elections and the party conventions at our 2008 U.S. Election page. In the center of the page, there's a page element based on the AJAX API that is designed to deliver a large amount of very specific election-related news and information in a very compact form factor. The tab labeled "Blog Posts" is simply a search on a specific CSE -- the CSE itself is simple, it covers a dozen or so well-read political blogs. This page element demonstrates the ability to deliver this in a very small form factor that's easy to place on any page. You can do a "view source" on the page and you will see what I mean. The AJAX Search API blog provides more details on how the elections page element, included below, was constructed.

Another good example is VisualDx Health. This site delivers health-related information through a very sophisticated CSE. Under the hood, it's a straightforward application that takes advantage of the underlying API. Multiple search requests are fired off in parallel to create this powerful categorized display of results. Rolling Stone takes a similar approach, but this site uses news in addition to multiple CSEs to create a segmented search experience to tailor the content to its specific audience.

Let's take a quick look at how CSEs are accessed using the AJAX Search API.

Take a plain old web search URL like this

http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=barack%20obama

Add a new CGI arg called &cx to refer to an existing custom search engine

http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=barack%20obama&cx=010222979794876194725:pqldevwuapa

Check the responses -- it's pretty easy. If you prefer using JavaScript objects, something like this also works:

var search = new google.search.WebSearch();
search.setSiteRestriction("010222979794876194725:pqldevwuapa");
search.execute("barack obama");

You can dig deeper to view how these applications are built. Using Firebug, you can also take a look at the "Net" tab to focus on the JS request/response stream while performing searches.

The AJAX API team is working to deliver Google services through a RESTful JSON/JSON-P based system, as well as the large collection of JavaScript classes and high level page elements that developers use to build these very clever sites. The combination of the AJAX APIs and CSEs is a pretty powerful system. Learn more by reviewing our documentation and posting questions/suggestions on our developer forum.

Creative uses of Custom Search

2008-08-18T15:05:00.000-07:00

Posted by: Rajat Mukherjee, Group Product Manager and John Skidgel, Senior Interaction Designer

If you've been to the 2008 Summer Games site that we created to help you stay updated on the happenings in Beijing this month, you'll notice a Custom Search box in the upper right corner that will offer you results from a set of sites that cover the games. This customized search experience is similar to what many of our users have been doing with Custom Search - defining their own slice of the web to search.

Since we launched Custom Search, developers have found interesting ways to use the platform and the API (did you see our new developer guide?). One interesting way to build a customized search experience is by using linked Custom Search Engines (CSEs). With linked CSEs, you can create a dynamically defined search engine that can be updated automatically.

A cool application to create a personalized search over your del.icio.us bookmarks is described in the tutorial Build a Custom Search Engine using your Social Bookmarks. You can export your bookmarks to define a CSE and search across all the stuff you care about.

Many social bookmarking sites make this kind of application easy to build by providing tools for exporting bookmarks or by listing all bookmarks on one webpage. Other services provide similar functionality. Here's a screenshot from a CSE builder application that does this.

You can create a CSE on the fly using the MakeCSE tool. Provide the URL of a page that has a bunch of hyperlinks and MakeCSE will extract the links, construct a CSE, and provide you the search results instantly. As the links on the page change, the CSE is automatically updated.

Here are a couple of other interesting CSEs:

A CSE for your blog. Try out this Blogger widget that searches your Blogger blog and blogrolls automatically.
A CSE that uses our XML API within Google App Engine allows you to save search results or email them to friends and manage them in personalized lists. Here's a screenshot from the UXFind application:

Want to build interesting CSEs? Here are a few ideas:

CSE for bookmarked URLs that have a particular tag on del.icio.us, for example, http://delicious.com/tag/photography?setcount=100.
CSE 'for searching pages with links'. This would be great for webpages that have a lot of links, such as the directory category pages on the Open Directory Project website.
CSE that searches across URLs that your friends are embedding in their Twitter messages.

Let us know in your feedback if you've built interesting tools using the Custom Search platform. We'd also like to hear if there are things we can do to make Custom Search easier to use.

Spanking new developer guide

2008-06-11T14:14:00.000-07:00

Posted by: Kevin Grendelzilla Lim, Technical Writer

We created a new developer guide from scratch and moved it to code.google.com, where many other API documents reside. This means that the bells and whistles of Google Code are available to the Custom Search developer guide. For example, you can now search for information across multiple APIs (Not that we're bragging, but Custom Search powers the search on that website).

Fine, we'll admit that the new doc is not exactly a-thrill-a-minute, but it's definitely stuffed with more examples (and pretty pictures). The new organization, navigation, and search box make it easier for you to find information. The guide also discusses background information, explains complex concepts, makes recommendations, and points you to the right direction.

We're still tinkering with the doc and adding more stuff into it. We'll keep you posted about our progress.

Happy reading!

Custom Search at the core of Google Site Search

2008-06-03T01:47:00.000-07:00

Posted by: Matt Wytock and Vrishali Wagle, Software Engineers

Today, we announced Google Site Search, a hosted website search product that takes advantage of the Custom Search platform to offer high quality search to any website. Google Site Search integrates features that offer site visitors a search experience closer to what they're used to on Google.com. Here are a few of the new features:

Expanded coverage: We already search through all of your pages that are in the Google.com index. Tell us about more pages on your site by submitting a Sitemap. We'll crawl your Sitemap and add these pages to your custom search engine so that your search engine has the maximum coverage. (These additional pages will only be available to your custom search engine; your PageRank and Google.com rankings won't change in any way.)

Custom Synonyms: You can now define custom synonyms for your custom search engine. For example, you can define "cd" as meaning"certificate of deposit." When a visitor searches for "cd" on your search engine, we will return pages that contain either "cd" or "certificate of deposit." You can specify these synonyms in the XML definition of your search engine.

Date Biasing: Fine-tune the relevance of search results by specifying a bias for newer documents. We allow various levels of biasing, the highest of which approximately sorts by date.

Top Results Biasing: If you want the first N results to always match a refinement, you can specify that as a property of the refinement. We will try to fill up the top N positions with results matching that refinement before showing other relevant results.

We're not done with Custom Search and are always thinking about ways to add new features. Stay tuned, as we've only just begun.

Custom Search expands to AdSense for Search

2008-05-06T09:42:00.000-07:00

Posted by: Vrishali Wagle and Tom Duerig, Software Engineers

Since the launch of Custom Search, we've seen more and more webmasters, publishers and organizations enable high-quality search engines on their web sites and blogs to help their users find what they're looking for. Thanks to your continuous feedback, we've been able add new features and global support, resulting in even more growth.

Now, the Custom Search platform is expanding further by simultaneously improving our indexing for your search engine and broadening our reach to power AdSense for Search. AdSense publishers can now create Custom Search Engines (CSEs) and take advantage of the most popular customization features right within their AdSense accounts.

As you may know, Custom Search is built on top of the Google index, which means that pages that are available on Google.com are also available to your search engine. In addition, Custom Search now maintains its own index for enhanced coverage for web sites included in CSEs.

If you have pages that aren't currently being indexed in Google.com, you can let us know about these pages by submitting a Sitemap through Webmaster Tools, and pages in your Sitemap that aren't included in the Google index will be detected and indexed for your search engine. Improved index coverage is not instantaneous as it takes some time for the pages to be crawled and indexed. Please also note that this improved indexing only affects search results within the search engine you create, not your rank and indexing on Google.com.

We'd like to welcome all AdSense publishers to the Custom Search community! By growing our community, we're hoping to work with more of you to help users across the web find exactly what they're searching for.

Searching for Creative Commons licensed content

2008-04-23T17:52:00.000-07:00

Posted by: Tania Bedrax-Weiss, Software Engineer

If you've ever wanted to let your users only see content that is free to use and share (such as through Creative Commons), here are a few tips on how to do it. Assuming you've indicated the CC license in your pages, you can now specify that you want only content with specific licensing terms to show up in your CSE. We've made available four labels for CSE creators to use based on the CC licenses:

free_use_share (by-nc-nd)
free_use_share_commercially (by-nd)
free_use_share_modify (by-nc-sa)
free_use_share_modify_commercially (by-sa)

You can either use one of these labels as a background label to restrict all of your content to these terms, or you can reference these labels in the facet items section of the CSE specification and they will be exposed to users as additional refinements in your CSE. Note that these are filter labels; you cannot boost according to these labels.

As you generate content, please keep in mind that there are a number of ways Creative Commons licenses may be added to an HTML doc:

with <rdf:RDF>...</rdf:RDF> in the HTML head or body
using <meta name="DC.rights" content=uri of cc license>
using <a rel="license" href=uri of cc license>
using <rdf:RDF>...</rdf:RDF> in a comment

If your documents use any of these methods, we likely already have licensing information. See for example: http://joshua.smcvt.edu/linearalgebra/ that illustrates method number 3:

<a rel="license" href="http://creativecommons.org/licenses/by-sa/2.5/"> ...

and also illustrates method number 4:

<!-- <rdf:RDF xmlns="http://web.resource.org/cc/" ... >
<License rdf:about="http://creativecommons.org/licenses/by-sa/2.5/">...
</License>
</rdf:RDF> -->

See the file source for the complete text and don't forget to visit our Webmaster Central for more information.

Labels in Custom Search

2008-03-27T07:49:00.000-07:00

Posted by: Phil Liao, Software Engineer

Last month we demonstrated how to specify URL patterns in your CSE. We now explain another powerful concept in Custom Search: labels, which are also known as refinements. Labels are typically displayed above the search results.

Let's revisit our I Love Veggies search engine. When the user searches for "eggplant", we display some categorical refinements above her search results such as: recipes, nutrition, farmer's markets, and growing tips. When she clicks the "recipes" refinement, for example, our CSE emphasizes eggplant recipes in her search results.

To make refinements, do the following:

1. Create refinement labels.

To add a refinement to your custom search engine, go to the Refinements tab in the control panel, then click the Add Refinement button. Enter a refinement label and its associated keywords, and choose how to prioritize the labeled sites. You can enter one of our predefined refinements or create one of your own.

2. Tag sites with refinement labels.

Once you have configured your refinements, go to the Sites tab in the control panel. Choose some sites (URL patterns) using the checkboxes, then choose a label from the Label actions drop-down list. You can attach any number of labels to each site. For example, if a site includes nutritional information with its recipes, you could label it with both the "recipes" and the "nutrition" refinements. Labeling your sites properly enables your users to find sites that are relevant to each refinement. In addition, by showing meaningful labels under each search result, you enable your users to find similar results quickly. The Custom Search Help Center has helpful hints on how to choose good refinements.

And this is how to create refinements and assign their labels to your sites using the Custom Search control panel. We'll show you advanced refinement management using XML in another post soon.

Making the most with AdSense

2008-03-18T09:39:00.000-07:00

Posted by: Talia Brodecki, AdSense Product Marketing

If you're currently making money from CSE with AdSense, here are a few optimization tips to help you improve your current performance. (If you're not already earning money this way, scroll down to learn more about how you can start doing so, using your search box.)

1) Search box placement: Search boxes should be easy to find, typically under the header or on the left navigation bar, to encourage users to simply search for what they're looking for instead of spending time browsing through your content. Try to keep your search box in the same place on every page so users know where to return for future searches.

2) Two search boxes per page: For pages with a lot of content, we recommend placing two search boxes per page - one at the top of the page where users can immediately go to to find what they're looking for and one at the bottom of the page where users can quickly search for something else after reading the content on the page. You can track the performance of each by creating a custom channel.

3) iFRAME: By choosing to host the search results, you can integrate search results and ads within your site so users don't leave your site when they perform a search. If users don't find what they're looking for in the search results, they can easily click to another section of your site through your site's template.

4) Search box on search results page: Remember to place another search box on your search results page in the case that a user wants to perform another search.

5) Ad placement: For better monetization, we recommend choosing to place ads at the top & right of the search results for added visibility.

6) Appearance customization: Customize the search results with your site's colors to match the look and feel.

If you aren't using AdSense, we encourage you to sign up and get paid for every valid click on ads you're already showing. Just go to the Make Money link in the CSE control panel and fill out the short form to sign up for an account. If you already have an AdSense account, you can easily link the two to start earning money immediately.

Now then -- better go figure out what to buy with all that extra cash. ;)