Custom Search Engine

Spanking new developer guide

2008-06-11T14:14:00.000-07:00

Posted by: Kevin Grendelzilla Lim, Technical Writer

We created a new developer guide from scratch and moved it to code.google.com, where many other API documents reside. This means that the bells and whistles of Google Code are available to the Custom Search developer guide. For example, you can now search for information across multiple APIs (Not that we're bragging, but Custom Search powers the search on that website).

Fine, we'll admit that the new doc is not exactly a-thrill-a-minute, but it's definitely stuffed with more examples (and pretty pictures). The new organization, navigation, and search box make it easier for you to find information. The guide also discusses background information, explains complex concepts, makes recommendations, and points you to the right direction.

We're still tinkering with the doc and adding more stuff into it. We'll keep you posted about our progress.

Happy reading!

Custom Search at the core of Google Site Search

2008-06-03T01:47:00.000-07:00

Posted by: Matt Wytock and Vrishali Wagle, Software Engineers

Today, we announced Google Site Search, a hosted website search product that takes advantage of the Custom Search platform to offer high quality search to any website. Google Site Search integrates features that offer site visitors a search experience closer to what they're used to on Google.com. Here are a few of the new features:

Expanded coverage: We already search through all of your pages that are in the Google.com index. Tell us about more pages on your site by submitting a Sitemap. We'll crawl your Sitemap and add these pages to your custom search engine so that your search engine has the maximum coverage. (These additional pages will only be available to your custom search engine; your PageRank and Google.com rankings won't change in any way.)

Custom Synonyms: You can now define custom synonyms for your custom search engine. For example, you can define "cd" as meaning"certificate of deposit." When a visitor searches for "cd" on your search engine, we will return pages that contain either "cd" or "certificate of deposit." You can specify these synonyms in the XML definition of your search engine.

Date Biasing: Fine-tune the relevance of search results by specifying a bias for newer documents. We allow various levels of biasing, the highest of which approximately sorts by date.

Top Results Biasing: If you want the first N results to always match a refinement, you can specify that as a property of the refinement. We will try to fill up the top N positions with results matching that refinement before showing other relevant results.

We're not done with Custom Search and are always thinking about ways to add new features. Stay tuned, as we've only just begun.

Custom Search expands to AdSense for Search

2008-05-06T09:42:00.000-07:00

Posted by: Vrishali Wagle and Tom Duerig, Software Engineers

Since the launch of Custom Search, we've seen more and more webmasters, publishers and organizations enable high-quality search engines on their web sites and blogs to help their users find what they're looking for. Thanks to your continuous feedback, we've been able add new features and global support, resulting in even more growth.

Now, the Custom Search platform is expanding further by simultaneously improving our indexing for your search engine and broadening our reach to power AdSense for Search. AdSense publishers can now create Custom Search Engines (CSEs) and take advantage of the most popular customization features right within their AdSense accounts.

As you may know, Custom Search is built on top of the Google index, which means that pages that are available on Google.com are also available to your search engine. In addition, Custom Search now maintains its own index for enhanced coverage for web sites included in CSEs.

If you have pages that aren't currently being indexed in Google.com, you can let us know about these pages by submitting a Sitemap through Webmaster Tools, and pages in your Sitemap that aren't included in the Google index will be detected and indexed for your search engine. Improved index coverage is not instantaneous as it takes some time for the pages to be crawled and indexed. Please also note that this improved indexing only affects search results within the search engine you create, not your rank and indexing on Google.com.

We'd like to welcome all AdSense publishers to the Custom Search community! By growing our community, we're hoping to work with more of you to help users across the web find exactly what they're searching for.

Searching for Creative Commons licensed content

2008-04-23T17:52:00.000-07:00

Posted by: Tania Bedrax-Weiss, Software Engineer

If you've ever wanted to let your users only see content that is free to use and share (such as through Creative Commons), here are a few tips on how to do it. Assuming you've indicated the CC license in your pages, you can now specify that you want only content with specific licensing terms to show up in your CSE. We've made available four labels for CSE creators to use based on the CC licenses:

free_use_share (by-nc-nd)
free_use_share_commercially (by-nd)
free_use_share_modify (by-nc-sa)
free_use_share_modify_commercially (by-sa)

You can either use one of these labels as a background label to restrict all of your content to these terms, or you can reference these labels in the facet items section of the CSE specification and they will be exposed to users as additional refinements in your CSE. Note that these are filter labels; you cannot boost according to these labels.

As you generate content, please keep in mind that there are a number of ways Creative Commons licenses may be added to an HTML doc:

with <rdf:RDF>...</rdf:RDF> in the HTML head or body
using <meta name="DC.rights" content=uri of cc license>
using <a rel="license" href=uri of cc license>
using <rdf:RDF>...</rdf:RDF> in a comment

If your documents use any of these methods, we likely already have licensing information. See for example: http://joshua.smcvt.edu/linearalgebra/ that illustrates method number 3:

<a rel="license" href="http://creativecommons.org/licenses/by-sa/2.5/"> ...

and also illustrates method number 4:

<!-- <rdf:RDF xmlns="http://web.resource.org/cc/" ... >
<License rdf:about="http://creativecommons.org/licenses/by-sa/2.5/">...
</License>
</rdf:RDF> -->

See the file source for the complete text and don't forget to visit our Webmaster Central for more information.

Labels in Custom Search

2008-03-27T07:49:00.000-07:00

Posted by: Phil Liao, Software Engineer

Last month we demonstrated how to specify URL patterns in your CSE. We now explain another powerful concept in Custom Search: labels, which are also known as refinements. Labels are typically displayed above the search results.

Let's revisit our I Love Veggies search engine. When the user searches for "eggplant", we display some categorical refinements above her search results such as: recipes, nutrition, farmer's markets, and growing tips. When she clicks the "recipes" refinement, for example, our CSE emphasizes eggplant recipes in her search results.

To make refinements, do the following:

1. Create refinement labels.

To add a refinement to your custom search engine, go to the Refinements tab in the control panel, then click the Add Refinement button. Enter a refinement label and its associated keywords, and choose how to prioritize the labeled sites. You can enter one of our predefined refinements or create one of your own.

2. Tag sites with refinement labels.

Once you have configured your refinements, go to the Sites tab in the control panel. Choose some sites (URL patterns) using the checkboxes, then choose a label from the Label actions drop-down list. You can attach any number of labels to each site. For example, if a site includes nutritional information with its recipes, you could label it with both the "recipes" and the "nutrition" refinements. Labeling your sites properly enables your users to find sites that are relevant to each refinement. In addition, by showing meaningful labels under each search result, you enable your users to find similar results quickly. The Custom Search Help Center has helpful hints on how to choose good refinements.

And this is how to create refinements and assign their labels to your sites using the Custom Search control panel. We'll show you advanced refinement management using XML in another post soon.

Making the most with AdSense

2008-03-18T09:39:00.000-07:00

Posted by: Talia Brodecki, AdSense Product Marketing

If you're currently making money from CSE with AdSense, here are a few optimization tips to help you improve your current performance. (If you're not already earning money this way, scroll down to learn more about how you can start doing so, using your search box.)

1) Search box placement: Search boxes should be easy to find, typically under the header or on the left navigation bar, to encourage users to simply search for what they're looking for instead of spending time browsing through your content. Try to keep your search box in the same place on every page so users know where to return for future searches.

2) Two search boxes per page: For pages with a lot of content, we recommend placing two search boxes per page - one at the top of the page where users can immediately go to to find what they're looking for and one at the bottom of the page where users can quickly search for something else after reading the content on the page. You can track the performance of each by creating a custom channel.

3) iFRAME: By choosing to host the search results, you can integrate search results and ads within your site so users don't leave your site when they perform a search. If users don't find what they're looking for in the search results, they can easily click to another section of your site through your site's template.

4) Search box on search results page: Remember to place another search box on your search results page in the case that a user wants to perform another search.

5) Ad placement: For better monetization, we recommend choosing to place ads at the top & right of the search results for added visibility.

6) Appearance customization: Customize the search results with your site's colors to match the look and feel.

If you aren't using AdSense, we encourage you to sign up and get paid for every valid click on ads you're already showing. Just go to the Make Money link in the CSE control panel and fill out the short form to sign up for an account. If you already have an AdSense account, you can easily link the two to start earning money immediately.

Now then -- better go figure out what to buy with all that extra cash. ;)

Custom Search on Google Help Centers

2008-03-04T15:09:00.000-08:00

Posted by: Christine Moschella, Online Operations

Custom Search Engines (CSEs) are powerful tools for searching over the pages of your site, as well as providing links to your favorite resources across the web. Because they provide great search functionality with the option to specify a subset of Google's web index, they're perfect for us to use in our Google Help Centers.

We've rolled out CSEs to many of our product Help Centers already, including the Custom Search Help Center, and we're also using it on our main Google Help page.

The Help Center search engines use several key features of CSEs such as refinements and labels to identify result sources, and Subscribed Links for certain help queries, like this one in the Analytics Help Center.

Because Help Center searches will now include content from product pages, developer documentation, Google blogs, and help groups, we hope that search will be an even more effective tool to help you find answers to your Google product questions. For example, failed searches - searches that don't return any results - decreased by 23% on the Analytics Help Center after we added a CSE.

Specifying patterns for your Custom Search Engine

2008-02-21T16:24:00.000-08:00

Posted by: Vrishali Wagle, Software Engineer

Creating a basic Custom Search Engine (CSE) is very easy. You enter a list of sites, select a few basic preferences, and you are done, right? But in fact there's more to Custom Search -- consider it a very powerful way of building your own search engine on top of Google search. You can exclude sites, add labels for drill-down and even change the ranking of results for your search engine. In this blog post, we look at the basic element of Custom Search - URL patterns

URL patterns specify the part of the web you want to search or exclude from your search. Custom Search is based on approximation algorithms that use these patterns to give you your customized results.

Consider the "I Love Veggies" search engine that we created. Here's how the "I Love Veggies" search engine made use of patterns effectively:

Be very specific. Use the longest possible pattern for specifying a site. For example, in the "I Love Veggies" search engine, we wanted to search all of www.goveg.com, so we added "www.goveg.com/*" as a pattern. But we wanted to search only the vegetarian part of the "allrecipes.com" site. So instead of adding all of "allrecipes.com/*" we added the more specific "allrecipes.com/Recipes/Everyday-Cooking/Vegetarian/*".
Specify multiple pages in a site with a "*" at the end of the pattern. If you specify just "www.goveg.com", Custom Search will search just the single page http://www.goveg.com. You need to remember this only if you are write your XML file of annotations directly. If you are using the Control Panel, it automatically adds the "/*" at the end for you, unless you indicate otherwise.
Sometimes, you might have a few hosts on a domain with the same path that you want to search. In our example, we wanted to search "mideastfood.about.com/od/vegetarianrecipes/*" and "indianfood.about.com/od/vegetarianrecipes/*". In such a case it is better to specify these patterns individually instead of a very general "*.about.com/od/vegetarianrecipes/*" as more specific the patterns, better the approximation.
You can only use the * in the hostname at the beginning of the pattern and it can only represent a full token. For example, "*.about.com/*" is a valid pattern and so is "*.food.about.com/*". However, "*ood.about.com/*" is not valid, nor is "food.*.about.com/*".

Keep reading this blog for more tips and tricks as we develop our "I Love Veggies" search engine. If you have specific questions or feature requests you can visit our Help Center or ask a question on the Discussion group.

Promoting useful information and web pages

2008-02-08T15:19:00.000-08:00

Posted by: Rajat Mukherjee, Group Product Manager

Webmasters often want the ability to promote specific information or web pages when users search for specific things via their Custom Search Engine (CSE).
Here are some examples where this is useful:

* You have a travel site, and want to draw attention to a spring promotion above the search results for keywords "hawaii", "maui", "kona". You want a nice image of the beach and the three most popular packages listed right on top.
* You've created a soccer search engine, and you want soccer fans to quickly get to the results of the Cup of Nations tournament in Ghana for queries [africa], [ghana], and [cup of nations]
* Your company just launched a brand new product, and you'd like people to know about it via a headline link to your product blog post when they search for older products in the same category.

We've always had the ability to do this in Custom Search (via a feature called Subscribed Links) but now, we've tried to make it a little easier for you.

When you go to to the control panel for your CSE, you'll see a new option in the "Preferences" section towards the bottom of the "Basics" tab. Selecting this option will enable your Subscribed Link to be triggered in your CSE for the keywords you've specified. If you don't already have a Subscribed Link defined, you can create one in minutes. Just specify the trigger keywords, the summary text you'd like displayed, and the URL of the target web page when the link is clicked.

When visitors search on your CSE using the special trigger keywords, your special link will show up right on top of the results, where they won't miss it.

You can manage your Subscribed Link via the Subscribed Link console. Check the developer documentation for advanced options. Remember that subscribed links display differently on your CSE (above the results) versus on Google.com (inline).

Displaying the most relevant refinements

2008-01-07T11:04:00.000-08:00

Posted by: Vrishali Wagle, Software Engineer

Over the past year, a number of you have asked to show only the matching refinements above Custom Search results. Today we're announcing that very feature: You can have as many FacetItems as you like in the XML definition of your CSE and control the number that display on a search page using this feature. Just add a "top_refinements" attribute to your context definition's CustomSearchEngine tag.

<CustomSearchEngine top_refinements="n">
</CustomSearchEngine>

Where n is the number of refinements you want to show.

For example, consider the context definition of a CSE I created: http://vrishali.googlepages.com/food_context.xml. It has sixteen FacetItems -- however, the top_refinements is set to 4. This means that it will display the top four matching refinements for every query. Check out the query chicken. Observe how the refinements displayed at the top are different for the query food safety.

Now consider the default implementation of the same CSE, that is without the top_refinements option: http://vrishali.googlepages.com/food_context_no_top_refinements.xml. All queries in this search engine will display the same refinements, for example chicken and food safety.

This feature is enabled for Linked CSEs as well as Google stored CSEs. To use it in your Google Stored CSE, download the context file in the "Advanced" tab of your control panel, add the "top_refinements" attribute to the CustomSearchEngine tag in your XML and upload the modified file.

If you don't want to display refinements at the top of the search results, turn them off by setting top_refinements="0". Your search results will still be labeled with the refinements that match.

As always, let us know what you like, or don't like, about this feature!

Ads now in harmony with search results

2007-12-17T15:35:00.000-08:00

Posted by: Tom Duerig, Software Engineer

Many folks have pointed out that all the results on the page -- both the search results and the advertisements -- should be appropriately customized so that the ads are in sync with the search results. We are happy to tell you that this is now indeed the case. Keywords that you use to tune search results will also be used to tune the ads.

To demonstrate the power of keywords for tuning ads, we've created 3 search engines. Note the differences in ads when we search for [leash] on all three:

A search engine about children, with the keywords [child children]:

A search engine about dogs, with the keywords [dog dogs]:

And, a search engine about surfing, with the keywords [surf surfing]:

So, keywords are even more important than before to your search engine. Make sure that you choose the keywords that provide the best overall search experience!

Custom Search in your language of choice

2007-11-20T05:30:00.001-08:00

Posted by Matt Wytock, Software Engineer

Since we launched Google Custom Search last year, webmasters and businesses around the world have been able use specific language controls to build customized search experiences. However, the Custom Search administration console was only available in English -- until now. Starting today, the Custom Search platform supports 40 languages in close to 80 countries worldwide.

Webmasters can use CSE to create tailored search experiences on community web sites; businesses can add hosted site search to their web sites; users can add search to their blogs and web pages; and developers can build search right into their applications with the Custom Search APIs. And there's no software to install or hardware to maintain. Custom Search engines can be built in minutes and are easy to customize and manage. You can also control the appearance of the search results to match the look and feel of your web site. Now, you can administer your favo(u)rite CSE in the language of your choice.

Custom Search is available as a free, ads-supported version, and through Custom Search Business Edition (CSBE). With CSBE, further customization of search results is possible using an XML API, and ads are optional. CSBE offers options for email and phone support. This launch also brings Custom Search Business Edition (CSBE) to your country. So if you're looking for fast, easy and relevant search for your website in your language, you'll be pleased with this launch.

As always, we'd love your feedback. Visit our User Group to tell us more about your Custom Search implementation.

Custom Search Results Overlay

2007-10-30T22:20:00.000-07:00

Posted by: Matt Wytock, Software Engineer

The ability to integrate Custom Search with your site is an important feature , and we recently added a new option to the Code tab that makes this integration easier. Using AJAX techniques, you can now get your search results in an overlay that appears on top of the page containing the search box. Here's an example.

Go to the Code tab of your search engine's control panel and select "Host results on a non-Google site" and "overlay" to get this new code.

Behind the scenes, this is powered by the AJAX Search API, which gives you complete control of your search results and is easy to use with Custom Search.

Custom Search for Salesforce Developers

2007-09-18T15:28:00.000-07:00

Posted by: Ryan Pollock, Product Marketing

I recently met with Adam Gross, VP of Developer Relations at Salesforce.com, to hear how developer.salesforce.com uses Google Custom Search. The company is building a community and knowledge base for developers on its Apex platform, and needs a search engine across a variety of content sites, including blogs and wikis. Adam says that Custom Search Business Edition is ideal for their needs, since it's easy to search multiple sites and access search results through an XML API. From the time he learned about Custom Search Business Edition, Adam says it only took a few days to install and create a look and feel for search results that matches the rest of the site.

To hear the story in Adam's own words, check out this video.

And if you're interested in hearing more about our work with Salesforce, we're at their Dreamforce conference this week, so come by to see us.

Better search results

2007-09-12T06:14:00.000-07:00

Posted by Tom Duerig, Software Engineer

We've implemented two frequently-requested features that we think will improve the quality of results in your Custom Search Engine.

1. Keywords now have a much stronger effect on Custom Search results. This helps give better targeted results and more flexibility than before. You can change your search engine's keywords in the "Basics" tab of your search engine's control panel. If you don't like the effect that keywords have your search results, you can also remove them.

Compare the results on this yoga search engine when looking for a "mat" with and without the keyword "yoga" --

with keywords:
http://www.google.com/cse?q=mat&cx=005239880967462049052%3A412vkh_dcla

without keywords:
http://www.google.com/cse?q=mat&cx=005239880967462049052:mdwcyik6y9m

2. For the XML hackers, FILTER labels now are effected by scores and weights just like BOOST labels. This simplifies rearranging your search results so that the content you care about comes up first. In order to prioritize a set of pages, give annotations for those pages' higher scores.

Compare the results of using scores in this vision search engine to prioritize *.ucsd.edu/* over other schools.

with scores (XML definition):
http://www.google.com/cse?q=camera&cref=http://www.google.com/cse/samples/ucsdvision.xml

without scores (XML definition):
http://www.google.com/cse?q=camera&cref=http://www.google.com/cse/samples/ucsdvision-noscores.xml

As always, feel free to give us your feedback in the discussion group.

Additional Capacity for Business

2007-08-27T16:18:00.000-07:00

By: Matt Wytock

In July, we launched Custom Search Business Edition, which those of you who are webmasters and product developers at companies can use to easily set up an ads-free search engine for your websites and completely customize the look and feel of search results through an XML API.

Earlier, the online checkout process allowed purchases for search of up to 50,000 web pages. A number of businesses asked us about site search for larger sites, so now we have expanded the capacity of Custom Search Business Edition. We've introduced two new plans that you can purchase online:

Search up to 100,000 web pages: $850 per year
Search up to 300,000 web pages: $2,250 per year

If you want to search more than 300,000 pages, contact us .

You can convert your existing Custom Search Engine to the Business Edition , or you can create a new Business Edition search engine.

Now CSE for businesses

2007-07-17T07:46:00.000-07:00

Posted by: Patrick Riley, Software Engineer

Ever since we launched Custom Search Engine (CSE), businesses of all sizes have been talking to us about specific features, more flexibility on presentation, and tighter integration with their websites and internal databases. We've heard you! Over the past year, we've launched powerful features, including improved language support, statistics and Linked CSEs. We've made Custom Search APIs available to developers, so that nifty search-enabled applications can be built atop the Custom Search platform.

Today, we're announcing Custom Search Business Edition (CSBE), an offering tailored to businesses that want more control over the search experience on their site. Search results can now be delivered through XML, offering further presentation control and flexibility to business users. Businesses now have the option to turn off ads. And CSBE provides options for email and phone support and Google branding is optional for business customers.

CSBE builds on the same foundation as CSE and offers all the benefits of Google search technology -- powerful algorithms, scale, reliability, performance and most importantly, world class search results.

You can directly sign-up for a new CSBE or convert your existing CSE (new Business tab on the control panel) to Business Edition for as low as $100 per year.

Search your blog world

2007-07-12T09:51:00.000-07:00

Posted by: Matt Wytock and Patrick Riley, Software Engineers

With our launch of Linked Custom Search Engines (CSEs) we made it possible for developers to construct CSEs on the fly.
Using the Custom Search APIs, you can now dynamically construct CSEs, and Google will automatically update and maintain them for you.

You may already know how you can use our APIs to create innovative search-enabled applications. Today, we're delivering a Google Search widget for Blogger built using the AJAX Search API. Configure this widget on your Blogger blog to immediately search not just your blog posts, but across all the link lists/blogrolls you've set up on your blog and the links you've made from your posts -- in other words, the world around your blog!

Try out the Search Box widget at Blogger's experimental site, draft.blogger.com. Once you've logged in and configured the widget, visitors to your blog will see a search box there. The search experience inherits your blog's look and feel, and is uniquely flavored around pages you've linked to from your blog.

To add the widget:

Edit your blog's layout.
Click on "Add a page element", and configure the "Search Box" widget.

Your link lists will automatically show up as optional tabs for your search; you can decide which ones you want to configure.
Preview the search experience and save your changes. Your blog is instantly custom-search-enabled.

If you're not using Blogger, you can still use our AJAX Search API to put Linked CSEs on your website.

Let us know what you think, and what CSE-enabled applications you're working on.

Custom Search Engine APIs

2007-06-26T14:21:00.000-07:00

Posted by: Matt Wytock, Software Engineer

A couple of weeks ago we blogged about a new feature and a new kind of Custom Search Engine (CSE) that you could create on the fly. Today, we thought we'd dig a bit deeper and describe the underlying infrastructure that powers this. With our new Linked CSEs, we are exposing the API to create and control CSEs.

Until now, you created a CSE either by using the wizard or by writing an XML file and uploading it to Google (via the "Advanced" tab on the control panel). To change any aspect of the CSE, you had to either use the control panel or upload the new XML specification. This imposed several limitations:

Creating and maintaining a CSE was a manual process.
It was difficult to create a large number of CSEs.
It was difficult to use other data sources such as iCal, RSS, Google Base, etc. to programmatically create CSEs.

The search box code for these CSEs (found on the "Code" tab in the control panel) includes a "cx" parameter with every search request (for example, <input type="hidden" name="cx" value="005946352831473999820:qs1idu8ptku" />), which specifies an internal identifier for the CSE.

Linked CSEs overcome these limitations. In short, you can now specify your CSE using a "cref" parameter that points to a URL, anywhere on the web. You update this URL at your end and don't have to upload it or edit your CSE using our tools. The URL can take arguments to produce dynamic CSEs, based on the current page, the current user visiting your site, etc. You can see this in action on our "on the fly" demo page: when you type "http://www.cs.berkeley.edu/~russell/ai.html" in the form text field, the javascript on that page constructs a "cref" parameter that contains http://www.google.com/cse/tools/makecse?url=http://www.cs.berkeley.edu/~russell/ai.html. This URL (visit it!) contains an XML specification for a CSE. You can use any script you want, or reference a static file, when creating your CSE. And there's nothing special about our makecse example script: we're hoping that our developers and the developer community will build many other such CSE-generating tools.

How does this work? With Linked CSEs, you designate a CSE specification URL with each search request (as a hidden form field in your search box HTML code). Google retrieves the CSE specification from the URL when your user searches in the CSE. We cache and refresh the results so that only the first search to your CSE incurs any delay. The flexibility to specify how your search engine should behave, just when your user is doing the query, using whatever data sources you want, opens up many possibilities:

You can use our makecse tool to generate CSEs from different sources of links:
- HTML: http://www.cs.berkeley/~russell/ai.html
- RSS: http://reddit.com/.rss
- ATOM: http://lipstadt.blogspot.com/feeds/posts/default
- OPML: http://medicalconnectivity.com/gems/blogroll.opml
You can combine multiple sources of links using our makeannotations tool and the <Include> tag. For example, its easy to create a search engine from the links on the front pages of techmeme, slashdot and digg.
You can write your own tools to produce <Annotations /> XML from other data sources such as Google Calendar or iCal feeds, Google Base or any other structured source of information.
You can automatically generate any number of CSEs, each possibly tuned to a particular user. For example, we've created a sample that builds a CSE from a user's digg.com friend network and submissions using the Digg API. Try it out and view the source. This makes use of two simple python CGI scripts:
- diggannos.py generates <Annotations> from the specified user's submitted stories
- diggcse.py generates <GoogleCustomizations> from the specified user's friend network. For each friend, it generates an <Include> element pointing to the appropriate diggannos.py URL

You can test any Custom Search Engine XML by going to http://www.google.com/coop/cse/cref and entering the URL. Putting a search box on your site is as easy as copying a small bit of HTML code and modifying the "cref" parameter.

Linked CSEs are a very big step for Google Custom Search. We hope you will find them as cool as we do. As always, thank you for your support and keep the feedback coming.

Finding your community

2007-06-22T11:14:00.001-07:00

Posted by: Vrishali Wagle, Software Engineer

Since we launched Custom Search Engines (CSEs) last year, we have seen search engines created on a variety of topics. Many of these search engines are open to volunteers and would love to have you contribute to them.

Now, it's easier for you to find all of these CSEs. Go to the Featured Examples page and type relevant words into the search box at the top of the page. We will search for these words in the fields for the search engine's name, description, keywords, and popular queries. If you're interested in finding a search engine to contribute to, search specifically for search engines that allow volunteers. For instance, if you're most interested in non-profit organizations, search only for non-profit search engines. You get the gist.

All of this searching over search engines is done via Google Base. It's the place to submit your content and make it searchable and accessible on Google. You can look at the hundreds of custom search engines that are related to cricket, podcasts and lyrics. We also have search engines on pet rats and many flavors of legal search.

Please note that only a small fraction of existing CSEs are included in this search. We've filtered search engines based on quality, recency, traffic and other metrics.

We hope this new Custom Search Engine search feature enables you and other CSE creators to find search engines more easily, and that it ultimately provides you with better access to information in your community.

Custom Search on the fly

2007-06-12T12:31:00.000-07:00

Posted by: Rajat Mukherjee, Product Manager

Starting today, there's a new feature that makes Custom Search Engines (CSEs) even easier to create and keep up to date.

You can now create a CSE by simply placing a small piece of tailored code on a page on your site. With that one piece of code, Google's search technology will automatically include in your new CSE all of the sites you have linked to from that page, creating a dynamic, powerful and tailored search experience really quickly. Moreover, your new CSE will update itself periodically to include any new links added to that page.

So, if you have a blog or a directory-like site and don't feel like listing all of the URLs you want to search across, you can leave the work to us. With this new feature we'll automatically generate and update your CSE for you. For example, try the query 'sculpture' on this CSE dynamically created from a page of links to kids museums or the query 'planning' on the search engine about Artificial Intelligence we created from the page of links at Berkeley.

Pretty cool, eh? We think so too. There are many powerful things you can do with this new feature, and in the near future we'll be talking about different possibilities. In the meantime, however, feel free to get your dynamic Custom Search Engine up and running. We'll be back in an instant.

Keep the feedback and great ideas coming!

Star Power

2007-05-17T10:46:00.000-07:00

Posted by: Jay Davies, Community Manager

If you've created a Custom Search Engine you've likely encountered the term 'URL patterns': the method for specifying sites and pages to include in a search engine. You can create a Custom Search Engine without ever giving URL patterns much thought, but there's added control that comes with understanding how they work and how best to use them. To that end, here are a few tips on creating useful URL patterns.

The simplest type of pattern is the URL of a specific page. You can use an asterisk (or star, in techie vernacular) as a wild card in your pattern to specify a swath of URLs by identifying their common characters. Here's how it works: rather than add each page of www.site.com to your search engine, you can merely add www.site.com/* and we will include every URL that begins with www.site.com. Similarly, rather than individually adding each subdomain of site.com, you can use the star to include them all: *.site.com will apply to www.site.com, server.site.com, etc.

Because most users adding sites to their search engines expect the subpages to be included, we've made the addition of the star the default behavior. This is what we mean in the Sites tab of the control panel when we say 'Include all pages whose address contains this URL.' If you leave that option selected, we will add a star to the end of whatever URL you provide. Additionally, if your URL is just a domain (like site.com/), not including a sub-domain (like www.site.com/ or server.site.com/), we will add a star at the beginning. This means:

site.com/ becomes *.site.com/*
server.foo.com/ becomes server.foo.com/*
othersite.co.uk/ becomes *.othersite.co.uk/* (yep, we handle two level top-level domains correctly)

If you'd like to control exactly where stars get added, you can select 'Include just the specific page or URL pattern I have entered.'

Another way to use patterns is to identify and distill common characters among a set of pages you'd like to include in your index. For example, if I know that all car review pages on my favorite consumer reviews site contain the words 'car' and 'reviews' in the URL, I can create the pattern www.reviewsite.com/*reviews*cars and include these pages. (I just need to be careful that only the pages I want contain these terms, or I may include more than I bargained for.) These types of patterns are especially useful when creating search refinements.

There is also one more application of the star, which is before a top-level domain like .edu or .ca. One use of this pattern is for when you'd like to refine a search by country: for example, you can create a pattern like *.ca and refine your search to Canadian web sites only. That said, you cannot otherwise insert a star into the second-level domain, such as www.*.com or www.dogs*.com. For a few more concrete examples of URL patterns check out our documentation.

The careful creation of URL patterns allows you greater control over the content of your Custom Search Engines, and can be particularly helpful when breaking up a site's content for refinements. Patterns may also help reduce the overall number of sites in your Custom Search Engine account, so look for redundant URLs that can be included more economically by the use of, shall we say, a little star power.

Bigger and Better Group

2007-04-10T10:19:00.000-07:00

Posted by: Christine Moschella, Online Operations

Good news, Custom Search Engine fans! We've redesigned the discussion group just for you; in particular, we've added categories to make information easier to find, including several sections dedicated to CSE features and options.

We invite you to join the Google Custom Search Help Group at http://groups.google.com/group/google-custom-search to ask questions, find answers, and lend your expertise to other Google Custom Search Engine users. Our own Custom Search Guides will also be posting to this group from time to time to help facilitate the discussion with tips and clarifications.

As for the original Co-op group, we've moved it into a subcategory of the new Custom Search group and archived the posts. You can still search for topics and read posts, but you won't be able to post new messages to the group. We hope you'll find the right category for your questions in the new Custom Search group.

Before posting to the group, try searching for the answer to your question in the FAQ or in the group archives. Also, please review the Custom Search Help Charter for more details about the group, including posting tips and etiquette. We hope to have you as an active group member soon!

Show popular queries on your site

2007-03-21T08:57:00.000-07:00

Posted by: Matt Wytock, Software Engineer

We've just added the ability to show popular queries for your Custom Search Engine, so that your users can see what others have been searching for on your site. Popular queries are updated daily, and the code snippet dynamically shows the most recent data.

To show your CSE's popular queries, go to the "My search engines" page (after you've signed in) and click on the "statistics" link. Cut and paste the popular queries code where you would like them to appear on your website. You can also subscribe to the feed for your CSE popular queries using your favorite feed reader -- the link to the feed is also available on the statistics page.

Better branding part II

2007-03-06T15:41:00.000-08:00

Posted by: Jennifer Hyman, Product Marketing Manager

In tandem with the search results branding changes, we've just updated and implemented new search box branding for CSEs as well. You can now select from 7 different search box branding options to fit the look and feel of your site. You'll find these different branding options on the Code page in your Custom Search Engine Control Panel. If you were previously using the watermark branding option that sits within the search box, we have automatically updated this logo to match the new updated branding.

We're always working to improve and enhance CSEs look, feel and functionality, and hope you agree that this new branding is a step in the right direction. To learn more about CSE branding, visit our branding guidelines. And as always, please keep the feedback coming.