It took me a few years to feel confident writing articles that try to bring something new (and avoid rewriting what TechCrunch or other CSEs-focused blogs like Comparison Engines or Comparison Shopping Engine Strategies can say). But slowly, the blog started to gain readers; and Twitter, that I use since early 2009, has been a clear booster. The main benefit is that I’ve developed some very interesting contacts in this small industry.

This 200th article is the opportunity to thank you all for reading this blog :)

Current state of product navigation

There are so many user behaviors when it comes to shopping: those who know precisely what to buy, those who take months to do product research, those who are impulsive, those who are eco-conscious… Still, CSEs often provide pretty “static” navigation to browse products whatever the user goal can be, relying on the commonly-accepted pattern that is faceted navigation.

Faceted navigation is indeed a convenient way to filter down a result set, considering the facets are correctly defined and the number of facets remain acceptable (there are obviously other aspects to think about, but let’s consider those two for the sake of argument :) ). This leads most CSEs to implement a static and small list of facets per category; and this can lead to a pretty frustrating product research for those users who don’t find the filters they need…

Another limit in the way CSEs implement faceted navigation is that they tend to only provides descriptive or technical facets. That implies to find a camera by brand, megapixels, optical zoom ; or a pair of pants by brand, color, material… While those facets are obviously mandatory, I feel they become too restrictive especially on fashion or home decoration where people are more impulsive than on electronics or computing universes.

CSEs have then developed other ways (but not as efficient as faceted navigation) to go beyond descriptive facets:

• Implementing user / expert review is a basic way to provide usage-oriented data to appreciate if a product fits user’s need. However, reviews  are rarely used to filter down products: they are usually only reachable from product pages.
• Buying guides provide interesting guidelines but are rarely integrated into the browse / search experience: they become quite difficult to spot in the site hierarchy.
• A nice emerging pattern is to provide shortcuts to the faceted navigation, in a way to start integrating buying guides and filters.

Beyond those classic patterns to product navigation, there are also some new types of navigation that are emerging, i.e:

• Geographic / local search for those CSEs providing a mix of online and offline shopping,
• Visual search, to find similar products.

PriceRunner providing shortcuts to its faceted navigation, using Q&A style:

Moving from descriptive features to usages

This is a topic I’ve discussed several times: I believe it’s time to enrich faceted navigation with filters related to product usage. Buzzillions and WisdomTap use the potential of user reviews to enhance product navigation with usage-related filters: a clever way that requires advanced text mining algorithms to be relevant.

Buzzillions has “Pros” and “Use” filters:

Without the need for such advanced algorithms, CSEs could rely on the “ratings per feature” to enhance faceted navigation. Wouldn’t it be nice for a user looking for his/her first digital camera to be able to choose a “type:compact” camera that is easy to use (“pros:ease of use”)?

Yahoo! Shopping US and ratings per feature:

Hunch uses another approach to provide usage-related attributes to products: its community creates shopping assistants with, obviously, a human touch that is very interesting.

Adjusting faceted navigation to the level of expertise of users

Talking about Hunch: Hunch doesn’t use faceted navigation, but an assistant-like approach relying on questions & answers. I think it’s an interesting way to quickly understand what is the goal of the user (product research, quick finding…). I imagine a user interface where assistant-like approach and faceted navigation would be mixed. Such interface could embed and replace buying guides, and quickly adjust itself to the user. It should be able to support any number of facets in such a way only facets useful for the user are displayed.

For buying a camera, this interface would be able to answer uneasy questions such as: “I want an easy-to-use compact camera, what are my choices?” (beginner, product research), “Should I buy an advanced Bridge camera or a SLR?” (intermediate, product research), “I want a Canon EOS for any weather – which model should I get?” (intermediate, narrowed product research), “Nikon 3D, Nikon 3Ds, Canon EOS 1D Mark III, Canon EOS 1D Mark IV – which one?” (expert, precise product search).

Choosing the facets to be displayed using YouTellMe and its very precise product database:

Aggregating expert content

There are now so many relevant data sources that provide facts, ratings, reviews on products that I foresee big CSEs will become more and more content aggregators to enrich their browse / search experience and meet user expectations. Today, such partnerships between content providers and CSEs are frequent; but usually the integration on the CSE side remains limited, mainly consisting on link exchange, when this content could be used to enrich filtering options. Some examples of such content providers:

• GoodGuide provides health, environmental and social facts about products, with a current focus on food & health/beauty categories;
• For food products, Zeer gives you nutritional facts;
• Snooth has an impressive wine product database with its own ranking built by its community

Summary / Final thoughts

If product browsing cannot be limited to faceted navigation (a good integration between browse and search is mandatory; cross-linking modules are important…), faceted navigation remains the right pattern to refine and filter down a result set; and an area where CSEs could innovate to better address user’s goals. Among those evolutions, I’ve discussed:

• The evidence that faceted navigation should be adjusted to the level of expertise of users.
• The potential of mixing assistant-like user interface and faceted navigation, to quickly understand user’s intent;
• The need to go beyond descriptive facets and explore how usage-related facets could be supported;
• The interest of aggregating relevant content to products, and exposing this content through faceted navigation.

The challenge is of course to mix all those “requirements” while supporting search / browse integration and advanced filtering options like geographic search or visual search through one interface. An exciting challenge in terms of user experience and data management I would love to work on :)

You may recall that towards the end of 2009, we began testing a new feature called AdWords Comparison Ads. This feature lets users compare multiple relevant offers with ease, and it provides advertisers with a new sophisticated and flexible cost per lead format. Speed is of the essence for many users, so we’ve made sure that Comparison Ads shows targeted offers in less than a second.

The initial test covered mortgage-related queries in the US. Today, we’re delighted to announce that the test is being extended to cover credit card-related queries in the UK and that some major issuers have signed up to be part of the initial run.

Comparison Ads improves the ad experience on Google by letting users specify exactly what they’re looking for and helping them quickly compare relevant offers side by side. There are no long forms for users to fill in, and Comparison Ads will not send advertisers any personally identifiable user information (in fact, we don’t send any user information at all unless the user explicitly applies for an advertiser’s offer).

While it seems difficult to spot ads with link to this new service, it can be reached directly at the following URL: http://google.com/comparisonads/ukcredit . This is a typical comparison service providing faceted navigation to filter results down, product pages with all details for a given credit card, and a feature-by-feature comparison tool. Simple yet efficient service following the classic Google look&feel.

The drawback of this service is that it currently lists only credit cards from few affiliated card issuers… Today, the service only lists 20 different credit cards, which considerably reduces its potential (to be compared to the ~170 credit cards proposed on MoneySupermarket). For sure, this is only a test and we could except more card issuers to participate if the product goes final.

What strikes me is the different approach from Google Product Search. While Product Search is free for merchants (with the temptation from Google to start having sponsored results on this service), Comparison Ads is only available for advertisers, following a business model similar to CSEs.

Another thought: Google has started testing similar service in the UK for loans in May 2008 (almost 2 years ago); we can wonder why Google is moving so slowly to roll out such service… Maybe thinking how it could fit in the grand scheme of shopping search and unification with AdWords?

Product page:

Feature-by-feature comparison tool:

Yahoo! recently talking about its search strategy, and how it can still innovate on search:

People don’t care about finding 5 million pages about Roger Federer, they want the one page with all the relevant information. If someone is looking for Black Eyed Peas, we want to show users a bubble with all the info they want – main site, most popular songs, etc. And we want to do this for all the bands, not just popular ones. And movies too, restaurants, etc. The web of things.

Microsoft is doing the crawling, we’re innovating how you display it.

So how is Yahoo going to lead this “web of things”? It’s like digitizing all your CDs – the hard part is building up a database of this information you’ve made. On the web the information about these recording is noisy. You just can’t manually edit metadata – you need killer machine learning and data mining.

While Yahoo! is talking about web search, this vision of the “web of things” also makes sense for shopping search. In a sense, this notion of “things” already exists in the CSEs world for years, mainly through the duality between products and offers:

• A product can be purchased in different stores, on different terms and conditions, at a different price and in different condition (new, used, refurbished…)
• An offer is a product being sold by one specific store, at a given price and in a given condition, with specific terms and conditions.

The problem with soft goods

The problem is that, for now, CSEs have done the easy part: creating or licensing product databases for hard goods (electronics, computing, music, books…). For soft goods (fashion, health & beauty…), where the number of product references is far more important, the cost to build and maintain such product database becomes too high and shopping engines have therefore built site navigation based on offers rather than products. While this approach allowed CSEs to provide an acceptable solution to browse soft goods for a rather low cost, it also induced some major problems in terms of usability.

The main issue is that search result pages on soft categories have two dimensions:

• The “product” dimension – Searching for “levis jeans”, you can have 3 offers for “levis 501” product, 2 for “levis 512” product…
• The “store” dimension – The 3 offers for “levis 501” certainly belong to different stores with different price, delivery cost…

This offer-based navigation works ok when searching for a specific product (i.e “levis 501”) as the first dimension is basically removed. But that becomes a problem when doing product research (i.e searching for “levis jeans”): too many results and information overload to take the right decision!

Another issue is that it is difficult to attach any content (like user reviews, coupons…) to offers due to their inner nature (short lifespan, linked to a specific merchant), therefore reducing the services CSEs can provide to shoppers on those soft categories. Among those services, the inability to provide a true price comparison service is overwhelming.

Last but not least, CSEs tend to mix products and offers for search result pages that trigger results from several categories, which can be confusing or frustrating for the users.

Some CSEs have tried to solve this problem

Several CSEs have experimented with product grouping on soft goods (Yahoo! Shopping, PriceGrabber…) using clustering technologies rather than product databases. Most of the time, those experiments have not been successful, as the number of offers being grouped and the quality of results remained low.

Among all those experiments, TheFind and Buzzillions have reached some really promising results. TheFind has launched its price comparison service last November; this service is still in beta, as the algorithms need to be tuned, but the clusters created are good enough to provide a valuable price comparison not only on soft goods but also on hard goods (without any product database)… Buzzillions (not a CSE stricto sensu) provides a pretty good product navigation on soft goods; still some “products” remain duplicated. Those two initiatives show that building product clustering on soft goods is not an utopia, but still requires some research and smart people and algorithms to become a reality!

An example of product aggregation on clothes category (Buzzillions):

Another challenge: product definition must be flexible

Another challenge that most CSEs failed to solve with their product database is that the aggregation of offers as products must be flexible enough to cope with the various purchase scenarii of users. I’ve already discussed this problem in a previous article, highlighting several examples (different bundles for a same camera, different color for a fridge model) and concluding that the way products are defined can lead to successful or inefficient search experience.

YouTellMe, a Dutch CSE, has implemented a really good solution to solve this problem for hard goods (relying on a very detailed product database that understands product variations). For soft goods (i.e: size and colors for fashion goods), some retailers have implemented clever solutions, but as far as I know, no CSEs have succeeded implementing such mechanism.

Product variations for Levi’s 514 jean at Amazon:

Building the next-gen product database

Thinking what could be the perfect CSE for 2010, I believe CSEs need to work again on their basics – how to organize and display search results – in order to bring their user experience to the next level:

• They need to reduce the clutter of browsing offers by providing product navigation on all categories, including soft goods. It’s time to build a real product search rather than the existing “offer searches” or “product/offer searches”,
• They need to ensure product definitions can be adjusted to user goals; which obviously will imply new challenges to let users filter results.

Such shift would require CSEs to find innovative approaches – surely a mix of manual and automated solutions – to build a next-gen product database, for instance:

• Using advanced “killer machine learning and data mining” (as Yahoo! suggests) to identify new products from merchant offers;
• Using crawling on manufacturers websites;
• Using open source databases or crowd sourcing solutions (like Freebase, ProductWiki…).

In a sense, I would say that such next-gen product database nearly exists: this is the Amazon product database, accessible for Amazon partners through web services. Amazon covering many categories and starting to provide CSEs-like features (marketplace, ProductAds), its database is surely the most comprehensive product database today, and can show the way for CSEs.

Final thoughts

The “web of things” is an appealing vision, but I doubt Yahoo will execute this vision on the shopping domain, considering the recent announcement related to Yahoo! Shopping. And Google Product Search and Bing Shopping still mix products and offers in their shopping results.

To differentiate themselves from those starting points that are search engines, CSEs need to (re)think what is their value proposition for users. Thinking about “things” (I’ve discussed “products” so far but think about brands / product lines / artists…) rather than offers / results has the potential to lay down some brand new user experience for shopping search.

For a shopping-oriented query, the “shopping” link, redirecting to Product Search, is displayed along with the “fewer shopping sites” / “more shopping sites” search options. Nice.

Previous design: (screenshot from Gizmodo)

New design:

Put simply, to change the tide, shopping engines need to become innovative again. At one time they were cutting edge in their ability to deliver multiple offers, additional merchant information and consumer feedback. Literally the CSE worked as a complete resource, aggregating purchasing data which helped merchants target consumers, and in turn which helped the consumer’s purchasing decision. Shopping engines need to find some of that creativity again. Whether this is done through dynamic social engagement, iPhone enabled scanning devices, or many other options is irrelevant. The point is that another large e-commerce shift is occurring, and if the shopping engines don’t get out in front of that change, then shopping engines face the prospect of becoming e-commerce relics of a by-gone area. That, would be too bad.

While the situation may slightly be different outside the US, I think ChannelDollars is right about its key message: innovation is mandatory for CSEs to survive. As I have some free time these days, I will start a series of articles on this topic, trying to define what could be the perfect CSE from the customer point of view. Some rough ideas for the moment:

• Reconsider what problems CSEs seek to solve for the end-users, and execute against this goal – a way for CSEs to increase user loyalty;
• CSEs must re-invest in technology, as tech innovation will lead to product innovation (TheFind currently leads the pack in this area);
• Technology is not enough, and CSEs must find the right mix between technology and manually-produced content (produced internally or by the community);
• I’m convinced CSEs could take benefits of social features, even if no CSE has been successful in this area yet (but Hunch shows there is some potential). How could CSEs smartly leverage the benefits of the social web?

Would you like to contribute to this series? How would you define the perfect shopping engine? Please leave comments :)