flickr.com/roxycraft

At the heart of every great enterprise search user experience lies a clear and concise information architecture. Why – because search without structure is nothing but a disheartening solitary struggle to make sense of chaos. In order to implement a successful and sustainable search information architecture, you need to identify and design around relevant information entities, categories and facets. Categories in particular have a special role to play in the forming of search information architecture, one which reaches far into the following concept development and interaction design.

Why am I so geared-up about categories? I believe that design transcends technology, and that a well-designed (and possibly low-tech) enterprise search provides a better return on investment for businesses aiming to provide employees or customers with the best possible information access. And dividing by categories is one of the most effective ways to design coherent and noise-free search results, addressing problems that can’t be solved with relevancy alone.

Entities are the information archetypes we are seeking, pieces of information that documents are merely containers for. In a corporate setting examples of such entities may be employees, customers, projects, case files etc. Categories are natural and meaningful groupings of these entities, ideally satisfying several criteria that we’ll be looking into shortly. Facets are useful dimensions for refining the selection of entities, and groups of facets usually cluster around categories. Following our corporate examples, facets for employees may be years of employment, department, professional skills etc.

When you’re working to discover a suitable information architecture for an enterprise search solution, be on the lookout for categories that satisfy the following 5 criteria (in decreasing order of importance).

1. Matching the Users Mental Model

Categories in Google Search Options

As humans we’re masters of categorization. We do it all the time, and it’s how we make sense of the world. When I see a dog, I instantly know that it is also a mammal, that it’s name is Fido, that it has an owner who’s name is Bob and that Bob is also my neighbor. I know this because previous experience has allowed me to form the categories dog, mammal, Fido, dog owner, Bob and neighbor, as well as the meaningful relationships between them, together making up a mental model. (I realize that it may be more correct to call this model an ontology, but I’ll stick to mental model for now.)

Mental models hold true for digital information as well, and search result categories should match the mental model of the user. An information scent is then provided to the user, helping her anticipate the consequence of choosing to see search results from just one particular category, and building her confidence in the effectiveness of doing so. Card sorting is one way to discover with the help from users, what the natural and meaningful categories should be for a particular domain and enterprise search solution.

One good example of search categories have recently been unofficially launched by Google, as an improvement to their also-quite-recent search options. Categories like images, news, books, and maps are easy to comprehend, and it’s a good fit to our mental model of what is out there on the Internet.

2. Effective Disambiguation of Queries

Query Disambiguation by Bing

Search terms may be ambiguous, meaning that the user hasn’t provided enough information about the exact intent of the query. Does dolphins refer to Miami Dolphins (an American football team), or marine mammals? In the spirit of HCIR and exploratory search, it makes sense to simply ask the user to clarify the confusion – to disambiguate the query – rather than just guessing. Effective disambiguation is then the starting point for further refinements of the search results.

Bing provides a dynamic set of category refinement for most queries, capturing some of the different word senses for the search terms. The categories are semantically related (mined from search logs I believe) and therefore give rise to meaningful disambiguation.

3. Unique Facet Combinations

Facets describe what is typical about a type of documents or information entities, and which attributes they have in common. All cars have a make, a model, a color, a mileage etc. All employees have a title, a department, a supervisor etc. These attributes represent facets of that it means to be a car or an employee, and they are effective tools for search results refinement and clustering.

Entities described by a distinct set of facets should be separated from other entities described by different sets of facets. Using categories is a natural way to achieve this separation in a search user interface. Cars don’t have a department like employees do, and employees don’t have a color like cars do (it would be highly unethical to filter employees by color, at least). What happens then, if cars and employees are mixed together in the search result, and the user refines by department? All cars disappear, contrary to the impression that cars and employees alike are governed by the same department facet.

It’s better to avoid this possible confusion by separating the categories, and to show unique facets only when the user has chosen a particular category. Common facets can be shown all along, of course.

4. No Need for Cross-Comparison Between Categories

Categories, like any form of tabbed navigation, has at least one major drawback. By physically separating entities in the user interface, comparison across categories becomes difficult. If the user needs to compare features or prices between different makes and models of washing machines, do not organize these items in separate tabs. Items must be shown side-by-side if effective comparisons are to be made. Otherwise the user is forced to pogo-stick between tabs or dialogs.

Same applies for categories in search results. Do not place entities into tabs that need to be compared, like employees working in different departments or customers from different countries. God says that the only right use of tabs in user interfaces are for alternative views of the same information. That makes it sound like tabs should be great for alternative list views and map views in classifieds search results. I won’t dive into that problem right now, and I hope to write more about it some other time.

5. Even Distribution Between Categories

The utility of search result refinements (including categories) increase with even distribution of documents/entities across categories. With an even distribution, each category holds roughly an equal number of items. That makes each category an effective segmentation of the result set. Two categories with a distribution of 99%-1% is not very effective. If you select the first category you still got 99% of the search results to sort through. A 50%-50% (or any other even distribution) is generally preferable, provided that this constraint does not violate the previously mentioned criteria of good match with mental models, effective query disambiguation, unique facet combinations and no need for cross-comparison between categories.

If you found this post interesting or helpful, you may want to read Enterprise Search – Information Accessibility by Design or Technology?

Written by Vegard Sandvold

Facets for used car classifieds like I would design them.

Facet design may seem like a trivial task at first. After all, it’s just a list of links representing a range of facet choices. By clicking on one choice after another users simply refine and narrow down the search results. Sort them by popularity (frequency) and show those neat hit counts on each facet choice, and you collect instant bonus points for improved usability and information scent. It’s not a lot more to it, right?

Wrong. You can and should put a lot more consideration into how you give shape to facets, in terms of both information design and interaction design. Poor facet design hurts. It alienates and confuses users by disregarding established mental models and overloading them with information. Good facet design on the other hand, makes use of web design conventions and familiar vocabulary to increase affordance and help users anticipate the effects of their actions.

It’s probably true that real knowledge of what works and what doesn’t in design should be based on empirical studies and hard evidence. But I also believe that best practices and reasoning takes us a long way towards something that works better. Design patterns, evidence-based or not, is a great asset for us all, and great things should be shared. For what it’s worth, I would like to share 3 of my own favorite design patterns for faceted search. I have good experience with these, and I hope they may serve you just as well.

Pattern 1: Multi-select with check boxes
A facet design with check boxes is ideal for multiple selection within a facet, like when you’re looking to buy a car and want your search to target a few chosen manufacturers. A facet with check boxes lets you choose all these in one go, saving you both time and frustration.

You should use check boxes for multi-selection facets because:

• It’s a strong visual design element that tells users by convention to “select as many of these as you like”. Choices are implicitly combined with a logical OR, finding items matching one or more choices.
• They work well for disjunctive (mutually exclusive) categories when it’s necessary to cross-compare items from different categories. Tabs or links, on the other hand, will not let you cross-compare.
• They support concept composition, allowing users to form idiosyncratic concepts like “Japanese cars” or “Reliable used cars” without a need for prior classification by the content editors.
• You avoid tiresome pogo-sticking between categories like you would have to with regular link list facets.

You should not use check boxes when:

• Don’t use check boxes for facet that are very large (50+) or hierarchical. Multi-selection with check boxes doesn’t scale with size.

I also add a few extra details to my multi-selection facet designs:

• I include an extra set of radio buttons (Show All/Choose Makes) to avoid confusion when all check boxes are initially empty (saving the diligent user who concernedly checks all the boxes for unnecessary work).
• I initially present a short teaser list (7±2) sorted by hit count. A link to “More choices” reveals a longer list of facet values sorted alphabetically (if another sorting criteria isn’t more suitable).
• I let the facet value hit counts indicate how many search results you add/remove by checking/un-checking an option.

Pattern 2: Single-select with radio buttons
Sometimes it’s better to think of facets as a way to toggle between different views of the result set. Examples could be a facet for low, mid and high-priced cars, or a facet for mileage ranges. These options are mutually exclusive (a car can’t have both a low and a high price, or both low and high mileage) and the number of options are few. With radio buttons it’s painfully obvious to the user that just one choice can be made at a time, and it’s reassuring to know that radio button choices can easily be changed.

You should use radio buttons for single-selection facets because:

• They work well for a small number of successive inclusive or overlapping choices, like price, date and other numerical ranges.
• It’s a strong visual design element that tells users by convention to “select either this or that”.
• Undo is really easy, since all options are visible all the time.

You should not use radio buttons when:

• Don’t use radio buttons for facets of even moderate size (9+), since long lists of options take up a lot of screen real-estate. You may also run the risk of choice overloading the users.
• Don’t use radio buttons when items need to be compared across categories. Use multi-selection with check boxes instead.

A few details I make sure to include in my single-selection facet designs:

• I include a default “Show All” option in the list that implies not having made a choice.
• Even options with zero results are visible, but disabled.
• I show facet value hit counts that indicate how many search results each choice will produce.

Pattern 3: Repeated selection with link lists
The dominant design for facets is by far the link list. In this example we have a list of features you may want in a used car. Each successive click on a facet value adds a constraint to the search, helping you find cars that match all the criteria that are important to you. Each selection also reduces the number of options left to choose from. On the upside, this interaction style takes you quickly from a larger general result set to a smaller specific one. The downside is pogo-sticking.

You should use links for repeated selection facets because:

• They work well for large facets with overlapping categories, both flat and hierarchical. Choices are implicitly combined with a logical AND, finding items matching all choices.
• Links are safe – everybody understands what a link is (not everybody understands what facets are, however, but that’s a different discussion).

You should not use links when:

• Don’t use links for facets if check boxes or radio buttons (or drop-down lists or tabs) suit your problem better.
• Facets with links have one major drawback – undo is less obvious as options disappear. Choices you make eliminate choices you haven’t made, since those options are no longer available and visible.

Some details I make sure to add to my link list facets:

• I include a “Show All” link that removes all choices made for the facet. A positive wording may encourage users to explore this feature, as opposed to a more negative “Remove Choices”.
• I provide clear visual feedback on chosen values (bold face and check mark graphics).
• Hit counts for chosen values are usually equal to the number of search results, so I simply hide those.
• I sort the teaser (7±2) on hit count, and will perhaps change the sorting to alphabetical for the full list.

More search design patterns
Peter Morville has a nice collection of search pattern screenshots. Marti Hears has published her book Search User Interfaces online for free. Rumours has it that Endeca is planning to release their UI Design Pattern Library sometime soon.

If you found this post interesting or helpful, you may want to read Search Design Patterns for Ideas and Inspiration

Written by Vegard Sandvold

flickr.com/kevinomara

Facets add value to the search user experience by helping users navigate the usual ranked, best-first list of documents in a search result. The quality of faceted search, however, is at risk when search result precision is traded for recall. Users should ideally be able to find all documents relevant for a given query (high recall), and nothing other than these relevant documents (high precision). Balancing precision and recall is a constant challenge for enterprise search practitioners, since it’s notoriously difficult to achieve enough of both at the same time. When a trade-off has to be made, it may seem safer to err on the side of high recall. Then nothing of potential interest is left out, at least. But we shall see that high recall plays tricks with faceted search, forcing us to reconsider this assumption.

By design, facets and facet values summarize the entire search result set, assigning hit counts to each unique facet value. A Brand facet may tell you that the search result contains 15 Canon cameras, 14 Nikon and 7 Sony cameras. If you want to refine you search to target a particular brand, you simply choose the corresponding facet value.

flickr.com/morville

It was while reading a recent book on faceted search by Daniel Tunkelang that I came across the answer to a question that has troubled me for some time. What does facet values and their hit counts really say about the corresponding documents and their relevance to your query? How can you know up-front whether you’re making a good refinement choice or not? It defeats their purpose if facets and facet values are poor indicators of relevance, and I would like to know how facet ranking and presentation effects the user experience. Are facet values with high counts always more relevant than those with low counts?

With ranked retrieval (as opposed to set retrieval) documents are scored according to how well they match the user’s query, and the documents in the search result are ranked on this score, sorted from highest to lowest. A query that favors recall may cast a wide net, allowing the user to search freely through all parts of the document, perhaps with linguistic processing and synonym expansion applied to the query. Such a query will hopefully retrieve most of the interesting documents, but it may also dig up a lot of documents that are not particularly relevant.

It’s usually safe to assume that ordinary relevance ranking will banish the lesser relevant documents to the dark depths of the search result, well hidden from all but the most insistent users. Faceted search is played by a different set of rules, however, and low ranking documents may very well contribute significantly to the facets seen by the user. Facet values are usually sorted on hit count (if not alphabetically or hierarchical), but this ranking does not necessarily reflect the relevance ranking of the documents themselves. On the contrary, a facet value with a high count is not more relevant if it represents many irrelevant documents, as the case may be with a query that favors recall.

Knowing about the hidden menace of mindless recall, we can take some measures to ensure a satisfactory faceted search user experience:

1. Increase precision by restricting queries to specific document fields, and limit the use of linguistic processing. It may not be necessary to search everywhere with full-blown synonym expansion and phonetic normalization of the query. In short, err on the side of high precision instead of recall if possible.
2. Restrict facets to summarize only the most relevant part of the search result. Computing facets from e.g. just the 4000 highest ranked documents may dampen the noise introduced by lesser relevant documents. In FAST ESP, this technique is referred to as shallow navigators. Others may call it hedging.
3. Rank facet values according to a relevance score based on document relevancy. I imagine it would be possible to compute a utility measure similar to tf-idf suitable for ranking facets, and that this measure would favor facet values originating mainly from high ranking documents. I admit this is speculation on my part, and mostly off the top of my head, and some readers may even tell me that this feature is out-of-the-box in their favorite enterprise search software.

Finally, Faceted Search: The Book is in my opinion a must-read for all enterprise search practitioners. I give it my warmest recommendations.

If you found this post interesting or helpful, you may want to read Enterprise Search – Information Accessibility by Design or Technology?

Written by Vegard Sandvold

flickr.com/jkim1

Okay, so the website is up, I started blogging, and I’m promoting my blog on Twitter, which again updates my Facebook status. I connect with interesting and like-minded people on LinkedIn, who all spend too much time online connecting with interesting and like-minded people on LinkedIn, just like I do. Now, in the interest of further self-promotion and personal development, what more can I do to increase my web presence?

I want to write an e-book about search user experience, based on some of my latest blog posts (and all the great discussions they have sparked). I started writing this summer, and that is partly why blogging has been a bit slow for me. I hope to get the e-book done in a few weeks time. Meanwhile, I would like to give you a little LOOK INSIDE! the text I have titled 4 Kinds of Search User Experience.

The text should have some real-world use cases as examples of the 4 kinds of search user experience. E-commerce is a good, but obvious example of Diligent Search. Please leave a comment if you want to make a suggestion.

Chapter Outline – 4 Kinds of Search User Experience

The Search User Experience Manifesto
Search is about more than fast algorithms, beautiful code and large indexes. Relevance is in the eye of the beholder, and user experience design is the ultimate frontier. I believe that design transcends technology, and that a well-designed (and possibly low-tech) enterprise search provides a better return on investment for businesses aiming to provide employees or customers with the best possible information access.

A Model of Search User Experience
Part of my job as an information architect is to correctly align user needs and business goals with available technology and resources. A poor choice of technologies or design patterns may very well cripple the entire user experience. We’ll see how 4 kinds and 2 characteristics help us understand the particular strengths and weaknesses of various search concepts and technologies, as well as their applicability to real-world use cases.

Kind 1: Plain Search
The Plain Search experience is well-suited for known-item search, where information needs are easily resolved with basic query/response behavior. Sometimes all you have are a few keywords, and a ranked best-first list of 10 blue web page links is just what you need. Less can be more, but expect difficulties with precision/recall trade-offs and ambiguous queries, as well as excessive query re-formulation if the user is unsuccessful at completing the search task.

Kind 2: Superficial Search
The wisdom of the crowds is something of a holy grail, and tapping into the behavioral patterns of millions of users have brought fame and fortune to Amazon and YouTube, and recently also to Twitter. Recommendations and real-time search leverages user behavior to feel the pulse of communities, and to surface popular and current material, even as events unfold. Superficial Search is very efficient when you don’t need to dig deep into the information space, but it’s vulnerable to feedback loops and other kinds of popularity bias.

Kind 3: Ingenious Search
The Ingenious Search experience improves upon the weaknesses of Plain Search with a model-knows-best approach. Like rubbing on Aladdin’s lamp, sophisticated algorithms automatically disambiguate query intent and extract semantic structures from content. Direct answers are provided instead of documents in an attempt to increase precision. Clever algorithms can be said to be cost effective, but they’re comparably more difficult to implement and execute well. Also be aware of awkward query formulation and high expectations on behalf of the user.

Kind 4: Diligent Search
It’s sometimes easier to ask than to guess. Diligent Search regards the human intellect as a better judge of relevance, and empowers users to increase precision by refining both query and content themselves. Faceted search, a common feature on modern e-commerce sites, enables efficient exploration of the information space (shopping) by assisting users in iteratively expressing their information needs. Choice is great, but don’t get carried away. Having too many options can be paralyzing and counter-productive for the user.

Objections
I expect some readers to disapprove of how I have grouped, named and described search-related concepts and technologies in this text. That’s alright, and I’ll try to meet you half-way by reproducing and commenting on some of the constructive feedback and criticism I have received already from engaged and enlightened readers.

Resources
This is a collection of blogs, books, e-books and pattern libraries I find useful in my work with search user experience. I hope it may serve you just as well.

Written by Vegard Sandvold

flickr.com/waisian

When working with search user experience, it’s often necessary to discover a suitable information strategy for an organization or business. An information strategy can potentially have a huge influence on how the search for information is facilitated. Are the users in question generally seeking precise answers to quick questions, or are they seeking to explore a broader selection of opportunities? Question answering will generally require a more ambitious use of technology, while exploration and discovery places greater focus on communicating with the users. Perhaps an early decision to acquire software with semantic analysis capabilities will lead the project towards greater use of technology, while users studies may reveal that users are keen on spending time researching the cheapest flights to their holiday destination. Understanding the integration of business goals, user needs, and technological possibilities is in any case vital for a successful and sustainable information strategy.

Two recent articles in Technology Review reflect some of my latest writing on 4 different approaches to search user experience. The article by Daniel Tunkelang entitled “To Search, Ask” is very much in line with what I like to call Diligent Search, while the article on Wolfram Alpha is describing something closer to Ingenious Search. I find these articles very interesting, and I believe you will think so too.

Daniel Tunkelang (CTO, Endeca), a strong believer in faceted search and set retrieval, and an advocate for HCIR, prefers search engines that are more like conversational librarians. Rather than guessing what the users need, these systems provide users with opportunities to clarify and elaborate their intent. If the engine isn’t sure what users what, it just asks them.

Read Daniel’s article at MIT Technology Review →

Stephen Wolfram, the physicist and maker of Mathematica, has recently launched the computational knowledge engine Wolfram Alpha. In response to questions, Alpha is meant to compute answers rather than list Web pages, and pushes technology to the limit in order to anticipate what the users need. It attempts to achieve this by means of a constantly expanding collection of curated data sets, an elaborate calculator, and a natural-language interface for queries.

Read the article about Wolfram Alpha at MIT Technology Review →

You may also want to read more about my views on mediating information and information accessibility.

If you found this post interesting or helpful, you may want to read Wolfram Alpha – Re-Inventing the Command Line

Written by Vegard Sandvold

What is your name?

The hype has passed, and it’s clear that Wolfram Alpha isn’t the new Google killer as initially anticipated. But what is this new computational knowledge engine, and how shall we use it? It seems to me that we have been given some sort of a command line tool for a comprehensive information database. The search box acts as a peeking hole into Wolfram Alpha’s wonderful inner world of facts and figures, giving us a sense of what’s hidden inside there, while at the same time making everything slightly inaccessible. Like with a command line, you have to know what you’re asking for, and you must know how to phrase that question in a way Wolfram Alpha understands, in order to get a decent answer.

Wolfram Alpha is perhaps less of a search engine, and more like a question answering engine. Being something other than a search engine may be a strength in the fight for survival alongside Google, Yahoo and Bing, but it also means that the general Internet population must learn how to use this new tool. And that doesn’t happen overnight, especially since we’re already so accustomed to the quick and dirty web search. Wolfram Alpha can be both tedious and puzzling to interact with, and must therefore provide some serious extra value if it expects us to make the extra effort.

Your wish is my command.

I’m comparing Wolfram Alpha to a command line, and if you’re not a geek, you may not know what a command line is. In short, it’s a way of giving instructions to the computer, like deleting a file, by typing text commands into a special window, instead of clicking and dragging the file icon to the trashcan with your mouse. The command line is loved by many for it’s efficiency (I do!), but it’s also an interaction paradigm hampered with some serious poor usability issues. Jono DiCarlo of Humanized does a great job at summarizing these issues in his post about the problem with language-based interfaces. Here’s why he believes command lines are inherently difficult to use:

• Not discoverable: There’s no guidance given to a first-time user. You type some letters and nothing happens: it feels like shouting into a void. If you don’t already have the basic commands memorized, there’s no way to figure out what they are.
• Cryptic names: Whether for historical reasons or for brevity, the standard names of commands, programs, and locations are all called stuff like ‘tar’ and ‘mkdir’ and ‘/usr/local/bin’. Because these names are unnatural and unfamiliar, they have to be learned by rote.
• No feedback: I just entered a command and all I got back was a blank line! It worked, but what did I just do?
• Options are hard to remember: Does the ln command take the source file first or the destination file first? What does the -z option on tar do again?
• Really easy to make mistakes: One wrong character and your innocent command is transformed into a ruthless file murderer. And there’s no undo!

Let’s look at each of these misfeatures in turn, and see if any of them apply to Wolfram Alpha.

Not Discoverable
Not entirely true for Wolfram Alpha, since the welcome page provides a multitude of helpful hints on how to get started using the engine. There’s examples of mathematical formulas, geographical locations, dates, stock quotes, demographic data and other computations Wolfram Alpha is capable of handling. But even with this collection of well-chosen examples, Wolfram Alpha’s special flavor of knowledge computing feels a bit like a guessing game. And that makes it difficult to learn how to type in the right questions.

The normal rules of web search (like typing in whatever is on your mind) don’t apply here, and feeding Wolfram Alpha the same kind of search input as you would to any other search engine seldom works. Considering that Google sees about 20-25% new and unique (never seen before) queries every day, it’s natural to assume that Wolfram Alpha’s natural language processing will have a hard time providing sufficient query formulation flexibility to searchers and their diverse information needs.

Cryptic Names
Wolfram Alpha is riddled with scientific abbreviations and mathematical notation, which are rather unfamiliar to most people. These specialized features of the input language reinforces my impression that Wolfram Alpha is a search engine for specialist (in various domains). You have to possess some prior expert knowledge in order to ask the right questions, without which you can’t get access to possibly valuable information.

No Feedback
If you successfully managed to execute a query, Wolfram Alpha gives you a lot more back than just a blank line with a blinking cursor. But if you don’t succeed, it’s usually the case that “Wolfram|Alpha isn’t sure what to do with your input.” Not very helpful, unfortunately, but I guess it’s hard to do more, especially while the engine is still in its infancy.

Instead of just dismissing unlucky searchers, Wolfram Alpha could start to back-fill unsuccessful queries with web search results delivered by eg. Google. That would avoid creating a dead-end feeling. Sure, there’s a link to web search in the sidebar, but it’s well hidden. Back-filling with web results could make Wolfram Alpha more appealing to the general audience.

Options Are Hard To Remember
Not really transferable to Wolfram Alpha, but as with cryptic names, search operators like “/” must be taught to the user before they provide any value.

Ruthless file murderer?

Really Easy To Make Mistakes
Relax, there’s no risk of you accidentally erasing Wolfram Alpha, or the Internet for that matter. But it is really easy to make mistakes when you’re typing in a query. Wolfram Alpha seems to be equipped with a decent spell checker (Levenshtein, perhaps), but it doesn’t work as well for various abbreviations. For example, gbp norway gives a different result altogether than gdp norway. Sure, no harm done, except perhaps to the user’s sense of pride and confidence in the engine.

5 Things Wolfram Alpha Does Better
So much for my Wolfram Alpha bashing. Stan Schroeder of Mashable tells us what he thinks are the bright sides of Wolfram Alpha, the computational knowledge engine. He makes a list of 5 things Wolfram Alpha does better (and vastly different) than Google:

1. Complex Queries – Like finding information about our solar system by typing in the names of a few planets. Wolfram Alpha gives individual information about each planet together with different comparisons, while Google simply returns the best matching web pages.
2. Localization – Currencies and units of measure are easily converted when treated as first-class information objects, like Wolfram Alphawho’s mostly restricted to returning whatever is on the web page, localized or not. does. It’s not so easy for Google,
3. Precision – Wolfram Alpha surely can’t answer all questions. But it tries to be as precise as possible with the answers it does give, and refuses to deliver anything half-baked. I would say that precision is nice, but it’s not everything. Exploration and discovery usually starts with a few good alternatives as basis for further research and analysis.
4. Calculation – Google provides direct answers to some specialized queries, like paris hilton height and 100 USD in EUR, but nothing close to what Wolfram Alpha does.
5. Comparison – Comparing information across web pages does not work that well for document-centric search engines (like Google). But with Wolfram Alpha it works like a charm.

Wolfram Alpha is a bit like a box of chocolate – you never know what you gonna get. If at first you don’t succeed, try again. If you happen to love command line interfaces above everything else, you may want to check out Goosh – the unofficial and curious Google command line. And if you you’re looking for some light entertainment, check out these Wolfram Alpha easter eggs.

If you found this post interesting or helpful, you may want to read What Is Your Approach to Information Retrieval?

Written by Vegard Sandvold

flickr.com/allaboutgeorge

You’re the user experience designer hired in to give shape to a new enterprise search solution. Your client has already decided upon a technology vendor, and has purchased a license. The responsibility now rests with you to provide employees or customers with the best possible information access. Will you design the solution around the assumption that IR algorithms are able to directly deliver the relevant documents in response to most queries, without much further involvement from the user? Or will you design for optimal communication, making the user more responsible for retrieving relevant documents through engaging interaction? Since I don’t believe in technological silver bullets, I argue that good design takes you further for less money when you’re aiming for higher information accessibility.

This post is a continuation of my previous posts on a topology of search concepts, and mediation in information retrieval. I have received a lot of valuable feedback, and one reader commented specifically on my use of the term information accessibility as one of the dimensions for classifying approaches to search. Here’s a snippet from one of the emails I received (reproduced with permission):

When we came up with the definition of accessibility, we also very clearly spoke about the cost required to reach the resource (i.e., user effort to find the document). You would need to compare the costs that users associate with looking through a linear ranked list, versus reading the facet names and then interpreting them and then making a choice and then potentially not finding the document. This across a wide range of potential queries. Only then can you say that one form makes something more accessible than the other.

We were at this point discussing how the definition of accessibility in information retrieval applies to pure best-first (linearly ranked) search vs. faceted search. Put in other words, will a clever ranking algorithm generally provide a shorter path to the document sought by the user, or is the user generally a better judge of relevance? What I’m trying to understand from this discussion as a user experience designer, is how my design considerations may effect the final outcome to the better for businesses and users. I’m also aware that my chain of reasoning does not qualify as proper scientific argumentation.

Technology – What to Reasonably Expect
Many enterprise search customers acquire a search platform for the purpose of implementing search for a web site, an e-commerce portal, or several company systems like email and CRM. Whether the brand is Google, FAST/Microsoft, Autonomy, Endeca or Lucene, these platforms and their ranking algorithms must be tailored to the specific information needs of the company’s employees of customers. If you spent a lot of money on that software license, it’s reasonable to have high expectations for what this magical search box can conjure up. But how much effort will it take to fine-tune the relevance ranking, and can you expect it to perform top-notch across a wide range of potential queries?

There’s little doubt that web search engines like Google, Microsoft Live and Yahoo spend an awful lot of time and money perfecting their ranking algorithms, simply because the credibility of their brands depend on always delivering the best results first. But chances are that your enterprise search client is not able to pour that kind of money into all kinds of relevance trickery. With that in mind, can we reasonably expect ranking algorithms to outperform humans in terms of efficient information seeking? I believe that faceted search holds the upper hand on linear ranked lists (possibly combined with successive query reformation) when accessibility is considered.

Design – How Far Can You Get
I have to admit that I’m partial to set retrieval and faceted search. I believe strongly in solving information seeking problems with good user experience research and design, and have what I like to believe is a healthy skepticism towards technological silver bullets. It’s my experience that some up-front design gives a better return on investment than excessive technical tinkering later on.

A critical step in the design of faceted search is to find meaningful facets that convey a strong scent of information, and the only sure path to well-design facets goes through extensive user research. The set of facets and their values must correspond to the mental model of the users, and it must be immediately understandable to them what results they should expect upon making a selection. Well-designed facets also provide more support to the user as compared to multiple queries / query reformulation, which is more of a guessing game.

When a user reads and interprets the names of facet values, makes a choice and not finds the document he or she was looking for, I think one of 2 things may have happened:

1. Either the facets are poorly mapped to the mental model of the user.
2. Or the sought-after document simply doesn’t exist.

In the first case, a linear ranked best-first list would’ve been less misleading and would probably have served the user better. That could possibly have provided a shorter path, resulting in higher accessibility. In the second case, the user got early feedback that the search was futile, and that his or her time could be better spent looking somewhere else. In any case, I believe well-designed faceted search is very likely to provide a high level of information accessibility.

A Case of Ingenious vs. Diligent Search?
I have previously written about 4 different approaches to search user experience, where Ingenious and Diligent Search are descriptions of concepts and technologies that bring about high accessibility, with the difference that some are more algorithmically powered, while others are more user powered. I consider the Ingenious approach to include Google’s Universal Search, clustering, LSI, question answering / NLP, search intent analysis and more, where algorithms primarily make the judgment about what is relevant. With the Diligent approach, on the other hand, it’s the user who primarily makes this judgment, which includes concepts like set retrieval and faceted search.

I’m a search user experience designer, and I believe that a well-designed (and possibly low-tech) enterprise search can provide a better return on investment for businesses aiming to provide employees or customers with the best possible information access.

If you found this post interesting or helpful, you may want to read What Is Your Approach to Information Retrieval?

Written by Vegard Sandvold

Especially the comments by Gene Golovchinsky helped reveal some short-comings of the dimension I had named “Algorithm vs. User Powered”. Following our discussion, I have come to believe this dimension can be better described as “Algorithmically vs. User Mediated”. Let us find out by exploring what mediation means in the context of search and information retrieval.

The goal of searching is to resolve an information need, and that resolution process can be mediated (or facilitated/supported) by algorithms, users (alone or in groups), or a mix of both. Put into other words, the mediator (or facilitator) is the entity responsible for taking actions to retrieve information that will fulfill the user’s information need. If those actions are taken by an algorithm, like extracting latent structures from the data, or analysing query intent, the search approach can be thought of as “algorithmically mediated”. On the contrary, if the actions are taken by users, like navigating facets, or sharing search results with a partner, the approach can be thought of as “user mediated”.

It’s difficult to imagine information seeking that isn’t a symbiosis between algorithm and user. Almost every search starts with a user action, whether it’s to submit a few keywords, or to just produce something that can be the starting point of a query. And unless I’m asking a question directly to someone else, an algorithm has to pull up a set of possibly relevant documents for me to investigate further. But if we now accept this pre-condition, who plays the role as primary mediator for the information seeking process?

In an attempt to break it down, I have compiled a list of search concepts that I believe are primarily mediated by either algorithms or users. Whether you agree or disagree, please share your thoughts.

Algorithmically mediated (primarily):

• Keyword Search (best-first)
• PageRank (authority-base search)
• Related Searches
• Search Suggestions (type-ahead, auto-completion)
• Universal Search (structured results)
• Clustering
• Implicit Collaborative Search
• Questions Answering (natural language processing)

User mediated (primarily):

• Collaborative Filtering
• Best Bets
• Social Search (whatever that is – Twitter?)
• Explicit Collaborative Search
• Set Retrieval
• Faceted Search

If you found this post interesting or helpful, you may want to read Enterprise Search – Information Accessibility by Design or Technology?

Written by Vegard Sandvold

A Topology of Search Concepts

Search is a wicked problem, with no apparent universal solution in sight. Different technologies and approaches to search exist side by side, serving a multitude of business goals and user needs. In my work with search user experience I find it important to understand the particular strengths and weaknesses of search concepts like Best Bet and Faceted Search, since part of my job is to correctly align goals and needs with available technology. A poor choice of technologies or design patterns could very well cripple the entire user experience. But how do I know which concepts will work?

To answer my own question, I’ve spent some time the last few days putting together a topology of search concepts, which I’m thrilled to share with you now. I’ve made a scatter plot of a handful of search technologies and patterns, with the purpose of revealing some basic structure and similarities. The main elements of this topology are the set of search concepts, the 2 dimensions of the scatter plot, and the descriptions of each quadrant. You can see the result so far in the illustration at the top of this page.

This is still work in progress, so I’m eager to get feedback from you. I may have left things out, plotted things the wrong way, or described things poorly. Whatever you have to say, please leave a comment on this post.

Here is how I define the 2 dimensions of the scatter plot for the purpose of this topology:

• Algorithm vs. User Powered – In the absence of a better name, positioning along the algorithm vs. user powered dimension reflects to what extent human or machine intelligence is responsible for retrieving precise and accurate information in response to the query.
• Information Accessibility – By implementing a search concept for a given information space, information accessibility is a measure of how much easier it becomes to find any document of interest within that space. If the time it takes (and/or the number of steps required) to retrieve a particular document goes down, the general information accessibility goes up. Read more about accessibility in information retrieval.

The scatter plot is divided into 4 quadrants, so that the concepts positioned within each quadrant share a common set of characteristics related to business value, user experience and technological capabilities. Here is how I describe these quadrants:

1. Simple Search – When you know what you want, and you can express that need with a few keywords, the simple approach to search fits the bill. It’s by no means trivial to create simple experiences, but the general information accessibility is quite low for these concepts. Google, FAST/Microsoft, Exalead and Lucene are some of the champions of simple search.
2. Superficial Search – The superficial approach to search leverages user behavior to feel the pulse of communities, and to surface popular and current material. Superficial search is often very efficient when you don’t need to dig deep into the information space. Amazon, Twitter, PostRank and Twingly are some of the champions of superficial search.
3. Ingenious Search – With a model-knows-best approach, ingenious search relies on sophisticated algorithms to determine user intent and content semantics. Clever algorithms can be said to be cost effective, but are comparably mode difficult to implement and execute well. Autonomy, Powerset, Grokker and Wolfram Alpha are some of the champions of ingenious search.
4. Diligent Search – The diligent approach to search favors human intellectual effort over clever algorithms. Given an initial search result, users are asked to further disambiguate their queries in order to effectively explore the information space. Endeca, eBay, Freebase and Apache Solr are some of the champions of diligent search.

I won’t get crossed if you say my matrix looks a bit like Gartner’s Magic Quadrant, but there’s at least one big difference. Contrary to Gartner’s quadrants, this matrix does not suggest any better-than/worse-than relationships between the data points. Each quadrant is just different from the others, and the search technologies and patterns found within simply serve different purposes, having their own particular strengths and weaknesses.

Whether you agree or disagree with my way of mapping the world of search concepts, please share your thoughts. I’ll make sure to give credit to everybody who contributed significantly when I publish the final result. Cheers!

You may also download a poster with both the diagram and the quadrant descriptions.

If you found this post interesting or helpful, you may want to read Enterprise Search – Information Accessibility by Design or Technology?

Written by Vegard Sandvold

flickr.com/netliferesearch

Gulltaggen claims “Norway’s premier event for digital marketing” (presentations by Seth Godin and Chris Anderson to mention a few) and it was held on the 28th and 29th of April. There’s also a steady stream of #gulltaggen tweets. We’ll give a quick recap on two talks: building successful websites through understanding what users want to achieve on your site and how marketing changes from a mass focus to the spreading of ideas.

“The Long Neck” with Gerry McGovern
Gerry’s presentation focused on page views and content productions as our traditional view of site effectiveness. But what about user satisfaction?

Microsoft’s Excel web site had around 11 000 pages. All pages were searchable in major search engines and they were adding more and more content to ensure that they cover every aspect of Excel. A long tail focus, but what about the majority of users: The long neck?

The majority of users might be interested in how to do a basic sum of values in excel, but queries on Google would present a page on the INSUM function, a far more specialized aspect of Excel. Thus, content was beginning to clog down their web site. Page views were rising but user satisfaction was declining as users were constantly running into dead ends within the enormous amounts of pages.

The solution was to simply start removing content; a concept highly unpopular in many organizations where countless hours have been placed in content production. The web site has to focus on its core message – what does the majority of users need to do online when entering the your web pages? Looking through support records and behavior analysis to find out the most frequent actions users want to take on your site. If you are not covering your users basic needs – how can you start focusing on the long tail?

As a result page views went down drastically, another unpopular concept with senior management – but user satisfaction went up! Shouldn’t we be measuring satisfaction over of page views?

Other examples were campaign pages (Hillary Clinton, John McCain) where the pages are focused around the single most important aspect – contribution.

“Marketing, Leadership & Being Remarkable” with Seth Godin
Seth’s presentation circled around the concept of the “purple cow”. How can organizations keep on making average products for average users? People have seen cows and “cows are boring” Seth continues, but a purple cow would gets everyones attention.

Seth claims that something is fundamentally wrong with todays mass marketing model on line – where organizations are constantly trying to interrupt users in order to get their attention, examples being banner and popup ads. The Internet is about attention giving , not an attention taking, and only by getting your users permission can you successfully push your ideas at them, avoid the spam filters and make the stories spread.

The Internet is not TV on a laptop but a tool for the spreading of ideas.
- Seth Godin

The ideas are what made recent Apple products so successful. The ideas that people want to talk about, and show their peers that they are using Apple products. Apple started out with a small group of engaged mac users. Apple had permission to tell these people (aka the “fan boys“) about their new products. These people would in turn acts as marketeers for Apple products, thus spreading the idea.

Seth had several other good stories in a frantic hour of a web-streamed talk both amusing and enlightening. However, I must confess that I occasionally find cows quite amusing.

Some links:
- Gulltaggen and on twitter: #gulltaggen
- Seth Godin’s blog
- Gerry McGovern

If you found this post interesting or helpful, you may want to read Topic Pages – Content re-use and news site usability (part 1)

Written by Thomas Kjelsrud

Increasing content life span
Large news sites publish several stories by the hour, both costly to produce and short lived as they are moved down the front pages. Content only a few days old leave for obscurity by the grace of site search and search engines.

Topic pages can be a step to maximize re-use and thus increase the life span of the content produced; through the creation of landing pages that can act as a second home for articles after leaving the front page spotlight.

Basic components of topic pages
Topic pages usually start out with an introductory abstract and a region of related news stories, brought to life through search. Attention can be captured by related materials like pictures and media clips. A community aspect through a forum or visible thread could show interest, build a community around the topic and work as a social proof for future readers.

Users will quickly see through fully automated pages that try to disguise search results as a topic pages. Good topic pages needs a human touch and special care when considering what articles are relevant and not.

Why they are important
An optimal topic pages give users important background on news stories, crucial when reading outside ones field of knowledge. Topic pages solve some informational needs directly on your site, and may keep users from having to cross-read through other sites.

Journalist pressed for time can not elaborate on the basics in every article, nor should they have to. By relating and linking back from the articles to the topic pages, readers can brush up on the basics in between news reading. Thus making news content more accessible and pedagogic in nature.

Check back later for follow ups to this story!

- How to capture search traffic using topic pages

- Creating communities using topic pages and creating topic pages using communities

Let us hear your opinions on topic pages!

If you found this post interesting or helpful, you may want to read “Gulltaggen” – The long neck and spreading ideas online

Written by Thomas Kjelsrud

My vote according to Hunch.com

I like Barack Obama, but that is not why I have put his face on my blog. I have been playing around with Hunch.com the last few days, both answering and creating questions. I have discovered that Obama is my 2008 US presidential candidate, that I should live in Portland, Oregon (partly because I like big cities, a warm climate and the ocean), and that I should switch to a Mac (could have told you that). Hunch is meant to help you decide, by breaking large and small questions down into series of smaller decisions. Every path down the decision tree brings you to a result, which is the recommended (and most probable) answer for you. All topics, questions and results are user generated. If you feel a topic needs another question, add one. If you feel there’s something wrong about a result, change it.

I’m no expert on politics, but I believe that Hunch works much like libertarian paternalism, an “experts rule” where decision are engineered by those who know better. In this case, the wise guys are the authors of the Hunch topics, questions and results, rightfully (or not) posing as domain experts and advisers. Their knowledge is meant to be my guide. Hunch gives me a feeling of controlled and rational decision-making, while the real control is exercised by those who frame the questions. Of course, no one forces me to accept anything suggested by Hunch, so ultimately I’m still in control. Some healthy scepticism is always in order, but disregarding every suggestion sort of defies the purpose.

Libertarian paternalism (a.k.a. politics for dummies) builds upon behavioral economics, a field of study with fancy theories about the irrationality of human decision-making. Habits, emotions and circumstances influence our decisions more than we’d like to admit, even when these decisions are non-trivial and potentially life-altering. I also believe The Power of Defaults play an important role. When forced to decide between choices that have unknown or unpredictable effects, we tend to choose the default option. It’s easy to go with a recommendation, especially when it’s framed in a convincing manner. Like when your bank’s financial advisor recommends you an annuity loan, when a serial loan with lower total interest costs may actually be better for you.

What has politics got to do with search user experience? Libertarian paternalism is a progressive political philosophy, “designed to help irrational people who are not advancing their own interests while not interfering with the autonomy of those who are making rational, deliberate, decisions.” In other words, protecting the idiots from themselves, while leaving the sensible people alone. I personally believe we’re all functional idiots to some extent, so this kind of choice architecture makes a lot of sense to me. Search (and recommender systems) are also decision-making tools, and I have a hunch that some of these concepts  (libertarian paternalism, behavioral economics, power of defaults) may be used to increase the decision-making effectiveness of both search and recommendations, enabling a better user experience. I also believe the question of choice effectiveness and overall satisfaction is connected with the problem with recommender systems and The Paradox of Choice.

On a final note – Hunch gave me a very pleasant experience today. Last night I started on a new topic which I named “Am I a Maximizer or A Satisficer?” I was quite tired by then, so I didn’t add any questions or results. But when I checked back today, I discovered that someone else had continued my work, adding four questions and two results. It was reassuring to see the crowd-sourcing at work.

If you found this post interesting or helpful, you may want to read Hunch!

Written by Vegard Sandvold

If you found this post interesting or helpful, you may want to read Behavioral Economics Meets The Power of Defaults at Hunch

Read the original article »

Written by Vegard Sandvold

If you found this post interesting or helpful, you may want to read The Future Of Social Search (Or Why Google Should Buy Facebook)

Read the original article »

Written by Vegard Sandvold

flickr.com/aldo

Topic Maps, an ISO standard for semantic networks, relies on authorities to create and maintain Published Subject Indicators (PSIs), uniquely linking single topics to single subjects out there in the real world. TopicMaps.Org has eg. published indicators for languages and countries. But who gets to claim authority over a particular set of topics? Conflicts between people and organizations with opposing interests are unavoidable, especially for more controversial topics. I believe Wikipedia may provide a solution, by acting a democratic and deliberated set of PSI’s for every topic worth writing about. The authority to create and maintain PSI’s is then effectively granted to the global Internet community as a whole, since everybody is essentially allowed to edit the content of Wikipedia.

The problem with authority becomes evident with controversy. Who gets to decide on the truth when (groups of) people disagree? This is an inevitable problem for Wikipedia, as it is for any organization in charge of a communication channel. Wikipedia’s solution is as elegant as it is un-complicated. Whenever a conflict over an article occurs, that article is marked with a warning about it’s controversial content (like this article on the 2008 South Ossetia war). The article authors are then responsible for working towards a version of the article that everybody involved can agree upon. This deliberative and democratic mechanism seems to be built into the very core of Wikipedia.

I believe Topic Maps can draw benefits from Wikipedia’s democratic mechanisms, by making Wikipedia the universal authority of Topic Map PSI’s. If your topic map needs a new topic, check for an existing article on Wikipedia. If it doesn’t exist, author a new article, and use it’s URL as your new topic PSI. If nobody feels a need to modify your contribution, congratulations! Your new topic PSI has earned it’s right to exist. Should someone disagree with you, however, you need to engage in a little deliberative democracy in order to establish a common understanding. Once that agreement is reached, we’re all better off.

Wikipedia sports many of the features defined for the Topic Maps standard. Each article can serve as a topic definition, and each article URL can the thought of as that topic’s PSI. When topic maps that contain common topics are merged, Wikipedia’s disambiguation mechanism supports the requirement that merged topics must include the union of the individual topics. Even scoping can be handled using a more flexible approach to subdomains, not unlike how it is used for languages today (eg. http://no.wikipedia.org for Norwegian).

Is the Topic Maps community willing to adopt Wikipedia as it’s democratic and user-generated repository of topic PSI’s? What actions must Wikipedia take to fully accommodate for the needs of topic map author out there in the trenches? Let your voice be heard.

If you found this post interesting or helpful, you may want to read Topic Pages – Content re-use and news site usability (part 1)

Written by Vegard Sandvold