TwigKit

The London Search Meetup is Back

2013-02-12T00:00:00-08:00

Yesterday evening we held the first meetup of 2013: a book-themed event featuring Martin White discussing his new book, Enterprise Search, as well as Tony Russell-Rose and myself talking about Designing the Search Experience.

A brief history

Twigkit organized the first London Search Meetup in December 2009. Throughout 2010 and 2011 a dozen meetups were held, featuring talks from the likes of Stephen Arnold, Tony Russell-Rose, Martin Belam, Jeremy Bentley, Max Wilson, Vegard Sandvold, and many others. During that period over 400 members joined the group, making the London Search Meetup the second largest enterprise search meetup in the world (the Bay Area Search group being the largest). After going strong for over two years, the meetup group then took a break in 2012 (we got a bit too busy, sorry); but we’re now very pleased to be reigniting the group in 2013, and aim to have bi-monthly events throughout the year.

Calling all search enthusiasts

So if you are in or around London, we hope to see you at a search meetup soon. And, if you’re interested in giving a search-related talk at a future meetup, do get in touch.

My slides from the meetup

A Handbook for Designing Search

2013-01-29T00:00:00-08:00

Search is ubiquitous. It has become embedded in our daily lives — not just for fact-finding, but from shopping and cooking to listening and watching. At the same time, organizations have embraced this disruptive technology to solve business problems, from increasing productivity, to making more informed strategic decisions, to providing better experiences for customers.

While the potential of search is tremendous, many search experiences don’t measure up. We’ve all been there: broken buttons, irrelevant results, confusing interfaces.

From the beginning, Twigkit has endeavoured to close this gap between reality and the ideal. Our rapid-development framework and library of refined user interface components have made it easier than ever before to deliver a superb search experience.

Our new book

That’s why we’re pleased announce a new resource for aiding the design of search user interfaces. Designing the Search Experience: The Information Architecture of Discovery is a new book by yours truly (Tyler Tate) and Tony Russell-Rose, published by Morgan Kaufmann, an imprint of Elsevier. The aim of the book is to help you:

Understand how people search and how the concepts of information seeking, information foraging, and sensemaking underpin the search process.
Apply the principles of user-centered design to the search box, search results, faceted navigation, mobile interfaces, social search, and much more.
Design the cross-channel search experiences of tomorrow that span desktop, tablet, mobile, and other devices.

We are very pleased to feature essays in the book from experts such as Daniel Tunkelang, Louis Rosenfeld, Greg Nudelman, Martin White, and others. The book has also received praise from thought-leaders including Ricardo Baeza-Yates, Marti Hearst, and Peter Morville:

“Search is among the most disruptive innovations of our time, and in myriad contexts from ecommerce and enterprise to mobile and social, it remains a massive user experience headache. If you dare to tackle this wicked problem, be sure to bring along Designing the Search Experience as your faithful guide.” — Peter Morville, author of Search Patterns

Where to find it

The book is available in print and for the Kindle at Amazon.com and Amazon.co.uk. You can also download a free chapter and find out more about the book at DesigningTheSearchExperience.com.

Designing Mobile Search

2012-05-18T00:00:00-07:00

In 2010 there were more computers manufactured than mobile devices—but just barely. Starting in 2011, the number of smartphones and tablets began outpacing the number laptops and desktops being sold. While these devices are being purchased primarily by consumers rather than corporate IT departments, there’s a growing trend to Bring Your Own Device (BYOD) to work. Mobile devices are becoming ubiquitous—both at home and at work.

Search has long been the de facto method for interacting with information, and mobile devices are no different. Yet the temptation to replicate desktop-oriented user interface conventions on mobile devices is far from ideal. That’s why designing mobile search is an important topic.

Data from Beyond the PC by The Economist, Oct 8th 2011.

This week I’ve been in New York City talking about just that. On Tuesday afternoon I spoke at the Enterprise Search Summit, and then again at the NY Enterprise Search User Group meetup in the evening, kindly hosted by the friendly people at Shutterstock. The four areas I focused on were:

Mobile searchers and what differentiates them from desktop users. In particular, the information needs of mobile users—a topic originally written as a blog post, but was recently included as a paper at the Search4Fun workshop collocated with the European Conference for Information Retrieval.
Design principles for mobile search: content trumps controls, answers over results, contextual sensitivity, and cross-channel continuity (these have also appeared as a blog post).
Practical design solutions for constructing mobile search user interfaces. From inputting the query, to displaying results, to refining the query.
And lastly, the pros and cons for three different implementation strategies: native applications, web-based applications, and responsive websites.

The slide deck is available on Slideshare, and I’ll be giving the talk again at Enterprise Search Europe in London later this month.

Designing Mobile Search, NYC Edition

View more presentations from Tyler Tate

Design Principles for Mobile Search

2012-01-13T00:00:00-08:00

Apple’s iOS Human Interface Guidelines, Google’s Android Design Guidlines, and others such as Luke Wroblewski’s Mobile First book provide valuable guidance for designing general mobile applications. Yet there are a number of design principles unique to crafting mobile search experiences in particular. Namely: prioritizing content over controls, providing answers over results, being sensitive to context, and ensuring cross-channel continuity.

Content Trumps Controls

Search is often accompanied by a number of knobs and dials: filters, breadcrumbs, sort controls, pagination; the list goes on. When moving from the design of desktop to mobile search interfaces, the temptation is to replicate all of these controls on the main search results screen. Yielding to this temptation, however, leads to cluttered, frustrating interfaces that add stumbling blocks to the user’s path.

The primary search screen of a mobile application should be focused on the clarity of search results; bells and whistles must take a back seat. Mobile users, after all, often use their devices for short bursts of time, enter fewer queries per search session than do desktop users, and may seek answers to simple, lookup-based information needs. These realities suggest that navigation bars should be kept to a minimum, filtering and sorting displaced off-screen, and pagination controls omitted so that the search results receive as much screen space as possible.

Left: 50% of AOL’s mobile search interface is consumed by chrome.

Answers over Results

In addition to minimizing search controls and emphasizing content, focusing on precision over recall is usually in the best interest of mobile users, particularly time-sensitive smartphone users. Precision, as you know, describes the accuracy of the top results. Because mobile users reformulate their queries less often than desktop users, prioritizing the relevance of the top few results is generally more useful than providing high recall.

Providing direct answers to users’ lookup queries can make the mobile search experience more efficient still. Rather than force users to click on a search result to discover straightforward facts, such as “director of third man movie”, for instance, a more desirable approach is to provide a computed answer directly on the search page, eliminating the need for further action.

But tablets are a different story. While phones are often used for short periods of time to accomplish tangible goals, tablets are more likely to be used for longer durations and with more casual information needs. Optimizing for precision over recall and answers over results is a good rule of thumb, but should be balanced with the user’s context.

Google provides a direct answer to the query.

Contextual Sensitivity

Few argue about the importance of context on mobile devices. But effectively designing for context is another matter. The trick is to carefully interpret the user’s current context so that the interface can be optimized accordingly, while being accurate and forgiving enough that users don’t get frustrated when the system identifies the context incorrectly. Interpreting the user’s context could include determining their task—driving, cooking, or doing bicycle repairs, for example—whether or not their physical location is important, or whether they’re alone or with others.

Each of these aspects of context, if correctly identified, provides opportunities for optimization. A query for “basel”, for example, could be interpreted as a city if the user is driving in Switzerland, or as an ingredient if the user’s task is cooking. A trivia-lookup query originating during a conversation with friends could prompt a direct answer, while a casual ‘window shopping’ query originating on a tablet via the sofa could be better addressed with an image grid that encourages undirected browsing.

Amazon’s Windowshop iPad app invites casual browsing.

Cross-Channel Continuity

Every business recognizes the value of consistency across channels: customers benefit from a coherent, holistic experience where the learning from one channel can be applied to all the rest. A user familiar with Amazon on the desktop will instantly recognize the similarity of Amazon’s mobile application, for instance. But while consistency ensures the learnability of each channel, continuity makes it personal. Continuity is adding an item to the shopping cart via a desktop computer, and having it appear in the shopping cart on your phone; it’s saving a search on your phone and returning to it later on your tablet. In other words, continuity ensures that your actions aren’t performed in isolation, but propagate from the source channel to each of the others.

Continuity between channels can facilitate the figure-it-out-later approach often taken by users, as well as reduce the number of information needs that fall through the cracks. For starters, search history should be synchronized across devices so that inconclusive information seeking can be easily followed up later. What’s more, allowing saved searches that are accessible from every channel enables users to organize and return to important, ongoing information needs.

Zillow allows users to save searches, but fails to synchronize them across devices.

Mobile Information Needs

2011-12-06T00:00:00-08:00

Mobile is the new desktop. The adoption of mobile Internet makes the introduction of the World Wide Web seeming glacial by comparison; Morgan Stanley predicts that mobile Internet usage will outpace desktop-based access in just three years.

Search—or information seeking more broadly—is pivotal to circumnavigating this convergence of the digital and physical worlds. But before designing experiences for the new frontier, we must first grasp the needs of users. Specifically, we must understand the information needs of mobile users.

“Information need is an individual or group’s desire to locate and obtain information to satisfy a conscious or unconscious need.” — Wikipedia

Two dimensions: scope and type

Mobile information needs can be assed by two criteria: scope and type. Scope describes the sophistication of the information need, the degree of higher-level thinking it involves, and the time commitment required to satisfy it. The lookup, learn, and investigate elements of scope are derived from Gary Marchionini’s work on exploratory search, while the casual component has been more recently advocated by Max Wilson and others.

Type, on the other hand, is concerned with the genre of the information being sought. Broder is often cited for recognizing the informational and transactional nature of many needs, while the geographic and personal information management goals identified by Church & Smyth are especially prominent for mobile users.

Scope

Casual. Undirected/semi-directed with a hedonistic rather than task-driven purpose.
Lookup. “Known item” searching for a concrete fact. Most common form of search.
Learn. Iterative information gathering requiring greater interpretation and judgement.
Investigate. Long-term research and planning demanding high-level of thinking.

Type

Informational. Any search for information about a topic.
Geographic. Points of interest searching or retrieving directions between locations.
Personal Information Management. Private information not publicly available.
Transactional. Goals which are action-oriented rather than informational.

A matrix of information needs

While the scope and type provide a framework, they don’t actually tell us about the information needs themselves. That’s where ethnography comes in. Sohn et al. and Church & Smyth have each conducted diary studies in which smartphone-toting adults spread across the globe were instructed to record any information need that arose over a period of weeks. This research enables us to construct a matrix of mobile information needs:

Below are quotes of information needs recorded by participants in the two studies mentioned above. Those without quotations indicate my own additions.

Informational

Checking Notifications. I’ll glance over status updates to kill time while waiting.
Trivia. “What did Bob Marley die of, and when?”
Information Gathering. “How to tie correct knots in rope?”
Research. What is Keynesian economics and is it a good idea?

Geographic

Friend Check-ins. “Where are Sam and Trevor?”
Directions. “Directions to Sammy’s Pizza”
Local Points of Interest. “Where is the nearest library or bookstore?”
Travel Planning. Flights, accommodations, and sights for my trip to Italy.

Personal Information Management

Checking Messages. “Email update for work”
Checking Calendar. “Is there an open date on my family calendar?”
Situation Analysis. “What is my insurance coverage for CAT scans?”
Personal Planning. What should my New Year’s resolutions be this year?

Transactional

‘Window’ Shopping. I don’t know what I want. Show me stuff.
Price Comparison. “How much does the Pantech phone cost on AT&T.com?”
Online Shopping. I want to buy a watch as a gift. But which one?
Product Monitoring. I know the make and model of used car I want. Alert me when new ones are listed.

A penny for your thoughts?

Though this matrix does accommodate the most common information needs identified by Sohn et al. and Church & Smyth, it is still a work in progress. As such, I would very much appreciate your feedback.

Book Review: Search Analytics by Lou Rosenfeld

2011-10-13T00:00:00-07:00

This is a book review of Search Analytics for Your Site: Conversations with Your Customers by Louis Rosenfeld, published by Rosenfeld Media, July 2011.

Financial services company The Vanguard Group had just purchased a shiny new search engine to improve search for their 12,000 employees. There was only one problem: the search results were worse than what they had before.

John Ferrara, an information architect who had helped select the new search platform, blew the whistle, asking the project to be delayed so that relevancy could be improved before the search engine went live. Unfortunately, he failed to make a convincing case to his IT colleagues. Technically, the new platform was running just fine. Besides, the search vendor undoubtedly new more about this kind of stuff than the internal guys.

But not to be deterred, John Ferrara turned to the search logs. He aggregated the most popular search queries from the old platform and, going down the list one by one, measured two metrics for each: relevancy and precision. To measure relevancy, John typed in the query (“company address”, for instance), and checked to see how far down the best match appeared from position #1. To measure precision, on the other hand, John looked at the top five results for a given query and marked each as either relevant, nearly relevant, misplaced, or irrelevant. The output from each of those exercises is shown below.

Source: Rosenfeld, Louis. 2011. Search Analytics for your Site. New York: Rosenfeld Media.

By turning to the data, John was able to convince his colleagues that there was, indeed, a problem. With hands on deck, the team was fortunately able to tune the new search platform before it went live, bringing relevancy and precision up to parity with the old system. Thanks to search analytics, John had saved the team from what could have been a serous blunder.

What’s it all about?

Lou Rosenfeld opens with this relatable story in his latest book Search Analytics for Your Site: Conversations with Your Customers. The book is a very practical guide on how to exploit query logs to improve your company’s search experience. Lou outlines a collection of simple but potent techniques for analyzing search logs, spotting insightful patterns, and putting those insights to use.

As mentioned in the Vanguard story, Lou demonstrates the value of understanding users’ most common queries. But he also goes much further. From studying failure situations (which queries lead to zero results), to session data (who searched what when), audience segmentation, and goal-based analysis (using key performance indicators), Search Analytics presents a sweeping collection of techniques for turning search logs into an organizational goldmine.

And if those techniques weren’t practical enough already, Lou ends the book with a comprehensive list of tips for using search data to improve your website’s search, content, navigation, and metadata.

Search Analytics is well-written, to the point, and does what it says on the tin.

Search Analytics is well-written, to the point, and does what it says on the tin. The author is authoritative — he wrote Information Architecture for the World Wide along with Peter Morville — as well as experienced, having consulted for companies such as PayPal, Caterpillar, Ford, and others. I was able to put into practice techniques I learned within days of first picking up the book. If you in anyway share responsibility for search in your organization, this book is well worth your time. I highly recommend it.

Resources

Use promo code TWIGKIT to receive 20% off at RosenfeldMedia.com
Purchase from Amazon.co.uk or Amazon.com
View Lou’s presentation, “Site Search Analytics”, on Slideshare

The Social Side of Search

2011-09-26T00:00:00-07:00

Have you ever noticed a honeybee fly in a tight loop? Individually a bee will set out on its in own in search for food. But rather than hoard an abundant source of nectar for itself, the honeybee will return to his swarm and dance in a figure-eight pattern to alert the others of the new food source.

Similarly, our information seeking is situated within a backdrop of social activity. While your fingers alone may type a query into the search box, our need for declarative (know-what) and procedural (know-how) knowledge — as well as the means by which we acquire it — is inseparably intertwined with other people.

In The UX of Learning I recently outlined six stages of learning — initiation, selection, exploration, formulation, collection, and action — which can also be thought of as stages of information seeking. Here I’d like to look at the two directions of social activity which occur throughout that six-stage process: pull and push. Understanding the social context of search will better position us to design the collaborative search applications of the future.

Pull

I’ve been having bicycle trouble recently. My pedal fell off, I had to replace a crank, and most recently, the chain snapped in two (not all at once, fortunately!). At some point I’ll have to sit down at my computer, do a bit of research, and order a replacement part. In the meantime, I’ve received several words of advice from friends, including things like:

“You should get the KMC X10 SL 10 Speed Chain Gold, it’s amazing!”
“Just ask the guys at Evans Cycles, they’re really knowledgable.”
“Hmm, you’ve been having so many problems lately I’d replace the entire crankset.”
“Yeah, I think you’re right: If I were you I’d save money and only replace the chain.”
“I talked to the guy at my local cycle shop and he strongly recommended the KMC double durability chain.”

Researches Rob Cross and Lee Sproull interviewed 40 managers from a major accounting firm to analyze the types of “actionable knowledge” that are shared through personal interactions. The feedback I received about bicycle chains fits into the five categories they identified:

Solutions. Direct answers to a specific questions.
Referrals. References to a person or information source.
Problem reformulations. Altering the question that is being asked.
Validation of plans. Confirmation of the approach.
Legitimization. Consulting an authority figure in order to expropriate their authority.

Pulling occurs when you interact with other individuals in order to satisfy your own information need.

Push

Pushing, on the other hand, is not associated with furthering your current information need, but revolves around sharing ideas with others.

In 1993 Vicki O’Day and and Robin Jeffries interviewed clients of professional intermediaries to discover how search findings were shared within the organization, while Preben Hansen and Kalervo Jarvelin conducted a similar study in 2005 with patent researchers. When combined, the two studies reveal a number of self-initiated push actions, including:

Sharing documents. Such as articles, reports, and presentations.
Sharing information about documents. Including annotations, references, citations.
Self-initiated broadcast. A tweet, status update, mass email, or blog post.
Archiving. Recording information for future retrieval.

In Conclusion

Social networks have come to define how we interact with our friends. Yet the enterprise — which stands to gain enormously through better collaboration amongst its employees — has yet to figure out how to use technology to best facilitate collaboration. Push and pull are certain to play a big role in the future of enterprise search.

Redesigning Wikipedia's Search

2011-08-19T00:00:00-07:00

Yesterday was the deadline to submit a redesign of Wikipedia’s search page to the search design contest organized by Greplin. While the contest isn’t officially endorsed by Wikipedia, we found the challenge too much fun to turn down.

The Status Quo

Most people use Google to search Wikipedia. What’s broken about Wikipedia’s search?

1. Search gets second-class treatment on Wikipedia

Can you even remember what Wikipedia’s search page looks like? Despite having a search box at the top right of every page, Wikipedia only shows search results as a last resort. They first try to send users directly to other Wikipedia pages. Type in “London riots” and you’re redirected straight to List of Riots in London. Or type “US” and you get the US Disambiguation page. Only when no page exists, such as for ”egypt pyramid”, does Wikipedia show search results. Even then, the search results are treated as en error state, with a message saying “Sorry, this page doesn’t exist.”

2. Unnecessary clutter pushes results too low on the page

Even excusing the Wikimedia notice, there are a number of unnecessary elements on the page which cause less than three results to appear above the fold. The “Search Results” heading isn’t needed, a link to the search help page is mentioned not once, but twice, and there’s that long error message complaining that the page doesn’t exist. None of these are needed.

3. Lack of helpful filtering options

A look at Wikipedia’s contents pages reveals a robust, top-down categorization of knowledge into structured categories. Unfortunately, none of this encyclopaedic categorization makes its way to the search page. If searching for “beatles”, for instance, it would be nice to filter by either “Nature” or “Culture”.

4. Uninformative search results

A short title, a 20-word description — that’s all the user has to go on. Titles could be augmented with disambiguation information, the category of the article could be listed, descriptions could be longer, images could be displayed. There’s a wealth of useful information Wikipedia could pull from to improve the display of search results.

The Redesign

But enough complaining. What would a useful, usable Wikipedia search look like?

1. Results with strong information scent

Users quickly scan the page looking for trigger words and visual concepts meaningful to their search. By using images, longer descriptions, and providing greater context through categories and related articles, our redesign would helps users find what they’re looking for more quickly and more reliably.

2. Emphasize the best match

Rather than skip the search results page altogether by taking users directly to an article, we would instead use the result list to emphasize the most relevant article. We would us a visual container, a larger image, and links to sub-sections to highlight the top article.

3. Meaningful faceted navigation

There are at least two dimensions to Wikipedia’s content: aboutness and format. Aboutness is captured at a high level by Wikipedia’s portals: arts, biography, geography, history, mathematics, science, society, and technology. We chose to place these as tabs at the top of the page, allowing users to filter their search to any one high-level subject. Format, on the other hand, is represented by Wikipedia’s content types: article, list, picture, portal, sound, and topic. We thought these best-suited for the sidebar, where we also added a category facet to help users narrow in on lower-level sub-topics. To complete the faceted navigation, we also added a breadbox to keep track of any filters users apply.

The Last Word

There are doubtless many other approaches which could improve Wikipedia’s search. Our goal was to design a search interface in keeping with Wikipedia’s style and ethos, which they could conceivable implement in the real-world with a modest amount of effort. In fact, only the search results themselves were mocked-up in Photoshop; the layout, tabs, searchbox, faceted navigation, and surrounding elements shown in the redesign were all built using the TwigKit toolkit and are already functional in the browser. We are of course happy to donate these design ideas to Wikipedia should they come across this post.

Read more about the Wikipedia search design competition on TechCrunch.

Revisiting Faceted Navigation

2011-07-29T00:00:00-07:00

Today I’ve been experimenting with a different take on faceted navigation. TwigKit’s current facet widget is already robust – it displays both flat and hierarchical facets, it’s expandable, it allows users to exclude a given filter, the list continues. But today, there are two additional use cases that I set out to address:

The Challenge

View all filters. Not just expanding from 7 to 20, but viewing all the available filters.
Search within a facet. If the filter of choice doesn’t appear at the top of the list, allow them to search for it.

Neither of these features are necessary in the average search interface (in fact, many interfaces are better off without them). However, they are useful when detailed analysis is required. An intelligence analyst, an academic researcher, and even a very dedicated shopper would appreciate the flexibility that these controls provide.

Design Philosophy

Once I understand what the user needs, I tend to look through three lenses when designing the how. In order of priority:

Minimize cognitive load
Minimize visual complexity
Optimize browser performance

Unfortunately, many of the search within/view all implementations I’ve seen are a bit heavy-handed. I was hoping for a gentler approach.

A Potential Solution

After experimenting with a few different approaches, I decided to prototype a design patten based on progressive disclosure. The series of screenshots below correspond to these four actions:

Default state. As few controls as possible are shown by default to minimize cognitive load and visual complexity.
Expanded state. Once the user clicks “Show more,” the container expands and the “search within” box appears.
Endless scrolling. Not only does the expanded container enlarge, but it also becomes scrollable. When the user hits the bottom of the list, we’ll ask the server for more and append them to the bottom of the list.
Search with autocomplete. As the user types into the facet search box, the filters will be refined as the user types.

(Here’s my partially-implemented webkit prototype for Safari and Chrome users.)

What’s Your Take?

A few reasons why I like this solution:

The search-within box and scrollbar are shown only after the user has expressed interest in the facet
It allows all the filters to be viewed without using an additional popover or modal.
The search box refines the list of filters below it, avoiding the need for the traditional popover-based autocomplete.

However, this is only the result of one day of experimentation. More than anything I’d love to hear what others think of this new take on an old design pattern.

A Call for High Quality, Open Source Demo Data

2011-05-20T00:00:00-07:00

There is a huge need for a standard corpus of high-quality, free-to-use demo data. When building search applications, for instance, getting your hands on actual data can be near impossible, forcing you to design for unrealistic situations and compromising the end result. Well-rounded demo data would help ensure you’re working towards the right target.

There is a second huge benefit in having a de facto corpus: benchmarking. Different retrieval platforms could be compared against a fixed standard, and academic studies would benefit from greater comparability.

What would this corpus look like? Above all, it must be open source. It should contain a critical mass of records (10,000 or greater), while still having a light footprint (under 100 megabytes). It should captured in a common standard (such as XML) which could easily be ingested by a range of technologies. Beyond that, the data should meet at least the following requirements:

Requirements

Descriptive titles. Records should have properly curated titles.
Free text. There should be large chunks of text.
Ranges. Numerical, range-based data such as ratings scores.
Temporal. There should be dates and times.
Spatial. Records should be geo-tagged or included addresses.
Multi-value fields. Fields with more than one value, such as tags or keywords.
Hierarchical. At least some of the records should belong to a taxonomy.

Nice to haves

Public URL. It would be a bonus if records contained a publicly-resolvable URL.
Public multimedia. It would also be a bonus if records contained references to publicly-available images or video.

What do you think?

Most importantly, what do you think? Have I carelessly overlooked an excellent corpus already in existence? If not, then how would you improve this list of requirements? Together, lets make something.

Why Developers Should Become UX Designers

2011-04-08T00:00:00-07:00

This is a follow-on piece to Why Designers Should Give a Bleep.

Why do you code? It’s probably not just for a pay check (lets face it, there are plenty of boring jobs out there that pay the bills). Maybe you code because you like working with the latest technology, or perhaps you take pleasure in crafting concise, elegant solutions to tough problems.

Did I hear you say, “I code to deliver value to users”? Hmm, didn’t think so. But you’re not alone: designers have their own set of motivations devoid of the user, from seeking the praise of others to creating a work of art.

It’s imperative that both designers and developers fight against our natural inclinations and treat the user as king. Whatever you’re working on, whether it’s an API for a payment gateway or a new request handler for Solr, you’re building it for the people who will use it. Want to become a better developer? Then start designing the user experience.

Want to become a better developer? Then start designing the user experience.

What UX is not

UX is not about style; it’s primary concern is not colors, textures, or font faces. These elements deserve attention, but they’re secondary to the user’s interaction with your application.

What UX is

User experience design is about helping users accomplish their goals as efficiently as possible. Sounds simple, right? It’s a straightforward concept that’s complex to achieve. Good user experiences are:

Discoverable. Can the user easily figure out how to accomplish their task?
Efficient. Are the steps involved in accomplishing the task irreducibly complex? Are there any wasted clicks?
Economical. Is the cognitive burden placed upon the user as minimal as possible?
Tactile. Is immediate and responsive feedback provided for each interaction? Does it feel right?
Recoverable. Is the potential for errors minimized? Can users recover gracefully when errors do occur?

While you might recognize the value of these rules of thumb, how can you as a developer achieve them in practice?

Developing a great user experience

You might have seen this joke floating around Twitter recently:

Q: How do you generate a random string?
A: Put a fresh student in front of vi and tell him to quit.

It’s a good reminder that the knowledge and preferences of developers are mostly likely very different from that of your users. To achieve a great user experience, you must put yourself in the shoes of your users. Don’t assume they’re like you. Talk to real users. Watch them use your application (without any hints from you) and notice where they stumble. Do customer support and see which issues keep coming up. It’s simple, but it will revolutionize the way you work.

To achieve a great user experience, you must put yourself in the shoes of your users.

In conclusion

Every application has users. Great applications are the ones built by developers who care about the users enough to exert the time and effort required to understand them.

Why Designers Should Care

2011-03-30T00:00:00-07:00

The designer triumvirate of information architects, user experience professionals, and interaction designers have a potent skill set for creating smooth, delightful experiences for users. While those skills are masterfully applied to browse-based navigation, search is far too often an afterthought.

“Afterthought search” leaves just one leg for the experience to stand on, crippling usability. Designers must be concerned with both sides of the coin. If browse has been the pervasive mode of interaction since the GUI was invented in 1984, Big Data is ensuring that search will be the prevailing mode of tomorrow.

“If browse has been the pervasive mode of interaction since the GUI was invented in 1984, Big Data is ensuring that search will be the prevailing mode of tomorrow.”

O designer, where art thou?

Why have designers been reluctant to work with search in the past? I can think of three reasons:

Small vs. big business. There are many more small businesses than large corporations in the world. Small businesses usually need small websites, and small websites don’t often require search.
Company politics. Big companies with big data, on the other hand, segregate employees into departments, meaning designers and IT people — the traditional owners of search technology — don’t often cross paths.
It’s a different world, sort of. Many design patterns are unique to search: autocomplete, pagination, sort controls, related searches, spelling suggestions, the breadbox. This may push first-time search designers out of their comfort zone, though it’s quickly remedied with a bit of experience.

While these reasons may help explain the disconnect between designers and search, they doesn’t excuse it.

The data explosion

Remember the Yahoo! Directory? It was a curated classification of websites which users could browse to discover sites of interest. It wasn’t until 2002 that Yahoo! actually switched from being a directory to a search engine. As you’d expect, Yahoo! realized that in order to cope with an ever-increasing number of websites, browse just wasn’t a viable option; search was a no-brainer.

Data volume has exploded since 2010 from under 100 exabytes to over 1,000 exabytes today, and shows no sign of slowing down. Popular online stores like Amazon, eBay, and iTunes feature millions of items. Wikipedia has over 20 million documents, LinkedIn and Facebook have hundreds of millions of users, and Flickr hosts over 5 billion photos. Search is here for good.

The convergence

Don’t cry! While search is here to stay, browse still has it’s place. In fact, search and browse are complimentary paradigms. Unfortunately the search page has traditionally been isolated from the rest of the website, forcing users to choose at the outset one path or the other; a false dichotomy. Consider this user journey.

Peter goes to www.amazon.com and navigates to the “Books” section. He types “design” into the search box, and presses return. He then clicks on a filter within the “Format” facet to only show hardcover books. Rather than viewing books across all categories, Peter clicks on the “Art, Architecture & Photography” sub-category.

In this example Peter began with a browse action, performed two search operations, and then ended with another browse action. He didn’t have to choose between search and browse. The future of interacting with information almost certainly lies in the user being able to search and browse in tandem.

“The future of interacting with information almost certainly lies in the user being able to search and browse in tandem.”

In conclusion

Search has been the red-headed stepchild of information architecture for too long. Designers of all shades should wholeheartedly embrace search. Users already have.

Comments?

We’ve just revamped our blog (moving from Wordpress to Jekyll), but haven’t got around to supporting comments just yet. We’d love to hear your thoughts though. Use the hashtag #bleepingdesigners when you tweet and we’ll join in on the discussion.

Search as a Flow Experience

2011-02-09T00:00:00-08:00

When was the last time that you were “in the zone”? Do you remember being so absorbed in an activity that you forgot about the outside world, time seemed to fade away, and you felt invigorated? Maybe you’re an avid tennis player and remember a rigorous game when you seemed on fire. Or perhaps you’re a musician and recall feeling as if the notes were flowing through your fingertips.

Psychologist Mihaly Csikszentmihalyi calls this state of optimal experience flow. In his research, he found that musicians, composers, athletes, and even chess players all used the same words to explain their enjoyment. Csikszentmihalyi identified 8 elements that contribute to a flow experience:

Clear goals
Concentration
A loss of self-consciousness
Lost sense of time
Immediate feedback
An adequate level of challenge
A feeling of control
Intrinsically rewarding

Flow certainly has its roots in the physical world, but it happens on the web as well. Hoffman and Novak describe flow in the digital world as:

The state occurring during network navigation which is characterized by a seamless sequence of responses facilitated by machine interactivity, is intrinsically enjoyable, is accompanied by a loss of self-consciousness, and is self-reinforcing.

Search is particularly ripe for facilitating flow. Search results respond to the user as the user responds to the search results, entering a dance as each influences the other in an ongoing, two-step tango.

But that’s when everything goes right. Far too often, things go wrong. The user can’t figure out how to express his information need, or the search engine can’t find the right documents. What then?

Above all, users need to feel like they’re always headed in the right direction. There are four principles than can guide us in designing search experiences that always provide a next step for a users, helping them enter a state of flow.

Help users form their query

It’s easier to pick something from of a list than it is too pull it out of then air, so use the power of suggestion to help users get off on the right foot. Autocomplete, bookmarks, and saved searches can all help.

Help users review the results

Once the user has entered the initial query, they’ll then (hopefully) have a list of results to review. The most important consideration is to have descriptive titles for each result (hint: filenames usually make for poor titles). “Hit highlighting” can help the eye quickly parse the results for important words, and grouping similar results together has been shown to improve productivity.

Help users reformulate their query

After reviewing the results, the user may find the answer he’s looking for, click on one of the results for more information, give up unsuccessfully, or — what often happens — try a slightly different search. Showing related searches, providing sort controls, and offering faceted navigation can help make sure the user always has a way forward.

Help users recover gracefully

The worse thing that can ever happen in a search interface is for there to not be any search results. Try everything to avoid it. Automatic spelling corrections, stemming, lemmatisation and other query expansion techniques can all help alleviate this situation. Just be sure to clearly notify the user if you change their query without asking them.

In the words of Csikszentmihalyi:

The combination of all these elements causes a sense of deep enjoyment that is so rewarding people feel that expending a great deal of energy is worthwhile simply to be able to feel it.

The Slides

I presented this material as a talk at a UX Brighton meetup on February 8, 2010. Thanks to Harry Brignull for inviting me to speak alongside Tony Russell-Rose and Glenn Jones.

A review of Foodily’s recipe search

2011-02-01T00:00:00-08:00

This past week I discovered a new recipe search engine called Foodily. Both the interaction and the visual design are superb, and Foodily makes use of several novel patterns that I thought would be worth pointing out.

With and Without

For starters, instead of using a single search box, Foodily have two. The first is for entering the ingredients that you do want in a recipe, while the second is for excluding ingredients that you do not want. I don’t remember having encountered this before, but it seems to work quite well in this case and doesn’t really require any additional effort from the user (pressing enter while in the first search box will still submit the query).

Where this seems slightly inconsistent to me is if I initially enter “apples” in the first box, and “oranges” in the second. When I’m then brought to the search results page, Foodily strangely chooses to combine these terms into the first search box, reading “apples without oranges.” To me this seems unnecessarily inconsistent; I would keep “oranges” in the second box instead of switching things around.

Breadcrumbs and suggested searches

Foodily also provides the staples of breadcrumbs and suggested searches. There are two aspects worth pointing out here, however. The first is that Foodily have managed to fit both of these features onto the same line, which helps conserve valuable vertical space, especially important since this website is optimised for tablet devices. (The breadcrumbs did break on me when I applied more than three filters, but this is nothing that Foodily can’t rectify easily enough).

The second very interesting technique at play here is that Foodily have synchronised the breadcrumbs with the contents of the search box. This enables the user to iteratively refine their query by adding or subtracting words from the searchbox (whereas in most search interfaces typing something new in the searchbox clears all the previously-applied filters).

Filtering on ingredients and sources

The last convention that I’ll draw attention to are the plus and minus icons that are shown by each ingredient, as well as beside the original source of the recipe (i.e. Williams-Sonoma). The plus icon clearly indicates to the user “show me only recipes from Williams-Sonoma,” while the minus icon indicates “don’t show me any recipes using butternut squash.” So far so good.

What I dislike is the popover that appears when either the plus or minus icons are hovered. While it gives the user more power, it is also adds confusion. The problem is amplified by the fact that the order of the menu options is inverted between the plus and minus icons, causing the user to have to very carefully study each phrase to figure out which filter they’re applying. I would get rid of this popover.

So, on balance I think Foodily is an elegant, usable recipe search site that I plan to use myself. It just has a couple of small nagging issues that the team will hopefully work to improve.

Foodily’s response

“Here at Foodily, we throw around the word love a lot. It’s in the origin of our name after all (Food, I Love You), but as company’s design lead, I wish I could be more original than to say I reaaalllly love this review. You’ve touched on many of the design decisions that we’ve labored over and re-visited many times over the last year. For example there we many nay-sayers for the double inputs on the homepage. But as you’ve noted they do more than just provide an extra control. Together they state what we do differently from a conventional search engine and they do so without requiring much effort from users. We’ve worked hard to eliminate everything that is not completely valuable to users (my personal motto is “a pixel is a terrible thing to waste”) and with feedback like this we can get even better. It’s been a long road to our launch. Thank you so much for making sweating the details seem so worthwhile. Cheers from San Francisco.” - Foodily

UI Components for Search

2011-01-14T00:00:00-08:00

Last week we published an article on UX Magazine under the title, From Pattern to Component, and released a new section of our website revealing more about TwigKit’s user interface components. What gives?

We’ve been very pleased to see so much conversation taking place around good design patterns for search. In the last year and a half, Marti Hearst has published Search User Interfaces, Peter Morville written Search Patterns and popularised his Flickr collection, and Endeca released a design pattern library for search. All this discussion has been immensely valuable for increasing both search literacy in the UX world, as well as greater UX literacy in the world of search.

But there still seems to be a disconnect: now that there’s a greater understanding of how to best design search, why do so many companies’ search experiences still su… ahem still have so much room for improvement?

The answer is that even though we now have a better set of blueprints, the construction process is still costly and time-intensive. Many companies just don’t have the budget or timeline to go beyond a barebones implementation of search.

Our mission is to help companies achieve an uncompromising search user experience, while simultaneously reducing the time and cost of development.

But it doesn’t have to be that way. At TwigKit, our mission is to help companies achieve an uncompromising search user experience, while simultaneously reducing the time and cost of development. We accomplish this have-your-cake-and-eat-it-to wizardry through user interface components. While design patterns consist of words and pictures, TwigKit’s UI components are composed of all the HTML, CSS, JavaScript, and server-side code needed to make them work. They’re also made to be easily and highly configurable, quickly tailored to the exact needs of each project.

So, we encourage you to take a look at our gallery of UI components and their corresponding code snippets, read From Pattern to Component on UX Magazine, and take a look at our 75-second video on the TwigKit Markup Language. As always, get in touch with us if you’d like to learn more about TwigKit, and do watch this space; we’re very excited about what’s in store for 2011!

Search Solutions 2010 (BCS-IRSG)

2010-10-21T00:00:00-07:00

Today I was fortunate enough to attend the Search Solutions conference in London put on by the Information Retrieval Specialist Group of the British Computer Society. Here are my notes from each talk of the day.

Behshad Behzadi on Web Search Freshness

When searching the web, it’s crucial to strike a balance between new results, and very relevant results.

Sometimes users express a desire for fresh results in their query, such as “latest news chilean miners.” In this case, the user’s desire for recent results is obvious.

Other times, users don’t explicitly ask for freshness in the query, but it can be inferred if that query is spiking in popularity. For instance, when there is a large surge of users searching for “chilean miners,” they are probably looking for news stories, not encyclopaedia entries.

Some queries have seasonal patterns. “American Idol Winner,” for instance, should show results for this year’s winner, not last year’s. But this isn’t universally true — a query like “Turkey recipe” shares the same query trending pattern as “American Idol Winner”, but this year’s turkey recipe is not necessarily better than a recipe from 5 years ago.

Challenges to freshness

Different queries have different freshness granularity needs. (Traffic = minutes, news = hours, politics = days)
How to you know how old a result is? It’s difficulty to actually reliable pin down the age of the page. The search knows when the page was first indexed, but not necessarily when it was originally created. It’s also difficult to know how fresh the page is based purely on when it was modified (because a small modification to the page may not actually affect the main content of the page).
When there are many fresh results, how do you decide which ones are worth showing? Freshness is at odds with relevancy.

Solutions to freshness

Show the date in the description synopsis
Allow users to restrict results to a particular date range
Offer a realtime search option

Vishwa Vinay on Click evidence — Signals and Tasks

One approach to scoring results is to rank them as a probability that the document will be clicked. This score can be constructed by using each click on a result as a “vote” for that document.

An advantage of a probability-based ranking is that the scores are comparable across indexes, making it easy to merge results when doing federated search (50% is always better than 25%).

Downsides of probability ranking are “rank bias” and “lock-in”, where users click on the first result not because it’s the most relevant, but simply because it’s the most prominent. To combat this, the search engine can perform A/B testing of results by ordering them in descending order in one case (1,2,3), ascending order in another case (3,2,1), and comparing the outcome.

Another method to combat “rank bias” is to keep track of how many results are clicked by a given user searching for a given query. If the user clicks on three of the results, for instance, it can be inferred that he is most satisfied with the last document clicked. If the user only clicks a single result, it’s a strong sign of satisfaction with that document.

Vivian Lin Dufour on How to help searchers become better searchers

Yahoo track trending topics to help users discover timely results, and use several techniques:

Query suggestions
Result suggestions
Related searches

Nick Patience on The trends shaping the future of enterprise search 2010-2013

There are three trends affecting enterprise search.

Information Governance

There are several opportunities for search in information governance:

Archiving
eDisclosure
Records & storage management

There two approaches to information governance, both involving search technologies:

Store everything and rely on a great search engine to find it (simple, but large volumes of data)
Rely on a great search engine and dedeuplication to selectively delete what is not needed and then search what’s left (harder to achieve, but saves on data volume).

Search-based applications

Search is gaining traction over relational databases as a primary driver of domain-specific applications, and have the potential of being hugely disruptive in data management. Success lies in:

Building partnerships with application vendors
Focusing on developers
Focusing on user interfaces

In 2010, search-based applications tend to be be custom-built and expensive. Over the next few years, expect to see search-based apps appearing more as software as a service.

Enterprise Search

The smallest area of the three, enterprise search for intranets looks destined to be dominated by Google Search Appliance, Sharepoint, and open source search.

Chirag Gandhi on I still haven’t found what I am looking for

The history of enterprise search is one of businesses purchasing very expensive search systems and building very complicated applications, while users have been neglected. Companies often fall into a number of traps.

Gotchas

Throw it all in. Too often organisations thoughtlessly toss any and everything into their search index, rather than carefully considering the documents and metadata
No attention to the user experience. Companies just want it to look like Google
No proper support for multiple languages (especially individual documents that contain multiple languages)
Format conversions — search engines don’t digest a wide variety of documents very well

Trends

Federated search. As organisations begin to centralise their infrastructure, federated search is often required
Finding people, not just documents

Dusan Rnic on Enterprise Search and its evolution

Why enterprise search?

Efficiency
Compliance
Centralised finding

Information growth is accelerating at a phenomenal rate (thanks to emails, file systems, CMS, databases, etc. But there is a long tail of searches (a very few extremely popular queries, but a large body of rare queries). eCommerce was the first sector to successfully tackle the long tail problem, and they addressed with techniques such as dynamic classification, spotlighting results, cross-content navigation, visualisation.

In a word, eCommerce websites helped shoppers explore and discover products by focusing on the user experience. By telling using what content is there (through the likes of faceted navigation), you help users discover the needle in the haystack. “Start any search project by thinking about the user experience.”

Greg Lindahl on Instant indexing

Blekko are attempting to build a brand new web search engine from scratch. The company was founded in 2007 with $24 million of investment, and they’re currently in private beta. They are attempting to completely reinvent the technologies used in web search, and apparently they have a unique approach to indexing.

Charlie Hull on What’s the story with open source?

Search is no longer a bolt-on, but a platform for innovation, and open source is no longer the outsider.

Indexing

Content is created for publication, not for search
Content isn’t published consistently or available to all
Ranking is never simple
Must be able to publish rapidly
Essential metadata — byline, title, source
Content restriction and embargo data
Solution: lightweight, customisable index scripts using open source libraries

Searching

Free text with boolean operators
Filters for metadata and date ranges
Combine date and relevance ranking
Faceted search
Saved searches and alerting
Similar results
Solution: template-based user interface scripts, again using open source libraries

Why open source?

Flexible, extendable
Powerful and scalable
Lower cost
Commercial support is available
Freedom to innovate

Looking to the future

More and more content including social media
Multiple delivery platforms
Search-powered websites and applications
No-SQL
Cloud computing

Roberto Cornacchia on Search by strategy

Current search engines do a poor job of understanding what users are actually looking for. For instance, “amsterdam canals balcony faces west” is unlikely to return desirable results from a traditional search engine. The Spinique approach break the query down into its component parts (location, desired attributes, etc). What’s unique about Spinique is that it doesn’t these attributes as cut and dry filters, but instead provides sliders for the user to indicate the importance of each attribute.

The combination of attributes expressed by users can be thought of as domain-specific preferences, and can be used to influence the ranking of results for users in general.

Till Kinstler on Current trends in library search

I wasn’t able to attend this talk, unfortunately.

Mihai Lupu (Information Retrieval Facility) on Scaling up innovation

The innovation cycle brings together information professionals, scientists, and technology experts to share one technical platform, one set of standers, and one community.

The Information Retrieval Facility (IRF) was founded in 2007 in Vienna to encourage cooperation between researchers and industry. Their key areas of research include multiple indexing, text annotation, information extraction, document categorisation, image retrieval, and machine translation. They are looking for partner companies who can help put their research into practice in the real world.

Rob Stacey on Reconciling facts

True Knowledge are a Cambridge-based company who strive to provide direct answers to user’s questions (rather than just a list of search results). They currently know 300 million static facts. True Knowledge check new, incoming knowledge, and compare it to previously existing knowledge. When they contradict, the historically more reliable source wins out.

Panel: What will search look like in 2015

Stefan Rueger (Open University): Search will be more immersive than it is now, performed less at the computer, and more on the move in the real world. Search will also offer more of a browsing experience than it does now
Jody Goodall (Trader Media): More data, and more commoditisation of data.
Charlie Hull (Flax): open-source search will be the dominant player as search technologies become commoditised. There will be an increase in the number of applications that are driven primarily by search, as opposed to search being an after thought.
Nick Patience (451 Group): Open source search will be big, there will be more leveraging of social search tools to improve relevancy.

In Summary

While Search Solutions was heavily represented by computer scientists and academics, there were quite a few references to the importance of the user experience of search. Dusan Rnic from Endeca, for instance, advocated: “Start any search project by thinking about the user experience,” while Nick Patience, an analyst at 451 Group, claimed that the success of search-based applications lies in “a focus on the user interface.”

Thanks to Tony Russell-Rose, Andy MacFarlane, Alex Bailey, Leif Azzopardi, and Udo Kruschwitz for organising a great event!

Why you should start your own search meetup

2010-10-19T00:00:00-07:00

About this time last year Stefan Olafsson and I were running around a search conference handing out hastily-prepared fliers to anyone who would take them. The fliers were advertising a brand new event about a month later that we were calling the “Enterprise Search London Meetup.”

When the evening of the meetup came, exactly two of our friends showed up. Not the crowd we were hoping for, but enjoyable conversation nonetheless.

At the second meetup about a month later, we had eight people show up. Then, we decided to start doing things properly and arrange for someone to give a talk at the third meetup. So, we booked the only person who was willing to do it: me.

Since these very humble beginnings we’ve hosted a number of excellent presenters including Tony Russell-Rose from Endeca, Vegard Sandvold from Comperio, and Udo Kruschwitz from the University of Essex, and drawn an average crowd of around 30 people. Last night we held the most successful meetup yet in conjunction with The Guardian. Martin Belam curated an outstanding event of 7 lighting talks that drew a crowd of 60 participants (with many more on the waiting list).

What’s you story?

As you can tell from our experience, you don’t have to be a rock star to start a meetup. Anyone can do it. All you need is enthusiasm and a bit of persistence.

A week and a half ago at the Lucene Revolution search conference in Boston we launched SearchMeetups.com, an initiative to encourage other people to start search meetups in their own city. We’ve seen how the London meetup has helped people grasp the true value of search and encouraged greater awareness of best practices, and we want to see the same effects take place at a global level.

If you are interested in search (which you must be since you are reading this), then you should consider starting a search meetup of your own. Not only will you be giving back to the community, but you’ll get to meet new people and be seen as thought leader.

So what are you waiting for? Read our tips on SearchMeetups.com and get started!

The Scent of Search

2010-07-05T00:00:00-07:00

Search is an evolutionary, iterative process. Like a Grizzly Bear foraging for food in the forest, people jump from one information source to the next as we seek to satisfy our curiosity.

Presented at Apache Lucene EuroCon 2010 in Prague and as an article on Johnny Holland Magazine, “The Scent of Search” seeks to apply the principles of Information Foraging Theory, and in particular, Information Scent, to the usability of search interfaces. Below are the main recommendations we set forth in the talk.

The Search Box

The searchbox should look like a searchbox. Drastically changing its appearance will result in fewer users discovering it.
Place the searhbox in the top right corner of the page. Users have come to expect it in this location, so moving it anywhere else will reduce the number of users who actually find it.
Provide as-you-type query suggestions. Search suggestions reduce spelling errors, save time, and make users more confident of their query.

Search Results

Indicate the number of results. This helps users gage the validity of their search. Numerous results can act as a vote of confidence, while few results may raise a red flag.
Titles should be easily comprehendible. Filenames make for lousy titles. Use natural language titles that accurately describe the contents of the result.
Matching words should be highlighted. Emphasising queried words when they appear in the search results makes the result list easier to scan.
Make sure visited links are easily discernible from unvisited links. Visited link colours prevent users from accidentally reviewing the same result twice.
Differentiate results when real differences exist. Clearly identifying which category a result belongs to can make results easier to scan.
Avoid having zero results. A search results page with no results is a roadblock to users that could make them give up searching altogether. Consider using automatic spelling suggestions.

Faceted Navigation

Indicate the number of results that match each filter. Filter result counts give insight into the shape of the data.
Use breadcrumbs to reflect the user’s query and applied filters. This helps users know where they are, how they got there, and get back if necessary (breadcrumbs should be removable).
Make field values clickable. Important fields in the search results should be applied as filters when clicked, making it easy for the user to filter the results.
Find ways to meaningfully visualise facets. Some facets may work best as a list, others as a chart, still others plotted on a map.

The Google Redesign

2010-05-05T00:00:00-07:00

This morning I got out of bed, ate my cereal, took my shower. Everything was proceeding pretty predictably. But then I did a Google search — usually a pretty mundane task — but this morning, Google looked very different than it did yesterday.

Word on the street is that Google is rolling this new design out to everyone over the next 48 hours. As with any change, some people are bound to complain, but I think the redesign introduces many significant improvements.

What’s Changed

The Google logo is about 30% larger.
There is now a permanent sidebar left of the results that allows filtering by news, blogs, images, etc., as well as time range filters and options for changing how search results are displayed.
The searchbox is now the full width of the results column, slightly taller, and has a slight drop shadow rather than the previous inner shadow.
No more top vertical bar. The result count now sits just below the searchbox and is much smaller than before. The filter for searching locally moved from under the searchbox to the sidebar.
Search results are now 55 pixels higher on the page and have have a higher density overall (there’s slightly less vertical space between results, and indented results have only one third of their previous margin).
Cached, Similar, Show more, and Related links all changed from a muted purple to a brighter light blue and now only have an underline on hover.
Pagination is about 30% larger, and there is still a searchbox below the pagination, though the blue background has been removed.
Related searches are now displayed much more compactly.

The Sidebar

The new sidebar provides an inviting mechanism for narrowing your search to a specific channel. In particular, blogs and books now have much greater emphasis than ever before. Hidden under the “more” button are updates and discussions, and lower down in the sidebar are “latest” and “past two days” time filters, so there is definitely a push towards recent and real-time search. What isn’t clear to me, however, is why the sidebar filters are in a different order than the top bar options, and why some items from the top bar (shopping, for instance), didn’t make it into the sidebar at all. It definitely seems like Google should ditch the topbar options completely and place all these channel filters in one consistent location.

Wonder Wheel

There is also an option to show the wonder wheel, which is another mechanism for presenting similar searches. Instead of showing a simple list, however, the wonder wheel groups related searches into clusters and allows the user to navigate from one node to another. While this visualisation seems useless for simple lookup — finding the population of Spain, for example — it could be useful for exploring broad topics (such as “UK Politics”). My biggest complaint is with its name. “Wonder wheel” is completely undiscoverable and has absolutely no information scent. It’s impossible to guess what it does without actually interacting with it.

In Conclusion

The Google redesign offers many improvements — the sidebar is a step in the right direction, the searchbar feels much more intentional. Not all of additional views, from the wonder wheel to the timeline, feel all that useful, and the top bar now seems redundant. But overall the design itself feels crisper and more concise. I like it.

ECIR Industry Day 2010

2010-04-01T00:00:00-07:00

The event consisted of 12 different speakers each presenting for exactly 20 minutes, with about 10 minutes of Q&A after each. I particularly enjoyed the presentations from the major search engines: Yahoo, Google, Bing, and Wolfram Alpha. A topic that seemed to arise in each of those talks was how query reformulation data can provide a feedback loop to make search better. But without further ado, here are my summaries of each talk.

Mining the Web 2.0 to Improve Search

Ricardo Baeza-Yates, Yahoo Research

Ricardo Baeza-Yates talked about how web usage data can to be used improve relevance and accurately provide related queries.

He first talked about how user tagging can be combined with user-generated image annotations to increase the relevance of image search. What I found most interesting, however, was how Yahoo keep track of all the iterative queries people enter in order to get to a specific result (for example, if the user typed ‘furry animal,’ reformulated the query to ‘white black furry animal,’ and then chose an article on Pandas, Yahoo would associate both queries with ‘Panda’). Yahoo would then use this data to suggest related queries (i.e. if the user searched ‘furry animal’, Yahoo would suggest ‘white black furry animal’ as a related concept).

Google Squared: Web Scale, Open Domain Information Extraction and Presentation

Dan Crow, Google

Dan Crow opened by asserting that complex tasks, such as planning a trip or writing a research paper, are still very difficult on the web. Google’s user studies revealed that people make spreadsheets, email themselves things to remember, and add post-it notes to their computer monitors when undertaking these complex search activities. They also found that people seemed to love tables (and chose to adopt this table style for data presentation themselves).

Google Squared is an attempt to help people cope with comparison-heavy, list-driven search activities and it operates at three different levels. First, it seeks to discover the topic behind the user’s query (does “Ford” refer to U.S. Presidents or the car manufacturer, for example?). Once the topic has been isolated, Google Squared then tries to find attributes of that topic (price, horsepower, colour). And finally, it tries to fill in the values for each of the attributes.

Google Squared combines offline analysis (such as mining wikipedia categories and combing the web for attribute / value pairs) with run-time queries for finding specific values.

Relevance Challenges at Bing

Milad Shokouhi, Microsoft Research

I really enjoyed Milad Shokouhi’s talk about clever ways of boosting relevance in search, though I must say it stretched my vocabulary a bit. His initial slide outlining the challenges to relevancy included temporal queries, heterogenous verticals, and pre-retrieval query alteration. He did go on, however to clearly articulate each and offer concrete examples from Bing.

Like Ricardo Baeza-Yates, Milad Shokouhi touched on how query reformulation data can be used to enhance relevance of ambiguous terms. An example from Bing was the ranking of results for the query “wow.” The unadjusted top hit for this query was a cable company. But a look at the query reformulation logs indicated that a majority of people who queried for “wow” went on to search for “world of warcraft.” Bing then adjusted the ranking of the results so that world of warcraft appeared first.

He also talked about query trends, specifically the “spiking” of certain queries. Some queries quickly spike, but then quickly disappear. Other queries (like “iPad”), spike and remain high. Still others spike seasonally (such as “Halloween costume”).

These spikes of queries could be used to trigger the appearance of a news story, seasonally suggest related queries, or even forecast future events (elections, for instance).

Search User Experience, the Essentials of Great Search Design

Vegard Sandvold, Comperio

Vegard Sandvold argued for cross-disciplinary collaboration. Everyone should be involved in the design process, from stakeholders, to techies, to users. “Innovation happens happens where disciplines intersect.”

Vegard advocated a “Sprint Zero” phase of one to four weeks in which to set forth a plan of action for the project. During this time, he typically talks to stakeholders about their goals, interviews users, creates personas, and attempts to identify all of the problems that must be solved in the project. He also strives to prototype and test both the basics of the interaction design and the capacity of the underlying technology.

His talk can be summarised by his final slide: “We discover the best solutions together.”

Getting Value from the Search Master’s Toolbox

David Hawking, Funnelback

David’s talk was primarily about fine-tuning relevance in enterprise search deployments within organisations. He advocated picking a selection of random queries (that realistically reflect the distibution of queries within an organisation), trying those queries yourself, and tuning the engine until those queries produce the most relevant results.

Enterprise Search: State of the Market 2010 & Beyond

Nick Patience, The 451 Group

Nick Patience is a market analyst and provided some revealing insights into the industry. It’s a bit smaller than I would have guessed: in 2009 it totalled $1.3bn and is estimated to reach $2.8bn by 2013 (at a 22% annual growth rate).

Project Plaza – A New Approach to Information Management in the Construction Sector

Rob Blackwell, Active Web Solutions

Rob and his colleague demonstrated a search-driven web application for managing construction projects.

Rethinking the Library Catalogue

Sally Chambers, The European Library

Sally Chambers from the European Library talked about the challenges they faced in integrating a vast number of library corpora using a federated search approach. She described the issues they faced in providing relevant results due to the asynchronous nature of the system; where certain searches might time out and providing a blended set of relevant results was not feasible.

With users expecting the same user experience they get on the web, they initiated a move towards a single index of bibliographic and full text (including optically recognised) content. Some of the more persistent challenges that still remained had to do with the multi-lingual nature of the content, and the disparity in the metadata formats used by the individual institutions (such as MARC vs. Dublin Core).

Collaborative Research, Technology Transfer and Networking

John Tait and Francisco Webber

John Tait presented Francisco Webber’s ideas on the information retrieval innovation cycle. He advocated enhancing a mutually beneficial ecosystem between academia, government, and industry by aligning the incentives: academics need to publish papers, politicians need to get re-elected, and businesses need to generate revenue.

He insisted that innovation in technology hinges on open data. There was a rigorous discussion in the Q&A over the merit and drawbacks of open business models. A member of the audience asserted that companies aren’t realistically going to give information away for fear of competition exploiting it. John defended this criticism well by arguing that businesses who have embraced open business models have benefited from higher volume and the formation of communities around their products (his example was GE).

Wolfram Alpha – the New Computational Knowledge Engine

Jon McLoone, Wolfram Alpha

Jon McLoone began by saying that knowledge consists of four components: opinions, facts, methods, and understanding. While traditional search is geared towards retrieving opinions and facts, Wolfram Alpha focuses on revealing method and understanding, while leaving out opinion altogether.

The Wolfram Alpha approach asserts both that the desire for information does not indicate the ability to use it, and that computed data is more valuable than data alone (for example, turn-by-turn directions are more valuable than just a longitude and latitude).

This computational approach is evident in the examples that Jon demonstrated. Every type of data has a corresponding visualisation. Planets get plotted on a sky chart, financial markets are graphed, chemical elements are shown on the periodic table of elements, and recipe ingredients are even turned into a nutritional value chart.

One comment brought up in the Q&A session was the discoverability of all these visualisations, which is a concern that I share.

Using AI to get Answers from the Internet

Simon Overell, True Knowledge

True Knowledge is a platform for answering questions. It takes natural language questions and returns numerous well-cited answers by gathering information from both structured and unstructured sources.

Panel Discussion: Leveraging Semantics to Enable Better Search Experiences

Dan Crow (Google)
Gjergji Kasneci (Microsoft Research)
Jon McLoone (Wolfram Alpha)
Simon Overall (True Knowledge)

The primary thrust of the panel discussion was a debate over curated knowledge verses the wisdom of crowds.

Google Squared doesn’t bake in any knowledge or curate any information, it relies completely on scraping the open web in search for the answers. Wolfram Alpha, on the other hand, relies completely on authoritarian facts. True Knowledge takes a middle ground of presenting differing facts and opinions from around the web, and clearly citing the sources.

Search Suggestions

2010-02-08T00:00:00-08:00

You used to be expected to type for yourself. But today people have come to expect a reasonable amount of help at even this task. Our phones now help us form correctly-spelled words, our browsers fill in long addresses after we’ve typed only a few characters, and search engines recommend searching for “Humphrey Bogart” after we’ve typed just “boga.”

But not all as-you-type search suggests are created equal. Careful observation seems to reveal three different approaches: completion, suggestion, and instant results. These approaches range in cognitive burden on the one hand, and utility on the other. We’ll look at several examples of each and consider when they should be used.

Completion

The simplest of these three approaches is query completion. The sole role of completion is to reduce the friction involved in getting an idea out of your head and onto the computer screen. Just like a fluent translator improves the conversation between two people who speak different languages, so query completion helps a person and a computer communicate a bit more coherently.

Completion makes the most sense when dealing with a narrow strand of finite data. It’s commonly employed for concepts like geography (airports, cities, countries), finance (ticker symbols, company names), and people.

Query completion has many advantages over a plain-text search. It saves the user time by cutting out unnecessary keystrokes (i.e. I could type “Birm” and accept the suggestion of “Birmingham”). It also helps prevent spelling mistakes (I can just type “Mitsub” instead of remembering how to spell “Mitsubishi”). And, unlike some of the other forms of search suggestion, query completion does not suffer from being complicated or mentally tiresome. On the contrary, it can be such a natural behaviour that we almost don’t even realise it’s occurring.

My consensus, then, is to use completion whenever and wherever possible. It will make your users more productive without the risk of confusing them.

Suggestion

Suggestion is by far the most ubiquitous auto-complete approach. The obvious examples are search engines such as Google and Yahoo, though it’s used on virtually every category of website.

While completion helps get an idea from your head onto the screen, suggestion actually throws new ideas into the mix. It’s not too dissimilar to what happens when you tell your travel agent that you’d like to go on holiday. The agent comes back with a handful of suggestions like the Swiss Alps, Jamaican beach, or Vietnamese jungle. Similarly, if I type “guitar” into eBay’s search field, I get suggestions for “electric guitar,” “acoustic guitar,” and “guitar hero.”

Not only do suggestions save the user keystrokes, but they can actually help construct a more useful query than the user would have thought of on their own.

But there are a number of opportunities for maximising the power of query suggestions that are often overlooked, but which Google seem to get right. The lowest hanging fruit of these are spelling suggestions. If I type “guitaf” into Google, the first suggestion that appears in the list is the correctly-spelled word, “guitar.” Surprisingly, this elementary feature is missing from giants like eBay, Amazon.com, and Yahoo. A second technique is to display the number of results for each selected term, as Google do with a right alignment. This has a risk (which we’ll get to in a bit), but it can help the user gauge the popularity of a given suggestion. And lastly, it’s often helpful to infuse suggestions with recent searches. One approach, which Safari utilises, is to simply have two groups, one for query suggestions and another that lists the user’s recent searches. A more sophisticated approach could be to use search log data (either for one user, a group of users, or all users) to augment the scoring of the query suggestions themselves.

When deciding what features to include in search suggestions, it is important to consider the mental load that they will exert upon the user. Every addition of new information will add precious milliseconds of processing time for the user’s brain to digest that information. The trick is to strike the right balance between usefulness and cognitive burden within the context of your application.

Instant Results

The third approach differs greatly from the first two forms of search suggestions. In fact, instant results doesn’t offer query suggestions at all, but instead offers actual results to the user, often organised into a handful of categories.

Apple is perhaps the most well-known purveyor of this technique, both on their website and on OS/X’s desktop search, Spotlight. When I type the letters “mou” into the search box on Apple.com, I get five search results, the first being Magic Mouse. If I accept that suggestion, however, I am not taken to a search results pages with the query of “Magic Mouse,” but am actually taken directly to the Magic Mouse product page.

A key feature of instant results is that it can often diminish the need of having a dedicated search page. Instead, search results are provided in a contextual drop-down without leaving the page. The benefit is of course that the user is able to view results as they type, cutting out the additional time required to load a second page.

However, there are risks associated with instant results. First of all, the ability to provide query suggestions is lost since results are shown instantaneously. A second concern is that instant results usually only have space to show a handful of results, and often require quite a lot of space even at that. More specifically, instant results carries with it a much greater cognitive burden on the user than do either completion or query suggestion. It simply takes more time to parse through the multiple categories and apprehend all the metadata associated with each result.

But there is a time and place for everything. Instant results work great when there is highly structured data neatly divided into a handful of categories. Take the forthcoming Nutshell CRM application, for example. The search field only covers three entities: leads, accounts, and contacts. In this context, query suggestions would be useless. It would also be very rare for a result to be buried very far down the list, so the user would almost never need a full page of search results. In this situation, instant results helps the user find what they’re looking for quickly, and with as little friction as possible.

These three approaches differ greatly in both their usefulness and the effort required to use them. Completion is easy to understand and almost always results in a net benefit for the user. Suggestion, because it throws new ideas in the mix, is slightly more complex, but that is usually outweighed by its helpfulness. And finally, instant results work well for simple, highly structured data, but at times runs the risk of overwhelming.

Further Resources

Tyler’s Flickr collection of search suggestion examples
Peter Morville’s collection of search patterns
Google’s original blog post announcing the release of search suggestions
Jared Spool’s article on spending quality time with your search log
Smashing Magazine’s article on designing the search box

The Deck

Tyler presented a talk on search suggestions at the Enterprise Search London Meetup on February 4, 2010. The presentation is available on SlideShare.

Security in Search Applications

2010-01-22T00:00:00-08:00

When a search engine is brought to bear on content with restricted access, it becomes evident that security and preserving the integrity of permissions can be an important and often thorny issue.

The problem

Consider the case when enterprise search is applied to a document management system. Each and every document might well end up in the same index, meaning that when searches are conducted it is imperative that confidentiality is maintained and users are only shown results they’ve got access to. This is typically known as ‘document-level’ or mapped security, where each piece of content is accompanied by the associated security metadata (known as an Access Control List, or ACL).

The problem is further compounded by the fact that search engines are being increasingly used to break down silos of data by making searchable, disparate sources of data that don’t share a single access control system.

Under the hood

This security metadata, or entitlements, get stored in the index, thus forming a part of the query evaluation process carried out by the search engine. What that means is that the set of documents found to correspond to the user’s query will be further whittled down by matching this metadata with the user’s credentials (authorisation). Given how intrinsic this is to the overall process of matching, this piece of the puzzle will generally fall under the remit of the underlying search engine. The other constituent part is the authentication or the process of establishing who the user is, so that his credentials (such as the groups he belongs to) can be provided along with the query. However, many providers of search technology have seen the former as their only remit in the security life cycle. To be fair, this lack of vendor support for the entire security life cycle has been somewhat understandable, given the plethora of authentication mechanisms out there. But the problem’s not going away.

How do we deal with it?

Given this, and the fact that almost every internal search project faces this dilemma, we often have to provide or facilitate a solution. We therefore chose to provide a TwigKit Security Module which offers integration with leading authentication providers such as Active Directory, LDAP servers, databases, OpenID, and proprietary systems, via a variety of methods and protocols such as form-based, basic, NTLM, Kerberos) delegation, X.509 certificate exchange, and ‘container managed’ authentication. What this means is that if an organisation already has a single sign on infrastructure we’re can easily tap into that, but equally importantly, in greenfield scenarios, we can quickly implement an effective security solution. Thankfully, there was no need to reinvent the wheel since the Twigkit Security Module builds on the industry-leading Spring Security framework.

The TwigKit Security Module essentially fills three roles. First, it initiates the authentication of the user by using any one or combination of the supported methods. Second, once the user is authenticated, it will pass the identity of the user to search engines that support security using each platform’s specific methodology (e.g. the FAST Security Access Module). Finally, the TwigKit Security Module provides simple methods for restricting access to individual components or aspects of the search user interface. This allows us to in effect go a level deeper than document-level security, restricting access to individual fields (somewhat analogous to column level security in a database).

Some code

Let’s say our search user interface is built using the TwigKit Tag Library and provides access to some kind of product catalogue. In this case we might want certain parts of the product information to be only available to a group of ‘Product Managers’. This could be information stored in the search engine, such as as profit margins, or functional aspects of the interface like the ability to manipulate discount levels. In the following code sample you can see how you would go about displaying product attributes such as features and price, whilst restricting other parts to members of the group “PRODUCT_MANAGERS”:

<search:field fieldName="productFeatures" label="Features" />
<search:field fieldName="priceGBP" label="Price" prefix="£" />

<!-- Only show margins and discount level link to 'Product Managers' -->
<security:conditional allowedGroups="PRODUCT_MANAGERS" deniedGroups="">
    <search:field field="profitMargin" label="Profit Margin" />
    <a href="/EditDiscount/1234/">Change Discount</a>
</security:conditional>

Any information or components placed within the ‘conditional tags’ from the security library (highlighted) will only be visible to members of the groups mentioned, or with an alternative configuration, visible to everyone unless they belong to the groups being explicitly denied access. This example shows you how to interject at a user interface level, but TwigKit provides other hooks to apply similar logic using Query and Response Processors to correspondingly protect against potential injection attacks or do even do last minute checks and filtering. We will be covering processors in a separate post.

To wrap up, other tags from the TwigKit Tag Library provide more innocuous facilities such as providing particular attributes and features for authenticated users:

<security:userDetails authenticated="true">
    Hello <b>${user.name}</b> (<a href="/logout">sign out</a>)
</security:userDetails>

If you’ve come across other frameworks that tackle this, or just have some thoughts on the matter, then we’d really like to hear from you – so put it in a comment or drop us a line.

Final thoughts

Unfortunately it’s hard to do such a complex topic proper justice in a short post like this. Admittedly it covers the problem at a very high level, ignoring other aspects of securing search such as ensuring the integrity of the search index when permissions change, and the overarching concerns addressed in security policies (such as physical and network security).

Despite these caveats we hope it you found it helpful from a functional perspective. Last, but certainly not least, I want to thank my good friend and search guru Christian Moen from Atilika for all his invaluable thoughts on the subject.

Making Data Meaningful

2010-01-17T00:00:00-08:00

Most modern enterprise search platforms provide some inherent capability to illustrate the shape and nature of the data within. Take for example faceted search.

Facets will quickly break down the dimensions in all the data we’re storing or even just the stuff that meets our search criteria. In either case we can get some form of statistical feedback e.g. on which top-level categories exist, their names and how many documents each represents. Take this search for positions as a ‘project manager’ as an example. Using faceted search, we can quickly see that some of these are are in the ‘Engineering’ field, with still more for IT professionals.

Not only does this give the user insight into what information is available, but also guides them in their search, allowing them to slice and dice the data to get precisely to the information they’re after. The question is, how do we best represent this information and make it useful (and meaningful) to us?

As you saw in Tyler’s previous posts in most cases there might be sufficient utility in just getting the broad strokes, preferably in a manner that minimises the cognitive burden of taking it in. In some cases proportions may give us the visual cues we’re after. For example it may be useful enough for us to see that there are 1) almost no orders pending shipment this week (phew), 2) a bunch in transit, with 3) the vast majority already delivered. And, thanks to faceted search all the detail on each group or dimension is a mere click away.

Facet information displayed with TwigKit as a 3D pie chart To achieve this, the TwigKit UI libraries provide widgets that will turn facet information from the search platform into pretty pictures, charts and graphs. Traditionally, a developer would have written some code to extract the necessary information from the facet, integrated a visualisation library, and displayed the result on a web page. But we’ve done all that for you.

In the code snippet below you can see how to create visualisations using the TwigKit JSP Tag Library. All you’d need to do is specify which facet to display, the format (such as column, line or pie chart) and the result is an interactive visualisation – where clicking a particular aspect will further refine your search. Easy as pie :)

<widget:facetChart
    type="Column3D"
    facet="${response.facets.manufacturer}"
    numberOfFilters="6"
    color="ffbb33"
    backgroundColor="fbfbfb"
    query="${query}"
    width="700"
    height="250"
    title="Top Manufacturers"
    subTitle="Number of products per manufacturer"
    showAverage="true" />

Simple example of Facet information on products, broken down by manufacturer and represented as a column chart. The important thing here is that search engines have a myriad of ways to efficiently mine vast volumes of data, providing insights that simply weren’t achievable in the traditional relational paradigm. However it is often the little things that transform that analysis into meaningful, every day tools that truly alters the way we consume information.

Behind the Scenes at ITV

2010-01-14T00:00:00-08:00

Just before Christmas we put the finishing touches on a prototype internal search application for British broadcaster ITV. We’ll be working with ITV in the coming months to roll out the application across their entire organisation, but we wanted to give you a sneak peak in the meantime.

Information Architecture: The Big Picture

The application searches ITV’s product catalogue, which contains a large archive of television shows and scheduling information. Our largest effort surrounded the information architecture for the application. Our goal is always to create an empowering user experience that helps people easily digest information and nimbly find what they’re looking for. To accomplish that goal, we first had to understand the data ourselves, and, just as importantly, figure out how the different user groups at ITV understand the data.

We realised that seemingly everyone at ITV was familiar with the organisation’s programme hierarchy of programmes (such as “The Office”), series (Season 2), episodes (Episode 1), and finally productions (International Version). Because it was so ubiquitous, we chose to make it the backbone of the search application. We placed the hierarchy prominently at the top of the page and branded each piece of the hierarchy with its own colour, which we also used to categorise each search result. Our goal was to help users easily traverse the hierarchy and to instinctively understand the level of each result.

The hierarchy remains prominent after the user drills down to a single result. We included a breadcrumb trail so that the user is aware of her current position. In addition, every detail page (of a programme, for instance), lists all of its children (series, in this case). This means the user can continually navigate both upwards and downwards throughout the hierarchy.

Information Architecture: Metadata

In addition to thinking about the top-level hierarchy and navigation, we also focused on the presentation of metadata. We wanted the metadata to enhance the search experience and help the user hone his search. We also wanted to prioritise the information, make the important bits really jump out. We worked to shape the data in two primary ways:

Any metadata that the user can facet on gets styled as a capsule. We chose to style keywords in tag-like containers, and other metadata, such as “partners,” in pill-shaped capsules. This communicates to the user that the items are clickable and will refine the search query.
Important metadata should be visually prominent. (It’s obvious, we know.) We created large boxes to display the most important metadata and placed the boxes at the top of the page. Users should not need to hunt for important information, it should jump out at them.

Data Visualisation

If information architecture was our top concern, data visualisation was a close second. The ITV search application offers two primary perspectives on the data—a catalogue-centric perspective and a schedule-oriented view. We utilised visual metaphors that fit best with each of the two perspectives: a tag cloud for for the keyword-based catalogue view, and a date histogram for the scheduling information.

Video Walkthrough

Common Problems with Pagination

2009-12-19T00:00:00-08:00

The purpose of search is to help people find what they’re looking for as quickly as possible. Search engines attempt to facilitate this by taking the user’s query and responding with results, placing what it thinks are the most relevant results first.

Unfortunately, pagination doesn’t always do the best job of guiding the user through the search results in the most beneficial manner. I’ve noticed three common problems.

Problem 1: The “Last” button

If the first page of results is the engine’s best guess at what the user is interested in, the last page of results is inherently the engine’s worst guess. Which begs the question: why do you often see a link to the last page of results?

For the user to find what they’re looking for, it’s important to get highly relevant results in front of them. Encouraging users to look at irrelevant results is simply counter-productive. Avoid having a “last” link.

Problem 2: Pagination above results

A second harmful practice is placing the pagination above the search results. Again, the purpose of search is to present the most relevant results first. It doesn’t make sense to encourage users to skip to page 2 before they’ve encountered all of page 1.

Problem 3: Next

If the first two problems are akin to sending the user on unnecessary detours, the third problem is like not giving the user a green light when we should.

Imagine you enter a query, scroll down the page as you look over the results, and reach the bottom of the page without finding what you’re looking for. What would you want to do next? Go to page 7? No, you’ll most likely want to go to the next page of results. So, why doesn’t the pagination always offer us a big, whopping “Next” button?

In Summary

All three of these side effects are the result of one illness: forgetting to optimize pagination for helping users move from highly relevant results to less relevant results in a proper progression.

For TwigKit, we also wanted to make it easy for people to get back to the earlier pages of more relevant results once they have navigated to a later page.

Precise to a Fault

2009-12-05T00:00:00-08:00

Can a number be so precise that it actually hinders users rather than assists them?

Take search results for example. It is common practice to indicate the total number of results found for a given query. Google tells me that there are 221,000 web pages about “rubber duckies,” while a recipe site indicates that there are 113 salads and 7 desserts with “avocado.”

I can think of two good reasons for why showing the number of search results has become a standard. First of all, it provides a sanity check. If “Barack Obamaaa” only returns 3 results, it is an indicator that I’ve made a mistake in my search query. Second, it helps me compare one collection with another. The fact that there are twice as many two bedroom flats as one bedroom flats in Greenwich informs me about the area.

The problem: big numbers

Knowing that there are 324,768 results is no more helpful to me than knowing that there are about 324,700, which is really no more helpful than knowing that “there are a bunch of results.” And when comparing collections, knowing that there are 41,327 species of vertebrates and 1,589,583 species of invertebrates isn’t nearly as useful as knowing that there are 38 times as many invertebrates as vertebrates.

Perhaps I’ve convinced you that the utility of precise numbers decrees as the size of numbers increase, but can they actually harm the user experience? The danger comes with the added cognitive burden of processing precise numbers of results into something that is meaningful. Result counts themselves are a step removed from what we’re actually after, and it’s the mental cost of translating the abstract to the practical where the problem with precise numbers lies.

The solution: be less precise

The answer to the problem is simple: make big numbers less precise. A less precise number, such as “3 million,” is less taxing to comprehend than a precise one like “3,185,391.”

In TwigKit, we decided to precisely render numbers 999 and below. For 1000 and up, we felt like 2 digits of accuracy struck the right balance between precision and low cognitive burden. We also decided to provide numbers in units of thousands, millions, billions, and so forth. Thus 2,239 would become 2.2k, 723,826 would be 720k, and 12,296,148 would just be 12m.

Not only does this system of two-digit precision make the meaning of numbers more accessible, but it also allows us to fit big numbers into compact spaces.

For it

Agreed. I did a bit of usability consultancy for the recent JustGiving redesign (http://www.justgiving.com). One thing that I thought was quite important was the precision of the numbers at the top: “Over 8 million of you have helped raise £550m for 8,382 charities.” Classic social proof, but the original wireframes had these numbers in full. We put forward the idea of rounding them to aid quick comprehension & recall, and to save some layout space. I think the results work well, although of course I’d need to usability test it to know for sure. — Cennydd Bowles

Against it

I don’t agree. When I look at that list of numbers, I actually have to read them to see which number is greater. Looking at 324,768 I will not read the number, just compare it’s physical size to 23,745 and 100. In your example, 24K looks smaller than 9.3K. You should also decimal-align the numbers to facilitate scanning and comparison. “9.3K” is user-hostile tech-speak, shame on you. — Jon Gunnar Wold

TwigKit

The London Search Meetup is Back

A brief history

Calling all search enthusiasts

My slides from the meetup

A Handbook for Designing Search

Our new book

Where to find it

Designing Mobile Search

Design Principles for Mobile Search

Content Trumps Controls

Answers over Results

Contextual Sensitivity

Cross-Channel Continuity

Related Reading

Mobile Information Needs

Two dimensions: scope and type

Scope

Type

A matrix of information needs

Informational

Geographic

Personal Information Management

Transactional

A penny for your thoughts?

Book Review: Search Analytics by Lou Rosenfeld

What’s it all about?

Resources

The Social Side of Search

Pull

Push

In Conclusion

Redesigning Wikipedia's Search

The Status Quo

1. Search gets second-class treatment on Wikipedia

2. Unnecessary clutter pushes results too low on the page

3. Lack of helpful filtering options

4. Uninformative search results

The Redesign

1. Results with strong information scent

2. Emphasize the best match

3. Meaningful faceted navigation

The Last Word

Revisiting Faceted Navigation

The Challenge

Design Philosophy

A Potential Solution

What’s Your Take?

A Call for High Quality, Open Source Demo Data

Requirements

Nice to haves

What do you think?

Why Developers Should Become UX Designers

What UX is not

What UX is

Developing a great user experience

In conclusion

Why Designers Should Care

O designer, where art thou?

The data explosion

The convergence

In conclusion

Comments?

Search as a Flow Experience

Help users form their query

Help users review the results

Help users reformulate their query

Help users recover gracefully

The Slides

A review of Foodily’s recipe search

With and Without

Breadcrumbs and suggested searches

Filtering on ingredients and sources

Foodily’s response

UI Components for Search

Search Solutions 2010 (BCS-IRSG)

Behshad Behzadi on Web Search Freshness

Challenges to freshness

Solutions to freshness

Vishwa Vinay on Click evidence — Signals and Tasks