Comperio consultant, Marcus Johansson, co-authors ‘Working with Microsoft FAST Search Server 2010 for SharePoint’.

As most search techies will testify, the life of a consultant working on complex enterprise search projects with large customers, can be quite a demanding one. To commit to investing personal time, outside of all this project work is, to put pen to paper and document a series of learnings and tips in a structured and user-friendly format, is no mean feat. So on this note, we salute Comperio consultant Marcus Johansson and congratulate him on the release of his first book earlier this month - Working with Microsoft FAST Search Server 2010 for SharePoint.

Marcus co-authored the book with Mikael Svenson and Robert Piddocke, who are also highly regarded contributors to the search community. Here, in Marcus’ own words, is some background on the book:

The book is split into two parts: the first one explains what you’ll need to know to deploy and administrate a solution. The second one is targeting developers who want to build their own search solutions on top of FAST Search for SharePoint. Although we expect most readers to have a SharePoint background, people who’s worked with FAST technology in previous incarnations will hopefully feel at home as well.

Recognizing  FAST Search for SharePoint’s popularity, and seeing that the product is many people’s first exposure of Enterprise Search technology, the first two chapters introduce the reader to important concepts and terminology. We felt it was critical to get this backdrop right as there are vastly different data and user experience opportunities (and challenges) in Search technology, than in e.g. a database-driven solution.

The next few chapters  target the IT Pro audience and deal with such things as architecture, scale-out, deployment and security. Scaling, in particular, is covered in depth seeing how FAST Search for SharePoint is often used for top-tier solutions both in  query load and content volume. Additionally, day-to-day operations and the various methods of maintenance is covered in detail. We show you how to interact with and follow-up the system through the SharePoint GUI, but also how to work with the solutions through PowerShell and, for those adventurous people who wants to truly integrate the platform in their application environments, how you can integrate against FAST Search for SharePoint’s native APIs even for administrational and operational tasks.

The second part of the book is a good fit for application developers who want to create their own search solutions on top of FAST Search for SharePoint, whether that means extending a default FAST Search Center or completely build their own search-driven application truly taking advantage of the powerful platform. Doing so, developers will realize an Enterprise Search platform, such as FAST Search for SharePoint, relies not only on a competent and flexible index, but also on a framework for advanced content processing and the possibilities to query into the data using a rich query language.

Finally, the book is wrapped up with a tutorial-like section on how to resolve common problems and how to attack frequent development scenarios.

At Comperio, we see how more and more of our clients realize the potential in not only implementing a global search experience in their enterprise, but to deploy targeted search-driven applications across their organizations. One user group might need a mash-up of data aggregated from several internal content sources, and perhaps another group needs to trigger a certain function that he or she perceives as a single operation, but in reality hits 2-3 different source systems in the backend. A search-driven interface is the very sweet spot for building such applications, and  I’m hopeful this book will give you the tools of the trade to implement them in reality.

Happy reading!

In recognition of Marcus’ achievement, the team here in Comperio have given him temporary clearance to use lines like ‘As I say in my book, Working with Microsoft FAST Search Server 2010 for SharePoint, …’ in internal meetings and possibly even at customer presentations.

The book is now available for purchase on Amazon - or you can win one of three free copies being offered by Comperio, by being one of the first to tweet this post mentioning @comperiosearch.

This image search app provides a neat solution to the history/speed issue by simply displaying results in a push-pop stack. As you type in new search terms, the image thumbnails are prepended to the resultset. Your search history stays on the page, but is simply pushed further down as new thumbnails appear. Its is really fast and simple to browse your image search history.

Also, this search app operates entirely in 30 lines of HTML and JavaScript by using a JSON feed to the flickr API. Simple and elegant.

Click here for the demo

Or copy and paste the code below this line:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

 
<html>
  <head>
    <script src="http://code.jquery.com/jquery-latest.js"></script>
    <title>Comperio Super Simple Image Search</title>
  </head>
  <body>
    Search for <b>cats, dogs, cakes</b>, or anything else that takes your fancy :-)
    <br />
    <input id="searchterm" />
    <button id="search">search</button>
    <div id="results"></div>
    <script>
      $("#search").click(function(){
        $.getJSON("http://api.flickr.com/services/feeds/photos_public.gne?jsoncallback=?",
        {
          tags: $("#searchterm").val(),
          tagmode: "any",
          format: "json"
        },
        function(data) {
          $.each(data.items, function(i,item){
            $("<img/>").attr("src", item.media.m).prependTo("#results");
            if ( i == 10 ) return false;
          });
        });
      });
    </script>
  </body>
</html>

Dr. Olstad speaks on how Norway is a hotbed for search technology development and this search ecosystem has it’s source at NTNU in Trondheim, which have been a feeder university to search giants like Yahoo, Google and Microsoft.

With the tidal growth of data (particularly unstructured) in the past decade, it is no surprise that enterprise search has seen impressive increases in levels of demand. Hence, says Dr. Olstad, the competition for bright young knowledge management or search graduates has intensified. He also points out that several search start-ups have risen out of Norway, benefitting from the knowledge and experience in the local search ecosystem.

One example of these Norwegian search start-ups is Comperio, which has matured to being 50 employees strong with offices in Oslo, Stockholm, London and Boston. Comperio is a Systems Integrator (SI) focused on implementing FAST Search projects for enterprise customers. These search solutions, more recently developed on the FAST Search for SharePoint platform, offer an alternative to traditional integration between IT systems. ‘Today, a lot of IT budgets devoted to integrating old and new solutions. It costs time and money. We rather use search technology’, says Comperio founder and CEO Jørn Ellefsen.

Comperio has developed search solutions in Norway for such recognisable names as, Sintef, Posten, DSS, DNV and Innovation Norway. Outside of Norway, Comperio has delivered large projects for the likes of UBS and Shell. ‘The search-based technology we are developing is largely generic, and it allows us to reuse the solutions from previous projects with new customers. This means that it is both cheaper and better for everyone. This market is starting to wake up in earnest and will grow quickly when you discover how effective the technology is. I think the market for our services will increase tenfold over the next three years’, added Ellefsen.

Comperio are currently hiring for their Oslo and London offices. See the open positions @ http://www.comperiosearch.com/about-comperio/work-for-us/.

Original article from http://www.tu.no/it/article292738.ece.

Let’s take it from the beginning. Using Best Bets, i.e. hard-wiring a certain result item to a particular search term, is a great way for Search Administrators to make sure end-users find what they’re looking for. Interestingly enough, there’s sometimes a slight contempt towards Best Bets in the technical community. Some people seem to think that the search solution’s relevance model should be able to stand on its own, and make sure the best results are always returned at the top of the results anyway.

But as a Search Administrator, if you could increase the perceived end-user experience a lot by a just a little manual (or semi-automatic) labor, wouldn’t you do it? Sure you would. One way of doing that is simply to look into the top X search queries in your query logs, and spend a few minutes on each search query and hard-wire a certain result item against it.

Anyway – we’re on the Technology section, so let’s get dirty.

In FS4SP, Best Bets are managed through the FAST Search Keywords page under Site Collection Administration. Adding a new one is easy: you start by creating a keyword, and then attach a Best Bet to it. Simple enough.

Let’s say you added the keyword “Comperio”, with the definition “Microsoft Search Partner of the Year 2011”. You then attached a Best Bet to it, linking to Comperio’s web site. With the minimal configuration, you’d end up with something like this:

Cool! You got yourself a Best Bet, and this is how they’re shown on the majority of FS4SP deployments out there. However, the Best Bet is not the only thing showing. The first star is showing the keyword itself, a long with its definition.

Let’s say you want to save some pixel real-estate, and decides that showing just the link is enough. At the Edit Keyword page, there’s no switch saying if to show the definition or not, but you can remove the text altogether from the text area shown below. That ought to do it!

Unfortunately, the result is not much better. We still get two stars, only now the keyword definition text is removed. The keyword itself is floating around by its own above the Best Bet – and not a single pixel was saved from vertical misery.

There’s no other setting on the FAST Search Keywords page to fix this – but there’s still a nice work-around. Enter PowerShell!

In an FS4SP shell, get hold of a Search Setting Group object corresponding to your Site Collection. If you only have one site collection, you can do:

$ssg = Get-FASTSearchSearchSettingGroup

If you have more than one, and haven’t named the group manually, pass in the id of your site collection to the the cmdlet’s –Name parameter. Running the two lines below in a SharePoint Management shell will print the id:

$site = Get-SPSite -Identity http://<site collection root>
$site.ID

When you’ve got hold of the search setting group, you can use it to view all internal information about the keyword:

PS C:\FASTSearch\bin> $ssg.Keywords.GetKeyword("comperio")
Synonyms        : {}
BestBets        : {Comperio}
FeaturedContent : {}
Promotions      : {}
Group           : Microsoft.SharePoint.Search.Extended.Administration.WCFClient.SearchSettingGroupImplWCF
Definition      : <DIV>&nbsp;</DIV>
Term            : comperio
LastChanged     : 1/27/2012 12:36:57 AM
Id              : 2

Looking at the Definition property, you’ll notice that’s it not actually empty at all – even though you’ve clearly removed all text from the definition text area. It contains the HTML entity for a non-breaking space wrapped in a <div> tag! Turns out that if you use the GUI to clear the definition, this is literally as empty you can make it. Even on a brand new keyword, without even specifying a definition. Bug or feature? I don’t know. But now knowing what the problem is, you can easily fix it for yourself by running:

PS C:\FASTSearch\bin> $keyword = $ssg.Keywords.GetKeyword("comperio")
PS C:\FASTSearch\bin> $keyword.Definition = $null

1. Enlarge your search box! Why are you hiding it ? Screenshot: TMZ.com  My favorite search box: 2. Make your search button big and clickable. Screenshot: CNN.com (Edited. CNN uses the correct version) 4. Use colors common to your users! Screenshot: google.com

5 quick fixes for non-designers who want to enhance their search results page:

1. Enlarge your search box. Almost every second site I see, internal or external, “hides” their search box – maybe because designers think they’re ugly. If you want your people to search: make it easy. I like to add a little gradient on the top of the search box, and to make the corners round. That makes it look inviting to write in.
2. Make your search button big and clickable. Make sure the search-button is as tall as your search box, but make it come out of the page, with a clear color or an embossment. The goal is not to make it look cheesy… but to make it look clickable.
3. Use already designed material as an inspiration. Make sure that your search results, and the rest of the website for that matter, matches your company brand identity or design hand book. Use the correct colors and fonts – they are probably thought through by a designer already.
4. Use colors familiar to your users. - Blue is for links – Green is for URLs etc. telling visitors where the result came from – Red is for alerts and error messages If your brand identity doesn’t have hex-codes for these colors … ask your designer or design partner for then. Kindly.
5. Enlarge your body text size. Many designers tend to user small text in their sketches … maybe to get more room for nice pictures? Your users are probably interested in the result when they are searching, so why don’t we make the text readable. A nice size for a readable body text is 14 og 15px, with a line-height of 17/18 px. Not 12 px Arial…

And a general tip: Don’t try to reinvent the wheel! When working with search design, see how Google, Amazon and Bing format their URLs, links, file type-icons, etc. and make your version of it. Surely you can try another way, but then you should also user test it properly before you launch it.

What are your suggestions to improve search design?

A year ago the Comperio Design Team started with planning and researching for what today has resulted in a new brand identity. It includes a new logo, new branding of services and products, more colors in our color palette, the redesign of our offices in Oslo, Stockholm, London and Boston, and Comperio employees who proudly hand out their new business cards.

The research we did in advance of the rebranding was threefold:

1. A simple analysis of what didn’t work with the existing profile, how we used the logo and the logos we are being seen next to.
2. An internal survey to uncover Comperio employees’ thoughts about “who we are”.
3. Data collection from materials featuring those we consider to be the world’s leading brands, to see the characteristics of a brand as “not just a logo” – how the brands use colors, fonts and graphical elements.

The output of this research, was a brief outlining the need for a prouder, stronger, more contemporary logo, with heavy typography and black as the main color. It must be solid and without 3D effects or shadows, and in a sans serif font (such as Arial and Helvetica).

After several directions, sketches and internal meetings, this is our new logo:

A simple typographic logo, with details you do not see until you see it several times, where our new payoff, “Search Matters”, is given the attention it deserves.

This is our color palette:

We have had the orange and cyan color since the birth of Comperio, but the new profile has more focus on black and gray as well. And not least, we have added a new color: purple – that will get us a little less technical, and showcase the valuable creativity we have so much of, among other things, the design team’s resurgence.

Now we are looking forward to show of Comperio in magazines and newspapers, on websites and in social media and at conferences around the world for years to come … And we look forward to with even greater confidence to state that; Search Matters ®!

Some glimpses from our rebranded office in Oslo:

A little bonus material for my fellow design nerds:

This … is my new business card:

A 400 gram coated paper, with 5 four-color printing (purple Pantone color of the Twitter bird to avoid halftone in its beak). The back of the card has a 4-colored light gray (C3 M3 Y4 K0), with white logo, where the logo is raised with relief varnish.

Yes, it’s just a business card … and Jaguar is just a car

For the most part, Microsoft has done a great job integrating FAST into the SharePoint world, but there are certainly some areas where you notice there are actually two paradigms behind the scene: how things work in SharePoint, and how things (used to) work in FAST.

The primary key of the FS4SP index is one of those areas. In this post, I’ll try to make this a little more understandable.

First of all, the place where most people will notice that a primary key actually exists is in the crawl log on the FAST Content SSA.

Each indexed item is assigned an auto-incremented integer referred to as Item ID. All right, so let’s dig a little deeper.

If you’ve been reading up on FS4SP, you probably already know that there is an internal FAST-process called “qrserver”. It is responsible for receiving queries from the FAST Query SSA and internally forwarding the query to the actual index. You might also know that this process exposes a small web interface. For security reasons, it’s only available from within the server it runs on. More specifically: http://localhost:13280.

Searching for something here will return results in an internal XML format. The actual result items are listed under the <RESULTSET> tag a bit down in the XML. There are lots of things to talk about here, e.g. that the naming convention used internally in FAST is quite different to what is used when the results come back through the FAST Query SSA. A managed property for example, is referred to as a “field” internally. But let’s not fret over that now. Instead, let’s look at the first few properties of the first result:

After a few quick comparisons of the crawl log on the FAST Content SSA and the search results from the qrserver, it’s clear that the Item ID is stored in FS4SP’s index inside the property contentid. When comparing with the Item IDs listed on the FAST Content SSA, we also notice that SharePoint is prefixing the Item ID with “ssic://” when it’s stored in the index. In other words, the true primary key as used internally, is based on the pattern “ssic://[auto-incremented integer]”.

But as anyone who’s been using FAST pre-Microsoft can tell you, the contentid is actually not the primary key of the index. The real primary key is what’s stored inside the property internalid. The value of this property is the MD5 digest of the contentid, concatenated with the name of the content collection it is stored in. Let’s double-check, using our example. We had these two:

<FIELD NAME="internalid">8a832873c701c00135ce827d6c64c09c_sp</FIELD>
<FIELD NAME="contentid">ssic://33</FIELD>

Calculating the MD5 digest of the contentid correctly yields the internalid (sans the collection suffix):

md5(“ssic://33”) == “8a832873c701c00135ce827d6c64c09c”

Since the internalid is suffixed with the name of the collection, we can actually put several items with the same contentid into the index. The requirement being that they’re stored in separate collections, so that the concatenated internalid value becomes unique. In FS4SP however, we often use only the default “sp” collection. Luckily, SharePoint makes sure to assign the Item IDs so that they’re unique across all collections, hence creating unique internalid:s even though the items are in the same collection.

However, there are ways in FS4SP to index data without going through the FAST Content SSA, i.e. you can index data without letting SharePoint know about it. This happens when you’re using any of the FAST Search specific connectors or the command-line tool docpush. These tools talk directly with the index, bypassing SharePoint completely. Thus, the content id item won’t be assigned an Item ID using the “ssic://” pattern.

So, what happens instead? Let’s try it out. Using the docpush tool, we can send an arbitrary web page into the index:

Using the qrserver web interface, we inspect what was indexed:

In this case, the contentid property is the URL of the web page we specified. This makes sense as the URL is unique for the whole web, and as such it is also a good candidate for being a primary key in the index. A URL is just a special case of a URI, which is what many of the FS4SP command-line tools use when referring to the primary key of the index. Examples being the docpush tool (when deleting a item from the index with the –d switch) and the waadmin tool (used for retrieving link cardinality data for an indexed item).

To sum up with some key points:

• The primary key of the index is stored in a property called contentid, though in the SharePoint GUIs it is referred to as an Item ID and look slightly different. They relate to one another as: [contentid] = ssic://[item id]
• Items that are indexed using the connectors of the FAST Content SSA are assigned a contentid on the form “ssic://”…
• Items that are indexed with the FAST Search specific connectors or the docpush tool do not follow the same pattern, but are typically a proper URL or a value from a database.
• If a command-line tool calls for a “URI” to an indexed item, use whatever is stored in the item’s contentid property.

The video is 70mins long and roughly divided into three sections: 1) the challenges you face as an implementer (quite interesting), 2) the nuts and bolts of how you overcome this, and 3) Some real life examples. At the end of the presentation I showcase some really nice SharePoint search front-ends that Comperio made for Microsoft a few months ago.

How do you make great UX? What can be done to make your Fast Search For SharePoint application a joy to use? Can Scottish people make themselves understood to Americans? What tips would you give the up and coming UX designer?

The Microsoft SharePoint Conference is consistently the number one event on SharePoint. This time, even though it wasn’t even a release year, more than 7000 participants attended the conference, beating both the Microsoft Partner and the Build conference.

Microsoft SharePoint Conference 2011 venue center

We’re sincerely impressed by the professionalism and the amount of planning that must have gone into this event. Microsoft really did an amazing job; all the way from when the first invitations were sent off to when the last booth had been disassembled. As presenters on the conference, we also had the pleasure of working with Microsoft behind the scenes, and we can assure you that the success of SPC 2011 was in no way a coincidence.

Early Monday morning, Jared Spataro, Jeff Teper and Kurt DelBene kick-started the conference with an inspiring keynotes session. Besides giving the many thousands in the crowd an outline of the week, they also shared a glimpse of their opinions and the outlook of SharePoint technology going forward. If you didn’t get a chance of attending the conference, make sure to watch the keynote online.

As a company specializing in Enterprise Search and FAST Search for SharePoint, we’re of course extra thrilled that Microsoft – in the actual keynote -  gave us a live demonstration of a SharePoint and FAST Search installation containing more than 100 million documents. Of course, they didn’t just show it to us; an assistant was brought onto the scene and pulled the power cord to the server rack, thus triggering a full farm failure. In just a few seconds, the solution had recovered and was back online, fully operational. The crowd was sheering!

As for the actual sessions, SPC 2011 really had something for everyone. The amount of sessions, hands-on labs and activities going on was simply staggering. The Enterprise Search area alone, whether having a business-oriented or technical focus, was covered by more than 30 sessions! Including our own day-long pre-conference training session Learning to develop a high-end search solution with FAST Search for SharePoint 2010 and our break-out session on User-Centric Design for Deploying FAST Search Server 2010 for SharePoint.

Comperio hands-on lab on FAST Search for SharePoint at SPC 2011

But besides all interesting sessions, attending SPC 2011 was of course about more than just learning new material. We also had an absolute blast at Disneyland, rented just for this occasion, and at the many other venues set up specifically for us attending SPC 2011. Most of the time, the partying started right when we stepped out from the elevator, as the lobby in our hotel turned out to be one of the hot spots for grabbing a “SharePint” after the sessions.

We’ve met so many interesting and inspiring people from all over the world; whether that someone was a SharePoint developer from down-under in trouble with their FAST Search installation or people from Fortune 100 companies around the globe looking to finally to tackle their Knowledge Management challenges.

Everyone at the conference must have heard the key message from Microsoft as loud and clear as us at Comperio: Enterprise Search is the solution to a wide range of problems in your organization, and FAST Search technology will help you working towards solving those problems.

This is an exciting time to be in the search business!

Knowing one’s roots is important, and it feels good to point out where I stand as a practitioner of user experience design. After ploughing through years of bookmarks and downloaded PDFs – with subsequent strict self-censoring – I came up with this list of 12 extra important sources of inspiration. The selection is wide and eclectic – something pre-Internet, something from yesterday, something obvious, and perhaps even a surprise. I learn something by integrating this knowledge into my own design process, and I hope you find something you like as well. Enjoy!

Principles of Traditional Animation Applied to 3D Computer Animation [PDF] by John Lasseter (1987)

Good animation = good interaction design! John Lasseter, the grand old man of Pixar and Disney, explains the principles of communication through movement.

The Myth of Metaphor [PDF] by Alan Cooper (1995)

Use conventions, not metaphors! Good conventions are easily learnt and remembered forever – metaphors just limit our thinking to old habits and frames of mind.

The Paradox of Choice: Why More Is Less [PDF] by Barry Schwartz (2004)

Few, but good alternatives are best! Read the whole book, which explains why low expectations are the key to happiness.

Nudge: Improving Decisions About Health, Wealth and Happiness [PDF] by Thaler & Sunstein (2008)

As designers we have an obligation to help people choose wisely. Not because people are stupid, but because we all would rather not think for ourselves.

How To Kill Creativity [PDF] by Teresa M. Amabile (1998)

Purpose, mastery and control (and pleasure) are fragile qualities that flourish with care, and wither if neglected.

The Analysis-Synthesis Bridge Model by Dubberly, Evenson og Robinson (2008)

An analytical and straightforward presentation of the creative process, which also puts the need for design documentation into context.

Design Thinking by Tim Brown (2008)

User-centered design – said with other words. Design is not just about practical knowledge, but also important personality traits – such as curiosity, empathy and “formidlingsglede”.

Bringing Holistic Awareness to Your Design by Joseph Selbie (2009)

Cross-disciplinary collaboration permeates everything we do with user experience design. Nobody can find all the answers on their own!

5 Decision Styles. What is Yours? by Jared Spool (2009)

There’s not just one true good design process. We all take shortcuts – and it’s important that we know about the consequences of doing so.

What Goes Into a Well-Done Critique by Jared Spool (2008)

Giving and receiving critique is perhaps the most difficult – and also the most important – thing we can do to help each other become better user experience designers.

Guiding Principles for UX Designers by Whitney Hess (2010)

Design is a creative decision-making process, where principles are more important than your particular choice of user research methods.

The Fall and Rise of User Experience by Cennydd Bowles (2011)

What does it really mean to design great user experiences, and are really anything more than self-centered beta junkies with turtle-neck sweaters and Apple logos on our breakfast cereal?

Other sources of inspiration:

• So you wanna be a user experience designer — Step 1: Resources
• Essential Interaction Design Essays and Articles