Gus: OK Vlad, just imagine a sales funnel. People don’t just come to your website and within 30 seconds they have placed an order. There is a process of the visitor becoming engaged with the brand.

Vlad: I still don’t understand what ‘engaged’ means.

Gus: See it as a necessary step before an order is placed; that the visitor is interested in your brand, your products or service, and is ready to place an order. That could happen on their first visit but it might be their fifth visit.

Vlad: I have read Eric T Petersen’s writings about ‘scoring’ visitors on the basis of what content they viewed, whether they were returning, if it was a direct visit, length of session and whether they completed other non-transactional goals. But it’s not easy to get your head around.

Gus: I agree, it’s not straightforward. Plus, each business will weight these factors differently and therefore come up with a different ‘engagement score’. My issue is that there are ways of engaging a visitor with your brand off-site.

Vlad: What do you mean ‘off-site’?

Gus: I think social media sites such as Facebook and Twitter are playing increasingly important roles in delivering an ‘engaged’ visitor to your website.

Vlad: You mean, Facebook or Twitter as a referring site?

Gus: Yes, but more than that. I mean that a website visitor has spent time interacting with your brand on your Facebook page, and becoming comfortable with what you offer, learnt more about the business and the people – and then they come to website far readier and more prepared to transact. Ultimately, back to sales funnel approach, if you have more ‘engaged’ visitors coming onto your site you’ll have higher conversion rates.

Vlad: Are you suggesting that our clients should be investing in Facebook; building and maintaining pages and content?

Gus: I am, yes. There are other reasons why businesses should be seriously looking at Facebook as an opportunity

Firstly, the number of visits that Facebook receives is now higher than Google. You see the how Google’s traffic in blue was surpassed by Facebook’s in 2008.

With over 400m Facebook accounts, every demographic is on there. Even you have got a Facebook account, Vlad. This means that our client’s target audience is there already.

Vlad: Are you saying that as well as maintaining an e-commerce website, our clients have got to be building other sites as well?

Gus: If that’s where there audience is, then yes, of course.

Vlad: OK, but how do you think most businesses are currently thinking about Facebook?

Gus: I suppose it divides into three groups. Those who haven’t realised the importance of Facebook and so have done nothing. Those that have got a presence, but not really understood how to use it to create engaged visitors, and I think a small group currently of companies who are making a good fist of their Facebook presence.

Dave Chaffey lists some examples of business who have built engaging Facebook pages including STA Travel, Dell’s Social Media for Small Businesses, Howies, Marmite, The National Trust and Wiggly Wigglers.

Vlad: I have heard of Wiggly Wigglers, they do these fantastic off-the-wall podcasts about wormeries and composting.

Gus: It’s also interesting to hear from Justin Kistner that big brands like Unilever and Coca Cola are reallocating all the budget for developing microsites, and instead investing these funds in developing their Facebook presence, including Facebook apps.

Vlad: Presumably, because they already have a lot of traffic going to their current Facebook pages, whereas they will need to build the traffic to these microsites from scratch?

Gus: Exactly.

The most commonly cited reasons for businesses to invest in Facebook are firstly, to get found by people looking for your product and service ..

Vlad: But that’s what Google does?

Gus: I know, but since there’s now more traffic going to Facebook than Google, you also need to have a presence on Facebook.

Secondly, Facebook is good way of connecting, and back to our theme, of ‘engaging’ your customers and prospects. Three, it’s a good environment in which you build a sense of community around your business and merchandise, and also it’s a great way to promote the content you’ve got.

Vlad: Plus, there will also be some SEO benefits as well.

Gus: Right, you have a lot of control over both the content and Facebook that can help with your SEO ranking. Creating links to and from your website will help, having specific landing pages or tabs” will help direct traffic to the right place as well as posting keyword rich content into particular parts of your Facebook profile. There’s a lot of help out there to improve your SEO from Facebook.

Vlad: All of that’s good, but whenever Facebook is mentioned on the news it seems to be always some issue with privacy.

Gus: Yeah, I know. I think as a business they are walking a fine line between ‘helping people to share their content’ and giving control to users over their visibility on Facebook. But I am sure that they aware that if they screw up on privacy, the consequences would be monumental.

Vlad: Isn’t that an argument to sit it out and see what happens?

Gus: It could be. It depends on how one’s competitors might react to the Facebook opportunity.

Given the momentum behind social media, I think the ‘do nothing’ approach is the riskiest one.

Vlad: OK, but before we dive into Facebook with our clients, I need to understand what makes an engaging Facebook presence, what type of page they should have, etc. etc.

Gus: Agreed. The first thing to realise is that as a business you can opt to set up a Group or a Page on Facebook.

Vlad: What’s the difference?

Gus: In a nutshell, brands should build a Page rather than a Group. Pages are indexed by search engines and you can add apps from the list of 55,000 available Facebook applications. With groups you don’t get these benefits, and that’s why Pages work better. Here’s a good link to explain the difference between Pages and Groups in more detail.

Vlad: But, presumably you can’t just stick up a Facebook and not maintain it?

Gus: Right, not maintaining it and keeping it up-to-date could look worse than not having a Facebook presence at all.

Vlad: So our clients are going to have commit resources to maintaining their pages. And if they are spending time on this then the client will want to measure their impact of their efforts.

Gus: Exactly, and you know a few things about measuring social media, but first of all I want to look at what makes a great Facebook page.

Vlad: OK, let’s talk about measurement later.

Gus: Here’s a list of things to get right when setting up your Facebook page

1. Get a vanity URL that is easy to remember and includes your brand name, e.g. http://www.facebook.com/Marmite.
2. Spend time on making sure you have a compelling landing page – see Oxfam’s
3. Just like you would with a website home page, make it obvious what you offer and a what visitor can do on your Facebook page
4. Don’t let visitors arrive on your Wall – you have less control of what your fans are saying, so the first time that a visitor arrives you want to ensure that they understand the benefit your business offers
5. Use the Custom Tabs, as Oxfam have done, to segment your visitors into those with particular interests
6. Make sure there is up-to-date, relevant content on your Pages
7. Cross-promote your other channels – retail outlets, catalogue , online shop, Twitter feed, etc
8. Investigate what Facebook apps are likely to be appealing to your target audience
9. Finally, remember to comments on your fans’ comments and participate in the discussion

Vlad: You mentioned that there are lots of apps that you can add to your Page. What types of apps are popular.

Gus: There are over 50,000 apps so there is going to be something for everyone. The obvious ones to include are slideshows of your pics, product feeds, surveys and polls, discount coupons, the Facebook online shop, events as well as wishlists and games. A good place to start is Involver who develop both free and paid-for apps.

Added to this you have got FMBL, which is an evolved subset of HTML. This gives you even more control over your environment.

Vlad: And what about the wisdom of advertising on Facebook.

Gus: I agree with Justin Kistner of Webtrends, although the Facebook’s advertising platform is nowhere near as mature as Google Adwords, it is worth building up expertise in using Facebook’s platform.

If the Facebook’s traffic continues to build then more and more businesses will start to consider migrating some of the paid search budget to Facebook. Ideally, you want to be ahead of that curve, so you are not coming to it totally green.

Vlad: From my reading, it seems Facebook is investing in giving its Page owners more usable data.

Have you seen the analytics interface ?

You get to see demographic data of your fans, as well as the interactions taking place on your page.

Interestingly, Facebook are offering an API where you can get more data from this source than you can from these report.

Gus: I thought you had some reservations about the tagging up your Facebook page with your tags from your analytics provider?

Vlad: I do. Based on reports, I understand that Facebook are prone to change some technical architecture, often with warning. This means that your tags may not collect the data in the way you first envisaged.

If you are using your tags inside your Facebook Pages then you need to monitor their accuracy far more regularly than you would on your website. It’s down to the fact that you’re not in control of the platform.

Gus: What other options do you have to measure your efforts on Facebook.

Vlad: Apart from exploring what data you can get out of the Facebook API, you should set up a filter to isolate your Facebook traffic in your analytics tool. By default, we set up a social media filter for our clients on the Google Analytics account, and then reassign the medium of traffic as ‘social ’ rather than ‘referral’.

Here’s the list of referrers we capture and reassign as medium = ‘social’

blog|forum|board|community|group|answers\.yahoo\.|twitter|facebook|linkedin|flickr|digg|livejournal|myspace|tumblr|del\.?icio.?us|faves|reddit|squidoo|stumbleupon|technorati|netvibes|newsgator|wikipedia|wordpress|typepad|PRweb|ow\.ly|gumtree|feedburner

This means you can look at how social media traffic behaves differently to other sources of traffic. But unless you have tagged up your Facebook pages with your own tags, and accepted the extra monitoring you will need to do, you should be focussed on analysing the results from two reporting systems – Facebook and Google Analytics.

Another thing you can do is to use Hubspot’s Facebook Grader. The tool works by analyzing the number of friends you have, how important those friends are, the completeness of your profile is, how many wall posts you have and how many groups you belong to.

But measuring Facebook activity is, in principle, no different from measuring website activity. It requires our clients to focus on what business outcomes they want to achieve and always be tying the data back to these objectives. We recently produced a book review on Social Media Metrics and Jim Sterne is adamant that just because it’s ‘social media’ doesn’t mean the measures we use shouldn’t be ‘hard’.

Gus: I think for me, the interesting thing about developing a Facebook presence is what this might mean for the future. If the momentum of the ‘social graph’ continues, then the idea that we should be trying to drive visitors to our little ol’ standalone e-commerce website may seem very old-fashioned. If people are living their lives on social media platforms, then that is where businesses should be.

We’ve already heard reports that conversion rates are higher on Facebook where the conversion occurs on Facebook. With the growth of apps, it’s hard to conceive a scenario where a Facebook visitors would need to leave that environment to complete a process or transaction.

And we haven’t even begun to talk about the impact of mobile devices..

Vlad: That’s for another day.

We were lucky to have some great presentations from Phil Pearce on blackhat analytics as well as an update from Vicky Brock, Board Member of the Web Analytics Association, about the newly launched certification process.

Vicky Brocky and Lawrence Dyson

Phil Pearce’s insightful presentation on blackhat web analytics is available here, as well as other photos from the meeting.

Our next meeting is scheduled for Wednesday (no surprises) August 18th, at a venue to be confirmed.  If you are interested in coming along, you can sign up here. You can also take part in discussions and get updates from the WAW Manchester LinkedIn group as well as use the #wawmanchester hashtag on Twitter.

We are always looking for speakers and co-sponsors for future events, so if you are interested contact Dan on 0800 990 3580 or email dan@appliedwebanalytics.com

Introduction

The importance of having Jim turn his attention to how to measure and optimize social media should not be overlooked. Here’s someone who has charted, chronicled and challenged many of the online developments we now take for granted. Whilst social media has been around a lot longer than we think, the “Social Media” that has got both marketers and CEOs both excited and anxious is still in its infancy. It’s the same social media that has now become headline-making and time-consuming, so that we are forced to say ‘hold on, are we getting anything out of all of this?’. This is where Jim steps in.

What I like immediately about this book (and I like books as my way of digesting information) is that Jim is very clear about what this book is and is not about. Thankfully, it’s not another diatribe about how game-changing social media is and you need to get your hands dirty with this range of tools. This book is about how to measure your investment – and although social media may be free, your time is not – in social media.

Who’s this book for?

In my view, these groups of people will find this book especially useful:

• Organisations and individuals who have made some investments in social media and want to quantify the results they are getting
• Those who remain sceptical about the chaotic nature of social media, but are afraid that they are about to get left behind – and therefore want to bring some structure to their future endeavours.
• Practitioners and consultants who want to help their clients develop a system of measurement that will stand up to scrutiny, and can be discussed sensibly at board level, without hysteria or hyperbole

Sructure

I get a great deal of comfort from a book of this type that starts off each chapter with a verb. The author has understood that the reader wants to do something, learn a few things, take away a bit more and ultimately take some action. Jim’s favourite verb is “getting” – getting focused, getting attention, getting respect, getting emotional – see what I mean.

Chapter 1 starts as the book goes on – setting your objectives. No investment, social media or otherwise, can be justified unless it meets one of the three big goals, namely:-

• Increase revenue
• Lower costs
• Improve customer satisfaction

The rest of the chapters, read chronologically, form a pattern of a traditional sales funnel. Chapter 2 is concerned with Getting Attention and reaching your audience, most importantly, not just any audience – but your audience. Chapter 3 – looks at how to spread your message through your networks in Getting Respect. Getting Emotional, the next chapter, recognizes that your message is an empty vessel unless we ascribe a sentiment to it. Understanding and classifying sentiment – positive, negative, neutral, sarcastic, humourous, etc. whilst not straightforward is vital in measuring the impact your social media activities are having.

Chapter 5 and Getting Response looks at the question of how you can track back the interactions you having the social media space to some desired outcomes. Getting the Message, in the next chapter, takes a small diversion from the ‘sale funnel’ approach and concentrates on the skills and techniques social media practitioners require for being both a good communicator, and most importantly, a good listener. Chapter 7, Getting Results, ties back your efforts to the overall objectives you set in Chapter 1. Chapter 8 outlines the way in different ways to ‘sell’ social media to the various tribes of doubters and naysayers.

The final chapter looks to the future and what the metathemes of social media might signify for the relationship between buyers and sellers in a networked world.

It ain’t new

Despite the furore and headlines around Twitter, it’s worth remembering that many forms of social media, in relative forms, have long histories. Message boards, review and opinion sites, blogging, bookmarking sites as well as media sharing platforms are well-known and familiar. The phenomenal growth in Twitter usage and Facebook’s rivalling of Google has naturally brought attention to how these newer community tools can be used. Some of this attention is directly from businesses and marketers working out how this form of ‘free marketing’ can be exploited to their benefit.

But a networked view of markets and buyers is not new either. The Cluetrain Manifesto of 1999, with its 95 theses, already understood that ‘markets are conversations’, and that most companies were failing to be involved in those conversations out of ‘obsolete notions of command and control’. From an offline perspective, David Ford and the Industrial Marketing and Purchasing Group, 9 years before Cluetrain, wrote persuasively about a chaotic, non-hierarchical view of industrial relationships. The Internet, and its newer forms, has simply given (?) these type of ‘relationship’ tools to the marketers.

Equivalence

One of the interesting themes that Sterne explores, is that of equivalence in the chapter, Getting Respect – Identifying Influence. Whilst the goal of many retention campaigns is to create ‘raving fans’, Sterne looks at one company’s approach to measuring fans. Baekdal of Baekdal.com defined fans as “one who either follows everything you do, or actively points other people to you – or both”.  Based on his web stats, Baekdal was able to calculate that one fan was ‘equivalent’ to 445 regular online visitors. With an average fan having an average of 445 fans themselves, 15 fans commenting on a Facebook page broadcast this message to 6,675 others.

For those who have struggled to monetize ‘engagement’ activity, here was a clear-cut example. If you can put a value on a website visitor, and you can identify the equivalent number of visitors a fan is worth, you can monetize your social media marketing efforts. Added to that, if you can examine the success of fans in recruiting other fans, the ROI of social media just got a whole lot better.

Like every good book on web analytics there is the caveat about data purity, and Sterne makes the necessary right noises about the need for careful expectation setting. Data quality issues in the online space are well-known, and Sterne is right to make the case for correct presentation of the exactitude of the data, rather than sweeping the issue deftly aside.

Visualising and measuring networks

As well as putting a value on a fan, the book talks meaningfully about measuring ‘interconnectedness’ using a variety of tools. Companies like FMS Advanced Systems Group, Axiom, Rapleaf and Unbound Technologies have built tools that allow you to see, in visual form, how the form of these networks change over time and interact with one another.  Whilst some tools focus on following ‘conversations’, others look at the linkages, via blogrolls for example, between actors.  The goal of all this spidery network visualisation is to identify clusters, trending topics and important actors within your network. This could all be useful intelligence in targeting the right message to a significant group within your network.

Based on Pareto’s law, many of those in your network will be connected, but as Sterne asks, are they ‘respected’? Authority and impact are two measures for assessing the value of individual actors, whilst tools like Peterson’s Twitalyzer focus on generosity, signal velocity, clout and influence. However, like many activities in web analytics, you need to rely on more than one tool. Technorati may be great if many of your prospects and customers are blogging away; Twitalyzer is a powerful tool (and let’s not forget free) for those active on Twitter, but the real value, as well as hard work, will come when you come to aggregate these views into a single system. Until the technology catches up, it would appear you still need to do some pulling together of spreadsheets and Powerpoints for the exec team.

Sentiment

One of the reasons automation has struggled with social media metrics is that of recognizing sentiment. Whilst it is straightforward to capture mentions and retweets of items containing your brand name, what exactly were they saying about you? Think about emotions like sarcasm and irony, and see if you can work out ways for this type of emotions to be correctly classified. Not easy.

Although Sterne outlines frameworks for classifying sentiment – polarity and intensity – the text analysis tools necessary to automate and accurately classify millions of posts, tweets and reviews appears wanting, as Sterne opines

“Perhaps machines can eventually be taught. But who will teach them? Machines have a hard time with this because humans do, too”

Whilst we recognize that listening is just as important as participating in these ‘conversations’ their sheer quantity force us to look towards technology for the answer, when the real difficulty lies in codifying our own language. Layering on top of this semantic challenge is listening to those speaking in other languages, from different cultures and orthogonal perspectives.

One answer might be to look at what people do, rather than what they say – isn’t this after all the difference between market research and testing?

Measuring response

There are many ways to record actions of both prospects and customers, and Sterne has arranged this into a pyramidal “Engagement Food Chain”. The chain (why not a pyramid?) starts with Saw, advancing to Saved and Rated, and ends with Purchased and Recommended. Leaving aside some of the data challenges of identifying individuals, this model works well, particularly if you have an entirely online business model. This sales funnel view of the world makes the assumptions that the more people you have say, Commenting, ceteris paribus, the more you will have at the Purchasing stage. It’s classic stuff, and I am inclined to believe the underlying assumption.

Customer service and crowdsourcing

If you are still having difficulty convincing the Exec team after showing them how many visitors each fan is worth and how your volume of recommendations is positively correlated to your recent boost sales, try the Customer Service pitch. Many organisations have seen quick wins from listening to complaints from customers, and going on to help them out. For those, worried about getting involved in the ‘naked conversation’, using social media to resolve customer issues feels manageable, and can bring some personality. Virgin America, for example, has met those tweeting in-flight about poor service at the landing gate with “customer recovery coupons”. My sense is that, over time, customers will come to expect this type of interaction and woe betide the large corporations who have the wherewithal to do this, but sadly don’t.

One of the most interesting developments of social media is that of crowdsourcing – using the power of the many to develop innovative ideas as to how products and services can be improved. Asking your customer what could be done better, is not terribly new, but the volume of responses that can be garnered and the feedback loop as to what changes have been implemented make this a powerful tool.  So when Dell.com gets 2,000 appeals for Linux to be pre-loaded on their laptops and PCs, it can respond, and did within three months of the first request. Measuring the number of ideas submitted, votes on each idea as well as further comments is a great way of understanding the power of your customer community. But of course, this is a big-brand exemplar, and you have to ask yourself: what would you think of Dell if it wasn’t doing something like this?

Return on investment

Overall, though, can we tie back our efforts to meeting business objectives? Jeremiah Owyang thinks so. He cites three critical measures to monitor in order to justify investment in social media:

1. Improvement in marketing efficiency (through greater amplification of your message);
2. Reduction in support costs (avoiding the costly call centre or a visit to the store);
3. Improvement to sales (subject to a robust tracking capability, of course).

With determination and thought, I am certain that most organisations will be able to use these three measures above to compare the investment they are making in social media with business outcomes

The future

Sterne at the beginning of the book promises not to be an evangelist for social media, and for the most part he keeps his word. But at the end of the book, the inevitable crystal ball-gazing chapter, John McKean of the Centre for Information Based Competition points out our central false assumptions. Our current thinking is based on the belief that innovation is occurring within the organisation.

In fact, McKean argues, “..the vast proportion of innovation is happening on the “buy side” of consumer interactions. The “sell” side innovations are progressively less and less.”. Organisations have failed to address their audiences in anything measuring like a success, when fail-to-respond rates are between 95%-98%. Instead of organisations continuing to try and ‘guess’ what the consumer wants, the efficient response is that consumers communicate their needs through a “consumer initiated dance (C2B) of Search, Find (engage) and Negotiate.”  Like many codas to technology marketing books, only time will tell, but suffice to say when visits to Google are now being rivalled by those to Facebook, searchability is being replaced by socialability.

Evaluation

Jim Sterne has done an excellent job in pulling together many important strands from the body of, often self-inflated, research and practice of social media. He has stripped out much of the puffery in the debate, and concentrated on the fundamentals. Sterne has constructed a stirring narrative, whilst tying technology, techniques and tools back to their impact on the business.  The shape of the book, like an old-fashioned sales funnel, gives us confidence that what has been included is useful and important to business outcomes. He has charted some difficult territory – data quality, sentiment analysis as well as old and the new of social media.

However, there are a few gaps which could have done with plugging. The book title includes the term ‘optimisation’, and for my money, there were few examples of this. I would like to have seen a few case studies where businesses had ‘measured’ their social media efforts, understood that they were sailing off course, and taken corrective action to ‘optimise’ and get better results. By concentrating only on the ‘measurement’ side of the equation, the book feels slightly lopsided.

Secondly, a good number of the examples were B2C and featured the global brands. I know there is some value in featuring case studies from household names with household products, but I fear this is somewhat short-sighted. Many of the people reading this book will be responsible for business-to-business marketing programs, and will want to be inspired by these stories, no matter whether the brand name is global or not. In fact, some small business owners will see social media as an opportunity to compete on a more level playing field with their more well-resourced competitors, and would be interested to hear more about businesses who have taken on the goliaths in their sector.

Finally, I would have liked to see a greater exploration of Owyang’s maturity model. The model presented lacked depth and does not bear comparison with Stephane Hamel’s Web Analytics Maturity Model. Allied with this shortcoming, would have been a more detailed consideration of how to resource social media marketing – what type of people make great community managers, where to find them, how to define their responsibilities, what organisational  guidelines should exist for using social media  – all of these would have been important questions to answer.

These gaps notwithstanding, Sterne has written an important book about how we will measure social media and evaluate the results of our efforts. It deserves to be on the bookshelf of every serious marketer.

Vlad: There it is again.

Gus: What?

Vlad: The number of visits has dropped to zero between 1am and 2am.

Gus: Well, that’s quite possible. Most people are visiting this site during the day and at weekends.

Vlad: That might be plausible, but how do you explain the fact they had 213 visits from midnight to 1am, and then 99 visits from 2am to 3am?

Plus, this is happening two or three times a month.

Gus: Well, we should alert the client and get him to give their hosting supplier a hard time.

Vlad: Yes, of course, but I think the actual performance of the website, including availability, speed and errors found should be something we should be reporting on to the client.

Gus: But we’re not responsible for these things, and secondly, our job is to help optimize the visitor’s website experience, not get involved in hosting and tuning the web server.

Vlad: Gus, you’re right in one way and so wrong in another.

If the visitor can’t reach the website, or is getting loads of Error 404s ..

Gus: 404s?

Vlad: Pages not found. What I’m saying is that the technical performance of the website is inextricably linked to the website experience.

Not only that Google are now paying attention to your page load speed, and have a got some tools to identify ways to increase the speed of your website.

Gus: What do you mean paying attention?

Vlad: One of the factors that will determine where your website will rank in Google is your page load speed.

Gus: What kinds of tools are available?

Vlad: Assuming you’re using Firefox, you need to install Firebug, and then download a script called Page Load Speed. Once that’s in place, you simply press F12 you are on any website and the application gives the website a score out of 100, and identifies issues you need to fix to get higher page load speed.

Look, the European Parliament’s website has a few issues.

example of Page Load Speed report

Gus: OK, apart from page load speed, how would we get an alert if one of our client’s website went down?

Vlad: I would use a low-cost service like ChartBeat or Montastic to monitor server availability – it’s going to be a lot easier than trying to spot times in the middle of the night when visitors have gone down to zero. Alternatively you could set up custom alert in Google Analytics but that doesn’t monitor number of visits by the hour, only by the day.

Custom Alerts in Gooble Analytics

Gus: What other KPIs would you monitor?

Vlad: As I said, the number of Error messages that the website is producing, such as 404s.

Gus: Can you report on that from Google Analytics?

Vlad: Not currently, but you can modify the Google Analytic Tracking Code to trap the number of these type of error pages.

Gus: And then what do you need to do?

Vlad: You use the Navigation Summary report in Google Analytics to see what was the previous page before the Error 404 page.

Gus: I suppose another way you could look at the site is to examine the conversion rate, assuming it is an e-commerce site, by browser type and connection speed.

If there were any large differences in conversion between different browsers and connections versus the average conversion rate, then you might say that the website isn’t serving those using certain browser and connection speeds very well.

Vlad: Good point.

Gus: I know this is supposed to be all about technical parts of the site, but one other KPIs I would want to look at under this heading, is the number of failed searches.

Vlad: Failed searches?

Gus: Those searches that result in no matches found.

Vlad: That doesn’t come out of the box with GA does it?

Gus: No, but ROI Revolution have suggested an easy way to capture the search term used in and number of times you get a failed search. But it does require some changes to your GATC code.

Vlad: So, what are we going to capture now for our client?

Gus: I think the list goes something like this:

• Number of times site went down and for how long
• Page Load Speed Score
• Number of 404 errors
• Conversion rate by browser type and connection speed
• Number of failed searches, and the failed search terms

Vlad: Great. I let you put all that data together for the next meeting with the client.

___________________________________________________________________________________________________________

Related posts:

How  to select website KPIs

Making web analytics a continuous process

Vlad: I don’t get a good feeling about this workshop. I am sure it’s not going to go well.

Gus: Oh Vlad, come on not another ‘I am going to jump into the canal’ speech. What are you worried about this time?

Vlad: These guys are quite new to online, and whenever I sit down with them to review their analytics data their eyes just glaze over.

Yet, at the same time, when I talk about the results we have achieved for other clients  – made a change here or there to their site or shifted around some of their online marketing budget – they are bouncing up and down with excitement.

I just don’t get it!

Gus: What kinds of words have you been using to describe the data? You haven’t gone all uber-technical on them, have you?

Vlad: Of course not, I just said that there Google Analytics currently has 103 dimensions and metrics, and that once you understand what each metric or dimension means, then putting together custom reporting is a doddle.

Gus: Whoa, whoa, whoa!.

Vlad: What?

Gus: Dimensions, metrics, custom reporting!! Remember, these guys are quite new to online. They may understand what they want to achieve on their website, but you need to put it in language that they can understand quickly.

Vlad: How difficult can it be to understand CPM or CTR, for heaven’s sake?

Gus: Look, I think we need to take a different approach. Let’s make sure they understand the terms we are going to be using, before you move onto what reports and measures they will need to monitor.

Vlad: How do you suggest we do that?

Gus: Well, you know I always like to make a game out of everything?

Vlad: Yyyyesss. I am not sure I know where this is going?

Gus: Well, the last place I worked at, we tried to make learning fun, especially around key definitions or terms that people needed to use.

I have put this together this little learning pack for dimensions and metrics in Google Analytics, and there are simply loads of games you can play with a new client.

Vlad: What’s this little pack got in it, then?

Gus: In Google Analytics there are 5 groups of dimensions and 5 groups of metrics. So I have put together 10 Title cards for each one of the 10 groups of dimensions or metrics.

Vlad: I know I am not really into card games, but I can’t quite see the point.

Gus: Patience, Vlad. The game hasn’t started yet.

So Game #1 is that you lay out all the 10 Title cards and you give out all the 103 cards to your workshop participants.

Vlad: Is that the cards with the definition of each metric/dimension or just the name of the metric/dimension?

Gus: Game #1 is not about using the definition cards just yet. You are simply asking individuals, or maybe groups, to put the categorise the metric/dimension into one of the 10 metric/dimension groups.

Vlad: So 103 cards, grouped into one of the 10 Metric/Dimension groups? Got it.

That could be quite difficult with real newbies to web analytics.

Gus: Sometimes it is. That’s why I have suggested different colours for each of the 10 groups of metrics/dimension cards.

Once the players put the right card under the right dimension/metric group, such as “Time on Page” under “Metrics – Site Usage” and it’s a light blue card, then soon people work out that all “Metrics – Site Usage” are on light blue cards, and suddenly the game picks up in pace.

If you want people to work a bit harder, then you don’t need to colour code the cards at all.

Vlad: I like it, you could also get teams to race each other to see which ones could get it all the cards under the right cards in the shortest time.

Gus: I thought you would like the competitive element of the game.

Vlad: What’s Game #2?

Gus: Game #2 is where players have to use the Definition cards and lay them on top of the Label cards.

So players will have a definition like “The number of new visits by people who have never been to the site before” and they have to match it to the Label card – “New Visits”.

Again you can use the same colour for each group of Dimension/Metrics card, so that they start knowing what colour of Definition cards they need to consider.

Vlad: And if you want to make it more difficult and randomize the cards, you suggest not using any coloured card at all, right?

Gus: Exactly.

One thing I forgot is to say how I explain the difference between “Metrics” and “Dimensions”.

Metrics, I tell players, are always numbers, such as number of Visitors, Bounces, etc. Dimensions are filters on a number, e.g. Visitors by Country. I explain that Dimensions are usually numbers, but not always. For example, Days Since Last Visit is actually a Dimension even though it is a number.

Vlad: Best to explain that up front.

Gus: Agreed.

Vlad: Where can I get hold of your little ‘metrics and dimensions’ learning pack?

Gus: Here’s the link

Vlad: I think, if we start with a few of your little games – the people at the workshop will be far clearer about the terms we are using.

Gus: If they are clear about what each web analytic term means, it’ll result in a far higher level of contribution in coming up with KPIs to measure their website.

Vlad: That was not a good meeting. We’re not going to get any more work from them, I am certain. This is the last thing we need. And they are going to tell everyone we are a complete waste of money.

Gus: Wait a minute before you jump into the canal, at least tell me why you think it went so badly. From where I sat, it went OK – not brilliant, but not terminal. How do you know this client?

Vlad: Dimitri put them onto us. I phoned Anna, the Marketing Director and we talked about what they were doing with analytics – not much at the time – and how they wanted to use analytics strategically throughout the business.

I suggested a workshop to agree their business goals and the most relevant KPI’s, and we had this workshop about three months ago. We also agreed that we would coach their in-house analyst to make sure that they were implementing a ‘world class approach to analytics’, as Anna calls it.

Gus: So why do you think the meeting went so badly?

Vlad: Because after seeing some immediate improvements in their conversion rate after the workshop, it has started to plateau, and we are getting the blame for it. And probably, Anna’s right, they are not ‘world class’ and we’ve let them down, and you know Anna’s well-connected and she’ll be tweeting about us right now – saying what a poor job we’ve done.

Gus: So, you didn’t say the workshop was actually a ‘disaster’. Assuming that went well, something has clearly happened between the workshop and now, right ? Who came to the workshop?

Vlad: Lots of people were down to come but there were quite a few no-shows. I don’t think Anna had spelt out why they should come along and what role they were expected to play. Plus, by the end of the workshop there were just so many website KPIs that they were going to measure and optimize.

Gus: OK, I can see not having some key people at the workshop not helping matters right now, but why did you let them apply KPIs to absolutely everything ?

Vlad: Anna was insistent. She said that they had just invested in a new analytics tool that could produce these wonderful reports at the push of the button. She didn’t want to tell her boss, who had agreed to spend over 30,000 Euros on this system, that her team was only looking at less than 10% of these fancy reports. I kept saying that she was going to overwhelm her colleagues but she wasn’t listening.

Gus: The web analyst guy, Alex, it seemed Anna was pretty tough on him as well.

Vlad: I think he’s been put in a really difficult situation. He sits in the IT department, even though he is supposed to be working for Anna and her marketing team. He’s been told that he’s responsible for raising the conversion rate and yet because he’s got an IT background and sits with all the other IT people, he gets dragged into other projects.

Gus: It seemed to be that there was something else going on. When Alex presented the results on the last split test they’ve did on the product page I thought Anna’s head was going to explode

Vlad: Just like in ‘Scanners’.

Gus: What, what are you talking .. anyway. No, I meant that Anna’s was clearly not expecting a test to produce anything less than a fantastic result. When Alex showed her the figures from testing the original version of the product page against one we had recommended there was just silence.

Didn’t you explain that part of the testing process is that you sometimes find out what doesn’t work, as well as what does?

Vlad: I thought I did, during the workshop. But the way she spat out the word ‘failure’ I don’t think she has really taken it in.

Gus: But I thought there were some positives – Alex showed that some KPI’s had improved.

Vlad: That’s right, but because we had run out of the time during the workshop, partly having so many KPI’s to agree, we never got round to setting achievable targets for each of the measures. So, even though we have got some improvement, no-one is going to get a pat on the back to say ‘you’ve beaten your target’ because none were set.

Gus: Just thinking about the web analyst Alex, he spent most of the meeting going through all the reports that he was generating. He’d clearly no time to spend on investigating or playing the detective.

Vlad: Well, we’re supposed to be helping him with that area. But when I went to see him 6 weeks ago to help him delve a bit deeper into the data, all he was doing was getting the reports set up and scheduled to be delivered to just about everyone in the building.

Gus: And did you notice the reports?

Vlad: No, what do you mean?

Gus: What did you think about how the reports looked ?

Vlad: They looked alright, apart from there being far too many.

Gus: Vlad, they looked awful, all numbers in densely packed tables – not a graph to be seen. No series of traffic lights – you know, red, green and amber.

Vlad: He was probably too busy to worry about what they looked like. But I suppose you’re right – no-one is going to take much interest if they look like my old stats exam papers.

Gus: And what happened to bringing in results from the visitor surveys?

Vlad: Mmmm. I spoke to Alex about this. A fair number of survey responses were pretty negative about the product imagery on the site, and he was so worried that Anna was going to have a go at the art director, Johann, that he said he had ‘forgotten’ to download the data from SurveyMonkey.

Gus: But that type of insight is exactly what they need to understand what to improve.

Vlad: I know, I know. There’s no way we are going to get out of this with our reputations intact. The next email I get from Anna will be her terminating our retainer.

Gus: Hold on Vlad. Look I’m sure we could have done a better job in the workshop of trying to get Anna to understand that analytics is a process that sometimes gives results which aren’t what we would like, but I am sure the situation can be salvaged.

Let’s arrange another meeting to go through the issues we’ve just discussed. We’ll explain that the results from her analytics project could be *even* better if she understood that analytics is more than just a technical process or solely about conversion rates. She needs to stop ‘blaming’ members of her team and start seeing this as a journey – with its ups and downs.

Don’t forget, with our help, Anna did get a big boost in conversion in the early days – her boss will still be thinking that she’s doing a great job – but we know that if we don’t get this process right now then six months down the line we will have a much harder job getting it back on track.

Vlad: Plus, we need to move Alex physically to sit with marketing team, and prioritize his reports with the marketing team, at least to give him some time to be explore what the data is telling him.

Gus: And don’t be so hard on yourself. Just because we’ve run a workshop with Anna and her team doesn’t mean that we can all just relax and assume it’s going to work. This is about making a process successful, not just analytics.

Vlad: You’re right. I just expected people to do what I recommended, and then follow the tips we gave them. I didn’t expect that we might have to remind them of what we’d told them.

Gus: I think if we forget to consider the people involved, then of course the process will suffer.

Vlad: You’re right. I’ll give Anna a call on Monday to set up a meeting.

____________________________________________________________________________________________________________________________

Other posts you might like:

The benefits of web analytics

How to select website KPI’s

Gus: Hey Vlad, we have been helping our clients work out what’s been going on on their website for ages now, but how many of them have actually got a clear set of website KPIs for their business?

Vlad: Most have, but some haven’t. But each business will have different objectives

Gus: Correct, but there are some common business models, like e-commerce, lead generation, advertising, and customer support, where the KPIs are likely to be similar.

Vlad: So for e-commerce sites, what would the KPIs you’d look at be?

Gus: The list would go something like this:-

• % of new to returning visitors
• Bounce rates on key pages
• Referring websites
• Search keywords and phrases
• Conversion rates
• Average order value
• Sales per visitor

Vlad: OK, OK, but I don’t believe that each business, whatever model they fall into, would want to use exactly the same set of reports.

Gus: No, you’re right, and that is why if we are going to help these guys, we need to show what they should be measuring, specific to their business

Vlad: I think I may have seen a few models that could help us, like those that Petersen, Jackson, Sterne and Kaushik have come up with

Gus: What can these models show us?

Vlad: Well, before looking at that we need to understand what is involved in the creation of a KPI.

I remember Jackson talking about a “core score” – you know, a measure of the whole organisation’s efforts. He talked about the positive effect on the UK Prison Service when it moved from measuring itself on the number of escapees to judging its effectiveness in terms of number of repeat offenders.

That switch meant that the organisation considered itself as part of a system, rather its role within the system.

Gus: That’s a good example, but most of our clients are in the private sector where being judged on their revenues and share price is the *only* number they care about.

Vlad: Yes, of course,  somewill see the success in those terms, but think about a goal like  “the creation of happy customer”. Creating lots of happy customers creates great financial results for the organisation, but also if you unpack the term ‘happy customers’ you’ll see there are lots of hard measures a client would need to monitor and manage – levels of repeat business, customer satisfaction, loyalty and so on.

And that’s my point, you might have a strategic goal, but in order to succeed at that objective you need to have a good number of operational or intermediate goals as well.

Gus: And what type of operational goals could those be ?

Vlad: Well, according to Jackson and the others, it’s all a variation on this theme

1. How well are we doing at reaching our target audience (or visitor acquisition)?
2. How well are we engaging our audience (or converting them to the opportunity)?
3. How well are we doing at activating them, or turning them into a customer?
4. And how well are we doing at retaining them or nurturing them?

But the important thing is how each business defines this ‘core score’ as well as these operational goals.

In the past I have used a KPI workshop to generate these goals, and out of these come measures for the website.

Gus: Who do you get to come to this workshop?

Vlad: I think there are three important sets of people to have come along

• People who run departments and manage budgets
• Those that are responsible for what appears on the site – whether that is product, or service description, or editorial
• And finally, the detectives, us analysts who can help them with evidence – you know analysis and reports and, and so on.

Gus: Lock in the room and let them thrash it out then ?

Vlad: Not at all. I have always played the facilitator – you know armed with a the marker pens and the flipcharts.

The first thing you do is explain the Reach, Engage, Activate, Nurture model (I prefer Jackson’s model over the others). I always draw four boxes on a flipchart

Second, you ask everyone to brainstorm the questions they need the answers to to allow them to do a better job.

Gus: For example?

Vlad:

• Where are the visitors coming from ?
• What do people search on the site ?
• Which products convert the highest ?
• Why do people buy our products?
• What is most profitable way to acquire new customers ?

Questions will be a mix of Reach, Engage, Activate and Nurture

Gus: Then, I am guessing, you read out each question from your workshop people and put them into the correct box on the flipchart

Vlad: Absolutely, and then you jointly agree a measure how that goal will be measured. Here, have a look at this sketch I have done.

Website KPIs

So for example, looking at most profitable way to acquire new customers,  you would generate a report on different forms of traffic segmented by ROI.

Gus: Aren’t you forgetting a couple of things

Vlad: Like what?.

Gus: Firstly, you need to what action will be taken as a result of this KPI. There’s no point getting people to produce reports if no action will be taken.

Vlad: Agreed, 100%. And your second poist.

Gus: You should have some form of benchmark.

Vlad: What type of benchmark?

Gus: Well, I know there’s a lot of benchmark data on the web, but at least you should benchmark your own performance. As well as benchmarking your performance against previous results you need to set a target for improving each metric over a period of time.

Vlad: Agreed. The last point is that you need to really embed this process, and ensure you’re doing this on a continuous basis

Gus: And what about how the reports look, isn’t that important?

Vlad: Too important for a brief conversation, let’s come back to that one another time

__________________________________________________________________________________________________________________________

Other posts you might find useful

What is web analytics?

The benefits of web analytics

Setting website goals – part 1

Setting website goals – part 2

Setting website goals – part 3

Any book that has a foreword by Steve Krug, author of my all-time favourite website usability book, Don’t Make Me Think, has got my attention.

Added to that if you just flip through the pages of Forms That Work, you just know Jarrett and Gaffney have emulated Krug’s folksy style and lots of before-and-after visuals – and that’s a good thing. Why ? Because let’s be honest, very few people get excited about online forms, so anything an author can do to make the topic accessible and even winsome deserves praise

Forms That Work

My summary

Let me start with my conclusion first – if you like waiting to the end of anything to know what someone thinks, I’m sorry I am not that kind of book reviewer – this book is better than you think it is.

Primarily because after reading several sections of the book, you end up saying to yourself ‘Of course, everyone knows that. Doh!’ and that leads you to think that perhaps Jarrett and Gaffney have pulled some kind of con trick. Have they really written a book about online forms that just states the obvious? Initially, I thought they had, and was all ready to give this book the thumbs down. The reality is this, we often complete forms online that are tortuous, long-winded and ill-conceived. So, if the book’s contents are just plain commonsense, a good many people (and the websites they manage) have still a lot to learn.

‘Forms That Work’ begins with naming the three layers of an online form;

• the relationship – that between the individual completing the form and the organisation
• the conversation – the questions asked, its instructions and structure
• the appearance – the way it looks, input areas, use of graphics and text

Book structure

These three layers form the structure for the book, as chapters proceed to illustrate the dos and donts in the creation of a successful online form. What I particularly liked was how Jarrett and Gaffney segment users of online forms into three types:

• Readers
• Rushers
• Refusers

The interesting point is how online forms have to cater for all type of users, and presumably for those who simply refuse to complete the form, there has to be other ways in which the organisation can garner the same information. In most situations this is likely to result in a telephone call or a visit to an office – this is an expense. So, given the known costs of someone taking a telephone call or spending time with a customer who feasibly could have completed the online form without further help, there is a business case for investing improving your online forms. Well-conceived online forms will save you money and give the user a good experience, and add to your standing as an organisation that’s ‘easy to do business with’. It’s a point that the book could have made strongly – the ROI of great online forms. A title for my next book, perhaps?

Relationship

A crucial element in the form’s ‘relationship’ between the user and the organisation is the need to reduce social costs. For many, filling in an online form can feel like hard work, and if the form ‘punishes’ you for getting it wrong, or makes you feel stupid or inadequate, then no surprises when users either abandon the form altogether or pick up the phone . Three pieces of advice really stand out:

• Ask for answers: don’t demand them
• Use error messages that respect the effort the user is making
• If the user does make a mistake, preserve as much of that user’s work as possible

Plain commonsense when you think about it, but remind yourself how many times you’ve become frustrated when trying to fill in a form, and for some reason all your work has been lost and you have to start again.

Conversation

Onto the Conversation layer of an online form, and once again my initial response to a section titled “The four steps in answering a question” is ‘oh, for goodness sakes, surely people know how to answer a question’. But actually some people in charge of websites do have a particular knack for making the answering of a question far more difficult than it needs to be. Ask yourself these questions:

• How many times have you seen online forms with acronyms, abbreviations or industry jargon that you didn’t understand?
• How often have you tried to work out the meaning of a sentence with one too many double negatives?
• How frustrated do you get when a form asks whether you want ‘Standard Delivery’ or ‘Express Delivery’ but with no information on the difference between the two options?

Sometimes, part of the difficulty as Jarrett and Gaffney explain is that instructions are wordy, and or use unfamiliar words when a common one will suffice. The book mentions a great tool called ‘Vocab Profile’ which tells you how many of your words are outside the 2,000 most commonly used. Steve Krug’s maxim of ‘Don’t Make Me Think is super appropriate when choosing words to include on your online form.

Word order is also crucial to understanding the question. ‘Forms That Work’ demonstrates the point with the use of “title”. Think about this.

• If you ask for “title” before name, it means “salutation” (Mr, Mrs, Ms, etc)
• If you ask for “title” after name, if often means “job title”
• If you ask for “title” without any names, it often means book or film title

Appearance

Lastly, the section on Appearance of forms is really useful, with lots of hints and tips on how to improve online forms based on eye-tracking studies. Particularly revealing was what users see around an input box.

ldeal label position for input fields

A few improvements

I have already told you what I think about this book – on first skim it appears to state the obvious, but the science and rational applied to careful dissection of the common mistake website owners make, show the value of a book like this.

There are some irritations, though. The redaction or blocking out of hardly sensitive information like postcodes and company names seemed pointless, and spoiled the flow of the text. Secondly, the emphasis on analysing online forms appearing on government websites somewhat pigeonholed the book. Online forms appear on all sorts of websites, albeit that government ones unfortunately are those that we struggle with the most.

Finally, the authors could have spent some time explaining that although much of what they say is borne out of their experience and although it may appear as commonsense, the number of poorly constructed forms is evidence enough that further work is required. As more processes, from renewing your car insurance to booking a meal can be completed online, the greater the importance of making online form work, and work well.

The costs of users refusing to use your online forms are very real.

____________________________________________________________________________________________________________________________

Related Posts:-

Book Review: Advanced Web Metric with Google Analytics (2^nd Ed.) by Brian Clifton