I needed to create a post to check that my new RSS feed is working. Apparently (and thanks Charlie Southwell for pointing it out) they haven’t been working so well for a few months. Should be fixed now.

I’ve been interested by forums for a while. It often seems that the Social Media establishment overlooks them in favour of shinier platforms; unjustly, in my opinion. Forums require more time investment than other social platforms, grow more slowly, often cater to a slightly less attractive demographic, and (significantly) are more intrinsically hedged against commercial activity from marketers.

But that doesn’t stop them being interesting to marketers; if for no other reason than that they often perform better against common “problem searches” than brands’ own sites.

But I think there are many other reasons we should keep an eye on forums. Which is why I’ve been noodling around with a few recently. What follows are some notes on what I found on one such forum; MoneySavingExpert.com.

I used a perl script to gather the following semi-public data from a sample of the users:

• numeric uid
• total posts
• join date
• date of last activity

I sampled roughly every 500th member – giving me information on just over 3.6K members (or around 0.35% of MSE’s approximately 1m members.) The earliest record in the set dates back to March 2003 (the year that Martin Lewis founded the site).

Assuming that member numbers are assigned sequentially, I plotted member number against join date to get some sense of how fast the forum grew over time.

Take a look at the left hand side. There’s something strange going on there. If we zoom in, we can see that my assumptions about sequential member numbers might be called into question for the very earliest members.

The numbers since June 2005, on the other hand, grow much as I’d expect.

A plausible explanation might be that there was a major change of forum infrastructure around May 2005; and that old members were migrated to the new system; but that their numeric uid (which isn’t of any great importance to anyone) was somehow reassigned in the new system.

How long do users remain active members? By plotting a line between their join date and last active date, we can get a glimpse of that.

While it looks at first glance as though the members are fairly loyal, that’s probably just an artefact of the data density. Zooming in on a segment at random shows that loyalty isn’t so much the norm:

There’s got to be a better way of looking at this.

Plotting the chart so that we can see the number of posts a member has made gives us this:

It appears that there are a relatively few big contributors; this is more or less in line with experience. How important are they? Let’s do a little pareto-like analysis, plotting the percentage of members ranked by contribution against the percentage of posts:

There’s an old 1:9:90 rule of thumb when it comes to these things; misapplying it slightly to suit my own purpose, here’s how that works out for this data set.

The red bar shows the post contribution of the top 1% most active members. The green bar shows the contribution of the next 9%. And the blue bar shows that 90% of the members are hardly pulling their weight. That’s not really a surprise; many people join a forum (a prerequisite for posting) to ask a single question, or to express a single complaint. They aren’t really part of the community.

There’s some potentially interesting desk research into what kinds of content are being posted by the most massively frequent posters. Are they mostly responding to posts, or starting threads? Might it turn out, for example, that the balance between creators and contributors on forums isn’t what social media orthodoxy would have us think (few creators, more contributors & commenters), but rather a few key community leaders who are kind enough to respond to the many random posts left by the careless majority?

What will the impact of this be?

In anticipation of this change, Simply Measured (a web metrics company) recently published a pre and post study of brands that had been early adopters of the new Facebook Timeline format. In it they made the headline-grabbing claims that “new Facebook Pages drive higher engagement rates” and even put a number on it: brands, they promise,

get 46% more engagement per post with timeline.

This would make more sense if the majority of a brand’s audience engaged with their posts on the Page – but they don’t. The great majority of a Page Post’s impressions are served on the audience’s newsfeeds and news tickers (it’s a current source of irritation to me that we can’t distinguish between the two) and only a tiny fraction (and generally less than 1 percent) are served to Page visitors. So the great majority of fans won’t even have seen the new Timeline layout. How then could it have any impact on engagement?

I decided to look at the numbers in some more detail, particularly the two pages with the highest increase in engagement: Livestrong (161% increase in post engagement) and Toyota (156% increase in post engagement.) That’s a hefty increase.

Methodology

I queried the Facebook Insights API to get a list of Page Posts for both Livestrong and Toyota, and counts of the Comments, Likes and Shares for each of those posts. I’ve not done anything clever like adjust for fan counts or post frequency, and I can’t get any data on paid promotion (that sort of thing is private to the Page admins.

I used the free statistical analysis language R together with the graphing package, ggplot to analyse and plot the data.

Livestrong

Here’s what we see for Livestrong (each dot is a post.) The red line shows when Livestrong page admins switched to the new Timeline format. I’ve also had the graphing tool add a kind of trendline to describe the data better.

It’s hard to see that huge uplift in engagement, but it’s not particularly easy to read. These box plots grouping Page Post engagement by week may tell a clearer story.

Again, the red line shows when the new Timeline came in. Can you see evidence of the 161% increase in engagement? I can’t!

What I see is a great deal of variance; both within each week’s posts, and from week to week. Looks to me that Livestrong didn’t get a lot of post engagement for the first four weeks of 2012 for example, but that they had a strong Thanksgiving and Christmas period.

Toyota – a different story?

The Toyota data, on the other hand seems to show a huge spike in engagement following the shift to Timeline! Could I be mistaken?

One of the things to notice here is how the the height of the box plots has increased — indicating that there’s greater variance following the launch of Timeline. That suggests that there’s something going on — something that’s not quite as simple as “engagement per post is up”. Here’s another way at looking at that variation: I’ve plotted them in order from the highest engagement posts to the least:

That’s quite a curve. It suggests to me that the increase in the overall and mean engagement could have something to do with international sympathy, good causes or cute cats; together, these posts account for only 28% of the posts, but 49% of the engagement. There’s a bit of me that suspects that content might have more of a role to play here.

What can we conclude?

I think it’s hard to draw concrete conclusions. I’m not saying that SimplyMeasured are wrong; but I would like a closer look at their methodology and numbers.

I’d say, however, that in the absence of this we can argue that there’s sufficient variation week on week, and from post to post that it’s very hard to prove that Timeline has had any effect.

There are too many factors affecting engagement; including fan growth, paid activity, competitions, and – well, simply “good content.”

We may be able to replicate the strong user engagement enjoyed by games (and social games) by adding game elements to a non-gaming business.

This isn’t about companies creating advergames (of course that hasn’t prevented advergame developers from repackaging their services as sexy "gamification" products and muddying the waters.)

Instead, businesses are learning to engage with consumers through "game-like" behaviour.

Example: Foursquare turns "coffee-shop loyalty cards" into a game by publishing league tables based on your social graph.

Example: Facebook (inadvertently?) turns "acquiring fans" into a "strategic social media goal" for companies by publishing the number of fans on Facebook Pages.

Points, levels and collectables are becoming a tool for marketing and behaviour-change as real-world activity increasingly becomes game-like.

Here’s my top-line review of the situation as I understand it. Which isn’t much. It’s mostly told through links to other people’s presentations — and as such this is intended as a jumping-off point for people new to the concept.

10,000 hours of video gaming

Jane McGonigal (Director of Games Research & Development at the Institute for the Future in Palo Alto, California) estimates that the average 21-year old growing up in a gaming culture will have played 10,000 hours of games. This establishes both an expertise in game playing and a set of preconceptions about the world.

She proposes that gaming teaches 4 "superpowers"

• Urgent Optimism – gamers act immediately to solve a problem with a belief that there is a reasonable hope of success
• Social Fabric – gamers learn trust and cooperation
• Blissful Productivity – gamers are willing to work hard if given the right work
• Epic Meaning – gamers love to be attached to awe-inspiring missions

Putting the Fun in Functional

Amy Jo Kim’s company Shufflebrain develops social games and social content for businesses. She specialises in using game mechanics to improve the usability and engagement of software and social software. This presentation from 2008 was the first in which I came across the concept of gamification (although Kim doesn’t use the term), and it’s still relevant today – and the intervening years (remember when World of Warcraft only went to 60?) may even make it easier to understand.

Game Mechanics are the systems and features that make games fun, compelling and addictive:

• Collecting and Completism
• Points and Levels
• Feedback (system & social)
• Implicit and Explicit Exchanges (structured social interactions)
• Customisation

Design outside the box

Jesse Schell is a professor at Carnegie Mellon, and runs Schell Games, a company that creates everything from online games to theme park attractions.

In a speech at DICE 2010 he suggests that with the advent of gestural controls the world of videogames is "busting through to reality", and that conversely the real world (and the world of marketing and product design) is increasingly game-influenced.

He posits a real world RPG (role playing game) where brands and gonvernmental departments reward consumer behaviours and activities that further their interests with experience points that help the consumer level up in an all-embracing game.

It seems that at least part of this presentation was intended to be a dystopic vision of the future to encourage game designers to ask themselves important ethical questions; questions that have often been ignored by marketers.

Schell says he wanted to encourage people to think carefully about which kinds of games and experiences were appropriate to develop. But not everyone picked up on those subtleties. (The Curse of Cow Clicker)

Epic Win App

UK-based Supermono Studios notes that it’s "easier to complete a 40hr RPG than drag yourself to the gym or wash your dirty car." Their response is the Epic Win iPhone app; an app that is both a to do list and a streamlined RPG. As you complete real world goals, you gain experience points, uncover loot, and level up your character.

I think we can assume that this is intended as a clever satire, along the same lines as Geek Overcomes Social Anxiety By Turning Life into RPG

High Score House on the other hand seems to be wholly without irony. It allows parents to repurpose their childrens’ household chores into games. Thanks Dad. Worst. Game. Ever.

Criticism of the numbers

Raf Keustermans (ex-EA Games online games marketer) points out in this 2011 presentation that loyalty on Social Games is in fact very low (only 15% of players are active in month 2, fewer than 2% will spend money.) Of the 80K games on Facebook, only 200 have 1m monthly active users (MAU) or more. He claims that 0.07% of social games have both scale and lasting engagement.

Why – then – would adding the handicap of non-gaming business requirements be a sensible way to improve your chances of success?

Criticism of the approach

Sebastian Deterding is a user experience designer with a strong interest in gamification. He notes that too many entrants into this space mistake the superficialities of points and levels for motivation- and that the pleasure in games stems from mastering challenges.

Richard Bartle: Extrinsic vs Intrinsic Rewards

Richard Bartle is a Professor of Gaming at the University of Essex (and creator of the first virtual world — the original MUD.)

There’s a good deal of debate around intrinsic/extrinsic rewards; the hardliner argument (expressed in Bartle’s deck) is that emotional response ("Epic Win!") is the only intrinsic reward. The more liberal thinkers seem to accept that in-game achievements and other Maslow-type psychic rewards are OK too.

Shore Ditch

However there’s common agreement among all the practitioners that (despite product offerings from Big Door and Bunchball – who’ve done work gamifying porn for Playboy) points and levels aren’t what people play for (neatly illustrated by a few slides from Sebastien Deterding’s Don’t Play Games With Me! Promises and Pitfalls of Gameful Design).

If you want to experience this, and not just understand it, have a quick play with Progress Wars.

Bartle is also known as the creator of the Bartle Test, which can be used to typify MMO players according to four stereotypes. He argues that extrinsic rewards such as "points, levels, badges, leaderboards etc. only appeal to achievers!"

Bartle ends on a strong line:

Gamification will not be ubiquitous after five years. The more it happens, the less effective it becomes.

See also

Here’s a well-put together deck that offers a good summary of the points raised in this article – citing many of the same people.

If it turns out that you could probably just have read that and not bothered with this post, then you know where to leave the comments. Here’s a quick summary for those who don’t want to read the slides: today’s gamified apps are flawed qua games; they lack true playfulness.

The Wired article The Curse of Cow Clicker doesn’t properly concern itself with gamification (although it is mentioned in passing.) However it does look in some detail at the impact of companies like Zynga who have brought business targets like ARPU, user growth and loyalty to gaming. Which seems like a neat reversal…

It’s a matter of time — within the next five or so years — before more business will be done on Facebook than Amazon
(Sumeet Jain, Principal, CMEA Capital)

Now it looks like as though we’ll need to find a new trend for 2012: Bloomberg has posted a bearish story tracing the collapse of the Facebook store front. It comes as no surprise to us; that’s just not how people use Facebook.

Facebook marketing communications are all about optimising your content for the News Feed, and not running your Facebook Page like some kind of half-baked microsite. We routinely see fewer than 1% of our Daily Active Users (DAUs) hitting the Tabs (including the Wall Tabs) of the Pages we manage or promote. The chart above shows fewer than 0.3% of clients arriving on a brand Page (generally to ask a question or register a complaint.)

Here’s a chart for another client. Client 2 has a fair amount of promotional activity pointing to their Page, so there’s a noticeably higher proportion of on-Page visits.

Look at your own Facebook Insights data again; where are you talking to your fans?

So the Bloomberg article reflects an overdue readjustment of businesses’ Facebook strategies. But this market readjustment mustn’t be taken to mean that Facebook Commerce (F-Commerce) is dead. Sure, Asos’s F-Store gets a paltry 40 DAUs and 700 MAUs at the time of writing and that number is likely to fall still further.

But Asos (who are a savvy operator, and well worth watching) probably doesn’t care; they learned early from their mistake and made changes. They now use Facebook to drive traffic into its core web store. Yesterday’s Asos Facebook promotion, for example registers over 23,000 clicks. Here’s a snapshot of the 24 hour period since 07:00 hours yesterday:

…of which the lion’s share come from Facebook. See where they posted again to catch the afternoon traffic? They went on to post again at 21:00 hours to mop up those who hadn’t already joined the campaign. That last post accounted for an incremental 6K visitors.

Facebook Commerce still looks pretty healthy from Asos’s perspective. Take a look at these results or even these.

So – the Facebook Store Front is dead, long live Facebook Commerce?

Mobile

It’s worth noting that a large share of the traffic (again looking at yesterday’s promo) comes from a mobile device.

Mobile devices don’t have a great history with Facebook Apps; and it seems that Asos prefer to send traffic direct to site (they have a well-put together mcommerce site) rather than lose those users. This is something they appear to have learned last October; their first Savvy Sunday Sale pushed traffic to a Facebook Tab, their second sale (a week later) to a Mobile sniffer.

The increasing share of mobile traffic has been a good reason to avoid F-Stores.

Should I always send my traffic out of Facebook?

In a word, no.

It’s still notably cheaper to land traffic from Facebook Ad Units within Facebook; your average cost per clicks will be lower (as much as 45% cheaper?)

So there’s a clear argument to be made for testing landing pages within Facebook for the paid traffic. Links for earned audience, however should probably point outwards to the brand’s own site.

Anyway — thanks to SMG partners like Sysomos, PeopleBrowsr, and Twitter themselves, my hard drive is bursting with data and charts. Here’s a quick selection, presented with as little commentary as possible.

Twitter responds to TV

When it comes to planning Twitter media campaigns, it seems clear (for the present at least) that brands will have to find predictable trends to form the backbone of their activity. Nothing is more predictable than TV.

The Only Way Is Essex: Seasons 2 & 3 Twitter engagement

The Apprentice: Season 7 Twitter engagement

X Factor: Season 8 Twitter engagement

Optimal buying requires real time engagement

X Factor Tweets: 12 November 2011

Twitter responds to TVCs. Just about.

Champions League Final 2011: Twitter mentions of Heineken

Heineken’s sponsorship of the Champions League was particularly successful last year: research from STAP (the Dutch Institute for Alcohol Policy) reveals that the Heineken logo was on screen for almost a third of the match. Still, it seems that viewers had other things to tweet about.

England vs Argentina (RWC 2011): Twitter mentions of O2

A particularly fine TVC from O2 generated a bit of noise, but not all that much.

Twitter may be most useful as a data source

This chart, from the wonderfully creative Tony Hirst (@psychemedia) maps out some of the key accounts followed by people who tweeted about Who Wants to be a Millionaire.

It turns out that we can compare the “twitter fingerprints” of different programmes — and compare these, in turn, with the twitter fingerprints of our brands.

The chart above illustrates the emergence and resolution of a flame war[1] on Waitrose’s Facebook Page last November.

The horizontal axis represents sequential Posts on Waitrose’s Wall while the vertical axis represents the individual contributors to the "conversation" (really it was more of a barney than a conversation.) Each blue dot plotted on the chart represents at least one comment posted by a specific contributor on a specific post.)

So the more blue dots in a column mean the more unique users have commented on that post; the more blue dots in a row, the longer that unique user has continued engaging with the overall conversation (or to put it another way, the greater their appetite for the fight.)

The flame war in question more or less dominated Waitrose’s Facebook Page for more than a day and a half; accounting for 70% of all Posts and 72% of all Comments until it finally ran out of steam.

Much as I’d enjoy going into them, the ins and outs of the matter have little bearing. For the sake of this post, I’m only interested in what the numbers tell us about how Page Admins should deal with these emerging crises when they appear on their Facebook Walls.

Because, as it turns out, the accepted wisdom may be misleading.

Accepted Wisdom

Both Crisis Management and Social Media experts will tell you that it’s essential that a brand under attack does all it can to take control of a crisis situation as soon as possible. As one commentator on the Waitrose incident wrote:

The key to successfully managing an issue and ensuring that it doesn’t become a crisis is to keep communicating.

(Sarah Williams, Senior Lecturer in Public Relations, MMUBS)

This is relatively easy to say, but in practice, it’s often hard to identify emerging crisis situations in Social Media platforms.

What did Waitrose do?

As it happens, Waitrose Admins did comment on one of the early posts in the sequence.

For most of the sequence, though, they were more notable by their absence. After apparently ignoring the situation Waitrose admins finally weighed in at the end of the day with three identical posts in rapid succession:

Why did Waitrose post the same message three times?

Here’s an apposite quote from Stuart Bruce speaking in Communicate Magazine:

The volume of comments on the page makes it extremely hard to follow. One sensible thing Waitrose is doing is to repost its response, as if they did it just once and left it then it could be easily missed.

Stuart is one of the smartest chaps in this space; but he’s wrong here. I’m not trying to show him up, instead I’m using his quote to illustrate how easy it is to get Facebook Community Management wrong. It should make us all feel better to know that we’re in good company.

Why is Stuart right?

Let’s look at how he’s correct first. There was a lot of activity on the Page. The Facebook Wall is strictly chronological, and each time Waitrose posted the message it was pushed down and off the screen by subsequent user posts. By re-posting the message, they maintained a dominant role on the Page[2].

Why is Stuart wrong?

Here’s that chart again.

This time I’ve highlighted four areas. You’ll recall that the the vertical axis represents the individual contributors — the gradient represents new contributors joining the conversation. Together, the four highlighted areas represent 95 new contributors adding their voices to the mix.

The impact of the admin Posts outweighs the impact of the user Posts by an order of magnitude, as illustrated by the following chart:

In this particular instance, there were a mere 4 admin posts compared to the 65 posts from users. But those admin posts had a disproportionate impact on drawing new people into the conversation (more than 12 times as great: an average user post attracted 1.88 new contributors, an average admin post attracted 23.75 new contributors.)

New contributors have more "energy" for the fight (and will revive and rehearse old arguments and conflicts) so I think we can say that on the whole new contributors to a flame war are bad no matter how good their intentions.

You’ll recall that the rows represent the continued presence of a given contributor in the ongoing debate — so we can highlight the legacy of those initial three admin posts:

Why does this happen?

Facebook Pages are inherently asymmetric.

1. People interact with content in their News Feed and rarely visit Pages (for many of our clients, the figure is as low as 0.5% of Active Users.)
2. Posts made by an admin will be sent out to appear in a proportion of the Page’s Fans’s News Feeds[3]; reaching (in Waitrose’s case) tens of thousands of users.
3. Posts made by Fans will only be seen by visitors to that Page, and a (vanishingly tiny) proportion of their Facebook friends’ News Feeds.

Taking this into account, let’s retell the Waitrose Flame War story as follows:

A small number of people had a fight on Waitrose’s Facebook Page. None of Waitrose’s customers was really aware until the end of the day, when Waitrose admins broadcast three identical messages to as many of their Fans as possible.

What should they have done? What should I do if a Flame War breaks out on my Page?

What’s the lesson here? I’d say that — when managing Facebook Pages, concentrate on the News Feed, not the Wall.

Here’s what I’d suggest:

• Do not post Status Updates unless you want to reach a wider audience than is already aware of the issue.[4]

• Instead, comment on every new thread — even if it’s only a holding statement. These comments will be seen only by those who are already involved. Bear in mind that – as threads expand beyond 50 comments, Facebook pages the output and that as a result earlier comments may not be seen by later viewers.

• To avoid copy-and-paste fatigue, you should to maintain and link to a single page with the latest details. A blog would be a good place for this sort of content[5]; it really doesn’t need to be on Facebook. Whatever you do, keep it away from the Wall (publishing a Note, for example, would be an error.)

• Consider switching the default view to ‘Only Posts by Page’; but monitor carefully: it is possible that users may begin to add their complaints as off-topic comments on existing brand Posts.

9 days of activity on the ASOS Facebook Page

This chart (click for bigger) represents 9 days of activity on ASOS’s Facebook Page. Compare it to the Budweiser flow and you’ll see how focused ASOS is on Customer Service by comparison.

And yet their post frequency is high: an average of more than 3 posts per day. They promoted 11 individual links during the period which between them delivered over 120K clicks through to the ASOS site.

So Customer Service is only half the story; there’s a robust DR element here.

Here’s a brief summary:

Automated sentiment analysis uses a combination of two approaches to determine sentiment: lexical analysis (which looks for emotionally-weighted words) and machine-learning (which relies on a “training corpus” of manually scored documents to predict the emotional content of new documents that it processes.) Continued human intervention (“training”) in the machine learning process may improve results, but makes it more or less meaningless to compare the results over time.

No two sentiment analysis tools on the market agree closely; with most disagreements occurring and around “sentiment-neutral” comments. Since these form the bulk of the content, there is much room for disagreement.

Sentiment analysis on short content (e.g. Tweets) lacks sufficient context for accurate judgments, whereas analysis of longer content often lacks sufficient relevance (e.g. the search term may only be mentioned in passing, and the sentiment score refer to a different object)

Human-based/manual sentiment analysis also faces reliability challenges. There is often as much disagreement between two human analysts as there is between two automated systems. Worse still, research demonstrates that the same person – when presented with the same text on different occasions – may score it differently each time.

Conclusion: Sentiment Analysis may be used to guide customer service engagement, but should not be used as a KPI

Have a glance at this chart showing three years of search traffic (click for a live version) 

In previous years, search traffic has more or less followed the announcement – but for the (still mythical) iPhone 5, search traffic began to grow well ahead of time (a situation — as Scott Thompson has pointed out to me — probably only exacerbated by the delayed announcement.)

Bloggers and journalists are increasingly using social signals (search & twitter volume) to determine their editorial policy. After all, search and social are a good indicator of what interests your audience – and major traffic drivers.

In this case, however, the process created a vicious feedback loop. Even the more sensible commentators and analysts found that — when the conversation was about the iPhone 5, there was no virtue or value in writing about it in any other way.

I used Google to count the posts that mentioned “iPhone 5” in the title on some of the top tech blogs, MSM and online news sources — the numbers tell a story.