The Net-Savvy Executive

On the (Non-) Automation of Insight

2009-10-23T14:57:55Z

Is it a problem of overpromising/underdelivering, or are people developing these unrealistic expectations on their own? Either way, I'm seeing more examples of people who seem surprised that software doesn't do all of the work in social media analysis. I really don't think this is controversial: regardless of your choice of tool, there's a necessary human contribution to the process.

This notion that the software isn't good enough because it requires a person to do something with it seems to be picking up speed. The first post that really kicked off a conversation was probably Asi Sharabi's. I saw a couple this morning, including one from Mark Schaefer that focuses on the graphics:

I’ve been spending time studying the trends in social media monitoring and have been impressed with the rapid progress. But there is still a lot of noise like this chart that really tells us nothing. The fact is, the most meaningful keyword and sentiment analysis is all still being done MANUALLY.

I'm not arguing that anybody's tool is perfect (the steady stream of updates strongly implies that the vendors don't think that, either). This is still a new category, and the software will evolve. So observations about which pieces work well—and which pieces need improvement—provide a valuable contribution. But we're not going to see a product that (a) analyzes the world, (b) develops meaningful insights, and (c) delivers it in a tidy, executive-ready package.

Building the social media spreadsheet
Think about spreadsheet software. When you first open a new spreadsheet, the software gives you a blank page. In the right hands, the software is a power tool for running financials, forecasting results, analyzing historical data... I've seen some impressive examples, but the most powerful spreadsheet software is useless without someone who knows how to use it. Which, if you think about it, is true of most software.

Social media analysis tools are software; they do some of the work, but to get the most out of them, you need someone who knows how to use them. The more you expect from your tool, the more the user needs to know. Anything that's fully automated either isn't doing much, or it has a lot of human effort baked into it.

Regardless of the tools used, at some point people take over. It may be earlier in the process (manual content analysis) or later (analysis and reporting), but eventually, a person takes what the computer produces and does something with it. All of those agencies that sell services based on the same SMA platforms presumably think this is where they add value.

If you want it done for you...
There is an answer for the company that wants the insights without putting in the effort, of course. Have someone else do it. You can't bypass the requirement for a human analyst, but you don't have to do it yourself. When you're shopping for social media analysis, just be sure to include analyst services among your requirements. That eliminates some of the best-known software companies, but it opens the door to an entirely different set of service providers.

If you want to make spreadsheets, you buy Excel. If you want financial projections, you roll up your sleeves or hire a financial analyst. It's up to you to decide whether that analyst will be an employee or work for an external service provider.

Where's the disconnect? Are unrealistic customer expectations coming from vendor hype, or is it just hope that things will be easy?

Taking Social Data To Go

2009-10-14T15:36:48Z

I try not to be too obnoxious with my iPhone, but it's hard to avoid being impressed with how it's changed my expectations about mobile computing. I've been wondering when social media analysis apps would start showing up, and now I've found one to play with: iCrossing's new Say What?

Compared to the tools I usually look at, Say What? (iTunes link) is pretty basic. It runs searches across Twitter, Digg, forums and blogs, returning a sampling of the results from each. It's not much, but it might be enough to get a quick sense of what's going on with a topic (especially if it's currently controversial).

Why are people talking about that?
Say What? is best used for getting a clue about a current topic—especially if it's controversial or newsworthy. Looking at results, four per screen, you're looking for someone to provide a hint about what's going on.

Looking up (Rush) Limbaugh, I immediately found mentions of his interest in buying into an NFL franchise. Ford returned items about the latest product recall. But when the company isn't making news, the results are far less interesting—people are apparently having breakfast at Dunkin' Donuts.

From Search to Analysis
I've been looking into web-based collaboration and project management tools for my own company, and as soon as I realized that some of these tools have iPhone apps, that became a requirement. We have so many web-based tools for monitoring and analyzing social media; who's going to be the first to offer a simple dashboard that delivers clients' data to smart phones?

Linking Social Media to Intelligence

2009-09-29T18:30:33Z

Think of your favorite model or metrics for measuring social media activity. Flip through Olivier Blanchard's presentation on social media ROI. Now, with that in your head, read Tom Davenport's 2007 book, Competing on Analytics. How far do you get before realizing that the enterprise analytics crowd is asking some of the same questions as the social media crowd, but looking for answers in different data? What if the two groups met?

When we asked CIOs to identify their visionary plans for enhancing their enterprises' competitiveness, business intelligence and analytics was the top answer, selected by 83 percent of our sample... "Facts drive decisions," said an Insurance CIO. "Plans for imbedded analytics need to enable data capture at the customer touch point."
— IBM's 2009 Global CIO Study (via KDnuggets)

What would happen if you were to analyze social media data alongside operational data to look for insights in the interaction between what people do online and what they do with your company? You could measure the ROI of marketing in social media, but that's a defensive move (protecting your job/budget). Beyond learning what works and what doesn't, what would you learn by looking at the data together?

Are you doing this now? I'm looking for companies to interview for my research.

Agency Approaches to Measuring Social Media

2009-09-24T05:08:24Z

Measuring Social Media must be the new black. Everybody's doing it—the in-crowd is, at least, and the rest are starting to realize they're missing something. Just look at the agencies who've suggested their own take on the little black dress—that is, on how marketers should measure social media.

Conversation Impact, Ogilvy PR

Ogilvy proposes a framework with three sets of metrics that correlate to the traditional marketing funnel: Reach and positioning, based on a combination of web analytics, media analysis, and search visibility; Preference, based on media analysis and traditional research; and Action, based on measurable client objectives (such as sales conversions).
Social Influence Measurement, Razorfish (with TNS Cymfony and Keller Fay Group)

The SIM score, as introducing in the Fluent 2009 report, compares sentiment for a company to sentiment for its industry. The report also mentions share of voice and weighting for influence, although the formulas for the metric do not.
Digital Footprint Index, Zócalo Group (with DePaul University)

Evaluate a brand's online presence in three dimensions: Height, which represents the total volume of brand mentions; Width, based on consumer engagement with online content; and Depth, based on message saturation and sentiment.

Three agencies, three models that fit fairly neatly into measurement silos. I've grouped them on the somewhat arbitrary basis that they're all backed by marketing agencies, but they're not answering the same question, are they?

It was my understanding that there would be no math.
—Chevy Chase, as Gerald Ford

Breaking eggs, making omelets
Ogilvy's Conversation Impact tracks marketing effectiveness with its explicit alignment with the marketing funnel. I like that the framework intermingles different sources of data, including traditional research. The Action category, linking measurement to specific business outcomes, should help keep strategy and measurement on point.

Razorfish's SIM score is all about perception. How does the client look compared to its competitors and industry? Despite the "influence" in its title, this score is entirely about sentiment, with none of the usual indicators of influence. As a single metric, the SIM should be compared with the Net Promoter Score or MotiveQuest's Online Promoter Score, but I'll be honest here. I'm having trouble figuring out the significance of this ratio of ratios. I tried a few scenarios to get a sense for how the numbers change and got some strange results: divide by zero errors, negative scores... The intermediate Net Sentiment metric is the more meaningful number here.

Zócalo's DFI addresses PR effectiveness, as telegraphed by the "earned media" headline in the announcement. The focus on presence, engagement and sentiment pick up on important aspects of social media, but without more detail on the math behind the overall index value, this seems like another framework rather than a metric.

What are we measuring, exactly?
I'm not the first to say it: the golden metric that will answer every question does not exist. To be fair, the authors of these examples don't claim to have the ultimate answer, anyway. Social media initiatives can support diverse objectives, and so the metrics used to evaluate those initiatives or to answer questions will be equally diverse. But it is nice that we have people sharing their efforts to find appropriate metrics for some common objectives and questions. Thank you, and keep it up.

While working through the math, I was reminded of an old trick from undergrad science classes: if you're losing track of your formula, do the math with the units included. If the resulting units don't make sense (comments^2, for example), the value won't, either.

Photo by alist.

Five Modes of Listening

2009-09-17T17:29:47Z

I'm working on a theme that's all about expanding our idea of listening—it's so much more than defensive monitoring, but we need to get beyond first steps. After the last post, Sam Flemming commented on the importance of distinct terms for communicating outside of the bubble, and he's right. After we expand the concept of listening, we need to break it into manageable pieces. Fortunately, the pieces will look familiar.

As a set of activities, listening breaks down into these five modes:

Searching
Search is so familiar that we don't always think about it, but look at the advice on getting started in social media. That first step: find out what people are saying, where they meet—you know, the 5 Ws—when you do that as a snapshot, that's search. Don't neglect the value of familiar methods.
Monitoring
The usual starting point for a discussion of listening. Through automated methods (typically a dashboard or RSS reader), find and read new posts, comments, tweets, etc. that are relevant to your business. Focus on individual items for action.
Alerting
Similar to monitoring, but the system notifies you through email, instant messaging or text when a new item is discovered. Alerts can also be based on measurement thresholds, such as a sudden increase in negative commentary. No requirement to revisit the platform to receive alerts.
Measuring
Add a quantitative element to monitoring. Whatever your choice of metrics or measurement silo, measurement is about aggregation and numbers. For the purposes of this list, I use measurement to refer to the generation of regularly updated metrics.
Mining
Add a quantitative element to search, and you have data mining, which looks for meaningful patterns in archival data. Although it has a lot in common with measurement (as used above), I'm seeing different practices and benefits that justify separating the two.

I know some knowledgable people in the space will disagree with my definitions, but my point is not to start another semantics argument. And I'm certainly not discounting the importance of looking at the data and interpreting its significance. The point of making these fine distinctions is to point out areas where we may be missing some of the value in listening.

For example, if you're doing routine measurement—you're looking at meaningful metrics on a regular basis—is there an opportunity to find different value by taking a mining approach, looking for insight in a snapshot of historical data? A slim distinction, but the point is to step back, walk around a bit, and look at the data from another angle.

Actually, lots of other angles, but more on that later.

Photo by bdu

Setting Social Objectives, Inside and Out

2009-09-17T15:14:27Z

National governments represent a special category of large organizations: they're far larger than any company, and they're in a funny kind of business. But their talent for generating documents occasionally leads to something of value in the business world. Would you believe a strategy document that frames the relationship between social media and Enterprise 2.0 in a sidebar?

Though it's not what most people will be looking for, the new 2009 [US] National Intelligence Strategy (PDF) neatly categorizes two types of objectives for the intelligence community (IC). If you squint a little, I think you'll see how these categories could be repurposed for the 2.0 crowd:

Mission Objectives

MO1: Combat Violent Extremism
MO2: Counter WMD Proliferation
MO3: Provide Strategic Intelligence and Warning
MO4: Integrate Counterintelligence
MO5: Enhance Cybersecurity
MO6: Support Current Operations

Enterprise Objectives

EO1: Enhance Community Mission Management
EO2: Strengthen Partnerships
EO3: Streamline Business Processes
EO4: Improve Information Integration & Sharing
EO5: Advance S&T/R&D
EO6: Develop the Workforce
EO7: Improve Acquisition

Identifying external and internal objectives
Obviously, I'm not suggesting that national security and social technologies are the same thing. If you're not in the national security business, then "combat violent extremism" isn't your first objective. Instead, look at the framework. I think that the distinction between mission objectives and enterprise objectives might just clarify the relationship between externally-focused social media and internally-focused Enterprise 2.0 initiatives.

Social Media for Mission Objectives
Mission objectives are closely linked to the overall objectives of an organization. At the enterprise level, these are measured in terms of financial success; in marketing, they're the familiar product- and customer-oriented objectives that lead to financial success. These are the kinds of objectives we see in social media discussions (especially when social media for business is interpreted as social media marketing):
- Combat negative impressions of the company
- Improve customer communication and responsiveness
- Increase brand visibility
- Enhance customer loyalty
- Integrate market intelligence
...add your favorite social media objective here. The social media focus on connecting with the worldwide conversation in support of the business reflects an emphasis on mission objectives.
Enterprise 2.0 for Enterprise Objectives
The E 2.0 case is even easier to make—look at E04 on the NIS list (improve information integration & sharing). Look at the list through your new technologies lens, and you'll hardly need to edit to start applying it. Enterprise objectives are about making the operation work better, so the prescriptions are generic, not specific to the organization's mission. Despite the idealistic rhetoric, improving the operation is the argument for E 2.0.

Aligning social media and Enterprise 2.0
I don't think the internal/external view of social media and E 2.0 is all that new, but I do think it's instructive to see the two types of objectives neatly linked in one document. If the evangelists of social business strategy succeed, I think we'll see more explicit alignment of these high-level categories.

Thanks to Andrew McAfee for pointing out the new document.

Expanding Our Idea of Listening in Social Media

2009-09-10T15:18:28Z

Everyone says that listening is central to social media success, but over time, we've fallen into a too-narrow interpretation of the metaphor. Think about it: if listening means monitoring, then we have too many words. Fortunately, they don't need to mean the same thing. We just need to expand the way we think about listening.

Here's the definition of listening implied by many posts and presentations:

Defensive keyword monitoring of social media for customer problems and complaints that need a communications or customer service response.

In the social media buzzword compendium, that's a great example of listening. But as a working definition, it leaves a lot out. Almost every word imposes a limitation on finding all of the value in a listening strategy. We can do more.

How can we expand the definition of listening?

From a defensive posture to developing valuable market intelligence.
From keyword monitoring to applying all of the technologies available to discover and analyze relevant online content and activity.
From monitoring to metrics, mining, and interpretation. It's a metaphor, so there's no reason to be stuck with the word's literal meaning.
From social media to all media and customer communications.
From a focus on problems and complaints to an interest in all relevant conversations.
From PR, marketing, and customer service to anywhere the information has value to the business.
By collaborating across measurement silos to find the right methodology for the task.

More formally, I think of listening as the application of intelligence and analytics to social media (and other sources), but that's so many syllables. If you don't mind, I'm going to continue to say "listening," and when I do, you'll know that I'm talking about a lot more than monitoring Twitter for your brand name. 'k?

Social Media on Healthcare Reform

2009-09-10T14:32:43Z

Find a topic that a lot of people care about, and you'll find a great pool of data for social media analysis companies to use in a demonstration of their work. Over the last few years, we've seen examples based on Super Bowl ads, American Idol, and national elections. Now it's healthcare reform in the US, where discussions are—uh, generously seasoned with sentiment. Just the thing to show off your analysis chops.

Here's what's shown up so far:

Obama Health Care Speech: What Did Twitter Think? (Crimson Hexagon)
Reactions to Obama’s Health Care Speech to Congress (Echometrix)
How Obama lost the healthcare debate online (Market Sentinel)
Healthcare debate analysis (MotiveQuest)

Anyone else working on an analysis they'd like to share?

photo by Rob Stemple.

Scaling Human Analysis

2009-09-04T17:33:35Z

One thing about sentiment analysis: it really stirs up the opinions. Apparently, it's good for attention, too, because yesterday's post has gotten a lot of it. So what is it about automated analysis that's so controversial, and what can human analysts do to offset the advantages of automation?

Automation offers four basic benefits:

Scale
Keeping up with all of the relevant conversations as volume grows.
Speed
Processing new items sooner; computers "read" faster than humans read.
Consistency
Software doesn't get less accurate with fatigue or mood, and it doesn't consider contextual knowledge if it's not supposed to. It just follows instructions, over and over.
Availability
Automated systems don't sleep, so they won't be the limiting factor in determing your 24/7/365 operations plan.

The trade-off—or the development challenge, depending on your point of view—is in accuracy and interpretation. Most of the discussion focuses on these accuracy issues, so let's think instead about the less controversial benefits and how a software-assisted human analysis approach can compete with them (remember, SAHA is human analysis within a software-mediated environment for operational efficiency).

Competing with automated systems
Scale and speed are related, and the hands-on approach is simple: add more people to the process. Speed (latency) will still be limited by the ability of your analysts to read quickly. You won't compete in the sub-second latency market, but you can get ahead of the daily-update crowd.

Consistency in human scoring comes down to training and process. I won't pretend to teach the media analysis pros how to do that job, but it's going to be more formal than the eyeball ratings I gave out yesterday.

Availability is an interesting challenge, but it's not the first time companies have addressed the issue. If you're going to work through nights and weekends, your choices are to create some undesirable jobs at home or switch to follow-the-sun operations overseas.

The rise of offshore outsourcers
Combine the need for a lot of people (analysts) with the desirability of around-the-clock operations, and a lot of people will reach the same conclusion. From the beginning of the social media analysis business, some of the better-known vendors have had development and analyst groups in India. Now, I'm starting to hear from companies that are offering offshore human analyst services as a specialized service.

The interesting bit is that they're unbundling the content coding, so clients (or vendors) can add human-powered sentiment analysis to any platform that provides user coding or tagging.

This won't be an easy group to track. It's largely a traditional outsourcing approach, and any company that provides full-service social media analysis using human analysts could unbundle the coding piece, too. But if clients end up selecting human analysis over the automated version, expect more offshoring of the manual effort.

Photo by vitorcastillo / CC BY 2.0

The Sentiment on Sentiment Analysis

2009-09-03T17:31:48Z

Since the recent New York Times piece on sentiment analysis, it seems everyone has an opinion on sentiment analysis (how appropriate, yes?). Without actually counting, I'm getting the impression that the overall score is negative, although with the colloquialisms and subtle innuendo, I'm not always sure. :-)

This is a round-up post, so I'm going to start linking to posts I've seen in a minute, but first, we have a problem: a buzzword alignment problem on what to call companies who monitor and analyze social media content. The article uses sentiment analysis to refer to the industry, but sentiment analysis is better understood as just one of the types of analysis used in the field.

This industry has a history of picking up a new label almost every time someone new writes about it. Forrester Research has called it brand monitoring and listening platforms, depending on which year and analyst you ask. I picked social media analysis when I had to choose, but even that is more limited than the state of the art tools and services. I don't have an answer to that one that makes me happy just yet.

Scoring the conversation
Oh, OK, I'll count. Really, how could I resist? Isn't this the obvious way to collect the posts on this topic?

Positive

Sentiment First, Ask Questions Later, Joseph Fiore (RepuMetrix)
Text Analytics for Ye of Little Faith, Manya Mayes (SAS)

Negative

Sentiment Analysis, Scott Berkun
Sentiment analysis for online content: Honest?, Urs Gattiker (CyTRAP Labs)
La tarte à la crème du sentiment analysis, Guilhem Fouetillou (Linkfluence)
Sentiment analysis crap in social media, Laurent François (Ogilvy)
Is Sentiment Analysis Reliable? Andy Beal (Trackur)

Neutral

Sentiment mining: new term, new field. A new web? Claude Théoret (Exvisu)
The sentiment analysis party has plenty of elbow room, Christine Sierra (Lexalytics)
Sentiment Analysis, Eric Friedman (Union Square Ventures)

This was an ongoing discussion long before the Times article. Mike Marshall made for the case for automation of large-scale analysis in the first guest post on SMA. I suggested additional models for the human vs. computer dichotomy in early 2007. I don't imagine we'll settle this any time soon.

This list is an example of document-level sentiment analysis by a human. Anyone want to make the case that it might not be 100% accurate?

Listening Is Not (Only) Defensive

2009-09-01T13:12:16Z

All together, now: "Companies should listen to social media." We all know the advice, but do you have the impression that listening is a purely defensive strategy? It's not. You just have to move beyond the common, but limited, interpretation of listening.

How often does your defense score?
In a recent survey of management, marketing and HR executives in the US, Russell Herder and Ethos Business Law found a strong defensive leaning in respondents' current use of social media. The top reasons they use social media?

Read what customers may be saying about our company (52%)
Monitor a competitor's use of social media (47%)
See what current employees may be sharing (36%)
Check the background of a prospective employee (25%)
None/personal use only (16%)

Not exactly the way I would put it, but this isn't entirely a bad start at listening. At least they've gotten some of the message. It's a little heavy on the fear motivation, but it's a start. The trouble is, it's only a start.

Put your listening on offense
Think about my earlier list of conversations you should care about, and let's come up with some things you can do with the information you find. Defensive ideas are easy (and rampant). Let's focus on putting some points on the board. I'll start:

Spot sales leads where prospects ask questions or contact you through public channels.
Figure out a competitor's plans from their public statements and personnel changes.
Figure out a customer's plans (and needs) from their public statements.
Identify a competitor's weakness in online complaints; launch a product or program to exploit it.
Identify a product or service opportunity in online discussions; fill the gap before competitors notice it.

That's a short list; what does putting listening on the offense make you think of?

Listening can be defensive—and if you're not monitoring for customer complaints and other problems, start. But don't stop with defense; think about how to apply it to advantage, too. Although it sounds passive, listening doesn't have to be either passive or defensive. Don't be satisfied until you find the path to profit for your business.

Thanks to Deni Kasrel and Bill Ives for pointing out the report and the defensive tone of responses to the question.

Photo by avinashkunnath/

No More Tag Posts

2009-08-27T20:29:24Z

I turned off the automatic tag posts from Delicious today. Murphy being on the job, the post linking to the recent New York Times article on sentiment analysis promptly got a comment when I did that—a good one, too. But the tag posts are gone. The list of recent posts was overrun with "links for [date]" entries, and I lost some email subscribers who were probably tired of them, too. So they're gone.

I'll continue to use Delicious; it's part of my publishing workflow, as well as my bookmarking service of preference. I'll also continue the practice of adding editorial content in the comments to my tags. Depending on the tag, these items will continue to appear on my web sites. If you want to see all of them, they're also available as an RSS feed.

I have some good stuff in the drafts folder. I really don't want it to be lost is a sea of links posts.

I don't know if this is one of the recognized corollaries of Murphy's Law, but I've long known that you can't game the system. If you count on Murphy to make something work out in the end, it won't.

Social Buzzword Alignment

2009-08-27T16:42:15Z

Talking with a friend who is smart but outside of the bubble, I was surprised that one of my usual comments surprised her. If you really want to find all of the insights on social media that might be relevant to your business, you need to track down some closely related buzzwords: Web 2.0, Enterprise 2.0, community, and WOM.

These buzzwords aren't synonyms, but they're very closely related. Serious discussions of one tend to bring up the others. The catch is this: events, organizations, suppliers, and thought leaders tend to be aligned with one of them, so it's easy to miss significant contributions to the discussion if you focus on only one.

For those just learning social media, I usually recommend looking up some of these other topics. Here's why:

Web 2.0
This buzzword has dropped off the hype charts, but before social media caught on, people were thinking about many of the same trends under the Web 2.0 banner. Web 2.0 lives on in the 2.0 appended to so many buzzwords, such as...
Enterprise 2.0
Social media inside the firewall—that's E 2.0 in a nutshell. I realize the vision is a bit different, but the tools are the same, and those who start thinking about using social media inside their companies should know that a different group of thinkers is already on the case. We're seeing the realization that doing social media well (in business) and applying E 2.0 principles are closely related; Dachis Group's social business design construct is an early example of linking the trends.
Community
Who you're trying to connect with through social media. Emphasizes the strategy of connecting with people instead of the tools. Can you really talk about social media for more than ten minutes without using the word community?
Word of Mouth
What the marketer wants to encourage through social media. Go to a WOMMA meeting, and much of the talk is about WOM in social media.

The map is not the territory; the buzzword is not the thing. Niall Cook gets this. Each of these buzzwords is a label—a handle to help us get a grip on a new set of concepts. There's even a longer list, with social computing and consumer-generated media to focus attention on different attributes of what's going on.

Rather than debating the merits of a label or limiting ourselves to label-induced intellectual silos, let's focus on figuring out the concepts and making them work. The whole is way more interesting than the sum of the parts.

links for 2009-08-26

2009-08-27T02:01:01Z

10 things you should cover in your social networking policy

Interesting high-level outline.

(tags: policy socialmedia corporate msm)
Let me Show You Inside a Secret Blogging Alliance

A group of bloggers work together to make each successful. Hmm.

(tags: blogging)
E-Reputation Market Map / Carte du marché de la e-Réputation

Not just a list, a periodic table!

(tags: socialmediaanalysis lists)

links for 2009-08-25

2009-08-26T02:00:40Z

Sentiment Analysis Takes the Pulse of the Internet

I'm struck by the variety of terms used to describe this field and the different results from searching/tracking the variants. This NYT article doesn't include "brand monitoring," "listening," or "social media analysis" as it focuses on "sentiment analysis," but they're all more or less the same thing.

(tags: socialmediaanalysis textanalytics)