Most out-of-the-box tools such as Radian6 usually raise many false negatives. For a broad search the majority of sentiment is generally ‘neutral’. Though for the 10% or so which may be negative or positive, what is the best way of identifying or increasing your confidence scale? In future posts I’ll discuss workflows and optimizing your confidence rating. The majority of false negative sentiment stems from colloquialism and slang. A field of computer science and artificial intelligence named Natural Language Processing was developed to interpret human speech via computers. Today this field has advanced to the point where IBM’s Watson is able to obliterate all-time Jeopardy! winner Ken Jennings. Wondering how you can place decades of research into your own marketing campaign and decipher human speech into digital signals?

Here are three great open-source resources:

OpenNLP: supports the most common NLP tasks, such as tokenization, sentence segmentation, part-of-speech tagging, named entity extraction, chunking, parsing, and coreference resolution. These tasks are usually required to build more advanced text processing services. OpenNLP also includes maximum entropy and perceptron based machine learning.

Natural Language Toolkit: NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning.

The Stanford NLP Group makes parts of our Natural Language Processing software available to the public. These are statistical NLP toolkits for various major computational linguistics problems. They can be incorporated into applications with human language technology needs.

Why apply NLP for sentiment analysis?
A quick use-case on why such tools are important for contextualizing your conversation streams. A broad keyword such as “Ford” will yield a spectrum of results such as: Ford Motor Company, Toronot’s mayor Rob Ford, Betty Ford Clinic, Henry Ford Museum, etc. If you’re only interested in automotive related Twitter conversations then the rest is simply noise. A finely-tined NLP post-process can filter the noise and allow you to target only your high-value data streams.

On February 12th I’ll be discussing using Sentiment Analysis within your digital marketing campaigns. Join the chat every Tuesday at 5PM PT/ 8PM ET using the hashtag #MetricsChat.

How would the Emperor have tried on his new clothes with today’s technology? Given his vanity, and thirst for public affection he would most likely have had a presence on Instagram. He surely would have racked more love than Zooey Deschanel, Snoop Dogg Lion, and the Kardashian sisters, combined. The Emperor would have perfected the self-mirror shot better than Harry Styles, both of whom, could always be found in their dressing room.

If two weavers today, would claim they design the most beautiful clothes with the most magnificent fabrics imaginable, all one would have to do is look up their store on Yelp. What’s more accurate than crowdsourced reviews? This is of course, one of the attractive features Sir Zuckerberg pitched to the Facebook community marketers & investors. In 2013, Andersen’s story would have ended right here, for the Weavers 1-star ratings would have them kicked out of the Royal court. They also lacked a Foursquare listing, because they never could verify their phone number (it was yet to be invented in the 1830s,) and they were after all swindlers.

Our scenario of course is hypothetical, and the Emperor gullible, so let us nonetheless continue. The Emperor’s advisors demonstrated some level of intelligence. Using Snapchat they shared Gilt.com’s latest 70% off designer clothing and securely shared it with the Emperor who insisted every message be deleted in precisely 10 seconds or less. In case he felt the public would disapprove, the Emperor wanted no digital trace that he contemplated undesirable threads in his digital footprint. For this same reason, the advisors set the Emperor’s Facebook privacy settings accordingly so he could gather the opinion of his latest outfits with only his noblest noblemen, prior to sharing it publicly with his 1 billion fans. On the day of the fitting, the weavers carefully placed the new delicate outfit upon him and he quickly snapped away a few photos through the reflection of the looking glass for his private circle to approve. Not wanting to seem unfit for their position, collectively the advisors all gave it a big thumbs up.

The evening prior to the procession, a Storify bug had publicly exposed the Emperor’s private story. A young lad, who also happens to be a Redditor picked it up and posted it online. Known for their sharp wit and quick judgment, one member of the community commented “But he hasn’t got anything on.” With many upvotes, the story quickly gained steam and went viral, it spread across the Web on Twitter, Tumblr, even Buzzfeed compiled Top10 Royal Faux pas. Three days later when Joan Rivers finally heard about it she ripped apart the non-outfit on Fashion Police.

###

It is inevitable that as strong as a security encryption may be, it is only as strong as its most secure app. Bugs will expose vulnerabilities, and private data will be leaked. The more public or sensitive the data the higher the likelihood for this occurrence. Are we gullible like the Emperor to think it’s secure or is it simply a calculated risk? If so, why do we take the risk? Drs. Keith Wilcox and Andrew Stephen from Columbia Business School and Pittsburgh University respectively, found that when people interact with close friends on social media, they get a boost in self-esteem. Unfortunately, they also demonstrate less self control. Much like the child in the original tale, do we know the only way not to expose ourselves is to simply not post something controversial, or is it simply a difference in generational behavior? To answer this we can look toward today’s Gen-Y. Recently at CES a 22-year-old fashion designer named Darius summed up his generation’s attitude by saying: “We live in public.” He is aware everything he shares on Twitter and other social platforms is “out there” and remains extremely conscious about what he posts.

Over the holidays even Randi Zuckerberg fell victim to a security loophole when a friend of a friend unknowingly nabbed a photo and shared it on Twitter. Facebook Graph Search is bound to amplify these types of instances. Numerous articles have been published such as Venturebeat’s steps to take before Graph Search embarrasses you.

TechCrunch called Graph Search humorless, creepy and doomed to disappoint. While Forbes described it as a “disruptive minefield of unintended consequences.” Peppering the article with other classifications of “Engineering narcissism” and “Ironic ambiguity.” Would this type of scrutiny pass for an unknown startup? While it presents a lot of potential value, Graph Search’s focus may be disguised to profit the swindler at the expense of the Emperor, who the platform initially held with high regard. This seems to be the cycle when any social channel reaches critical mass which Facebook achieved long before its IPO. Despite transparency to its TOS and privacy policy, Zuckerberg et al need to answer to a board of directors.

Harmony is possible to achieve with certain regulations while maintaining privacy and security. A shift in paradigm must occur for the people to see the value and for the gatekeeper of the data to be properly accountable and adhere to a high standard of security. Otherwise we would have learned nothing from this centuries-old tale.

This is my story which I have related. If it be sweet or if it be not sweet, take some elsewhere and let some come back to me*.

The original tale by Hans Christian Andersen: The Emperor’s New Clothes

* Traditional Ashanti end to a story

Prior to the 2012 baseball season I collected a set of social media metrics to measure social engagement across all 30 MLB teams. The San Francisco Giants, who made it to the World Series happen to be the most engaged team in April. The Detroit Tigers held an honorable 5th place position behind both LA teams and the Texas Rangers. Impressive seasons for the Tigers and Giants, and post season wins was reflected in their social profiles. While both teams increased Facebook and Twitter fans, the Tigers roared their way to social media greatness.

Baseball is filled with statistics, such as HBP, LOB, RISP; and social introduces a few new variables. Let’s take a look at some of the numbers. As expected, pre and post-season social fan volume increased for both teams. The Giants began their season with 1.35M fans and the Tigers were 50K shy of the million fan mark. The Giants and Tigers increased their fan volume by 16% and 17% respectively during the 2012 season. Another interesting stat is PTAT (People Talking About This) which is measured by the ratio of Facebook fans to comments in Facebook discussions. Intuitively, both the AL and NL champs significantly increased their PTAT index. San Francisco began the season with a PTAT index of 4.5 and are presently at 19.4. The Tigers, however, which began the season with an index of 3.15, now account for a PTAT index of 30.6; nearly 10 times their April 2012 score.

Twitter fan volume also increased significantly for both World Series teams. As you can see in the table below, San Francisco increased their Twitter fan base by 42.5%. The Tigers, nevertheless, who began their season with a humble 55K Twitter fans, increased their count by over 220% to nearly 180,000 fans.

Surely, having the Tigers’ Miguel Cabrera being the first player since 1967 to earn a Triple Crown, amplified Detroit’s social engagement. Game-time social promotions such as “Tweet your Seat” also engage fans throughout the season while at the ballpark with giveaways. Though Twitter was founded in San Francisco, this year, the Tigers definitely grabbed the platform and ran with it. When it comes to social media they’re essentially batting a thousand! My official pick is Tigers in 6, what’s your prediction?

Locally, the annual debate regarding the Dream Cruise heats up during the weeks preceding the event. There is little polarity among Metro Detroit residents and business owners who reside or own a business along the main artery; they either hate it or love it. The social landscape, however, provided a different outlook. By mining the conversations for sentiment using Radian6 linguistic engine, only 7% of conversation streams contained negative sentiment. The majority of negative comments contained the word “traffic” such as this Tweet below.

An estimated 1.35 million classic car owners and fans turned out for this year’s Dream Cruise which some described as Woodstock on Wheels, only instead of dirty hippies, waxed Hemis and shiny Camaros were on display. Detroit’s Big 3 provided the crowd, and auto enthusiasts alike with plenty of horsepower to stay captivated with the City which put America on wheels. Chrysler Group LLC, in its largest collection ever had 75 vehicles on display including the 2013 Dodge Dart Rally Car, the 2013 SRT Viper, Ram’s Raminator Monster Truck and more than a dozen classics from the Walter P. Chrysler Museum.

GM, a sponsor of the event celebrated its Chevrolet brand’s 60th anniversary of the Corvette. Detroit’s graffiti artist Shades painted a Corvette inspired mural to positive social acclaim.

Ford’s annual Mustang Alley featured more than 800 Mustangs from 1964 through today. Mustang fans could also preview the 2013 Ford Shelby GT500, the Boss 302 and an array of SVTs (Special Vehicle Team) commemorating their 20th anniversary.

Offline events translated well into online influence as each brand generated a dominant market share and digitally engaged the social media crowd. Chevy and Dodge also took Top 10 social honors, along with mainstream media accounts for the Detroit News and Free Press. Ryan Brisco, the Australian IndyCar driver and part of Birmingham owner’s Team Penske rounded off the Top 10 list. Brisco was in town for the Pure Michigan 400 Cup at the Michigan International Speedway this weekend. Though a big fan of the auto industry, and auto racing he may have contributed to the 7% negative sentiment when he was quoted saying: “I’m not much for this traffic, but I could see myself in one of those lawn chairs by the side of the road with a couple of beers watching this parade go by.” We can’t blame the Aussie, especially since he’s used to traveling at several hundred miles per hour.

Batman Makes An Appearance

The Labattmobile

Many custom vehicles also rode alongside the classic, and Batman even made an appearance. Incidentally, the most unique ride of the 2012 Woodward Dream Cruise belonged to the Labattmobile which demonstrated that driving what you drink is perfectly legal. Be proud Canadians! Til next year.

Photos by David Coates | The Detroit News

Curiosity isn’t the first non-earth object to enter the Twittersphere, its predecessors Spirit and Opportunity, as well as Phoenix communicated updates via the popular social medium. Their status, however, doesn’t compare to Curiosity’s unprecedented interest. The voice behind the account is the Jet Propulsion Laboratory (JPL) social media manager, Veronica McGregor and her team Courtney O’Connor and Stephanie Smith. Together, along with engineers and scientists they share the latest findings from the Red Planet.

Countless Internet memes such as this Tumblr account highlighting JPL’s Bobak Ferdowsi acted as a catalyst to an already popular campaign. The top post to date received over 72,000 Retweets and nearly 15,000 favorites:

The 2nd most popular Tweet with over 30K Retweets demonstrated the JPL social media team’s healthy sense of humor:

Even the Whitehouse contributed to amplify Curiosity’s social signal when President Obama called to congratulate the team.

This kind of Tweetention may make Lady Gaga envious, especially if the rover finds little monsters, otherwise known as microbes. Nevertheless, it’s captivating to see the buzz surrounding science and research opening up the curiosity for us Earthlings. Here’s to a million more fans, and many more exciting Tweets from JPL’s mission control!

Twitter is one of the most popular means of global communication today, with some even dubbing the London 2012 games as the “Social Olympics.” For a communications director to ask to suppress the most popular form of user-broadcasted information is simply absurd. This would be analogous to the department of transportation asking to avoid a major freeway due to potholes, or any other obstruction to quality, instead of addressing the root of the issue. The 500 million+ Twitter users, must have reacted the same way Alexander Graham Bell felt when the ‘Mute’ button was invented for the telephone.

Now that the games have concluded, we can see the popularity of the medium from which spectators were asked to restrain just two weeks prior. More than 150 million Tweets about the Olympics were transmitted over the 16 day period. In fact, the 2012 London Olympics Opening Ceremony generated more than 9.6M tweets, surpassing the entire 2008 Olympics games in Beijing. One way to assess excitement is to measure Tweets per minute, and several moments generated a remarkable volume of Tweets. Usain Bolt winning the gold in the 200m sprint generated 80K+ TPM. Andy Murray securing the gold over Roger Federer accounted for over 57K TPM. When Jamaica won gold and set the world record in the men’s 4×100 relay 52K TPM were broadcasted, and when Team USA defeated Spain in basketball over 41K Tweets per minute populated the digital medium.

Certain athletes and individual Twitter accounts produced noteworthy mentions throughout the games. Tom Daley, the British Diver and hometown favorite who secured the bronze medal in the 10 meter platform competition was mentioned over 1.4 million times during the Olympics. Although Ryan Lochte couldn’t beat Phelps in total medal count, he surpassed him by over 70,000 mentions during the games. The above chart is the cumulative mentions per Twitter account for the 2012 Olympics.

The London games even proved popular enough to generate a Presidential Tweet when @BarackObama congratulated Michael Phelps on his world record medal count. Yet despite the fierce competition amongst the world’s greatest athletes, the TPM gold medal went to none other than Great Britain’s Spice Girls. Their closing ceremony on August 12, 2012 inspired more than 116,000 Tweets per minute.

Will Twitter be able to evolve, and sustain its popularity through Rio? Stay tuned…

• 50% of all mobile phones in US are smartphones
• 88% of people check their email via mobile daily
• Projected by end of 2012 more people will read emails on mobile than on desktop
• 63% of Americans would close or delete email not optimized for mobile

Source: Return Path

Data is ever flowing and often times requires a creative solution to arrange and present the data in a visually appealing manner. The below is a list of blogs which are regularly updated by renowned statisticians, and some cases artists in their respective fields.

Eager Eyes

EagerEyes is Robert Kosara’s place to reflect on the world of information visualization and visual communication of data. The goal is to help digest things that are happening in the field and discuss developments that may be tangential or early, but that are likely to have an impact.

Junk Charts

Kaiser Fung finds the not-so-good and explains how to improve them. He is a  professional statistician with over a decade of experience applying statistical methods to marketing and advertising businesses. He holds an MBA from Harvard Business School, in addition to degrees from Princeton and Cambridge Universities. Fung is a statistician for Sirius XM Radio.

Visualising Data

Run by Andy Kirk, launched in February 2010. This site offers readers contemporary insights into the skills, techniques, resources, applications and best practices around this exciting subject.

Data Pointed

The home of artist and scientist Stephen Von Worley’s data visualization research; a journal of interesting information imagery and news from around the world; and a place where you can spend a few minutes, have a laugh or two, and discover something new.

Jim Vallandingham

A software developer with experience in database-driven website creation, and data analysis and visualization. Visuals in Geo, as well as Bioinformatics, occasionally publishing code to complement the graphs.

Statistical Graphics and more

Statistical Graphics, Data Visualization, Visual Analytics, Data Analysis, Data Mining, User Interfaces, a mashup of it all.

Off the bat, a sample of Facebook fans and Twitter followers established an intuitive fact that the NY Yankees and Boston Red Sox are the two most popular teams in baseball. Combined, they have won over 1/3 of World Series titles so it’s no surprise their reign influences the social media world. With a whopping 5.8MM Facebook fans for the Yankees, and 3.7MM Red Sox fans, they eclipse their nearest competitor, the Phillies by over 1.8MM.

For the same reason, the AL East is on the verge of 10 Million combined Facebook fans, significantly higher than any other Division as seen below.

** As of April 2012

World Series rings and trophies, however, do not correlate to social media engagement. For the 30 days leading up to the first pitch on April 4, using Facebook Graph API, I sampled the Fan Count per team as well as the ‘Talking About’ metric to measure each of the 30 teams’ social engagement rate. This presented insight into how some teams garner major league social media influence with their fan base. The social engagement index was calculated by measuring the 30-day trending average of fans per discussions, and thus the social media scorecard was compiled.

The National League accounts for 7 of the top 10 most socially engaged teams.  The Giants, Angels, and Dodgers, respectively are the top 3 most engaged teams in the 2012 pre-season, all three are also California based teams where social activity is higher than the national average. Three popular teams did not make the cut for social engagement, most surprising, the Red Sox fell to the bottom of the index. Unable to leverage their popularity, they average approximately 25% of the engagement rate of the #1 San Francisco Giants. This could be a result of fair weather fans, or their social fan base less engaged due to postseason woes in recent years.

The Detroit Tigers, the 10th most popular team by volume, had a positive shift to the 5th most socially engaged, and are most likely to be the next baseball team to reach the 1 Million fan mark this season. And while on the topic of standout stats, not charted, the Phillies have the highest Twitter following, surpassing the Yankees by over 100K.

The first official game of the 2012 season is technically Opening Night on April 4th, as the Cardinals take on the Miami Marlins with the opening of Florida’s new ballpark. This will be followed by six openers the next day April 5th. Marlins fans have a welcoming statistic on their side, as 5 of the last 6 ballpark christenings have been won by the home team.

Many exciting plays are sure to happen this year, so keep following your favorite team and I’ll keep an API on the scorecard!

The raw social scorecard data sorted by fans:

The digital takeaway:

• Over 3 Billion Google Searches/Day
• 1 in 4 people abandon a web page that takes more than 4 seconds to load
• 50% of mobile users abandon a page if it doesn’t load in 10 seconds