Tomorrow I’m getting married, so I probably won’t be posting for a while. Not, of course, that I’ve been posting a lot recently.

Without wanting to get sentimental (it’s not that kind of a blog, and I’m not that kind of a man) I can say that not only did I never believe that I’d find someone like Krista, but that now I have found her, I still can’t really believe it.

I’m saving the rest of what I have to say for my speech tomorrow evening. There are all sorts of little surprises planned for the day, but one of the biggest surprises right now is “what Mat will be saying in his speech” because I’ve yet to write it. Tim has told me “be nice to everyone and try not to sound like Hugh Grant.”

Anyway, the hashtag for my wedding will be #wediaczar. Given that the audience is startlingly low on digital media bods, it’s not like I think it’s going to trend or anything, but it seemed like too good a hashtag to waste.

Geek alert: if the title of this post isn’t a dead giveaway I should tell you — unless you’re interested in APIs and badly-put-together bits of code — this probably isn’t for you.

I’ve recently found myself using a service provided by Damon Clinkscale called DoesFollow. All it does is answer the simple question “does twitter user A follow twitter user B?” Apart from a frill which lets you reverse the order of your question (“does twitter user B follow twitter user A?”) that’s all it does. You can even interrogate it from the address bar like this: http://doesfollow.com/barackobama/mediaczar

While I was thinking about how useful a service this is, I was suddenly struck by a moment of clarity. A lot of the research I’ve been doing could be simplified by something like this.

Quite often I want to find out whether MPs or congressmen or PR people follow each other on Twitter.

The way that I’ve been doing this until now is

1. make a list of the people who I’m interested in researching
2. for each person on that list, grab the list of all the Twitter people whom they follow
3. process the list so that only relationships between the people on the list show up

If all I’m doing is checking to see who follows whom, then this is a horribly wasteful way of doing things. The Twitter API limits the number of calls one can make on it — so this wastage leads to things taking much longer.

If only I could cycle all the names I want to check through something like DoesFollow!

Well – it turns out that I can. And in theory it’s not much harder than using DoesFollow. The Twitter API (which is what DoesFollow uses, after all) has a method called friendship/exists. All we have to do is send Twitter the following request:

http://twitter.com/friendships/exists.xml?user_a=barackobama&user_b=mediaczar

and it will come back with the answer:

<friends>true</friends>
or
<friends>false</friends>

Kludge-y perl code

(This fabulous picture courtesy of There, I Fixed It)

So I tried to do this using Yahoo! Pipes, but there are too many nested loops. You need to do something like this:


get list of names

}


There’s no easy way to get Pipes to do this, as far as I can see (I’ll keep trying, but if someone else can help, I’d be v. grateful.)

So I’ve pulled together a badly-written perl script to do the work for me.

The script

[code lang="perl"]
#!/usr/bin/perl
# checks the Twitter API to find the friendships between a list of usernames
# this should really use the NEW API call that would let us halve the number
# of API calls
# author: Mat Morrison
# date: Friday July 10, 2009
use warnings;
use LWP::Simple;
# set up variables
# we're just using a whitespace delimited list for the moment
my @usernames = qw(kerrymg mediaczar timhoang titusbicknell);
# let's build the matrix with a hash of hashes...
# to begin with, we'll include diagonal values -
# that is -- we'll check to see whether @mediaczar follows @mediaczar
foreach $user_a(@usernames) {
foreach $user_b(@usernames) {
# we should put in a conditional clause that will check for the diagonal values
# and not bother checking whether someone is a friend of themselves...
$url = 'http://twitter.com/friendships/exists.xml?user_a='
.$user_a
.'&user_b='
.$user_b;
# get XML file from Twitter -- it's an astonishingly simple XML file that reads
# true
# or
# false
# so we don't need to do much with it...
$follows = get $url;
die 'Can\'t get $url' unless defined $follows;
# strip the tags - I'm using a generic "HTML stripping" regex
$follows =~ s/<(.|\n)+?>//g;
# we should probably convert "true" values to 1 and "false" values to zero or blank
# now let's push data into the matrix
$matrix{$user_a}{$user_b} = $follows
}
}
# spit out the data as a tab-delimited table
# print the top line first
for $user_b ( keys %matrix ) {
print "\t$user_b";
}
# now print the values
# they're all neatly arranged in the matrix so we
# can just print them out sequentially
for $user_a ( keys %matrix ) {
print "\n$source";
for $follows ( keys %{ $matrix{$user_a} } ) {
print "\t$matrix{$user_a}{$follows} ";
}
}
print "\n";
[/code]

Where next?

Most of my thinking is included above in the code comments. An obvious mistake I’m making is checking to see whether, say, @mediaczar follows @mediaczar. That wastes n API calls per search. But a more serious mistake is not to be using the new friendships/show method. Because it tells you whether user A follows user B and whether user B follows user A at the same time, it would save me lots of API calls. How many lots? Well take a look at this.

This is what I’m doing at the moment — checking each and every cell in the matrix:

This is what I’d be doing if I removed the diagonals:

And this is what I’d be doing if I used the newer API call:

I had to look up the formula for working this out without colouring in little boxes. With a little tweaking (to prevent the diagonals from creeping back in), here it is:

((n-1)^2)+n-1)/2

So — for a list of congress people (159 on twitter as at Tuesday July 14, 2009) that’d be ((156-1)^2-1+156)/2 = 12,090 API calls. Which is still a lot and will require some careful throttling, but (literally) not half as many as the 156^2 = 24,336 API calls that I’d need to run it as the script currently stands.

So – back to the drawing board for a while. I really can’t work out a programmatic way of doing this. Hmph.

• Chart: Who Participates And What People Are Doing Online
  An old (2007) infographic from BusinessWeek that I just stumbled across. The (incorrect) implication is that one falls into a single category.
  (tags: audience segmentation)

• The Newspaper Licensing Agency – charging for weblinks!?
  Francis Ingham — DG of the PRCA — weighs in on the story in the comments section of this post.
  (tags: newspaper legal idiots)
• Wrong-Headed Newspaper Biz Tries Again To Charge For Links
  paidContent:UK has a nice straightforward article on the NLA's mindless attempt to charge for free content.
  (tags: legal newspaper idiots)
• Newspaper Licensing Agency – What Will it Cost?
  OMFG ROFL — The Newspaper Licensing Agency wants to charge us for sending URLs to clients (see the linked PDF — <a href="http://www.nla.co.uk/pdf/Guide%20to%20Electronic%20Distribution%20April%202009b.pdf">A Guide to Electronic Distribution</a>.) Can we take this seriously?
  (tags: newspaper idiots)
• Johnson & Johnson's "Real Moms" playlist
  J&J <a href="http://jnjbtw.com/2009/05/calling-mommy-vloggers/">called upon Mommy vloggers</a> to submit videos on topics "relevant to them" — a nice idea, but one, as you'll see, that generated little interest either in the way of video submissions (8 videos submitted after 6 months as at July 9, 2009) or views (fewer than 15K.) Needs more investment, or more thought.
  (tags: video mommyblogging pharma healthcare)
• JNJ BTW
  This looks like a best-practice example for pharma company blogging. There's little evidence for consumer interest: most of the inbound links seem to be talking about this as an example of "corporate blogs" and "marketing" rather than as a source of what we might broadly term "medical information." But there's some great evidence of <a href="http://www.postrank.com/feed/38238b9c95b44e75f747871bccf68c70">outreach and linkbait</a> — clearly this is a platform for activity on the broader web, rather than a simple content site
  (tags: blog blogs corporate healthcare health marketing pharma blogging)

• Instapaper, Del.icio.us, Yahoo! Pipes and being Slack.
  A masterful Delicious/Pipes/Instapaper mashup by Daniel Catt — with clever use of Yahoo! Alerts (a service of which I was previously unaware, but which seems to outperform Google Alerts in significant ways.) Actually, I was just looking for a way to get around a URL Encode problem I was having with Pipes when I stumbled across this (Catt's "sloppy" solution works for me.) But now I want to implement this whole hack for myself.
  (tags: pipes yahoo! delicious hack instapaper iphone rss)

A couple of weeks ago, I asked a bunch of people to send me their OPML files (for those of you who aren’t aware, an OPML file is what tells your RSS reader what feeds you’ve subscribed to — it can act as a way of moving your subscriptions between readers.) Some of the more trusting among them agreed, and that gave me the raw material for the first bit of my experiment.

Some red herrings

Along the way I uncovered a couple of things that were interesting but not (entirely) relevant to the experiment.

1. Some people are cagey about sharing their list of feeds: whether they consider it intellectual property, or whether they think that it may be too revealing, I don’t know.
2. Lots of people said things like “oh — my RSS reader? Haven’t looked at that in a while. I get all my news off Twitter these days.”

So what’s the experiment about?

You’ll probably know that I’m interested in networks of people and how information flows through those networks. I’m also interested in things like influence and whether and how we can identify and track its effects.

I’d heard about a word of mouth marketing campaign which was set up along the following lines: a sample set of pupils at a given school were asked “who’s the coolest kid in the school?” Clearly some names came up more often than others, whereupon the researchers went to those kids and asked them the same thing. By the end of the process, they had a good idea of who might be the key influencers.

This seemed like a good sort of thing to be doing. It was a simple idea, and apparently easy to execute. Everything else I was working on (the citation analysis and the network analysis in particular) seemed to be complementary.

A little history

So last year we did a version of this experiment for a client. We emailed and phoned a whole load of journalists and analysts whose beats covered our client’s interests and asked them:

Who do you read on a regular basis?

Who (other than you) should we be talking to?

If you were looking for information, where would you start?

Then we approached the people that they’d recommended and asked them the same questions. The purpose (as I’m sure you’ll have guessed) was to create a network map of who was whose go-to guys and girls. We’d take the data that we collected, push it through the usual processes and hey presto! we’d know who was really influential.

But what we actually found was that the journalists and analysts we asked seemed not to have any specialist sources. The people whom they cited were (in no particular order) their colleagues, the companies they covered (and their public relations representatives and agencies). Oh, and Google. Google was cited by everyone.

There were two obvious explanations for this:

1. It’s the truth: they really didn’t have any better sources. We’d all read about the collapse of journalistic standards; perhaps we were encountering it at first hand?
2. The privacy and intellectual property argument: maybe the journalists and analysts were constitutionally loathe to reveal their sources?

All my training and experience, however, points to the following reason:

3. Poor questionnaire design led to an inability to think (when put on the spot) of names and sources whom they believed were their influencers.

This is less unlikely than it may seem. If I were to ask you who your big influencers were, would you be able to answer?

All in all, it was a deeply unsatisfying exercise. We had envisioned what wild success would look like and this wasn’t even close.

Where we are today

I thought that it might be easier to ask people for their OPML files than it had been to ask them who were their influences. This is the equivalent of — say — asking to see a musician’s CD collection, instead of asking them about their musical influences. It’s not necessarily more accurate, but it might help expose a different picture.

Seven of my friends, colleagues and acquaintances sent me their OPML files, and that was enough to get started.

Early results

Between them, the first seven people subscribed to just over 1.5K RSS feeds, which gave me a lot of data to process. Here’s a picture of the network before I started processing the data.

You can see (I think) pretty clearly that there are several blogs (or RSS feeds — I’m using them interchangeably here) in the middle of the map that are linked to by several of the respondents. And there are some (around the edges) that are linked to by a couple of respondents. And there are lots that are linked to by only one respondent.

At this stage we’re interested in “indegree” — or “the number of OPML files in which we find RSS feed X.” Clearly the great majority (just over 1.3K) of the RSS feeds are single hits — that is, they appear in only one respondent’s OPML file. But that still leaves 180 RSS feeds with multiple hits:

Those 180 RSS feeds with multiple hits left us with a nice (and fairly predictable) distribution that looks like this:

Clearly the first-generation results aren’t very meaningful or accurate. Most often cited were author Seth Godin, ex-blogger-turned-lifestreamer Steve Rubel, and PR blogger Stephen Davies who were each read by five of the seven respondents. But more than half the respondents read my blog. Despite what I’d like to think, this is clearly an artefact thrown up by the sampling frame (my friends, colleagues and acquaintances).

Where next?

I’m hoping that we can extend this out a generation — and then keep iterating. I’m going to approach everyone with an indegree of 3 or above, and see how many of them will send me their OPML files.

I don’t hold out much hope of getting OPML files from people like Messrs Godin and Rubel, but if I don’t ask, I’ll never know, will I?

The current approach to WOM is to try to stimulate positive WOM while addressing or countering negative WOM. A sort of “accentuate the positive, eliminate the negative and don’t mess with Mr In-Between” strategy.

But what if we could do it a different way?

This idea stems from a conversation I had back in February with Martin Kelly and Andy Cocker of Infectious Media. Since that time I’ve chatted it through a couple of times with various interesting people. It’s not properly thought through yet, but following a chat a couple of weeks ago with Ketchum London’s new Head of Digital, the excellent Fernando Rizo, I’ve decided to put the idea out into the public domain to gauge what (if any) interest there is and whether I should continue to work on it.

“Word of Mouth” is hard to do well

I’ve read lots of word of mouth marketing case studies (there’s a great list over at WOMMA) and it strikes me that WOM is hard to do well for a few reasons. I don’t want to go into these in too much detail, but here are a couple of the structural issues:

1. Unless I’m a journalist, an A-list blogger or media personality or have some kind of platform, I probably have a very low reach.

   Despite everything pointing towards personal contact being the best impetus for positive word of mouth, most word of mouth campaigns compensate for my low reach by trying to get me to self-service my relationship with the brand and the campaign.

2. “Viral” distribution just doesn’t work the way most people seem to think it does; and this is particularly true when it comes to WOM.

   While I’m quite likely to tell stories about my personal experience of a brand and fairly likely to tell stories that involve a mutual friend, I’m much less likely to tell stories about other friends’ experience, and not likely at all to tell stories about friends-of-friends.

   Furthermore because of the ‘clumpiness’ of most people’s social graphs, geometric progression (the “I tell two people and they each tell two people and so on” effect) just doesn’t happen.

Homophily

One of the many reasons that WOM works is a thing called homophily — which roughly translates to “birds of a feather flock together”, or “you can tell a man by the company he keeps.”

I’ve written about examples of this before: for example, my analyses of twittering US Congresspersons and Westminster MPs which showed that one can predict with some reasonable degree of accuracy the political colouration of any given twitter account based on their mutual friends and follows (if you want to know more about the methodology, it’s worth reading Robert Hanneman’s chapter on cliques and subgroups.)

But there’s another side to the homophily coin; the social pressure to conform to the group’s norms.

Why I bought an iPhone

Let’s take a concrete example of this: it often seems to me that everyone I know has an iPhone. I made a conscious decision a few years ago not to buy an iPhone, but I’ve finally succumbed to the pressure.

While it’s true that several people have take time out of their busy schedules to tell me exactly how good the iPhone really is, I’m far more affected by what I perceive as the omnipresence of the iPhone. Everyone, it sometimes seems, has one except me.

Of course, on those occasions when I head down to visit my family in rural Hampshire, I’m reminded of the obvious truth: most people don’t have an iPhone. But that’s not the way the world appears to me.

How about turning it on its head?

What if we could identify people who are under social pressure to buy our products — who are being influenced by what their friends are doing and saying?

Let’s say for example that Apple and O2 (iPhone’s carrier partner in the UK) could work out that a significantly higher proportion than average of my Facebook friends access the service using the iPhone client, and could target me with special offers and rates to push me over the edge.

Instead of looking for WOM influencers, why don’t we look for areas of high potential — and target those people who are likely to be receiving lots of WOM stimuli?

• The PR lessons from Neal's Yard Remedies public debate U-turn
  Neals Yard Remedies agreed to join a public online debate, then wimped out, leaving little more than a reputation disaster. Here the Guardian's Adam Vaughan (together with Max Clifford and Mark Borkowski) offers a critique.
  (tags: guardian pr pneo001 fail)

• Yahoo! Pipes Auto-retweet recipe for Twitter
  Ryan Price documents his Auto-retweet pipe. So many interesting possibilities here.
  (tags: Yahoo twitter pipes tools)

Here’s an interesting ethical question: is it OK to ask employees to share company and client news through their personal social networks?

Here’s a hypothetical example. An agency has just launched a new ad campaign and posted the TV spot on YouTube. Is it OK to send an all-hands email asking people to share the link on Twitter and Facebook?

Let’s take it a little further. Is it OK to ask them to sign into YouTube using their personal accounts, and rate the video? It seems harmless enough, doesn’t it? You’re not telling them how they should rate it, after all.

But what if you asked them to leave comments? Any normal agency or client side social media policy will tell them that they have to disclose their relationship with the makers of the video. And you wouldn’t really want a whole bunch of comments that start “Hi, I work for the agency that made this ad and I think it’s really great,” would you? What makes the two things different?

Digging for victory

OK. Another hypothetical example. The social news site Digg is a huge source of traffic for most news websites. The Telegraph, for example, gets around 75K visits a day from the service. That’s an awful lot of traffic.

The thing is, with Digg, you really want to make the front page if you want the big traffic. Around 10K stories are submitted by users to Digg every day, and only 150 or so make it to the front page. So we’d need to be smart.

Digg is less open to being gamed than it used to be, but let’s say that a smart agency could still deconstruct Digg’s algorithm sufficiently that it can use its network of staff to improve the chances of a story (or review) making it to the front page that shows their client in a positive light.

All they’d need to do (say) is send an all-hands email that mobilised your staff to digg a particular story at the right time of day. Would this be legitimate? Digg clearly thinks not, but are they the best judge?

Ballot stuffing

Let’s say a client gets nominated for a Webby or one of the other user-voted awards out there. It’s common and acceptable practice for web services to use any means at their disposal to beg for votes.

So is it OK to send an all-hands email asking your staff to register and vote? Is it OK to ask them to use their Facebook and Twitter accounts to ask their friends and followers to vote?

Here’s what I think

I think that this is an ethical minefield, but I’ve got a couple of clear points of view that are up for discussion. First off, and from a purely business perspective:

If it’s valuable then clients should pay us to do it. If it’s not valuable we shouldn’t do it.

Ignore the ethics. When was the last time you sent an email that said, “Please share this with your friends and use billing code xxx.xx when you record it on your time sheet”? Of course, it only takes a few seconds to relay a message (fewer if you simply copy and paste the message from the all-hands email to your social media presences). What the hell – it’s all just part of the team spirit, isn’t it?

But if this service is of any value at all to the client, then we’ll see the demands on our time beginning to increase. Soon we’ll find ourselves doing several a day. Larger agencies with more staff will offer a more valuable service to clients than boutique agencies (“We have a thousand trained staff with Twitter and Facebook accounts primed to promote your campaign!”)

We could even work out some kind of ratcheted scale that said that people with more Twitter followers could bill at a higher rate; or we might start looking at unduplicated reach.

This might seem fanciful, but stories like the uSocial’s offer to game Digg’s front page for around $200 tell us that the thriving black hat market for this sort of thing is getting a little greyer.

If clients value this, then they should pay us. If we give it away as a “value added” service, then are we sure that we’ve communicated this properly so that the client understands the value we added?

How can we be certain we aren’t just undermining our digital value proposition?

Now, here’s the tricky bit where I bring ethics back into it. If we’re paying our staff to relay messages to their networks on behalf of our clients, what makes this different from spam?

Employees personal networks are their personal property

I’m one of those people who believes that the boundaries are blurring between our personal, public and professional lives. My colleague Chris Nee has posted about the presentation of self in social media, expressing a lot of how I feel rather better than I could myself, although that hasn’t stopped me from covering this elsewhere — notably in a discussion about our social media policy where I said:

I think that a good PR person is someone who manages their relationships well; who can tread the fine line between doing good work for their clients without abusing or exploiting their relationships. Who recognizes the value of their personal network, and their personal brand.

Do we have a right to ask our employees to use their personal networks on behalf of our business? I think the answer is – of course we do. We pay experienced staff more partly because of the social capital that they’ve managed to accrue in their address books. When you leave your job to go to a new agency, you’ll take your contacts with you along with all the shared experience, the favours you’ve done, and the favours you owe.

So if part of the reason we’re paying them more is because of that network, then we clearly value their network.

So why on earth would we encourage our staff to spam their personal networks?

I trust my peers. I don’t trust spam monkeys

Part of the reason for the success of social networking tools like Facebook and Twitter is that I can choose who I follow and who I don’t. I can restrict conversations to a bunch of people I trust and respect. Sure, I’ve got lots of Twitter followers with names like “Sophie1982″ and “EdelmanHR” but that just means that they hear my inconsequential blatherings, not that I have to hear theirs!

And I follow a lot of my colleagues’ Twitter streams. I’m pleased to say that these are — on the whole — full of meaty chunky content and devoid of spam.

But if we increase the spam content, what will happen? Here’s what (in the absence of evidence) I believe: their follow rates, retweet rates and mention rates will all begin to drop off. From your experience, what do you think?

Where does this leave us?

I’ve put a somewhat one-sided argument here. I’m sure I’m open to all sorts of rebuttals like, “surely, if the content is interesting/useful/entertaining then our staff will only be adding value to their networks?” and “but surely it’s up to them whether they want to pass it on to their friends?” I’ll wait for these to roll in before I start trying to address them.

But here’s what I think. I think that we’re trying to teach our colleagues to learn from managing their personal communities in order better to manage our clients’ communities. Anything that teaches them to prostitute their networks is a retrograde move.

Am I wrong?