That’s all well and good, of course. But exactly how high is that? The Washington Post published an amazing graph the other day that tries to show just that.

The fact that they take you all the way to sea level first and then let you climb the page is so well thought out! Sometimes the counterintuitive turns out to be the most intuitive! Go there.

(via FlowingData)

The post Climbing Mount Everest appeared first on René C. Nielsen.

Anyway, yesterday we at UN Global Pulse announced a partnership with social data provider DataSift. Thanks to that partnership, we have been able to do a big project on the Post-2015 agenda together with the UN Millennium Campaign. The Post-2015 agenda is basically a process where the UN, civil society, governments of the world, and so on agree on a set of goals to aim for when the Millennium Development Goals end in 2015, so we went for rather broad stokes and global coverage. Global coverage pretty much means social data, so the partnership with DataSift really came in handy!

About a year ago, we created a microsite with crude numbers and three data visualisations.

A Spinning Globe

The spinning globe showing the top-20 countries talking on Twitter about 16 different development topics as they were defined in the massive survey MY World. We chose the spinning globe, as we had to create a visualisation that would be good for an exhibition taking place at UNICEF HQ. That meant that we wanted an interactive visualisation that could potentially update and be explored while passively looking at it (in the end we did manage to get a mouse connected, though). The main takeaway (or story!) we wanted to show was that these 16 very diverse development topics are talked about all over the globe. One problem is not just a problem for a particular continent or similar. Hence, a globe giving you a tour of the world seemed appropriate. And thanks to Mike Bostock, the skeleton had already been laid out.

A Trend Graph

We also included a trend graph showing how much the 16 topics are talked about monthly in 193 different countries. One of the first questions we get is usually “How about my country?”, so we definitely needed the ability to drill down a bit. We also added basic lists of most used words (unigrams only) when hovering over each data point, so the user can get a basic answer when asking the question, “What do people talk about when they talk about topic X?” We went for dimple on this one (mainly thanks to the fairly easy use of an interactive legend) and a simple MySQL database for word count lookups. The graph is slightly buggy, so we may go in another direction at some point if time allows.

A Map

Finally, we also included a map of the world where you can choose between the different topics. The trend graph makes it possible to look at one country at a time, while the map makes it possible to look at one topic at a time. We used CartoDB for that one.

New: A Bipartite Graph

Now, to celebrate that the partnership with DataSift was all wrapped up, I wrote a blog post yesterday on the Global Pulse blog, called “Sifting Through, and Making Sense of, Big Social Data” where I for example wrote about creating a taxonomy consisting of 25,000 keywords. Yes, 25,0000 keywords. Not your ordinary hashtag monitoring…

It also happened to be the day where we updated the microsite with data from June, seeing the full dataset get above 200 million tweets. To be precise, we have now categorised 206,784,678 tweets from 35,474,126 Twitter users into 16 development categories. To me that meant that it was time for a new data visualisation. We haven’t really looked much at global aggregates as it would be heavily biased towards users from especially the US, UK, Canada, and Australia, but as this was a quick celebratory visualisation, and not meant for policy decisions, I thought it was time. So this is what I created for the “Sifting Through, and Making Sense of, Big Social Data” blog post:

RAW includes Density Design’s old Fineo visualisation, and I really like those. And I especially like them when used with only two axes, making it into the more traditional bipartite graph. Each individual line is rather unimportant, while the overall most dominant colours do show you what’s most important. Most importantly, though, the sizes of the nodes are what you want to look for if you’re looking for rankings of either most talked about topics, or when we found the most tweets (Twitter seems to be just growing and growing). And, well, the nodes are already ranked, so there.

RAW makes it possible to resize the nodes, and I did play around with very wide nodes to make it apparent that that’s where a lot of the information is. But the readability (basically reading the differences in height of nodes) seemed to be just about the same whether they were 15 or 150 pixels wide. So I went with the better looking 15px.

I also considered making the tooltips (when hovering over the nodes) look better, but didn’t really think it was worth it (RAW doesn’t give you that option). I also considered re-colouring the text labels or giving them a background, but again, couldn’t be bothered. It’s simply good enough as it is!

The post Visualising Tweets About Global Issues appeared first on René C. Nielsen.

Not that I ever get to read the avalanche of tweets entering my stream, although I do go through it cursory somewhat daily. I even occasionally write something myself. But mainly, I analyse tweets. Millions of them.

Doing that can be quite daunting/discouraging at times, of course. There’s a lot of … erhmm … not too politically correct tweets out there. But generally, it’s just ridiculously interesting and useful. And a truckload of fun!

Having previously worked in (social media) communications for quite a few years, one of the things that surprise me repeatedly is the scattered use of campaign hashtags. Or, rather, that social media communications campaign people “create” hashtags for this, that and everything. Or “take ownership” of a conversation via hashtags. It makes it easier to track, sure, but next to nowhere else does the user have to “pay” for campaign complacency. It’s bad form.

Although I haven’t really researched the effectiveness of different uses of hashtags, I would be utterly surprised to see that the use of about a gazillion hashtags promoting the same issue does anyone any good. You only have 140 characters, right? Why would you think that people would try to fit in all of #eventX, #eventX2013 and #eventXsessionY in one tweet?

Anyway, that was just a stray thought while doing the below explorations. I know there are good reasons for any of the hashtags I’ve tracked around the UN General Assembly 2013, but it does seem they’re generally competing, not being used in conjunction. That said, #MDGs and #post2015 seem to be used a lot together, and that surely does make a lot of sense. And soon we’ll probably see #SDGs take over both.

The most interesting thing to me is probably that #MDGs and #post2015 are used as intended: They’re being used during the whole period as topic keywords, whereas the other hashtags analysed are event hashtags and only supposed to be used for a few days.

Tweets per Hashtag

Trends by hashtag around GA Week

Long-term trends (2009-2013)

This streamgraph shows the daily number of tweets using each of these 6 hashtags: #UNGA, #FutureWeWant, #MDGMomentum, #MDGs, #MDGSuccess, and #post2015. The total height of each stream on any day indicates the volume. Although the graph centers around the middle, there are no negative values.

[html collapse="true"]
<!DOCTYPE html>
<meta charset="utf-8">
<link href='http://fonts.googleapis.com/css?family=Open+Sans:400,300' rel='stylesheet' type='text/css'>

<style>

body {
  font: 300 10px "Open Sans", Arial, sans-serif;
  font-color: #96999b;
}

.axis path, .axis line {
  fill: none;
  stroke: #96999b;
  stroke-width: 1px;
  shape-rendering: crispEdges;
}

.tick {
  fill: #96999b;
}

button {
  position: absolute;
  right: 50px;
  top: 10px;
}

</style>

<body>
<script src="http://d3js.org/d3.v3.js"></script>

<div class="chart">
</div>

<script>

chart("sep2013.csv", "globalpulse");

var datearray = [];
var colorrange = [];

function chart(csvpath, color) {

if (color == "globalpulse") {
  colorrange = ["#00aeef", "#e1f4fd", "#96999b", "#e1d8ad", "#cf5c42", "#00447c", "#f4d5e3"];
}
else if (color == "blue") {
  colorrange = ["#045A8D", "#2B8CBE", "#74A9CF", "#A6BDDB", "#D0D1E6", "#F1EEF6"];
}
else if (color == "pink") {
  colorrange = ["#980043", "#DD1C77", "#DF65B0", "#C994C7", "#D4B9DA", "#F1EEF6"];
}
else if (color == "orange") {
  colorrange = ["#B30000", "#E34A33", "#FC8D59", "#FDBB84", "#FDD49E", "#FEF0D9"];
}
strokecolor = colorrange[0];

var format = d3.time.format("%m/%d/%y");

var margin = {top: 20, right: 60, bottom: 30, left: 30};
var width = document.body.clientWidth - margin.left - margin.right;
var height = 600 - margin.top - margin.bottom;

var tooltip = d3.select("body")
    .append("div")
    .attr("class", "remove")
    .style("position", "absolute")
    .style("z-index", "20")
    .style("visibility", "hidden")
    .style("top", "30px")
    .style("left", "30px");

var x = d3.time.scale()
    .range([0, width]);

var y = d3.scale.linear()
    .range([height-10, 0]);

var z = d3.scale.ordinal()
    .range(colorrange);

var xAxis = d3.svg.axis()
    .scale(x)
    .orient("bottom")
    .ticks(d3.time.days);

var yAxis = d3.svg.axis()
    .scale(y);

var yAxisr = d3.svg.axis()
    .scale(y);

var stack = d3.layout.stack()
    .offset("silhouette")
    .values(function(d) { return d.values; })
    .x(function(d) { return d.date; })
    .y(function(d) { return d.value; });

var nest = d3.nest()
    .key(function(d) { return d.key; });

var area = d3.svg.area()
    .interpolate("cardinal")
    .x(function(d) { return x(d.date); })
    .y0(function(d) { return y(d.y0); })
    .y1(function(d) { return y(d.y0 + d.y); });

var svg = d3.select(".chart").append("svg")
    .attr("width", width + margin.left + margin.right)
    .attr("height", height + margin.top + margin.bottom)
  .append("g")
    .attr("transform", "translate(" + margin.left + "," + margin.top + ")");

var graph = d3.csv(csvpath, function(data) {
  data.forEach(function(d) {
    d.date = format.parse(d.date);
    d.value = +d.value;
  });

  var layers = stack(nest.entries(data));

  x.domain(d3.extent(data, function(d) { return d.date; }));
  y.domain([0, d3.max(data, function(d) { return d.y0 + d.y; })]);

  svg.selectAll(".layer")
      .data(layers)
    .enter().append("path")
      .attr("class", "layer")
      .attr("d", function(d) { return area(d.values); })
      .style("fill", function(d, i) { return z(i); });

  svg.append("g")
      .attr("class", "x axis")
      .attr("transform", "translate(0," + height + ")")
      .call(xAxis);

  svg.append("g")
      .attr("class", "y axis")
      .attr("transform", "translate(" + width + ", 0)")
      .call(yAxis.orient("right"));

//  svg.append("g")
//      .attr("class", "y axis")
//      .call(yAxis.orient("left"));

  svg.selectAll(".layer")
    .attr("opacity", 1)
    .on("mouseover", function(d, i) {
      svg.selectAll(".layer").transition()
      .duration(250)
      .attr("opacity", function(d, j) {
        return j != i ? 0.6 : 1;
    })})

    .on("mousemove", function(d, i) {
      mousex = d3.mouse(this);
      mousex = mousex[0];
      var invertedx = x.invert(mousex);
      invertedx = invertedx.getMonth() + invertedx.getDate();
      var selected = (d.values);
      for (var k = 0; k < selected.length; k++) {
        datearray[k] = selected[k].date
        datearray[k] = datearray[k].getMonth() + datearray[k].getDate();
      }

      mousedate = datearray.indexOf(invertedx);
      pro = d.values[mousedate].value;

      d3.select(this)
      .classed("hover", true)
      .attr("stroke", strokecolor)
      .attr("stroke-width", ".5px"),
      tooltip.html( d.key + ": " + pro )
      	.style("visibility", "visible")
      	.style("color", "#FFF")
      	.style("font-size", "40px")
      	.style("background", function(d, i) { return z(i); });

    })
    .on("mouseout", function(d, i) {
     svg.selectAll(".layer")
      .transition()
      .duration(250)
      .attr("opacity", "1");
      d3.select(this)
      .classed("hover", false)
      .attr("stroke-width", "0px"), tooltip.html( "

" + d.key + "
" + pro + "

" ).style("visibility", "hidden");
  })

  var vertical = d3.select(".chart")
        .append("div")
        .attr("class", "remove")
        .style("position", "absolute")
        .style("z-index", "19")
        .style("width", "3px")
        .style("height", "600px")
        .style("top", "10px")
        .style("bottom", "30px")
        .style("left", "0px")
        .style("background", "#00aeef");

  d3.select(".chart")
      .on("mousemove", function(){
         mousex = d3.mouse(this);
         mousex = mousex[0] + 5;
         vertical.style("left", mousex + "px" )})
      .on("mouseover", function(){
         mousex = d3.mouse(this);
         mousex = mousex[0] + 5;
         vertical.style("left", mousex + "px")});
});
}
</script>
[/html]

Hover over the graph to see the number of tweets using that hashtag on that day.

Trends, September 2013

Trends, September 2013 (No #UNGA)

Trends, GA Week 2013

Trends, GA Week 2013

Trends, GA Week 2013 (No #UNGA)

Trends, GA Week 2013 (No #UNGA)

The post Hashtag Musings appeared first on René C. Nielsen.

Divided into 5 areas, the social power visualization is particularly interesting, and it is easy to see some important connections between coalitions.

Designed by Fathom, it’s no surprise that it is both useful and beautiful, so do give it a spin.

(via information aesthetics)

The post China Explained appeared first on René C. Nielsen.

The post Robert Kirkpatrick on Tweets and UN Global Pulse appeared first on René C. Nielsen.

Global Pulse is an innovation initiative launched by the Executive Office of the United Nations Secretary-General, in response to the need for more timely information to track and monitor the impacts of global and local socio-economic crises. The Global Pulse initiative is exploring how new, digital data sources and real-time analytics technologies can help policymakers understand human well-being and emerging vulnerabilities in real-time, in order to better protect populations from shocks.

Dybest set er det et innovationsinitiativ, der forsøger at gøre udviklingsbranchen bedre informeret, hurtigere, via analyser af Big Data (sociale medier, mobildata, satellitfotos, offentligt tilgængelige data etc.). Min kommunikationskollega Anoush Rima Tatevossian præsenterede os for nylig på et DataKind Meetup, så hvis du er interesseret i at få præsenteret initiativet og se nogle af de vanvittigt spændende projekter, Global Pulse har lavet de seneste par år, er 30 minutter af din tid givet godt ud i denne video (mit projekt bliver præsenteret 48:53-51:10):

Som det også fremgår af klippet ovenfor, er jeg i første omgang ansat til via sociale medier at undersøge hvad mennesker taler om i verden. Jeps, jeg skal forsøge at indfange “Den Globale Samtale” om emner, der er relevante for Post-2015-agendaen. Med lidt god gammeldaws underdrivelse vil jeg mene, at det er et forholdsvis bredt emne.

Post-2015

Mit projekt er en del af et initiativ, hvor FN vil inkludere verdens indbyggere i prioriteringen af, hvordan udviklingsarbejde skal foregå, når den enorme indsats for at nå Millennium Development Goals (på dansk: 2015-målene) om 995 dage slutter. For verden er ikke fri for undertrykkelse, sult og fejlernæring, dårlig uddannelse, dårlige sundhedssystemer osv. om 995 dage. Langtfra. Men hvilke problemer er vigtigst for post-2015-agendaen?

FN, de enkelte stater, NGO’er, akademia, private virksomheder (og en masse andre) er i fuld gang med at diskutere, hvad der – ud over bedre samarbejde – er vigtigst. Og det er de gode til. De er trods alt eksperter i netop det. Men de forsøger også at inkludere de mennesker, der rammes hårdest af globale kriser, fødevaremangel, oversvømmelser, tørke, diskimination, osv. Det er ikke nemt, men det afholder heldigvis ikke UN Millennium Campaign og deres partnere fra at forsøge. Der er især to processer i gang, der forsøger at inkludere de mennesker, udviklingsarbejdet forsøger at støtte.

World We Want

Den ene proces er World We Want. Det er en lang række af diskussioner, som vigtigst af alt indeholder nationale konsultationer, hvor de enkelte lande kan diskutere, hvad der er vigtigst netop i deres land:

The consultations will seek to ensure that the vision of the world we want to live in takes into account the perspectives from a broad base of civil society, marginalized groups, and others previously left out of discussions on development priorities.

De midlertidige resultater blev præsenteret for ganske nylig og data – og dokumenterne fra de forskellige lande, der har deltaget – er offentligt tilgængelige på data.worldwewant2015.org.

MY World

Den anden store, globale proces er en gigantisk spørgeskemaundersøgelse, MY World, hvor der på nuværende tidspunkt er knap 300.000 mennesker, der har fortalt, hvilke 6 emner ud af 16, de synes skal prioriteres efter 2015. Undersøgelsen er en blanding af representativ dør-til-dør surveying i nogle udvalgte udviklingslande og en online-udgave for alle vi andre. De nuværende tal er derfor ikke på nogen måde repræsentative for verdensbefolkningen. Faktisk er halvdelen af stemmerne lige nu fra Nigeria, hvor der allerede er lavet en gigantisk indsats for at få repræsentative svar.

Kun 426 danskere har på nuværende tidspunkt stemt, så der er plads til flere (det tager kun et par minutter!). Globalt er det på nuværende tidspunkt udannelse og sundhed, der løber med de fleste stemmer. De 426 danskere, der har stemt, har denne top-3:

1. A good education
2. An honest and responsive government
3. Access to clean water and sanitation

Det er nøjagtig samme top-3 som alle de rige lande samlet (eller rettere, lande med meget høj HDI). Landende med lavest HDI har på nuværende tidspunkt stemt disse emner ind i deres top-3:

1. Better healthcare
2. A good education
3. An honest and responsive government

På 4. pladsen har disse lande “Better job opportunities”, som kun ligger nummer 12. på høj-HDI-listen. Alt sammen meget spændende og jeg glæder mig allerede til at lege med de endelige tal (selv om det ikke er mit job…).

Den globale samtale

Mit job er at tage de 16 emner fra spørgeskemaundersøgelsen og se efter, hvad folk taler mest om på sociale medier. I første omgang kigger vi kun på Twitter og kun på engelsk. Hvilket betyder, at jeg på nuværende tidspunkt jonglerer med omkring 25 mio. tweets, men jeg regner med at ramme 50 mio. før jeg har mit endelige datasæt. Det er ret mange.

Jeg tør næsten ikke tænke på, hvor meget computerpower, vi skal bruge, hvis vi får mulighed for at udvide med blogs, debatfora, Google+, Facebook, Sina Weibo, osv. … og med de andre officielle FN-sprog (fransk, spansk, kinesisk, russisk, arabisk), portugisisk og indonesisk. Anyway, det er heldigvis vores data-ingeniørers hovedpine…

Jeg har indtil videre ingen resultater, jeg kan dele, men forvent at finde dem på min datablog på et tidspunkt, når der er noget officielt. Indtil videre må du tage mit ord for, at jeg har det sejeste job i verden. For jeg har det sejeste job i verden.

The post Nyt job appeared first on René C. Nielsen.

Anyway, while researching for a good taxonomy, an insight- and helpful colleague re-reminded me of Infomous, and this blog post is basically just to let the world know that it has been a good encounter. It’s very easy to work with, and although it’s a bit noisy and rough around the edges, the drill-down capabilities have really helped inform the taxonomy I’m working on.

Here’s one visualization based on a very simple search on the different variations of “Post-2015” on Google News, Google Blog Search, and Twitter:

From the Infomous FAQ:

• The size of each word reflects the frequency with which it appears in the source.
• If you click on a word, a drop down list appears with links to articles that are related to the specific word. The drop-down will also appear if your mouse lingers over a word. By clicking on a link in the list, you will navigate to that specific article.
• Topics become linked when they are mentioned in the same context or discussed together multiple times. Related terms and concepts are linked together with lines so you can grasp the context of any relevant topic.
• The words in the Infomous cloud are organized in groups of related words. This provides you with a quick glimpse of which topics belong together in conceptual clusters.

The post Taxonomy Exploration with Infomous appeared first on René C. Nielsen.

Aggregator

I have only come across one aggregator of university level courses, and that is Class Central. It makes it very easy to find some truly inspiring courses from the big boys in Massive Open Online Courses (MOOCs):

• Coursera
• Udacity
• edX
• Canvas Network

Coursera have so many great courses that I can’t exactly say that I have checked them all. I did take part in the Computing for Data Analysis course last time around, and that definitely brushed up my R skills. I will hopefully find the time to also participate in Data Analysis.

Data Visualization

Alberto Cairo’s course “Introduction to Infographics and Data Visualization” from the Knight Center for Journalism in the Americas is absolutely top class.

Katy Börner at Indiana University will run a Information Visualization MOOC very soon (starts 22 January). I’ve signed up in the hope that I will find the time necessary. It seems extremely interesting.

Andy Kirk also travels the world with his one-day introductory training courses, so although it’s not an online course, it is quite global anyway. I attended his course in New York back in May, and it’s definitely worth the buck.

Visualization by Code

c<>de school is a new player on the market, and they already have a course where you can Try R. Which is free. And quite amazing for the effort. It’ll only take a couple of hours of your time. They have a range of other courses, some of which would be relevant to people doing information graphics and data visualizations.

Codecademy is not so much targeted at data analysis or visualization, but they do have introductory courses in JavaScript and Python, which is relevant for just about every person that would like to one day be able to make her own interactive visualizations.

For D3.js I believe Scott Murray’s D3.js tutorials and the tutorial he, Jeff Heer and Jérôme Cukier made for VisWeek 2012 are good places to start. I haven’t yet given this tutorial from Stanford much of a look, but as the only required reading is a paper by Stanford’s own Michael Bostock, Vadim Ogievetsky, Jeffrey Heer who are also the authors of D3, I think it should be a requirement for anyone digging into D3. The most comprehensive collection of tutorials seems (unsurprisingly) to be the D3 wiki page, but Dashing D3.js also seems like a great source. Still looking very much forward to find the time to go through all these D3 materials.

I have not yet looked into Processing/Processing.js at all, but they have a learning pages on the websites, so that would probably be the best place to start.

Visualization Without Code

Tableau have their own great training section. I don’t think you need to go anywhere else. I haven’t looked much into it yet, though. I will as soon as the Tableau Mac version is out.

There must be a few Google Fusion Tables tutorials out there by now. Not sure if there is any centralized or at least structured learning anywhere, though. The best place could easily be Google Fusion Tables help section.

And well, there is of course Excel, which is pretty great for more visualization purposes than than it usually gets props for. If you tweak every single graph a bit. For non-intuitive Excel graphs, I find The Excelchart’s tutorials pretty great.

Illustrator is probably the go-to tool for graphics, and all I’ve tried so far is the tutorials Alberto Cairo made for his course mentioned above, although I’m sure there are a lot of great tutorials relevant at Lynda.

Big Data

I only know Big Data University, and I have not yet followed any of the courses there, but as you would expect, you will be introduced to Hadoop.

I probably missed something necessary. If so, let me know.

featured image by Maxbo10

The post Going to Data Visualization School appeared first on René C. Nielsen.

Designed by Lunametrics

The post Billedstørrelser på sociale medier appeared first on René C. Nielsen.

Men noget, der er tankevækkende for mig, er, at de bedste kilder lige nu (igen) er nyhedsbreve. Jeg er især glad for:

• DAWNS Digest (Development and Aid World News Service Digest), som koster $2,99 om måneden. Det er hver en cent værd.
• UN Wire – United Nations Foundation laver et bragende godt gratis nyhedsbrev med FN-nyheder
• U-landsnyt.dk – nyheder på dansk. I hvert fald for det meste.
• Min chef sender dagligt en email ud til vi kommunikationsansatte, som minder meget om DAWNS Digest i format og omfang. Det er ikke til offentlig deling, men når DAWNS gør det samme, kan du jo også være lige meget, ikke?

Men altså … nyhedsbreve!

Det viste sig, at revolutionen var reaktionær.

The post Udviklingsnyheder appeared first on René C. Nielsen.