I was reading the description of a new data visualization contest coming out today, the Nielsen Data Visualization Contest, and an apparently insignificant sentence caught my attention: “The challenge is to make data tell a story, conveying what’s most important effectively and efficiently.”

There is a lot of attention lately around using visualization to “tell a story” and I can understand why: visualization, when designed properly, has a tremendous effect on people. Not only it has the power to convey a clear message and to make complex concepts very easy to grasp, but it also has the power to persuade. I guess the main reason being that when a statement is backed up by data then people believe it is true(er).

I have nothing against using visualization to tell stories, to the contrary I am fascinated by this use of visualization and I think it’s very relevant. For instance, raising awareness about important facts or democratizing access to complex information are very noble intents of visual story telling, and I fully support them.

But, I don’t know, call me old-style, conservative, bigot: I am concerned by an excessive focus on story telling. It’s an itch I cannot scratch. And because I cannot express it in a closed form the only thing I can do is to make a list of concerns I have (hoping your comments will make it easier to dispel the fog).

There’s no story telling without data exploration. Creating a story with visualization doesn’t mean there is not role for data exploration in visualization in its making. People looking at the final product might think the power of visualization is exclusively in the effective presentation of the facts. But what people don’t see is the amount of exploratory work behind every story. I know as a matter of fact that many great visualization designers start with a thorough visual exploration of the data at hand using standard tools like Tableau or R. Without this preliminary phase it’s very hard to tell a compelling story and it is also very hard to come up with an enlightening visualization.

It’s the data that makes the story not the visualization. I always laugh a bit when people complain about David McCandless’ work. They say that their visualizations are not optimal and that he makes many “mistakes”. In a way I agree but why does he have such a big success then? I think the reason rests in his ability to select amazing stories to tell. The story is hidden in the data. Well, not even in the data, I guess everything starts in his mind, the rest just follows naturally. So, if we are passionate about visualization and dare about its proper use I believe story telling is (maybe) not the most challenging area to test it.

Many people need visualization to build our future not to tell a story. While I cannot resist a catchy well-crafted data visualization that tells a compelling story, I also know from my experience how desperately professionals of all kinds need visualization to just do their work best. I am talking about doctors, engineers, biologists, policy makers, etc. Part of our life, or of our future generations, might depend on them and we have the opportunity to help them help us. Don’t you think this use of visualization is a bit under represented on the web when compared to the whole set of story telling visualizations out there? For instance, why don’t we have contests to help these people with their data and have plenty of those asking to vaguely find a story to tell in this or that data set?

A story is not THE truth. I have no evidence for that but my feeling is that visualization can be used to more easily persuade people. By the mere fact of being built on top of data people might think it is truer than other kind of stories. Again, you can see that in McCandless’ work. Many of his pieces are evidently conceived to be provocative and touch hot topics. But I bet that for every provocative visualization out there there is the possibility to build a counter argument with another one. I might be proven wrong on that but I haven’t seen any evidence on the contrary so far.

Not all stories are worth telling. Since the power of a story resides in the data, it is not always possible to tell a compelling story. Regardless the beauty or inventiveness of your visualization if the data is dull you might not get a compelling story. And I have experienced it so many times that I am almost inclined to say that this is pretty much the standard for any given data set. You can see it in the recent Information is Beautiful Award: there are many cool and pretty entries, some that I really like from the design point of view, but is there anything really interesting there to see? Do we leave the stage enriched by new knowledge?

As promised yesterday here is the answer I received from Visualizing after sending them a draft of my post. Given their answer and the whole bunch of controversial but constructive comments I received (check them out, they are full of insights) I am really glad to have started this. I have the feeling this can in a way help all of us, regardless our opinions, make the whole field at least a tiny bit better.

—

Enrico,

First, thanks very much for taking the time to share your feedback and for your thoughtful suggestions. We’re all committed to advancing the field of data visualization and healthy debate towards that goal is always useful.

The aim of the Visualizing Marathon program, which we started in 2010, is to encourage design students around the world to take up data visualization and generally to use design to help improve our collective understanding of complex world issues. The structure and format of the event is constantly evolving in support of this aim based on what is working and not working (we collect surveys from the students, for example) and so we’re most grateful for any and all feedback!

To your specific points:

1. Judging: All Marathons (and challenges) are judged based on three criteria, which we’ve previously shared on Visualizing:

• Understanding: How effectively does the visualization communicate? How well does it help you make sense of this issue? (out of 10 points – we agree with you this is most important and that’s why it gets the most weight)
• Originality: Are the approach and design innovative? (out of 5 points)
• Style: Is the visualization aesthetically compelling? (out of 5 points)

Our global jury selected 1 Winner and 2 Honorable Mentions in each of the 5 cities. These are the top 15 projects based on these metrics.

Importantly, however, the Grand Prize was selected by us out of the top 15 based on a different metric: how well does the project help illuminate new insights to the complex problem the students were given (in this case, sustainable development). Because data visualization is not only a tool for communication but also a tool for exploration, we sought to highlight and amplify the latter with this particular prize. It’s why it comes with a $10,000 grant to support further research and education. We felt the winning project best delivered on this metric (the approach and analysis detailed in their accompanying essay is particularly noteworthy). And as we noted in the prize announcement, we hope very much that the students use the additional time and resources they have been granted to take the visualization further (including putting their 3D shape to work as outlined, and perhaps evolving a simpler overall design). Also, we’re sure they would enjoy hearing suggestions directly.

We have been exploring how we might incorporate a “People’s Choice” aspect into the program, though there are some potential complications with this format that we are trying to be mindful of.

2. Time: There is no question that time (usually) improves quality – and our Visualizing Challenges, for example, typically run 4-6 weeks based on that logic. With our Visualizing Events, like Visualizing Europe and the Visualizing Marathons, we want to create the space and opportunity for people to come together in a shared and collaborative environment where they can meet, learn from one another, and develop new partnerships/relationships. We hope that after each event, the conversation continues in a way that can push forward the field of data visualization. We know from direct feedback from students and their professors that there is a tremendous didactic, creative, and inspirational value in working together with 2-3 of your friends in a common space with other students for 24 hours towards a common goal (we are also mindful that there is a real limitation to the amount of time students can commit to an extra-curricular activity). As you rightly pointed out, there are high quality projects among the entries, so clearly it is possible to produce something of quality in the allotted time. We also believe that overall quality from students will improve year over year as the professional field and its accompanying science mature and codify what works. That said, we are in fact experimenting with time this year to help improve overall quality and welcome any suggestions.

3. Training: We agree that training is important. In the spirit of openness, we allow students of all levels and disciplines to participate in the Marathons and learn by doing. As you mentioned, data visualization has become mainstream only recently, especially in some of the cities where the marathons have taken place. To provide greater training before the Marathon, this year we just have started providing registered students with various resources and helpful links (including this one) well in advance to encourage them to learn more about data visualization. And since the beginning of the program, we have been running data visualization lectures and workshops hosted by design professionals during the Marathons to teach best practices (based on feedback, we recently moved these lectures and workshops to the start of the marathon program so lessons can be incorporated from the outset).

Again, thanks Enrico for all your support. As we are ever committed to developing the best possible Marathon program, we’re very much open to ideas.

The Visualizing team

—

Thanks Visualizing for accepting openly my criticism. I think this is simply great!

“Huston we have a problem …”

I just received this in my inbox:

We want to again express our sincerest gratitude for your help in making the Visualizing Marathon 2011 such a resounding success. Your participation was instrumental and the 376 students who competed in Sydney, São Paulo, New York, London, and Berlin told us how excited they were to meet you and have their work reviewed by such an esteemed global jury.

We just announced that the winner of the $10,000 “Imagination at Work” Grand Prize is Columbia University for E-Cube-Librium [...] Out of 15 finalists, the Grand Prize was awarded to the project that “best illuminates a new insight or solution to a complex problem through data visualization.“ [bold is mine]

I receive this because I was part of the jury for the Marathon in Berlin. Visualizing Marathon is a series of events (inspired by the more famous hackatons) organized by Visualizing.org around the world to promote visualization. Groups of students develop a visualization for a given dataset/problem in 24 hours and Visualizing.org gives awards to the best entries.

Being a juror was fun and and an honor for me, as well as being one of the speakers at Visualizing Europe last year. I am grateful to Visualizing for the great work they are doing in terms of promotion, and also for their commitment to building a solid platform for visualization designers. Nonetheless, I think we have a problem.

I look at E-Cube-Librium  and I cannot help but think: “Is this the best 376 students from all over the world can produce?“ It just doesn’t match with my definition visualizations that “best illuminates a new insight or solution to a complex problem“.

I am really sorry I have to say that, especially because I am sure the students did their best and are probably proud of their work, and also because I am sure the guys at Visualizing.org have the best intensions in their mind. But I am also concerned that people around the world would look at the best prize winner and think this is the gold standard of visualization. We have to be careful, especially now that visualization is in the mainstream, about what message we give. I have seen too many times visualization dismissed altogether because people think it’s only pretty picture. Our reputation and future is at stake here.

Now, we have a nice series of events organized around the world and I am all in favor of data visualization evangelism, but why are the results disappointing? Is it an intrinsic problem of marathons and contests or maybe we can engineer the whole thing to make it more effective? Here are some potential explanations I can tell from my experience:

1. Time is too short to produce quality results. Every time I complain about the quality of the results there is someone who points out that time is too short. I am not fully convinced time is the main problem however, even though I do think time is too short. Basic design choices do not depend on the amount of time. It doesn’t take time to know a 3D visualization of numeric data should not be your first choice when designing a visualization, it takes knowledge.
2. Students are not well prepared. That students are not knowledgeable enough to produce quality results is not surprising. Visualization has become mainstream very recently and there is not a clear path to follow if one wants to become an expert. Nonetheless, some of the entries I personally reviewed as a juror were more than reasonable, especially given the 24hrs constraints! Also, giving a look to the page with the whole set of winners and honorable mentions it’s surprising to notice how neat solutions coexist together with very questionable ones.
3. Jurors select the wrong entries. Another possibility is that jurors just pick the wrong entries. I don’t know who selected the grand prize winner, I was not involved in the process, but my feeling is that here we might have the biggest mismatch. When I participated as a juror it became clear to me how things can go wrong. Some people put clarity and information throughput before everything else (guess who?), others judge things from their coolness factor. I know, it’s sad but that’s the way it is.

• Give more time. If time is too short why not giving more time? The marathon format does not lend itself to data visualization. Visualization is a process, a tortuous process actually, with lots of dead ends along the road. Pretending to visualize data effectively in 24 hours might be an unrealistic goal.
• Train students before the marathon takes place. If students are not good enough why not giving them some training before running the marathon? There are many professionals out there who are able to explain in a concise manner what are the no nos and the good practices of visualization.
• Run marathons without prizes. Maybe a marathon could be held without giving a prize? I don’t know … is the perspective of receiving a prize that motivate students to do their best? Maybe not. Maybe just knowing that they will have the opportunity to get trained by a professional and to have a certain level of exposure will motivate them enough. I think competition is totally overrated.
• Let people judge in place of jurors. One option could be to have “better” jurors but then we would have to discuss what we mean by “better”. As an alternative, why not letting people judge? I am not sure the result would be better but at least we could claim it is a democratic process and it wouldn’t embarrass any jurors.

And you? What do you think? Do you have any concerns with contests and marathons? How would you shape your own marathon event? Do you have any suggestion on how to improve the situation? I’d love to hear your voice.

—

IMPORTANT NOTE: I had the chance to discuss with some people at Visualizing before publishing this post. Since I totally respect their work and wanted to avoid slashing them with an overly unfavorable post, I decided to let them read it before publishing it. Apart from a few sentences here and there the post is still the same as the original draft.

Charlene Manuel was also so kind to send me a long reply to this post which I decided to publish soon as a post rather than a comment so that everyone will get the feeling of how Visualizing is handling this criticism.

I am very satisfied with this process. I think we all have to be happy to see that it is possible to have constructive criticism and make the whole field thrive without unnecessary battles.

—

UPDATE: here is the answer from the Visualizing team.

When I think Visualization and Excel there are two names that come into my mind: Jorge Camoes and Jon Peltier. If you want to do serious data visualization with Excel, stop here, they are the names. Since I was more familiar with Jorge’s work and had more opportunities to discuss with him I decided to interview him to cover the Excel part of this series, but you can give a look to Jon’s web site if you have any additional questions.

Jorge had been developing visualization in Excel for a long time now and I still remember the time when I saw one of his dashboards in Excel: “Wow, can Excel do that?” Give a look with your eyes to his dashboard courses. Pretty amazing isn’t it?

I have been following Jorge’s blog for a long time now and I often enjoyed his short and catchy blog posts. If you are not following him, give it a try. It’s worth it.

How did you start using Excel?

I started my professional career as a desk researcher. I had to create information products with lots of charts using market and socio-demographic data, and Microsoft Office was the only tool available. Like everyone else, I had no data visualization training, so you can imagine how bad those charts were. On the one hand, that’s very depressing, from a personal point of view. On the other hand, this proves that data visualization skills are easily acquired, once you become aware of what data visualization is all about.

What’s the best and worst aspect of Excel?

We must emphasize that Excel is not a data visualization tool, so you cannot directly compare it to other tools. That said, you can learn and practice sound data visualization principles using Excel. Its chart gallery is poor, but you can make new charts using some more or less clever tricks. Check Jon Peltier‘s site to see how you can extend the Excel chart gallery. So, its flexibility and general availability are the best aspects.

Unfortunately, defaults that emphasize marketing and sales pitch are responsible for a generation of users that don’t really know what a chart is. That’s the worst aspect of Excel. Also, because of it’s flexibility, many users do not recognize that they need stronger data management skills.

How is the learning curve vs. return-on-investment of Excel?

Most business users have access to Excel training. They just need to be brainwashed to remove all they think they know about charts. :) Corporate data visualization culture is so poor that applying simple rules can greatly improve insights and ROI, and you can do it using Excel.

Ok, I am a beginner and I want to learn Excel, where do I start?

Chances are, you already know Excel. If you don’t, I’d recommend Chandoo’s Excel School or Daniel Ferry’s Excel Hero Academy. And Jon Peltier’s site, mentioned above.

What other tools would you recommend other than Excel?

90% of all charts you need in a business environment can be done in Excel. But if it takes a full day to code a chart that you can do in minutes using a different tool you have a good argument to make the switch. I would recommend Tableau, Qlikview or Spotfire. They are well-aligned with currently accepted data visualization best practices and they force you to learn more about structuring your data.

—

Some comments from Jorge … and my answers:

• J: Business users hate programming. You can’t explain a product manager that a simple recorded Excel macro can make all the difference. You can’t tell them that they need programming skills to make a chart.
  E: I think this is totally fine and probably a reason behind the big success of Tableau.

• J: If it can be done in Excel managers will not spend more money getting a new tool.
  E: Ok, but then they have to be ready to pay someone to let Excel do the job right? I see a great potential for consultants here.

• J: But managers are becoming aware that they need a serious (visual) reporting tool; Tableau is one of the options; traditional BI tools are moving fast. If a BI tool supports sparklines that’s a good starting point.
  E: I think managers will feel more and more pressure as visualization becomes mainstream. I think we just have to wait a little to see some stuff flourishing. I am not too pessimistic.

• J: I believe tools matter, and matter a lot. Tools are not neutral (Tufte says that regarding Powerpoint). If you have to fight them they’ll make your life miserable. Try to apply Tufte’s principles to Crystal Xcelsius. I already wrote about this in my blog using fable about the scorpion and the frog (“it’s in my nature”).
  E: Sure, tools matter. Especially if you know how to switch from one to another according to your needs. Nonetheless, I still believe principles come first. And in order to select the “right” tool and understand its limitations you have to have a clearer idea of what you want to achieve.

• J: Life is short: I would argue that it’s better to learn about perception, statistics, data management and graphic design. Delegate the programming part. I’ve been making some dashboards and I spend more time programming than exploring better ways to show the data. Hate that.
  E: Cannot agree more. Eve though I think we are still in a phase where it’s really really hard to split between the designer and the implementer. The two things are so intertwined that trying to outsource the implementation may very easily lead to unsatisfactory results. But sure, the real skill is in the design IMO.

• J: If you don’t include R in your list you’ll get into troubles :)
  E: Sure! It’s in the pipeline :-)

Yesterday I wrote this on twitter: “I must confess I very rarely read data visualization blogs, most are depressingly predictable and shallow.” Yes, it’s not the nicest sentence I could write, but it’s true: most data visualization blogs suck. They do not inform, they do not entertain.

At VisWeek, last month, we organized a pretty successful Birds-of-Feathers (BoF) titled “Blogging about Visualization”. I and Robert advertised the thing a bit and we managed to gather a pretty cool bunch of people around a table. We spent at least a couple of hours all together and then we enjoyed a wonderful dinner at a Greek restaurant.

During the BoF we discussed several aspects related to blogging (check the nice summary wrote by Dominikus to know more) but what struck me the most is the following: (1) people desperately want to know how to succeed with blogs; (2) people think it is a sort of black art when in fact it’s only a matter of mindset and hard work; (3) there are endless possibilities to open new blogs.

Yet, the decent blogs around can be counted with the fingers of one hand. And I want to see more great stuff, because either we grow as a community or nobody will grow. Here are some personal thoughts about blogging and a number of tips I want to share with you, hoping they will convince some among you to open the best data visualization blog ever.

The Data Visualization Showcase is Dead

When I think about why many data visualization blogs are so useless, the reason number one that comes into my mind is that they try to replicate a dead model: the data visualization showcase (I fell into this trap twice before creating FILWD, so I know what I am talking about). The showcase model is this: “Hey folks, look how cool this is“. Stop. Iterated x-times per week.

You don’t need a blog for this. It was maybe true 5 years ago but with the advent of Facebook and Twitter it’s totally useless. Also, and even more important, there’s no way for you (and for me) to compete with Infosthetics and Flowing Data (@Andrew: I know you don’t agree with me on the death of the data visualization showcase, but what can I do? This is what I think ).

Let me clarify. I don’t think these two blogs are useless. Andrew and Nathan did an enormous service to our field and we all have to thank them from the bottom of our heart, but it’s foolish to believe we need more of that.

Three key reasons why (vis) blogs suck

I could name a hundred, and by the way if you buy a book on blogging (like the classic mainstream ProBlogger) you will find millions, but here I’ll focus on those I believe are especially troublesome for vis blogs (apart from the data visualization showcase which is the most troublesome).

Trouble #1 – Taking it as a hobby. This is the most problematic. People write blogs casually, once in a while, when they have nothing special to do or when they feel something is so cool they have the urge to share it with the world, that is, three friends. Amateur blogs are everywhere and pollute the whole web. If you want to succeed with your blog it’s important for you to realize that you have to sweat your damn shirt. On the contrary, if you don’t want to succeed, why polluting the web with your blog? Think about it, it’s an ecology thing: every piece of information you put on the web may decrease the already feeble signal to noise ratio we have. Do you want to contribute to the noise?

There’s no other way to succeed than taking it as a serious endeavor, believe me. Blogging takes a lot of planning and work. Every single post may take many hours distributed across days, weeks, or even months. And that’s just the effort needed to create content, without counting administration and marketing. You might not see it, but behind every single post here there is a huge amount of work, and I know it’s the same for other fellow bloggers.

Being serious about your blog then it’s not only a matter of content but also of being committed to have a somewhat regular schedule, especially at the beginning. People hate dead trees and for a good reason. Please do me a favor: if you are considering opening a blog, take the whole thing very seriously. You need a good reason for opening a blog and if you don’t have one, sooner or later you will give up. I don’t want to discourage anyone, to the contrary, I want to see more great blogs! But I am also tired of shallow blogs and dead trees.

Trouble #2 – Providing limited value. What is *special* about your blog? I know, it’s a tough question. But, if you are not totally honest with yourself about that, you will have problems. People stop by and read your blog only because you are able to deliver some kind of value. What value? I don’t know … you name it. As a general rule people read for two main broad reasons: to learn or to be entertained (or both). Are you able to deliver unique knowledge that other people cannot deliver? Or do you have a special irresistible style that people love so much they are eager to see what’s next? That’s the trick, that’s the obsession you have to have to succeed.

Many, many, many vis blogs are shallow just because they do not give in, they do not have anything special to offer. They don’t even try to differentiate themselves from the rest. It’s a game in which you lift the bar 1 inch higher every single time you write. The web is a jungle, people jump from one web page to another in a matter of seconds, how do you plan to let someone stop and read through what you write? Let’s take the data visualization showcase mentioned above: do you think you can attract people by showing new visualizations every day? Do you think you are more skilled than the current main players in finding new stuff? I have several doubts.

When I opened FILWD it was clear to me I could not compete with the big guys (and I didn’t want to anyway) so I asked myself: “what skills or knowledge do I have that I can use to gain a competitive advantage?” And my answer was that I have direct access to vis research and researchers and that I know vis theory better than the average geek. I am sure you have your own uniqueness so try to think hard how to use it.

Trouble #3 – Forgetting to show a real face.People are too busy to absorb the bare information, and information by the way is not a scarce resource anyway. Many blogs are plain dry, it looks like the writer does not exist or hides behind the curtains. Where are the emotions, opinions, and fun? Writing about scientific stuff does not imply being serious, objective or dry. The best bloggers show their face and risk their reputation every single post. Sometimes I feel a pain in the stomach before hitting “publish”. I happened to think: “people will kill me for this one “.

Similarly, many bloggers don’t spend any time thinking whether they have a style or not. But *your* style matters a lot and you’d better know what it is. There are a million styles and be careful not to fake it. Your style has to be natural but it also has to shine through your words and visual design. Take for instance Stephen Few: Oh boy … I hate the way he expresses his opinions, he makes me cling my teethes at times, but you rest assured I read every single line of what he writes. What is your style then?

How do you create (or revamp) a successful vis blog?

Hey this is slippery terrain: every single blogger has his own formula and you can find a million sources on the web on how to make your blog successful. I don’t pretend to be a blog guru, but I can share with you the things that really worked for me, with the hope they will assist you in case you want to open your blog.

Tip #1 – Find your final cause. How do you plan to change the world? Why do you want to open a blog? Once you put aside all the legitimate ego trip we all make what is left for the others? Successful blogs are centered around the readers, they want to make the world better. They strive to provoke shifts in people’s mind. How do you plan to be ridiculously helpful for people? With FILWD I planned from the very beginning to help people become visualization experts, then I discovered I could sometimes help them think in unconventional ways. What’s your cause? I’ll give you an example: do you know anything about Data without Borders? That’s a cause folks!

Tip #2 – Study a lot. Before starting FILWD I read an endless amount of material about blogging, I trashed many and kept some. I studied the strategies of many many successful bloggers in many other areas out of visualization. I could name hundreds of sources but you have to do your own research. Among the thousands things I read, there are two gems that really shine: Trust Agents, a must read even if not an easy read, and Think Traffic, the best blog about blogging ever.

Tip #3 – Plan ahead and find your style. Before starting FILWD I wrote down a thousand plans and eventually came up with two key pieces of information: (1) my target posting schedule; (2) a very few number of post categories. The posting schedule does not have to be very tight but it has to be somewhat regular, especially before your blog is established; people hate guessing when you are going to post the next article (and of course I am still struggling with it). Having a number of predefined post categories is the best piece of advice I can give, it helped me being totally clear about what I wanted to write and especially what I did not want to write. For instance, I very rarely write about other people’s work unless it is an inspiration for a broader argument. You can check my categories on the blogs and you will see they are very few. When I write a new post I think: “what category do I want to write in today?”

Tip #4 – Be ready to walk through the dark and deep valley of loneliness. Blogging reminds me when I started learning how to play guitar many years ago. At the beginning it’s so frustrating, it looks like you will never be able to play two chords one after another. With blogs the problem is that at the beginning you have zero readers and you have to spend a lot of time preparing these stupid posts nobody will ever read. Very painful. But it’s totally transitory: if you keep doing the good work, people will come and will love your post written to nobody in the past. That’s a very key element of blogging: being able to go through the deep valley and wait until it blossoms. You have to have faith: it will be great.

Tip #5 – Find your own buddies. What is life without friends? I don’t have to tell you how to use twitter, Facebook, or Google plus right? Plus I don’t think there is a unique formula. But hey, make sure to build a thriving environment around your blogs and your ideas. Somebody said “No man is an island”, well this is especially true in this business. Find some buddies, share your ideas with them, test your ideas before writing a post, be exceedingly generous and genuine and people will gather around you.

Tip #6 – Experiment. Blogging is a constant experiment. You write a very successful post with a given strategy, you try to replicate it and it doesn’t work. I like to think about blogging as a radio knob you have to manipulate to find the right frequency to tune with your audience. The frequency is always shifting and your work is to be able to seek the right spot all the time. Sometimes it works, sometimes it doesn’t. But it’s not a big deal as long as you keep trying. Blogs, for instance can accommodate very different media and it’s a good idea to experiment with them. I experimented a few times with video and I was scared shitless because of it.

There are of course many other things you can do to make your blog successful, many of which I don’t know yet. Everyone has his own path, you have to find yours. I know one thing for sure: hard work always pays off. Always.

Need a good reason for opening a blog?

Hey, I hope I did not scare you too much up to this point. There is one thing I want to make sure you get out of this blog post: opening a blog may be one of the smartest choices you can make in your life. Again I could name hundreds of reasons why blogging is great but for me the most important one is that it feeds my mind in a way I could not get with other means. Blogging so far helped me, at least, in these many ways:

• I became a much better writer
• I became a sharper thinker (thanks to having to write what I think)
• I know much better how the web works
• I know many more great thinkers … and they know me
• My ideas are debugged by a large crowd of people
• If I have a burning question I have lots of people to whom I can ask
• I get invited for talks
• It feeds my research and my research feeds it
• I might write a book one day thanks to it
• …

I can testify that all the effort is definitely repaid by the myriad of benefits you can get. Some people do blogging for the money, and some are pretty successful, and some other for the glory. But whether you do it for the bling bling or not, the formula is always the same: you have to write epic shit. There are altruistic and egoistic benefits from blogging and they are all fine as long as you have a good balance. Blogging makes you grow internally, you find yourself improving in many ways, and it helps you having a powerful interface with the world. But it also helps people thrive thanks to your work, and that’s absolutely priceless.

Start a kick-ass visualization blog today!

Let me add one final remark. If you are thinking of opening a data visualization blog, a good one, please do it! We have a desperate need for quality content and I want to have my inbox filled up with exciting ideas. If you need more help send me a line or ask to professional bloggers. I do think there is a huge space for new blogs in this area, you just need to find your niche. For instance, I am looking forward to data visualization blogs related to one specific application area. Or, another great one I’d love to see is a blog with a frequent posting of interesting little visualization experiments. It’s up to you now, let’s make data visualization better together!

Hi There,

VisWeek is approaching! This is just a short notice to let you know I am organizing a Birds-of-Feather with Robert Kosara titled “Blogging About Visualization” at VisWeek. The goal of the BOF if to meet people who are interested in data visualization blogs (bloggers and readers) and have a chat about current practices and future developments.

Here is the information about where and when:

Blogging about Visualization
Time: Tuesday 4:15-5:55pm
Location: Bristol
Description: A meeting of visualization bloggers, readers, and anyone interested in discussing visualization on the web. We will discuss experiences, uses, purpose, benefits, and future goals of blogging about visualization.

If you are a regular reader of this or other data visualization blogs and has something to say or just want to meet us and join the conversation please drop by, I am sure it will be fun. Me and Robert plan to give some advice on blogging, in case people want to hear it, and the BOF will be totally informal. It could also be the starting point of new endeavors, collaborations, ideas, etc.

Cocktail & Dinner.

Another good reason to participate is that we will have a cocktail and dinner just after the BOF. We still have to fix the location but we plan to meet at 7pm at the lobby and go there together. If you are interested, even if you cannot come to the BOF, please sign in the google group I created for this purpose: Vis Blogging Dinner. We are already a good bunch of people.

Take care and hope to see you soon.

P.S. In case you are reading this and will not be able to come to visweek don’t worry I plan to provide plenty of information from the conference this year. Stay tuned!

When I saw for the first time a visualization developed by Jan, the Ghost Counties, I was totally fascinated. It’s brilliant. It took me a while to understand how it works, but once I got it I could not help but admiring the strange mix of complexity and simplicity it provides.

Despite he looks so serious in this picture on the left, he has a big smile and he is fun. I met him for the first time at Visualizing Europe and since then we exchanged many emails. Plus, he is a regular commenter here (and everywhere) and I love him for that.

I don’t know how much I have to add to convince you his advice is a valuable one. Just give a look to his portfolio and judge yourself. He is IMO one of the most interesting data visualization freelancers recently appeared on the scene.

I know, by talking with him, he is proficient with several technologies but he has a passion for D3.

How did you start using Protovis/D3?

I’ve always been someone interested in the latest technologies. So, since I follow the data viz community very closely, I was aware of Protovis very early on, and I was aware of the development of D3 even before it was released to the public. I have a software development background, so I don’t have too much trouble finding my way in new programming languages, and since it excites me to work with new technologies and frameworks, I just started playing with Protovis and D3 as soon as it became available.

What’s the best and worst aspect of Protovis/D3?

The best aspect of Protovis is that it is a domain specific declarative language, which means that is fairly easy to start writing code, using visualization related keywords and functions. The best aspects of D3 is it’s flexibility (more direct integration with SVG) and better performance. The worst aspect of both D3 and Protovis is that it’s hard or impossible to get it working on older browsers, and the learning curve for D3 may be somewhat harder than for Protovis.

Ok, I am a beginner and I want to learn Protovis/D3, where do I start?

I think Protovis is easier to start with than D3, but Protovis is no longer under development. I also see that the Protovis mailing list is not very active anymore, while the D3 mailing list is very active. But, I guess Protovis would be a very good way to start if you’re a beginner. Basic Javascript programming skills would be recommended, both for Protovis and D3. For some great Protovis tutorials you should check out Jerome’s blog: http://www.jeromecukier.net/blog/category/protovis/page/2/ (part 1 – 5: working with data in protovis). Also, the Protovis website has quite some examples that are pretty good. For D3, documentation, examples and tutorials are still under development: http://mbostock.github.com/d3/api/ so, with D3 you’re more on your own right now (and of course the mailing list). But things are improving rapidly.

How is the learning curve vs. return-on-investment of Protovis/D3?

Protovis is really a good way to learn visualization and programming at the same time. Protovis is a language that is really geared towards the visualization (and diagrams) domain, so it really makes sense to talk about axis, marks, lines, bars, pies, etc. Also, there are really quite some good examples on the Protovis website, so it’s fairly easy to get started. However, right now Protovis is not supported anymore, and people are really moving to D3 now, so getting support may become a little tricky. Also, Protovis does not perform as well as D3 with very complex graphs for instance, and also, compared to D3, in Protovis you’re a little bit limited to the animations you can achieve. So, overall, a good way to start and also good to make some nice standard diagrams and visualizations, but if you really want to do ‘heavy’ visualization stuff, you might consider moving on to D3.

D3 is more powerful, more flexible and seems to have more capabilities (and better performance) than Protovis. The flip side is that in order to have a more flexible programming language, the language is also more abtract. Though many concepts of Protovis are also implemented in D3, and there are also quite some predefined visualization layouts, it’s also more useful (compared to Protovis) to gain some knowledge of SVG, since it’s more likely that you might do some low level stuff. D3 does give you much better animation capabilities, better performance, more flexibility, so, once you get the hang of D3 and some SVG, you’re able to create some very compelling interactive visualizations.

What other tools would you recommend other than Protovis/D3?

The tools you mention are some of the best right now. I also think that Raphaël is a fairly good alternative if you want to do Javascript-based visualization that works in older browsers as well. Personally I don’t have much experience with Raphaël yet. Also Processing now has an Android mode, which is great if you want to create visualizations that run on Android phones, and the upcoming Processing 2.0 also has a Javascript mode, so you can easily create HTML5 canvas based visualizations with the Processing development tool.

A recommendation I’d like to add: when I work on Protovis or D3 visualizations, I use TextMate on the Mac. This allows you to open a preview window which renders your visualization near real-time when you are typing in your code. I’m sure that are similar tools that do this. This is really great for getting immediate feedback while you’re coding.

This is a short notice to all FILWD readers who are going to VisWeek 2011. I will be there the full week starting from Sat Oct 22th and I would love to meet some of you guys. There are two main events you might be interested in.

Data Visualization Blogging Dinner

If you are going to visweek it would be just awesome to spend some time together in front of a hot meal and a good drink. I’d love to see your faces in real flesh!

If you are interested (even if you are not sure to come) please sign up to this google group I just set up - Vis Blogging Dinner – VisWeek 2011 - and we’ll find a suitable place and time to spend some time together. Ok? Please sign up as soon as you can so that I will be able to reserve a place to stay.

Blogging about Data Visualization BOF

I am trying to set up together with Robert Kosara (EagerEyes) a Birds-of-Feathers during visweek to meet people interested in data visualization blogging. Everyone is welcome (bloggers, aspiring bloggers, blog readers, etc.) The meeting will be totally informal and we plan to give advice on how to blog about vis (in case you want to hear it from us of course :-) and to hear from you guys whatever idea you have on how to make data visualization blogging better/funnier/stronger/etc.

Catch me!

Apart from that we can always meet in the hotel during the week if you feel like. If you see me around and want to have a chat or share a drink or just say hi please catch me! I added a picture of myself here below so that it’s easier for you to recognize me (I hope).

If you are not going to VisWeek …

I plan to record a video from visweek at the end of each day to provide a summary and thoughts. I also plan to write more about what will happen at visweek later this week. Stay tuned!

I wanted to title this post “giving constructive feedback about visualization and its long-lasting effect” but it didn’t sound as good as this one.

The Story

I was about to write my next long post (don’t worry, almost done) when I received an email from a guy working for Hotels.com:

“Hope you’re well. I’ve seen you’ve covered infographics in the past and thought you might be interested in a new one from Hotels.com that looks at how people from around the world eat and sleep when staying in hotels. The research was conducted among 3,339 people in 20 countries. You can view, download and embed the infographic at: http://press.hotels.com/en-gb/infographics/“

Here is the infographic (click on it to see the details):

I gave a quick look to the image, read the findings, and just discarded it as crap. I said to myself: “Here we go again … another email with crappy infographics”, pushed delete, and moved on to the next task. After a while, my sadistic brain could not resist and I wrote a quite cryptic message on twitter trying to see if I could catch some fish:

“From hotels.com: “I’ve seen you’ve covered infographics in the past and thought you might be interested in a new one” http://bit.ly/nGiUiC“

A few people replied and again I moved on to the next task.

Some other people like Stephen Few would have maybe started a long rant about all the reasons why this was crap, while some others, maybe, would have taken it seriously and tried to analyze it in details. Me, I just shook my head a never replied to the guy.

Here is where the true story begins: after a few minutes I receive an email from Andy Kirk of visualisingdata.com (bold is mine):

“Enrico – I received the same Hotels.com email today and had a good exchange of emails with the guy promoting them.  

To be fair after our conversation he was really appreciative of the advice and said he will do his best to try and affect a change in approach. Really interesting how this particular market has erupted though isn’t it – the fact Hotels.com has a dedicated Infographics section under its PR pages…”

And later on:

“… it is becoming more and more difficult to stay on top of these type of requests but I’m taking the longer view that if I can offer constructive feedback it might in the smallest way have an impact on improving practice“

Let me repeat this sentence from Andy:

“If I can offer constructive feedback it might in the smallest way have an impact on improving practice”.

The Lesson

What a lesson have I learned! It was like a diamond in my head. Thanks Andy.

If you have been reading this blog for a while, you might have noticed I tend to be quite pacific, but at the same time when it’s time to say crap, I say crap.

We in the community have learned to have an automatic reflex: we look at some crappy visualization and in the best case we shake our head, in the worst, we write long rants a la Stephen Few.

I must confess I use to shake my head more often than writing long rants, also because otherwise FILWD would only host such type of big-ego content which I don’t like.

After reading Andy’s email I completely changed my mind. It’s way too easy to look at some stuff and think  ”oh yes, the usual crap”. I did it so many times! And it’s even funnier when you share the “crappyness”  with some friends or tweet about it “hey look … how could they be so idiotic to draw this and that in this and that way”. And we fill our mouth with words of wisdom.

Question: Do we make visualization any better by ignoring or, worse, mocking people who design bad visualization?

I know some of you might say that publicly criticizing bad stuff with big words will make people notice and be more cautious about what they publish. True. But will this strategy pay off in the long-term? I am not sure.

What do you think? Is it more beneficial a loud voice or a humble and cheerful suggestion? Especially, when people ask for an opinion. Do we need both? Do we have to treat different people with different strategies? Or should we just ignore everybody and do our work the best we can?

A few months ago I wrote in my post on Visualization Consumerism:

“I think we have to acknowledge the problem and do our best to educate people. But wait a moment …. educating people is a dangerous idea! I agree. But let me explain what I mean. When I say educating people I mean doing it bottom-up; by giving the right examples and striving for creating a thriving environment“

It looks to me like if these words had been written by someone else! The words are good, my behavior just does not match. We will build this thriving environment only if we learn to shake our head less and learn to help people in every possible way to make great visualization.

Sorry, now I have to go … I have to write a reply to the guy from Hotels.com.

Thoughts?

Oh gosh, I was almost going to miss it: Fell in Love with Data turns one! One year has passed and so many things happened in the meantime. Where do I start? Well, let me start with the obvious but important:

Thanks to all of you guys who are reading, commenting, re-tweeting, sending messages, etc. You gave to me much more than what I gave to you. I owe you something.

Numbers have been growing fast during this year but FILWD is not my personal toy to boost my ego (even though it helps in these regards), it’s a tool to advance data visualization. It’s for you, it’s for me, it’s a work in progress made to help us making this whole business damn better … with some fun in between if possible.

Lessons Learned.

There are many things I learned during this year, far more than I am able to write in this blog post. Here are a few that come to my mind right now as I am writing:

• Blogging is a fantastic platform and I cannot think of doing without it anymore. Now, after one year, I cannot think of how one can pretend to be influential without a blog (especially in academia my dear fellows).
• Writing blog posts helped me far beyond my expectations towards clarifying ideas to me and taking all these vague concepts I had in my brain and transform them into something concrete. What is really surprising to me is to notice how not only my research work helped me writing blog posts, but also writing blog posts has had a strong positive influence on my own research (one more reason for blogging my dear friends). Far more than I expected.
• Blogging is at the same time much harder and much easier than people think. It’s harder because you have to spend a lot of energy and thoughts to make a blog successful. It’s a damn serious job, it doesn’t happen by chance. For me everything changed when I realized that it was totally nonsense trying to compete with Infostethics and Flowingdata and that I had to offer something different. But blogging is also much easier than people think because it just takes you to come up with a solid concept, set up a blogging account anywhere, and write, write, write an let it flow. I am surprised by how many people are scared by it.
• What matters is not success in terms of numbers (even though numbers count), but influence. At the end of the day if you have only 100 readers but you are blowing their mind it’s a lot better than having 100.000 casual readers passing by and say: “hey cool” and then they go back to their own stuff. It reminds me a notable statement from Tufte: “differences that make a difference”.
• No matter how much planning you put on your blog and how many blog posts you have in the pipeline, a blog is a living entity with its own dynamics and you cannot anticipate how people will react. This means being always ready to adapt and write about what matters now. What people need to read in this very moment, not the idea you had three months ago.
• The space for the data visualization showcase is shrinking (and thanks god!) The only way to be successful in data visualization is to do solid stuff that people need. Yes, there are still a couple of consumerist visualization readers out there but who cares? Do they make any difference at all?
• It doesn’t matter how clever and innovative the things I write are, the biggest value of the blog is YOU. In innumerable instances the real value of my posts came from my readers and their comments. I especially enjoyed those with opinions alternative to mine. They helped me re-think my ideas and make them more solid. Thanks a looooot!
• The best posts I wrote are those that scared my butt off, those where I felt I was stretching my intellectual capabilities. Often with this kind of posts I experienced the tension between trying to be as accurate and informed as possible, with the realization that I just don’t have enough time and means to study everything in every single detail. That’s hard but it’s also very very rewarding.
• Data visualization is a huge trend, far beyond the close knit of academics I was used to deal with. Plus, people out there are in desperate need of solid information because the Internet is a chaos and the field is not mature enough. Also, we people in academia have the responsibility to lead the way (did you hear that guys?)

Retrospective.

In retrospective what could I have done differently? I don’t know … maybe you can tell me what you think. I don’t think I could have done anything too differently, I am pretty satisfied of how things evolved.

If I have to mention one single thing I would like to do better, it’s to achieve a much more regular posting rate. But in the end it’s a compromise, what is better: to write more often but more crappy stuff or write only when I have something to say? Dilemma.

I am proud of the following posts:

• How to Become a Data Visualization Freelancer | Interview with Moritz Stefaner: Because it blends surprisingly well useful knowledge and fun (thanks Moritz!)
• Visualization Consumerism: Because it came from my gut and I managed to let some people think.
• Data Visualization is NOT Useful. It’s Indispensable: Because it’s a little manifesto about how important visualization is to me.
• Do visualizations need to be “accurate”?: Because I overcame the fear of questioning the most fundamental theory we have in vis (and it hurt).
• Can visualization influence people? I mean can we prove it?: Because I managed to leverage on an apparently insignificant personal life event.
• 7 Classic Foundational Vis Papers You Might not Want to Publicly Confess you Don’t Know: Because it spread some fundamental knowledge around (it’s the most successful post I’ve ever written so far).

Things that blew my mind.

• That my post on the 7 classic foundational vis papers had almost 4000 visits on the date of publication. People are thirtsty for knowledge!
• That some renown researchers in the field are reading my blog and contact me for the things I write.
• That some people invited me to talk for my blog and not for my research work (though a bit disappointing! :-))

Special thanks to …

• Robert Kosara: for showing me  with his blog that it was possible to write a blog like FILWD.
• Andrew Vande Moere: for instilling in me some doubts before I started.
• Prof. Tamara Munzner and Prof. George Grinstein for giving me so much fuel.
• The Data Visualization Cartel: you guys know why.

Plans for the future?

I always have plans for the future which I regularly abandon and the list is so full of stuff that I know I will never do it all. So what can I say? Maybe you have something to suggest:

• How do you see FILWD evolving in the future?
• What are the posts you liked the most and would like to see replicated in the future?
• Is there anything useless in FILWD that I should definitely stop doing?

I can anticipate a few things I’d like to do:

• Do more videos, especially if they are fun.
• Create the FILWD Newsletter to have a more intimate communication channel with some of you.
• Create an e-book out of the Beginners Toolkit.

What do you think? Do you like these ideas. Do you have anything to suggest to make them better? Thanks.

—
With Love,
Enrico.

I am really excited to announce my first interview for the “Tools from the Pros” series! We start with a very good one: Miriah Meyer talks about Processing.

Miriah is assistant professor at University of Utah. I met her only briefly during a couple of conferences but I am a huge fan of her research work on interactive visualization systems for biological data analysis (be sure to check them out!) Her tools are a rare example of well-crafted design studies in interactive data visualization and, as far as I understand, they are all developed in Processing.

I really like this interview because it covers many of the things beginners (and more advanced users) need to know. One above all: the rapid prototyping approach Processing makes possible and the whole mindset behind copying and pasting code to explore alternative designs.

Thanks Miriah! I think people has a lot to gain by reading this interview.

—

How did you start using Processing?

I started using Processing in 2008 when I helped design a new undergraduate visualization course at Harvard. We chose Processing as the language for the course, and I learned the core bits of the language putting together homework assignments. I quickly came to appreciate how Processing got rid of all the annoying parts of graphics programming — setting up a rendering window, registering callback functions, dealing with linking and libraries and compiling to multiple platforms, that ridiculous gluPickMatrix, and not to mention the headache of type.

We had Ben Fry come to the class to give a guest lecture that spring, after which we went out to lunch. I’ll never forget his answer to my question of why he created Processing. He said (well, I’m paraphrasing here) that he wanted a sandbox to play in, to quickly develop prototypes without getting bogged down in the architecture of the code. He emphasized Processing as a language to try different designs, with real data and with real interaction. And that cutting and pasting code in Processing is totally cool if it gets a design up and going faster. He wanted a language that lets people totally focus on the visualization concept and design without having to think too hard about the code underneath.

Well, that sounded great to me. And I quickly became a total convert, cutting and pasting code until things got so messy that I had to just rewrite an entire project. I found this philosophy totally liberating and that my work benefited immensely from rapid prototyping. Processing is a language that supports this style of development.

What’s the best and worst aspect of Processing?

In short, the best aspect of Processing is the amount of code it takes to get a simple scene with callbacks going — it is a small fraction of what it would take with OpenGL. Simple primitives like circles, squares, text, etc. are nicely abstracted into one-line function calls. Mouse and keyboard callbacks are automatically handled. There is a wide variety of common graphics helper functions available, like lerp-ing colors. Full-screen apps work without having to grab weird OS handles. The PDF library that exports the current scene as vector-graphics has forever changed figures for papers for me. And the ability to export an application to a variety of operating systems in a single go is absolutely invaluable when working with users on a variety of platforms.

Despite all the simplification of the underlying graphics library, Processing still feels like you are in complete control of every mark you make on the screen. I almost never feel like I need to find a way around a function to get the sort of control I want. The design decisions that went into creating the Processing API are fabulous. Really.

As for the drawbacks, there aren’t any really great libraries (yet) for basic user interface widgets. Which for me is ok because I’m kinda neurotic about how my scroll bars look and act. But for graphics beginners this can be a real time-sink. Same goes for more sophisticated types of visual representations like basic charts, maps, and networks. Other languages like Protovis provide built-in algorithms for handling these very common types of representations. In Processing, you’ll have to implement your own graph layout algorithm (or, find one on the web). Again, this can be a hurdle for people with less programming experience.

And as a small gripe — Processing has implementations of Bezier and Catmull-Rom curves … but where is the love for b-splines???

Ok, I am a beginner and I want to learn Processing, where do I start?

Go to www.processing.org. Click on the Learning tab. Explore.

On the Learning page you’ll find a whole series of tutorials and examples that can walk you through the basic functions of Processing. The next step is to peruse the inspiring demos in the Exhibition, many of which will include example code. When you see a function you don’t understand, the Reference page has wonderful documentation for the language.

If you are new to programming or graphics programming the two books I recommend are:

• Getting Started with Processing, by Casey Reas and Ben Fry
• Learning Processing, by Daniel Shiffman

You can work through the Getting Started book in a day. It’s short and sweet. If you find that you need more help, the Shiffman book includes more details on how to program and lots of paper and pencil, and coding, exercises. Daniel Shiffman wrote this book from course notes he created in teaching design students at NYU about coding and Processing. It’s intro to programming via Processing.

If you are an experienced graphics programmer all you need is what you can find on the Processing website.

How is the learning curve vs. return-on-investment of Processing?

If you know OpenGL and are familiar with Java, the learning curve is super short and shallow. If you are new to graphics, it will take you less time to wrap your head around Processing than OpenGL. And if you are new to programming, Processing is a really fun way to learn the basics.

With that said, it is still a programming language. Reading in data from a file requires basic coding skills, as does just about any interesting interactive visualization. You have to be comfortable with for-loops and arrays. Processing makes graphics programming way easier, but it doesn’t automatically generate visual representations of data. You have to code that.

If you want control over every aspect of your visualization and interaction designs, then you really just have to program. Processing is one of the best languages to use for that. If you just want to see what your data looks like, then there are other tools that can do this quickly with built in visual representations (like Tableau, ManyEyes, Matlab, R, etc).

What other tools would you recommend other than Processing?

I’d recommend any of the tools and languages I’ve mentioned previously. Another gem is ColorBrewer for selecting great colormaps.

Still, nothing beats OpenGL for truly understanding how graphics works. If you are serious about developing interactive visualizations, I think that taking an intro to graphics course that uses OpenGL is invaluable. Understanding the rendering pipeline and how it is implemented in a computer will make the seemingly quirky aspects of even a language like Processing make sense.

Hallo everyone! I am happy to introduce a new series nested in the Data Visualization Beginners Toolkit: “Tools from the Pros”.

In my last post on data visualization tools I suggested a number of strategies to choose the best tool for you and I provided a list of those I think are the best bets currently available. Now, while I think this list is already very useful, I decided to give you more and I interviewed one data visualization expert for each tool mentioned in the list.

I will be publishing the interviews during the next weeks. Some of them are still in preparation and the list might be expanded in the future as comments and requests come in (please feel free to ask!) What I can tell you from now is that I have the following interviews in editing stage and that the first one will come very very soon:

• Miriah Meyer on Processing.
• Joe Mako on Tableau.
• Jorge Camoes on Excel.
• Jan Willem Tulp and Jerome Cukier on Protovis and D3.
• Drew Conway on R.

Each one is a real pro in his area and knows very well the tool he or she uses to make effective visualization. I am sure you will get a lot of useful information out of them.

To each one I asked the following questions:

• How did you start doing visualization with X?
• What’s the best and worst aspect of X?
• Ok, I am a beginner and I want to learn doing visualization with X, where do I start?
• How is the learning curve vs. return-on-investment of X?
• What other tools would you recommend other than X?

I hope you’ll enjoy it. Stay tuned! The first one is coming very soon.

Important: if you are an expert and are willing to answer these questions about your favorite tool I’d be happy to include you in the list!

More important: specific request for other person/tool interviews are welcome! Who else would you like me to catch? About which tool? I cannot assure you anything but I’d love to receive your requests.

Take care guys, and have fun.

Big Pluses: visualizations delivered directly through a web browser, compact code, good community size and excellent documentation.
Some Minuses: the code is a bit tricky and it requires some getting used to, it is not as diffused as other technologies (but this is going to change soon), it might be discontinued in the future the same way as Protovis was.
Notable Examples: Jan Willem Tulp’s Urban Water | D3 Examples Page

Big pluses: can create visualizations in a snap, very easy to explore many alternative views of the same data, does not require programming, very large user base.
Some minuses: not as flexible as using a programming language, it’s expensive, takes some time to understand how it works.
Notable examples: Tableau Software’s visual gallery | Clearly and Simply’s Tableau Posts

Big pluses: universal platform, everybody understand excel, practically free, easy to go from data to chart, integrated with the spreadsheet functionalities.
Some minuses: the defaults are crap, harder to go beyond standard charts, slow with big data.
Notable examples: anything from Excel Charts gurus Jorge Camoes and John Peltier.

I kept my promise: the interview with Moritz Stefaner on data visualization freelancing is finally here! And I am really excited.

As I said in my introductory post, I think data visualization freelancing is one of the most exciting trends in visualization; even though it’s a little bit hidden. After recording the interview, I must say I am really satisfied. I learned something out of it and I am sure the same will be totally true for you.

The video is a bit long (see the content breakdown below) but it’s really worth it: we covered a very large number of questions and they all came directly from the readers (thanks to all of you guys)!

Any comment, question, or suggestion for me and Moritz is more than welcome. You can write a comment here below or contact us directly on twitter (@FILWD, @moritz_stefaner). Have fun!

Video content breakdown with timing:

• [01:00] Starting Out (building a portfolio)
• [08:30] Design Practice (iterative approach, designing 20 prototypes!)
• [16:18] Skill Building and References (books, tools and libraries, doing without programming?)
• [27:47] Dealing with Clients (what clients want vs. what is right, freelancer vs. agency)
• [34:08] Pricing (billable time, tracking yourself, strategic prices, the “pain coefficient”)
• [39:44] Time Management (avoid working 24/7, have kids!, having a rhythm)
• [41:55] The Freelancing Market (gaps in the market)
• [44:05] The Role of Research (searching and reading papers)
• [47:24] Summary of Tips for Wannabe Freelancers

Additional versions of the interview

• Download mp3 file to listen on your own player.
• Interview transcription (if you want to read it) – coming soon.

Do you want more? Let us know.

As you can see in the interview, me and Moritz are thinking of recording additional videos. Who knows … this might even become a regular meeting. We would love to have your opinion on that. Especially we would love to know: what else would you would like to hear? Is there anything we missed? How can we help you further? For sure, we would like to record a new one with a more extensive discussion of design practices and the overall data visualization process. Stay tuned and let us know!

Notes

•  First of all thank you guys for sending all your questions for Moritz! This was very useful and I am sure the interview is much much better than what it would be without your help.
• A big big thank to Moritz. I really enjoyed talking with him (as usual) and I think the end-result is really helpful for people who want to know more about freelancing.
• The quality of the video is not perfect, I apologize. There is so much to learn! My phone started ringing at some point, the line dropped because I forgot to plug the power plug, and there’s no video editing apart from very basic stuff. Nonetheless, I think that content is king and what matters is that you are going to learn something. This is work in progress and it will get better.

One of the main goals of this blog, other than challenging the status quo with reflections at the intersection between academics and practitioners, is to help people become data visualization experts. It’s not rare for me to receive emails from people who are enthusuastic about visualization but have little guidance about how to become an expert.

I have been posting some few articles in the past with this specific goal but I realized that they are too scattered and not organized in a way to represent an organic resource for the readers.

For this reason, I decided to create a series specifically designed to help those of you guys who are excited about visualization but really don’t know how and where to start. The series is meant to be part of a permanent collection in FILWD and it’s my first serious attempt to react to my own call to action: “When will we decide to provide lots of value?“.

Introducing the series

The Data Visualization Beginner’s Toolkit will function as an orientation guide for poeple who need guidance in finding the right resources to become data visualization experts. In the guide I will not be teaching visualization directly, nothing technical or theoretical about it (I have plans for this later), but I will show you the resources and one path.

Having such a guide is particularly important today because data visualization is really just like a jungle. There are plenty of opinions, blog posts, research papers, consumerist visualizations, books, etc., and it’s very hard to separate the wheat from the chaff.

When reading this series please keep in my this is my very personal view and, as such, is limited to my own experience. Also, whatever list I will propose is certainly neither unique nor exhaustive. If you are looking for an exhaustive list of resources I highly recommend you Andy Kirk’s collection of data visualization resources.

Here is a tentative list of topics I am planning to cover in the series (subject to changes):

1. Books and Other Resources
2. Programming Languages and Tools
3. Sources of Good Examples
4. Research Papers
5. University Courses

Please if there is anything else you would like to be covered let me know! Send me a message or add a comment below.

Books about Visualization

There is a reason why I start the series with a list of books: if you don’t know the basics of data visualization you will always be an amateur. And what’s worse, visualization experts will notice it and will not take your work seriously.

Also, orienting yourself in the mess we have right now might prove discouraging and prone to errors. If you type “data visualization” in Amazon the result is a disaster, believe me.

Finally, even if you end up picking up very good books, it is definitely possible they are not the right ones given the amount of knowledge and expertise you currently have. Here I suggest the following path (in order).

Show Me the Numbers: Designing Tables and Graphs to Enlighten (To acquire solid foundations). This book teaches the basics of visualization by using only tables and simple charts. You won’t find fancy and colorful visualizations, only scatter plots, bar charts and stuff like that. But that’s the way to go! If you understand the basics then it’s a lot easier to spot the limitations of basic graphs and go beyond them. Plus the book contains the best summary of visual perception applied to visualization I know. It really is a true gem. Don’t make the mistake to be attracted by fancy stuff and skip the basics, start here and you will have very solid foundations.

Readings in Information Visualization: Using Vision to Think (Chapter 1 only) (To go beyond simple charts). Once you understand how charts work and you have learned the basics of visual perception, you are ready to explore fancier stuff. Yet you need some guidance on how to explore the huge data visualization space. The first chapter of this book is the best self-contained piece of work I know. It’s able to provide all it’s needed to start thinking more creatively, but in a structured manner, about advanced visualizations. The book also has a strong emphasis on interaction which is important. If you want to go beyond the first chapter fine, but the book itself is a collection of papers and many of them are totally outdated. But wait a moment, this doesn’t means you cannot find useful material there! In the collection you can find fundamental papers that are totally worth a read: the work of Jacques Bertin above all.

The Visual Display of Quantitative Information and the rest of Tufte’s books  (To learn what “graphical excellence” is). People go crazy with Tufte’s book and I understand why: they are totally beautiful, the cover, the format, the colors, the contet, everything. But regardless their beauty, I have always thought it’s really hard to learn something out of them; they require you to think really deeply about what you see. Basically they are “just” a collection of images. The Visual Display of Quantitative Information is the first one and is the only one I truly recommend because it give more guidance than the others. The others are wonderful but you will have to spend more time on them to translate their content into design practices.

Information Visualization: Perception for Design (To know what happens in our brain when we see a visualization). If you have read all the books cited above congratulations! You have learned really a lot. Now, information visualization is deeply rooted in visual perception and cognition. If you want to master the art of visualizization, at some point you will have to know these basics; especially if you aspire at designing innovative visualizations that fit people’s needs. This book starts from the very basics of human vision (e.g., how the eyes work) up to how we think with visualizations. It’s a tough read but it’s totally worth it. You will have to spend quite some time thinking how these theories apply to your specific projects, but believe me, it’s a true investment. I experienced countless situations where a visualization design problem was deeply rooted in one of the issues discussed in this book. You will find yourself referring back to it all the time.

More Books about Visualization

Important: Are the books not mentioned above bad or not worth it? Absolutely not.

It is important to consider two factors: (1) there are several books I have never read or even skimmed through which might provide some additional value to you; (2) there are extremely valuable book I’ve not included just because they are either too advanced or don’t fit the progression of readings I am proposing here. Please keep in mind: I am suggesting you to read these books in the order I gave above.

A few additional books that come into my mind, which need at least a short mention are:

• Any other book written by Stephen Few.
• Any other book written by Edward Tufte.
• The statistics-flavored and super-classic Visualizing Data and The Elements of Graphing Data by William Cleveland.
• The monumental Semiology of Graphics by Jacques Bertin, which I did not include because it is still hard to get despite a new edition came out and because it’s really a hard read for non-experts.
• The extremely beautiful and information rich Visual Language for Designers by Connie Malamed, which I did not include because I haven’t finished reading it yet.
• The deep and dense How Maps Work by Alan MacEachren, which despite the title teaches visualization and makes you think deeply about it.
• The not known enough and little gem Designing Visual Interfaces by Kevin Mullet and Darrel Sano, which teaches aesthetics in a functional and systematic manner.

Books NOT about Visualization

It’s important to acknowledge that not all the knowledge a visualization expert needs comes from data visualization books. I have no intention to write another long list of related disciplines’ books, but it’s important for you to know that a good data visualization expert may have strong foundations in areas such as: statistics and data mining, data management and manipulation, human-computer interaction and cognitive science.

I don’t want to scare you: you can start doing visualization without these, but little by little you likely will find yourself digging more into these areas.

Also, let me stress the importance of human-computer interaction and related areas. While the rest is normally acquired, at least on a superficial level, by using various technologies you encounter along the way, human-computer interaction has a less technical flavor and you might not learn anything of it unless you seek it.

Knowing how people reason and interact with user interfaces is a crucial skill, the real differentiatior, that you’d better acquire if you want to become a pro. I cannot stress this point enough. Visualization, as any other user interface, happens in people’s mind, not in the computer! And if you want to design great ones you’d better learn how people’s mind work.

There is only one book I feel like suggesting as a starting point: the brilliant, super-practical, and freely-available Task-Centered User Interface Design.

Other Learning Resources

Unfortunately, other than the books I mentioned above, there are not many other sources from which you can really learn something. But luckily there are some few notable exceptions! Tamara Munzner and Jeff Heer, top-researchers in the field, share the material of their courses freely on the web and you should not miss them for any reason:

1) Tamara Munzner’s InfoVis Course Slides at University of British Columbia
2) Jeff Heer’s InfoVis Course Slides at Stanford University

These are university courses, with a specific target, but I cannot think of a more carefully and better organized set of information covering the whole theory and practice of information visualization. What is really unique in these courses and their material is the way this information is organized. Information visualization is still a young discipline and nobody really agrees yet on the content and order to use when teaching it. These two courses found in my opinion the perfect balance between coverage and organization.

Another great source for learning data visualization are Stephen Few’s articles and white papers, which teach a whole lot of fundamental data visualization skills with his usual concise and effective style.

Can I get all the knowledge I need with these books? No.

Conclusion

That’s all folks. I really hope this series will be useful to you. Let me know what you think and if it helps. Also, I’d really love if you could enrich it with your suggestions. You can write comments below or send message to me on twitter at @FILWD.

Please do not forget to share this with your friends or people who might benefit from it. Its main purpose is to let you guys become better data visualization experts. Help me to spread the word around.

—

Take care, have fun,
Enrico.

One of the most exciting and silent phenomenon I have seen developing during the last years/months in data visualization is the growing number of people who are transitioning or already succeeded to make a living by being a data visualization freelancer.

Who is a data visualization freelancer? Basically a self-employed person who sells data visualization services to companies and institutions. What kind of services? I don’t know … the sky is the limits.

There are people like Jan Willem Tulp, who designs pleasurable and accurate visualizations and is just turning into a full-time information visualizer. There is Andy Kirk, who writes the notable blog visualisingdata.com and teaches visualization in 1-day workshops. There is of course a veteran like Stephen Few who consults big BI companies like Tableau Software and writes among the best books in the field. And people like David McCandless who designs insanely popular (and discussed) beautiful visualizations.

Finally, there is Moritz Stefaner who works from home and seems to have the pretty rare ability to conjugate aesthetics with function and make everyone happy; as with the lately acclaimed OECD Better Life Index.

The brief story of me and Moritz at the airport.

And it’s exactly by discussing with Moritz that I came up with the idea of digging deeper into this fantastic world. The thing went more or less like that: I met Moritz at the airport, when we were both invited for a panel at Visualizing Europe, and during a casual chat he told me something along these lines: “You know Enrico … academia and research are cool but at some point I wanted to make real stuff for real people“. Yeah sure I understand. “… I always had small consultancy jobs during my studies so I had some experience … at some point I decided to become a data visualization freelancer“. Cool! “… so you know what? I work from home, I can plan the time myself  and make sure I play with my kids and talk with my wife” Uh!? “… and of course I am being successful and I am invited here and there so I am travelling a bit around the world and meeting interesting people“.

Ok, are you salivating already or what? I must admit it, despite I really love my academic job, I felt a bit jealous.

Send your questions! I will interview Moritz next week.

Ok so … I will be interviewing Moritz next week about data visualization freelancing. I started collecting a number of questions for him but I need your help! What would you like to ask to Moritz? What are you curious about? Is there a nasty question no one has the courage to ask? I think it would be much much better if you guys tell me what *you* want to know. So, don’t miss this opportunity. You could realize that being a data visualization freelancer is not a dream. It’s definitely possible! And Moritz can tell you how or at least provide some indications.

(On a side note, I will be experimenting with skype-based video interviews for the first time. I am totally excited by this new format and I’d love to have your feedback once it is done. The only video I currently host here is my interview with JD Fekete on Jacques Bertin, but the quality is really bad. I did some initial tests and the results look amazing. I really hope you will like it.)

Few additional reflections.

1st – Freelancing and working from home is not exclusive to data visualization, it is part of a bigger trend and it’s in my opinion absolutely awesome. The web is full of bloggers and small entrepreneurs that make a living by writing their blogs and giving their services by working from home. If you want to know more give a read to the super-successful 4-hour workweek. Note: some people love it and some people absolutely detest it, but no one can deny it captures a strong trend in our society. You can decide to ignore it or to read it and take the risk to have your life changed. You decide.

2nd – Being a professor used to be one of the best jobs in the world for the amount of freedom professors have (and still has a large degree of benefits in my opinion) but being a freelancer and working from home is a good competitor. Academia has its glow of knowledge and a little bit of mystery on its side but this whole segmentation of professions is going to disappear anyway. I see academia and freelancing somewhat similar as they both feature really special amounts of freedom. Academia maybe gives more opportunities to really do whatever comes to your mind but freelancing, as far as I can understand, has a considerable reduction in the amount of bureaucracy you have to bear with.

3rd – Most importantly, I believe freelancing is just a perfect fit for data visualization. I am not a big fan of generalist visualization because I think people get the best out of it when it meets the specific needs of a project. And this happens when you have a competent person able to listen, understand, and offer a tailored solution. For this reason I am a strong proponent of data visualization freelancing. It pisses me off that nobody is talking about it because it’s really a great trend. Plus, the more competent visualization designers we will have around, the more we will be able to show people great examples to take inspiration from.

Again: send  your questions!

Ok, let me repeat it again. I will be interviewing Moritz some time next week. This means the interview will appear here no sooner than about 10 days. I will be able to take your questions into account only if they come during the next few days. Don’t miss this opportunity, send your questions or ideas to me and Moritz. The easiest way is to add a comment here below. Otherwise you can contact us on twitter @FILWD and @moritz_stefaner or send me a private message.

Take care, have fun. We are waiting for your input!

A few days after my post on indispensable visualization I received an email from Prof. Georges Grinstein (what an honor! he is one of the fathers of visualization) asking for explanations about my use of the word “visualization consumerism”. I defined visualization consumerism as visualizations found on the web, used solely for communication purposes: mostly static, with little interaction and digested information.

Georges rapped a bit over my knuckles, and I understand why: ”The largest applications involves mapping. Can you drive without some form of a map? Yes, but even GPS dependent individuals look at the images most often. That’s great consumerism.”

Sure I see the point. I was a little too hasty in defining the whole set of visualization for communication “visualization consumerism” and I am ready to amend and apologize. But  ”visualization consumerism” does exist and in this post I intend to explain what I mean.

Defining visualization consumerism

I define visualization consumerism the careless production and shallow consumption of visualization. Consumerist visualizations are pre-digested depictions of data with the only effect of generating an “how cool!” effect. They do not inform, they do not let you think. They do not even pretend you to spend some time thinking. At best they entertain, but only for a few seconds: the time to click, give a look, say “cool”, and leave.

Visualization consumerism is an interplay between the consumer and the producer. It’s an attitude. It’s a feedback loop between the two, like consumerism in general. Eventually, you don’t know whether companies produce crappy products because consumers want them or the other way around.

Consumerist designers don’t spend too much time thinking or simply lack the knowledge, culture and attitude necessary to build “sophisticated” artifacts.

Consumerist consumers are people surfing the web, maybe with a slight interest in visual things, who stumble across a visualization and just say “cool!”. Again, they lack the knowledge, culture and attitude necessary to appreciate good design and to distinguish it from crap.

The origin of visualization consumerism

Am I a dogmatist, fanatic, orthodox, conservative visualization theorist? No, because I don’t blame anyone for visualization consumerism. I just think we have to recognize it and act in a way to let a larger portion of our society be able to make beautiful stuff and appreciate it.

Visualization consumerism is not different from many other effects we observe today, especially in the way we consume information on the web. Reading full articles has become a luxury, we just skim over everything, and of course we do the same with visualization. Finding good articles has become a luxury (by the way, this is why I strive to write long quality articles instead of a kaleidoscope) and again visualization is no exception.

Who’s to blame for that then? No one. Really, I firmly believe the large majority of people are honest and well-intentioned. It’s only the system we live in that produces these effects. And by the way I have no intention to write a long rant against modern society because I totally love the 21st century.

It’s also not my intention to point my finger to this and that designer so that I can generate a lot of voracious comments between two gangs: the purists and the creatives. I am personally annoyed by Stephen Few’s crusade against David McCandless, I think it’s detrimental, narrow-minded and, more importantly, disrespectful.

What is to be done?

Said that, what can be done? Should we just accept it and passively complaint about how mistreated “pure” visualization is? No. I think we have to acknowledge the problem and do our best to educate people. But wait a moment …. educating people is a dangerous idea! I agree. But let me explain what I mean. When I say educating people I mean doing it bottom-up; by giving the right examples and striving for creating a thriving environment:

1. Providing people with excellent study material
2. Showing people sophisticated and enlightening visualizations
3. Providing people with professional (and gentle) criticism
4. Showing our radical passion for excellence

I know. It’s tough. But that’s the deal guys if we want to do something. I know very well from my own experience how many moments in life lead us to produce and diffuse sub-optimal stuff. I don’t deceive myself with impractical ideals, and I am not ingenuous, but I am convinced we have to lean towards doing great stuff. Always. And this makes a difference in the long run, I am sure.

Is visualization for presentation consumerism?

Going back to the initial question and Geoges’ email, a clarification is due. No, I don’t think using visualization for communication purposes is consumerism. My initial reaction is due to the fact that I see a disproportion between the use of visualization as an exploratory tool versus a presentation tool. Up to the point that people might get an impression that communication is the only purpose of visualization. This is what Georges had to say:

“Visualization really fits into 3 classes which are exploratory, confirmatory, and presentation (see data vis book). Now exploratory visualization is only as good as the tool AND the analyst. A great analyst will be able to use visualization to generate many hypotheses. Once these are there then the next step of course is confirmatory and most often stats is the only too used. However combining stats and visualization provides a more grounded explanation of the variations in the numbers, more context, more connection with other hypothesis. Finally the presentation is what I would call your consumerism and we’ve not had many tools to make that fluid (from the exploration onto the presentation).“

Totally agree. What is missing is the connection between communication and exploration.

“We all encounter daily, daily, presentation visualizations. Most encounter confirmatory ones rarely as that’s the realm of researchers and statisticians. Very few encounter exploratory ones. But most often the presentation ones are the result of a great deal of exploration.“

So true! I’ve heard so many visualization designers talking about the painful exploratory process needed to produce a visualization for communication. And I can testify myself that this is always the case.

But again, what kind of impression do we want to give to people? Do we want to send the message that the main purpose of visualization is communication? Do we want people to only consume visualizations made by others? Or do we rather want to empower people with powerful “tools for thoughts“? I still believe we have to help people discover how visualization can be an indispensable tool for them. I want to realize the big vision of augmenting the human intellect not to making it flatter.

Is visualization for the masses consumerism?

In his email Goerges mentions “visualization for the masses” as one of the main challenges for visualization. It’s a buzzword that started circulating a few years ago when ManyEyes was launched and, as far as I understand, it means giving people easy access to visualization and enabling collaboration through visualization.

I must confess “visualization for the masses” does not make my heart beat faster (I am sorry Georges). I think the word “masses” is really unfortunate and resonates too well with consumerism. Who are these masses? Do we really want to give visualization to the masses? No. I personally want to give visualization to Paul, Cindy, Frank, and Anne … people with a real face and a specific need. I think it’s very dangerous to think of a generic audience in need of visualization, because there’s nothing like visualization for everyone.

My opinion is supported by the large failures we have seen in developing general purpose visualization services on the web like ManyEyes and Swivel. Even big and successful product like Spotfire and Tableau, which seem to provide general purpose solutions, started with a very clear target population in mind. The way I see visualization used most successfully is when it is designed and targeted to a specific population (often a small one) not for a general audience. Even less for the masses!

But if visualization for the masses means trying to push for the diffusion of tools that can support the full data visualization process and allow some people learn to use visualization and not only consume it, that’s definitely great! I am not against it. But I just won’t call it “for the masses”!

(This blog post is the result of a talk a gave a few days ago at Visualizing Europe. You can give a look to the program, you can give a look to my slides directly, or you can see a preview of the pictures taken during the event. Also, this is a quite long post, I am sorry. But I really needed this space to turn my slides into a blog post. If you don’t have time to read it now keep it for later.)

Background

I just came back from Visualizing Europe, a great event organized by Visualizing.org in Brussels to gather several key data visualization actors in the European scene and discuss the state of the art and potential future of visualization.

I had the honor to be invited in the panel session organized by Andrew Vande Moere to discuss the power and potential of visualization together with a bunch or great data visualization designers like Santiago Ortiz, Moritz Stefaner and David McCandless.

To tell you the truth I felt a bit like being the black sheep; but not with a bad feeling! I was the only one in the panel with a core CS visualization research background and the audience too was not to “researchy” as the usual audience I am used to talk to. What a wonderful occasion I thought!

So, when I got the invitation I started thinking about a talk that could send a strong message and stick into people’s mind. And suddenly I recalled I blog post I wanted to write since a long time about the questioned usefulness of information visualization; a complex that afflicts many of us.

I was originally inspired by a message I received from Jorge Camoes (ExcelCharts) in which he pointed me to a couple of interesting blog posts on visualization usefulness.

The first one is from Jorge and is titled: “Is Data Visualization Useful? You’ll Have to Prove It“. In this post he argues that believing that data visualization is useful is an act of faith and that we have no scientific evidence of its usefulness. Along similar lines the second post is from Stephen Few who in his blog post “True Stories about the Benefits of Data Visualization” confesses he has a hard time convincing people of visualization’s usefulness.

After reflecting a little bit about their content and their frustration I recognized the same frustration in me and decided it was time to react. And my reaction, I decided, had to be a strong one. I realized in fact that the whole problem of visualization usefulness can be removed if we change our mindset. I realized that one of the reason why we ask ourselves whether visualization is useful or not is because we don’t have a clear focus on those problems in which visualization is not just useful, but plainly indispensable.

In the rest of the post I’ll try to articulate this idea further, with the hope I will convince you it is important to change our mindset and focus more on problems where the net gain of introducing visualization is extremely high. But let me start by providing a few examples of problems I have personally experienced where I think visualization is clearly indispensable.

Examples of indispensable visualizations

In the following I provide three example of indispensable visualizations. I hope this will convince you that when visualization is indispensable it is evident.

Example #1: Bart and his moving animals
Bart is a biologist and he studies animals. More precisely he wants to understand how animals move and how contextual factors influence their behavior. Think birds migrating from one place to another exploiting the right wind at the right moment or lions searching for a prey and trying to optimize their energy expenditure.

Problem is that animal movement analysis today is a very technological endeavor. Animals have GPS and other sensors attached and can be tracked continuously around the world and generate enormous amounts of data. Plus, this data can be enhanced with contextual factors like weather conditions, making the whole thing more difficult. How do you deal with that? Can you do it without visualization? The problem is that not only they want to test some hypotheses and assumptions they have, but they also want to be able to come up with new ideas.

My colleague and dear friend Florian Mansmann (together with his valuable students) helps Bart and his colleagues understanding how these animals move, and developed a nice tool to visualize traces in time and to correlate them with contextual factors.

Again my question is: do you think it is possible to do that without visualization? I think this is another case of indispensable visualization.

Example #2: Joachim and his hidden molecules
Joachim is a chemist at the University of Konstanz, where I work, and his job is to study small molecules that inhibit a fundamental process in cellular division called “cytokinesis”. I am not going to give any biology lesson here, I am totally inadequate, but it suffices to know that  Joachim studies this stuff because it has implications on understanding cancer. Yes, cancer. Serious stuff.

Problem is that modern biology is done with computational tools and robots and he has to deal with tens of thousands of chemical reactions at once. He uses a technology called High-Throughput Screening by which a biological target can be tested with thousands of molecules in parallel in a matter of few hours. Isn’t it crazy?

But these data must be analyzed then and again there are no clear-cut preconceived hypotheses. Joachim has to delve through long excel spreadsheets and make sense of them. How primitive is this? And how indispensable is visualization here?

So we developed a simple tool to let him explore the association between the activity level of these molecules (how much they react to the given target) and their molecular structure. Nothing too complicated from the visualization point of view: a very interactive and flexible scatter plot with well-designed mappings.

When I showed the capabilities of visualization to him he was blown away, he couldn’t believe this was possible. Won’t you call this indispensable?

Example #3: Security officers keeping us safe
Even if we don’t notice it in our daily life, there are a number of people around us whose job is to monitor our infrastructures to keep us safe and to ensure that we have a steady delivery of fundamental services. We just started a new project here at the University of Konstanz to develop visual analytics tools to help this people monitor big amounts of information in real-time. These are for instance the guys who monitor the power grid infrastructure, a monster with millions of nodes spitting data 24hrs a day, and ensure that you can always recharge your PCs, switch your TV on, or heat up your food with your microwave. These guys do their job in control rooms like those depicted in this picture.

Of course, this data monster is by no means tractable by visualization alone, data analysis and reduction algorithm, as well as complex simulations are needed, but do you think it is possible to do without visualization here? Do you have the feeling it is conceivable to make sense of what is going on there without visualization? I don’t think so.

But that’s a handful of highly skilled people!

A common objection I get when I explain this theory is something along these lines: “ok Enrico this is true, but how many people are in the condition of desperately needing interactive visualization up to the point it is indispensable? The people you describe here are a few very skilled knowledge workers which represent a tiny proportion of the overall population! Visualization has the power to affect a much much larger group”

I agree. It’s actually true that these are a few skilled workers if we reason in terms of proportions. Nonetheless, we have to realize two things:

• The “tiny” proportion is millions of people. It’s not small! Jerome Cukier from OECD approached me after my talk and suggested: “You know what? I think you underestimated these numbers. There are 500 millions Excel users around the world!” Good point.
• The “tiny” proportion are those who work to make our life better. There is a whole army of skilled professionals who study to:
  • find solutions to cure our diseases
  • find solutions to reduce poverty
  • find solutions to make our planet cleaner and safer
  • find solutions to provide innovative and pleasurable products
  • find solutions to keep our cities safe
  • … add another thousand rows

Shouldn’t we spend some time to help these people? I think we certainly have to. If we have the tools with the potential to expand their mind and do great things we have the responsibility to do it!

Visualization use and the cartography cube

Finally, I think it’s necessary to reflect on how we are using visualization today, how we could use it in the future, and how this is related to indispensable visualization.

Fortunately, while thinking about the idea for this talk I stumbled upon a very useful diagram of visualization use. I found it in Alan MacEachren’s “How Maps Work” (by the way, this book is blowing my mind, it’s not at all only about maps and if you are very serious about visualization you should read it).

The model describes visualization use according to three main dimensions:

1) Private vs. Public: Is the visualization targeted to one person or a restricted group of people to solve a specific problem or it is more intended to address public audience?

2) Static vs. Interactive: Is the visualization a static representation or an interactive tool that people can use to explore alternative views?

3) Revealing Knowns vs. Revealing Unknowns: Is the goal to communicate a number of findings or messages to an audience or to provide people ways to explore the data and make sense of it?

My feeling is that visualization today, especially if seen under the lens of what the web and similar media offer to the average reader, is largely used as a communication tool, that is we are at the upper corned of the cube: largely public, mostly static or with little interaction, and mostly to reveal information that has been digested by someone else.

I call this use of visualization “visualization consumerism” (I hope nobody is offended by this name, it’s not my intent), that is, the visualization is prepackaged by someone in a format that can be easily and quickly digested by a large number of people. To be clear, I am not against it, as long as it is done well. Visualization Europe was full of fantastic designers and I love their work. But I think we should do more. Visualization is not only something to consume, we need to look at it in a different way.

What I propose is that we take more care of the knowledge workers I mentioned above and provide them with the indispensable tools they need. I call this: cognitive cyborgs. I think we have the great chance here to help people become cognitive cyborgs. Visualization, as I explained in an older post, has the power to make us more intelligent and we should strive to do that.

And looking at the cube I think we have to shift our attention to the opposite corner of the cube if we want to help people become cognitive cyborgs.

We have to build visualization for private use, highly interactive, to allow easy exploration and focus on revealing unknown, that is, help people generate new knowledge.

Conclusion

I will be totally candid with your here. It’s evidently a bold statement to claim that visualization is not useful but indispensable. To some extent it is provocative on purpose. And it is also dangerous to claim that visualization is indispensable, because some problems might benefit from it but still be solved without it. But I really think it is important to change our mindset and focus more on the impact we might have.

I confess I believe it’s not a totally black and white here, there are several shades of gray in between. But I see a huge opportunity cost here: the more we spend our energies on communication issues, the less we can spend on putting these tools directly in the hands of the people who need them. I see an army of incredibly skilled visualization designers that could do a lot more if only they had the opportunity to work on projects where visualization is really indispensable.

The problem is that this opportunity is not easily visible because many of the people we could empower with our tools don’t know it! That’s the real challenge.

I know, this is a very personal point of view and I am ready to accept some criticism, but that’s it. That’s the way I think and I would love to hear your opinion if you do not agree with me.

I personally don’t want to see visualization trivialized to a mere communication tool of digested data. I want to see cognitive cyborgs all over the place. They are everywhere guys! They are waiting for us but we have to chase them. Let’s make this happen, there are several opportunities at stakes here! Good luck.

If you are a regular reader of this blog you might have noticed how easily I fall into the trap of going into long rants against this and that. Not too big a problem, rants seem to work well in the blogging arena, but I realize that while I have some good criticism here and there, I almost never ever spend time praising someone else’s work. This is partly due to my not being too excited by the average visualizations I encounter on the web, but rest assured there is good stuff to talk about nonetheless. And here we are. I am very pleased to break this habit today and spend some time praising a recent project you might have noticed on the web: the OECD Better Life Index. Let me first explain what it is and then I’ll tell you why I think this great work.

The Better Life Index

The Better Life Index is a visualization developed by the OECD (Organisation for Economic Co-operation and Development ) under the supervision of Jerome Cukier and his colleagues and the help of two external designers, Moritz Stefaner and RauReif. You can find a very interesting description of the the process and the role of the designers in Jerome’s post mortem article.

“The idea was to come up with some kind of Progress Index, which we could communicate once a year or something. Problem – this was exactly against the recommendations of the commission, which warned against an absolute, top-down ranking of countries.

Eventually, we came up with an idea. A ranking, yes, but not one definitive list established by experts. Rather, it would be a user’s index, where said user would get their own index, tailored to their preferences.”

So, the OECD had a bunch of indicators summarizing aspects that concur to quality of life and happiness and wanted to find a way to represent this data in a way that makes people compare one country to another, but in a flexible and not totally pre-ordered way.

The final result, after a number of iterations is a interactive visualization based on icons (they call it flowers). The user can give a weight to the various factors building the index and see how the countries change their ranking. If you want to better understand how it works give it a try, it’s much easier than trying to figure out how it works from my description.

Here is the default result taking into account the whole set of factors.

Here is my own Better Life Index.

What I like and why

Why do I like this visualization so much? I have a number of reasons that I will list in a minute but in summary I like it because it’s a an example of how function and aesthetics can serve each other’s purpose.

There have been endless (and useless IMHO) discussions about form and function in visualization, where people are always a bit right and a bit wrong at the same time. My take at it is that when something works you can see it, no need to spend a thousand words on it. The Better Life Index, is beautiful and functional at the same time and this is the way to go.

The first and most important reason why I like the BLI is that it uses a technique I technically call “multi-dimensional icons”, probably the most neglected tool in the history of data visualization. The thing they call “flowers” it’s actually an icon (some core vis people would rather call it glyph) representing multi-dimensional data.

Every country is an icon, and each icon is made out of a number of “petals”. Each petal represents a data dimension (a better life factor in this case) and its length is proportional to the value the given country has on the corresponding factor. Plus, each factor is colored with a unique hue.

The same technique can be used every time you want to represent multiple dimensions of a series of objects at the same time. If you think about it carefully you will see that there are not many alternatives around. The only problem of this technique is that it doesn’t scale to a large number of objects, but this is not a problem here.

I also like the fact that color was used redundantly to represent the data dimensions. In principle each petal has a unique angle and position so it shouldn’t be a problem to identify which petal is which, but with color it obviously works much better. Plus, it works nicely as a cross-reference to the legend.

Using height as a way to convey the total score is a good choice. You can also change the ordering of the countries from alphabetical to by rank, which I actually prefer (see the gap in the middle now?)

Position is the most powerful visual primitive we have and it makes sense to use it to convey the most important information: how the countries score according to the selected factors.

Speaking of interaction, it is quite simple but effective. It’s easy to understand the function of the menu on the right hand side (see it on the web site) and as soon as you use it you get an understanding of how it works. The animation makes the whole thing smooth and calm, giving the feeling that it’s always possible to return on your owns steps and make several experiments.

To some extent it is possible to get the feeling of which country changes the most when a new set of factors are used, but it doesn’t really work too well, as following all the movements at the same time it’s not easy.

This is the only limitation I have found so far. I would like to have a way to better understand how the rank changes when a new set of parameters is chosen. But I cannot really blame anyone for it. This is the kind of design where the more you try to add the more complex and less appealing it gets. Adding such a functionality would require a lot of additional complexity and there’s no simple and elegant solution that comes into my mind right now.

Money doesn’t buy happiness

I have been playing with it for a while and if you didn’t do it yet I encourage you to try it out. It’s a lot of fun! Try to ask yourself some questions or see whether there is anything strange you notice. I have a number of interesting things I’ve noticed or learned but this one is the one that really stands out for me: the best countries score quite badly in the income domain.

Regardless the composition of factors you choose, there are always a bunch of countries that are leading the BLI, e.g., Norway, Sweden, New Zealand, Canada. Give a look to their flowers: even if they are almost always at the top, they score very badly in one domain: income.

Where should I live?

I don’t know how you interpret the data depicted in the BLI. Personally, the first reaction I had was: “ok then … so where should I live next?” And driven by this idea I started exploring several alternatives. But after a while I felt like two main things were missing.

The first one is information about weather. While in fact I give a great deal of importance to things like work-life balance, I often find myself thinking about weather when I think about a potential new country to live in. Maybe this is the heritage of my Italian origins, but it’s crazy to see how good weather is poorly correlated with the average well-being of a country. Isn’t it? I think some information about weather would be a great addition to the BLI. I would expect to see a lot of shift in the ranking and … well I am sure Italy would score much much better!

Another subtle but more important factor is that it’s really hard to judge quality of life in terms of whole countries. Take the USA, can you really consider these parameters descriptive for any of the main cities in the U.S.? I am sure there are cities that score a lot better and a lot worse than the average. And probably the same is true for any other country. I think it would be really fantastic to see the same data on a city level. Do you think this is even possible? Is the data out there? I would be much more interested to know how the BLI is in Rome, Berlin, Toronto, New York, etc … because in the end when you move to a new place you move more into a city than into a country.

Conclusion

In summary as I said I think the BLI is a very nice example of simple and effective visualization. It takes into account all the constrains posed by the project and comes up with a little nice tool that works just right for its purpose. Is it the best for everything? No, it is not of course. But it’s very well done for the goal people at OECD had. Everyone can learn from it.

My sincere compliments to Moritz and Jerome with whom I often interact on Twitter, and to all the other guys who worked on it. Very well done guys!

—

Did you like this review? Please do not forget to tweet about it and/or comment on it below. Thanks!