I just came back from CHI 2013, the premier conference on human-computer interaction (Paris was chilly and expensive. Yet, dramatically beautiful, as always). Here is a selection of interesting visualization papers I picked up from the program.

Using fNIRS Brain Sensing to Evaluate Information Visualization Interfaces. Interesting study from Tufts University on the feasibility of using brain scanning techniques to study mental workload in visualization.

Weighted Graph Comparison Techniques for Brain Connectivity Analysis. Excellent study on the ever-lasting battle between node-link graphs and matrices (to visualize weighted graphs in this case). Matrices win over node-links almost in every task. Very good example of exploration and evaluation of a specific design space. A lot to learn here.

The Challenges of Specifying Intervals and Absences in Temporal Queries: A Graphical Language Approach. Visual and interaction design study to allow end-users (doctors in this case) to specify complex temporal queries without writing a single line of code. It makes me think how visualization can and should be used not only as an output device but also a way to facilitate inputing data into a system.

Evaluating the Efficiency of Physical Visualizations. User study comparing 2D and 3D bar charts on a standard computer display to physical bar charts fabricated with a laser printer. Physical 3D is more effective than display 3D. Why? See the paper. (Side note: we featured this work in a Data Stories episode on Data Sculptures)

Contextifier: Automatic Generation of Annotated Stock Visualizations. Automatic annotation of stock market line graphs by extracting text from news articles. Annotation has been neglected for a while in vis (maybe because text is not considered part of the visualization?) but I think it’s super important. This is a great first step in the right direction.

Motif Simplification: Improving Network Visualization Readability with Fan, Connector, and Clique Glyphs. We all know how easily graphs can turn into hairballs. Motif simplification is a smart way to reduce the complexity of graphs by aggregating nodes into predefined glyphs.

Evaluation of Alternative Glyph Designs for Time Series Data in a Small Multiple Setting. User study on the comparison of icon-sized time-series visualizations. Two aspects are evaluated: layout (circular, linear) and value coding (length, color intensity). The study leads to a number of design guidelines (and hey … I am one of the co-authors here :))

—

I hope you’ll enjoy reading these papers. There is a lot of food for thoughts here. Comments, requests, criticism, always welcome.

Take care.

I don’t  (want to) buy the idea that it’s too hard to quantify/demonstrate impact of visualization. Yet I want more evidence/stories.

He is my very humble initial step (form your comments and twitter messages):

• Persuasive graphics in Al Gore’s An Inconvenient Truth
• Hans Rosling’s explanation of world development with Gapminder
• Florence Nightingdale’s Mortality Diagram
• John Snow’s Cholera Map
• Tableau having a $127M revenue in 2012 and going public
• >50k downloads of ggplot2 in the last 4 months
• Better diagnosis for hearth disease
• …

What else? Please send me more!

I see a lot of visualization around me now and I am extremely excited about it. Yet, are we making any real difference? I mean, are we having any real impact in people’s life other than telling them beautiful stories?

Yes I know, impact could be defined in a million different ways and it may be hard to capture. But why? Why I never stumble into an article or blog post showing, I don’t know, for instance, how visualization helped a group of doctors doing something remarkable with visualization?

Is it just because this stuff does not get reported or what?

Here are a few possible explanations:

• Explanation#1: Impactful visualization is hidden. Those people who are using visualization successfully, who have a real impact, are too busy to report their success.
• Explanation #2: Visualization is just a fragment of a much larger process. Visualization, when is not used as a communication/story telling tool is part of a much larger process, which includes many other steps and tools so simply success is not ascribed to visualization.
• Explanation #3: Visualization impact has yet to come. Maybe we just have to wait a bit longer and we’ll get all the success we want.

What do you think? Do you have other explanations? Is my question just too pretentious? Or did I just miss a ton of success stories and this post is totally nonsense?

P.S.1 On a side note: other areas of data analysis, especially automatic approaches like machine learning and data mining have plenty of stories to tell. Why? Food for thought …

P.S. 2 After writing this post I discovered my friend Andy Kirk has written a much longer post on this issue.

Being a visualization researcher means a very large body of my work revolves around pushing the boundaries of visualization further. I do that by mostly developing innovative techniques but also trying to better understand how humans interact with this amazing tool we call visualization.

You might think I have at least a rough idea of what progress means in visualization then, but in fact I don’t. And I guess I am not alone: researchers are trained to dive into tiny details and speculate for ages. The purpose of this post is to explore bigger questions:

• What is progress in visualization?
• How do we make progress in visualization?
• And how do we measure it?

I ask that because honestly I don’t see a direction in what we are doing. We researchers are mostly focussed on developing yet another technique, practitioners on (understandably) satisfying their customers. But what is our ultimate goal? Here I propose s few ways we can look at progress in visualization.

Progress As Real-World Impact

First and foremost I propose progress in visualization is the extent to which we are able to help people do remarkably useful things with data. This is for me the gold standard, the holy grail. It is a broad and vague definition but it helps. When I say “remarkably useful” I mean: can we say visualization played a critical role in curing or preventing diseases? Reducing poverty? Solving or preventing economic crises? Make people richer or happier? Etc. Think about it, why not? Why do we do visualization if not for these purposes?

Despite some few isolated cases I don’t see this happening now. We should keep our eyes open and focus more on having an impact in the real world. Visualization has this potential, I am sure, and progress is made, I believe, when we help people do remarkable things. The VisWeek conference used to host a very nice session called Discovery Exhibition with the specific intent to showcase success stories. Unfortunately, (its hurts to admit it) I think it was quite a failure. I remember a similar frustrating post from Stephen Few some years ago: “True Stories about the Benefits of Data Visualization“. And I have yet to see persuasive answers to his call.

Progress As Knowledge Construction

I have to admit measuring progress exclusively in terms of impact and success stories might be a bit fuzzy, not very practical and ultimately a bit subjective. Another possibility is to define progress as the accumulation of knowledge that permits to build more effective visualization. But what do we need to know that we don’t know yet? Broadly speaking we need to know:

1. How humans work.
2. How to translate knowledge about humans into visualization design.

Are we doing that right now? Partly, in academic environments and a bit outside, but not enough in my opinion. It’s surprising to see how much more foundational work has been done in the past and how little today. We have a rough idea of how visual variables (position, length, color, size, etc.) work in isolation but very little understanding of how they interact in complex environments. We have alternative visualizations for the same kind of data and little understanding of how they influence information extraction (parallel coordinates vs. scatter plot matrix? node-link diagrams vs. matrices? maps or abstract representation? animation or small multiple?) And we have not even started scratching the surface of muddier issues like semantics, influence, persuasion, etc.

Progress as Technical Achievement

I don’t even know if I need to comment on this one, it’s pretty straightforward: technical achievement is the development of visualization and interaction techniques that solve unsolved technical problems or improve performance over existing solutions. Typically this takes the following form:

• New visualization or interaction design.
• Faster and/or more accurate algorithms.
• Increased scalability in terms of data size and dimensionality.
• Accommodation of new data formats and tasks.

I think it’s safe to say academic research is mostly focused on this. I am not sure whether technical achievement translates into real benefits in real-world applications but from time to time we have really useful stuff coming out. Edge bundling and horizon graphs are the first things that come into my mind. Are we making progress in this area? Yes. Would I like to see more? Yes and no … In a way sometimes I feel like we are spinning the wheel (please note that I include myself into this description and I am not immune to many many faults) so I’d like to see less spinning-the-wheel technical contributions and more useful stuff. But I also realize we cannot invent a new edge bundling every year. Progress happens with valleys and peaks.

Progress As Education and Adoption

Maybe this is the most neglected kind of progress, yet it very much lies at my heart. The last way to define progress in visualization I propose is the extent to which we are able to teach people how to judge and use visualization effectively and how many people will use visualization in their work. We need to reach more people (visualization at school?) but more importantly we need to teach proper visualization. We need courses, seminars, teaching material, web sites, and a whole army of evangelists. I am lucky enough to know quite a bunch of them but we need more.

I want to measure progress in a few years by counting how many people are able to criticize a chart. I also want to measure progress by assessing whether visualization will be part of the standard toolbox of scientists, business men and decision makers around the world.

Conclusion

This is what I had to say about progress. I know it’s not perfect, it’s just a draft. And now it’s your turn. How do you define progress in visualization? Are we making progress? How would you measure progress in visualization in, let’s say, 5 or 10 years from now?

And by the way, do you care about making progress? Why not? It is not necessary to be “a researcher” to make progress, you can make progress in a thousand ways. The only thing we need is to bring more focus. Or maybe we just have to let things happen and have some fun? I am looking forward to hearing from you guys. Thanks for reading.

—

On a side note: I have been out of the scenes with FILWD for a very long while. There are good reasons why that happened (I’ll tell you more about that later) but I want to assure you FILWD is not going to fade away. To the contrary, I have many plans on how to grow it further and offer a better service. If you are still there reading me after so much time well … thank you so much from the bottom of my heart! -Enrico

I was reading the description of a new data visualization contest coming out today, the Nielsen Data Visualization Contest, and an apparently insignificant sentence caught my attention: “The challenge is to make data tell a story, conveying what’s most important effectively and efficiently.”

There is a lot of attention lately around using visualization to “tell a story” and I can understand why: visualization, when designed properly, has a tremendous effect on people. Not only it has the power to convey a clear message and to make complex concepts very easy to grasp, but it also has the power to persuade. I guess the main reason being that when a statement is backed up by data then people believe it is true(er).

I have nothing against using visualization to tell stories, to the contrary I am fascinated by this use of visualization and I think it’s very relevant. For instance, raising awareness about important facts or democratizing access to complex information are very noble intents of visual story telling, and I fully support them.

But, I don’t know, call me old-style, conservative, bigot: I am concerned by an excessive focus on story telling. It’s an itch I cannot scratch. And because I cannot express it in a closed form the only thing I can do is to make a list of concerns I have (hoping your comments will make it easier to dispel the fog).

There’s no story telling without data exploration. Creating a story with visualization doesn’t mean there is not role for data exploration in visualization in its making. People looking at the final product might think the power of visualization is exclusively in the effective presentation of the facts. But what people don’t see is the amount of exploratory work behind every story. I know as a matter of fact that many great visualization designers start with a thorough visual exploration of the data at hand using standard tools like Tableau or R. Without this preliminary phase it’s very hard to tell a compelling story and it is also very hard to come up with an enlightening visualization.

It’s the data that makes the story not the visualization. I always laugh a bit when people complain about David McCandless’ work. They say that their visualizations are not optimal and that he makes many “mistakes”. In a way I agree but why does he have such a big success then? I think the reason rests in his ability to select amazing stories to tell. The story is hidden in the data. Well, not even in the data, I guess everything starts in his mind, the rest just follows naturally. So, if we are passionate about visualization and dare about its proper use I believe story telling is (maybe) not the most challenging area to test it.

Many people need visualization to build our future not to tell a story. While I cannot resist a catchy well-crafted data visualization that tells a compelling story, I also know from my experience how desperately professionals of all kinds need visualization to just do their work best. I am talking about doctors, engineers, biologists, policy makers, etc. Part of our life, or of our future generations, might depend on them and we have the opportunity to help them help us. Don’t you think this use of visualization is a bit under represented on the web when compared to the whole set of story telling visualizations out there? For instance, why don’t we have contests to help these people with their data and have plenty of those asking to vaguely find a story to tell in this or that data set?

A story is not THE truth. I have no evidence for that but my feeling is that visualization can be used to more easily persuade people. By the mere fact of being built on top of data people might think it is truer than other kind of stories. Again, you can see that in McCandless’ work. Many of his pieces are evidently conceived to be provocative and touch hot topics. But I bet that for every provocative visualization out there there is the possibility to build a counter argument with another one. I might be proven wrong on that but I haven’t seen any evidence on the contrary so far.

Not all stories are worth telling. Since the power of a story resides in the data, it is not always possible to tell a compelling story. Regardless the beauty or inventiveness of your visualization if the data is dull you might not get a compelling story. And I have experienced it so many times that I am almost inclined to say that this is pretty much the standard for any given data set. You can see it in the recent Information is Beautiful Award: there are many cool and pretty entries, some that I really like from the design point of view, but is there anything really interesting there to see? Do we leave the stage enriched by new knowledge?

As promised yesterday here is the answer I received from Visualizing after sending them a draft of my post. Given their answer and the whole bunch of controversial but constructive comments I received (check them out, they are full of insights) I am really glad to have started this. I have the feeling this can in a way help all of us, regardless our opinions, make the whole field at least a tiny bit better.

—

Enrico,

First, thanks very much for taking the time to share your feedback and for your thoughtful suggestions. We’re all committed to advancing the field of data visualization and healthy debate towards that goal is always useful.

The aim of the Visualizing Marathon program, which we started in 2010, is to encourage design students around the world to take up data visualization and generally to use design to help improve our collective understanding of complex world issues. The structure and format of the event is constantly evolving in support of this aim based on what is working and not working (we collect surveys from the students, for example) and so we’re most grateful for any and all feedback!

To your specific points:

1. Judging: All Marathons (and challenges) are judged based on three criteria, which we’ve previously shared on Visualizing:

• Understanding: How effectively does the visualization communicate? How well does it help you make sense of this issue? (out of 10 points – we agree with you this is most important and that’s why it gets the most weight)
• Originality: Are the approach and design innovative? (out of 5 points)
• Style: Is the visualization aesthetically compelling? (out of 5 points)

Our global jury selected 1 Winner and 2 Honorable Mentions in each of the 5 cities. These are the top 15 projects based on these metrics.

Importantly, however, the Grand Prize was selected by us out of the top 15 based on a different metric: how well does the project help illuminate new insights to the complex problem the students were given (in this case, sustainable development). Because data visualization is not only a tool for communication but also a tool for exploration, we sought to highlight and amplify the latter with this particular prize. It’s why it comes with a $10,000 grant to support further research and education. We felt the winning project best delivered on this metric (the approach and analysis detailed in their accompanying essay is particularly noteworthy). And as we noted in the prize announcement, we hope very much that the students use the additional time and resources they have been granted to take the visualization further (including putting their 3D shape to work as outlined, and perhaps evolving a simpler overall design). Also, we’re sure they would enjoy hearing suggestions directly.

We have been exploring how we might incorporate a “People’s Choice” aspect into the program, though there are some potential complications with this format that we are trying to be mindful of.

2. Time: There is no question that time (usually) improves quality – and our Visualizing Challenges, for example, typically run 4-6 weeks based on that logic. With our Visualizing Events, like Visualizing Europe and the Visualizing Marathons, we want to create the space and opportunity for people to come together in a shared and collaborative environment where they can meet, learn from one another, and develop new partnerships/relationships. We hope that after each event, the conversation continues in a way that can push forward the field of data visualization. We know from direct feedback from students and their professors that there is a tremendous didactic, creative, and inspirational value in working together with 2-3 of your friends in a common space with other students for 24 hours towards a common goal (we are also mindful that there is a real limitation to the amount of time students can commit to an extra-curricular activity). As you rightly pointed out, there are high quality projects among the entries, so clearly it is possible to produce something of quality in the allotted time. We also believe that overall quality from students will improve year over year as the professional field and its accompanying science mature and codify what works. That said, we are in fact experimenting with time this year to help improve overall quality and welcome any suggestions.

3. Training: We agree that training is important. In the spirit of openness, we allow students of all levels and disciplines to participate in the Marathons and learn by doing. As you mentioned, data visualization has become mainstream only recently, especially in some of the cities where the marathons have taken place. To provide greater training before the Marathon, this year we just have started providing registered students with various resources and helpful links (including this one) well in advance to encourage them to learn more about data visualization. And since the beginning of the program, we have been running data visualization lectures and workshops hosted by design professionals during the Marathons to teach best practices (based on feedback, we recently moved these lectures and workshops to the start of the marathon program so lessons can be incorporated from the outset).

Again, thanks Enrico for all your support. As we are ever committed to developing the best possible Marathon program, we’re very much open to ideas.

The Visualizing team

—

Thanks Visualizing for accepting openly my criticism. I think this is simply great!

“Huston we have a problem …”

I just received this in my inbox:

We want to again express our sincerest gratitude for your help in making the Visualizing Marathon 2011 such a resounding success. Your participation was instrumental and the 376 students who competed in Sydney, São Paulo, New York, London, and Berlin told us how excited they were to meet you and have their work reviewed by such an esteemed global jury.

We just announced that the winner of the $10,000 “Imagination at Work” Grand Prize is Columbia University for E-Cube-Librium [...] Out of 15 finalists, the Grand Prize was awarded to the project that “best illuminates a new insight or solution to a complex problem through data visualization.“ [bold is mine]

I receive this because I was part of the jury for the Marathon in Berlin. Visualizing Marathon is a series of events (inspired by the more famous hackatons) organized by Visualizing.org around the world to promote visualization. Groups of students develop a visualization for a given dataset/problem in 24 hours and Visualizing.org gives awards to the best entries.

Being a juror was fun and and an honor for me, as well as being one of the speakers at Visualizing Europe last year. I am grateful to Visualizing for the great work they are doing in terms of promotion, and also for their commitment to building a solid platform for visualization designers. Nonetheless, I think we have a problem.

I look at E-Cube-Librium  and I cannot help but think: “Is this the best 376 students from all over the world can produce?“ It just doesn’t match with my definition visualizations that “best illuminates a new insight or solution to a complex problem“.

I am really sorry I have to say that, especially because I am sure the students did their best and are probably proud of their work, and also because I am sure the guys at Visualizing.org have the best intensions in their mind. But I am also concerned that people around the world would look at the best prize winner and think this is the gold standard of visualization. We have to be careful, especially now that visualization is in the mainstream, about what message we give. I have seen too many times visualization dismissed altogether because people think it’s only pretty picture. Our reputation and future is at stake here.

Now, we have a nice series of events organized around the world and I am all in favor of data visualization evangelism, but why are the results disappointing? Is it an intrinsic problem of marathons and contests or maybe we can engineer the whole thing to make it more effective? Here are some potential explanations I can tell from my experience:

1. Time is too short to produce quality results. Every time I complain about the quality of the results there is someone who points out that time is too short. I am not fully convinced time is the main problem however, even though I do think time is too short. Basic design choices do not depend on the amount of time. It doesn’t take time to know a 3D visualization of numeric data should not be your first choice when designing a visualization, it takes knowledge.
2. Students are not well prepared. That students are not knowledgeable enough to produce quality results is not surprising. Visualization has become mainstream very recently and there is not a clear path to follow if one wants to become an expert. Nonetheless, some of the entries I personally reviewed as a juror were more than reasonable, especially given the 24hrs constraints! Also, giving a look to the page with the whole set of winners and honorable mentions it’s surprising to notice how neat solutions coexist together with very questionable ones.
3. Jurors select the wrong entries. Another possibility is that jurors just pick the wrong entries. I don’t know who selected the grand prize winner, I was not involved in the process, but my feeling is that here we might have the biggest mismatch. When I participated as a juror it became clear to me how things can go wrong. Some people put clarity and information throughput before everything else (guess who?), others judge things from their coolness factor. I know, it’s sad but that’s the way it is.

• Give more time. If time is too short why not giving more time? The marathon format does not lend itself to data visualization. Visualization is a process, a tortuous process actually, with lots of dead ends along the road. Pretending to visualize data effectively in 24 hours might be an unrealistic goal.
• Train students before the marathon takes place. If students are not good enough why not giving them some training before running the marathon? There are many professionals out there who are able to explain in a concise manner what are the no nos and the good practices of visualization.
• Run marathons without prizes. Maybe a marathon could be held without giving a prize? I don’t know … is the perspective of receiving a prize that motivate students to do their best? Maybe not. Maybe just knowing that they will have the opportunity to get trained by a professional and to have a certain level of exposure will motivate them enough. I think competition is totally overrated.
• Let people judge in place of jurors. One option could be to have “better” jurors but then we would have to discuss what we mean by “better”. As an alternative, why not letting people judge? I am not sure the result would be better but at least we could claim it is a democratic process and it wouldn’t embarrass any jurors.

And you? What do you think? Do you have any concerns with contests and marathons? How would you shape your own marathon event? Do you have any suggestion on how to improve the situation? I’d love to hear your voice.

—

IMPORTANT NOTE: I had the chance to discuss with some people at Visualizing before publishing this post. Since I totally respect their work and wanted to avoid slashing them with an overly unfavorable post, I decided to let them read it before publishing it. Apart from a few sentences here and there the post is still the same as the original draft.

Charlene Manuel was also so kind to send me a long reply to this post which I decided to publish soon as a post rather than a comment so that everyone will get the feeling of how Visualizing is handling this criticism.

I am very satisfied with this process. I think we all have to be happy to see that it is possible to have constructive criticism and make the whole field thrive without unnecessary battles.

—

UPDATE: here is the answer from the Visualizing team.

When I think Visualization and Excel there are two names that come into my mind: Jorge Camoes and Jon Peltier. If you want to do serious data visualization with Excel, stop here, they are the names. Since I was more familiar with Jorge’s work and had more opportunities to discuss with him I decided to interview him to cover the Excel part of this series, but you can give a look to Jon’s web site if you have any additional questions.

Jorge had been developing visualization in Excel for a long time now and I still remember the time when I saw one of his dashboards in Excel: “Wow, can Excel do that?” Give a look with your eyes to his dashboard courses. Pretty amazing isn’t it?

I have been following Jorge’s blog for a long time now and I often enjoyed his short and catchy blog posts. If you are not following him, give it a try. It’s worth it.

How did you start using Excel?

I started my professional career as a desk researcher. I had to create information products with lots of charts using market and socio-demographic data, and Microsoft Office was the only tool available. Like everyone else, I had no data visualization training, so you can imagine how bad those charts were. On the one hand, that’s very depressing, from a personal point of view. On the other hand, this proves that data visualization skills are easily acquired, once you become aware of what data visualization is all about.

What’s the best and worst aspect of Excel?

We must emphasize that Excel is not a data visualization tool, so you cannot directly compare it to other tools. That said, you can learn and practice sound data visualization principles using Excel. Its chart gallery is poor, but you can make new charts using some more or less clever tricks. Check Jon Peltier‘s site to see how you can extend the Excel chart gallery. So, its flexibility and general availability are the best aspects.

Unfortunately, defaults that emphasize marketing and sales pitch are responsible for a generation of users that don’t really know what a chart is. That’s the worst aspect of Excel. Also, because of it’s flexibility, many users do not recognize that they need stronger data management skills.

How is the learning curve vs. return-on-investment of Excel?

Most business users have access to Excel training. They just need to be brainwashed to remove all they think they know about charts. :) Corporate data visualization culture is so poor that applying simple rules can greatly improve insights and ROI, and you can do it using Excel.

Ok, I am a beginner and I want to learn Excel, where do I start?

Chances are, you already know Excel. If you don’t, I’d recommend Chandoo’s Excel School or Daniel Ferry’s Excel Hero Academy. And Jon Peltier’s site, mentioned above.

What other tools would you recommend other than Excel?

90% of all charts you need in a business environment can be done in Excel. But if it takes a full day to code a chart that you can do in minutes using a different tool you have a good argument to make the switch. I would recommend Tableau, Qlikview or Spotfire. They are well-aligned with currently accepted data visualization best practices and they force you to learn more about structuring your data.

—

Some comments from Jorge … and my answers:

• J: Business users hate programming. You can’t explain a product manager that a simple recorded Excel macro can make all the difference. You can’t tell them that they need programming skills to make a chart.
  E: I think this is totally fine and probably a reason behind the big success of Tableau.

• J: If it can be done in Excel managers will not spend more money getting a new tool.
  E: Ok, but then they have to be ready to pay someone to let Excel do the job right? I see a great potential for consultants here.

• J: But managers are becoming aware that they need a serious (visual) reporting tool; Tableau is one of the options; traditional BI tools are moving fast. If a BI tool supports sparklines that’s a good starting point.
  E: I think managers will feel more and more pressure as visualization becomes mainstream. I think we just have to wait a little to see some stuff flourishing. I am not too pessimistic.

• J: I believe tools matter, and matter a lot. Tools are not neutral (Tufte says that regarding Powerpoint). If you have to fight them they’ll make your life miserable. Try to apply Tufte’s principles to Crystal Xcelsius. I already wrote about this in my blog using fable about the scorpion and the frog (“it’s in my nature”).
  E: Sure, tools matter. Especially if you know how to switch from one to another according to your needs. Nonetheless, I still believe principles come first. And in order to select the “right” tool and understand its limitations you have to have a clearer idea of what you want to achieve.

• J: Life is short: I would argue that it’s better to learn about perception, statistics, data management and graphic design. Delegate the programming part. I’ve been making some dashboards and I spend more time programming than exploring better ways to show the data. Hate that.
  E: Cannot agree more. Eve though I think we are still in a phase where it’s really really hard to split between the designer and the implementer. The two things are so intertwined that trying to outsource the implementation may very easily lead to unsatisfactory results. But sure, the real skill is in the design IMO.

• J: If you don’t include R in your list you’ll get into troubles :)
  E: Sure! It’s in the pipeline :-)

Yesterday I wrote this on twitter: “I must confess I very rarely read data visualization blogs, most are depressingly predictable and shallow.” Yes, it’s not the nicest sentence I could write, but it’s true: most data visualization blogs suck. They do not inform, they do not entertain.

At VisWeek, last month, we organized a pretty successful Birds-of-Feathers (BoF) titled “Blogging about Visualization”. I and Robert advertised the thing a bit and we managed to gather a pretty cool bunch of people around a table. We spent at least a couple of hours all together and then we enjoyed a wonderful dinner at a Greek restaurant.

During the BoF we discussed several aspects related to blogging (check the nice summary wrote by Dominikus to know more) but what struck me the most is the following: (1) people desperately want to know how to succeed with blogs; (2) people think it is a sort of black art when in fact it’s only a matter of mindset and hard work; (3) there are endless possibilities to open new blogs.

Yet, the decent blogs around can be counted with the fingers of one hand. And I want to see more great stuff, because either we grow as a community or nobody will grow. Here are some personal thoughts about blogging and a number of tips I want to share with you, hoping they will convince some among you to open the best data visualization blog ever.

The Data Visualization Showcase is Dead

When I think about why many data visualization blogs are so useless, the reason number one that comes into my mind is that they try to replicate a dead model: the data visualization showcase (I fell into this trap twice before creating FILWD, so I know what I am talking about). The showcase model is this: “Hey folks, look how cool this is“. Stop. Iterated x-times per week.

You don’t need a blog for this. It was maybe true 5 years ago but with the advent of Facebook and Twitter it’s totally useless. Also, and even more important, there’s no way for you (and for me) to compete with Infosthetics and Flowing Data (@Andrew: I know you don’t agree with me on the death of the data visualization showcase, but what can I do? This is what I think ).

Let me clarify. I don’t think these two blogs are useless. Andrew and Nathan did an enormous service to our field and we all have to thank them from the bottom of our heart, but it’s foolish to believe we need more of that.

Three key reasons why (vis) blogs suck

I could name a hundred, and by the way if you buy a book on blogging (like the classic mainstream ProBlogger) you will find millions, but here I’ll focus on those I believe are especially troublesome for vis blogs (apart from the data visualization showcase which is the most troublesome).

Trouble #1 – Taking it as a hobby. This is the most problematic. People write blogs casually, once in a while, when they have nothing special to do or when they feel something is so cool they have the urge to share it with the world, that is, three friends. Amateur blogs are everywhere and pollute the whole web. If you want to succeed with your blog it’s important for you to realize that you have to sweat your damn shirt. On the contrary, if you don’t want to succeed, why polluting the web with your blog? Think about it, it’s an ecology thing: every piece of information you put on the web may decrease the already feeble signal to noise ratio we have. Do you want to contribute to the noise?

There’s no other way to succeed than taking it as a serious endeavor, believe me. Blogging takes a lot of planning and work. Every single post may take many hours distributed across days, weeks, or even months. And that’s just the effort needed to create content, without counting administration and marketing. You might not see it, but behind every single post here there is a huge amount of work, and I know it’s the same for other fellow bloggers.

Being serious about your blog then it’s not only a matter of content but also of being committed to have a somewhat regular schedule, especially at the beginning. People hate dead trees and for a good reason. Please do me a favor: if you are considering opening a blog, take the whole thing very seriously. You need a good reason for opening a blog and if you don’t have one, sooner or later you will give up. I don’t want to discourage anyone, to the contrary, I want to see more great blogs! But I am also tired of shallow blogs and dead trees.

Trouble #2 – Providing limited value. What is *special* about your blog? I know, it’s a tough question. But, if you are not totally honest with yourself about that, you will have problems. People stop by and read your blog only because you are able to deliver some kind of value. What value? I don’t know … you name it. As a general rule people read for two main broad reasons: to learn or to be entertained (or both). Are you able to deliver unique knowledge that other people cannot deliver? Or do you have a special irresistible style that people love so much they are eager to see what’s next? That’s the trick, that’s the obsession you have to have to succeed.

Many, many, many vis blogs are shallow just because they do not give in, they do not have anything special to offer. They don’t even try to differentiate themselves from the rest. It’s a game in which you lift the bar 1 inch higher every single time you write. The web is a jungle, people jump from one web page to another in a matter of seconds, how do you plan to let someone stop and read through what you write? Let’s take the data visualization showcase mentioned above: do you think you can attract people by showing new visualizations every day? Do you think you are more skilled than the current main players in finding new stuff? I have several doubts.

When I opened FILWD it was clear to me I could not compete with the big guys (and I didn’t want to anyway) so I asked myself: “what skills or knowledge do I have that I can use to gain a competitive advantage?” And my answer was that I have direct access to vis research and researchers and that I know vis theory better than the average geek. I am sure you have your own uniqueness so try to think hard how to use it.

Trouble #3 – Forgetting to show a real face.People are too busy to absorb the bare information, and information by the way is not a scarce resource anyway. Many blogs are plain dry, it looks like the writer does not exist or hides behind the curtains. Where are the emotions, opinions, and fun? Writing about scientific stuff does not imply being serious, objective or dry. The best bloggers show their face and risk their reputation every single post. Sometimes I feel a pain in the stomach before hitting “publish”. I happened to think: “people will kill me for this one “.

Similarly, many bloggers don’t spend any time thinking whether they have a style or not. But *your* style matters a lot and you’d better know what it is. There are a million styles and be careful not to fake it. Your style has to be natural but it also has to shine through your words and visual design. Take for instance Stephen Few: Oh boy … I hate the way he expresses his opinions, he makes me cling my teethes at times, but you rest assured I read every single line of what he writes. What is your style then?

How do you create (or revamp) a successful vis blog?

Hey this is slippery terrain: every single blogger has his own formula and you can find a million sources on the web on how to make your blog successful. I don’t pretend to be a blog guru, but I can share with you the things that really worked for me, with the hope they will assist you in case you want to open your blog.

Tip #1 – Find your final cause. How do you plan to change the world? Why do you want to open a blog? Once you put aside all the legitimate ego trip we all make what is left for the others? Successful blogs are centered around the readers, they want to make the world better. They strive to provoke shifts in people’s mind. How do you plan to be ridiculously helpful for people? With FILWD I planned from the very beginning to help people become visualization experts, then I discovered I could sometimes help them think in unconventional ways. What’s your cause? I’ll give you an example: do you know anything about Data without Borders? That’s a cause folks!

Tip #2 – Study a lot. Before starting FILWD I read an endless amount of material about blogging, I trashed many and kept some. I studied the strategies of many many successful bloggers in many other areas out of visualization. I could name hundreds of sources but you have to do your own research. Among the thousands things I read, there are two gems that really shine: Trust Agents, a must read even if not an easy read, and Think Traffic, the best blog about blogging ever.

Tip #3 – Plan ahead and find your style. Before starting FILWD I wrote down a thousand plans and eventually came up with two key pieces of information: (1) my target posting schedule; (2) a very few number of post categories. The posting schedule does not have to be very tight but it has to be somewhat regular, especially before your blog is established; people hate guessing when you are going to post the next article (and of course I am still struggling with it). Having a number of predefined post categories is the best piece of advice I can give, it helped me being totally clear about what I wanted to write and especially what I did not want to write. For instance, I very rarely write about other people’s work unless it is an inspiration for a broader argument. You can check my categories on the blogs and you will see they are very few. When I write a new post I think: “what category do I want to write in today?”

Tip #4 – Be ready to walk through the dark and deep valley of loneliness. Blogging reminds me when I started learning how to play guitar many years ago. At the beginning it’s so frustrating, it looks like you will never be able to play two chords one after another. With blogs the problem is that at the beginning you have zero readers and you have to spend a lot of time preparing these stupid posts nobody will ever read. Very painful. But it’s totally transitory: if you keep doing the good work, people will come and will love your post written to nobody in the past. That’s a very key element of blogging: being able to go through the deep valley and wait until it blossoms. You have to have faith: it will be great.

Tip #5 – Find your own buddies. What is life without friends? I don’t have to tell you how to use twitter, Facebook, or Google plus right? Plus I don’t think there is a unique formula. But hey, make sure to build a thriving environment around your blogs and your ideas. Somebody said “No man is an island”, well this is especially true in this business. Find some buddies, share your ideas with them, test your ideas before writing a post, be exceedingly generous and genuine and people will gather around you.

Tip #6 – Experiment. Blogging is a constant experiment. You write a very successful post with a given strategy, you try to replicate it and it doesn’t work. I like to think about blogging as a radio knob you have to manipulate to find the right frequency to tune with your audience. The frequency is always shifting and your work is to be able to seek the right spot all the time. Sometimes it works, sometimes it doesn’t. But it’s not a big deal as long as you keep trying. Blogs, for instance can accommodate very different media and it’s a good idea to experiment with them. I experimented a few times with video and I was scared shitless because of it.

There are of course many other things you can do to make your blog successful, many of which I don’t know yet. Everyone has his own path, you have to find yours. I know one thing for sure: hard work always pays off. Always.

Need a good reason for opening a blog?

Hey, I hope I did not scare you too much up to this point. There is one thing I want to make sure you get out of this blog post: opening a blog may be one of the smartest choices you can make in your life. Again I could name hundreds of reasons why blogging is great but for me the most important one is that it feeds my mind in a way I could not get with other means. Blogging so far helped me, at least, in these many ways:

• I became a much better writer
• I became a sharper thinker (thanks to having to write what I think)
• I know much better how the web works
• I know many more great thinkers … and they know me
• My ideas are debugged by a large crowd of people
• If I have a burning question I have lots of people to whom I can ask
• I get invited for talks
• It feeds my research and my research feeds it
• I might write a book one day thanks to it
• …

I can testify that all the effort is definitely repaid by the myriad of benefits you can get. Some people do blogging for the money, and some are pretty successful, and some other for the glory. But whether you do it for the bling bling or not, the formula is always the same: you have to write epic shit. There are altruistic and egoistic benefits from blogging and they are all fine as long as you have a good balance. Blogging makes you grow internally, you find yourself improving in many ways, and it helps you having a powerful interface with the world. But it also helps people thrive thanks to your work, and that’s absolutely priceless.

Start a kick-ass visualization blog today!

Let me add one final remark. If you are thinking of opening a data visualization blog, a good one, please do it! We have a desperate need for quality content and I want to have my inbox filled up with exciting ideas. If you need more help send me a line or ask to professional bloggers. I do think there is a huge space for new blogs in this area, you just need to find your niche. For instance, I am looking forward to data visualization blogs related to one specific application area. Or, another great one I’d love to see is a blog with a frequent posting of interesting little visualization experiments. It’s up to you now, let’s make data visualization better together!

Hi There,

VisWeek is approaching! This is just a short notice to let you know I am organizing a Birds-of-Feather with Robert Kosara titled “Blogging About Visualization” at VisWeek. The goal of the BOF if to meet people who are interested in data visualization blogs (bloggers and readers) and have a chat about current practices and future developments.

Here is the information about where and when:

Blogging about Visualization
Time: Tuesday 4:15-5:55pm
Location: Bristol
Description: A meeting of visualization bloggers, readers, and anyone interested in discussing visualization on the web. We will discuss experiences, uses, purpose, benefits, and future goals of blogging about visualization.

If you are a regular reader of this or other data visualization blogs and has something to say or just want to meet us and join the conversation please drop by, I am sure it will be fun. Me and Robert plan to give some advice on blogging, in case people want to hear it, and the BOF will be totally informal. It could also be the starting point of new endeavors, collaborations, ideas, etc.

Cocktail & Dinner.

Another good reason to participate is that we will have a cocktail and dinner just after the BOF. We still have to fix the location but we plan to meet at 7pm at the lobby and go there together. If you are interested, even if you cannot come to the BOF, please sign in the google group I created for this purpose: Vis Blogging Dinner. We are already a good bunch of people.

Take care and hope to see you soon.

P.S. In case you are reading this and will not be able to come to visweek don’t worry I plan to provide plenty of information from the conference this year. Stay tuned!

When I saw for the first time a visualization developed by Jan, the Ghost Counties, I was totally fascinated. It’s brilliant. It took me a while to understand how it works, but once I got it I could not help but admiring the strange mix of complexity and simplicity it provides.

Despite he looks so serious in this picture on the left, he has a big smile and he is fun. I met him for the first time at Visualizing Europe and since then we exchanged many emails. Plus, he is a regular commenter here (and everywhere) and I love him for that.

I don’t know how much I have to add to convince you his advice is a valuable one. Just give a look to his portfolio and judge yourself. He is IMO one of the most interesting data visualization freelancers recently appeared on the scene.

I know, by talking with him, he is proficient with several technologies but he has a passion for D3.

How did you start using Protovis/D3?

I’ve always been someone interested in the latest technologies. So, since I follow the data viz community very closely, I was aware of Protovis very early on, and I was aware of the development of D3 even before it was released to the public. I have a software development background, so I don’t have too much trouble finding my way in new programming languages, and since it excites me to work with new technologies and frameworks, I just started playing with Protovis and D3 as soon as it became available.

What’s the best and worst aspect of Protovis/D3?

The best aspect of Protovis is that it is a domain specific declarative language, which means that is fairly easy to start writing code, using visualization related keywords and functions. The best aspects of D3 is it’s flexibility (more direct integration with SVG) and better performance. The worst aspect of both D3 and Protovis is that it’s hard or impossible to get it working on older browsers, and the learning curve for D3 may be somewhat harder than for Protovis.

Ok, I am a beginner and I want to learn Protovis/D3, where do I start?

I think Protovis is easier to start with than D3, but Protovis is no longer under development. I also see that the Protovis mailing list is not very active anymore, while the D3 mailing list is very active. But, I guess Protovis would be a very good way to start if you’re a beginner. Basic Javascript programming skills would be recommended, both for Protovis and D3. For some great Protovis tutorials you should check out Jerome’s blog: http://www.jeromecukier.net/blog/category/protovis/page/2/ (part 1 – 5: working with data in protovis). Also, the Protovis website has quite some examples that are pretty good. For D3, documentation, examples and tutorials are still under development: http://mbostock.github.com/d3/api/ so, with D3 you’re more on your own right now (and of course the mailing list). But things are improving rapidly.

How is the learning curve vs. return-on-investment of Protovis/D3?

Protovis is really a good way to learn visualization and programming at the same time. Protovis is a language that is really geared towards the visualization (and diagrams) domain, so it really makes sense to talk about axis, marks, lines, bars, pies, etc. Also, there are really quite some good examples on the Protovis website, so it’s fairly easy to get started. However, right now Protovis is not supported anymore, and people are really moving to D3 now, so getting support may become a little tricky. Also, Protovis does not perform as well as D3 with very complex graphs for instance, and also, compared to D3, in Protovis you’re a little bit limited to the animations you can achieve. So, overall, a good way to start and also good to make some nice standard diagrams and visualizations, but if you really want to do ‘heavy’ visualization stuff, you might consider moving on to D3.

D3 is more powerful, more flexible and seems to have more capabilities (and better performance) than Protovis. The flip side is that in order to have a more flexible programming language, the language is also more abtract. Though many concepts of Protovis are also implemented in D3, and there are also quite some predefined visualization layouts, it’s also more useful (compared to Protovis) to gain some knowledge of SVG, since it’s more likely that you might do some low level stuff. D3 does give you much better animation capabilities, better performance, more flexibility, so, once you get the hang of D3 and some SVG, you’re able to create some very compelling interactive visualizations.

What other tools would you recommend other than Protovis/D3?

The tools you mention are some of the best right now. I also think that Raphaël is a fairly good alternative if you want to do Javascript-based visualization that works in older browsers as well. Personally I don’t have much experience with Raphaël yet. Also Processing now has an Android mode, which is great if you want to create visualizations that run on Android phones, and the upcoming Processing 2.0 also has a Javascript mode, so you can easily create HTML5 canvas based visualizations with the Processing development tool.

A recommendation I’d like to add: when I work on Protovis or D3 visualizations, I use TextMate on the Mac. This allows you to open a preview window which renders your visualization near real-time when you are typing in your code. I’m sure that are similar tools that do this. This is really great for getting immediate feedback while you’re coding.

This is a short notice to all FILWD readers who are going to VisWeek 2011. I will be there the full week starting from Sat Oct 22th and I would love to meet some of you guys. There are two main events you might be interested in.

Data Visualization Blogging Dinner

If you are going to visweek it would be just awesome to spend some time together in front of a hot meal and a good drink. I’d love to see your faces in real flesh!

If you are interested (even if you are not sure to come) please sign up to this google group I just set up - Vis Blogging Dinner – VisWeek 2011 - and we’ll find a suitable place and time to spend some time together. Ok? Please sign up as soon as you can so that I will be able to reserve a place to stay.

Blogging about Data Visualization BOF

I am trying to set up together with Robert Kosara (EagerEyes) a Birds-of-Feathers during visweek to meet people interested in data visualization blogging. Everyone is welcome (bloggers, aspiring bloggers, blog readers, etc.) The meeting will be totally informal and we plan to give advice on how to blog about vis (in case you want to hear it from us of course :-) and to hear from you guys whatever idea you have on how to make data visualization blogging better/funnier/stronger/etc.

Catch me!

Apart from that we can always meet in the hotel during the week if you feel like. If you see me around and want to have a chat or share a drink or just say hi please catch me! I added a picture of myself here below so that it’s easier for you to recognize me (I hope).

If you are not going to VisWeek …

I plan to record a video from visweek at the end of each day to provide a summary and thoughts. I also plan to write more about what will happen at visweek later this week. Stay tuned!

I wanted to title this post “giving constructive feedback about visualization and its long-lasting effect” but it didn’t sound as good as this one.

The Story

I was about to write my next long post (don’t worry, almost done) when I received an email from a guy working for Hotels.com:

“Hope you’re well. I’ve seen you’ve covered infographics in the past and thought you might be interested in a new one from Hotels.com that looks at how people from around the world eat and sleep when staying in hotels. The research was conducted among 3,339 people in 20 countries. You can view, download and embed the infographic at: http://press.hotels.com/en-gb/infographics/“

Here is the infographic (click on it to see the details):

I gave a quick look to the image, read the findings, and just discarded it as crap. I said to myself: “Here we go again … another email with crappy infographics”, pushed delete, and moved on to the next task. After a while, my sadistic brain could not resist and I wrote a quite cryptic message on twitter trying to see if I could catch some fish:

“From hotels.com: “I’ve seen you’ve covered infographics in the past and thought you might be interested in a new one” http://bit.ly/nGiUiC“

A few people replied and again I moved on to the next task.

Some other people like Stephen Few would have maybe started a long rant about all the reasons why this was crap, while some others, maybe, would have taken it seriously and tried to analyze it in details. Me, I just shook my head a never replied to the guy.

Here is where the true story begins: after a few minutes I receive an email from Andy Kirk of visualisingdata.com (bold is mine):

“Enrico – I received the same Hotels.com email today and had a good exchange of emails with the guy promoting them.  

To be fair after our conversation he was really appreciative of the advice and said he will do his best to try and affect a change in approach. Really interesting how this particular market has erupted though isn’t it – the fact Hotels.com has a dedicated Infographics section under its PR pages…”

And later on:

“… it is becoming more and more difficult to stay on top of these type of requests but I’m taking the longer view that if I can offer constructive feedback it might in the smallest way have an impact on improving practice“

Let me repeat this sentence from Andy:

“If I can offer constructive feedback it might in the smallest way have an impact on improving practice”.

The Lesson

What a lesson have I learned! It was like a diamond in my head. Thanks Andy.

If you have been reading this blog for a while, you might have noticed I tend to be quite pacific, but at the same time when it’s time to say crap, I say crap.

We in the community have learned to have an automatic reflex: we look at some crappy visualization and in the best case we shake our head, in the worst, we write long rants a la Stephen Few.

I must confess I use to shake my head more often than writing long rants, also because otherwise FILWD would only host such type of big-ego content which I don’t like.

After reading Andy’s email I completely changed my mind. It’s way too easy to look at some stuff and think  ”oh yes, the usual crap”. I did it so many times! And it’s even funnier when you share the “crappyness”  with some friends or tweet about it “hey look … how could they be so idiotic to draw this and that in this and that way”. And we fill our mouth with words of wisdom.

Question: Do we make visualization any better by ignoring or, worse, mocking people who design bad visualization?

I know some of you might say that publicly criticizing bad stuff with big words will make people notice and be more cautious about what they publish. True. But will this strategy pay off in the long-term? I am not sure.

What do you think? Is it more beneficial a loud voice or a humble and cheerful suggestion? Especially, when people ask for an opinion. Do we need both? Do we have to treat different people with different strategies? Or should we just ignore everybody and do our work the best we can?

A few months ago I wrote in my post on Visualization Consumerism:

“I think we have to acknowledge the problem and do our best to educate people. But wait a moment …. educating people is a dangerous idea! I agree. But let me explain what I mean. When I say educating people I mean doing it bottom-up; by giving the right examples and striving for creating a thriving environment“

It looks to me like if these words had been written by someone else! The words are good, my behavior just does not match. We will build this thriving environment only if we learn to shake our head less and learn to help people in every possible way to make great visualization.

Sorry, now I have to go … I have to write a reply to the guy from Hotels.com.

Thoughts?

Oh gosh, I was almost going to miss it: Fell in Love with Data turns one! One year has passed and so many things happened in the meantime. Where do I start? Well, let me start with the obvious but important:

Thanks to all of you guys who are reading, commenting, re-tweeting, sending messages, etc. You gave to me much more than what I gave to you. I owe you something.

Numbers have been growing fast during this year but FILWD is not my personal toy to boost my ego (even though it helps in these regards), it’s a tool to advance data visualization. It’s for you, it’s for me, it’s a work in progress made to help us making this whole business damn better … with some fun in between if possible.

Lessons Learned.

There are many things I learned during this year, far more than I am able to write in this blog post. Here are a few that come to my mind right now as I am writing:

• Blogging is a fantastic platform and I cannot think of doing without it anymore. Now, after one year, I cannot think of how one can pretend to be influential without a blog (especially in academia my dear fellows).
• Writing blog posts helped me far beyond my expectations towards clarifying ideas to me and taking all these vague concepts I had in my brain and transform them into something concrete. What is really surprising to me is to notice how not only my research work helped me writing blog posts, but also writing blog posts has had a strong positive influence on my own research (one more reason for blogging my dear friends). Far more than I expected.
• Blogging is at the same time much harder and much easier than people think. It’s harder because you have to spend a lot of energy and thoughts to make a blog successful. It’s a damn serious job, it doesn’t happen by chance. For me everything changed when I realized that it was totally nonsense trying to compete with Infostethics and Flowingdata and that I had to offer something different. But blogging is also much easier than people think because it just takes you to come up with a solid concept, set up a blogging account anywhere, and write, write, write an let it flow. I am surprised by how many people are scared by it.
• What matters is not success in terms of numbers (even though numbers count), but influence. At the end of the day if you have only 100 readers but you are blowing their mind it’s a lot better than having 100.000 casual readers passing by and say: “hey cool” and then they go back to their own stuff. It reminds me a notable statement from Tufte: “differences that make a difference”.
• No matter how much planning you put on your blog and how many blog posts you have in the pipeline, a blog is a living entity with its own dynamics and you cannot anticipate how people will react. This means being always ready to adapt and write about what matters now. What people need to read in this very moment, not the idea you had three months ago.
• The space for the data visualization showcase is shrinking (and thanks god!) The only way to be successful in data visualization is to do solid stuff that people need. Yes, there are still a couple of consumerist visualization readers out there but who cares? Do they make any difference at all?
• It doesn’t matter how clever and innovative the things I write are, the biggest value of the blog is YOU. In innumerable instances the real value of my posts came from my readers and their comments. I especially enjoyed those with opinions alternative to mine. They helped me re-think my ideas and make them more solid. Thanks a looooot!
• The best posts I wrote are those that scared my butt off, those where I felt I was stretching my intellectual capabilities. Often with this kind of posts I experienced the tension between trying to be as accurate and informed as possible, with the realization that I just don’t have enough time and means to study everything in every single detail. That’s hard but it’s also very very rewarding.
• Data visualization is a huge trend, far beyond the close knit of academics I was used to deal with. Plus, people out there are in desperate need of solid information because the Internet is a chaos and the field is not mature enough. Also, we people in academia have the responsibility to lead the way (did you hear that guys?)

Retrospective.

In retrospective what could I have done differently? I don’t know … maybe you can tell me what you think. I don’t think I could have done anything too differently, I am pretty satisfied of how things evolved.

If I have to mention one single thing I would like to do better, it’s to achieve a much more regular posting rate. But in the end it’s a compromise, what is better: to write more often but more crappy stuff or write only when I have something to say? Dilemma.

I am proud of the following posts:

• How to Become a Data Visualization Freelancer | Interview with Moritz Stefaner: Because it blends surprisingly well useful knowledge and fun (thanks Moritz!)
• Visualization Consumerism: Because it came from my gut and I managed to let some people think.
• Data Visualization is NOT Useful. It’s Indispensable: Because it’s a little manifesto about how important visualization is to me.
• Do visualizations need to be “accurate”?: Because I overcame the fear of questioning the most fundamental theory we have in vis (and it hurt).
• Can visualization influence people? I mean can we prove it?: Because I managed to leverage on an apparently insignificant personal life event.
• 7 Classic Foundational Vis Papers You Might not Want to Publicly Confess you Don’t Know: Because it spread some fundamental knowledge around (it’s the most successful post I’ve ever written so far).

Things that blew my mind.

• That my post on the 7 classic foundational vis papers had almost 4000 visits on the date of publication. People are thirtsty for knowledge!
• That some renown researchers in the field are reading my blog and contact me for the things I write.
• That some people invited me to talk for my blog and not for my research work (though a bit disappointing! :-))

Special thanks to …

• Robert Kosara: for showing me  with his blog that it was possible to write a blog like FILWD.
• Andrew Vande Moere: for instilling in me some doubts before I started.
• Prof. Tamara Munzner and Prof. George Grinstein for giving me so much fuel.
• The Data Visualization Cartel: you guys know why.

Plans for the future?

I always have plans for the future which I regularly abandon and the list is so full of stuff that I know I will never do it all. So what can I say? Maybe you have something to suggest:

• How do you see FILWD evolving in the future?
• What are the posts you liked the most and would like to see replicated in the future?
• Is there anything useless in FILWD that I should definitely stop doing?

I can anticipate a few things I’d like to do:

• Do more videos, especially if they are fun.
• Create the FILWD Newsletter to have a more intimate communication channel with some of you.
• Create an e-book out of the Beginners Toolkit.

What do you think? Do you like these ideas. Do you have anything to suggest to make them better? Thanks.

—
With Love,
Enrico.

I am really excited to announce my first interview for the “Tools from the Pros” series! We start with a very good one: Miriah Meyer talks about Processing.

Miriah is assistant professor at University of Utah. I met her only briefly during a couple of conferences but I am a huge fan of her research work on interactive visualization systems for biological data analysis (be sure to check them out!) Her tools are a rare example of well-crafted design studies in interactive data visualization and, as far as I understand, they are all developed in Processing.

I really like this interview because it covers many of the things beginners (and more advanced users) need to know. One above all: the rapid prototyping approach Processing makes possible and the whole mindset behind copying and pasting code to explore alternative designs.

Thanks Miriah! I think people has a lot to gain by reading this interview.

—

How did you start using Processing?

I started using Processing in 2008 when I helped design a new undergraduate visualization course at Harvard. We chose Processing as the language for the course, and I learned the core bits of the language putting together homework assignments. I quickly came to appreciate how Processing got rid of all the annoying parts of graphics programming — setting up a rendering window, registering callback functions, dealing with linking and libraries and compiling to multiple platforms, that ridiculous gluPickMatrix, and not to mention the headache of type.

We had Ben Fry come to the class to give a guest lecture that spring, after which we went out to lunch. I’ll never forget his answer to my question of why he created Processing. He said (well, I’m paraphrasing here) that he wanted a sandbox to play in, to quickly develop prototypes without getting bogged down in the architecture of the code. He emphasized Processing as a language to try different designs, with real data and with real interaction. And that cutting and pasting code in Processing is totally cool if it gets a design up and going faster. He wanted a language that lets people totally focus on the visualization concept and design without having to think too hard about the code underneath.

Well, that sounded great to me. And I quickly became a total convert, cutting and pasting code until things got so messy that I had to just rewrite an entire project. I found this philosophy totally liberating and that my work benefited immensely from rapid prototyping. Processing is a language that supports this style of development.

What’s the best and worst aspect of Processing?

In short, the best aspect of Processing is the amount of code it takes to get a simple scene with callbacks going — it is a small fraction of what it would take with OpenGL. Simple primitives like circles, squares, text, etc. are nicely abstracted into one-line function calls. Mouse and keyboard callbacks are automatically handled. There is a wide variety of common graphics helper functions available, like lerp-ing colors. Full-screen apps work without having to grab weird OS handles. The PDF library that exports the current scene as vector-graphics has forever changed figures for papers for me. And the ability to export an application to a variety of operating systems in a single go is absolutely invaluable when working with users on a variety of platforms.

Despite all the simplification of the underlying graphics library, Processing still feels like you are in complete control of every mark you make on the screen. I almost never feel like I need to find a way around a function to get the sort of control I want. The design decisions that went into creating the Processing API are fabulous. Really.

As for the drawbacks, there aren’t any really great libraries (yet) for basic user interface widgets. Which for me is ok because I’m kinda neurotic about how my scroll bars look and act. But for graphics beginners this can be a real time-sink. Same goes for more sophisticated types of visual representations like basic charts, maps, and networks. Other languages like Protovis provide built-in algorithms for handling these very common types of representations. In Processing, you’ll have to implement your own graph layout algorithm (or, find one on the web). Again, this can be a hurdle for people with less programming experience.

And as a small gripe — Processing has implementations of Bezier and Catmull-Rom curves … but where is the love for b-splines???

Ok, I am a beginner and I want to learn Processing, where do I start?

Go to www.processing.org. Click on the Learning tab. Explore.

On the Learning page you’ll find a whole series of tutorials and examples that can walk you through the basic functions of Processing. The next step is to peruse the inspiring demos in the Exhibition, many of which will include example code. When you see a function you don’t understand, the Reference page has wonderful documentation for the language.

If you are new to programming or graphics programming the two books I recommend are:

• Getting Started with Processing, by Casey Reas and Ben Fry
• Learning Processing, by Daniel Shiffman

You can work through the Getting Started book in a day. It’s short and sweet. If you find that you need more help, the Shiffman book includes more details on how to program and lots of paper and pencil, and coding, exercises. Daniel Shiffman wrote this book from course notes he created in teaching design students at NYU about coding and Processing. It’s intro to programming via Processing.

If you are an experienced graphics programmer all you need is what you can find on the Processing website.

How is the learning curve vs. return-on-investment of Processing?

If you know OpenGL and are familiar with Java, the learning curve is super short and shallow. If you are new to graphics, it will take you less time to wrap your head around Processing than OpenGL. And if you are new to programming, Processing is a really fun way to learn the basics.

With that said, it is still a programming language. Reading in data from a file requires basic coding skills, as does just about any interesting interactive visualization. You have to be comfortable with for-loops and arrays. Processing makes graphics programming way easier, but it doesn’t automatically generate visual representations of data. You have to code that.

If you want control over every aspect of your visualization and interaction designs, then you really just have to program. Processing is one of the best languages to use for that. If you just want to see what your data looks like, then there are other tools that can do this quickly with built in visual representations (like Tableau, ManyEyes, Matlab, R, etc).

What other tools would you recommend other than Processing?

I’d recommend any of the tools and languages I’ve mentioned previously. Another gem is ColorBrewer for selecting great colormaps.

Still, nothing beats OpenGL for truly understanding how graphics works. If you are serious about developing interactive visualizations, I think that taking an intro to graphics course that uses OpenGL is invaluable. Understanding the rendering pipeline and how it is implemented in a computer will make the seemingly quirky aspects of even a language like Processing make sense.

Hallo everyone! I am happy to introduce a new series nested in the Data Visualization Beginners Toolkit: “Tools from the Pros”.

In my last post on data visualization tools I suggested a number of strategies to choose the best tool for you and I provided a list of those I think are the best bets currently available. Now, while I think this list is already very useful, I decided to give you more and I interviewed one data visualization expert for each tool mentioned in the list.

I will be publishing the interviews during the next weeks. Some of them are still in preparation and the list might be expanded in the future as comments and requests come in (please feel free to ask!) What I can tell you from now is that I have the following interviews in editing stage and that the first one will come very very soon:

• Miriah Meyer on Processing.
• Joe Mako on Tableau.
• Jorge Camoes on Excel.
• Jan Willem Tulp and Jerome Cukier on Protovis and D3.
• Drew Conway on R.

Each one is a real pro in his area and knows very well the tool he or she uses to make effective visualization. I am sure you will get a lot of useful information out of them.

To each one I asked the following questions:

• How did you start doing visualization with X?
• What’s the best and worst aspect of X?
• Ok, I am a beginner and I want to learn doing visualization with X, where do I start?
• How is the learning curve vs. return-on-investment of X?
• What other tools would you recommend other than X?

I hope you’ll enjoy it. Stay tuned! The first one is coming very soon.

Important: if you are an expert and are willing to answer these questions about your favorite tool I’d be happy to include you in the list!

More important: specific request for other person/tool interviews are welcome! Who else would you like me to catch? About which tool? I cannot assure you anything but I’d love to receive your requests.

Take care guys, and have fun.

Big Pluses: visualizations delivered directly through a web browser, compact code, good community size and excellent documentation.
Some Minuses: the code is a bit tricky and it requires some getting used to, it is not as diffused as other technologies (but this is going to change soon), it might be discontinued in the future the same way as Protovis was.
Notable Examples: Jan Willem Tulp’s Urban Water | D3 Examples Page

Big pluses: can create visualizations in a snap, very easy to explore many alternative views of the same data, does not require programming, very large user base.
Some minuses: not as flexible as using a programming language, it’s expensive, takes some time to understand how it works.
Notable examples: Tableau Software’s visual gallery | Clearly and Simply’s Tableau Posts

Big pluses: universal platform, everybody understand excel, practically free, easy to go from data to chart, integrated with the spreadsheet functionalities.
Some minuses: the defaults are crap, harder to go beyond standard charts, slow with big data.
Notable examples: anything from Excel Charts gurus Jorge Camoes and John Peltier.

I kept my promise: the interview with Moritz Stefaner on data visualization freelancing is finally here! And I am really excited.

As I said in my introductory post, I think data visualization freelancing is one of the most exciting trends in visualization; even though it’s a little bit hidden. After recording the interview, I must say I am really satisfied. I learned something out of it and I am sure the same will be totally true for you.

The video is a bit long (see the content breakdown below) but it’s really worth it: we covered a very large number of questions and they all came directly from the readers (thanks to all of you guys)!

Any comment, question, or suggestion for me and Moritz is more than welcome. You can write a comment here below or contact us directly on twitter (@FILWD, @moritz_stefaner). Have fun!

Video content breakdown with timing:

• [01:00] Starting Out (building a portfolio)
• [08:30] Design Practice (iterative approach, designing 20 prototypes!)
• [16:18] Skill Building and References (books, tools and libraries, doing without programming?)
• [27:47] Dealing with Clients (what clients want vs. what is right, freelancer vs. agency)
• [34:08] Pricing (billable time, tracking yourself, strategic prices, the “pain coefficient”)
• [39:44] Time Management (avoid working 24/7, have kids!, having a rhythm)
• [41:55] The Freelancing Market (gaps in the market)
• [44:05] The Role of Research (searching and reading papers)
• [47:24] Summary of Tips for Wannabe Freelancers

Additional versions of the interview

• Download mp3 file to listen on your own player.
• Interview transcription (if you want to read it) – coming soon.

Do you want more? Let us know.

As you can see in the interview, me and Moritz are thinking of recording additional videos. Who knows … this might even become a regular meeting. We would love to have your opinion on that. Especially we would love to know: what else would you would like to hear? Is there anything we missed? How can we help you further? For sure, we would like to record a new one with a more extensive discussion of design practices and the overall data visualization process. Stay tuned and let us know!

Notes

•  First of all thank you guys for sending all your questions for Moritz! This was very useful and I am sure the interview is much much better than what it would be without your help.
• A big big thank to Moritz. I really enjoyed talking with him (as usual) and I think the end-result is really helpful for people who want to know more about freelancing.
• The quality of the video is not perfect, I apologize. There is so much to learn! My phone started ringing at some point, the line dropped because I forgot to plug the power plug, and there’s no video editing apart from very basic stuff. Nonetheless, I think that content is king and what matters is that you are going to learn something. This is work in progress and it will get better.

One of the main goals of this blog, other than challenging the status quo with reflections at the intersection between academics and practitioners, is to help people become data visualization experts. It’s not rare for me to receive emails from people who are enthusuastic about visualization but have little guidance about how to become an expert.

I have been posting some few articles in the past with this specific goal but I realized that they are too scattered and not organized in a way to represent an organic resource for the readers.

For this reason, I decided to create a series specifically designed to help those of you guys who are excited about visualization but really don’t know how and where to start. The series is meant to be part of a permanent collection in FILWD and it’s my first serious attempt to react to my own call to action: “When will we decide to provide lots of value?“.

Introducing the series

The Data Visualization Beginner’s Toolkit will function as an orientation guide for poeple who need guidance in finding the right resources to become data visualization experts. In the guide I will not be teaching visualization directly, nothing technical or theoretical about it (I have plans for this later), but I will show you the resources and one path.

Having such a guide is particularly important today because data visualization is really just like a jungle. There are plenty of opinions, blog posts, research papers, consumerist visualizations, books, etc., and it’s very hard to separate the wheat from the chaff.

When reading this series please keep in my this is my very personal view and, as such, is limited to my own experience. Also, whatever list I will propose is certainly neither unique nor exhaustive. If you are looking for an exhaustive list of resources I highly recommend you Andy Kirk’s collection of data visualization resources.

Here is a tentative list of topics I am planning to cover in the series (subject to changes):

1. Books and Other Resources
2. Programming Languages and Tools
3. Sources of Good Examples
4. Research Papers
5. University Courses

Please if there is anything else you would like to be covered let me know! Send me a message or add a comment below.

Books about Visualization

There is a reason why I start the series with a list of books: if you don’t know the basics of data visualization you will always be an amateur. And what’s worse, visualization experts will notice it and will not take your work seriously.

Also, orienting yourself in the mess we have right now might prove discouraging and prone to errors. If you type “data visualization” in Amazon the result is a disaster, believe me.

Finally, even if you end up picking up very good books, it is definitely possible they are not the right ones given the amount of knowledge and expertise you currently have. Here I suggest the following path (in order).

Show Me the Numbers: Designing Tables and Graphs to Enlighten (To acquire solid foundations). This book teaches the basics of visualization by using only tables and simple charts. You won’t find fancy and colorful visualizations, only scatter plots, bar charts and stuff like that. But that’s the way to go! If you understand the basics then it’s a lot easier to spot the limitations of basic graphs and go beyond them. Plus the book contains the best summary of visual perception applied to visualization I know. It really is a true gem. Don’t make the mistake to be attracted by fancy stuff and skip the basics, start here and you will have very solid foundations.

Readings in Information Visualization: Using Vision to Think (Chapter 1 only) (To go beyond simple charts). Once you understand how charts work and you have learned the basics of visual perception, you are ready to explore fancier stuff. Yet you need some guidance on how to explore the huge data visualization space. The first chapter of this book is the best self-contained piece of work I know. It’s able to provide all it’s needed to start thinking more creatively, but in a structured manner, about advanced visualizations. The book also has a strong emphasis on interaction which is important. If you want to go beyond the first chapter fine, but the book itself is a collection of papers and many of them are totally outdated. But wait a moment, this doesn’t means you cannot find useful material there! In the collection you can find fundamental papers that are totally worth a read: the work of Jacques Bertin above all.

The Visual Display of Quantitative Information and the rest of Tufte’s books  (To learn what “graphical excellence” is). People go crazy with Tufte’s book and I understand why: they are totally beautiful, the cover, the format, the colors, the contet, everything. But regardless their beauty, I have always thought it’s really hard to learn something out of them; they require you to think really deeply about what you see. Basically they are “just” a collection of images. The Visual Display of Quantitative Information is the first one and is the only one I truly recommend because it give more guidance than the others. The others are wonderful but you will have to spend more time on them to translate their content into design practices.

Information Visualization: Perception for Design (To know what happens in our brain when we see a visualization). If you have read all the books cited above congratulations! You have learned really a lot. Now, information visualization is deeply rooted in visual perception and cognition. If you want to master the art of visualizization, at some point you will have to know these basics; especially if you aspire at designing innovative visualizations that fit people’s needs. This book starts from the very basics of human vision (e.g., how the eyes work) up to how we think with visualizations. It’s a tough read but it’s totally worth it. You will have to spend quite some time thinking how these theories apply to your specific projects, but believe me, it’s a true investment. I experienced countless situations where a visualization design problem was deeply rooted in one of the issues discussed in this book. You will find yourself referring back to it all the time.

More Books about Visualization

Important: Are the books not mentioned above bad or not worth it? Absolutely not.

It is important to consider two factors: (1) there are several books I have never read or even skimmed through which might provide some additional value to you; (2) there are extremely valuable book I’ve not included just because they are either too advanced or don’t fit the progression of readings I am proposing here. Please keep in mind: I am suggesting you to read these books in the order I gave above.

A few additional books that come into my mind, which need at least a short mention are:

• Any other book written by Stephen Few.
• Any other book written by Edward Tufte.
• The statistics-flavored and super-classic Visualizing Data and The Elements of Graphing Data by William Cleveland.
• The monumental Semiology of Graphics by Jacques Bertin, which I did not include because it is still hard to get despite a new edition came out and because it’s really a hard read for non-experts.
• The extremely beautiful and information rich Visual Language for Designers by Connie Malamed, which I did not include because I haven’t finished reading it yet.
• The deep and dense How Maps Work by Alan MacEachren, which despite the title teaches visualization and makes you think deeply about it.
• The not known enough and little gem Designing Visual Interfaces by Kevin Mullet and Darrel Sano, which teaches aesthetics in a functional and systematic manner.

Books NOT about Visualization

It’s important to acknowledge that not all the knowledge a visualization expert needs comes from data visualization books. I have no intention to write another long list of related disciplines’ books, but it’s important for you to know that a good data visualization expert may have strong foundations in areas such as: statistics and data mining, data management and manipulation, human-computer interaction and cognitive science.

I don’t want to scare you: you can start doing visualization without these, but little by little you likely will find yourself digging more into these areas.

Also, let me stress the importance of human-computer interaction and related areas. While the rest is normally acquired, at least on a superficial level, by using various technologies you encounter along the way, human-computer interaction has a less technical flavor and you might not learn anything of it unless you seek it.

Knowing how people reason and interact with user interfaces is a crucial skill, the real differentiatior, that you’d better acquire if you want to become a pro. I cannot stress this point enough. Visualization, as any other user interface, happens in people’s mind, not in the computer! And if you want to design great ones you’d better learn how people’s mind work.

There is only one book I feel like suggesting as a starting point: the brilliant, super-practical, and freely-available Task-Centered User Interface Design.

Other Learning Resources

Unfortunately, other than the books I mentioned above, there are not many other sources from which you can really learn something. But luckily there are some few notable exceptions! Tamara Munzner and Jeff Heer, top-researchers in the field, share the material of their courses freely on the web and you should not miss them for any reason:

1) Tamara Munzner’s InfoVis Course Slides at University of British Columbia
2) Jeff Heer’s InfoVis Course Slides at Stanford University

These are university courses, with a specific target, but I cannot think of a more carefully and better organized set of information covering the whole theory and practice of information visualization. What is really unique in these courses and their material is the way this information is organized. Information visualization is still a young discipline and nobody really agrees yet on the content and order to use when teaching it. These two courses found in my opinion the perfect balance between coverage and organization.

Another great source for learning data visualization are Stephen Few’s articles and white papers, which teach a whole lot of fundamental data visualization skills with his usual concise and effective style.

Can I get all the knowledge I need with these books? No.

Conclusion

That’s all folks. I really hope this series will be useful to you. Let me know what you think and if it helps. Also, I’d really love if you could enrich it with your suggestions. You can write comments below or send message to me on twitter at @FILWD.

Please do not forget to share this with your friends or people who might benefit from it. Its main purpose is to let you guys become better data visualization experts. Help me to spread the word around.

—

Take care, have fun,
Enrico.