BELIV: The Workshop Blog http://www.beliv.org/blog BEyond time and errors: novel evaLuation methods for Information Visualization Thu, 08 Apr 2010 11:16:41 +0000 http://wordpress.org/?v=2.8.4 en hourly 1 BELIV’10 Schedule is now online! http://feedproxy.google.com/~r/BelivTheWorkshop/~3/r46BqcrlCIg/ http://www.beliv.org/blog/?p=42#comments Thu, 08 Apr 2010 11:16:41 +0000 Adam http://www.beliv.org/blog/?p=42 The BELIV’10 Schedule is now online.  Please check here for any updates!

Day 1 – Saturday, April 10th 2010. 9AM – 5PM.

A day of presentations: get an overview of evaluations in Information Visualization from multiple perspectives and prepare for Sunday’s working session

09:00 – 09:30 Welcome to BELIV 2010!
09:30 – 10:30 Presentations: #New Metrics I
10:30 – 11:00 Coffee Break
11:00 – 12:30 Presentations: #New Metrics II; #Insight evaluations; #Tasks / Data
12:30 – 14:00 Lunch
14:00 – 15:30 Presentations: #Physiological measurements; #Methods
15:30 – 16:00 Coffee Break
16:00 – 17:00 Presentations: #Mass evaluations; #Evaluation lessons
17:00 – 18:00 Break
18:00 – 20:00 Workshop Dinner (optional)

Day 2 – Sunday, April 11th 2010. 9AM – 5PM.

A day of discussing future research themes in visualization evaluations.

09:00 – 9:10 Introduction to day 2
09:10 – 9:55 Keynote (Kasper Hornbaek)
09:55 – 10:30 Generation of discussion topics and groups
10:30 – 11:00 Coffee Break
11:00 – 11:30 Working session 1
11:30 – 12:00 Working session 2
12:00 – 12:30 Working session 3
12:30 – 14:00 Lunch
14:00 – 14:30 Working session 4
14:30 – 15:30 Group presentations (10 presentation + 5 discussion)
15:30 – 16:00 Coffee Break
16:00 – 17:00 Wrap up, feedback, awards, conclude
]]> http://www.beliv.org/blog/?feed=rss2&p=42 0 http://www.beliv.org/blog/?p=42 BELIV’10 Papers are now online http://feedproxy.google.com/~r/BelivTheWorkshop/~3/8uD6QJPx-6Y/ http://www.beliv.org/blog/?p=37#comments Mon, 05 Apr 2010 09:53:31 +0000 Adam http://www.beliv.org/blog/?p=37 As the BELIV’10 workshop is nearing, we’ve put all of the papers online at http://www.beliv.org/beliv2010/.  Revised versions of the following papers will be published in the ACM Digital Library in the coming months.

Research papers

Position papers

]]> http://www.beliv.org/blog/?feed=rss2&p=37 0 http://www.beliv.org/blog/?p=37 Announcing the BELIV’10 Keynote http://feedproxy.google.com/~r/BelivTheWorkshop/~3/D27hJAVRYSo/ http://www.beliv.org/blog/?p=33#comments Wed, 17 Mar 2010 14:34:40 +0000 Adam http://www.beliv.org/blog/?p=33 We are delighted to announce that Kasper Hornbæk will deliver the keynote at BELIV’10.  Here are his keynote details:

Conceptual and Practical Challenges in InfoViz Evaluations

Abstract: Research in information visualization (InfoViz) has developed considerably during the last 25 years. In particular, the field is now informed by a substantial and growing literature on evaluations of visualizations. To keep advancing InfoViz, I believe we need to address two limitations of our evaluations. On the one hand, few empirical studies are motivated by theory or are comparing equally plausible hypotheses. Mostly, the InfoViz literature proposes radical innovations (in the terms of William Newman) and does little to develop and test concepts. On the other hand, many of the practical, low-level decisions in InfoViz evaluations are problematic. Like most HCI researchers, we evaluate our own interfaces, use mostly simple outcome measures, rarely study the process of interaction, and select tasks somewhat randomly. This talk will outline the conceptual and practical challenges of evaluation and begin a discussion of how to overcome them.

Biography: Kasper Hornbæk received his M.Sc. and Ph.D. in Computer Science from the University of Copenhagen, in 1998 and 2002, respectively. Since 2009 he has been a professor with special duties in Human-centered Computing at University of Copenhagen. His core research interests are human-computer interaction, usability research, search user interfaces, and information visualization; detours include eye tracking, cultural usability, and reality-based interfaces. Kasper serves on the editorial boards of Journal of Usability Studies, Interacting with Computers, and International Journal of Human-Computer Studies (IJHCS). He has published at CHI, UIST, ACM Transactions on Computer-Human Interaction, and Human-Computer Interaction, and won IJHCS’s most cited paper award 2006-2008.

]]> http://www.beliv.org/blog/?feed=rss2&p=33 0 http://www.beliv.org/blog/?p=33 The Papers of BELIV 2010 http://feedproxy.google.com/~r/BelivTheWorkshop/~3/4q13JYvzJpw/ http://www.beliv.org/blog/?p=29#comments Mon, 15 Mar 2010 15:16:33 +0000 Adam http://www.beliv.org/blog/?p=29 The strong quality of BELIV continues with the 2010 edition.  12 research papers and 10 positions papers were accepted to our workshop.  Below is a preview of the titles and authors.  As BELIV gets closer, we’ll publish more information about each of these papers.  Stay tuned!

Research papers

  • Developing Qualitative Metrics for Visual Analytic Environments. (Jean Scholtz)
  • Do Mechanical Turks Dream of Square Pie Charts? (Robert Kosara, Caroline Ziemkiewicz)
  • Many Roads Lead to Rome. Mapping Users’ Problem Solving Strategies. (Eva Mayr, Michael Smuc, Hanna Risku)
  • Decomposing Visual Deciphering for Justification and Comparison of Representation. (Stephane Conversy, Christophe Hurter, Stephane Chatty)
  • Exploring Information Visualization – Describing Different Interaction Patterns. (Sylvia Wiltner, Margit Pohl, Silvia Miksch, Markus Rester, Klaus Hinum, Christian Popow, Susanne Ohmann)
  • Is Your User Hunting or Gathering Insights? Identifying Insight Drivers Across Domains. (Michael Smuc, Eva Mayr, Hanna Risku)
  • Comparing Benchmark Task and Insight Evaluation Methods. (Chris North, Purvi Saraiya)
  • Evaluating Information Visualization in Large Companies: Challenges, Experiences and Recommendations. (Michael Sedlmair, Petra Isenberg, Dominikus Baur, Andreas Butz)
  • Learning-Based Evaluation of Visual Analytic Systems. (Remco Chang)
  • Scanning Between Graph Visualizations: An Eye Tracking Evaluation. (Joseph Goldberg, Jonathan Helfman)
  • Towards Information-Theoretic Visualization Evaluation Measure: A Practical example for Bertin’s Matrices. (Innar Liiv)
  • Generating a synthetic video dataset. (Mark Whiting)

Position papers

  • Visualization Evaluation of the Masses, by the Masses, and for the Masses. (Jeffrey Heer)
  • Focus Groups for Functional InfoVis Prototype Evaluation: A Case Study. (Peter Kinnaird, Mario Romero)
  • Pragmatic Challenges in the Evaluation of Interactive Visualization Systems. (John Stasko, Youn-ah Kang, Carsten Gorg)
  • Look Before You Link: Eye Tracking in Multiple Coordinated View Visualization. (Chris Weaver)
  • Mutually Linked Studies – Balancing Threats to Internal and Ecological Validity in InfoVis Evaluation. (Niklas Elmqvist)
  • Beyond system logging: human logging for evaluating information visualization. (Nathalie Henry)
  • How is a graphic like pumpkin pie? A framework for analysis and critique of visualisations. (Hadley Wickham)
  • Proposed Working Memory Measures for Evaluating Information Visualization Tools. (Laura Matzen, Laura McNamara, Kerstan Cole, Alisa Bandlow, Courtney Dornburg, Travis Bauer)
  • Comparative Evaluation of Two Interface Tools in Performing Visual Analytics Tasks. (Dong Hyun Jeong, William Ribarsky, Tera Green, Remco Chang)
  • Implications of Individual Differences on Evaluating Information Visualization Techniques. (Ji Soo Yi)
]]> http://www.beliv.org/blog/?feed=rss2&p=29 0 http://www.beliv.org/blog/?p=29 Position vs Research Papers at BELIV’10 http://feedproxy.google.com/~r/BelivTheWorkshop/~3/nP9H5ZZOyOU/ http://www.beliv.org/blog/?p=26#comments Wed, 18 Nov 2009 09:15:48 +0000 Adam http://www.beliv.org/blog/?p=26 There has been some confusion about the difference between position and research papers for BELIV’10, as a potential participant asks:

I’m thinking about submitting a paper to BELIV, but I’m not sure what
category to put it in – it’s a bit more than a position paper but not
quite a full research paper – i.e. I think it’s a nice idea and it’s
worked for me in the past, but it doesn’t have any formal validation.
Am I better of submitting it as a position paper (which I presume has
a high change of being accepted) or should I try for a full paper?
(and can I still submit it as a position paper if it is rejected?)

Our co-chair Enrico thoughtfully responds:

I understand we have a problem here, you are the second person to ask.   Of course, even if we have had very good quality papers in the past, BELIV is just a workshop and we don’t expect lots of validation or strong results. What we are aiming at is more well-reasoned, innovative and, if possible, thought provoking, ideas.

So, if you think your paper has these characteristics, go on and send a full paper! Position papers are just a way to say: “hey I want to participate and this is what I have to put on the table when we will discuss”. And if anything goes wrong, you can of course always convert your paper in a position paper, no worries.

In any case I suggest to you to give a look to the papers of past editions. Many good papers don’t have any formal validation at all. You can for instance give a look to Catherine’s paper on MILC methodology (BELIV’06), or the one of Heidi and Tamara in BELIV’08 on the utility of empirical studies, or the one of Petra et al. on Grounded Evaluation.

]]> http://www.beliv.org/blog/?feed=rss2&p=26 0 http://www.beliv.org/blog/?p=26 BELIV 2010 Program Commitee announced… http://feedproxy.google.com/~r/BelivTheWorkshop/~3/IIsokScVw-M/ http://www.beliv.org/blog/?p=24#comments Thu, 29 Oct 2009 08:25:00 +0000 Adam http://www.beliv.org/blog/?p=24 BELIV 2010 continues the tradition of our workshop’s extremely strong program committee.    We are proud to announce and thank our current PC members:

Remco Chang (UNC Charlotte, USA)
Carla Dal Sasso Freitas (Instituto de Informatica UFRGS, Brazil)
Jean-Daniel Fekete (INRIA, France)
Georges Grinstein (UMass Lowell, USA)
Jeffrey Heer (Stanford, USA)
Nathalie Henry (Microsoft Research, USA)
Petra Isenberg (Univ. of Calgary, Canada)
Silvia Miksch (Vienna Univ. of Technology, Austria)
Tamara Munzner (Univ. of British Columbia, Canada)
Chris North (Virginia Tech, USA)
George Robertson (Microsoft Research, USA)
Jean Scholtz (Pacific Northwest National Lab, USA)
John Stasko (Georgia Tech, USA)
Jarke Van Wijk (TU Eindhoven, Netherlands)

Remember: Submissions to BELIV 2010 are due November 30!  Full details are available on our workshop page.

]]> http://www.beliv.org/blog/?feed=rss2&p=24 0 http://www.beliv.org/blog/?p=24 BELIV ‘10 Submissions Due November 30 http://feedproxy.google.com/~r/BelivTheWorkshop/~3/LqNQNTYSu84/ http://www.beliv.org/blog/?p=15#comments Wed, 21 Oct 2009 19:27:50 +0000 Adam http://www.beliv.org/blog/?p=15 BELIV 2010 submissions are due November 30.  Similar to 2008, you may submit full research papers or position papers.  The complete details are on the BELIV 2010 website:  http://www.beliv.org/beliv2010/

IMPORTANT DATES
  • Deadline for submissions: November 30, 2009 (5:00pm PDT)
  • Notification of acceptance: December 21, 2009
  • Camera ready papers due: Mid-March
  • Workshop: April 10-11, 2008
WAYS TO PARTICIPATE
We accept 2 types of submissions: position papers or research papers:
  • Research papers are longer (4-8 pages) and present new work and unpublished results. Research papers will be peer-reviewed by members of the program committee and selected according to their novelty, quality and relevance. Authors of accepted research papers will have a chance to revise their papers before they are published in the **ACM digital library**.
  • Position papers are short statements (1-2 pages) describing a participant’s relevant experience and ideas that can contribute to the discussion during the workshop. They will be made available to the workshop participants only.
]]> http://www.beliv.org/blog/?feed=rss2&p=15 0 http://www.beliv.org/blog/?p=15 Help Shape BELIV 2010! Take our survey… http://feedproxy.google.com/~r/BelivTheWorkshop/~3/UnTsxpB1RLQ/ http://www.beliv.org/blog/?p=12#comments Tue, 29 Sep 2009 12:21:36 +0000 Adam http://www.beliv.org/blog/?p=12 In order to make BELIV 2010 as productive and lively as possible, we are seeking your input on how the workshop should be organized.

Please take our survey and share your thoughts!

]]> http://www.beliv.org/blog/?feed=rss2&p=12 0 http://www.beliv.org/blog/?p=12 The BELIV Tradition Continues! http://feedproxy.google.com/~r/BelivTheWorkshop/~3/yRvJ8TuxgIk/ http://www.beliv.org/blog/?p=1#comments Sat, 19 Sep 2009 13:53:27 +0000 Adam http://www.beliv.org/blog/?p=1 The BELIV workshop will once again be taking place at CHI!  It will be held in Atlanta on April 10-11.  For full details, please check out the BELIV 2010 Wiki.

CHI 2010

BELIV at CHI 2010

]]> http://www.beliv.org/blog/?feed=rss2&p=1 0 http://www.beliv.org/blog/?p=1