Nicolas Garcia Belmonte

WebGL Camp

2011-12-09T00:00:00-08:00

I generally don’t do write-ups of events I go to (now that I actually check my blog I’ve never made one) but I had such a great time at WebGL Camp today and learnt so many new things that I think that (especially for the sake of my bad long term memory) I’ll do a write-up about my experience at WebGL Camp #4 at Mozilla HQs.

This is not going to be a chronological review but more like stuff I learnt about state-of-the-art WebGL, and the incredible community that we have here in the Bay Area around this technology.

On the things I learnt there’s that WebGLCamp this year gathered people that make great products and apps using WebGL; it also gathers people who like to write great WebGL frameworks either for games, art or data visualization and it also gathers people who define and participate directly on the making of the actual WebGL API. All of these people are together in the same place, discussing about these three things, and since this is a small group of very knowledgeable people, the outcome of this event is extremely valuable for the people there.

Just take a look at the agenda. In one day you will see how a profitable business like My Robot Nation uses WebGL as a core component of their product, you will also find out the technical aspects of implementing MapsGL in one of the most visited applications on the Web; you will also hear about how WebGL is an integral part for modeling and simulation in medical training applications and finally how the Department of Defense uses WebGL in their real time graphic simulation applications. Oh and yes, Autodesk also showcased their new WebGL applications.

So much changed since last year at WebGL Camp. While WebGL was still “alpha” and people built ideas on top of the thought that in the future it’ll become a solid and robust foundation, WebGL is now massively used in products/businesses that span to any possible field you could imagine!

Another very interesting aspect on this is that not only cool ideas around businesses show up, there’s a lot of discussion on technical stuff, on three different levels I would say: the standards body, the framework API designer/developer group and finally the consumers of these frameworks/APIs that create compelling demos, products and apps.

Just to name a few frameworks, Paul Lewis presented about A3 and Bartek Drozdz presented about J3D. Oh, and the Department of Defense said they were using GLGE for their applications. There was also a very interesting presentation of an Audio/Kinect/WebGL mashup that used CubicVR.js.

GLSLUnit was also presented by Google devs and a very interesting idea of using it to feature detect that a visitor to your site has an adecuate graphics card to run your WebGL application was suggested.

Another very interesting idea from the J3D framework was to design a special templating language that would render to proper glsl code to reuse shaders properly. Although J3D’s glsl language is pretty simple (only has a few special annotations), I would love to see something more like the Django templates in which inheritance, filters and include statements could be added in an easy way to reuse shaders.

Also, I really enjoyed Brandon Jones talk on texture (up and down)loading performance.

But what about me? I presented about PhiloGL last year at Google HQ’s for WebGLCamp #3, so this time it was all about meeting the people and not having any pressure on presenting anything :)

I also met today a couple of developers for OLPC that have been using and contributing code back to V8-GL for OpenGL/WebGL integration in the OLPC project. Can you believe it? V8-GL helps OLPC devs!

Finally there’s also the people who will talk to you about the future of the APIs you’re using. People working on the standard, who will actually listen to your experiences using the WebGL API and probably at the next iteration of WebGL Camp they will present some standard improvement that will include at least part of your ideas in it. And I’m not lying about this. You can witness that in the WebGLCamp #3 video (right towards the end) I was complaining about the fact that simple JSON was not enough as an interprocess/worker format, that it would be nice to be able to move typed arrays from one place to the other. This change was being added as I spoke and this year happened the exact same thing again. I talked two minutes to Ken Russell from Google about the structured cloning algorithm used for worker communication and how typed arrays still had to be cloned from here to there and he told me that transferable objects were just added to Chrome. Yay!

I really enjoyed this event and hope to see the community again for the next WebGL Camp. I had a great time and I think that there aren’t so many ocasions in which you can gather a group of people like this that have so many different backgrounds and ideas on such different topics related to WebGL like standards, products and frameworks.

A big props to Mozilla and Henrik Bennetsen from Katalabs for organising the event! Until the next WebGL Camp!

PhiloGL 1.4.0

2011-10-14T00:00:00-07:00

I’m very excited to announce that PhiloGL version 1.4.0 has been released!

This release focuses on two main things: a better picking algorithm that caches scene rendering and a simple image post-processing chain to add some nice fragment shader effects when rendering a scene.

Both these features are implemented in a revamped World Flights example I just released, so let’s take a look at how these things are implemented in the example.

World Flights (revisited)

World Flights is a data visualization that shows flying routes information for various airlines around the world. For 200 airlines, more than 57.000 flights and 6.000 destinations are displayed. You can find more information on the example here.

You can access the example here.

Image post processing

There’s a nice neon effect in the airline routes as well as in the cities displayed in this example. This is done by rendering two different scenes; one with a “black” planet:

…and another one with the current texture for the planet:

Here’s the code that renders them to a world and world2 FBOs and their associated world-texture and world2-texture textures:

  app.setFrameBuffer('world', true);
  program.earth.use();
  gl.clear(clearOpt);
  gl.viewport(0, 0, 1024, 1024);
  //do not use a texture but paint it black
  program.earth.setUniform('renderType',  0);
  scene.renderToTexture('world');
  app.setFrameBuffer('world', false);

  app.setFrameBuffer('world2', true);
  program.earth.use();
  gl.clear(clearOpt);
  gl.viewport(0, 0, 1024, 1024);
  //use a regular planet texture
  program.earth.setUniform('renderType',  -1);
  scene.renderToTexture('world2');
  app.setFrameBuffer('world2', false);

These two textures are then combined to generate the screen image:

  Media.Image.postProcess({
    fromTexture: ['world-texture', 'world2-texture'],
    toScreen: true,
    program: 'glow',
    width: 1024,
    height: 1024
  });

The glow fragment shader will take these two generated textures, create a bloom filter for the one with the black planet, and add it to the one that has the regular textured planet and provide a result for that:

#ifdef GL_ES
precision highp float;
#endif

#define BLUR_LIMIT 4 
#define BLUR_LIMIT_SQ 64.0

varying vec2 vTexCoord1;

uniform bool hasTexture1;
uniform sampler2D sampler1;

uniform bool hasTexture2;
uniform sampler2D sampler2;

void main(void) {
  vec4 fragmentColor = vec4(0.0, 0.0, 0.0, 0.0);
  float dx;
  float dy;

  if (hasTexture1 && hasTexture2) {
    //Add glow
    for (int i = - BLUR_LIMIT; i < BLUR_LIMIT; i++) {
      dx = float(i) / 512.0;
      for (int j = - BLUR_LIMIT; j < BLUR_LIMIT; j++) {
        dy = float(j) / 512.0;
        fragmentColor += texture2D(sampler1, vec2(vTexCoord1.s + dx, vTexCoord1.t + dy)) / BLUR_LIMIT_SQ;
      }
    }
    //Add real image
    fragmentColor += texture2D(sampler2, vec2(vTexCoord1.s, vTexCoord1.t));
  }
  gl_FragColor = vec4(fragmentColor.rgb, 1.0);
}

This is not the best way to make a bloom filter since it’s O(N * N), but it could be easily transformed into a two-pass bloom filter with the postProcess chaining API:

  //Make a horizontal pass
  Media.Image.postProcess({
    fromTexture: 'world2-texture',
    toFrameBuffer: 'bloom1',
    program: 'bloom',
    width: 1024,
    height: 1024,
    uniforms: {
      'orientation': 1
    }
  //Make a vertical pass on the result
  }).postProcess({
    fromTexture: 'bloom1-texture',
    toFrameBuffer: 'bloom2',
    program: 'bloom',
    width: 1024,
    height: 1024,
    uniforms: {
      'orientation': 0
    }
  //Combine the result to the original texture
  }).postProcess({
    fromTexture: ['world-texture', 'bloom2-texture'],
    toScreen: true,
    program: 'glow',
    width: 1024,
    height: 1024
  });

The result is:

Lazy Picking

The World Flights example displays around 6.000 cities served by the airlines. If you hover those cities a tooltip will display showing information about the city:

As mentioned in older posts, I implemented the color picking algorithm which assigns a different “picking color” to each city, then renders all 6000 cities to a texture with those picking colors and retrieves the color of the pixel pointed by the mouse. From that color it’s easy to identify which city has been hovered.

The current algorithm though renders this alternate scene each time the mouse moves over the canvas. A better approach would be to only take a snapshot of the scene when it’s updated in some way (the planet is dragged and dropped, a new airline is selected and the earth moves, etc.).

This can now be done by setting lazyPicking to true and then using a scene.resetPicking() method when the time is right to take a new snapshot. For all other cases a cache of the capture stored in a typed array will be used improving the performance of the algorithm:

events: {
  //enable picking and lazy picking
  picking: true,
  lazyPicking: true,
  //remove the center origin option for picking
  centerOrigin: false,
  onDragStart: function(e) {
    //do stuff...
  },
  onDragMove: function(e) {
    //do stuff to rotate the earth...
  },
  onDragEnd: function(e) {
    //reset the picking by removing the cached image capture
    this.scene.resetPicking();
  },
//more code here...

You can find more information on both these features in the Media and Scene APIs.

Thanks

Thanks to Luz Caballero for helping me out with the design of World Flights!

Upcoming Talks

2011-09-06T00:00:00-07:00

I thought I’d put my schedule for the upcoming talks here so I can convince whoever is reading this blog to come and meet me at some of these events! :)

I’ll be speaking at:

Data Visualization Group in Bay Area on New Tools for Visualization in JavaScript.
Visual.ly Data Visualization Meetup on The Ingredients of a Great Data Visualization.
Tech Talks at Mozilla on PhiloGL and its API.
SenchaCon 2011 in Austin, TX on WebGL fundamentals.

I hope to see you there!

PhiloGL 1.3.0 released

2011-08-07T00:00:00-07:00

Just a quick post to announce that PhiloGL version 1.3.0 has been released!

I’ve been working hard on this new release, which is mainly focused on many performance improvements and the ability to have reflection and refraction available in the default shaders.

Performance Improvements

Performance improvements happened on three different places of the framework:

Math functions

Math functions for Vector and Matrix operations are faster than ever. I changed the inner implementation of the math methods to use typed arrays whenever possible. The result is an amazing performance improvement that can be seen on the WebGL Matrix Benchmarks for Safari, Firefox and Chrome (these tests were made on a 13-inch MacBook Pro Core 2 Duo).

The API for the math module has changed a little bit. I explain most of the API changes in the documentation (look for new from version 1.3).

Deep typed array integration in O3D.Models

I changed how attributes where handled by models in the O3D module, integrating typed arrays very deep and reusing them whenever possible. In order to do this I created getters and setters for most of the model attributes like vertices, normals, color, etc. so you can feed to those properties either a typed array or a simple array of values and they will be converted internally into a typed array. The typed array will be reused whenever possible. When accessing some of these properties, you will always get a typed array too; even if you didn’t assign one to that property.

Micro Optimizations

Did you know that using forEach instead of a for-loop can be up to 50% slower in most browsers?. Some micro-optimizations take care of changing forEach calls into regular for-loops in the main rendering loop code.

Reflection / Refraction

I added the ability to have reflection and refraction in O3D. There are two new properties in O3D.Model: reflection and refraction which indicate the reflection and refraction index correspondingly.

I created two simple demos to demonstrate some of the new features in the framework.

The first one uses the marching cubes algorithm and web-workers as a divide and conquer technique to render reflective metaballs. I also use cube map textures and the new reflection index. You can access the example here.

The second example displays an interesting technique to render quaternion julia sets. Quaternion julia sets are four dimensional fractals that are created by using the same formula than the one used with two dimensional fractals, Zn+1 = Zn + C, but instead of using complex numbers as Z we use hyper-complex numbers called quaternions. Once we obtain the four-dimensional shape, what we do is project it into a three dimensional space to get “the shadow” of the four dimensional object. A very interesting article into how this works and also how this can be fully calculated via a fragment shader can be found here. I really enjoyed reading that article and recommend it to anyone interested in math :)

Anyway, you can access the example here.

SIGGRAPH 2011 WebGL BOF

Last but not least, I’ll be making a presentation on PhiloGL at SIGGRAPH 2011 in Vancouver on August 10th. If you’re there, come say hello!

PhiloGL 1.2.0 released

2011-06-06T00:00:00-07:00

I’m very excited to announce that PhiloGL version 1.2.0 has been released!

This release adds a lot of goodies to the Framework, like a much better design for multiple program management, a better design for picking enabling us to have finer grain picking, a brand new Quat class for quaternions and a couple of new primitives and improvements in O3D. All this with some updated documentation and a new demo!

Multiple program management

We always had the possibility to create multiple programs and use them in a PhiloGL application (see lesson 13), but in a more object-oriented model of applications we might want to assign programs to objects themselves than manually managing programs here and there.

You can define multiple programs right in the main PhiloGL constructor function as an array of program configurations with a specified id:

PhiloGL({
  program: [{
    id: 'program1',
    from: 'uris',
    vs: './vertex1.vs.glsl',
    fs: './fragment1.fs.glsl'
  }, {
    id: 'program2',
    from: 'uris',
    vs: './vertex2.vs.glsl',
    fs: './fragment2.fs.glsl'
  }]
  //other options here...
});

Then you can set a program id to any model you like by instanciating a model with:

var myObject = new O3D.Model({
  program: 'program1'
  //other options here...
});

and then each model will have a program assigned and used before being rendered. Easy!

Finer grain picking

The picking algorithm used in PhiloGL is a color picking algorithm. Each model is assigned a different color and the scene is rendered to a texture. Then, the pixel pointed by the mouse position is retrieved from the texture and the color of that pixel is used to identify the model.

Sometimes we want to know more than just which object has been picked. For example, we might want to know which face of that object has been picked. In that case the O3D constructor options pickingColors and pick are useful. By defining your own set of per vertex colors and a method that given a pixel returns special information on what part of the object has been retrieved, then it is possible to have finer grain picking. For more information about how to use this you can take a look at the World Flights example or go to the Google group of the framework.

More stuff

A Quat class has been added and also other interesting options in O3D.Model constructors have been included, like an attributes object to define buffers and attributes per model.

World Flights

World Flights is a data visualization powered by PhiloGL that displays the routes of all major airlines in the globe. Red markers indicate cities served by airports, and blue curves show the routes each airline makes and their destinations. You can drag and drop the globe, and also select new airlines from the controls on the right. The link to the application is here.

WebGL camp #3 at Google HQ

Last but not least, I’ll be making a presentation on PhiloGL at WebGL camp this Friday. Regirstration is still open, so I hope to see you there!

PhiloGL 1.1.1 and JavaScript InfoVis Toolkit 2.0.1 released

2011-05-05T00:00:00-07:00

Just a quick post to let you know about two patch releases for the JavaScript InfoVis Toolkit and PhiloGL respectively.

JavaScript InfoVis Toolkit 2.0.1

The new patch release for the JavaScript InfoVis Toolkit includes many bug fixes (that you can find here) and new configuration options for Area, Pie and Bar charts.

As I mentioned before I’m very excited that the project has been selected for the Google Summer of Code 2011 program, and that two student applications have been already accepted. The selected students will work on making the toolkit better by implementing new visualizations like Voronoi TreeMaps, Scatter Charts, Line Charts and other work related to code design and refactoring. Expect some interesting updates on the project in the next few months!

PhiloGL 1.1.1

PhiloGL version 1.1.1 has been released with a bunch of bug fixes and improvements. Multiple point lights (up to 50) are supported now. You can check a complete changelog here.

Our entry for Google's Data Visualization Challenge

2011-03-27T00:00:00-07:00

Launch the Visualization by clicking here! (browsers supported are the latest of Google Chrome, Firefox, Opera and Safari -not IE).

Luz Caballero and I have been working together for a couple of weeks to create an interesting visualization for Google’s Data Visualization Challenge.

Every year, Americans fill out income tax forms and make a payment to the IRS. Where does it all go? Using data provided by WhatWePayFor.com, we created a data visualization that will make it easier to understand how the government spends our tax money.

The Visualization

We want to help people grasp what these millions and billions mean… 43.03M on Federal aid to highways… is that too much or too little? The Federal Budget is traditionally divided in 20 “functions” that, under names like “National Defense” or “Energy,” show the purposes to which money is allocated. Our goal is to show not only how much money is spent for each function on a given year, but also

How does that compare to other functions? What are our priorities? Would cutting this give us money enough to fund that?
How has the amount spent on each function changed over the years? Hopefully this should give us all an idea on what has traditionally been considered normal highway spending ;)
Where does the money allocated to each function finally go? (Because we would never have guessed that we would find a USD 150M bailout under “Other advancement of commerce”!)

This visualization is made of two components. First, a tree layout shows, for each given year, how the budget (blue) is divided on each function (green), subfunction (yellow), and “account” (the most granular subdivision accounted for in the budget, which gives an idea of the use to which the money is put) (red):

The hegiht of each node/section in the tree is proportional to the amount of the budget spent on that function, subfunction or “account”. You can click on the nodes to navigate and filter functions and subfunctions. To go back you can right-click or click in the back arrow on the left of the visualization.

The second component of the application is a timeline that puts the amounts you are visualizing in the tree in a temporal context, showing the evolution of that particular spending category through time. Click on the bars representing the different years to navigate through time.

We wanted to allow our users to compare today’s spending to the 80s’ without artifacts caused by inflation, and changes in population and GDP. This is why our visualization shows inflation-adjusted USD, gives you the the option to see spending in USD, in USD per capita, or as a percentage of the time’s GDP. Plus, we wanted to give our users a good shot at pinpointing if/when spending changes are a consequence of government policy or of the economic climate. This is why you can overlay indicators like debt, GDP, inflation and yr/yr population change, to give you an idea of how they affect and are influenced by spending trends.

Interaction

What makes this application most interesting is the interaction between the components, which are tightly coupled.

When a function is selected, the timeline will adapt to show the budget spending trend for that particular function over the years.

When clicking on a timeline element, the tree will animate to show how the budget is spent on each function, subfunction and account, for the selected year.

If you are interested in comparing how the budget changed between two different moments in time, there is an option to transition between years using colors, red and green, to highlight a decrease or increase of total budget spending for each item.

Demo Video

We created a video that shows how to navigate and filter data and some interesting finds we made using our visualization. Click here to see the video in HD.

Acknowledgements

We would like to thank the people who gave us feedback on this visualization and helped us to improve it. Thanks for your help!

Click here to access the visualization!

Google Summer of Code and NASA

2011-03-22T00:00:00-07:00

The JavaScript InfoVis Toolkit selected for Google SoC 2011

The JavaScript InfoVis Toolkit has been accepted for the Google Summer of Code 2011!

An ideas page with some interesting tasks to complete for the project has been posted, and also we already got some awesome feedback from highly motivated students wanting to participate. The feedback has been great so far!

Being part of the Google Summer of Code 2011 is great not only for the JavaScript InfoVis Toolkit as a project, but also for our mentoring organization, Sencha Labs, to connect with students to promote Open Source activities and help build a new generation of JavaScript developers.

If you like interesting challenges with JavaScript and visualization don’t hesitate to contact me at nicolas at sencha dot com.

NASA

I have more exciting news: I’ve been invited by NASA to the first NASA Open Source Summit!

The summit will bring together 60 leaders from the open source community, NASA developers, and NASA policymakers to lay the foundation for improving how NASA develops, releases, and uses open source software.

I’m really looking forward to meet everyone there!

PhiloGL version 1.1.0

2011-03-11T00:00:00-08:00

PhiloGL version 1.1.0 has been released! This version includes a couple of new features like improved WebGL detection, linear fog, picking, added primitives like Cylinder and Cone and some bug fixes with directional lighting.

Better hasWebGL

Until now WebGL was currently detected via the existence of the WebGLRenderingContext object in the window object. This check is good to see if the browser we’re using provides access to the WebGL API, but doesn’t really check whether the hardware is capable to render WebGL - or at least create a WebGL rendering context.

In other words, we could have Firefox 4 installed on a ventilation system and get true for the check of !!window.WebGLRenderingContext.

In order to go a little further on the check I transformed hasWebGL into a static method that checks whether the WebGLRenderingContext exists but also checks whether the 3D context can be created. This is a noticeable step forward to check for WebGL capable machines. It’s worth a small API change (from static property to static method). You can check the documentation for this method here.

Picking

Picking is the ability to interact with 3D objects in the scene via events like mouseenter or click. Picking can be used by setting pick to true in the events configuration for the PhiloGL constructor. Internally the Event object uses the Scene pick method.

For example:

   //Create App
    PhiloGL('surface-explorer-canvas', {
      events: {
        //enable picking
        picking: true,
        //now the model is returned as the target of
        //the event.
        onMouseEnter: function(e, model) {
          model.uniforms.colorUfm = [1, 1, 1, 1];
        },
        onMouseLeave: function(e, model) {
          model.uniforms.colorUfm = [0.5, 0.5, 0.5, 1];
        }
      },
      onError: function() {
        alert("There was an error while creating the WebGL application");
      },
      onLoad: function(app) {
        /* do stuff with app */
      }
    });

Fog

Linear fog was added in version 1.0.2, but only now it’s official. Fog can be selected as a scene effect in the configuration object. You can find more information about how to configure linear fog here.

Primitives

I added Cone and Cylinder as new primitives. You can find documentation about them here.

Mini Demo

I made a video featuring fog, picking and our new primitives. In this particular example I only set cones and spheres as pickable objects.

I hope you enjoy the new features :) You can download the library here.

The White House uses the JavaScript InfoVis Toolkit

2011-03-03T00:00:00-08:00

I was surprised when I entered the main White House website the other day to find out that this logo was showing a very familiar treemap.

Clicking on the logo took me to another page where this treemap was featured. The application / visualization showing the 2012 budget is made with a TreeMap, in JavaScript, using the JavaScript InfoVis Toolkit. Click on this link to go to the White House page that features the visualization.

PhiloGL

2011-02-22T00:00:00-08:00

For some time now I’ve been working on a new project built around WebGL that today I’m very proud to release: PhiloGL.

PhiloGL is a WebGL Framework for advanced Data Visualization, Creative Coding and Game Development.

PhiloGL uses cutting edge technology and JavaScript idioms and good practices to deliver elegantly coded WebGL applications that are focused on performance. PhiloGL also provides a rich module system covering Program and Shader management, IO, XHR, JSONP, Web Worker management, Effects and Tweening, and much more.

PhiloGL is Open Source Software, licensed under the MIT license and owned by the SenchaLabs foundation.

PhiloGL has a complete API documentation with a detailed description of all the modules and class methods. Another very cool thing is that all Learning WebGL lessons have been ported into PhiloGL code so that you can easily learn PhiloGL, WebGL or both at the same time. You can find a list of the ported Learning WebGL lessons here.

In order to show how PhiloGL applications are made I created three featured projects that deal with Data Visualization in different aspects. In order to see these examples you’ll need the latest Google Chrome and/or Firefox 4. If you don’t have a WebGL capable browser then you’ll see a video showcasing the application.

Visualizing Temperature Anomalies

Access the example here.

NASA collects year by year data about temperature changes around the globe. This information has been collected since 1880 and tracks temperature anomalies (changes) in different points of the earth as a 2D heatmap. By loading these images into textures and then mapping them into a 3D histogram we are able to track the temperature changes around the globe interactively. Smooth animations between date ranges enable us to spot the overall differences in temperature across the years. Can you guess what the temperature changes will be for the next decade? Access the example here.

Real time 3D color Histogram Analysis

Access the example here.

Color decomposition on RGB or other color schemes takes a 3D form. In order to show the color decomposition of an image each dot on a 3D color histogram is assigned a different diameter. The following example renders fine grain and precise color decomposition schemes of each frame of a video in real time. Notice how color varies on each take, and feel free to change the color scheme or pause the video to have a closer look at the color decomposition. You can interact with the visualization by using drag and drop and zooming in/out with your mouse wheel. Access the example here.

Explore 3D Parametric Surfaces

Access the example here.

Exploring 3D surfaces is a fun way to learn about geometry, physical phenomena or even terrain modeling. Parametric surfaces are surfaces that also change their shape with an extra variable: time. Explore 3D surfaces by using drag and drop and the mouse wheel to zoom in/out. You can enable the t parameter (by clicking on the checkbox) and hit play to see it move. There are three predefined surfaces for you to explore or modify. Access the example here.

Go get it!

I hope these examples show what you can do with the framework and interest you enough to explore the code. I’ve had a lot of fun building and using PhiloGL and I can’t wait to see what you’re capable of doing with it, you can download it here!

Our entry for the Mozilla Open Data Visualization Challenge

2010-12-14T00:00:00-08:00

Launch the Visualization by clicking here!

Maria Luz Caballero and me have been working together for a couple of weeks to create an interesting visualization for the Mozilla Open Data Visualization Challenge.

Mozilla is releasing a new version of Firefox, version 4. This new version introduces a new UI and they have been monitoring how users interact with the different UI components of Firefox 4. They gathered some interesting information about how UI components are accessed and other elements taken by a survey to the Firefox users describe their expertise level, age, place where Firefox is used, etc.

We decided that this information could be visualized as a TreeMap in which the area of each node is proportional to the number of times a UI component has been accessed. For example in this image we can see that around 20% of UI interactions happen in the Navigation Toolbar, and it’s possible to compare the area of this rectangle to others to give us a clue of the overall interactions:

Each node also has a color that spans from red to blue, indicating an average of the expertise level of the user interacting with this component.

It’s interesting to note how some components are more often accessed by expert users than by beginners and viceversa. However since we provide an average we lack the information about the real distribution of user skills accessing the component. In order to avoid misleading results, when drilling down the TreeMap PieCharts indicate the real user skill distribution for each UI component interaction.

What I believe is by far the most interesting aspect of this visualization is the fact of using the ordinal and nominal information provided by the user survey to provide dynamic filtering for the TreeMap. Dynamic filtering (or morphing) enables an easy comparison between different categories of users on how they interact with these components. The filter panel we provide is shown below:

If you’re interested in taking a look at the visualization and playing with it please go to our entry. Browsers supported are Firefox 4, Opera 10.6+ and the latest of the Webkit family.

Animating Isosurfaces with WebGL and Workers

2010-12-10T00:00:00-08:00

If you just want to take a look at the JavaScript demos with WebGL and Workers you can click here or just go towards the bottom of the page

For some time now I’ve been considering rendering isosurfaces with JavaScript. Isosurfaces are 3D surfaces that can be defined with implicit 3D equations.

While “regular” 3D surfaces are defined as:

  z = f(x, y);

isosurfaces can also be represented as:

  c = f(x, y, z);

Some shapes that can be represented as implicit equations and that can’t be rendered just as “regular” 3D surfaces include the Sphere. While with regular surfaces we can have either the top or bottom part of a Sphere:

  z = f(x, y); // ->
  
  z = (+|-)Math.sqrt(1 - x * x - y * y); //either + or -

Implicit 3D equations can define the whole shape:

  c = f(x, y, z); // ->
  
  1 = x * x + y * y + z * z;

Implicit 3D equations can be used to model organs after taking a computed tomography, or to model other physical phenomena (like fluids), or even to do old-style OpenGL animated demos like Metaballs :P

How do I render an Isosurface?

There’s this very interesting Marching Cubes algorithm that describes how to create primitives that can be later sent to an OpenGL renderer in order to plot Isosurfaces (the primitives are triangles in this case). The main principle of the algorithm is to create some sort of differentials by dividing the space in which the surface is defined into “very small” cubes within the main volume where the visualization is contained and study how the surface intersects these small cubes (when it does). We can identify in this way 256 different cases in which the surface can “cut” these small cubes:

For each of these cases we can obtain the group of triangles corresponding to the part of the surface that intersected the small cube. That way we can recreate the surface to be rendered.

Approaching the Problem with WebGL

The power of WebGL lies in being able to compute these algorithms in some shader language. In the WebGL programable pipeline we can write shader code at two different stages: when processing a vertex (vertex shader) or when doing per pixel manipulation (fragment shader). Unfortunately this means that we are forced to send to the vertex shader an array of predefined vertices to render the surface before computing anything on the GPU. There is no way to send any other type of information through the shader that once arrived to the vertex shader would allow it to “emit” new vertices.

For example, one approach I would have enjoyed doing is to send the grid details to the shaders and then calculate the triangles and emit new vertices to render the scene from the shader. However I haven’t found a way to do this with WebGL because there’s no geometry shader.

So we’re stuck on the client-side to calculate all vertices and then send the information to the vertex shader with WebGL. This proves to be a little bit slow.

Web Workers to the rescue!

The divide & conquer nature of the algorithm made me think that I could divide the computing space into an octree and parallelize each octant with a Worker. A worker is a JavaScript API that provides asynchronous threads of execution. One can post messages to the workers and add onmessage listeners to them to do something in particular when they finish the computation or they send us some message. Workers can call other workers and also one can create multiple workers out of the same code. For more information about Workers read the link above ;).

So now what I needed is something like a WorkerGroup that would create a certain number of workers and map different configurations for each of them, and then reduce all workers responses into a single “merged” result of vertices to send to the shaders.

A way to implement the WorkerGroup would be:

//Create an array of workers
function WorkerGroup(fileName, n) {
  var workers = this.workers = [];
  while (n--) {
    workers.push(new Worker(fileName));
  }
}

WorkerGroup.prototype = {
  //create a configuration array to post to each worker
  //when reduce is called
  map: function(callback) {
    var workers = this.workers;
    var configs = this.configs = [];

    for (var i = 0, l = workers.length; i < l; i++) {
      configs.push(callback(i));
    }
  },
  //run all workers with their proper configuration and 
  //define a group-callback to merge the results
  reduce: function(opt) {
    var fn = opt.reduceFn,
        workers = this.workers,
        configs = this.configs,
        l = workers.length,
        acum = opt.initialValue,
        message = function (e) {
         l--;
         if (acum === undefined) {
          acum = e.data;
         } else {
          acum = fn(acum, e.data);
         }
         if (l == 0) {
          opt.onComplete(acum);
         }
        };
    for (var i = 0, ln = l; i < ln; i++) {
      var w = workers[i];
      w.onmessage = message;
      w.postMessage(configs[i]);
    }
  }
};

The performance improvements of using Workers are very noticeable. The algorithm remains on the client-side though, so if you want fast performance I’d recommend you to use some webkit nightly.

The Result

I made two demos, one is a Metaballs demo and the other one is an Isosurface visualizer. I really like the second example because you can specify your own formulas. There are predefined formulas like a Torus or Gyroid. To see these demos you’ll need a WebGL enabled browser, I do encourage you to install a webkit nightly for this.

Click here to take a look at the Metaballs demo, or you can watch a video here:

For the Isosurface generator you can click here or take a look at the video below:

GitHub

I’m pushing all experiments from now on to a Playground repo for you to check the code or improve. There’s also some 2D Canvas, WebGL and Scala code from Three ways to make 3D there.

My Talk at YOW! Developer Conferences

2010-12-09T00:00:00-08:00

Last week I was in Melbourne and Brisbane, Australia, to speak about interactive data visualizations with JavaScript at the YOW! Developer Conferences. My presentation was a one hour talk in which I covered some InfoVis Theory basics (some ideas from Jacques Bertin’s Semiology of Graphics); also I spoke about the main things the JavaScript InfoVis Toolkit has to offer and finally I presented some future work involving WebGL, web workers and some cool algorithms I’ll be presenting in a future post.

I want to thank Dave Thomas and the entire team for giving me the opportunity to speak in Brisbane and Melbourne. I had the honour to speak right before Ben Fry’s Computational Design talk and Dan Ingalls 40 years of fun with computers - both amazing presentations.

You can find the slides for my presentation here, the audience went wild with my presentation in Brisbane!

Creating Interactive Data Visualizations for the Web - YOW! Developer Conferences Australia

View more presentations from philogb.

One of the highlights of YOW! for me was to see Dan Ingalls, father of five generations of Smalltalk, winner of two ACM awards and now working with cutting-edge web technology as SAP fellow introduce bindings for the JavaScript InfoVis Toolkit in the lively-kernel:

Being at YOW! Developer Conferences was a great experience.

Sencha Acquires the JavaScript InfoVis Toolkit

2010-11-06T00:00:00-07:00

Sencha Inc., a company that focuses on client-side technologies acquired this Friday the JavaScript InfoVis Toolkit!

What does it mean?

Sencha now owns the toolkit. Following the Sencha Labs philosophy, the project will remain with an Open Source license (BSD), and will be part of the Sencha Labs foundation. This means that someone from Sencha will take some time to maintain and still develop the toolkit from now on. That person can be me, but I also encourage the Sencha Team to take a closer look at the project, there are many cool concepts in there that even trascend the fact that it’s done in JavaScript, and I would love to get some help on the development of the toolkit. Oh, and yes I’m looking at you too reader, the project has been up for more than two years and you still haven’t contributed? Now is the time!

What does it mean… for me?

I started this project all by myself three years ago. I still remember I was playing around with Hyperbolic Geometry when the experiment was featured for the first time in Ajaxian. My intention at first was far from building an Open Source project or gathering a small community of people interested in it, it was just about learning Data Visualization. After taking a look at Tamara Munzner’s Google Tech Talk, I was amazed by this field and tried to learn all I could about state of the art infovis.

Then I moved to France to work for Exalead. I was primarily working with the Java stack on enterprise products (not related to infovis). Then I would go home and spend more time learning infovis and putting into practice the things I learnt by hacking.

One of the things I found interesting about infovis papers is that they generally provide an overview of some visualization technique, they provide a user study to show how the visual encodings presented are better in some situations, but there is little concrete talk about the implementation itself. Of course, the papers are about Data Visualization, not Computer Graphics. We’re not supposed to be reading pseudo-code, but instead how a visualization provides new insights on data. That “lack of information” about a concrete implementation was very interesting to me at first: each visualization became a technical challenge. With the time I got to learn something very interesting though. The real challenge lies in the creativity required to invent a visualization that actually helps the user and increases his/her overall productivity when browsing some dataset.

At first the toolkit was… WYSIWYG: I was just experimenting with infovis and since I was building everything from scratch design errors showed up. There were some early adopters though, like OpenCRX that included the toolkit as part of their mobile application. With time the toolkit got much better, and people from Google, Mozilla, The Guardian, CERN, UBS, Platform Computing and more started using it.

Along with Protovis, I believe that the JavaScript InfoVis Toolkit became the most robust solution to provide Web based interactive Data Visualizations.

Thanks to the toolkit I got the chance to meet the InfoVis community in Paris by speaking at the Parisian Seminar on InfoVis and HCI, and to present the toolkit at JSConf in Berlin. I also began working on building visualizations at Exalead Labs, and the toolkit was also one of the reasons I moved from France to California to work for Sencha. Also I’ll be presenting the toolkit at the YOW! Developer Conferences in Australia this year.

Within this three years I also learnt how to create, license, develop and maintain an Open Source project and its community by hosting the project, accepting collaborations through GitHub, answering questions at the Google Group, etc. This might seem silly to some of you, but maintaining an Open Source project requires some hard work and dedication.

The project has reached a maturity I’m very proud of. And there are still much things to work on. Exploring new visualization techniques and robust WebGL support are just a few things that come to my mind.

Analyzing JavaScript Class Patterns Performance

2010-10-10T00:00:00-07:00

For a while now I’ve been considering different JavaScript class patterns and their performance when initializing and calling methods from them. Depending on the use-case some class patterns can show considerable performance improvements over the typical pseudo-classical patterns. In some cases, the use of closures and private member initializers in function constructors, as opposed to what’s stated by the MDC, can also show performance improvements over classical approaches.

JavaScript class performance can be analyzed by altering two variables: the JavaScript class can have public/private properties and/or
public/private methods. On the other hand, code performance can be measured on class definition, class constructors, or instance methods.

I’ll just state the conclusion before making the analysis: there’s not a unique class pattern that will make your code faster in all cases. Most of the time it just depends on your use-case. In most cases people try to delegate all computations at class definition, since this generally happens only once. When this is not possible there’s a tradeoff between having computations in the class constructor or performing these computations when a method is called.

The premise for this analysis is that the extra-computations can’t be avoided, and their complexity can’t be simplified. In some sense, they’re just like a black box.

In the next section I’ll show how you can use closures, variable scope and a class prototype to delegate computations at class definition, class constructor or to just keep the computations when calling a method.

Class.prototype 101

Any JavaScript object, (String, Object, RegExp, etc), generally has two type of properties: the own properties and the properties found in the prototype chain. Each object has a special property proto that actually points to another object (or is null).

When we try to access a property in an object:

var propertyValue = object.myProperty;

it will first try to see if it’s an own property, something that can be checked by doing:

Object.prototype.hasOwnProperty.call(object, 'myProperty');

and use that property if it exists. If it doesn’t exist then it will try to find that property inside the object pointed by the proto property:

object.__proto__.myProperty;

If the property isn’t found in the object pointed by proto then it will try to find it recursively in the prototype chain (object.proto.proto, etc) until it gets to the top of it.

When we create a Class with a prototype:

 function Person() {
    //constructor
 }

 Person.prototype.sayHi = function() {
    alert("hi");
 };

 var p = new Person();
 p.sayHi(); //"hi"

Internally JavaScript will assign to any Person instance proto property the Person.prototype object we defined above. This means that we actually define an object with methods only once, and any instance of Person will have a proto property pointing to those methods. Since all Person instances point to the same Person.prototype people usually say they share those methods, and so they’re shared properties across instances.

var p = new Person();
p.__proto__ === Person.prototype //true
p.sayHi === p.__proto__.sayHi //true

The convenience about this is that we don’t have to execute an entire code that actually re-creates methods and assigns them to a newly created object in the function constructor. Since each time new Person is called the Person function is executed, it would be better to just define the methods in the prototype object for once and not execute code for augmenting an object with new methods each time we call Person.

Let’s see how this could be used to increase the performance of certain class patterns.

Public Properties

In the public properties class definition we just move all properties definitions from the class constructor to the prototype object. By moving these computations we can see that there are some interesting performance improvements on constructor calls.

The code preparations are:

function ConstructorPublicProperties() {
  this.publicProperty1 = 1;
  this.publicProperty2 = 2;
  this.publicProperty3 = 3;
  this.publicProperty4 = 4;
  this.publicProperty5 = 5;
  this.publicProperty6 = 6;
  this.publicProperty7 = 7;
  this.publicProperty8 = 8;
  this.publicProperty9 = 9;
  this.publicProperty10 = (function() {
   var i = 100;
   while (i--);
   return 10;
  })();
}

function PrototypePublicProperties() {}
 
PrototypePublicProperties.prototype = {
  publicProperty1: 1,
  publicProperty2: 2,
  publicProperty3: 3,
  publicProperty4: 4,
  publicProperty5: 5,
  publicProperty6: 6,
  publicProperty7: 7,
  publicProperty8: 8,
  publicProperty9: 9,
  publicProperty10: (function() {
   var i = 100;
   while (i--);
   return 10;
  })()
};

The code we actually compare is:

for (var i = 0; i < 1000; i++) {
 new ConstructorPublicProperties();
}

vs.

for (var i = 0; i < 1000; i++) {
 new PrototypePublicProperties();
}

As you can see by the test case that you can run yourself moving computations to the prototype object can lead to considerable performance improvements in class constructors.

Public Methods

For public methods the case is just the same than with public properties. Moving all definitions and initializations to the prototype object leads to performance improvements in class constructors.

The preparation code for this is:

function ConstructorPublicMethods() {
  this.publicMethod1 = function(n) {
   return 2 * n;
  };
 
  this.publicMethod2 = function(n) {
   return 3 * n;
  };
 }
 
 function PrototypePublicMethods() {}
 
 PrototypePublicMethods.prototype = {
  publicMethod1: function(n) {
   return 2 * n;
  },
 
  publicMethod2: function(n) {
   return 3 * n;
  }
 };

The test code is:

for (var i = 0; i < 1000; i++) {
 new ConstructorPublicMethods();
}

vs.

for (var i = 0; i < 1000; i++) {
 new PrototypePublicMethods();
}

As you can see constructors are also faster when public methods are defined inside the prototype object.

Private Methods

Private methods can be defined in JavaScript by using the class constructor as encapsulation for other function definitions. This is a pattern I’ve seen done by Douglas Crockford and is used in many other places:

function ConstructorPrivateMethods() {
  function privateMethod1(n) {
   return 2 * n;
  }
 
  function privateMethod2(n) {
   return 3 * n;
  }
 
  this.publicMethod1 = function(n) {
   return privateMethod1(privateMethod2(n));
  };
 }

If you’re only defining private methods (and not private properties) you can also augment the prototype object with public methods and use encapsulation for defining private methods around the prototype object. I’ve seen this clever technique done by WebReflection some time ago:

function PrototypePrivateMethods() {}
 
 (function() {
 
  function privateMethod1(n) {
   return 2 * n;
  }
 
  function privateMethod2(n) {
   return 3 * n;
  }
 
  PrototypePrivateMethods.prototype = {
   publicMethod1: function(n) {
    return privateMethod1(privateMethod2(n));
   }
  };
 
 })();

The code tested is the same as in the previous sections:

for (var i = 0; i < 1000; i++) {
 new ConstructorPrivateMethods();
}

vs.

for (var i = 0; i < 1000; i++) {
 new PrototypePrivateMethods();
}

As you can see for yourself the prototype version has the fastest constructor. The performance improvements are considerable with this pattern which is interesting in use-cases such as when defining some helper functions to be used with a class.

Private Properties

The examples above are kind of simple to analyze: moving the computations from the class constructor to the prototype object can lead to faster constructors, and more generally can increase performance since the code is computed once at the class definition and not once per each constructor call.

With private properties there’s a tradeoff. Since private properties are generally accessed and/or modified by public methods, defining a private property inside the constructor forces the definition of the public method into the class constructor too, decreasing the overall performance of the class constructor. Another interesting technique could be to just use an underscore prefix for private members:

function ConstructorPrivateProperties() {
  var privateProperty1 = 1,
      privateProperty2 = 2,
      privateProperty3 = 3,
      privateProperty4 = 4,
      privateProperty5 = 5,
      privateProperty6 = 6,
      privateProperty7 = 7,
      privateProperty8 = 8,
      privateProperty9 = 9,
      privateProperty10 = 10;
 
  this.publicMethod1 = function() {
   return privateProperty1 
        + privateProperty2 
        + privateProperty3 
        + privateProperty4 
        + privateProperty5 
        + privateProperty6 
        + privateProperty7 
        + privateProperty8 
        + privateProperty9 
        + privateProperty10;
  };
 }
 
 function PrototypePrivateProperties() {}
 
 PrototypePrivateProperties.prototype = {
  _privateProperty1: 1,
  _privateProperty2: 2,
  _privateProperty3: 3,
  _privateProperty4: 4,
  _privateProperty5: 5,
  _privateProperty6: 6,
  _privateProperty7: 7,
  _privateProperty8: 8,
  _privateProperty9: 9,
  _privateProperty10: 10,
 
  publicMethod1: function() {
   return this._privateProperty1 
        + this._privateProperty2 
        + this._privateProperty3 
        + this._privateProperty4 
        + this._privateProperty5 
        + this._privateProperty6 
        + this._privateProperty7 
        + this._privateProperty8 
        + this._privateProperty9 
        + this._privateProperty10;
  }
 };

The code we’re testing is:

for (var i = 0; i < 1000; i++) {
 new ConstructorPrivateProperties();
}

vs.

for (var i = 0; i < 1000; i++) {
 new PrototypePrivateProperties();
}

As you can see for yourself using the prototype object yields faster constructors again. But there’s a catch: the public method for the augmented prototype object class access this.something instead of just accessing a local variable, which drops the overall method call performance considerably. You can see this for yourself by testing this code:

var instance = new ConstructorPrivateProperties();

for (var i = 0; i < 1000; i++) {
 instance.publicMethod1();
}

vs.

var instance = new PrototypePrivateProperties();

for (var i = 0; i < 1000; i++) {
 instance.publicMethod1();
}

The augmented prototype object version is the slowest. What’s also interesting is that using private properties defined in the function constructor can increase the overall performance of the methods called in your class (if they access instance properties).

So are you targeting faster constructors, class definitions or methods? The fact is that any of these three stages can be optimized, but generally with a tradeoff.

JSConf

2010-09-28T00:00:00-07:00

JSConf just finished. It was a great experience: I got to meet a lot of interesting people and attend awesome talks, and I even got the chance to speak about the toolkit!

The slides from my talk are here:

JavaScript InfoVis Toolkit - Create interactive data visualizations for the web

View more presentations from philogb.

All talks were great, some of the talks I really enjoyed were:

…and more.

You can follow JSConf for updates about when the videos will be released.

Finally I want to thank janl, cramforce and hblank for gathering all javascripters in Berlin, making an excellent event, and giving me the chance to introduce the toolkit to the JS crowd. You guys are awesome!

Jekyll

2010-09-20T00:00:00-07:00

So I just moved from WordPress to Jekyll, a Ruby based static site generator. Moving from WordPress wasn’t as pleasant as I thought, and the site might have some “bugs” (let me know if you find one!) but I’m much more happy to be writing this post in my favorite editor, and not having to rely on WordPress, PHP or MySQL to run the blog.

While moving stuff from WordPress I found a few interesting old posts, I hope you find them interesting too:

…oh well, there are more interesting posts, so I recommend you to go to the main page to see a list of them.

If you find some bugs please let me know!

JavaScript InfoVis Toolkit + WebGL

2010-09-17T00:00:00-07:00