A texture atlas is a large image that contains many sub-images, each of which is used as a texture for a different geometry, or different parts of the same geometry. The sub-textures are applied by mapping the geometries’ texture coordinates (UV) to different regions of the atlas.

In a scene where there are many small textures, this has the benefit of reducing state changes on the graphics hardware by binding once, instead of for each individual texture. It also reduces the number of HTTP requests for texture image files.

SceneJS state-sorts its draw list by shader (which is normally auto-generated by SceneJS), then by texture, so as long as each of the geometry nodes within the texture's subtree inherit the same configuration of parent node states, and can therefore share the same shader, the draw list will bind the texture once for all the geometries.

• Read the Wikipedia page on texture atlases
• Learn more about texture atlases in this white paper from nVIDIA.

Example

To show how texture atlases are done in SceneJS, we'll define a texture containing two sub-textures that will be applied to two separate quads, as shown below. You can run the live example here.

Shown below is our atlas texture, texture-atlas.jpg. The guy on the left is General Zod, from Superman:

Now we'll define a fragment of scene graph. To start with, we have a material node, which wraps a texture, which is our atlas:

{
    type: "material",
    baseColor:      { r: 1.0, g: 1.0, b: 1.0 },
    specularColor:  { r: 1.0, g: 1.0, b: 1.0 },
    specular:       0.9,
    shine:          6.0,

    nodes: [


        /*------------------------------------------------------------------
         * Our texture atlas
         *
         * The atlas is the General Zod portrait and the brickwall pattern,
         * arranged side-by-side within the same image file.
         *-----------------------------------------------------------------*/

        {
            type: "texture",

            layers: [
                {
                    uri:"texture-atlas.jpg",
                    applyTo:"baseColor",
                    blendMode: "multiply"
                }
            ],

Next, we have a couple of rotate nodes, just to tilt our quads a little bit:

nodes: [

                /*------------------------------------------------------------------
                 * Rotate our geometry a little bit to make it more interesting
                 *-----------------------------------------------------------------*/

                {
                    type: "rotate",
                    angle: -30.0,
                    x : 1.0,

                    nodes: [
                        {
                            type: "rotate",
                            angle: -30.0,
                            y : 1.0,

Next, we have the quads themselves, arranged along the X-axis with a couple of translate nodes. Although SceneJS has a special quad node, we'll manually define our quads in order to demonstrate how UV coordinates map to the textures. The first quad's UV coordinates map to the left half of the texture, to texture it with General Zod's image:

nodes: [

                                /* General Zod
                                 */
                                {
                                    type: "translate",

                                    x: 1.5,

                                    nodes: [
                                        {
                                            type: "geometry",
                                            positions : [ 1, 1, 0, - 1, 1, 0, -1, -1, 0, 1, -1, 0],
                                            normals : [ 0, 0, -1, 0, 0, -1, 0, 0, -1, 0, 0, -1 ],

                                            /* UV coords map to left half of texture image
                                             */
                                            uv : [ 1, 1, .5, 1, .5, 0, 1, 0 ],
                                            indices : [ 0, 1, 2,0, 2, 3]
                                        }
                                    ]
                                },

The second quad's UV coordinates map to the right half of the texture, to texture it with the brick pattern:

/* Brick wall
                                 */
                                {
                                    type: "translate",

                                    x: -1.5,

                                    nodes: [
                                        {
                                            type: "geometry",
                                            positions : [1, 1, 0, - 1, 1, 0, -1, -1, 0, 1, -1, 0 ],
                                            normals : [ 0, 0, -1, 0, 0, -1, 0, 0, -1, 0, 0, -1 ],

                                            /* UV coords map to right half of texture image
                                             */
                                            uv : [ .5, 1, 0, 1, 0, 0, .5, 0 ],
                                            indices : [ 0, 1, 2,0, 2, 3]
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ]
}

So there we have it - a useful technique to improve rendering speed by reducing WebGL state changes, while also reducing loading time by reducing the number of HTTP requests.

Note that we can have textures of various sizes in an atlas and use UV coordinates to scale them to fit the geometry vertices. 

Limitations

Take care if using mipmaps to arrange the textures so as to avoid sub-images being affected by their neighbours.

Note how SceneJS texture layers configure what geometry/material aspects their texture applies to (eg. "baseColor", "normals" etc). Therefore we can't currently have mixtures of those configurations within the same atlas, so we must have separate atlases for color-mapping, emission-mapping, normal mapping and so on. 

Future Work

• It would be very cool to make a server-side utility in Python or Node.js to optimise a SceneJS JSON scene by by aggregating it's textures into atlases. 

Permalink | Leave a comment  »

SceneJS ray picking finds the logical pick name of the picked object, plus the point in world-space at which a ray cast from the eye position intersects with object's surface. It's ridiculously fast because the Z-coordinate is found with the help of the GPU, while the X and Y are found by unprojecting the canvas mouse coordinates.

Ray-pick can therefore be done almost as fast as a normal frame render and scales up to huge amounts of detail, with no ray intersection testing against millions of triangles using JavaScript math.

In the screenshot below, we're doing an instantaneous pick on a scene containing a very large number of triangles, fast enough for mouse-over picking. Try the example live here.

The Technique

As shown in the wiki, a pickable scene contains name nodes that assign logical pick names collectively to the geometries within their subgraphs. Each name node has a "name" value, which multiple name nodes can share. In other words, we could have a house model with a name node around each window geometry, with each name having the value "window". Clicking on any window then picks "window".

The steps:

1. User clicks canvas at coordinates (X, Y). 
2. Do a render pass to a hidden frame buffer, in which the objects within each name node are are rendered in a colour that uniquely maps to node's "name" value (eg "window").
3. Read the colour from the framebuffer at the canvas coordinates, map the colour back to a name. Now we have the pick name.
4. Do a second render pass to another hidden frame buffer, this time rendering objects with each pixel colour being the clip-space Z-value packed into an RGBA value.
5. Read the colour from the framebuffer at the canvas coordinates and unpack it to the clip-space Z value. Now we have the clip-space Z, which will be in the range of [0..1], with near clip plane at 0 and far clip plane at 1.
6. Transform the canvas coordinates to clip-space. Make a ray from clip space (X,Y,0) to (X,Y,1) and transform that ray into world-space by the inverse view and projection matrices.
7. Linearly interpolate along ray by the value of our clip-space Z, to find the world-space coordinate (X,Y,Z).
8. Voila, we have the picked name, canvas (X,Y) and world-space (X,Y,Z) for the pick hit.

Packing clip-space Z in GLSL:

vec4 packDepth(const in float depth) {
     const vec4 bitShift = vec4(256.0*256.0*256.0, 256.0*256.0, 256.0, 1.0);
     const vec4 bitMask  = vec4(0.0, 1.0/256.0, 1.0/256.0, 1.0/256.0);
     vec4 res = fract(depth * bitShift);
     res -= res.xxyz * bitMask;
     return res;
}

Unpacking clip-space Z in JavaScript:

function unpackDepth(depthZ) {
        var vec = [depthZ[0] / 256.0, depthZ[1] / 256.0, depthZ[2] / 256.0, depthZ[3] / 256.0];
        var bitShift = [1.0 / (256.0 * 256.0 * 256.0), 1.0 / (256.0 * 256.0), 1.0 / 256.0, 1.0];
        return SceneJS_math_dotVector4(vec, bitShift);
    };

Calculating clip-space Z in the fragment shader:

Step (4) requires that we have the view-space position in the fragment shader, which we pass through from the vertex shader. It also requires us to feed the locations of the near and far clipping planes into the fragment shader (which we take from the scene's camera node). Using these, we calculate the clip-space depth like so:

float depth = (uZNear - vViewVertex.z) / (uZFar - uZNear);
gl_FragColor = packDepth(depth);

API

Through the API, the pick operation looks like this:

var hit = scene.pick(45, 150, { rayPick: true });  // Picking at canvas coordinates

if (hit) {
    alert("Picked 'name' node(s) with name '" + hit.name + 
             "' at canvas coords " + hit.canvasPos[0] + ", " + hit.canvasPos[1]
             "' and world coords " + hit.worldPos[0] + ", " + hit.worldPos[1] + ", " + hit.worldPos[2]);
} else {
    alert("Nothing picked");
}

SceneJS internally caches the hidden frame buffers to avoid re-rendering them. This means that when we do a subsequent pick, as long as a re-render is not neccessary after objects have moved or changed appearance, we just re-read the buffers without repeating any rendering passes.

For picking, many WebGL frameworks will save time by doing a picking render of only a 1x1 viewport at the canvas coordinates. SceneJS renders the entire view for picking so that it can cache the pick framebuffers as just mentioned. This is an optimisation geared towards fast mouse-over picking effects in model viewing apps, such as highlighting and tooltips.

Caveats

The technique described here trades accuracy for speed. Packing and unpacking the clip-space Z to and from a colour value is lossy. Hopefully in future it will be possible to instead read the WebGL depth buffer, which will preserve much more precision.

Another limitation is that this technique does not find any topological information on the pick hit: it only finds the name and a world-space coordinate. When picking a mesh for example, it does not report the actual face that was picked.

Permalink | Leave a comment  »

Last week saw the public beta release of BIMSurfer, an open-source  Web-based BIMServer front-end which was created by Rehno Lindeque and uses SceneJS as the rendering engine. 

Building Information Modelling (BIM) is the process of generating and managing building data during its lifecycle; BIMServer is the server that manages and queries that data, while BIMSurfer is a viewer that displays and interacts with it.

BIMServer is an open source project that tracks data on every part of a building under development: geometry, spatial relationships, light analysis, geographic information - whatever is useful, while supporting queries on the data. It also functions as a sort of version control system, allowing multiple users to collaborate on the model, while continually updating the overall data, sending notifications, preventing modification conflicts and so on.

One interesting feature is that, along with IFC models, BIMSurfer is able to import and export them as SceneJS JSON, which means they could be rendered using straight SceneJS. Another cool feature is the ability to author fly-throughs by creating a sequence of snapshots.

Permalink | Leave a comment  »

Today a SceneJS user asked the mailing list how content can be pulled into a scene graph on-demand, for an application in which the user browses a map view that loads and displays geometry, material and texture assets as they fall into view. 

Early SceneJS: The "instance" Node

Old versions of SceneJS had the instance node, which instantiated a target subgraph as a child. Really early versions of that were even able to pull the JSON for the target from disk and into the scene graph, on demand. The instance node was however removed in V2.0 because its implementation was so complicated, with the multiple parent paths it would create making engine optimisations impractical. It was replaced by shareable node cores in V2.0.

Latest SceneJS: Staging In-Core

On-demand loading of geometries can be done with a GeoLoaderService (the SceneJS plugin interface for custom mesh loading strategies), but there's nothing in V2.0 that loads whole subgraphs on demand, the way the instance node did.

Aside from implimentation simplicity, the reason is that SceneJS encourages staging content in-core. This means staging content in the scene graph, setting "enabled" flags false to cull it from the view until you want to render it,

For example:

{     
    type: "flags",     
    id: "my-teapot-content",     
    flags: { enabled: false },     
    nodes: [         
        { type: "teapot" }     
    ]
}

myScene.findNode("my-teapot-content").set("flags", { enabled: true });

Geometries, textures and shaders will be allocated on the GPU as the nodes are created in the scene graph. They will be bound on the GPU for rendering only when the geometries that use them are enabled.

You can stage as much content in your scene graph as your GPU has memory for, which these days is often as much as 1-2GB.

Internally, SceneJS optimises for in-core staging by maintaining a "visible list" that caches enabled (ie. visible) objects. Enabling an object causes the main draw list to be iterated on the next frame while saving the enabled objects into the visible list, which will then be the list that's iterated for each subsequent frame as long as more objects are not enabled. Then as objects are disabled, they are just plucked out of the visible list.

Creating and destroying Nodes

If your scene is too big to stage everything in the scene graph, then out-of-core content needs to swapped in and out by creating and destroying nodes:

myScene.findNode("some-node").add("node", { type: "teapot", id: "my-teapot" });
myScene.findNode("my-teapot").destroy();

Bear in mind that adding and destroying nodes cause scene-to-draw list recompilation (an internal optimisation) each time, so these add/destroys are best done in batches, and will cause a moment's delay when they happen.

If you're adding large numbers of nodes, you may want to pause the render loop on the scene graph while loading, just to ensure that the render interval doesn't keep kicking in and recompiling the draw list while you're still in the process of adding them:

myScene.stop();

// ..add many nodes..

myScene.start({
    // ...
});

So you can see how it's up to your application code to do this swapping, possibly under the direction of some sort of bounding volume visibility system. 

How to do this? In BioDigital Human we have our own application logic for swapping nodes and out of the scene, but that's driven by the user explicitly selecting what anatomy and conditions they want to view (Eg. switching gender, loading a beating heart etc).

A view-driven solution is a different kettle of fish. Way back in early SceneJS versions (around V0.7) I had "spheres of locality" centered about the viewpoint: a large outer sphere of say, 100000 units containing an inner sphere of 10000 units. The scene graph could have bounding volumes nodes which enclosed subgraphs: these nodes would load their subgraph when they intersected the inner boundary, then unload it as soon as they fell outside the outer boundary. SceneJS also performed frustum culling with the bounding volumes, enabling (making visible) the subgraphs whenever they intersected the view frustum.

All this was chopped out of V0.8 onwards to keep the engine lean and fast. As long as you are able to track the boundaries of things in application code, you should be able to implement this in third-party code using libraries like jsBVH. 

To be continued..

Permalink | Leave a comment  »

I've written a chapter on SceneJS for OpenGL Insights, a book on OpenGL and WebGL techniques which will be published in time for SIGGRAPH 2012. My focus is on the caching strategies and optimisations used to reduce JavaScript re-execution per frame, which may be relevant to others considering writing their own WebGL frameworks.

The book is shaping up to be seriously awesome, with quality chapters ranging from SpiderGL to efficient text in WebGL, plus many chapters on OpenGL ES techniques.

Permalink | Leave a comment  »

The SceneJS 2.0 release is finally out the door, focusing on high rendering speed for complex scenes containing many individually articulated and pickable objects, which is characteristic of model viewing applications for engineering, architecture and medical visualisation.

While most WebGL frameworks support multi-pass effects such as shadows and reflections, SceneJS 2.0 has evolved to specialise in optimised single-pass rendering of large amounts of detail. That's not to say, however, that we can't create some catchy effects using some of its new features, as shown in the live examples.

BioDigital Human (in development), with 1886 meshes and 126 textures visible, rendering at around 13-15 FPS in Chrome 14.0.835.202 on an i7 CPU and an NVIDIAGeForce GTX 260M GPU

What's New

New Wiki at GitHub

V2.0 has a new wiki over at GitHub, which I'm still in the process of building up. We'll keep the old V0.9 wiki around for a few more months to help people port to the new SceneJS release, then we'll archive it and make it available for download.

Custom Shaders

SceneJS normally generates shaders for us, but when special effects are required we can now inject custom functions into them to intercept and modify colours, matrices, coordinates etc, all parameterised via the JSON API. 

With custom shaders, we can intercept coordinate positions to do things like displace them for wobbling effects, or discard fragments for clipping effects.

In the fog shader example, we intercept the view-space position and fragment colour in the fragment shader, then fog the colour in proportion to the coordinate Z-depth.

The fancy transparency example creates the glass effect shown in the snapshot above by intercepting the view-space normal and fragment colour, then setting the colour's alpha to a value that is proportional to how directly the normal faces the view point.

We can also modify matrices, like in the sky sphere shader example, in which we zero out the translation elements to keep the sky sphere centered at the viewpoint.

The Tron Tank program even got a custom shader - note the blurred horizon line.

The colorTrans node from SceneJS V0.9 has been deprecated, since we can now do the same effect with something like this pixel colour shader.

Read more in the wiki page on shaders.

More Scene Graph Compilation 

SceneJS dynamically (re)compiles the scene graph to a draw list of optimised WebGL state changes, which in V2.0 is now further (re)compiled to a list of fast WebGL calls. SceneJS caches these lists to minimise JavaScript re-execution per frame, rebuilding only the affected portions of them whenever scene state updates occur.

The WebGL calls are higher order functions which memoize things like shader variable locations in closures. After adding this, I observed a ~30-40% speedup for some applications - expect the next version of BioDigital Human to be noticeable faster.

Shared Node Cores

Traditionally, re-use within a scene graph is done by attaching nodes to multiple parents. For dynamically updated scenes this can have a performance impact when the engine must traverse multiple parent paths in the scene graph, so SceneJS now takes an alternative approach with ”node cores”, a concept borrowed from OpenSG.

Shared node cores replace the instance nodes found in earlier versions of SceneJS.

A node core is the scene node’s state. Having multiple nodes share a core means that they share the same state. This can have two significant performance benefits: an update to a shared node core can write through to multiple draw list elements simultaneously, and there is increased chance of runs of same state objects in the draw list after state sorting, which SceneJS capitalises on by avoiding repeated execution of them.

Many of the examples (such as the custom shaders) demonstrate shared node cores. Read more on the wiki page on node cores or check out this rather plain example.

Video Textures

Movie files can be applied as textures - useful for complex animated textures, animated bump maps, moving backgrounds etc. Read the wiki page on video textures or try this this example.

Multiple Scenes

 Multiple scene graphs can now be created within the same JavaScript runtime environment, allowing us to update them concurrently and share JSON content between them. Here's a multi-scene example.

Mutable Geometry

Geometry VBOs and IBOs can now be dynamically updated. Try this very plain example.

Ray Picking

SceneJS ray picking finds the logical pick name of the picked object, plus the point in world-space at which a ray cast from the eye position intersects with object's surface. It's ridiculously fast because the Z-coordinate is found with the help of the GPU, while the X & Y are found by unprojecting the canvas mouse coordinates.

Ray-pick can therefore be done almost as fast as a normal frame render and scales up to large amounts of detail, with no ray intersection testing against millions of triangles using JavaScript math. Read the blog post or try out the ray pick example.

Picking Changes 

Subgraphs are now wrapped in name nodes which assign geometries to logical pick names. Pick listeners are no longer bound to nodes. Instead a pick operation is made on the scene, which returns a name if a hit occurs. Read more in the wiki page on name nodes or check out the basic picking example.

Simpler Node State Inheritance 

Many scene nodes (such as material, flags and texture) now completely override state defined by parents instead of augmenting it. This simplifies reprocessing of the scene graph after updates, giving a huge speed increase. I'll highlight this more in the wiki.

Permalink | Leave a comment  »

SceneJS 2.0 does a ton of caching to efficiently adapt its abstract JSON-based scene graph API to WebGL. 

I just added another layer of caching, which turned out to give a major performance boost - anyone who's not getting high enough FPS should try the latest build and see if it improves things.

The engine now compiles its internal draw list to a cached list of WebGL function calls, which execute very quickly. The calls are generated by higher-order functions, which cache parameters for the calls in their closures, such as variable locations in programs etc. A classic JavaScript optimisation strategy. 

1. Scene definition - a JSON definition is parsed to create a scene graph with resources buffered for its nodes on the GPU (VBOs, textures etc). 
2. Draw list compilation - the scene graph is traversed to compile a sequence of WebGL state changes.
3. Call list compilation - the draw list is compiled into a fast list of WebGL calls with arguments prepared from the draw list states. Call list nodes are functions that wrap WebGL calls, and are created by higher-order functions which prepare and memoize their arguments in closures.
4. State sorting - the call list nodes are sorted on their corresponding draw list states to minimize the number of state changes that will go down the OpenGL pipeline.
5. The call list is executed to render the frame.

Permalink | Leave a comment  »