Sculpture3D from cosmicollie on Vimeo.

There are many applications out there—some even available for free—that offer endless options for creating high-end digital content. I’m a longtime Maya user, and as I started learning it fifteen years ago, I was blown away when I realized what can be done with it. But some time later, I stumbled upon a program called Zbrush from the company Pixologic. The first time I tried out this software, I instantly knew it was the perfect tool for me. It is no animation package, but is very focused on sculpting and painting directly onto 3D models, and you can do this with an high amount of detail without losing track of brushstrokes. It’s simply amazing. You can get a free trial version from their website and try it for yourself. There is a huge community and a lot of free tutorials,  ranging from beginning to very advanced, that take you by the hand.

As I said, you can do incredibly detailed models and sculptures for video games, movies, concept art, or matte paintings with Zbrush. But as much as I like doing this kind of work, I always missed something: the ability to touch my 3D creations in the real world. If a traditional painter or sculptor creates a piece of art, you can walk around it and really see that it was handmade by a human being. Most people have the belief that if you create works with computers that it’s easy, and computer art doesn’t have the valence of work done “in the real world”. But we demosceners all know that it is not the computer that does the work for you, but the person sitting behind it. I think the problem was the same when people first connected their guitars to electrical power. I don’t want to have to use clay or paint to create my works to get respect as an artist. I want to use digital tools with all the benefits of the digital realm to do the same as a traditional artist does, but I also want the same fame and respect for my work as well. Not an easy task :)

So I stumbled upon a technique called rapid prototyping. It’s nothing really new and is used in the automotive industry, for example. If an engineer designs a blinker enclosure for a car, he actually prints out a plastic version of the 3D data set and places it on the car. If it fits well, the object goes into production. But in the art field I’ve seen very few people using this technique to bring their works to the real world. In the last few years this expensive technique has gotten more and more popular, and above all—affordable. There are many 3D printing services out there that can print out your works for a fair price. For the past four months, I was working on a big project I did on my own. Everyone I told about this was just blown away—there was no one who said “Meh, that’s boring and not real”—they even strengthened my idea that I have to show people what is possible and what you can create if you’re just willing to learn how sculpting on a computer works.

So I started thinking about what to create. I wanted to address a broad audience, but I didn’t want my work reduced to a mass production. I wanted to make something accepted by both younger and older generations of people, and improve my traditional anatomical sculpting skills, but I also wanted to make something which is hip so younger people could say “Hey, that’s cool.”

So I thought about it for a very long time, and I came to the conclusion to fuse the style of traditional sculpture you see in churches with something like street art—like graffiti—and I started to arrange some rough blocks on the digital canvas. I wanted to have the scene set on a huge graffiti background —but as some sort of mountain or rocky massive thing. I decided to create some sort of diorama where the scene takes place, and I wanted to tell a story. I planned to have a loving couple in it, and a threat like the Devil.

So the days went by and the creation evolved. I wanted to get more weight in it because of the Devil’s power, so I added an angel in front of the couple to give them some protection. In many sculptures and paintings you see Death placed in the highest position, but I didn’t want to present him that way. In my sculpture Death is placed at the lowest position—I disarm him. He got old—he’s using a cane. I turned him into a harlequin by adding flowing clothes and a hat with dice on it, but he is still dangerous and holds an ace of spades in his hand as a sign of seduction. The Devil is not the guy who seduces in my scene, but Death. The scene got too imbalanced by having only this small angel protecting the lovers against Death and the Devil. So another two elements went into the composition: a seraph flying high above the scene, and a guard on a horse watching the events from a distance and bringing balance to everything. Modeling everything took a very long time. I had to improve my knowledge of anatomy at the same time.

I took anatomy courses from the artist Ryan Kingslien, who runs Zbrush workshops and founded the Visualarium. I had to rearrange and throw away lots of work in finding the right composition. There were many technical matters to keep in mind, because with the 3D printing technique you have to follow several rules and conventions so the art can actually be printed.

I wanted the Devil to hold a mace with a long chain on it. For the chain, there were many long pieces hanging loose in the air so they could break if it got too long, so I had the idea to align the thin chain in regular gaps like the structure of pieces of fabric or cloth, so that they strengthened and held the delicate structure.

Another obstacle that had to be overcome: sculpting cloth that looks right is really hard to do. So I opened up Blender to do initial dynamic simulations to create what I was after.  That was a really hard and very tedious. A lot of trial and error. The fabric didn’t float the way I wanted it to, but in the end, with a lot of tweaking the elements by hand in Zbrush, I finally made it. The structure of the cloth was created with a plugin called NoiseMaker. The “only” thing you have to do is to have a clean UV layout for the cloth. The object was done after four months of work—I think it looks cool, although I’m aware that I still have to improve my anatomy skills.

When it came to printing the sculpture, another shock was inevitable. I searched for a company to do the printing and found Shapeways in the Netherlands. They are cool, but their pipeline only alows a 1 million poly count. My model had 26 million polys at that time. Oh my. Ok, there’s a plugin which ships with Zbrush called Decimation Master, and
it does a really awesome job reducing the number of polys in the object without losing its details and overall shape. But I had to do it for every single subtool each by each, and there were many of them :)

Ok. I reduced my model to a poly count of about 1 million, sent it to Shapeways, and wanted it to be printed in stainless steel. Yes, you heard right. Stainless steel. To 3D print in metal, a printer places fine layers of particles in cross sections on top of each other and then the whole thing goes into an oven where it is sintered. Just like a motor block.

Ok, I’m an artist, and my budget is in some ways very limited. I found out that if I wanted to print it in steel, I could only afford a print that’s around 14 cm in height. C’mon, I thought—that’s not fair. 1200 euro for a sculpture I worked so hard on, and I only get this size? I went through a very frustrating process to make all this, trust me, but I am a bullhead and wanted to have this sculpture I worked so hard for. Furthermore, Shapeways told me that this model could not be printed 14 cm high because the smallest parts in it would get so tiny that the printer couldn’t do it. So I powered up my calculator and did some arithmetic to find out what it would cost if I sized up the model to fit the specifications. The formula is easy: 1 cubic centimeter of stainless steel costs 20 dollars. If you double the size of it, you multiply by a factor of 8 and you get 160 dollars. The whole model resized so that the smallest elements could be printed would have cost me around 9000 dollars. So I thought about hollowing the object to save material. Days went by in doing this properly. The price didn’t change very much. After all of these calculations, I decided to print it out in a lower priced material. I chose nylon. Two weeks later the box arrived, and I could hold the sculpture in my hands, which were shaking from happiness when I did the unboxing :)

Guys, it is cool to create something like this on the computer —but seeing it as a physical object in the real world is breathtaking. Even the photos I took for this article can’t nearly show how amazing it looks in reality. I also did a lot of renderings from the scene and edited them in Photoshop. I created a photo book from all the prints, and I also wanted this book to be something special. There are so many providers out there offering photo book printing, but I wanted something really classy. So I found a company called Cinebook. They make high-end photo books which are a bit pricey, but the quality you get justifies the price. I’ve never seen any print that looks as sharp and crisp as what the guys from this company can produce.

I still wanted to see and feel how a metal version would look. So I designed an two finger ring in tiki style and printed it in stainless steel. Guys, this is so amazing. The details are all there and I can wear it and have it with me to show everybody what I’m doing. I wanted to take photos of someone wearing this ring, so a good friend of mine put it on and even cleaned his fingernails for the photoshoot, but I told him: “erm, this doesn’t look right— that’s not the look I’m after”. So I went to the local Harley Davidson guy and asked him if he could wear it for a picture.

At first he was a little bit annoyed, but the grumpy look on his face vanished and we made some cool photos. Thank you Mammut!—that´s his name—for your help. He also was fascinated by the technique and can imagine ordering steel skulls with flames for his next custom bike. :)

I want to show people my work and I’m hoping for a great audience, so I also did a free ebook available for the Ipad. I did lots of editing and added lots of movies and keynotes to it. At this point I want to say thank you to Marc Ewald from the Netherlands. He’s a composer and he created the music for the intro movie of the sculpture. Please download your free copy here:

I’m aware of the fact that there are lots of Android users out there, and people using other ebook formats. Exporting the book to PDF is technically possible, but it would be like printing out a movie on paper. I’m sorry for that, but there won’t be a format supporting other devices. If you want to take a look at the renderings you can do this without the book at my website under the gallery tab, or you can just look at my Facebook page to keep up to date with what I am working on.

See, I want to do art—I want to create things with value. And I really love what I’m doing. I don’t want to do stuff for mass production, but things that individual people like. Just like they do when they go to the tattoo artist they trust to get THE special kind of artwork they want. I can create belt buckles, busts of your children, emblems, pendants, sculptures, rings, earrings, canes, custom figures for tabletop role playing games, and so on and so forth.

The possibilities of this technique combined with good sculpting skills are endless. This is a brand new market and in the future everyone will have a 3D printer at home. Remember paying lots of money for a simple CD burner? Now you get them for almost nothing. And as easy and comfortable as the modelling tools might be, the skills you need to do really great work—sculpting, knowledge of human anatomy, style, and artistic ability—will still be in great demand because it is not the machine doing the work, but the person behind it.

Ödönke bánt.

In hopes that most people enjoy makingof’s as much as I do, the following series of articles would be an attempt to describe the creation process of my Revision demo called “Machine” by Ümlaüt Design, both from a technical and artistic standpoint, and perhaps shed some light on a few easily avoidable mistakes I’ve made along the way.

A disclaimer: I’m not aiming this to be a tutorial or a white paper; if you’re a competent demomaker, you will probably not find anything novel in these texts. This is more aimed at people wanting to get into demomaking, but are perhaps intimidated by the chore, or maybe made a few simple demos, but are unsure how to direct them better or approach the production process easier. Note that I’m not a very good visual effect coder, so don’t expect a lot of linear algebra. I’ll also try to keep a lot of the actual maths / technical stuff to the minimum, or at least digestible to a non-technical / artist audience.

Preparations

I first had the idea to do a demo for Revision around February. Reasons were simple: I haven’t done a demo for several years at that point (meaning actually worked on one instead of just supplying music), and I felt I needed to do something to get back into the loop, and to stay sane. Another motivation was that I found out about ASSIMP, a fantastic little loader-library for 3D formats; I would often discard ideas for a demotool simply because I felt the management of a 3D content pipeline was a right pain: I tried stuff like Flexporter or writing my own plugin, but it just never worked out quite right, and I also realized that to be really effective (and perhaps somewhere down the line, artist-friendly), I would need to use standard formats like COLLADA. So often I shelved ideas because of this, but then ASSIMP came along and I was in a sense reinvigorated, knowing that there’s a whole open source project who’ll gladly deal with these kind of problems.

An early concept model of the first robot.

My plan was simple: I had a track from around 2007 influenced heavily by Drew Cope’s Drum Machine animation, and I realized I never got around to making anything with it, so finishing the track already put me in a good position. I also had some really rudimentary basecode from around that time for the next generation of ÜD tools we wanted to make – this certainly spared me some time to mess around with datafiles and parsers and loaders. Of course this was by no means finished, but it was a good start.

So I started planning:

• I had roughly two months. First, I really don’t like party coding. (Why code in Germany when I can code at home any time I want? It’s a waste of a good time.) Second, my current portable computer is a netbook; a fantastic little machine for everything other than Shader Model 3.0 demos. These all meant that I had to finish at least a day before the party.
• Already had the soundtrack (or at least something that needed a day of work at most to finish), and it wasn’t particularly long either.
• Some basecode, mostly just boilerplate STL-like stuff.
• One person project – you’d think after 12 years in the scene I would know some willing graphics artists, but the problem here is that they were either uninterested or unavailable at the time, and my concept wasn’t fully fledged either, as I was anticipating to make some of the stuff up as I went along. (Plus there’s something to be said about masochism, megalomania and lack of human skills, but that’s a different story – let’s not dwell on that.)

Demomaking. Serious business.

I also planned what I’ll be needing:

• I’ll need a tool. You can argue about handcoding, but no. For the stuff I planned, razor-sharp editing and continuous flow, you’ll need a tool. And no, GNU Rocket won’t cut it – it just wasn’t precise enough or convenient enough for what I wanted. I’m a firm believer in that if you spend a week on writing a tool that saves you a week somewhere down the line, do it.
• I’ll need a 3D engine that can somehow load an animated 3D mesh and display it. I’ve written a few of these so I wasn’t concerned, although I really wanted it to be non-hacky this time, i.e. unified lighting, material system, and hopefully looking the same in the tool as in the modeller.
• I’ll need a couple of effects. Nothing special – I’m not good at them, so I’ll just keep them basic but hopefully interesting.
• A LOT of content. I was originally planning about 3-4 scenes, but listening to the music, I had to realize I needed at least double that.

I made a rough timeline for the project: about two weeks for the tool, two weeks for the engine + loader + effects, two weeks for content, and I allocated the last two weeks to edit the demo together. This last point might sound like an overkill, but I really wanted the demo to look polished and feel well thought-out, and I was also aware that making a demo, being a non-linear process, sometimes involves you going back to previous steps and fixing content or code in the phase when you’re supposed to edit (in film, I suppose this is what you’d call “re-shooting”), so in a sense I left myself some headroom to fail.

The tool

2012.02.11 – We all start somewhere.

I decided early on what I wanted my tool NOT to be: I didn’t want to write a content tool. I had no room (and in this case, no time) to make a full-fledged 3D editor, even to just place cameras or tweak materials. I decided that I’m better off to do those in the content package and then just make sure it looks as close (within reason) as possible. All I really wanted is to manage resources (to an extent), composite, but most of all, edit. That was it.

I also decided that since I don’t have much time, I needed to tone down on functionality: the basic idea of a tool is to make certain things easier, case in point editing and scratching back and forth in the demo. That’s all I wanted to have. The underlying system was capable of managing resources, loading them, reloading them, etc., but since my internal format was XML, I decided that as long as it doesn’t get too tedious, everything else I could just do by hand. Now, it’s up to each and every demomaker to decide what’s tedious and what’s not – for me, copy-pasting XML tags to insert a new envelope was a perfectly acceptable trade, even if I had to do it several times; compared to adding GUI to a “New Effect” window with foolproofing and all, it was really just a minor nuisance. Of course, I’ll finish the GUI properly at some point, sure, but then and there I had two months, of which I allocated two weeks for the tool. From scratch. I had to create a triage.

As far as the tool’s interface goes, I went back to what I’ve been using a lot: music editors, specifically DAW‘s. DAW’s have a particular way of arranging envelopes, clips, loops, and they allow a really precise level of editing, but the thing that struck me the most about them is how much ease they have developed over the years. I used to be a tracker person, and my switch to DAWs was somewhat bumpy, but now I don’t regret it – adaptive grids, triplet beats, beat snapping, envelopes, these are all things that one can use for a demotool and gain a lot of strength with them. With that in mind, I set out to essentially build a DAW by my own rules.

The underlying system was simple:

• I can load an arbitrary number of plugins.
• These plugins can produce an arbitrary number of resource loaders and effects. These are object factories that can instance any number of effects or resources.
• I have a common resource pool where I manage the outputs of the loaders: Textures, meshes, videos, shaders. If there’s a loader for it, it goes here. Resource loaders can also use this pool if they depend on other resources (e.g. 3D scenes can reach for textures already loaded)
• Each effect can specify a set of resources (by type) to use. None of them HAVE to be there – it’s a plugin’s responsibility to fall back gracefully.
• Each effect can specify a set of parameters to be wired out to envelopes or left as a static value.
• The demo then loads plugins, instantiates the loaders and effects, loads the resources, feeds them to the effects, loads the envelopes, and runs.

That’s it. That’s all I needed at the moment.

2012.02.16 – GUI shaping up slowly. Note the time signature change.

Early on one of the things I thought about is having arithmetic envelopes, i.e. I would be able to have two splines create an envelope by adding, multiplying, and so on (e.g. a pulsating beat that gets progressively stronger), but I ditched the idea (AFTER I wrote the code for it) because it was a hassle to manage, and I had a better idea: if it’s all just envelopes creating floating point values, why not subclass them? So right now I have one subclass for static values, one for a standard spline-based envelope, and then if I need specific GUI functionality later, I can just implement a new type. This all might sound like an overkill (I haven’t actually implemented new ones yet), and certainly arguing about GUI design can be pretty pointless, but I thought about how I would imagine an envelope to e.g. switch between cameras conveniently: with an ordinary spline-based system, you can set your splines to flat and just move your points up and down until it crosses to another integer value, but it sounds like a nightmare from a convenience point-of-view (and due to the time constraint, this is what I ended up using, so I can confirm it too). A better idea, of course, would be displaying the GUI itself as a list of ranges (i.e. “between 2:00 and 3:00, the value is constantly 7″), and not just a spline – like the pattern editor for the GlitchVST. While you could argue that this way you’re actually spending time on removing functionality, my perspective is that for certain parameters (colors are a good example too), a specific GUI feels much better, and with a subclassed GUI, that’s fairly easy to do without having to break something else. Once again, this might sound like a tangent, but the idea is this: As long as it spits out a floating point number, anything goes, and a bit of forward thinking pays off later.

One thing I kinda forgot about initially and then just sorta chose the simplest solution to take care of it was the actual rendering range, i.e. if there are all these possible envelope types, how does the demo know if it’s time to render an effect or if it’s time to leave it alone. This I decided to leave up to the discretion of the actual envelope class: each class had a bool function that returned whether according to that envelope, something should be happening or not, and when all envelopes said “true”, I rendered the effect. Constant values always returned true, and splines returned true if the play cursor was over one of their clips (= the white boxes visible on the screenshot below). This provided fairly convenient control over when effects were active or inactive, and again, with a different envelope type, I would’ve been able to define a whole new behaviour if I wanted to.

Another thing before I forget: For splines, I didn’t limit myself to a 0.0-1.0 range. When I did my first 4k synth, I realized a lot of the fun stuff comes from the out of range values, and if you want to extend your animation with 20 more frames, you don’t have to re-sync your entire demo. (This, by the way, is a hindsight. I did make this mistake.) Of course the question is, how to fit an infinite range to a GUI like this? Well, I went for the simple and stupid solution – default range is 0.0-1.0, a rightclick menu of “Set Range” can change that to anything. This is purely a GUI thing and the demo doesn’t even have to care about this.

As far as GUI goes, I went with MFC for a number of reasons: I’m really familiar with it, having worked with it for a number of years, I find it not only reliable, but also very convenient, however the REAL real real reason was docking panes. I fell in love with the Visual Studio-style docking panes on first sight and I wanted them in my tool for ages, not only because they’re really easy to rearrange, but I also wanted to have a level of multi-monitor support where I can see the demo on one monitor and edit the envelopes in the other – just like in a convenient DAW. (That hissing sound you just heard was the Ableton camp. Heh heh.) So for that I needed my docking panes to be able to leave the main window and maximize on the other screen.

2012.02.19 – The basic spline editor.

Another reason I enjoyed MFC was the library perks of the Document/View model – anything that keeps me from having to manage stuff like “Do you want to save the changes?” and the clipboard and keyboard shortcuts and loading bars and the whole New Document business is a bliss. To peek ahead a few weeks, somewhere down the line I occasionally found myself working with multiple UDS project files to cut down loading time and then just eventually manually merging them, all thanks to the fact that I was able to do a “New Document” trick fairly fast. (And yes, some of this sounds like an epiphany of a 3 year old finally figuring out how a toilet works, but hey. As said, it’s been years.)

The first iterations of the tool were obviously puritanical, but I wanted the idea to be clear from the word go – the View has a rendering window in the middle, rendering in the exact same resolution the demo is running in (optimally – resolutions can vary but there’s always an intended one, in this case 720p), and everything else is in panes, toolbars, menus, keyboard shortcuts, etc. One positive thing about MFC (and WINAPI in general, I guess) is that it encourages redundancy in GUI: if you have a menu item, you can immediately add it to the toolbar, assign an accelerator, and so on. It’s free, and it makes the GUI adapt the user instead of the other way around.

Now, this isn’t strictly code-related, but it’s worth mentioning that I noticed an interesting trick a few days into the development: I worked better, when I left basic stuff unfinished at the end of the day. My reasoning for this was that if I’ve done a 100% job that day, the next day I’d sit down in front of Visual Studio, and spend time wondering how to jump into the next task; I would start to procrastinate. However, when leaving really run-of-the-mill stuff unfinished, I would often spend the next day thinking about how I will solve a particular solution, and by the time I got home, I just jumped into the project and fixed the problem, and essentially carried on working without losing momentum. This worked best when the problem at hand was as simple as it gets: reproducible crash bugs, broken colors, textures being upside down, stuff you’re able to debug and fix in your sleep. But because they’re so easy, you don’t feel as if there’s a mountain to climb for the next task; by the time you fixed it, you’re “warmed up” and ready for the next big chunk. I highly recommend it if you feel that you’re able to lose motivation in democoding.

After I reached a satisfying stage with the basic envelope GUI, I switched to the next step: the underlying engine.

The engine

I had a pretty good idea from the get-go of what I wanted my engine to achieve, both as far as timing and compositing goes and as far as rendering goes:

For each “effect”:

1. Load some sort of standard mesh format
2. Use incoming splines to animate camera, light and meshes
3. Render in multiple passes, perform compositing

2012.02.21 – FK in action, rendering normals.

Step one used to be a huge pain; my first demos used to use 3DS files, but they’re a bit outdated by now, and they’ve always been a bit of a nuisance to handle. Later I made my own file format using Flexporter, a step above 3DS files but still a fairly dodgy solution: you’re using middleware to create even more middleware – things just go wrong. Finally, I attempted to write my own exporter, and that just ended up being torturous, because of the intricacies of the tools’ inner workings. At around 2007, as a result of this, I spectacularly gave up trying to create a content-pipeline, and with it, making demos.

Revelation came in the form of a set of Pouët posts talking about a library called ASSIMP, a 3D mesh loader library that takes dozens of known 3D formats and loads them into a unified data structure. Not only that, but it’s also able to do a bunch of really cool transformations on it, and it comes with a set of fun little tools to test your meshes as well. I took a look at the API, and I was sold immediately. I did some tests, and it worked so well that I decided to start integrating it into the engine with newfound enthusiasm.

Of course, it wasn’t all fun and games – the 3DS loader in ASSIMP is still kinda broken, but the COLLADA importer works. The COLLADA exporter for MAX, however, is rather lacking. The solution in this case was OpenCOLLADA, a unified exporter for MAX and Maya, which is slightly improved. A weird thing I noticed is that I had to flip my cameras around to get them to face the right way – this might be a bug in my code, but unfortunately ASSIMP’s reference viewer doesn’t use cameras, so I have no way to check. In any case, with the loader slowly creeping in the engine, and step 2 already mentioned above (splines and envelopes – I’m jumping a bit in chronology here), I was ready to tackle step 3.

Over the years I’ve done some intermediate rendering stuff, but I never really built a comprehensive engine where everything was combined with everything, and this time I really wanted to do something that “just works”. My wishlist was simple:

• Forward kinematics animation. This is easy on paper, comes from step 1, but of course the whole row-major/column-major/nodegraph/LHS/RHS matrix quagmire can take up a bit of time to sort out.
• “Correct” lighting, i.e. lighting done in the tool, and not the usual “it looks kinda okay if I change this to 0.5″ debacle.
• Normal maps. I’m a huge sucker for bump maps, and I’m sure I’ll die happy if they’re the last thing I see.
• Motion blur. Ever since the Realtime Generation invitation I’ve been infatuated with realtime motion blur, because I realized what a nice subtle film-like quality it adds to everything.
• Shadowing. Again, not a big thing, but it does make a considerable difference.

2012.02.24. – Shadow maps up and running.

This all doesn’t sound like a big thing, but I wasn’t sure how to manage all of this, as far as rendering order, render targets, etc go. The first “a-ha!” moment came when I started looking into D3DXEffect. Granted, after about 7-8 years of it being released, perhaps it was bloody time, but I guess I was always wary of particularly high level stuff. This time, it came like a breath of fresh air – all the tedious pass management is in there and more. The only problem was render targets.

Luckily, I came across a little note in the SDK about annotations. Annotations are essentially meta-data that allows the coder to assign variables and values to passes, and then read them back once the pass is running. This gave me the freedom I needed; I built my little system like so:

• During initialization, the effect plugin builds the required render targets and depth buffers, and stores them in a dictionary, designated by some sort of identifier e.g. “shadowmap” or “velocitybuffer”. (Later I built a facility where the effects would be able to request render targets from the engine core, and the core would either allocate a new one or hand out one of the older ones, depending on the parameters. This cut down on the video memory usage considerably.)
• The .fx file is built in a way that all the passes designate which buffer they want to render to. The actual rendering code then loops through the passes, sets up the render target, and pushes out all the draw calls. Also, since I designed .fx files to be handled as resources, I was able to switch between them easily during editing, changing one rendering chain to another.
• Because I wanted to do some compositing / VFX / etc., I extended this by adding another flag to specify whether the current pass is a “scene”-pass or a “post-processing”-pass. In the latter case, instead of pushing out the scene’s 3D geometry, the engine would simply render a 2D quad, and I’d be able to use the results of the previous passes as input.

Down the line, I also realized that if I wanted to integrate “hard-code” effects into the chain, I could just insert a pass where I render the geometry, and specify an annotation that e.g. I want to render the color pass into a certain buffer, but I want to use the depth values of the previous render. That way, my “effect” would be correctly occluded by the geometry around it.

To this end, I even introduced flags to specify that during certain passes, while the setup of the matrices would remain the same, the effect would “skip” rendering the 3D scene-graph and instead would render the custom effect code. Additionally, I subclassed my renderer, and made sure that all the “custom” code would remain in a separate class – this wasn’t necessary, but it made sure that my actual engine code remained lean and effective, and more importantly, reusable.

An example “technique” for one of the custom passes would end up looking something like this:

technique StandardRender
{
  pass passShadow<string renderTarget="shadow";>
  {
    AlphaBlendEnable = false;
    vertexShader = compile vs_2_0 vsDepthToColor();
    pixelShader  = compile ps_2_0 psDepthToColor();
  }
  pass passBlobObjectsShadow<bool blobs=true; string clear=""; string renderTarget="shadow";>
  {
    vertexShader = compile vs_2_0 vsDepthToColor();
    pixelShader  = compile ps_2_0 psDepthToColor();
  }
  pass passVelocity<string renderTarget="velocitybuffer";>
  {
    vertexShader = compile vs_2_0 vsVelocity();
    pixelShader  = compile ps_2_0 psVelocity();
  }
  pass passBlobObjectsVelocity<bool blobs=true; string clear=""; string renderTarget="velocitybuffer";>
  {
    vertexShader = compile vs_2_0 vsVelocity();
    pixelShader  = compile ps_2_0 psVelocity();
  }
  pass passRender<string renderTarget="colorbuffer";>
  {
    vertexShader = compile vs_3_0 vsRender();
    pixelShader  = compile ps_3_0 psRender();
  }
  pass passBlobObjects<bool blobs=true; string clear="color"; string renderTarget="blobbuffer"; string depthTarget="colorbuffer";>
  {
    vertexShader = compile vs_3_0 vsRenderBlobObjects();
    pixelShader  = compile ps_3_0 psRenderBlobObjects();
  }
  pass passBlur<bool postProcess=true; string renderTarget="blurbuffer";>
  {
    vertexShader = compile vs_3_0 vsPostProcess();
    pixelShader  = compile ps_3_0 psBlur( 2.0, samBlobmap );
  }
  pass passBlur2<bool postProcess=true; string renderTarget="blur2buffer";>
  {
    vertexShader = compile vs_3_0 vsPostProcess();
    pixelShader  = compile ps_3_0 psBlur( 4.0, samBlurmap );
  }
  pass passBlur3<bool postProcess=true; string renderTarget="blurbuffer";>
  {
    vertexShader = compile vs_3_0 vsPostProcess();
    pixelShader  = compile ps_3_0 psBlur( 8.0, samBlur2map );
  }
  pass passComposite<bool postProcess=true; string renderTarget="backbuffer";>
  {
    vertexShader = compile vs_2_0 vsPostProcess();
    pixelShader  = compile ps_2_0 psComposite();
  }
}

Nice and clean, and more importantly: doesn’t require touching the source.

2012.02.26. – Motion blur and lighting.

As far as the actual shading/effects go, I chose simple methods: I restricted myself to a single spotlight source (which worked well with the concept I had in mind), so I was able to do rudimentary PCF shadow mapping, which is jaggy as hell, but for some reason it worked really well with the gritty textures I was making. For normal maps, I used simple hand-painted stuff, and ran them through NVIDIA’s Texture Tools. For the motion blur, I used basic velocity buffering, which I achieved by doubling all my matrices: before I render anything, I pre-transform all my node matrices according to my animation values. (Since the cameras and lights are all just nodes in the scene-graph, these also transform with it for free and come with all sorts of fun perks – but more on this later.) Then I do another transformation calculation, but with my epsilon value subtracted from my animation timer. This will give me two matrices for each draw call, one being slightly behind in time compared to the other. Then for the velocity pass, I simply calculate their screen-space position and calculate their difference vector, which I store into an D3DFMT_G32R32F buffer, and then use that later as a directional blur input. This method is obviously not perfect, but my assumption was that motion blur is only visible when there’s fast motion, so while it doesn’t necessarily look good on a screenshot, in motion it does give that subtle film-like feel to the demo which I was looking for.

So I had my basic engine and tool-chain ready. It was time to sit down and come up with what I considered the hard part: content.

[to be continued, in theaters near you]

This extremely cool action seemed to trigger what can almost be described as “an avalanche of sources” because other demo groups and individuals started following their lead. Since this happened in various places online and in different shapes and forms, we decided that the best thing to do would be to simply collect all of them in a single post.

The source code packs

Below you can find links to the different source code packs as released by the various groups, plus links to example demos from each group. Enjoy!

Farbrausch

Known for both 64k intros as well as demos, this German group has released the sources for their demo tool and many of their productions, including the now famous Kkrieger — the 96k 1st person shooter game.

Included productions: fr-013: flybye, fr-022: ein.schlag, fr-029: dopplerdefekt, fr-024: welcome to…, fr-030: candytron, fr-038: theta, fr-043: debris and kkrieger (beta)
Download: Github

Conspiracy

Hungarian demo group known for their award and competition winning 64k intros. Included are the sources for two versions of their tool — aDDict and aDDict 2 (Advanced Digital Dynamite Intro Creation Tool).

Included productions: Project Genesis, A Place Called Universe and Chaos Theory.
Download: ZIP-archive

Excess

This Norwegian group is known for making demos, and have released the sources for their DX9 graphics engine as well as all demos made using that engine, including their sync tool called GNU Rocket, used to direct them.

Included productions: Scyphozoa, Regus Ademordna, Easter Guardians, Cortex Sous-Vide, GNU Rocket (tool) and more demos made with the same engine.
Download: Github

Moppi

Memon of Moppi, the finnish demogroup known for very stylish and trip-hop-ish demos, released the sources for their tool Demopaja. The tool was used in such legendary demos as Gerbera and the Assembly 2004 invitation. Unfortunately, the ZIP-archive does not include the datafiles for these demos, so you won’t be able to poke around with those, but the tool itself is very nice.

Included productions: None, but ships with sample effects.
Download: Mikko’s blog

Andromeda Software Development

Navis of ASD also contributed to the source code giveaway. Now, this piece is perhaps less of a sourcecode giveaway and more of a piece of C code as directed by David Cronenberg. Either way you look at it, it is an interesting peek into the programming mind behind some of the most memorable demos of the last decade.

Included productions: Spin (well, sort of..)
Download: part_start_lines.cpp

Anti-virus false positives

Due to the nature of how demoscene productions often push the limits in terms of compression and algorithmic content, they tend to be flagged by anti-virus software (like Microsoft Security Essentials), but this is simply a false positive due to lack of whitelisting. All of the source code and executables linked to above are absolutely clean, we assure you.

As the idea evolved in my mind and in conversations with friends, I decided to build a simple app to test out the concept and write about it as it progressed.

Before I dive in, give the current version a shot. You can access the bleeding edge at demoseen.com/genshaders and the current version at demoseen.com/genshaders-1 – I wanted to keep these separate so that you could see where I was at each point.

First Steps

My first thought for an interface was a simple 4×3 grid of WebGL canvases. Why WebGL? It’s easy, fast, and shaders can be used to generate the textures. The design was largely inspired by the Chrome new tab page — hey, it works!

From there, I had to actually start generating shaders. I decided that the best place to start was doing it purely randomly, so I created a shader builder based on a few simple practices:

• The code of the shader was gl_FragColor = vec4($genValue$, $genValue$, $genValue$, 1.0); Meaning each value going into the output color was generated separately.
• The genValue function — during shader generation — would randomly return one of the following:
  • An existing value (initially either the X or Y coord for the current pixel)
  • A random float
  • Two other genValue’d expressions combined with a random arithmetic operation
  • A genValue’d expression passed through a function such as sin, cos, tan, sqrt, etc

Based on that simple practice, I got some really interesting results:

But around this time, I realized that my initial concept of this being something that would allow you to drill down to get the exact texture you want, as a standalone tool, was simply not the best way to use this.

Evolution #1: Time

I mentioned earlier that genValue could pass back an existing value. Well, the first step I took away from generating static textures was to throw time (as floating point seconds) into the mix. Immediately, shaders that looked completely uninteresting before started looking amazing. Everywhere you looked, there was action. You’d get everything from old-school plasma effects to amazing shifting colors and rays of light.

Once I saw the effects of this, I decided to expand it. Instead of just floats and unary functions, I created vector creation methods and expanded the function catalog to include binary float functions (e.g. mod, atan, pow). But as the complexity increased, so did the odds of it going off the deep end; it’d often get into a loop where it was calling genValue/genVec* hundreds of times, recursively and causing stack overflows, or creating shaders so large that they simply wouldn’t compile. To get around this, I added a level value to each generator function, so that it could keep track of how deep it was in the hierarchy at any point.

One nice thing about having the level value around was that I could point the random generator in the right direction; if I was more than n levels deep, I wouldn’t generate vectors, for instance, because I knew it was highly likely that they’d end up tanking the rest of the code. I spent a while just tweaking the values here to get to a place where things compiled quickly and still looked good.

I tried to get a screenshot of this step that really captures what this looks and feels like, but without the animation it really just can’t be understood. I strongly recommend you take a look at the live demo at this point, if you haven’t already.

Next Step

The next few changes are going to be big ones. Rather than this just being a standalone bit of JS, it’s going to be attached to a server backend that will keep track of the genome. Everyone going to the site will be working on the same gene pool, so we end up acting as a collective fitness function.

In terms of the interface, it’s going to be considerably nicer. Rather than the big 4×3 grid, you’ll be presented with two shaders at a time and asked to choose the one you prefer; think Hot or Not for shaders.

To Be Continued

Well, that’s about it for where I’m at now. Things are still just random, but I’m loving what I’m getting out of it now — I can only imagine the insane things I’m going to get once everyone is unleashed on it. I plan on putting out these mini-articles as I work on this, so whether it works or not, you’ll get to see what I do and hopefully learn something along the way.

Welcome back! In this part we will start sculpting a humanoid character using ZSpheres. ZSpheres is a very useful (and powerful) tool in that it offers you a super-fast and highly flexible way of getting starting the sculpting process (often the hardest part). This means that you don’t have to create a base mesh in a 3D-modeling package first, you can just hit the ground running. So, let’s get started!

ZSpheres

Open the ZBrush application and create a new document. Hit the SimpleBrush icon in the Tool menu, and then the ZSphere icon as shown in the picture below.

Click and drag with LMB to draw the ZSphere on the canvas. Your screen should look like the picture below. Hit T to enter edit mode. Forget hitting T now and you will draw a new sphere every time you click on the canvas. If your accidentally draw new spheres, hit CTRL+N to clear the canvas, LMB-click-and-drag a new sphere, hit T, and you are OK.

The red ball you have no drawn on the canvas is your root ZSphere, to which you now can add new ZSpheres (in a tree-like structure that branches out from the root). All you have to do is add new spheres, and adjust the position, scale, and rotation of each while looking at your shape from different angles. Your goal is to create a structure in this manner that resembles a basic foundation for the shape you have in mind, which you will use for starting the sculpting process.

Here are the actions you need to create a base mesh using ZSpheres:

• Add new sphere: Hit Q to draw sphere. LMB-click-and-drag on a location on the sphere where you want to add a new one. Hold SHIFT if you want the new sphere to have the same size as its parent.
• Delete sphere: Hit ALT+LMB.
• Move sphere: Hit W to move sphere.
• Scale sphere: Hit E to resize.
• Rotate sphere: Hit R to rotate.
• Enable symmetry: Hit X.
• Preview: Hit A to preview the result.

So, as I said the the beginning, we’re going to sculpt a humanoid character. The way to do that is to use ZSpheres to create a shape like the one on the left in the picture below, and then proceed to make ZBrush convert the ZSphere object into a polygon mesh for sculpting (shown to the right).

The process

To create a base mesh using ZSpheres, you can use the following steps and repeat until you are satisfied with how your preview looks:

1. Add a new sphere on an existing one (or create a root sphere).
2. Move, scale, or rotate this new sphere while looking at your mesh from all views. (Enable symmetry if needed.)

The picture below demonstrates the process used for creating a humanoid shape. (Remember to look at the object from all sides and not only the front view!)

OK, so you have followed the process through several iterations and are satisfied with the preview. Time to convert to polygons!
Navigate to Adaptive skin in the Tool menu and hit Make adaptive skin. The polygon mesh will now be added to your tool menu, named Skin_… Select it and start sculpting! :)

That concludes this part. Feel free to add comments or pictures of your works in the forums. Have fun sculpting! :)

Hi and welcome to this series on sculpting in Pixologic ZBrush. In this series, we will investigate some of the game-changing techniques that have made ZBrush so popular among digital artists.

First, a quick primer on what ZBrush is. ZBrush is a digital sculpting and painting application (Autodesk Mudbox is another software package that offers similar capabilities). 3D sculpting means that you work on “digital clay”. You start out with a clay-like basic shape, like a sphere, and then you use a set of brushes to “sculpt” that clay. What makes ZBrush so powerful is that you can work on millions of polygons. In short, 3D sculpting is a very efficient and fun way to create high-polygon 3D models and/or normal-maps.

OK, enough with the sales pitch: let’s get to it!

There are basically two ways to start sculpting in ZBrush. One: make a simple “base mesh” in an external 3D application (such as Maya). Two: start with a simple shape or ZSpheres. ZSpheres are very cool, and we will save them for a later tutorial. In this part of the series we will start by importing a base mesh from Maya. You can refer to this tutorial if you need a refresher on Maya.

Creating a base mesh

There are basically three criteria for the base mesh:

1. Keep it simple
2. Shape it into a very abstract representation of your design
3. Make sure the mesh is clean and quads only

So, you may either have some concept drawings or a mental image as reference for your design. Then, block out a simple mesh in Maya that resembles that design, using only a handful of polygons. Now, this is very important: keep it clean and all quads. You will face quirky problems in ZBrush if your mesh does not satisfy this condition. All quads means that each polygon should have four edges (this has to do with the subdivison algorithms in ZBrush). Also: the mesh should not contain any holes, gaps, overlapping vertices or polygons with flipped normals. A good reference might be the section on cleaning up the mesh in my Maya tutorial series.

In the image below, I have created a base mesh for “Headmistress Daggerhorn”, which is one of the key characters featured in the game Evil Junior which I’m currently working on. It was created on top of great concept art from Atle Mæland as the reference.

Create your own base mesh, and then export your base mesh as .OBJ in Maya. Brace for impact in ZBrush!

Intro to ZBrush

Time to open ZBrush. Hit “Import” in the tool menu and browse to your .OBJ base-mesh. “Draw” your base mesh onto the canvas by click-dragging in the viewport. Then hit “T” to edit the object.

Now, a brief heads-up on the ZBrush interface (see the picture below). To your right you will find the “Tool” and “Geometry” menu. Current brushes and materials are located to the left. Here is what you need to know:

• Tool: Here you “Import” your base mesh. You save your work as ZTools by hitting “Save As”. Saved ZTools are loaded with – surprise! – “Load Tool”.
• SubTool: You can also import several sub tools and append them to your main tool, so that you can more easily hide and select individual objects on screen.
• Geometry: ZBrush “subdivides” your mesh when you hit “Divide”. Each division operation adds resolution to your mesh, measured as the number of “active points” in the HUD of the viewport. You can easily jump between subdivision levels by hitting “Lower Res” and “Higher Res”.
• Brush and material: Here you can edit your current settings for brushes and materials.

ZBrush both looks and feels like something out of this world. (You will probably either love or hate the interface.) I would highly recommend you to use a pressure-sensitive pen and tablet for sculpting in ZBrush. Here is how to navigate:

• Sculpt: LMB onto the mesh.
• Smooth: SHIFT+LMB onto the mesh.
• Quick-menu: RMB.
• Move: ALT+LMB in the background.
• Rotate: LMB in the background.
• Zoom: ALT+LMB in the background, then release ALT and hold LMB.
• Snap rotation into orthogonal view: Rotate and then hold SHIFT.

Here are a few hints to kickstart the sculpting process:

• Symmetry. Hit “X” to enable automatic mirroring.
• LazyMouse: Hit Stroke -> LazyMouse to enable computer-aided stability to your strokes. Hands like a surgeon!
• Brushes: Some brushes you may find useful are the Standard, Smooth, Move, mPolish, and Slash brushes.
• Strokes and alphas: Try experimenting with these to customize your brushes. You can add high-res details by painting an alpha in Photoshop and then apply it to your mesh using the Drag Rect stroke.
• Material: Try using other materials other than the very “ZBrushish” MatCap Red. All the MatCap materials are good. Switch between them to apply different effects to your mesh.
  
• Iterate: This is perhaps the most important thing in ZBrush. Only subdivide your mesh when you have maxed-out the current level of resolution. Iterate and add more detail for each subdivision level.
• Reference: The use of reference (anatomy guides, bodybuilding magazines, images on the web) can make all the difference. Study the anatomy of bones and muscle if you find the time. Be aware of nudity though.
• Masking: Hit CTRL+LMB or ALT+LMB onto the mesh to paint masks. Very powerful for adding details. You have more options under the “Masking” menu.

Basic sculpting in ZBrush

The sculpting process in ZBrush is actually very simple. It all boils down to iteration. In the beginning of the sculpting process, try using the Move brush to drag the mesh into the shape you want. You can also ALT+LMB with this brush to push or pull. When you are satisfied with the overall shape, use the Standard brush with LMB and ALT+LMB to paint detail at the surface. Use the Smooth brush in between to make the surface nice and clean. (You may want to dial down the brush modifier on the Smooth brush to make it less sensitive.) When you are happy with your sculpting at this level, subdivide and repeat the sculpting process. You can keep on subdividing like this till you drop – or more probably – you run out of RAM. Two million polygons should suffice – any more than that and your RAM will probably choke. As I gradually add more and more resolution for each iteration, I tend to use the brushes in the following manner:

• Move: I use this mostly at low resolutions to pull and push the clay into shape.
• Standard: This brush I tend to use at all resolutions, gradually decreasing the brush size for each subdivision.
• Smooth: I use this ALL the time.
• mPolish: Great to use at higher resolutions to flatten out a surface. Especially if you aim for a cartoony look.
• Slash: At very high resolutions you can use this brush to add scars and such.
• Pinch: Very nice for pulling two edges towards each other, for example if you want to “tighten” a scar.

In the picture below you can see how Daggerhorn takes shape from a simple base mesh to a high-ploygon bust of the Headmistress of Gloomcraft Academy.

That’s it

Piece of cake. Now, post an image of your sculpt in the gallery and stay tuned for the next installment! See you around :)

Oh, and one last thing: Here is an all-exclusive and yet unreleased work-in-progress of Headmistress Daggerhorn from Evil Junior!

Getting started

So you have a demo in JS. It’s pretty. It’s perfect. It’s 3k in a 1k competition. Well, damn.

You start with the obvious and run it through a minifier and you shorten all your variable names to a single character, you get it to 2500 bytes. Great, that’s progress, but you’ve still got 1476 bytes to go. You merge some functions together, fold a couple loops into each other, and soon you’re at 2200 bytes. Long way to go.

Rule #1 of shrinking demos: removing a byte becomes more difficult every time you remove a byte. It starts off trivial and runs very quickly into a brick wall.

Quick wins

These are a couple useful techniques to trim the fat at this point.

var a = g.getAttribLocation(x, 'pos');
g.uniform1f(g.getUniformLocation(x, 'time'), t);
g.vertexAttribPointer(a, 2, g.FLOAT, 0, 0, 0);
g.enableVertexAttribArray(a);

What’s wrong with this picture? Well, if we ignore whitespace (minification takes care of that): we’re using var (pfft, correctness), we have a shader attribute with a 3-byte name and a uniform with a 4-byte name, we’re using g.FLOAT (more on this in a moment), and we’ve got a lot of zeros.

g.vertexAttribPointer(
    a = g.getAttribLocation(x, 'p'),
    2,
    5126,
    g.uniform1f(g.getUniformLocation(x, 't'), z),
    0,
    0
)

Much better! We killed var removing 4 bytes, we removed 2 bytes by inlining the assignment to a in the first use and killing the semicolon, we eliminated a zero (the return of uniform1f is treated as a zero by the WebGL API in this case) and a semicolon for a savings of 2 bytes, we changed our shader attributes/uniforms to have 1-byte names (saving 5 bytes), and we inlined the FLOAT constant for a savings of 3 bytes. That’s 16 bytes removed from a block of 151 (after whitespace removal), or a reduction by 10.59%! Let’s do it again.

Before we move on, I just want to make a note that WebGL constants really are just that. You can inline them and trust them not to change.

Abusing globals

Look at the latest code above. Count the instances of ‘g.’. Four instances just in that tiny little snippet. If we could get rid of those in a small way, this could be a huge win, so let’s do it.

What is the global namespace in JS? Normally, it comes from the window object. So when you call foo(), it’s going to look in window if you haven’t declared it locally. So why don’t we shove our methods into the window object? Or even better, into top (in our case, they’re identical)?

Right now the declaration of g is as such:

g = z.getContext('experimental-webgl')

But let’s change it to:

for(k in g = z.getContext('experimental-webgl'))
    top[k] = g[k].bind && g[k].bind(g);

So we walk over every key in the WebGL context and put it into top, but only if it has the bind method and we can bind it to the WebGL context. Without that, it tries to treat the window as a WebGL context. The window doesn’t like this.

At this point, we just cut all instances of ‘g.’ out of the picture. This was a win of around 30 bytes (counting the cost of the globalization code) in my demo’s case, but we can go much, much further.

What’s in a name?

Let’s look at a list of all the WebGL methods I called in my demo:

attachShader
createProgram
linkProgram
createBuffer
bindBuffer
bufferData
viewport
createShader
shaderSource
compileShader
useProgram
getAttribLocation
getUniformLocation
uniform1f
uniform2f
vertexAttribPointer
enableVertexAttribArray
drawArrays

Holy moly, that’s a lot of bytes! But these are defined in WebGL, how can we change this?

Welllll… let’s look back at our globalization code. We get a name, k, and then bind g[k] intotop[k]. But we control what k goes into top. Having a map of long name into short name would be expensive, but what about a regex? I’m going to spare you the gory details, but after a while of tinkering with it, I determined that the optimal code for this is:

t[k.slice(1, -5).replace(/[ntalruicoh]/ig, '')] = t[k] = g[k].bind && g[k].bind(g);

In this case, t is top (because hey, 4 bytes!). We still assign the original name into t, but we also put the sliced and diced name into it. Why? Because this mangles two names we need to disambiguate: uniform1f and uniform2f. But that’s fine.

So what do the names look like after this?

S
eeP
kP
eeB
dB
ffe
e
eeS
deS
mpeS
seP
eb
efm
f
f
eexbP
beVeexb
w

Night and day difference, as you can see. You’ll obviously want to change the regex if you do this yourself, since your code will have different balances.

Other random wins

Arrays of digits tend to be pretty wasteful:

new Float32Array([0,0,2,0,0,2,2,0,2,2,0,2])

What about this instead?

new Float32Array('002002202202'.split(''))

The more digits you have, the bigger the win from this replacement is. Until we get to compression and everything changes.

Compression

Ok, we all know HTTP requests are trivially compressible, and this can improve load times and blah blah blah. Doesn’t matter when it comes to demos: if the size on the wire is 1k and the size on disk is 3k, your demo is 3k.

In the past, people have compressed their demos into images. This is an easy way to get a good size reduction — PNGs are just zlib’d data with a little header, basically. But these demos all had a PNG and an HTML+JS file that would load the PNG, draw it to a canvas, pull the pixels out of the canvas as a string, and eval that string. How can you make this into a single file demo?

Introduction to PNGs

The PNG file has an 8-byte signature followed by a series of simple chunks. The chunk format is as follows:

4 byte length
4 byte chunk type
<length> byte chunk data
4 byte CRC

Most of these are pretty clear, except chunk type. Chunk type is a FourCC, e.g. ‘IHDR’, which is mostly generic, with some exceptions which I’ll talk about shortly.

I’m not going to go into detail on the IHDR chunk (we can’t really mess with it, but I will say I use greyscale for simplicity purposes), but here’s what we start with.

8 byte signature
13 byte IHDR with 12 byte chunk header
X byte IDAT (covered below) with a 12 byte chunk header

The IDAT format is dead simple: 1 byte filtering method followed by your zlib deflated data.

Abusing PNGs

We start by defining our own chunk type (we’re allowed to do that!) before the IDAT chunk.

4 byte length
4 byte "jawh" (Just Another WebGL Hacker (TM))
X byte bootstrap
4 byte CRC

What is the bootstrap? Well, it’s what turns our PNG into code and runs it. Here’s the one I use:

<img onload=with(document.createElement('canvas'))p=width=4968,(c=getContext('2d')).drawImage(this,e='',0);
while(p)e+=String.fromCharCode(c.getImageData(0,0,p,1).data[p-=4]);
(t=top).eval(e) src=#>

The 4968 here is really the size of the decompressed data in bytes times 4 — there are 4 components to each pixel (red, green, blue, alpha) but we’re only using grayscale, so we need to offset that. If you look carefully, you’ll also notice that it walks backwards across the data, which should create a string in reverse, but I compensate for that when I compress the data, and reverse it before doing so. This saves a couple bytes. The image source is also important: rather than hard-coding the image filename and wasting space, it uses #, enabling it to treat itself as a PNG.

So what do we have now? We have a PNG containing some HTML. The browser first opens it as HTML (file extension is important here), then sniffs the MIME type and figures out it’s a PNG when it gets loaded into the image tag. We have a self-extracting PNG. But we can do better.

Why specs don’t matter

We’re currently using a chunk type of “jawh”. Clever and a nice little insider reference, but that’s 4 wasted bytes! The chunk type comes right before the data, so why don’t we make the chunk type into <img instead?

The spec says on the subject:

Four bits of the type code, namely bit 5 (value 32) of each byte, are used
to convey chunk properties. This choice means that a human can read off the
assigned properties according to whether each letter of the type code is
uppercase (bit 5 is 0) or lowercase (bit 5 is 1). However, decoders should
test the properties of an unknown chunk by numerically testing the specified
bits; testing whether a character is uppercase or lowercase is inefficient,
and even incorrect if a locale-specific case definition is used.

It then goes on to tell you that bit 5 of each of the bytes is:

First byte: 0 = critical, 1 = ancillary
Second byte: 0 = public, 1 = private
Third byte: reserved
Fourth byte: 0 = unsafe to copy, 1 = safe to copy

Since we could call it <iMg or <imG or any other combination of uppercase and lowercase, clearly we only care about the impact of < on the first byte. But if we look at bit 5 of <, we see that it’s already set high, so it’s an ancillary chunk — we’re in the clear and just saved 4 bytes!

Wait, no, neither Chrome nor Firefox load the image anymore. What the hell? This is the point where you realize: the spec doesn’t matter. No browser out there follows the PNG spec, whatsoever. Perfectly to-the-spec images won’t work, and horribly broken images will work. So let’s horribly break it. Note: Both Chrome and Firefox use libpng for image parsing, so you could go through the code and look for ways in which it’s mishandling data and go down that path, but for my purposes I found that just experimentally changing things and seeing how they break was Good Enough (TM).

Breaking PNG for fun and profit

Messing with the chunk type is out due to browser incompatibility, so what other options do we have? Well, who cares about CRCs.

Set the CRC to c=#>, save, refresh. Hey, it works in Chrome and Firefox! 4 bytes saved.

Well, we can’t push back towards the front due to the chunk type, and we can’t push forwards towards the IDAT chunk because of the size. But how do browsers handle chunk size mismatches?

If we set the size of the IDAT atom to c=#>, that unpacks to a size of 1042496867, which is obviously more than the size of the IDAT chunk. We shift ) sr into the CRC and set the IDAT size to c=#>, save, refresh. All browsers will set size = min(reportedSize, endOfFile – startOfChunk)! That saves us another 4 bytes.

So sure, we couldn’t get rid of jawh, but who cares? We just saved 8 bytes!

Compression caveats

Now you know how to make the PNG container small, but this says nothing about actually making your demo small. A few of the tips above are actually harmful in the context of compression. Why? Because repetitive data compresses really, really well. For instance, the Float32Array trick is nice when you’re dealing with uncompressed data, but ends up eating a couple extra bytes after compression. Reusing function and variable names is also very, very important. The more repetition you can introduce, the better off you’ll generally be.

Wrapping up

All in all, these tips (and general insanity) will let you build really small demos. My demo, Magister, went from around 3k down to just over 900 bytes when all was said and done. I ended up adding a message in the demo (“1k should be enough for anybody.”) to get it up to exactly 1024 bytes, which was my goal. Feels good to have to work up rather than down.

I hope this helped shine some light on the subject.

Above: Evan Wallace has once again created a much talked about tech-demo called, with great efficiency: WebGL Water. The tech demo features raytracing, soft shadows, caustics and “analytic” (meaning “fake” :) ambient occlusion. It all comes together in an interactive WebGL app that looks good and has a very decent framerate, even on middle-of-the-road hardware. Well done!

Above: Another piece from Evan Wallace, this time taking on path tracing to create more realistic lighting and shadowing in a scene. The tech-demo allows you to change the material, move the lights and add/remove objects in the scene, as well as moving them around. Fun to play with, and again, surprisingly fast.

Above: MKVDB has made a WebGL port of his native Windows 1k executable “Nautilus” intro. Goes fullscreen and is a little heavy for most mid-range computers, but if you have a fast one it looks really gorgeous.

Above: The famous Cadillac Ranch art installation in Texas has found a home online in the virtual Cadillac Ranch. You can move around (granted, the controls are not exactly top notch) and spray-paint the car-wrecks in the sand. Built on WebGL.

Welcome back! We created a character in part 1, using various 3D-modelling techniques inside of Maya (or your preferred modelling package). We now have our 3D-mesh and UV-layout ready to go, though our character looks kind of dull just with that default-gray Lambert-material in Maya.

Colors anyone?

In order to unleash your creativity and bring your character to life, you need one (or several) texture maps. We will start out with the diffuse map. In a nutshell, the diffuse map is a 2D picture that gets projected onto your 3D mesh, telling your application (or game) what color should be assigned to which polygon. Similarly, you have tons of other possible maps for things such as specularity, transparency, bumps and what not. The picture above depicts the notorious Sir Nigel Rochester (known from “Evil Junior” of course) as a 3D mesh with a diffuse-map containing all of his color information.

Let’s see how you go about making one of those diffuse maps!

Enter Photoshop

Photoshop is perhaps the application for painting 2D texture-maps (but feel free to give me a shelling in the comments field if you disagree). You may absolutely use other packages (or even hand paint your textures and scan them) but in this tutorial series I’m going to stick to Photoshop. Many 3D packages also provide more or less useful texture-painting capabilities, most notably ZBrush and BodyPaint. I will actually showcase some techniques in BodyPaint later on, so stay tuned for that.

Back to the action – fire up Photoshop and create a new file. I recommend the following settings for your texture maps:

• Width x height: 2048 x 2048 pixels
• Resolution: 300 pixels/inch
• RGB Color with 8 bits

This should result in a PSD file of ~12 MB. 2048 x 2048 pixels = “2K” in pixel-pusher lingo. 1024 = 1K. You get the idea. Now one piece of advice (which I learned for myself the hard way): don’t be cheap on your initial texture resolution — ever! By heeding this advice you will hopefully avoid the painful mistake of starting out with a texture resolution that is too low, only to find yourself starting from scratch because the texture you spent several hours working on looks blurry on your mesh. If your initial texture-file is 2K, you can always export your texture as 1K if need be. But a 1K master-file upscaled to a 2K texture map is surely an abomination.

Ok, so 2K should do for this project. Save the newly-created file as a PSD in the sourceimages directory of your Maya project. Name it something like: yourcharacter_diffuse.PSD.

Texturing preparations

Now you should have a clean 2K-canvas facing you in Photoshop. Let’s go through a few hints before you continue on your texturing business.

First, make sure you have exported a “UV Snapshot” from the UV Texture Editor inside Maya, as depicted in the picture below. I will use this snapshot as reference in the painting process.

Next, you can either save your texture map during the process as an image file and insert it into your map in Maya (as we discussed under “Assigning Maps” in part 1) — or — you could utilize this nifty trick: create a new material in your Hypershade Editor in Maya by selecting “PSD File”. Browse to your PSD file in “Image Name”. Select your mesh in Maya, then RIGHT-CLICK the “place2dTexture” node in Hypershade and click “Assign texture’s material to selection” (see image below).

You should be able to save some time by hitting “Save” in Photoshop and see the PSD material instantly update inside Maya. With that out of the way, let’s set up some layers inside of Photoshop.

Layers are among the texture artist’s best friends: they help creative (and often disorganized) people be effective (and in the process: more organized). They eat RAM, so don’t go crazy on your layers. But a few will help you experiment and save you a lot of time. Remember that you may also be part of a team and that other artists may need to work on your files. How to win friends in the art department: keep your files clean, properly named and organized.

The picture above shows my layers for Nigel’s PSD file.

• Put your UV snapshot at the top, with “normal” or “multiply” as the layer blending-mode.
• Create a folder for ambient-occlusion effects. This is a very nice trick I will dive into next.
• One folder for your “diffuse”, i.e. where you put all your paint layers.

Alright, now let’s bake that ambient-occlusion map…

Ambient occlusion

This step is optional, but with the process being a snap and the results being good — why not give it a try?

Ambient occlusion (AO) is an effect in which you let the renderer determine shadows for you. A white material is applied to everything in the scene, and then the scene is lit. Lighting and shadow information is calculated and the renderer “bakes” an AO-map containing only shades of black and white. By doing this, you basically let the renderer figure out shadows on your mesh. It can really add that little extra and also save you a lot if time in the texturing process. Lazy artists are good artists. I will briefly walk you through the process of baking AO-maps in Maya right now..

Ok, so this is what we will do: open up your 3D mesh in Maya and apply the ambient-occlusion material in the Hypershade Editor:

• Create the surface shader: Maya -> Surface -> Surface Shader.
• Create the AO shader: mental ray -> Textures -> mib_amb_occlusion.
• Connect the AO to the surface shader in the Work Area by MMB-dragging AO output to surface input (see picture). Choose default (outValue -> outColor).
• Select the AO shader and open the Attribute Editor (CTRL+A).
• Play with different settings under “Samples” (quality of output) and “Distance” (effect). Try setting “Samples” to 128 or 256.
• Apply this material to your mesh.

Now we need to “bake” (or in other words: render) this texture:

• Select your perspective camera in the Outliner. In the Attribute Editor set Environment -> Background Color to white.
• Select Rendering in the menu selection.
• Select Render -> Render using -> mental ray (you may need to enable mental ray in the plug-in manager) in the main menu.
• Select your mesh.
• Hit Lighting/Shading -> Batch Bake (mental ray) Settings. Set the resolution to your texture map resolution, e.g. 2K.
• Hit Convert when done.

The resulting map will be baked to renderData -> mental ray -> lightMap in your project directory. The result should look something like this:

Awesome. Now paste your freshly-baked AO-map onto its own layer in Photoshop, set the layer to “Multiply” and make sure it’s placed on top of any and all diffuse-layers you might already have. Adjust the blend-percentage to taste. Voilá!

3D texture-painting in BodyPaint

This step is optional but highly recommended if you have access to an application supporting 3D-painting. Painting a texture on a 2D canvas can be both limiting and frustrating because of distorted and slow feedback. 3D texture-painting enables a much more intuitive and responsive process because you paint directly onto the surface of the mesh. There are two methods for 3D-painting used by different solutions. ZBrush offers polygon-painting where you paint individual polygons directly. BodyPaint stores color information as an intermediary 2D-image. Maya also offers 3D-painting capabilities. I will use BodyPaint in this example. The principles are the same, regardless of application used. Here is what you need to do:

• Export the mesh you want to 3D-paint to an .OBJ file in Maya.
• Open BodyPaint. Create a new project. Import your .OBJ file.
• Select all mesh objects in the “Objects” tab.
• Create a new material in the “Materials” tab. RIGHT-CLICK the new material and hit “Apply”.
• Click on the material icon. In the “Attributes” tab click Color -> Texture -> Create New Texture. Alternatively, load your existing texture.
• RIGHT-CLICK the material icon and hit “Enable 3D Paint”.
• Create a new layer under the “Layers” tab.
• Start painting!

Now you can either make a texture from scratch or enhance your existing textures. 3D-painting is also great for painting in reference for 2D-painting. This is what I have done in the picture below. This way I can easily figure out where I want to paint Nigel’s shadows and skin in Photoshop.

When you are finished simply RIGHT-CLICK the texture icon next to the material under the “Material” tab and press Texture -> Save Texture as. Save as .PSD if you want to keep your layers.

Painting in Photoshop

Contrary to popular belief, you don’t need a PhD in Photoshop in order to paint awesome textures. All you really need is the paint brush. Here are a few additional tricks you may find useful:

Layer masks: Click in the background (outside the grid markings) in your UV-snapshot layer with the Magic Wand Tool (W). Click CTRL+SHIFT+I to invert the selection. Create a new layer. Then hit “Add layer mask” in the layers panel. You can see the result in the picture below. Next to the mini-version of your paint layer, you have a black-and-white representation of your layer mask. 100% black in the mask = 100% invisible, 100% white = 100% visible. In other words, layer masks are an extremely flexible and powerful tool in the painting process because you can “paint” in or out what should be visible in this particular layer. This is very nice when you want to blend your layers manually.

External textures: You can achieve insanely nice and lifelike effects by adding external textures and images onto a separate layer and experiment with the blending mode of the layer, e.g. “Multiply” or “Overlay”. Does your character wear armor? Try adding a metal texture with scratches and wear-and-tear.

Warping and liquify: Sometimes you need to push things around, like if you import a 2D texture that needs to be warped according to the grid lines of your UV snapshot. You can use Edit – > Transform -> Warp or Filter -> Liquify.

Pressure-sensitive tablet: This is a no-brainer. Texture-painting solely with a mouse is possible but extremely limiting. A pressure-sensitive tablet is a good investment (if you do not already have one) and will definitely make your day easier as an artist.

Pressure-sensitive brushes:  If you have a tablet, try experimenting with the brush settings under “Brushes”. Especially “Shape dynamics” (size) and “Other dynamics” (opacity).

Basic brushes: I tend to use a basic hard round brush, a soft one, and combinations of these with sensitivity for either size or opacity. Try making other brushes by painting a black-and-white pattern on a square canvas and hit Edit -> Define Brush Preset (or simply download free brushes on the net).

With those few tricks freshly memorized, it’s time to paint!

• Start with a color filling the entire background (left image).
• Import reference from 3D-painting if available.
• Using your UV snapshot, and popping in and out of Maya to check the effects of your actions, start painting in the basic colors with a hard round brush for each mesh piece or body part (middle image). You can also use your AO layer for reference.
• Once you have painted in all your desired colors, try enabling your AO layer to see how it looks (right image).
• Want more detail? Experiment with more advanced brush settings or add external textures with blending.

When satisfied, save out your work. I like saving as .PNG (which also stores transparency), though you may find a better choice depending on the requirements of your project (consult your art director or lead programmer if you have one). If you have an established pipeline, it will most likely do batch conversions of textures and assets at some point, so keep your original textures in high quality is always a good thing. Save the file in the same resolution as your PSD. Name it something like yourcharacter_diffuse.

Specularity and other maps

If you want additional control over other properties than just color, you can create other maps as well. For example, you may want Sir Nigel’s top hat to be extra shiny. Shininess, or rather specularity, is determined by a specular map. Create a new layer in your Photoshop texture-file and paint with black-and-white, black representing matte, gray semi-glossy and white very glossy. Paint as in the picture below:

You can use the same technique for controlling other surface-properties such as bumps (e.g. for hairs and fur) and transparency (the glass of Nigel’s monocle). A much-used map in video games is the normal map. Normal maps are color images providing rendering information for how light should be reflected on the surface of the mesh. They are often used for giving the illusion of indentations and protrusions on flat surfaces, such as muscles, bones and wrinkled clothing and so on. Normal maps can be created from high-polygon versions of your mesh.

Wrapping it up

OK, let’s conclude this part of the tutorial by adding our diffuse and specular maps in Maya:

• Create a Blinn material and assign it to your mesh.
• Open the Attribute Editor (CTRL+A) to edit the material properties.
• Enter your diffuse map in Common Material Properties -> Color.
• Enter your specular map in Specular Shading -> Specular Color.

If you want to render an image of the result, do the following:

• Add a primitive plane to your scene. Stretch it out so that it acts as the ground. Select two sides of the plane (the ones visible in your perspective camera), extrude those edges and “lift” them to create walls. When done, hit “2″ on your keyboard to smooth it. Apply a white Lambert-material to it.
• Then add an ambient light to your scene and position it.
• Set your renderer to mental ray.
• Position your perspective camera, open the IPR renderer and select a small portion of the scene for live preview-rendering.
• Now, play with the settings of your mesh’s material and the lights/background until you are satisfied with the result in the IPR renderer. Also try experimenting with the render settings such as quality.
• When happy, render the current frame with mental ray.

That concludes this tutorial part. We now have a 3D mesh with maps for diffuse and specularity. I hope you have enjoyed this tutorial, and please feel free to ask questions or add your own work in the forum!

Hi, I’m Ollie “cosmic” Borgardts, and I am the creator of the animated short “The Lacquerer”. I’m going to do my best to make this an interesting and entertaining read, but I should probably say up front that I will go into detail on how the movie was made. I’ll also touch upon the creative process and my personal experiences from making this production. At the bottom of this post I have attached a 30-minute walkthrough video in which I explain some of the techniques I used.

Behind “The Lacquerer”

The whole production took about 2 months (full time) to complete, and with such a long production time I feel I have plenty of tips about the production process and the ideas and techniques that went into the piece. The short was released at the Revision Demoscene Party 2011 where it won the Animation Competition.

The whole project started two months before the competition was to be held, when a good friend of mine, Pro from the demo group Nuance, told me about a promotional campaign for a 3D package named Messiah Studio at a very good price. I have been a faithful Maya-user for many years, but when I checked out the package, some of the features it offered were very interesting. Even though I had never used it before, I promptly ordered a license in the belief that I could learn what I needed to know in good time before the competition deadline got too close.

One trick I learned was to watch tutorials at double or triple speed to save some time

I really wanted to finish a cool production for Revision, so with hindsight being 20-20, I probably picked a bad time to start learning a new 3D package. I proceeded to spend quite some time just watching various tutorials online, to get to know the application and the differences from Maya. One trick I learned was to watch tutorials at double or triple speed to save some time (no joke, it’s what I did and it was very effective).

Just to touch on the fact that I decided to switch 3D packages. I do simply love Maya. I use an older version (6) which I bought several years ago and I’ve been using it so long that I know the program in and out and can therefore work quite efficiently in it. However, I do have some issues with it.

When creating digital characters I would begin in ZBrush and then bring it into Maya for rigging. To bind the bones to the geometry Maya requires you to go through a process called “weight painting”. This tells the skeleton how much influence each single bone has on the bound geometry. There are a few scripts that help you automate then I need to bind the bones to the geometry and do a process called weight painting. This tells the inserted digital skeleton how much influence every single bone has upon the bound geometry. There are scripts to partly automate the process, but to be honest: most of them suck, and doing it manually sucks even more.

There are most likely newer versions of Maya that has new features or techniques for rigging and skinning, but for me, taking the step over to Messiah really helped me out. One example: in Maya, if I want to change something after the bone structure has been set up and bound to the mesh, I would have to unbind all of it and repeat the process on the new or changed mesh.

For an artist, this workflow is killing productivity. I want to be free from the very beginnng, the planning stage, the modeling, texturing, animation and right up to the final export or the encoding process. I want to be able to make changes at any point in that toolchain. Period.

Another example: changing the character from a humanoid biped to, say, a 3-legged beast with 10 arms, I’d have to repeat the whole process. Again, this doesn’t work for me. This is one of the areas where Messiah shines. It has a very clever way of separating the binding/rigging-stage from the animation. If I wanted to add more bones, I’m able to do so, and the animation and other parts remain untouched. Awesome.

Thoughts on creativity (and art in general)

I was now in a place where I knew how to use Messiah — which is exactly the time that the creativity black hole hit me. I’d like to share some thoughts about creativity in general and about the approach I use when realizing my ideas.

My tools of choice are Maya (or now: Messiah), After Effects, Photoshop and ZBrush. Not necessarily in that order.

I believe that there are really no “new things”. Pretty much everything has been done before in some way. Even so I look at other artists and what they doing and I’m definitely learning something new every single day, and that’s a good thing. However, if it’s one thing I know it’s that creativity can’t be forced when you are on a deadline.

I want to surprise the people at the event with my production, but how could I surprise them if I can’t even surprise myself? For each project I start, my personal goal is always to surprise myself in some way — to create something I’d never thought of before it’s actually finished.

So how can this be achieved? Well, I know what I’m able to. I know what I’m too unskilled or too lazy for. I know my tools. I know my skills and my preference when it comes to art. I know these things inside and out, so how do I “trick” myself? Well, one approach is to combine existing things. Take, for example, three existing things and combine them in some random fashion. It doesn’t matter which way you do it. The point is to not be logical and simply negate the facts.

Just to give you a little example of the process that works well for me:

Look out your window at the scenery outside. Then go: “Why do trees have their roots planted in the groud? What if it was upside down?” Argument one: defined.

Continue with the simple assertion: “What would happen if the tree was planted upside down?” Well, for one, it wouldn’t be able to access any water. So the consequence of this could be that the roots would need to stretch to the clouds to access water. Argument two: defined.

But what with the birds that usually sit on the soft branches? They don’t want to sit on the ugly, muddy roots. No, they want to sit on the branches and leaves as before. Therefore the result could be that they hang themselves upside down instead, reaching for the branches. Argument three: defined.

..and boom — there’s your scene. The rest is just a matter of a technical completion, however you choose to proceed.

The image to the right isn’t exceptional, but I hope you get the idea how everything in an artistic world could be created in this way. Just let the things happen. You don’t want to control everything or be too logical here. That simply doesn’t work — or, to be more precise — it simply doesn’t work for me. I know a thing or two about the technical stuff, but in the creation process of something artistic, taking a logical approach is just disturbing the art.

However, when the creative process is completed, you can come back and go on working on it in a professional, logical and disciplined way.

When perfectionism and ego gets in the way

At an early stage of making “The Lacquerer” I just wanted to use ZBrush, model something and then animate it. I asked a friend of mine, Marc Ewald (a musician from the Netherlands), if he’d like to do a soundscape for it (just like he did for “Julie” my last short movie experiment for Breakpoint 2009). I believe he’s a genius in what he’s doing and so the brief I gave him went something like this:

Please think of a concept for a short film (but don’t tell me what it is) and then create a soundscape for it. No identifiable sounds like bird chirps, car horns and such. Just make something abstract that will fit with loosely defined terms of contrast, such as “fast”, “slow”, “large”, “small”, “oppresive”, “liberating”. Again: don’t tell me the meaning of it. I plan to let the story unfold itself while I’m making it.

The reason for this was of course to have no preference or preconceived notions when starting up. I could listen to the sounds as a small child would and just let the imagination kick in. What would I think of those sounds — what would they look like? Then I could concentrate on creating something visual around the sound completely disconnected from the visuals.

What I got from Marc was very good. Bombastic, epic, almost score-like in nature. I could picture it in a game such as “Metal Gear Solid”. It was absolutely fantastic and I was speechless. My problems started when I was trying to visualize them. Suddenly my creativity was .. gone.

I had vivid pictures in my mind, such as huge boss-fights in epic PlayStation 3-games. The sound was great, but I kept thinking: “How on earth am I going to be able to match this, visually? How can I live up to the expectations the sound is setting the scene for?”

I thought about all those things for a couple of days, and then, as it does, a happy accident occurred.

I stumbled upon some audio samples I had laying around on my harddrive. They sounded interesting so I chopped them up randomly to a 3 minutes collage. I also, for some reason, started modelling an old guy in ZBrush.

At this point I was slightly tired of the whole thing, but stuck to it anyway. Working on something was better than thinking about it. I believed that a solution would present itself, as it usually does.

I threw some bones onto the pretty sloppily created mesh and started animating it in a random way. No plot — just trying to forget how I would go about handling the miserable situation I was in. I had no idea on what to create, and I felt really bad for Marc who had created a great piece that simply wasn’t what I needed at the moment. I really wanted to use it, but it was dawning on me that it wasn’t what I was looking for this time.

I rendered out some seconds of crap and brought it into After Effects where I started playing with color correction and other generic things. I just wanted to keep working, waiting for the solution — the idea — the special technique.

This is when I more or less randomly started playing with a feature in After Effects called..

Time Remapping

Time Remapping squeezes and stretches your video footage by changing the timing and playback rate. You can slow down things or speed them up, and it does the same with the sound — just like a record on a turntable. It’s a very basic feature, and pretty much the first thing you learn when you start using it. However, I had I never thought about using this in a creative way. It was too simple, far away from the “epic effect” that you see in the movies. It’s totally overused, boring and just too simple.

In fact it’s so simple that I thought everyone using After Effects is using it. This is where ego kicked in.. “I wouldn’t use that — not me — the best digital artist in the universe — the one that only uses super-complex techniques — not even people at ILM do what I do..”

What utter bullshit! Ego struck again! The little bastard in my head took over and was controlling me in a very destructive way that lead to total stagnation — and I didn’t even realize it at the time. Instead of having fun with the new software, the digital tools and playing around with them to create something cool in a fun way, my ego — the perfectionism and logical approach — ruined everything.

This cheap and simple trick which I just played with by accident ended up as the concept for the shole short. All the content I was to create in the coming four weeks would be held together by this very basic tool. I decided to write it down so as not to loose it: “Create a very detailed creature, animate it in some way, sync it all with Time Remapping afterwards!”

This cheap, simple trick, found “by accident” was the concept for the whole movie. This would hold everything together. So I wrote it down that this won’t be lost again: “I’m going to create a creature with the best skills I have at this time – I’m going to bring it into messiah and animate it in some way, but in perfect sync to the audio to “destroy” it afterwards with simple time remapping“.

Then another problem presented itself: I still wanted Marc to do the soundscape. He has studied composition after all, and he’s by far my musician of choice to work with. But he needed some clips of the concept to create the audio, and that was a problem because none really existed at this point. My solution ended up being that I would animate the short with the glitchy audio I had already created, and he would create a new, better sound for it. Remixing it, in a way, but keeping the same major sync points.

Unfortunately, that never happened.

I was working non-stop with animating right up until the last day before Revision was about to begin, and therefore I couldn’t expect Marc to be able to create everything needed in only 12 hours. I’m quite sad about that, but I didn’t want to put him in the same position as I was in earlier — what if he wasn’t inspired with what I had created? Lesson learned: the next time I’m going to involve him at an earlier stage, promise.

Giving life to the creature (blendshape creation with 3D layers)

I spent some time modelling the figure, texturing it and creating the blendshapes for it. All of this was done inside of ZBrush. I love ZBrush, it has never let me down. In other applications you can very easily end up stuck in a one-way street somewhere down the pipeline or the production process, but with ZBrush I have never had this happen. At any point there is a way to backtrack and get back to what you need to change.

So what are blendshapes? You can think of them as layers in Photoshop, except in 3D space. You create different “versions” of your model on layers and with a slider you can switch between these and create stuff like facial expressions this way.

These are then exported with the original mesh and connected inside of the 3D application. Every major-vendor 3D package has those – Blender, mMx, Maya etc, but their lingo might differ (like “morph targets”). These imported blendshapes are then invisibly connected to your original model and can seamlessly be blended into each other. This again can then be keyframed and animated.

Principles of complex effects (or: “Hollywood has time and money”)

One look I wanted to create for the project the figure was one of having the character look like he was coated with something like clay or mud. If you’ve seen any of the X-men movies then you probably know the character “Mystique” (if not: this is what that looks like). Her skin morphs in a pretty crazy way. It’s not just blended though, it looks like there’s more going on under the surface, like tiny, greebly, flaky stuff. It looks really hard, but actually it is — once again — very simple.

I couldn’t find the technique described in any tutorial, book or in the 3D forums online, so I couldn’t get it exactly right. My effect isn’t as elaborate or worked-through like the one I mentioned, but I think it worked out well and I’m going to show you the basic principles behind it.

The basic idea is to create two morph targets — one untouched (a clean slate, if you will) and one for the tranformed shape (a mask or one with scales or whatever you like). Step one is taking these two models and blend them using influence modifier objects like a sphere or a cube or any shape you can think of. That gets you started, and you have great control over the look of the animation, but the desired “greeble-look” isn’t there yet. So step two is to add complexity where the blending areas of the structure are switching from one shape to another. For this I created a third map that distorts the model in a very overdone way. Very random and crazy.

At that blending point this third map is basically used to “destroy” the geometry in a small rim. This gave me that “wrapping around” effect you see when the figure is coated. I could have used an animated map as well to enhance this effect even more (and I think this is how they create the skin effect on “Mystique” as mentioned above), but I was satisifed with the way it looked.

Another, simpler, technique would have been to render out two versions of the animation and then simply blend them in post-production. However, this won’t give you correct shadows and the compositing job would have been at least triple the work in the end. Real geometry is nearly always best, if you can get it right.

The devil is in the Multi Displacement Map layering (aka “details”)

As you might know, the models you can get from ZBrush can have a very high poly-count. With an average computer with 4 GB of RAM you can easily make models containing up to 100 million polygons or more.

ZBrush can handle this because internally it’s not using polygons, but something they call pixols — more commonly known as voxels. These are pixels with depth information. It’s a clever fake so that you can work comfortably inside of ZBrush without lag, but once you actually export the model, it is converted into real polygons. The most common approach to maintain a high detail level is to create a displacement map or normal map and place this onto a lower poly-count version of the model.

Displacement maps are nothing more than grayscale images (colored ones for normal maps) that create the illusion of having the same high detailed level. A little clarification: with normal maps this is the truth, but with displacement maps the high poly count is acually created during the rendering process.

I took these mapps and added them to the blending process in Messiah with the same influence blend modifieres to bring in details in the model and color-space as well. However, a single displacement map didn’t yield the result I was looking for so I had to research the technique of using so-called “Multi displacement maps”.

Imagine a base model, like a sphere with 100 polygons. You can create a displacement map that deforms this sphere in a good shape, like the basic shape of a human head. With only one displacement map you can just pull “outwards and inwards” to create indentations. The side planes of each newly created “extraction” can then not be affected anymore by the same displacement map and this is a shame.

If you extract a normal with a displacement map that results in some sort of “cubic” form. Then you have no information left in the map to extract the faces again in direct way, because the “detailing side parts” are simply just being created during the rendering process. With this in mind, a single displacement map wasn’t really that useful. You need another map that provides that information, and the good thing is that you can stack tons of maps onto each other and get really deep details out of it. At the time I started this project I didn’t know how to do this, but as it turns out, it was surprisingly simple (as in: just the click of a button). You can watch the process in the making-of video at the bottom of this post.

Compositing is everything (or: “How to make crap look good”)

With the model, morph targets and textures done, it was time to getting it to look good. While I’m proficient in creating 3D models, it’d be a lie to say that this is my favourite thing in the process. I see it more as a necessary evil, because creating things in virtual 3D space eventually brings you to the point where it starts to suck. Or to put it differently: you can never expect to finish your projects in your 3D programme. You need post-processing. Even if your lighting setup is perfect, you animation looks good and the models are perfect — the true power comes when you step out of 3D space and enter post-processing land.

When I started toying with 3D modelling and animation almost 15 years ago I thought that the way the professionals worked was to set up a scene, model, animate it, set up the lights and click the render-button. In reality, no movie is made in this way, because it’s simply neither practical nor yield good enough results. Think about it like layers: what if the base rendering takes 15 hours to finish? What are you going to do if the shadows are too dark or the colors are slighty off? You can’t just render it again — a very bad workflow.

The correct (and only) way to do it is to composit different layers on top of another, with the base rendering being the lowest lever in your chain. Everything is done in layers, and when combining them (following some simple principles and a few standard rules) is what nails the look and feel of a movie. Once you know these rules you are also capable of breaking them, which is what I usually do. In all my pieces they were at some point at a stage that I just played around in Photoshop to try the find a look or a feel, but the basic workflow and the principles remained the same.

Warning — soapbox moment! It’s really about the basics. Turning on a plugin isn’t just lame, it’s counter-productive, because it means you’ll never know what’s truly going on under the hood, and therefore unable to control it well. Some plugins can make your life easier when you are in a hurry, but is no substitute for having a really good set of basic skills. Do your homework. Buy books on anatomy and lighting. Knowing these basic principles will make your work better.

Oh, and a top-tip: always render to still image sequences. If you suffer a crash or a powerloss during rendering to a movie-file, all is lost, while if you worked on a still image sequence, you can just set your renderer to continue where it left off.

Shading the character

The best looking renders, in my opinion, are the ones I get directly out of ZBrush. ZBrush is fast and uses lighting baked into so-called MatCap Materials. The only problem with ZBrush is that it has no real 3D space. Again, these are only pixels with depth information and so you can’t “go around them” (or I should rather say “walk inside of a model”), because you can’t rotate 2D. In ZBrush it’s actually 2.5D information you’re dealing with

To make skin look real you need a feature called subsurface scattering. Think of it as holding your hand in front of a strong light-source at night, where you can see the light through our fingers. This is due to the scattering of light inside of your skin. I really wanted this look in the movie, so to enhance it I set up a simple shading network inside of Messiah to get a rim shader.

For stills I get everything out of the programme: depth Pass, diffuse pass, SSS, specularity and so on. The trick lies in using these elements to make it look real in the compositing stage in Photoshop (if you’re working on a still image) or After effects (for movies).

The key things are: color, perspective, depth and hue.

I’m going to show you a simple way of getting a simple, really ugly and random scene to look realistic in some way by just following a few simple steps. Mind you, this technique is for still images only. You can see the end result in this image to the right, and for the how-to, watch the making-of video at the bottom of this post where I go through each step of the creation of this image.

Getting the look down to a sensible render time

I wanted to use the so-called MatCap Materials for the shading of the skin. Matcaps handle the way of shading in a very clever and fast way, and they look really cool, so I wanted to use these inside of Messiah. In Maya or Blender the setup for such a shading network is pretty easy, but not in version 6 of Maya. In messiah I had no clue on how do it, but luckily there is a nice and friendly Messiah-community from which I got help an tips on this. One of them even set up a full shading network for me, but sadly had no time to implement it due to the looming deadline. In my next production I’m going to use these techniques.

However, I still want to show you how the MatCaps can be used inside of Blender. The workflow can be most likely be applied to every other major 3D package.

In ZBrush, just render out a sphere with the desired material applied to it. Make sure the document width and height are at the same settings and that the sphere is perfectly placed in the center. Oh, and turn of perspective in ZBrush — using the orthogonal view will give you the same result. Export that texture and map it using to the color channel using the normals. Make sure that your shader is turned to shadeless because the light is already baked into the the texture, and the painted color information from the actual model is simply added on top of that. This way, you’ll get the exact same look as if you were still in ZBrush. That’s it.

With everything rendered out, it was time to have fun! I used Time Remapping here and there in After Effects and was fascinated by the sense of randomness and oppressive mood that it caused. The character was absolutely animated to the sound, but with the added effect of having the sound also stretched and squeezed where I used Time Remapping. I really liked how everything was coming together at this point.

One thing I haven’t touched upon is the glass panel in the movie. It was entirely done in post. Just a simple 2D tracking of the hands touching the glass and then a few dirt textures on a null object.

In terms of the story, my first idea was to the character break the glass when he hits it, but I found it more creepy and scary if he was still stuck behind the wall of glass.

“Making-of” and explore the scene files

If you’re interested in the scene files for this movie you can download them (project files & head), play with them and explore how they were done. I have also created a 30-minutes “Making-of” video which touches on many of the techniques described above.

I want to say thank you for your time and your interest in how all this was done. Of course not every single aspect and every problem that occurred during the creation process could be described here. Even if I’d tried to cover them here to warn you before embarking on your own projects, other problems will still occur. The devil is in the details.

It was really, really hard work to create this movie. Pain and fun and I earned a lot by going through the process. If they happen to read this, I would like to offer an apology to my friends and family who were really forgiving during my mood-swings. :) Also, thanks to the people sharing their comments and critique about the movie so far — I hope you will take time to watch it and comment below as well.

So, stay tuned, keep creating you own productions, don’t let discipline and perfectionism ruin your creativity — just don’t take life too serious and have fun (yes, I know — this applies to me as well :)