The first change was lighting the scene primarily with IBL. I painted out the green screen from the light probe images before assembly in Photoshop and then setup the rendering to use the image for lighting and reflections. The “practical” lights are still in the scene, but they are now used primarily for generating the shadows. The intensity of the lights are turned down to between 10-12% of their original values.

I rearranged the scene a little bit. I added a couple of more ticket booths and parked the Cadillac in the drive. There are some other cars in the background, but they are mostly lost in depth-of-field. I had to remove the window from the ticket booth. There was no way, with the footage that I have, to fake the reflections in the window.

This time around I did all of the work, after rendering the background plates, in After Effects. I have two cast shadows from the live-action. The first shadow, on the booth, was created by leveling down the live action plate to make it black and the using a displacement effect in combination with a z-depth pass of the scene I rendered out in Max. The second shadow uses the same leveled black comp and a slant effect to match in to the ground. The advantages with the this method are: significantly decreased render times, interactive editing of the shadow appearance, and much greater flexibility. Since the live-action and the shadows are all generated from the same pre-composition, I can move the live-action footage around and the shadows follow and still distort properly on the background.

I added a couple of fake reflections, as well. One on the hood of the caddy and one on the windshield. The reflections are displaced like the shadows using the z-depth pass. Mostly, they’re okay. I think they help better integrate the footage, even if they aren’t perfect.

The final look is also a big departure from the first test.  I tried to keep it clean with very little noise or diffusion. The biggest problem I still have is the actual key, but that is do more to the source material than anything else. I’ll be sure to shoot better footage next time.

I probably should have done the playback on the 7″ monitor, or taken the SD Card inside to look at what I shot. But I didn’t. It turns out, I’m slightly out of focus. Not much. Not enough that I noticed, but a little bit. This is going to make keying out the edges a bit trickier and the composite a little odd.

Step one, set up a clipping mask and key out the background. As I mentioned earlier, the mask is a little sloppy. I needed to mask out the edges of the stage where the green screen stops and mask out the cross references I’d taped to the green screen. The crosses are mostly masked out. There are some frames where they’re visible, but this was good enough for a test. I added some curve adjustments to put a little red into the highlights to simulate the red bounce that would came off the side of the ticket booth.

Next, since I was slightly out of focus, I added a depth of field camera blur in After Effects using a z-depth pass that I rendered out of 3DSMax. I was pretty tempted to rearrange the scene to put some foreground elements in front of me. It would have been much easier to keep my feet on the ground by hiding them behind something. For a test, though, that would have been cheating. Do the math. Show your work.

I needed some shadows to nail my feet to the ground. Again, as I mentioned before, I forgot to place the reference cameras in the scene. I tried to compensate by rendering a keyed-matte out of After Effects and putting it on a card in Max with the image mapped to the camera.

It would be nice if textures mapped using the Environment Screen method cast shadows, but it turns out they don’t. I had to manually align and scale the UV coordinates to match the camera. Raytracing the shadows took longer than I expected, about 50 seconds a frame, so this was going to be an overnight render. I cheated the second shadow a little too. The foreground shadow is just the keyed footage flipped and scaled then leveled down to render as black. Both shadows were blurred and multiplied against the background plate behind the keyed-footage.

It felt pretty unnatural that I was the only thing moving in the shot. I added some footage of rolling crowds to the background, lowered the levels, and blurred them a bit to better match the depth-of-field. I particularly like that you can see the clouds through the window.

Last, I did some color grading and effects using Magic Bullet Looks. I kicked up the diffusion a little more than I should have, but I liked the look of the bloom from the background spilling over the live action plate. The highlights are blown out, but I think it works for this shot. I experimented with a lot of different looks. You could lose yourself for days with all the buttons, knobs, and switches.

So, here is the final composite. I give it about a 7.5 on the Believablity Index. I’m going to shot for an 8, or an 8.5 on the tracking tests.

Link to video, in case the embedding video isn’t working.

I placed the probe on the stage and measured the height from the center of the ball to the ground and then put my camera on the tripod at the same level. The probe is then photographed at multiple exposures to capture the light in high, medium, and low levels. Three exposures is the minimum required to generate the high dynamic range(HDR) image. Commonly five exposures are used. I ran through all 11 f-stops on the camera for no other reason than I could.

The images are then assembled in Photoshop and cropped to remove everything but the probe. Adobe has a nice automation feature that loads a set of images and then aligns them. The new image is then saved as a Radiance file and imported into Paul Debevec’s HDRShop and transformed from a spherical image into a latitude-longitude map. When this image is mapped onto a sphere in a 3D application, it recreates the original light probe.

Usually this data is then used for image-based lighting (IBL). When the scene is rendered light is emitted from a virtual spherical enviroment the surrounds the scene. There are a couple of issues with this. One, lights in an HDR sphere have direction, but no distance. They are calculated as infinitely far away. Two, they don’t cast shadows the way a true light would.

To work around this, I mapped the image onto an inverted sphere in the scene and then used that sphere as a reference for placing “real” lights in the 3D scene. I had the real-world data for the lights used: (3)500 watt quartz halogens with 24″ softboxes and (1)500 watt Lowell omni with a full scrim. The scrim reduces the output of the light about one stop. The softboxes would also reduce the light output, but I considered the diffusion negligible. A trip to Wikipedia and several missteps with MS Calc later I had the four light represented in the scene fairly accurately. The rectangular photometric lights at 24″x24″ an one one photometric omni at half intensity.

I should have masked out the green screen in the source images. This would have certainly been necessary had I used the probe for IBL. I did use the HDR enviroment map for reflections, but none of the objects reflection the ground and so the green screen in the image wasn’t an issue.

With the lights at a 500W equivalent (341,500 lumens) and the exposure control in Mental Ray set (shutter-speed 1/24 sec., f-stop 5.6, and film speed ISO 100–which as you may remember was an educated guess since I didn’t have the EXIF data), I had a pretty good match. I left the white balance at 5600 kelvin to maintain the warm (3200 kelvin) temperature of the lights. In the future I might experiment with daylight gels on the lights for the green screen, though I’m not convinced it would make much difference when keying out the live footage.

I did two shots, one tripod-mounted with me in from of the camera and the other on the crane shooting props. To do tracking tests I put a series of cross references on the green screen about 24 inches apart. I used green spike tape hoping I could key those out with the green screen. The spike tape, it turned out, is a little too reflective and shows up close to white on the final frames. This meant that I would have mask them out manually during the compositing. Not too hard accept when I crossed in front of the markers. I made a pretty sloppy traveling matte in After Effects. I could have tighted it up, but this is only test footage. If I were only shooting static shots on the tripod, I would do it without the markers.

“Starring” in my own test footage presenting itself with its own problems. Since I was in front of the camera, I couldn’t monitor what was happening behind the camera. To compensate I put the camera on mostly auto settings and focussed on the props, placing a mark center stage where I should stand. This mostly worked. I neglected to shoot a still before filming. The EXIF data for the picture would have given me the lens setting, the ISO, and the F-stop. I had to guess at these and I got pretty close.

I have two Creative Vado HD pocket-cams that I meant to setup to have reference footage for rendering shadows and reflections. This was somthing else I forgot to do. It’s important to have a checklist before setting up and filming. This was reinforced by my experience.

The last oversight was not having a scene prepared as a virtual set. It was only after I shot the footage that I started making the set. The lighting setup turned out to be a little awkward. As with all my other little errors, this was something I would have to work around.

On my primary workstation I don’t have any drives under a terabyte and it turns out that Photoshop 7.0 doesn’t recognize drives that large as a scratch disk. It errors out on startup everytime. So I needed to upgraded to Photoshop CS 5.1. I didn’t have many positive experience with the CS versions of Photoshop when I was at Treyarch. Each session would eventually slow to a crawl after a few hours of use requiring a restart of the software. I don’t know whether it was the speed of the computer or that I was running it under Windows Vista, but on a faster computer with Windows 7 I don’t have any of those issues.

Since Attack of Tropes will be shot entirely on green screen, I needed a really good chromakey package. My tests in Vegas worked well, but everything I’ve seen and read shows that Adobe After Effects with Key Light is the one of best chromakey solutions out there. After Effects is also the de facto standard for composting and visual effects. Lots of compositing for this project? Check. It looks like I’d need a licence of After Effects as well.

At this point, the Adobe’s Production Suite is looking pretty attractive. Which means I’d be paying for Adobe Premiere whether I’d like to or not. Way back in the day, Premiere was my preferred editor. The latest version has tight integration with After Effects and is one of the only NLEs to support HD Canon footage without conversion.

Sony DVD Architect is the DVD authoring software that I’ve become comfortable with, but CS also includes Encore, Adobe’s own DVD authoring tool. The good news? I don’t need to upgrade Sony Vegas and DVD Architect. The bad news? I’m getting too old to learn new software. You win this time, Adobe.

I decided to go old school, a combination of Max renders with a toon shader and drawing everything else in Photoshop with a lot of layers. I’ve grown a little weary of drawing Brad and Susan over-and-over again. So far I’ve had to draw 17 boards to get to the bottom of page one. Only 11 more pages to go.

As an exercise, the boards are doing exactly what they should be. There is a crane shot at the end of the first scene that I wasn’t 100% certain how I was going to do it. It calls for a sweeping shot from the front of the Cadillac, up and over the actors and back along the trunk. The only problem is that my crane doesn’t have a pan head. Off to the Bat Cave.

I set up the crane once again in my living room and set about trying to find a workable solution. I tried mounting the camera with an L-bracket. Good, but it wouldn’t work with the tilt. I considered mounting my tripod head on the underside of the camera mount. Nope, I don’t have the right connectors in my tool box.

Finally I broke out the magic arms, clamped them onto the camera mount and voilà! The camera mount worked and I’m able to set  a fixed pan arbitrarily while keeping the camera at a 90º angle to the tilt mount. The shot calls for a dolly as well as the crane motion, so I still have to decide if it’s worth getting a dolly-track system or just managing with the dolly that fits on the crane. We’ll see as I finish the storyboards whether I have any other shots that require a track.

A bit more poking around on the internet and it looked like the 1959 Cadillac convertible fit the bill. It has front bench seats (the seat back is split, but not in a way that wouldn’t work) and, more importantly, I found one for rent in the area at Cinema Vehicle Services. Also, the homage remains intact. It turns out that The Brain That Wouldn’t Die was filmed in 1959 under the title The Black Door, but wasn’t released until 1962.

So I sequestered myself at home for the last couple of days working on the 3D stand-in for the shots where a practical car is either unnecessary or impractical. I realized today, that despite nearly two decades working in 3D, this is the first car I ever modeled. I haven’t done any texturing on the car, the renders are with simple Mental Ray shaders. I’m mostly pleased with the first completed 3D model for the short.

Click to enlarge.

Click to enlarge.

Click to enlarge.

I don’t want to just post about equipment, but I haven’t found many resources online where people talk about the gear that they are usig and their experiences with them. This is, after all, meant to be a chronicle of my experience shooting the short and equipment is a big part of that.

There are a lot of moving parts to keep track of and stay on top of during the pre-production and I hope to post more about the actual work that I’m doing. I need to shoot some test footage either this week or next to be sure that my plans will actually survive contact with reality. Expect some interesting adventures coming soon as I wrangle with lighting, shooting on the green screen, attempt camera tracking, and comping the whole thing together.

For the in-game lighting we used a series of light probes that were placed in the level by the lighting artists and then gathered the lighting data from a radiosity solution. That lighting data was then encoded into spherical harmonics (which were all the rage in games back then) using 27 coefficients. Those light probes were then connected to each other using a Delaunay triangulation that was editable by the lighters. At run-time the barycentric coordinates of the object to be lit were used to blend between the three connected light probes. This resulted in a pretty spot-in match between the pre-rendered lighting in the level and the dynamic objects that were lit in realtime.

The other piece of “new” tech they used in Real Steel was a solution for the large crowds in the arenas. They shot a number of their extras on green screen acting out various crowd motions at different angles. They then put together a Nuke script that populated the stands with a card or each member of the audience. The footage played back was then chosen by the script based on the angle of the card to the camera. John Versluis and Matt Rusch did a very similar thing at EA for back on Medal of Honor: Pacific Assault also in 2004. In Pacific Assualt the farthest level-of-detail for the trees were cards generated from the high LOD trees. These trees were pre-rendered from a number of angles and the appropriate angle was chosen based on the angle to the camera.

I guess everything old is new again.