Machin-X: Digital Puppetry

Blender Game Engine Evolution

2009-06-13T11:42:00.002-05:00

As you can see in this demo, Blender's Game Engine has come a long way in a short time (the video above was rendered in real-time, not frame-by-frame). Actually, the GE is capable of even more than what this video demonstrates; the latest version (2.49) was recently released and includes a bunch of cool features including video textures and real-time dome rendering as well as improved modifier support, speed, logic and physics.

As the GE gets better and better, more and more people seem to be interested in using Blender to create Machinima and digital puppetry. You can download the latest version of Blender from Blender.org.

The next release of Blender, version 2.5 is a major overhaul of the program's underlying code, some of which dates back to 1994(!). I'm especially excited about the proposed new animation system “Animato” and the new event system, which you can see demonstrated below:

Although I don't think the GE is the focus on the 2.5 release, you can bet that at least a few improvements are on the way. Experimental builds of Blender 2.5 can be downloaded from Graphicall, but they may not be stable and aren't necessarily suitable for production use yet.

Terminator Salvation: The Machinima Series

2009-06-02T11:45:00.002-05:00

I haven't had time to check out Terminator Salvation: The Machinima Series or the feature film from whence it sprung for that matter, but it certainly looks good. The six-part series serves as a prequel to the feature film and features Blair Williams, played in the film by Moon Bloodgood, who voices the character in the Machinima series.

The series was made entirely using in-game assets from the Terminator Salvation video game and was overseen by Terminator Salvation director McG, who's building an innovative little empire for himself in Hollywood these days. This is the first official studio-produced Machinima series and it's such a great example of cross-platform synergy that it's easy to foresee a lot more of these kinds of projects in the future.

Terminator Salvation: The Machinima Series is available to download from iTunes and the X-Box Live Marketplace. Free recaps of each episode are available to watch on Machinima.com.

Bryn Oh Second Life Machinima

2009-05-10T23:20:00.002-05:00

I love Bryn Oh's trippy, fantastical Second Life Machinima. Read more about her on PuppetVision and watch her latest work, filmed in the Immersiva sim in Second Life.

An Animator Praises Sid the Science Kid

2009-04-11T22:35:00.005-05:00

Keith Lango thinks the use of digital puppetry on Sid the Science Kid makes sense and I have to agree (but then you knew that, didn't you?). More and more I can see how puppets and pixels really are coming together. Check out the comments for the above-linked post to read some interesting discussion on puppetry, mo-cap and animation.

BTW, new episodes of Sid The Science Kid start the week of May 4th on PBS.

Creature Shop Open House Recap

2009-01-15T16:16:00.001-06:00

Debra Kaufman has written a nice recap of the previously-blogged Siggraph/Visual Effects Society event at the Creature Shop (see previous post). A couple people who were there Tuesday night wrote in to tell me about it yesterday; it sounds like it was a great event.

Digital Puppetry Open House at Jim Henson's Creature Shop

2009-01-10T13:51:00.007-06:00

Puppeteers bring characters to life with the Jim Henson Company's patented Henson Digital Performance Studio.

The Visual Effects Society and LA ACM SIGGRAPH are hosting a very cool-sounding open house of sorts at Jim Henson's Creature Shop next week called Innovation in Puppetry: From Animatronics to CGI next Tuesday, January 13th at 6 p.m. The program includes a presentation about the Creature Shop, a live demo of the Henson Digital Performance Studio, an audience Q & A, guided tours of Jim Henson's Creature Shop and self-guided tours of the rest of the Henson lot.

This event is free to LA ACM SIGGRAPH and Visual Effects Society members and $20 for non-members. New members who sign up on site and pay the $40 annual membership fee will be admitted for free. If you live in or near L.A. and have an interest in the Creature Shop's work or have always wanted to see the Henson lot, this sounds like an opportunity you shouldn't miss.

Note: You're supposed to RSVP at the Visual Effects Society web site, but I couldn't figure out how to do it so if you're planning to attend, it's probably best to email them directly to sign up.

Digital Puppetry with Animata

2008-12-14T10:57:00.005-06:00

Animata is open source real-time animation software, "designed to create animations, interactive background projections for concerts, theatre and dance performances". The goal of Animata is to make it really easy to create scenes with digital puppets that can be controlled via physical sensors, microphones or cameras.

Animata is another very promising digital puppetry system and it's already been used for a couple of interesting projects. Animata Jazz Pub is an interesting demo that shows how the Animata software can be used to react to live audio input. Reverse Shadow Theatre (seen above) is an installation that was created with Animata, inspired by Wayang Kulit, an Indonesian form of shadow puppet theatre.

If you want to try playing around with the software yourself, the Animata software can be downloaded for free for both the Mac and PC. The source is licensed under the GPL and can also be downloaded from Google Code.

Reverse Shadow Theatre link via Mary Robinette-Kowal.

How Waldo Worked

2008-10-15T03:37:00.004-05:00

In this clip from Secrets of the Muppets Jim Henson explains how Waldo C. Graphic - one of the first digital puppet characters - was created and performed (the Waldo segment clicks in at about the 5:20 mark).

Digital Puppetry with Microsoft's Touchless

2008-10-08T17:42:00.005-05:00

Touchless is the name of new multitouch software from Microsoft Office Labs. Much like the recently-blogged CamTrax software (see previous post), it allows you to use almost anything to create input for a computer via an ordinary webcam. MS Labs has released a software developer kit under an open-source Microsoft Public License that can be used to create custom apps, like controllers for digital puppetry purposes.

I really think that the concept of optically tracking everyday objects (I believe the technical term is "colour blob tracking") has enormous potential for (affordable) digital puppetry; much more so than motion capture. I've added this to my already-lengthy list of technologies to check out as soon as I have some time to spare.

Via TechCrunch.

A Wii For Everyone

2008-09-30T09:50:00.000-05:00

Okay, this is very exciting...a company called Cam-Trax Technologies has developed software that allows almost any PC webcam (they say 95% of webcams are supported) to track up to four objects in real-time and with very high accuracy and reliability. Essentially, this allows you to "build your own Wii" (minus the $250 console and controllers) and opens up lots of very exciting possibilities.

The way it works is that you download an emulator to your PC (it only works with Windows unfortunately) and build a controller for your favourite video game. While the four objects is a little limiting, I don't see why this couldn't be used for conrolling digital puppets.

I have to play around with this sometime in the next week or two.

Via TechCrunch.

Behind the Scenes of Sid the Science Kid

2008-09-28T13:25:00.010-05:00

The digital puppetry series Sid the Science Kid (see previous post) is now on the air and TV Guide recently went behind the scenes to see how the Jim Henson Company creates it. The Wall Street Journal has also been covering the show's unique production process.

Speaking of which, Brian Henson was in Silver Spring, MD at the AFI Silver Theatre yesterday and gave a presentation about the history of puppetry and how the Henson Co. has evolved over the years from Muppets to animatronics to it's latest work on Sid the Science kid using the Henson Digital Performance System.

TV Guide video via Puppeteers Unite.

29/09/08 Update: Brian Henson's presentation was at the AFI Silver Theatre in Silver Spring, MD and not the Centre for Puppetry Arts as I originally reported so I've updated the post accordingly. Brian actually spoke at the Centre for Puppetry Arts on Sunday and Harvey Deneroff has some coverage on his blog.

Panda Puppet Update

2008-09-23T15:12:00.006-05:00

It's been quite awhile since I updated Machin-X and for that I apologise; I'm hoping to be able to do more frequent updates in the future. I've gotten dozens of emails about Panda Puppet over the past few months so I wanted to post an update about the project, where it's at and what is (and isn't) happening with it right now.

I have not done any serious work on Panda Puppet for most of this year. This is due in part because I've been busy on other puppetry-related projects, but also because I have been waiting for a number of new features in Blender that I believe will improve it's usability for Machinima/digital puppetry purposes.

I do currently have a very crude working version of Panda Puppet (for Blender 2.46),but it's extremely basic, hacked together and not really suitable for public release (browse the Panda Puppet posts to see some of the old demos). There have been lots of great improvements to Blender's game engine this year (most of them developed for the Apircot video game project). The two huge new features that I've been waiting for are full character animation in real-time and real-time softbodies in Bullet, Blender's physics engine. All of the new features developed for Apricot will be included in Blender 2.48, which is tentatively scheduled for release next week. Once that is out and Ashid's great video texture plugin is updated to work with 2.48 I think it'll be time to get back to serious work on Panda Puppet again.

There seems to be serious interest in putting together a small developer's site for the project and I'm also applying for a grant that would help speed along development on Panda Puppet. In the meantime, I would love to get some help with the project. Is anybody out there handy with rigging armatures in Blender? If you are drop me a line at puppetvision {at} gmail dot com.

Second Life Lip Sync

2008-03-24T12:00:00.006-06:00

My significant other and I hosted a small dinner party over the weekend attended by a number of fantastically talented and creative folks, one of whom is an artist who has been deeply involved in Second Life for awhile now. I talked about Panda Puppet a bit with him and we had a long discussion about Second Life and it's potential. I had rejected the idea of using Second Life for digital puppetry/Machinima purposes, but his enthusiasm for it coupled with the recent announcement of a basic lip sync tool for Second Life makes me want to reconsider (helpful lip sync tips can be found on The Second Life Wiki).

Moviesandbox Looks Promising

2008-02-08T09:22:00.000-06:00

For quite awhile now Friedrich Kirschner has been developing Moviesandbox, a really promising set of open source tools for creating real-time animation. In the video above Friedrich demonstrates some of Movesandbox puppeteering possibilities using a Gametrak controller and a Wiimote. You can find more videos documenting the features of Moviesandbox here.

The latest beta version of Moviesandbox is available to download for free, but it requires you to have Unreal Tournament 2004 installed on your computer in order to make it work.

Video via Puppeteers Unite.

Video and Blender's Game Engine

2008-01-02T03:05:00.001-06:00

Work continues with Panda Puppet slowly, but surely. I haven't been able to share much of what I have been doing lately because my work is focused on two client projects that I can't really blog about at this point, but I am starting up my Bear Town web series project again and plan to be using Panda Puppet heavily on that in the coming months.

What has me most excited right now though is that "Ashsid" - who wrote a lot of the original Blender scripts that much of Panda Puppet is based on - has been working on a cool new video texture plugin for Blender's game engine that allows you to combine real-time graphics with various other sources like video files, live video, rendered 3d scenes, etc. inside Blender.

This has all sorts of interesting applications, the coolest of which is tracking the movement of a camera and applying that movement to a Blender object. Here's a demo of it in action with Suzanne (the famous Blender monkey head) combined with live webcam video:

What excites me most about this is that it enables puppeteers working with monitors to have physical puppets and digital ones interact in real-time. That isn't a new idea of course, but what's great about this is that now anyone can download the software and try it themselves.

Nice work Ashsid!

Head tracking with the Wiimote

2007-12-27T21:07:00.000-06:00

Johnny Chung Lee (see previous post) has done it again! In his latest video he explains how to use the Wiimote and Wii sensor bar to make a head tracking system. As always, you can download the software to do this yourself from Johnny's site (link goes directly to a .zip file).

The most obvious application that I can see for this in digital puppetry is a quick and easy way to track a camera in 3D space. It could also be used to allow a puppeteer to see through a character's "eyes" in a virtual environment/set.

So many things to do with the Wiimote, so little time!

More About New Henson/PBS Digital Puppetry Show

2007-12-21T10:49:00.000-06:00

Sid The Science Kid is the star of a forthcoming digital puppetry series for kids.

The Muppet Newsflash reports that work is continuing on the forthcoming Jim Henson Company/PBS Kids digital puppetry series What's The Big Idea? (see previous post), but the show has been retitled Sid The Science Kid. The series' concept is using short sketches to teach children lessons about science and discovery and is being produced using the Henson Company's HDPS Digital Puppetry System. Forty half-hour episodes of the show are currently in production; there's no word whether or not it has been affected by the current writer's strike in the United States.

Look for Sid The Science Kid on PBS sometime in 2008.

Multi-touch Interface using the Wii Remote

2007-12-13T11:57:00.000-06:00

This is another incredibly cool Wiimote hack from Johnny Chung Lee (see previous post) that suggests all kinds of interesting possibilities for controlling digital puppet characters in real-time. Since the Wii tracks IR lights, Johnny has discovered that almost any surface - a projector screen, a tabletop, or an LCD - can be turned into a multi-touch screen that you can use like the iPod touch or iPhone. All you need is the Wiimote, a ball point pen equipped with an LED and some free software that can be downloaded here. Watch the video for details.

And as cool as it is, I'm even more excited to see the results of what Johnny is working on next - 3D tracking and head tracking with the Wiimote!

Special thanks to Brian for sending this to me!

Track Fingers With The Wiimote

2007-11-26T10:12:00.000-06:00

Here's another potentially cool user interface for creating digital puppetry and Machinima...Johnny Lee at Carnegie Mellon University has worked out a method for tracking his fingers using the Wiimote's infrared camera. The basic idea is very similar to the iPhone and the Gesturetek technology I wrote about back in March.

Via MAKE.

Virtual Puppeteer Goes In To The Dragon's Den

2007-11-20T01:07:00.000-06:00

Virtual Puppet, a UK company that "specializes in motion capture technologies and involves itself with the production of advertising, face to face experiential marketing and performance for theatre and live interactive television," was featured this past weekend on the British edition of Dragon's Den. For those unfamiliar with the show, the idea behind Dragon's Den is that entrepreneurs pitch their ideas to secure investment from a panel of business experts and self-made millionaires ("The Dragons"). The rules of the show stipulate that an entrepreneur must secure all of the financing they request from one or more Dragons or they go home with nothing. Virtual Puppet's Creative Director David Field appeared on the show to pitch the Dragons for £200,000 (roughly $410,000 US).

Virtual Puppet performs digital puppets at trade shows and retail locations using Alias' MotionBuilder software and the Gypsy motion capture system manufactured by Animazoo. You can see some examples of Virtual Puppet at work in the video above.

Wondering how David made out? Find out for yourself this clip from the show (sorry, but IE on a Windows machine required to view the clip).

Low Budget Real-Time Animation System

2007-10-24T10:36:00.000-05:00

This is a very cool demonstration of a low-budget realtime animation system that Roland Dell'Mour developed for his diploma-thesis. The system uses Maya, PIE, a midi-motion-capture-server and several other tools. The neat-looking controllers are Saitek Cyborg EVO joysticks with game pads attached. I use Cyborg EVOs myself now for digital puppetry and I like them a lot.

Halloween Themed Digital Puppets Available To Download

2007-09-29T09:08:00.001-05:00

Now this is a pretty cool use of digital puppetry and something fun for Halloween; a company called ImaginEERIEing sells some basic computer programs that display a digital puppet on a Windows of Mac computer that can be controlled using a keyboard or microphone. It's also possible to playback a prerecorded performance too. It looks especially effective when the computer screen is hidden in a facade like it is in the video above.

There are currently three different characters available. The "Magic Mirror" character seen in the video can be downloaded for free; there's also a skull and a talking pumpkin available for $15 US each.

Via The Odd-Lot Puppetry Company.

Henson Announces New Digital Puppetry Series

2007-09-05T19:10:00.000-05:00

The Jim Henson Company is partnering with PBS’ KCET to produce a new educational digital puppetry series entitled What's the Big Idea? using Henson's HDPS real-time animation/digital puppetry system. The goal of the series is to promote science readiness in preschool children. Forty-eight episodes of the series have been ordered. The show is scheduled to debut in the United States on PBS stations next year.

Quick Experiment With Panda Puppet

2007-09-04T05:26:00.000-05:00

This was just a quick experiment that I did yesterday...I took a head that was modeled and rigged by Calvin over at BlenderArtists.org and set it up so that it could be puppeteered right inside a standard Blender window (i.e. not using the game engine) with a joystick. It was quick to set-up and fairly easy to do, I may experiment a bit more with this.

BusinessWeek on the Future of Machinima

2007-09-01T10:14:00.000-05:00

BusinessWeek recently ran a story on Machinima and how more and more game developers are packaging their titles with Machinima creation tools. It's a good read that outlines the history of Machinima and ponders where the artform is headed in the future.