Heavy Rain is the story of a father whose kid has been kidnapped and his endeavour to get him back.

Immersion

The game uses several tricks to create immersion.

The first and most simple one is that you are almost always playing. There are numerous cut-scenes which look like a video, but suddenly you have to perform an action. If you don’t have the gamepad in hand at this moment, you’ll fail the action. This forces you to always be alert and ready.

Then if you want to perform an action (you have the choice not to), you’ll have to do precise movements with your joystick, moreover at a correct speed; for example if you want to reach out to an object to your right, simply push the joystick to the right. If you want to open a door, you’ll have to do an half circle, mimicking the rotation of the door. Remember, gesture gives you more immersion. I can’t help but thinking how this would be much more natural in VR !!

Then the game happens in realtime which means you sometimes have to think and act fast : do I have to shoot this guy before he kills my partner (that I don’t like) ? But he might be useful to my investigation ! But I need my partner even more ? Will I be able to reason with him ?  Damn he might pull the trigger any second now !! *BAM* … Damn I shot him .. did I make the correct choice? Could I save him ? As in real life, you’ll never know.

That’s one beauty of the game: each of your action has consequences on the story. David Cage, creator of the game and head of the french game studio Quantic Dreams, has written more than 2000 pages for this game which has 23 different endings. If you don’t have your kid do his homework, he will be angry at you the next day because his teacher didn’t like that. If you don’t kill the guy you’ve been ordered to kill, you won’t have clues to find your son. What if you die ? The game goes on with the other 3 characters.

It is also very realistic because you have to use your brain realistically. No puzzles or crazy wayfinding. You’re in a rush and have to phone a room in a motel. Damn, can you remember the room number that you’ve seen several times ? Or your on a crime scene (but you don’t know that) before the cops arrive and you’ve touched several objects. Will you remember which ones to be able to clean them all and erase all your traces ? As in real life, you’re left on your own with your aging memory. Same for human interactions, will you have empathy? Will you be cold? Use your heart intelligence.

If you remember Mel Slater’s latest paper about presence, it talks about a concept called Plausibility Illusion :

Plausibility Illusion is the illusion that what is apparently happening is really happening. This results from a sense that your actions have effects on the VE, that other events of the VE affect your sensations, and that these events are credible.

This is exactly what is happening in Heavy Rain; through all the points mentionned above, they have managed to achieve Plausibility Illusion, or cognitive immersion.

Imagine if on top of that you had Place Illusion, or perceptive immersion. This would be Presence, with a capital ‘P’, the graal of immersive VR (iVR).

In a previous article, I also talked about how the game Mirror’s Edge feels quite real to me : when I play it, I can feel the wind on my face and the void beneath my feet as I jump from one building to another.

Mirror’s Edge is the first game to hack your proprioception. (…) When you feel like you’re truly inside your character, speed suddenly means something. The opposite is also true. Without a sense of physicality, speed feels lifeless.

So who will be creating Mirror’s Rain ? Or Heavy Edge ? Mixing this proprioception hacking with plausibility illusion.

A first person VR game like that would feel very intense.

Bonus

How could VR Geeks not like this game ? At some points in Heavy Rain you use a VR desktop :

I like the idea of extending the real desk.

Ok there are some negatives sides to the game. Athough everything looks very real, you’re often right in the uncanney valley; the faces of the characters are great but their movements are a bit stiff. And I’m not at ease with the phases that require quick actions on the PS3 buttons.

But appart from that, it’s really a milestone for games, and for me a major step towards VR games.

Immersive 4 sided Cube for sale

Following the crisis in the Emirates, one of our clients must quickly sell its 4 sided immersive cube installed in november 2008 :

• Active projectors Christie Digital Mirage S+3K (SXGA+)
• 4 faces : 3m wide x 2.3m high
• 5.1 sound
• ART Tracking

This CAVE, installed in november 2008 is worth 565′000€. It is sold at 350′000€ , not including the shipping costs. Filters and lamps are new.

If you’re interested let me know, I’ll pass it on, but only if you let me play with it!

(The following picture *is not* for sale, it’s the one from the CRVM in Marseille)

In the end of 2006, Sony was already talking about it. Now they’ve finally made a demonstration at GDC of their upcoming 3D tracker, named Playstation Move :

Tt has gyroscopes, accelerometers, magnetometer for high rate updates and the Playstation Eye webcam watches the glowing sphere to recalibrate the position at each frame. It should also have two distinct elements, ala Wiimote & Nunchuck.

Develop-Online has more specs :

Features
“The latency for the Playstation Move is under one frame” – Scott Rohde, vice president of product development, SCEA.

PlayStation®Move motion controller
Three-axis gyroscope
Three-axis accelerometer
Terrestrial megnetic field sensor
Colour-changing sphere for Playstation Eye tracking
Bluetooth® technology
Vibration feedback

PlayStation®Move sub-controller
Built-in lithium-ion rechargeable battery
Bluetooth® technology
2 DUALSHOCK® or SIXAXIS® Wireless Controller replacement capability.

PlayStation® Eye
Built-in four-capsule microphone array
Echo cancellation
Background noise suppression

Price
“Under $100” (£47)

I’ve been able to test a pre-version and must admit I was really impressed with the reactivity and precision of that device. As you can see, it can also be used for nice AR applications :

Yes, it’s much better than a Wiimote since it’s a real, absolute 6DOF tracker (as long as the camera sees the sphere) . Ok with the Wiimotion plus it’s getting better.

Contrary to the wiimote, which has an embedded camera that sees the infrared dots, the camera is now on your TV looking at the device.

They both have the same occlusion problem, meaning that if the camera doesn’t see the marker, the inertial data will rapidly drift to become useless. This can happen is someone stands in front of you or if the device is behind you.

It’s a good start for VR games, especially if you can stick one to your head !

And as the PS3 is supposed to support stereoscopic displays soon, this will get exciting

( Especially if, as the rumour says, Killzone 3 uses the Move and is S3D compliant !!!)

Queen’s University
Authors: Samuel Jordan, Michael Greenspan
Abstract: ARPool is an augmented reality system designed to assist shot planning and execution in a game of billiards. Using a projector-camera combination, ARPool is capable of detecting the ID and location of each ball on the table, as well as tracking the position and orientation of the pool cue in real-time. This information is fed through a custom pool physics simulator to obtain a complete table-state timeline of the shot. The shot data is dynamically rendered in real-time directly on the surface of the table using 2D graphics.

A useful, practical, usable (sellable?) tool if you ask me !

I’ll be in NY from 15th to 19th March before going to 3DUI near Boston, and I’d love to meet some VR geeks there !

Also if you have a bed for me, even for one or two nights, I’d happily exchange that with extensive VR discussions

What interesting VR centers are there in the big apple ?

Here’s a game concept by David Arenou, using AR markers to track the player’s movement, but also to register the position of some real life obstacles.

As cool as Time Crisis but with more body immersion!

DIPLOMA : Immersive Rail Shooter from David Arenou on Vimeo.

The basic concept of shuttering eyewear for viewing stereoscopic images isn’t mine.  You can find mentions of it in the literature before my work began.  Missing from the early work were the elegant electro-optical shutters that we now have and a good communications link between the display and the eyewear. (…)

I founded StereoGraphics Corporation in 1980 for the specific purpose of developing stereoscopic technology and offering products to industry. (…)

The initial StereoGraphics products used tethered or wired eyewear for viewing a CRT monitor.  We needed to modify the monitors to run at a high field rate. (…) We wound up making motherboards that were introduced into monitors to goose them into running at 120 fields per second.  We took the sync pulse from the video signal and used that to tell our tethered eyewear when to shutter. (…) The result was a decent stereoscopic image, but the shutters used pretty high voltage (200-300 volts), had low transmission, had parallel electrodes running through them and made an odd buzzing sound that wasn’t exactly comforting close to your eyeballs.

[Insert here *very* technical information about how they created the LCD glass that will allow the light to pass or not]

I knew that the ultimate package would be one that would not involve any cables or wires, or a big controller the size of a hi-fi amplifier.  But how to fit everything into a pair of eyewear?? [Insert very technical information about lowering the power needs of the LCD system] (…)

(…) I became one of the first people to use optical compensation to improve the performance of liquid crystal parts.  This allowed us to leap from a 15:1 dynamic range to one of more than 800:1. (…)

Marv Ackerman designed the infrared link using a pulse width modulation scheme to designate the left from the right fields and to make sure that the shutters were in phase.

The marketing of CrystalEyes was an example of the sale of vaporware. We presold the concept by allowing Silicon Graphics to believe that they invented it. (…) Six months later at SIGGRAPH we showed the first CrystalEyes.   If Jim Clark [one of the founders of Silicon Graphics] had thought about it at all he would have realized that it was impossible to have put anything like CrystalEyes together in six months.

We got them just in time for SIGGRAPH.  I was in my hotel room at the convention assembling them with a little jeweler’s screwdriver, slaving away putting them together.  We got enough of them to work and got orders for hundreds and hundreds of them from Silicon Graphics.  They wanted them in their color with their brand, and we were happy to comply.  In no time at all SGI ate Evan&Sutherland’s lunch with their computer and molecular modeling applications.

We sold over a hundred thousand Crystal Eyes to people in fields like molecular modeling, aerial mapping, oil and gas exploration, and CAD over a period of something less than 20 years.  I believe millions of shuttering eyewear are now going to be sold for 3D TVs.  A great deal of effort has gone into making high-field-rate displays in order to provide sharper images of objects inmotion.  So much time and effort to make LCDs look like CRTs.

That high-field rate technology can be applied to stereoscopic field-sequential television using shuttering eyewear.  It means that the customer can buy after-market shuttering eyewear, which in turn means that the cost of the set can stay low but the stereo function can be enabled by purchasing the eyewear.

I’m happy to be around to see all of this come to pass.  Shuttering eyewear is a good solution for stereoscopic television – providing that people will put up with wearing eyewear at home – because it will become a commodity product, and the end-user at home will be able to buy these eyewear at a low price. And they will get smaller and lighter and eventually be all but indistinguishable from polarizing eyewear.

I didn’t think there was that much research involved. I had imagined that the LCD glass was already existing. So thank you Lenny and your team for giving us my favorite viewing device (after HMDs) !

On a more personnal note, thursday was my last day at Virtools and it was quite moving. But to cheer me up, my colleagues offered me the Fujifilm Finepix Real 3D W1, a stereoscopic camera !! A VR geek come true !! I’ve only played with it for two days but I’m so in love with it.. I’ll try to make a review when I come back from Sri Lanka !

As for me, and as the title of this post suggests, I’m leaving Virtools / Dassault Systèmes.

I won’t discuss  the reasons here. It’s been a tough, personnal and political decision, but sometimes you have to do what you have to do.

I’ll really miss working with David, Geoffrey, the whole Virtools team and some of Dassault Systèmes’ people. I love you, you know who you are! Thanks for the amazing time with you, for all you’ve taught me and all that we shared.

But I won’t miss the changes that DS has forced on us.

Now that I’m regaining freedom, I’ll start by some travelling (is there some VR in Sri Lanka or Costa Rica ??). Oh and I’ve updated my photo gallery.

Then I’ll probably have some time if you need my VR expertise. I’ll remind you of that later but we can already start to discuss it

Here’s to a bright new future !

Marc Lerenard, ESIEA teacher and VR expert has sent me two new videos of the project:

This version is monoscopic, but they’re also working on an anaglyph and active stereoscopy version.

The only remaining problem, as for any VR application, is that the required head tracking takes some space (you don’t see that but there are at least six Optitrack cameras dispatched around) and that for the stereoscopic version you would need glasses. You wouldn’t be able to do that with four autostereoscopic screens since their viewing range is too small and they only have horizontal parallax.

Other solutions like the Holopub by french company BeInRelief exist :

Although it’s totally not holographic (I hate that when ‘holography’ is misused as it is 99% of the time!!), it’s a pretty cool marketing solution : a screen, a semi-transparent mirror, and you’re done !  No glasses, no tracking, illusion of depth (no stereoscopy, no parallax), integration of 3d images on top of a physical object.

We’re in the future baby!

Here’s a summary of these links :

- Barco about the light efficiency of their stereo systems http://bit.ly/6rTKjP

- Namco Bandai will use RealD stereoscopic API for their games : http://bit.ly/7BLR5d

- Google Goggles, incredible visual search : http://www.google.com/mobile/goggles

- Great AR algorithm demo by Oxford @ Ismar 2007 http://bit.ly/phGam

- Haptic Ring http://bit.ly/20XMgj

- Autostereoscopic shell for the iPhone, 49$ http://www.spatialview.com/en/node/489

- Firefighters in a CAVE : http://www.youtube.com/watch?v=qk1VdSKRa4U

- Youtube in 3D Stereo (old) : http://googlesystem.blogspot.com/2009/07/youtube-3d.html

- AlloSphere, A 360-Degree Virtual Reality Chamber http://www.scientificamerican.com/article.cfm?id=allosphere-ucsb

- A french 3D Webcam : http://www.gizmodo.fr/2009/11/22/la-webcam-3d-crazy-cam-vous-donne-du-volume.html

- Oxford AR group publishes all their vision paper : http://www.robots.ox.ac.uk/ActiveVision/Papers/index.html#tag2009

- VR setup at CNRS : http://www.limsi.fr/venise/index.en.html

- Asus has a laptop compatible with nVidia 3D Vision ! http://bit.ly/5MnIO

- Softkinetic, Optrima Partner On 3D Gesture Recognition Tech (http://www.gamasutra.com/php-bin/news_index.php?story=26095)

- Nice iPhone app to take stereo pix : http://www.juicybitssoftware.com/3dcamera/

I’m also going to Lyon for the big VR orgy, so see you there !