A couple of months ago, I wrote about a Korean magic mirror that lets you try on makeup. I thought the results were less than satisfactory:

Well, the Japanese have tamed the virtual makeup down. Created for cosmetic giant Shiseido by Fujitsu, the following application can be found in kiosks across Tokyo:

More details on Engadget

And, if you are not lucky enough to live in East Asia, Walmart and the British drugstore chain Boots are piloting the following kiosk by EZFace. Unfortunately, it works on a static image and doesn’t augmented live video feed, so it’s not really AR.

Sensor systems like cameras, markers, RFID, QR codes (and more) are usually done as single methods to align our augments.  One challenge for a ubiquitous computing enviroment will be meshing together the various available sensors so computers have a seemless understanding of the world.   

This video from the University of Munich shows us how a multi-sensor system works.  It appears to be from 2008 (or at least the linked paper is.)  Here’s the description on the project:

TUM-FAR 2008: Dynamic fusion of several sensors (Gyro, UWB Ubisens, flat marker, ART). A user walks down a hallway and enters a room seeing augmentations different augmentations as he walks on: a sign at the door and a sheep on a table.

In the process, he is tracked by different devices, some planted in the environment (UWB, ART, paper marker), and some carried along with a mobile camera pack (gyro, UWB marker, ART marker). Our Ubitrack system automatically switches between different fusion modes depending on which sensors are currently delivering valid data. In consequence, the stability of the augmentations varies a lot: when high-precision ART-based optical tracking is not available lost (outside ART tracking range, or ART marker covered by a bag), the sheep moves off the table. As soon as ART is back, the sheep is back in its original place on the table.

Note that the user does not have to reconfigure the fusion setup at any point in time. An independent Ubitrack client continuously watches the current position of the user and the associates it with the known range of individual trackers, reconfiguring the fusion arrangements on the fly while the user moves about.

The project brings up an interesting question.  Is anyone working with multi-sensor systems?  We know we’ll need a mix of GPS and local image recognition and markers to achieve our goals, but is anyone working on this complex problem for a real product?  We’ve seen good image recognition with Google Goggles or SREngine, and GPS/accelerometer based AR is popular, but I’d like to see an app use both to achieve their aims. 

If you’re working on a multi-sensor project.  We’d love to hear about it at Games Alfresco.

In Switzerland you can play Pong. Yeah, I know, you can play Pong for about 30 years all around the world, but you could never play it like this – outside, with your legs serving as paddles.

It’s nothing new either – you could have rent the setup for the game for at least a couple of years. The company behind it writes:

The setting is very simple: a reflector on each foot is the only physical tool to interact with Sportpong. The interface is integrated in the field which is projected on the floor. The players control the game with their feet, nothing else. This control is intuitive, naturalistic and very direct.

I really, really, can’t wait to try it out. Last year I had a session of Atari Pong (the first in twenty years) and enjoyed it immensely. This looks even better. Would be great having it on ARE or ISMAR.
More details on sportpong.ch via SwissMiss.

The time has come for another fun-packed linkfest.
But before we begin, a special message to those of you who like beer as much as AR, and live in the vicinity of Munich. Toby of augmented.org is inviting you to the first Munich AR regulars’ table. You can find more details here, and don’t forget to take pictures. Feel free to tell me about your own AR event/meetup!

And now, for the links:

• Whimsical infographics on GigaOM portraying mobile AR application that are going to change the world.
• ReadWriteWeb asks whether AR should be open or closed, and the answer is not that clear when comparing Layar and acrossair, each advocating a different practice.
• Did you know that Yahoo filed a patent on geo-located social AR? How Apple of them.
• Augmented Planet writes about best practices in augmented reality browsers
• CNBC interviews the guys from QderoPateo we mentioned last week – they aim to build a custom AR device and software, would be interesting to see how successful they’ll be.
• Layar is on Europe’s top 100 startups, according to Techcrunch.
• A Czech AR ad for Peugeot 5008. Yeah, I mention it just in order to put a link to that old post of mine.

There were a couple of other interesting things happening this week, don’t you worry, I’ll have a dedicated post about them in the very near future. In the meantime, this week’s video comes to us via @chrisgrayson. It’s very cool, but unfortunately, I don’t understand how it was done. Yes, ARToolKit was involved, but how was the robot augmented to fire rockets? What provided the distance readings? If any of you read Japanese and can shed light on those question – please do so in the comments:

Have a great week!

The biggest news about the movie Avatar has been the 3D experience and the way its blown the doors off the previous records. The movie has garnered huge success because it pushed the boundaries of technology and told an interesting story.

I loved the movie and the way 3D helped give more perspective to the enviroment. My own Star Trek loving mother didn’t even realize the Na’vi were CGI. She thought they were people in blue suits (really… I’m not joking.) And though storytelling will become important to later advanced augmented reality applications, it’s not what I wanted to point out.

James Cameron is part art-dude and part tech-geek. He waited for years for the technology to ripen enough to do the movie the way he wanted. One of the innovations that he created for the movie was the Fusion camera for the live-action sequences. Normally, scenes are filmed before a green screen and then the CGI is added afterwards. The actors play a game of make-believe and the director has to guess at how the enviroment will unfold around them. CGI movies tend view flatly because the emotions are added later by the special effects guys and not the actors on the scene. Cameron has changed all that.

The Fusion camera system is an augmented reality viewport into the CGI world. When Cameron was filming the actors, he was able to direct them and see the results. When he looks through his camera, he can see them interacting with the world Pandora as the nine foot Na’vi and help them tell the story. The camera itself wasn’t even a real camera in the sense that it filmed the action. The camera allowed Cameron to see the action being recorded by multiple sensors and cameras.  Once the action was recorded, he could go back and reshoot the action from a different perspective, even with the actors gone.

Facial expression was another hurdle they had to jump to make the movie work. So they added little cameras hanging on people’s heads to capture their range of facial expressions and then tweaked algorithms to get them to react correctly.  Even now we can pull off this trick.

Together these systems are similar to an immersive augmented reality world. While we don’t have the HMDs, complete camera access and processing power to pull off the world of Pandora now, time and continued improvement will make lesser versions possible.

If you look at the Fusion camera system, the camera is essentially the HMD display, albeit a large and bulky one. Multiple cameras, RFIDs and tracking markers help the computer understand the world, and complex and powerful computers put all the pieces together. I can only imagine that this system could be turned into a mind-blowing game in an empty warehouse with the proper HMDs.

Essentially, the movie Avatar teaches us that augmented reality has sky-high practical possibilities. All the components of his Fusion system can be ported to the commercial world (not now, but in three or four years) and used to make complex and believable environments overlaid our own world.

In the future, you too can be a nine-foot tall blue Na’vi and you won’t even have to have your soul sucked through a fiber-optic tree.

Total Immersion leads the augmented reality industry in total projects (around 125 last year and they’re expecting over 250 in 2010.)  They’ve successfully created world-wide campaigns like Coke Zero and the Avatar i-Tag game line.  So when they talk about augmented reality, I want to make sure I’m taking notes.  Iriny Kuznetsova from 2Nova interviewed Nicolas Bapst about the company and their current activities.  The interview was short, but had a few interesting insights.

Total Immersion has done work for the military in creating augmented reality solutions that put simulated objects on the battlefield.  This is a much cheaper alternative to war-gaming with real equipment.  Hopefully this encourages the military to fund more see-through AR HMDs. 

Total Immersion expects that AR mobile marketing will be the new trend in the coming year and shows off a brief demonstration.  They’re converting their PC software to mobile to take advantage of the smartphone growth.  I found Nicolas’ observation about how augmented reality marketing applications give you direct access to your customers interesting.  By moving people from static newspapers to the computer (and especially the smartphone), then they can find out exactly who is interested in their product and then leverage social media to spread the word.  Nicolas explains they doubled time on websites by adding augmented reality content.  I’m curious if this increase will sustain as the novelty of augmented reality wears off. 

Nothing game breaking here, but worth a few minutes if you’re not familiar with the company.

In January, flying Augmented Reality Drones stole the show at the Consumer Electronics Show: a quadricopter controlled with an iPhone, that unleashes augmented reality games. What a knock out.

Now it’s time for the land-based vehicles to show what they’ve made of.

Seac02 just published an SDK for developing augmented reality games for the WowWee Rovio.

The PC-based software (dubbed LinceoVR ) not only allows you to control the mobile-webcam Rovio , but can also recognize markers dropped in the perimeter, and overlay on top animated 3d models of enemy robots, weapons of augmented destruction, and more.

In fact, Andrea Carignano’s (Seac02 CEO) big idea is to empower young (and young at heart) out there to create their own augmented reality games.

Andrea explains:

“The Rovio is quite popular with several thousands of customers and a strong following among the tech community worldwide. This one of the reason we have chosen Rovio; the second reason is that you can use it even in an apartment or a really small room.

Rovio is currently available for $229 at Amazon.

The LinceoVR (AR enabling) software will be available next week for €25 and the SDK will ship in 2 months and allow any user to create her own new simple game, share on the internet, launch in a browser, start the plugin and control the drone from anywhere in the world. Nice.

Over the last 2 years we have seen growing interest in Augmented Reality in various events – panels, dev camps, meetups – and many more. Due to growing demand for knowledge and expertise in augmented reality (AR), a group of AR industry insiders, backed by the AR Consortium have put together the first commercial event dedicated to advance the business of augmented reality.

How is are2010 different from ISMAR…

…previously touted here as the “World’s best Augmented Reality event”?

Well, ISMAR is still the best AR event for the scientific community. If you want to learn about (or present) the latest advancements in AR research – you should be in Seoul this October for ISMAR 2010. However, for the rest of us, who wish to take advantage of AR in practice, in the commercial world, and build a business around it – there was a gaping hole.

That is, until now.

Meet the Augmented Reality Event.

Who’s this event for?

For established and start up AR companies -

For established and start up AR companies (such as Total Immersion, Metaio, Acrossair, Ogmento, Circ.us, Mobilizy, Layar, Zugara, Neogence, whurleyvision, Chaotic Moon Studios, and many more) – are2010 is a stage to showcase their products and services; a venue to form partnerships, learn about latest innovations, and most importantly speak with clients. Bruno Uzzan, CEO of Total Immersion will wow the audience with a cutting edge augmented reality show; Peter Meier, CTO of Metaio, will speak about his companies latest products. Early stage startups and individual developers will receive guidance from Cole Van Nice (Chart Venture Partners) for how to build a successful company in the AR space, including raising funding (from VCs that actually invest in AR), licensing technology and IP, legal aspects, forging partnerships, etc. Christine Perey will speak about the scope of the mobile AR industry today and it’s growth trajectory.

For Developers -

For developers, are2010 is a window into the latest AR algorithms, engines and programming tools. Learn from case studies and post mortems delivered by experienced developers from the leading companies in the space. Blair MacIntyre, director of the GVU Center’s Augmented Environments Lab at Georgia Tech, will speak about his experience with tools and technologies while developing augmented reality games. Daniel Wagner, one of the leading mobile AR researchers in the world, will bring developers into the wonderful world of mobile AR. Patrick O’Shaughnessey, which has lead the development of more webcam-based AR campaigns than anyone else I know – will share his knowledge of what works and what doesn’t. Mike Liebhold, Distinguished Fellow at the Institute for the Future , will speak about Technology foundations of an Open ARweb. Gene Becker, co-founder of AR DevCamp, will dive into augmented reality and ubiquitous computing, and Sean White, a pioneer in Green Tech AR will suggest concrete examples of how AR can help save the planet

For Mobile, Hardware, and Platform Companies

For Mobile, Hardware, and Platform companies (such as Vuzix, Nokia, Qualcomm, Intel, Qdero, Microsoft, Google, Apple etc.) are2010 consists of a captive audience to launch and showcase their latest devices, processors, AR glasses, sensors, etc. The best collective minds of the AR commercial world will be onsite to articulate the market demand characteristics and help influence the design of future hardware.

For Clients and Agencies -

For clients and agencies in entertainment, media, publishing, education, healthcare, government, tourism, and many more – are2010 offers everything you need to know about AR: how to leverage augmented reality to advance your brand, attract and keep your customers, and how to build successful campaigns and products that will delight users, including postmortems of landmark augmented reality projects.

Jarrell Pair, CTO and a founder of LP33.tv, will speak about “Augmented Reality in Music Entertainment: Then and Now”, Brian Selzer, co-founder and President of Ogmento, will deliver a crash course for clients and agencies about how to leverage AR in marketing campaigns. Marshal Kirkpatrick, lead blogger for ReadWriteWeb, will share the results of his AR survey collecting feedback from dozens of AR developers and their experience in delivering AR campaigns and apps. Kent Demain, designer of the visual effects in Minority Report, will open our minds with the talk: “Taking Hollywood visual effects spectacle out of the theatre and into your world”. And of course…

For any AR Enthusiast -

Are you an AR Enthusiast? If so, you’re going to feel like a kid in a candy store at ARE, with a soon-to-be unforgettable keynote by Bruce Sterling, demo gallery, exhibitors from leading companies, artists installations from AR artists such as Eric Gradman and Helen Papagiannis, and many more surprises.

If you are into Augmented Reality – are2010 is the one event you should attend this year.

Want to join the event? Early registration is now open!

It’s been a while since we last visited one of my favorite AR application, Sein Kanemura’s SREngine, and my, how it has grown!
What started as an AR browser-like image recognition based project, has now been released as a “Point&Find” like application for the iPhone. Available for free on the Japanese App Store, Eigaru, powered by SREngine, enables to look at movie posters and see associated trailers and reviews:

Now correct me if I’m wrong (I don’t have an iPhone), it does look like Eigaru works by analyzing the video input, and I thought that the API does not expose such functionality. So, is Eigaru the first of its kind?

More details on SREngine’s new home page (in Japanese)

I really fancy this next iPhone app. It’s called iButterfly and it transforms the habit of collecting coupons into a fun little game. Created by Japan’s Mobile Art Lab, the app tasks its users with catching virtual butterflies , each representing one or more coupons. You can even share
“butterflies” with your friends via Bluetooth.

Though the technology is not overwhelming (GPS based), the overall execution looks great, see for yourself:

Via MobileBehavior