After our latest video, you had consistently sent us a very interesting series of questions which can be summarized in two: what would happen if a Qbo has another of the same “species” identical to him in front of him? And, is Qbo conscious of its aspects and of what he sees? Here you’ll have some answers in which we have tried to simplify the debate without entering in philosophical or ethical issues, and of course always presenting our own point of view.

From an early age, humans learn to recognize themselves before a mirror using two mechanisms: we have learned how we look like since we are 8 months old, and we verify if our actions or movements are “replicated” in the image of the mirror. Because we are aware of the actions we take and  we can recognize those actions in the mirror’s reflection, we have the ability to distinguish ourselves from another “identical” being if we were placed in front of he/she. This is the problem that a robot like Qbo must face in the case of being in front of a mirror or find another robot identical to him.

Inspired by this process of self-recognition in humans, we developed a new ROS node that is executed when the node “Object Recognizer”, previously trained, has identified a Qbo in the image. Using nose signals to see if the image seen by the robot matches its action, a Qbo can tell in real time whether he sees his image reflected in a mirror or he is watching another Qbo robot in front of him. The sequence of flashes of the nose is randomly generated in each process of recognition, so the probability that two robots generate the same sequence is very low, and even lower that they start to transmit it at the same time.

After the green Qbo recognizes another member of his “kind”, both hold a short conversation. For this brief chat we have used the Julius software (speech recognizer) and Festival (speech synthesis), so that each robot recognizes what the other says and responds according to the question or the comment heard, using a small base pre-programmed knowledge base for this purpose. Thanks to the speakers, the microphone and the software systems mentioned, the Qbo robots are able to hold a conversation using synthetic speech.

CONSCIOUSNESS IN MACHINES

Some think that “consciousness” is an intellectual capability that enables to store knowledge about the world and exploit that knowledge to constantly make decisions and predictions, as suggested in his book “On Intelligence” Jeff Hawkins, co-founder of PALM & HandSpring companies. Jeff suggests that human beings can be seen as complex forms of statistics machines which constantly attempt to predict in real-time immediate responses of the actions that are generated in our environment.

However, might Qbo be aware of what he learns and sees around him? To find it out, we should first answer the question of whether it is possible to algorithmically program consciousness or not. There are several theories about it, some advocated by Roger Penrose, whose work is the most critical with the development of a true artificial brain?, claiming that self-consciousness may be artificially reproduced but not algorithmically simulated. Others, however, affirm that the self-consciousness is a process that emerges as a result of algorithms and an appropriate learning. Today, thanks to neuroscience and neurobiology, we know that what we do is the result of learned and unconscious mechanisms.

To accept that a machine can have some sort of consciousness could be a deep wound to human narcissism. However, the question I’m willing to ask you is: Is it necessary to have a complex corporeal system (for example, that senses pain or cold) for a machine to be truly self-aware? Personally, I think not.

If we take the theory and the words of Roger Penrose, Qbo is NOT self-conscious, since it just “simulates” a behavior algorithmically learned. If we adopt the other theory, Qbo can be seen as a conscious being because it exploits knowledge of its appearance and the actions it takes.

In 2007, Hod Lipson, a robotic engineer at the Cornell University, gave a brilliant talk about the mother of all designs “evolution” and claimed that the self-consciousness might emerge from the evolution of a robot’s intellectual capabilities, including a demonstration of his postulate. On one hand, Hod Lipson argues that we should stop creating robots manually so that they can evolve themselves, and on the other hand he states and shows that we have to let the machines learn through reinforcement and learning.

According to Hod Lipson’s point of view, at the moment Qbo is not more than a bunch of motors, electronic boards, cameras, etc., forming a mechanical being that, through software, can be submitted to natural evolution. By manipulating the rewards of a series of games, challenges, or just learning, we can mold Qbo behavior to, for example, recognize a face of the same person in different situations (which is already done by J).

Does all this mean that Qbo may be aware of his actions or surroundings? The answer is that, for the moment, as Hod Lipson would say, Qbo is just a programmable electro-mechanical set that can see, hear, speak and move. The ecosystem and the community around Qbo, that can program him and teach him, will tell in the future if we could witness the birth of  “some spontaneous intelligence”. One thing is for sure: it will be a terribly difficult task, impossible to achieve by only a few engineers. Only through the work of a true community of programmers, designers and professors, we can solve this challenge.

&

HAPPY NEW YEAR

This video corresponds to a small experiment in which we put Qbo in front of the mirror to see if he can learn to recognize himself. For that, we used the “Object Recognition” mode and the “Face Recognition” mode. Qbo, using its stereoscopic vision, selects his image in the mirror and, with the help of one of the engineers, learns how to recognize himself. This quite simple experiment touches interesting psychological aspects of self-consciousness, whose complexity can be proved by the fact I already mentioned of the few species that can recognize themselves in front of the mirror. In this first version, a human guide presents Qbo to himself, but we are working so as the robot could present and self-recognize himself autonomously when found in front of the mirror.

[ UPDATED Nov.30th ]

Due to the large impact this video is having on the Internet, we have seen fit to explain how this “real” experiment was carried out  in our laboratories.

In the video, Qbo transits to  the “Object Recognition” state from its internal state machine,  to learn to recognize its mirror image, as if it were a regular object. Technically, how does Qbo do it?

The “Object Recognition” state is implemented to execute simultaneously different ROS nodes: one responsible for the head and base movement; another to select an object from the image using the stereoscopic vision; another to recognize objects or learn new ones. The object recognition algorithm uses “SURF” descriptors and the “Bags of Words” approach (through  the OpenCV library), and stores images in Qbo’s internal storage system.

Qbo has several stored answers and behaviors in an internal knowledge base, that we upgrade as the projects evolves, to make questions or orders to Qbo such as “What it this? or “Do this”. Qbo interprets the object “Myself” as a an ordinary object, for which it has special answers in its internal knowledge base such as “Woah. I’m learning myself” or “Oh. This is me. Nice”. Qbo selects its reflection in the mirror in the image that he sees using the stereoscopic vision, and one of our engineers interacts (speaks) to him so that Qbo can learn to recognize himself as another object. For direct interaction, Qbo uses the open-source software Julius for speech recognition (in the video, you see how Qbo receives the order to turn around, and he responds to it by moving its base 90 degrees), and Festival for voice synthesis.

And to the question: “What would happen if a Qbo sees another Qbo in front of him?”. The answer will be out there pretty soon!

Cartoon by Carmen Cordoba

This has been one of the biggest surprises of the trip, for two reasons, one because I could enter in the future house of Microsoft by the hand of one of its funders, who is also a professor of the Stanford University, in which I could see the Microsoft’s vision of robotics.

However, the best part was the privilege to meet and talk personally with Alex Acero, who is one of the best engineers that Microsoft has in its facilities and who works there from the year 1994. Alex Acero joined Microsoft Research in 1994, became manager of the speech grup in 2000 and since 2006 is currently a Research Area Manager directing an organization with over 50 researchers and engineers working on audio, speech, multimedia, communication, and natural language. Prior to joining Microsoft, he was the manager of the speech group at Telefonica Investigacion y Desarrollo (1992-1993) and a Senior Engineer at Apple Computer (1990-1991). He has 93 granted US patents.

Since 2000, Alex Acero is also Affiliate Professor of Electrical Engineering at the University of Washington and has taught Spoken Language Processing. He has participated in the PhD thesis committee of 7 students.

Alex got his Ph.D. in EE from Carnegie Mellon University in 1990, his MS from Rice University in 1987 and a Telecommunications Engineering Degree from the Universidad Politecnica de Madrid in 1985, all in Electrical Engineering.

Alex Acero & me

His work, as he says, is to play while they pay him. We talked about voice recognition systems and artificial vision, which continue to be too far from those used by Qbo with the difference that the systems of Microsoft are far more trained than ours of course. To give you an idea its acoustic model consists of several hundreds of hours of training while Qbo uses the open source model of Voxforge that has no more than 50 hours,  even though, every day, the number of minutes trained increases through the open-source recognition system Julius that is used by Qbo.

For the recognition and image processing used to make the Kinect recognize every movement patterns of a person, they recruited hundreds of people of all ages, heights, clothes, etc ., encountering many problems when some people wore for example a hat or skirt. And we may have talked for hours and hours. However Alex is someone who you know that exists but which is almost impossible to access to, so it was a real pleasure to meet and chat with him in person about robotics. Thanks Alex for your support to the project.

IMG_0828.JPG

IMG_0826.JPG

IMG_0825.JPG

IMG_0821.JPG

IMG_0819.JPG

IMG_0816.JPG

Thank you so much Alex…

Cartoon by Carmen Cordoba

This is perhaps the strongest “dish” of the trip.

As many may know Qbo works over a Linux-type operating system called OpenQbo, that consists in a modification made by our engineers of the Ubuntu distribution. Over OpenQbo, you can find a robotic software platform named ROS that is developed and maintain by the company Willow Garage that is also in charge of the development and selling of the PR2 robot. Before this trip, we arranged a meeting so that both company (Thecorpora and Willow Garage) could finally meet.

For some time the person responsible for maintaining the company’s blog and one of the main developers of ROS, Ken Conley received us very kindly since the moment Qbo and I entered the facilities.

The facilities are quite big and they are divided in several departments of development, testing, lab and assemble, as well as a presentation room not only for their PR2 robot but also for other robots that they receive from other research centers, which I considered very well-done and with a completely different mindset to the European. When Qbo and I went to the facilities we encounter researchers from an university, learning the ROS system for 2 weeks now.

For more than three hours we talked about the possibilities of Qbo and of the integration of ROS in our robot. As we were in an open area, we could meet and greet each and every of the members of Willow Garage and its CEO.

Two things that caught my attention. The first is that the environment of Willow Garage reminded me of Google’s, but with a closer treatment. I saw once again Melonee Wise, who I met at the Campus Party in Valencia this year and a friend of hers whose name escapes me at the moment and with whom I had the opportunity to chat during lunch. The other thing that caught my attention was that they assembled each of the robots, having a fairly large laboratory where several engineers assemble by hand every piece of the robot allowing me to take pictures inside the open PR2, which until now I was forbidden in other places that I’ve been. To give you an idea, every PR2 takes a couple of weeks to be fully assembled.

In the facilities, I had the opportunity to meet PR2 in several of its versions and it is really impressive, as for its appearance and structure. I didn’t count the units that I saw throughout their facilities but they were many, some completely finished and other half finished.
Thanks to all the team of Willow Garage for receiving Qbo and me.

PR2 & Qbo

PR2 & Qbo

PR2, Qbo & Me

PR2 & Qbo

PR2, Qbo & me

TurtleBot & Qbo

Turtlebot & Qbo

Me in the Laboratory

Deutch University

Before beginning this new post regarding the third day, I must comment some aspects that I find important and that weren’t present in my last post.
If you decide to settle in Silicon Valley with all your team of engineers, you must know that it is possible that they end in another company soon after.
Another thing that I didn’t comment and that caught my attention is that the famous cable cars, which appear in so many Hollywood movies and that you’ve seen in previous pictures, is guided by a mechanical cable under the asphalt and two persons are in charge of guiding the cable car in each of the street’s intersections. Therefore, when you cross the street where a cable car circulates, you hear the cable under your feet.

Cartoon by Carmen Cordoba

… and the day to visit Google in Mountan View has arrived, in the company of Qbo and people from StepOne. As it could not have been otherwise, we first made a welcoming picture and then we went to the meeting room previous to the DNA signature and personal identification with an important warning that it was totally prohibited to take pictures inside the facilities and in the afterward meetings. For that reason, you won’t see any pictures of the inside of the facilities.

Know user
Know the magic
Connect the two

These were the first sentences that appear in the presentation of the company of Mountain View, accompanied by multiple videos identifying in some way each one of the sentences previously presented. Regarding this, I must say that Google must have one of the best marketing and advertising teams because they really impressed me in the videos, despite using an easy technique to reach to people’s heart through the employment of images of small children and elderly people.

After finishing the presentation, we started to walk around to the heart of Google, where a lot of expectations rose when Qbo arrived.  Imagine the scene: hundred of young engineers that love technology were passing from one side to the other while a robot walk around nearby. At this point, it is likely that hundred of pictures of Qbo are in the picture’s folder of these engineers in Facebook or in another social network.

Google is huge, very very big, thus you can find anything there, from a swimming pool to a small beach where young fellows in swim shorts play volleyball. Incredible but true. It was always said that Google is another world and I can assure you that.

Google

Google Logo

Qbo, me & Google

Me & Google

Swimming Pool

VolleyBall

Muscles

Qbo & flowers

Qbo

Garden

Garden

Eating in Google is another show. I’ve never have seen a place so big to eat nor so many people eating at the same time. Imagine the kitchen, the number of cooks and the structure that they must have in Google so that every day, thousands of workers may chose Mexican, Mediterranean, American, Italian, Asian or Indian food. And of course, all is free. My menu for that day was a delicious Mexican burrito. a brownie and a Pepsi.
The activity in Google is frenetic, hundred of the famous Google’s color bicycles moved in every direction. We could even see the famous car of Google earth.
As it could not have been otherwise, we could not miss visiting the Google Store where I bought some things to remember.

Google Earth Car

Google Bycicle

Google Earth Car

Eating

My food

My Food

Dessert

Mexican burrito

Qbo & Google Store

Cartoon by Carmen Cordoba

After the “tourist” visit to Google, Qbo and I headed to Intel in Santa Clara where 3 people and a video-conference system connected to Arizona were waiting for a meeting which, for obvious reasons, I will not talk in this post, although I will mention some curious details of it.

Intel breathes in another setting, not better nor worse than Google but completely different. You could feel a quiet and friendly environment as well as a direct treatment and a lot of professionalism. I do not know if it was because my visit to Intel wasn’t of a tourist kind, but I found a very human and close treatment, something that in Google I didn’t feel at any time. I had the chance to walk for about two hours in Santa Clara and I think that this mood is also strongly influenced by the quietly and peaceful area where Intel is located.

In the meeting with Intel, we spoke of general technical details, of the infra structure, of the technology that is yet to arrive, but above all of the security. Intel gives a huge importance to security issues, both regarding the cloud computing as the possibility to hack the systems, and after the meeting, my perspective was that Intel had a clear idea that the future and the developments of the robotic systems or any other type must resolve some issues regarding the security and access to the systems. I was pleasantly surprised on this topic that I didn’t expect at all.

During the meeting, we spoke of some specific aspects of the robot, my vision of living-room robots, of the technology it uses inside and of future plans for Qbo. For me, it was very important that Intel (though it was known by its engineers in Intel’s event of New York held a few months ago) had the opportunity of seeing the robot in action in Intel’s headquarters so that they could touch it and see it physically. And, in spite of knowing the project from Qbo’s blog, they were pleasantly surprised of the appearance of Qbo and the closeness feeling that Qbo emitted. Some videos and new aspects, of which you are going to be witnesses very soon, were also presented in this meeting.

Logo

Logo

Sculpture inside Intel

Photography

Moore's

A lot of chips

A lot of chips

Intel History

Intel History

Well yes, the famous search engine was very near to Intel, so, despite not having anything scheduled with them, I tried to show them the robot to see if someone could meet me to show me that at one time they were the best search engine in the world.

Unfortunately, and though it seems weird, there were nobody in Yahoo. Where were the people? 15 minutes waiting and no one appeared at the entrance or around Yahoo, so Qbo and I were in a building incredibly big but quite eerie. And if you don’t believe me here are some pictures.

Building

Building

Logo

Chairs

Chairs

Logo entrance

ROCKETSPACE

Today I started my real San Francisco trip. Our first visit was to a technology incubator center were teams and new projects consolidate their ideas with the help of professionals. Unlike other countries, in San Francisco, this type of incubators are managed by private companies and thus, it is difficult to access to one of them with advantageous conditions. It is important to mention that in San Francisco there are also the project accelerators but, unlike the incubators, these help the projects when they are in the initial stages.

By visiting the three floors of the building, I notices that there are many projects related with Web 2.0 that were being “cooked” in this incubator. By asking our guide if there were any company that merged software with hardware, he said “no”. Curious, isn’t it?

However, what does a incubator of this type offer us? Imagine a center were the companies share secretaries, phone lines, high-speed Internet, meeting rooms, security, printers, photocopies, furniture, legal help and the possibility of working 24 hours a day with your security card. Well, roughly, this is a technological incubator were some reject and/or select companies to work there. Many of them search for a ecosystem among all companies that you can find here.

Street

Logo

Logo

RocketSpace Companies

more RS Companies

..and more RS Companies

Working

Working

Working

Woorking

Working

Sara Herald

Friends

Miguel_RedKaraoke

After a quick introduction made by Sara Herald, of the Step One company, Miguel Angel Díez of the network company Karaoke and Koldo, two businessmen settled in the incubator a few months ago, gave us a little talk about their experiences and how some things in San Francisco and Silicon Valley are nothing but myths.
To not extend myself to much, I will put below the things that caught my attention the most:

1. – Myth: Nobody gives money away to any project. Here not everything counts, so if your project isn’t appreciated don’t doubt that they will tell you that without any problems.
2. – Truth: You can find in any coffee shop big investors with a lot of money searching for new ideas that won’t appear in any way to be so.
3. – Family: Koldo and Miguel Angel have made a lot of emphasis in this subject, since a lot of people with big ideas return to their home cities because their family don’t adapt to these places.
4. – To live in San Francisco or Silicon Valley. According to them, it is pretty clear that if you have a family the best thing to do is to stay in the Silicon Valley because of their colleges, but if you live alone it is better to stay in San Francisco.
5. – In general, when you have the opportunity to make a contact with an investor, you have to take into account that you won’t have more than 5 minutes to explain to him your project (this is called Elevator Pitch), so you have to prepare your speech before going out to the street.
6. – In San Francisco everything is very very expensive, specially the housing and the transportations.
7. – Here not everybody is successful and you have to be open-minded if you have to change your path rapidly. You have to be very fast.
8. – You probably think that you have the best idea in the world and here everyone will want to help you. However, the reality is that in most cases there is already someone who launched this idea, so don’t get desperate about it. You have to adapt and differentiate from your competitors.

This are the main aspects of some of the things I concluded after the talk of these two Spanish businessmen settled in San Francisco, and although I have few hours in San Francisco, I can notice that there is a real obsession with the software rather than the hardware. Are we witnessing a passing fad and will the next projects consist in mixing hard and soft?

And about Qbo?

Cartoon by Karmen Cordoba

Qbo caught the attention of all workers who went down the stairs to go eat, since the moment I pick it up from its box to solve a small problem with one of its microphones. In less than half an hour, I answered at least 100 questions regarding the robot, asked by different people. Guys, this is San Francisco!! I would like to thank to the people in the center who helped me with the robot and make me available all the material I needed. Thank you.

Meeting

Meeting

me & Qbo

My new friend Anna

And this finished our visit to this San Francisco’s incubator. After the launch, the law firm ORRICK, through Greg Heibel, commented some things to have in mind before settling in USA (patent issues, visas, etc.) and then Julio Deulofeu, a guru in startups and inversions spoke about his vision of how one should innovate.

And until now, here’s the summary of today. Tomorrow, two of the strongest “dishes” of the trip, Google and Intel.

GOOGLE
10:30AM  The culture and the innovation of Google, by the hand of Jeff Aguero
Visiting Google’s facilities
Google Shop
1:15PM   Launch at Google

5:00     INTEL

And Remember….

Miguel

Qbo travelling to Silicon valley

For any lover of technology, speaking of Silicon Valley means speaking of opportunities, of dreams, of the mecca of technology and of big companies forged by disturbing minds in small garages. It’s been more than six years, curiously also in a garage, since I started this project more as a game than as a  culmination of a personal dream that I had after watching Star Wars for the first time. Who of us, who belong to the generation X, never dreamed of having a R2D2 in our houses?

At first, when Qbo was only an idea, developments were slow and daily challenges were very few. However,  as these challenges increased and these ideas that were inside my mind materialized, this game started to develop into an obsession that today, 9th of October of 2011, made me travel to San Fransisco with Qbo.

From the 10th to the 16th of this month, I will visit many big companies such as Google, Willlow Garage, INTEL, Microsoft and Sap, among others, but I will also meet some bloggers residents in San Francisco that want to meet Qbo. Thanks to them, with small or big blogs, many people now know the project and Qbo.

During these days, I will share with you all in the blog each and every one of my experiences with Qbo in Silicon Valley.

MISCELANEA PICTURES ( DAY 0 )

Flight View

Arriving to SF

Jazz Club

Room view

Day 1 (Columbus Day)

[via Wikipedia] Many countries in the New World and elsewhere celebrate the anniversary of Christopher Columbus’s arrival in the Americas, which occurred on October 12, 1492, as an official holiday. The event is celebrated as Columbus Day in the United States, as Día de la Raza in many countries in Latin America, as Discovery Day in the Bahamas, as Día de la Hispanidad and Fiesta Nacional in Spain and as Día de las Américas (Day of the Americas) in Uruguay. These holidays have been celebrated unofficially since the late 18th century, and officially in various areas since the early 20th century.

Today nobody works in San Francisco, so I spent my day visiting the city, as all shopping centers were open with great discounts. The day was very cloudy and it was difficult to take pictures to the Alcatraz Island and the Golden Gate bridge (see photos).

Unlike many Europeans think, it is practically impossible to find free access to the Internet here in San Francisco, so I suppose many have had problems following me through Google Latitude (I’m sorry for that).

As the day began, I took my typical American breakfast in a coffee shop called “Lori’s Dinner”, famous for its waffles and pancakes (truly delicious), and afterwards I went to the fisherman area near the Golden Gate bridge and to the famous Alcatraz Island. In this area there are many shops of all kinds, including a bar of “oxygen-based drinks” and the possibility of seeing a pack of sea lions in their natural habitat. Finally, I climbed and hold to the bars of the famous San Francisco’s cable car, which I saw so many times in American movies, while crossing the city’s streets. A beautiful day in San Francisco spent in the company of exceptional people, in spite of the bad weather.

And if today was a relaxing day that included a “cocktail networking”, tomorrow the action will start by visiting the RockerSpace – Duncan Logan in Silicon Valley and from now on we won’t stop visiting sites and companies.

Let the action begin…

Lori's dinner

Lori's Dinner

Lori's Dinner

My new friends

No comments

SF Street

Imagine yourself being able to connect to your Qbo robot from any part of the world, wherever it may be: your house, an office, a museum or anywhere else. Using your Android mobile phone, you can hear what the robot hears, see what is sees, as well as control its movement in its environment, communicate and even express emotions using its leds.

In order to accomplish this, we are developing a mobile application for Android devices. For the Mobile-Robot communication, you must previously identify yourself as the owner of the robot to avoid any intrusions. Once authenticated, you can hear, talk and see trough the devices of the robot.

If you prefer, you can use standard 3D glasses with red/green lens to see the images previously processes by the robot. Therefore, you could see the anaglyph image in 3D, as a results of the stereoscopic vision, as well as the images generated by the object/face detection algorithms.

Moreover, two joystick are available on the mobile screen, one for the base control and another for the head movement, so as to control the robot as you wish.

Other features that are being developed or are already completed consist in:
- Being able to change Qbo’s expressions through your devices, selecting different face gestures (happy, sad, angry, etc.)
- Making Qbo say whatever you wish, using Google’s Speech Recognition software or speaking directly to your phone’s mic.
- Selecting on your screen whatever you want the robot to follow. For example, if you connect to your house and your son wants to show you something, you can select your son’s figure in the image and Qbo will follow him autonomously. Furthermore, you can execute this procedure when you’re standing next to the robot.

How the system works?

Internally, Qbo acts as a server, with REST as a communication protocol, sending information in JSON format. You will find an API, very easy to use and very handy. For example, in order to move the head just pack the coordinates into a JSON, and make a HTTP petition to, for example  “http://192.168.4.6:8880/control/qbo_arduqbo/headServos” and Qbo will manage itself, launching the necessary ROS nodes.

The audio connection uses the SIP protocol (Session Initiative Protocol). We wanted to make each Qbo independent from any external SIP server, so each one has the Asterisk SIP server installed to make calls directly to the robot. On top, it has a SIP client to accept incoming calls and send audio back. This enables Qbo to communicate with devices regardless of the platform, since there are applications available in each of these systems (MacOSX, Windows, Linux, Android, IOS…).

In the next posts, we will describe other features of Qbo’s Android application.

Some other pictures of Qbo’s Android application