Veterobot.org

Veterobot at InnoRobo2013

2013-03-25T12:57:00.000+01:00

From 19-th to 21-st of March we were presenting our BeagleBoard-based Veter robot at InnoRobo2013.

With this large poster we presented the Veterobot project.

In this year, there was a fist start-up competition organized by InnoRobo. As one of the five finalists, we got an opportunity to present our project at exhibition and make a short talk by the panel discussion with investors. It was a great success for us. We have made a lot of new promising contacts and this was our main goal.

We have got the small booth (but do not mind, it was for free ;-) ) and have made our best out of it.

Anna at the booth.

There were a lot of interesting robots presented at the exhibition. However, most of them are still extremely expensive. For example, we were looking for the decent robotics arm for the next generation of our robot, but the one I found interesting has $30K price tag! Lots of robots comparable with ours were also much more expensive than our offer. The main conclusion we have made after the exhibition is - target consumer market and try to solve concrete problem with your robot.

With this respect, I found the robot presented by our booth neighbors, the French company called Wall-Ye, one of the most impressive and useful robot of the whole exhibition.

This is autonomous robot for cutting vine. Nice mechanical design. We did not get a chance to find out more about software, but it is rather impressive if the robot can drive autonomously in outdoor conditions and find the parts to cut out based on the image processing! Hat off guys!

Despite some minor organizational problems (no single date for competition-related announcements were held, there was rather big delay and no words of excuse) we still think that organization committee has made a great job. The organizers were really nice and ready to help. The bus tour through the night Lyon was really impressive! So we want to expose our great appreciation for the possibility to participate in InnoRobo. We will try our best to be there next year also (hopefully with huge booth in the middle of the hall :-) ).

We are among finalists by "Call for Robotics Start-ups" at InnoRobo'13

2013-02-22T15:25:00.000+01:00

Veterobot team is among five finalists by "Call for Robotics Start-ups" issued by InnoRobo. It looks like we will get an area to demonstrate our robot. So if you are attending this conference or collocated exhibition, we will be glad to meet you there.

Testing the wheeled version

2013-02-12T16:52:00.004+01:00

As announced in our previous post, we are working on wheeled version of our robot. Tires and shaft adapters are arrived today and we mount them to conduct first test drive. Here are some photos:

Wheeled version of the "Shark" model

The first test drive was more or less successful. However, there are still some problems we need to solve:

Shafts are too long for adapters we are using and as a results wheels are sitting to wide. The simplest way to fix it would be just to cut about 10mm out. The front wheels on the second picture illustrate how it would be.
While turning with one wheel stopped works reasonably well. However, the on-place turn where wheels are rotating in different directions does not work as it should be. It happens because we are using two-wheel drive version of the chassis. I hope that four-wheel drive chassis will fix the problem.

Except of issues mentioned above, it works well and makes a lot of fun to drive because the robot can go faster than tracked version! Please feel free to visit our project web-site to learn more about the robot.

Kinect and wheeled versions under development

2013-01-30T11:03:00.000+01:00

Sungreen has made the 3D model for wheeled version of the "Shark". We were pointed out that tracked version might have problems on thick carpet or wet tiles. That is why we want to try the wheeled version and compare the performance on different surfaces. Any thoughts whether it will drive better then tracked version? Some pros/cons of wheels vs. tracks?

"Shark" with wheels.

In addition we are thinking about adding Kinect sensor. It is rather big and it is not 100% clear yet what is the best way to do it. We can try to mount it on the existing "Shark" model or modify the alternative model.

Alternative body with Kinect mounted on it.


Wheeled "Shark" with Kinnect mounted on it.

Any comments on it are very welcome. For more details about the whole project, please visit our web-site.

The new "Shark" model of our robot

2013-01-19T15:30:00.000+01:00

Last year we spend considerable amount of time building the new version of our robotics vehicle. Here is the result - the small tracked vehicle which might be interesting for researchers in robotics, AI, computer vision as well as for DIY enthusiasts. All the CAD models, assembling instructions and software are open source.

The "Shark" model

We are using Dagu's tracked chassis. The body of the vehicle is 3D printed. The BeagleBoard-xM is used as on-board computer to control wide range of sensors (such as ultrasonic range finders, compass, cameras, etc.). If you are interested, please check out our web-site for more details.

3D printed enclosure

2012-05-13T21:06:00.000+02:00

We are currently working on the new version of our BeagleBoard-xM based robot. The main reason for redesign was the lack of wheel encoders which prevent us from developing more sophisticated navigation algorithms. That is why we decide to use Dagu Rover tracked chassis. However, not all our electronic could be fit in to the chassis, hence the need for additional enclosure on top of it.
After couple of experiments with standard plastic boxes, we decide to design and 3D print our own custom enclosure. The following pictures are the very first concept ideas developed by great 3D designer Sungreen, who also developed 3D model for our cockpit application.

3D model of the Dagu chassis

Concept enclosure on top of chassis

We need to place BeagleBoard and other parts inside

It is surprising how hard it is to find precise mechanical drawings for typical hobby electronic parts. Neither BeagleBoard, nor ultra-sonic range finders as well as many other components have mechanical drawings which are easy to find in the Internet. So currently we are measuring and drawing sketches for all our mechanical and electronic components. As soon as we are done, we hope that our 3D printer will arrive and we can start experimenting with it. So stay tuned if you like to see how the enclosure will finally looks like.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

Precise PWMs with GPIO using Xenomai kernel module

2012-04-23T02:29:00.000+02:00

Introduction
This article is the follow up to our previous post on PWM generation with GPIO using Xenomai real-time Linux extension. Despite clear improvements with Xenomai, the results were not good enough for us. In particular, under the heavy system load, PWMs were generated with jitter which was high enough that our servo motor starts shaking. So we decide to continue our search for the better solutions now in the kernel space.

This article represents the results we obtained after implementing Xenomai RTDM kernel module. It also compares performance of the kernel-based solution with our previous user-space based approach.

Problem
Our previous solution performs well if the system load is low to moderate. However, under heavy load, we were unable to keep the servo motor stable. We believe, that one of the main reasons for it was increased jitter which results in the in-precise pulse length. To check this hypothesis, we connect the oscilloscope to the signal (GPIO) pin to see what is really going on there. The following video illustrates what we have seen:

We start our xenomai-based user-space application with no system load and after about 5 seconds run tar jcf lib.tar.bz2 /usr/lib command which leads to the top showing about 95% CPU usage. The distance between two horizontal lines around the falling edge of the signal corresponds to 50uSecs (as also shown in the bottom of the scope's screen). So the very rough estimation of the jitter (just visually) is about 40uSec. So with the whole pulse length of 1500uS (50% duty or middle servo position) it would be little less then 1%. This result roughly correlate with observable servo motor movements described in the previous post.

Solution - Xenomai RTDM kernel module
It is well known, that moving into the kernel space could provide much better timing. We were trying to avoid this step as long as possible (because it is easier to develop and debug in the user space), but based on the results mentioned above it looks like we can not improve timing any more from the user space. So we decide to write kernel module. Xenomai offers convenient API to write real-time kernel space drivers which is called RTDM - Real-Time Driver Model (please see introduction paper and API documentation for more details).

The result of these efforts (source code) could be found on git-hub.This repository contains the kernel driver and user-space test application. There are some instructions about compilation and how to run the test available in the README file. If you decide to compile and run everything yourself, we recommend to read the post about how we set-up our development and cross-compilation environment.

The core part of the driver is the 20mSec periodic task (function) which is toggling the GPIO pin in the endless loop:

void pwm_task_proc(void *arg) {
  const int which = (int)arg;

  // Toggling the pins
  for(;;) {
      //set_data_out has offset 0x94

      //set gpio pin to 1 (up)
      iowrite32(0x40000000, gpio + 0x6094);

      // wait requested pulse width time (duty)
      if(0 != rtdm_task_sleep(up_period[which]))
        rtdm_printk("PWM: rtdm_task_sleep() returns error\n");

      //clear_data_out has offset 0x90

      //set gpio pin to 0 (down)
      iowrite32(0x40000000, gpio + 0x6090);

      // wait until the next pulse should start (20mS interval)
      if(0 != rtdm_task_wait_period())
        rtdm_printk("PWM: rtdm_task_wait_period() returns error\n");
}

This loop essentially sets the pin to 1, then waits requested amount of time (pulse width length), then sets the pin to 0 and waits until the next period starts.

Running driver-based solution under the same heavy load conditions shows considerable improvements compared to the user-space solution. The following video illustrates the performance of this solution:

Similar to the case with user-space application, there are two lines around falling edge with 50uS distance between them. Here we add the system load after about 8-th second on the video. The jitter is smaller then in the previous test and as a result, servo motor is also much more stable.

Still not perfect
Unfortunately, developed kernel module does not solve the problem completely. Despite clear improvement, servo is still shaking. Much less than before, but still shaking under the heavy CPU load. So we conclude that there might be two potential sources of the problem.

The first one, is rather obvious - the jitter observed in the video above. But taking in account, the simple generation loop mentioned above, we do not see how it could be improved. So the question is - did we reach the limit of Xenomai+Linux abilities to provide real-time behavior? If yes, then it is rather sad news. BeagleBoard xM we are using for this experiment is rather powerful computer running at 800MHz. Yet we still can not control servo motor as precise as the one can easily do with micro-controller. It is clear that there is much more going on with real OS then with single application on micro-controller. Yet our hope was (and still) that 800MHz with real-time supervisor (Xenomai) running Linux kernel as a separate preempt-able task could provide better performance (much smaller jitter).

Looking for further possible reasons for shaking servos, we notice somewhat strange behavior which we could not explain. To check what we are generating, we decide to display 20mS (similar as our signal) square wave generated by the oscilloscope near our PWM signal. We also instruct the scope to synchronize on the rising edge of this reference square wave. Our expectation was that we would see our PWM signal staying still with respect to the test square wave (because they have the same period of 20mS). That is why, we were surprised to see the following:

Here, the yellow (bottom) signal is the square wave generate by the scope. The blue one (up) is our PWM signal which is drifting to the right. Currently, we do not know what might be the reason for such behavior. The only idea is that rtdm_task_wait_period()function does not work precise enough and the process is systematically waked up later (hence the drift). If yes, the again, the question is did we reach the limit of Xenomai+Linux abilities to provide real-time behavior?

Of course, there is a chance that we just misunderstood some programming concept and there is a bug in our code :-)

Anyway, it seams that we are not done with this problem yet and there still open questions which can not answer right now. That is why, we would highly appreciate any hints and suggestions about what might be wrong with our implementation and how to improve it. So your comments are very welcome!

UPDATE: please see the following discussion thread for the promising idea how to fix the jitter problem.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

Comfortable kernel workflow on Beagleboard XM with nfsroot

2012-03-30T19:44:00.002+02:00

Overview

For the last couple of days, I wanted to concentrate on reducing time required to boot our robot. The first action I did was to find out minimum configuration for kernel. After couple of rounds of getting SD card between computer and Beagleboard XM, which is the base for our robot, I decided to check how other people are developing kernel on embedded systems. The standard approach is to boot kernel from network.

Fortunately, the hard part was already done: u-boot support for network chip is already implemented. Here I wanted to document steps, required to get it working. Most of the information is available somewhere on the Internet, here is just place to get it all together.

Unfortunately, I could not configure DHCP server to get required parameters for TFTP boot, so...

The mission: u-boot loads kernel from nfs and kernel mounts nfs root.

Assumptions

Network configuration: I will be using static IP addresses to minimize influence of "bad" DHPC server. It will be good for many additional reasons. The beagleboard connected with ethernet cable to switch :)

beagleboard 192.168.0.9
server 192.168.0.2
router 192.168.0.1

The configuration is done on Debian stable (6.0), but should work on Ubuntu as well without major changes.

Preparation

Create user

In order to isolate work on this project, I created new user on the system and granted corresponding rights:


$ sudo adduser maker
$ sudo vigr

with the last step, add user maker to sudo group, as we need to use sudo pretty often.

Setup networking

as mentioned above, we are using static addressing, so first add beagleboard to /etc/hosts


$ sudo vi /etc/hosts

and add following line
192.168.0.9 beagleboard

in order to minimize hassle with regenerated rsa_key for ssh, add following to your ssh_config


$ sudo vi /etc/ssh/ssh_config

and add following
Host 192.168.0.9 beagleboard
  CheckHostIP no
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

then we need to grant permission for nfs clients, for easy configuration, just allow everyone. Obviously, if the server is used in some sort of public environment, this should correspondingly changed.


$ sudo vi /etc/hosts.allow

and add following
ALL: LOCAL

now we are ready to install nfs server, if it's not installed yet


$ sudo apt-get install nfs-kernel-server

Create local folder for remote root file system and export this with nfs server


$ mkdir ~/rootfs
$ sudo vi /etc/exports

add following line
/home/maker/rootfs  beagleboard(rw,fsid=0,no_root_squash,no_subtree_check)

and restart nfs-server
$ sudo /etc/init.d/nfs-kernel-server restart

check if export is ok
$ sudo exportfs

Setup toolchain

While, obviously, the toolchain could be compiled from sources, there is no need, as Angstrom distribution provides well tested and configured toolchain. There are plenty of versions, but I tested with Angstrom next toolchain

As I'm on x86_64, the file is angstrom-v20110117-x86_64-linux-armv7a-linux-gnueabi-toolchain.tar.bz2

download the file, extract and remove unnecessary structure


$ wget http://www.angstrom-distribution.org/toolchains/next/angstrom-v20110117-x86_64-linux-armv7a-linux-gnueabi-toolchain.tar.bz2
$ tar xjf angstrom-v20110117-x86_64-linux-armv7a-linux-gnueabi-toolchain.tar.bz2
$ mv usr/local/angstrom/arm .
$ rm -rf var usr

The installation of Angstrom toolchain, theoretically, should be done in root folder (/), but I don't like to have something in the root, which belongs to one user only. Thus, installation in the HOME folder.
Now, let's add some useful aliases and setup PATH correspondingly


$ vi ~/.bashrc

and add following to the end
export PATH=$PATH:${HOME}/arm/bin
alias cross='ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- CFLAGS=" -march=armv7-a -fno-tree-vectorize -mthumb-interwork -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 " LOCALVERSION="" '
alias sudo_cross='sudo ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- LOCALVERSION="" INSTALL_MOD_PATH=${HOME}/rootfs'

Pick the changes with . ~/.bashrc or logout and loging again.

Check, that cross compiler is available "arm-angstrom-linux-gnueabi-gcc -v".

Extract root filesystem

In order to boot from nfs root, we need a root file system. The easiest is to build required file system with Angstrom Narcissus or download demo root system from Angstrom beagleboard demo images . Required MLO and u-boot could be taken from there as well. The u-boot version already contains required code to boot from nfs. So, downloading following files:


$ wget http://www.angstrom-distribution.org/demo/beagleboard/Angstrom-Beagleboard-demo-image-glibc-ipk-2011.1-beagleboard.rootfs.tar.bz2
$ wget http://www.angstrom-distribution.org/demo/beagleboard/MLO
$ wget http://www.angstrom-distribution.org/demo/beagleboard/u-boot.img

note: we don't need uImage as we will be building our own kernel

now, extract root file system in ~/rootfs


$ sudo tar -C ~/rootfs/ -xjf Angstrom-Beagleboard-demo-image-glibc-ipk-2011.1-beagleboard.rootfs.tar.bz2

Prepare kernel sources

I don't put the link to kernel sources, as everyone knows where they are. We will be compiling 2.6.38.8 as this is the last version which is supported with latest Xenomai. Additionally, as example, we will be patching source with xenomai 2.6.0 patch and we need working defconfig, which could be taken from our source code at Veter project on Github. Download following files:


$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.38.8.tar.bz2
$ wget https://github.com/veter-team/veterlayer/raw/master/recipes/linux/linux-xenomai-arm-2.6.38.8/defconfig
$ wget https://github.com/veter-team/veterlayer/raw/master/recipes/linux/linux-xenomai-arm-2.6.38.8/linux-xenomai-arm-2.6.38.8.patch

now, let's unpack sources, setup privileges and patch files, with optional git in between


$ cd ~/rootfs/usr
$ sudo mkdir src
$ sudo chown maker src
$ cd src
$ tar xjf ~/linux-2.6.38.8.tar.bz2
$ ln -s linux-2.6.38.8 linux
$ cd linux
$ git init 
$ git add .
$ git commit -m "vanilla linux 2.6.38.8"
$ patch -p1 <~/linux-xenomai-arm-2.6.38.8.patch
$ cp ~/defconfig ./.config
$ git add .
$ git commit -m "xenomai 2.6.0 for arm"

Compiling initial kernel

now we have everything ready to cross-compile kernel for arm:


$ cross make -j16 uImage modules
$ sudo_cross make -j16 modules_install

Time for coffee !

let's fix some links to get it right from the beagleboard perspective


$ cd ~/rootfs/lib/modules/2.6.38.8
$ sudo ln -s ../../../usr/src/linux build
$ sudo ln -s ../../../usr/src/linux source
$ cd ~/rootfs/boot
$ sudo rm uImage
$ sudo ln -s ../usr/src/linux/arch/arm/boot/uImage .

For the above-mentioned image, which is still based on 2.6.32 kernel, we need to change console


$ sudo vi ~/rootfs/etc/inittab

change console line from ttyS2 to
S:2345:respawn:/sbin/getty 115200 ttyO2

and remove NetworkManager. Its "smart" logic is not smart enough to deal with nfsroot


$ sudo rm ~/rootfs/etc/rc5.d/S28NetworkManager

Prepare card

The micro SD card need to have only 1 partition. The exercise of creating partitions and formatting with fat is left for reader. Then copy MLO and u-boot.img to the root of the card. And the most important create uEnv.txt file with instructions as to where to look for a kernel and nfs file system. Assuming card is mounted on /media/boot/


$ cp ~/MLO /media/boot/
$ sync
$ cp ~/u-boot.img /media/boot/
$ vi /media/boot/uEnv.txt

and copy this content

serverip=192.168.0.2
ipaddr=192.168.0.9
netmask=255.255.255.0
gatewayip=192.168.0.1
hostname=beagleboard
console=ttyO2,115200n8
optargs=mem=80M@0x80000000 mem=384M@0x88000000 xeno_hal.cpufreq=800000000
mpurate=800
root=/dev/nfs rw
rootpath=/home/maker/rootfs
nfsdevice=usb0:off
nfsrootargs=vers=3
usbethaddr=6E:58:95:1B:82:65
nfsargs=setenv bootargs console=${console} mpurate=${mpurate} ${optargs} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}:${nfsdevice} nfsroot=${serverip}:${rootpath},${nfsrootargs} root=${root} 
uenvcmd=echo Initializing net ...; dcache off; usb start; echo Loading kernel ...; nfs ${loadaddr} ${serverip}:${rootpath}/boot/uImage ; echo Booting from nfs ...; run nfsargs; bootm ${loadaddr}

The content is self-explanatory. Now we are ready for a big moment !

Booting

now, unmount card, insert it into beagle and enjoy you kernel from nfsroot !!!

After this moment, all files are on the server, there is almost nothing on the beagleboard itself and you could just press the reset on board at any moment, the content is save on nfs.

Rebuilding kernel on server

Obviously, you want to change the kernel configuration and rebuild kernel couple of time, otherwise, all this effort is for nothing. Having set the links above, it's super easy, almost as it would be on the native system


$ cd ~/rootfs/usr/src/linux
$ cross make mrproper
$ cross make clean
$ cross make menuconfig
$ cross make -j16 uImage modules
$ sudo_cross make -j16 modules_install

And press reset on the beagleboard.

Building modules on beagleboard

After you made you perfect kernel, you may want to develop some kernel modules. Building them with cross-compiler is possible, but not very effective. On the beagleboard you have already tree compiled exactly for your version of the kernel. Why don't use them and compile modules natively ?

Sure you can do this. One thing you should just take care of is scripts in the kernel source. They were compiled on the server with x86_64 architecture and they don't work on the beagleboard. You just need to rebuild scripts.

Caution: now as a root on beagleboard !


root@beagleboard $ cd /usr/src/linux
root@beagleboard $ make scripts
root@beagleboard $ mkdir ../test
root@beagleboard $ cd ../test
root@beagleboard $ vi Makefile

and put this content in the Makefile
obj-m += hello.o
 
all:
 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
 
clean:
 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

root@beagleboard $ vi hello.c

and create simple module
#include 
#include 
 
int init_module(void)
{
 printk(KERN_INFO "init_module() called\n");
 return 0;
}
 
void cleanup_module(void)
{
 printk(KERN_INFO "cleanup_module() called\n");
}

root@beagleboard $ make
root@beagleboard $ modinfo hello.ko
filename:       hello.ko
srcversion:     4F856ABA1F3290D5F81D961
depends:        
vermagic:       2.6.38.8+ preempt mod_unload modversions ARMv7 
root@beagleboard $ insmod hello.ko
root@beagleboard $ rmmod hello

Now all combination of compiling on server and beagleboard is possible, you could press reset at any time and reboot beagle, network is configured and working.

Congratulations: MISSION COMPLETED !!!

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

One-dimensional robot localization example

2012-02-29T03:27:00.001+01:00

Here is my first example for the Unit 1 of the CS373 free online class. Please see this post for the introduction of this example series. This example illustrates how to perform one-dimensional localization using histogram filter we have learned in Unit 1 of the class. I will be using our Veter vehicle to illustrate how to control real robotics vehicle. To illustrate the one-dimensional localization algorithm, I have made the following test environment:

Figure 1: Test set up for the one-dimensional localization test

The Figure 1 illustrates our vehicle and the line of A4 paper sheets which represents the "wall". Each sheet is labeled with the number (0, 2, 3, etc., from right to left) which is the position within the line. Sheets with numbers 1 and 4 are removed to simulate the "door". We assume here, that robot can make discrete movement one position left or right. So the localization problem is to determine current position of the robot, i.e. in front of which sheet the robot currently is. So it roughly corresponds to the following situation :-) :

I hope I am not violating any copyrights here by re-posting this picture from Unit 1 Notes. Otherwise, please let me know and I will remove it.

For demonstration purposes, we decide to perform measurements using on-board ultrasonic range finder (aka. sonar). Since our robot has only one sonar on the front panel, we need to turn towards the "wall" (or "door", we do not know...) to make measurements, then turn back and drive forward to the next adjustment position. We assume, that we start at the arbitrary unknown position facing towards the wall. So the motion step is: turn left, drive forward, turn right back to the wall, query and process new measurement. For simplicity, I assume that if sonar shows distance less then 50cm, than we are in front of wall. Otherwise, if distance is bigger, then there is nothing in front of us, i.e. we are facing the door.

To perform precise motions, like for example turning exactly 90 degrees, additional sensor data required (for example compass) as well as control algorithms such as PID or whatever else. However, to keep this example as simple as possible and make sure that focus stays on the localization topic, I decide to implement the control in very naive way. I turn corresponding motors on, wait some period of time and then turn them off. For example, turning the left track clockwise and the right counter-clockwise will rotate the robot to the left. So I just measure the time needed to make ~90 degrees turn and turning motors off after this amount of time. It works only for concrete surface, varies when battery gets discharged and very imprecise. But again, it was not a purpose of this example and I want to keep it as simple as possible. There is control algorithms topic in the course syllabus, so hopefully I will be able to address issues mentioned above with following examples for the corresponding units.

The whole program, which is in localization.py, defines two classes SensorDataReceiverI which is a callback interface to receive sensor data pushed from the vehicle. The nextSensorFrame() method is invoked every time the new sensor data arrives. I store just the last received measurement for compass and sonar to read them later. The second class is Client which is derived from Ice.Application class. Client class defines sense() and move() methods which perform sensor data and movement processing steps of the localization algorithm. They are copy/pasted from the Unit1 lecture. The run() method is the application entry point. It connects to the remote vehicle, set up the callback interface and then executes five sense/move steps to update position estimation probabilities. In addition, it also invoke corresponding commands to control the vehicle. In particular, there is a function makeMotionStep() defined in the Client class to perform turn-left/move/drive-forward/turn-right motion sequence mentioned above. Finally, for each step, the location probability array is printed out.

All sources are available on GitHub. .ice files over there are interface definition for remote communication. They will be automatically processed by Ice.loadSlice("--all vehicle.ice") command. More details on how to use ICE middleware could be found here. Documentation for the Python language binding for ICE is available here.

The following video shows the test set-up and robotics vehicle made five motion steps.

Finally, the set of graphs below illustrate the position estimation (probability of being at certain position) for each of five steps. This graphs are made based on the output produced by the python control program described above. It is easy to see, that after the step 5, there is much higher probability of being at the second position from the left then somewhere else. So, we solve the localization problem.

Step 1

Step 2

Step3


Step 4

Step 5

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

Remotely control robotics vehicle with Python

2012-02-29T01:44:00.000+01:00

After very positive experience with the Artificial Intelligence class, I decide to continue with Udacities CS373 "Programming a robotic car" class taught by Professor Sebastian Thrun. Since the beginning of the class, I have seen a lot of question in the discussion forum about the way to try algorithms we are learning with real robotic vehicles. That is why, I decide to start a series of posts where I will try to use exactly the algorithms we have studied in each unit to control our robotics vehicle. In this series I will use Python and try to keep the implementation of the core algorithm as close as possible to what was presented in the lectures. I hope it would be interesting and helpful for the students of the CS373 class like I am.

Since it is a pure hobby project and all the activities are conducted in the evenings and over weekends I am not sure whether I will keep the pace and manage to produce the demo for every unit. But I will try hard. All relevant sources will be located at GitHub

I just posted the first examples from the series on localization from Unit 1.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

How to build BeagleBoard based WiFi robot

2011-11-11T14:35:00.001+01:00

As it often happens with typical software developers, we were working hard on implementation of different features but did not spend enough time documenting our development :-) . As a result, there were not too much documentation and details about our project. So with this post we would like to announce the availability of more detailed project documentation and make brief overview of some interesting aspects of the project. The whole available documentation could be found at our web-site and on the project Wiki. It is, as usual, :) not complete yet, but hopefully good enough to understand how everything works and even rebuild it yourself.

Here is the brief summary (details are available here) of what we have done so far. As a chassis platform we are using the one from Pololu. On top of it we mount the BeagleBoard with connected WiFi adapter, camera and GPS receiver. To control the motors we are using standard PWM-controlled speed regulators. Control PWMs are generated directly from BeagleBoard using available hardware PWM generators as well as using GPIO. In addition, there are compass and ultrasound range finder connected over I2C available on BeagleBoard.

Here is how it everything looks like now:

Or like this, depending on what chassis are used:

Video is compressed in to h264 stream using OMAP's DSP in real-time. TI's DSP optimized codecs integrated with GStreamer are used for video compression. After that, the h264 stream is sent to the driver console over WiFi. We also solved a lot of problems related to firewall and NAT traversing.

Experimentally we found out that maintain constant video frame rate is the very important to provide comfortable driving experience. It is not a trivial goal to achieve if the video is transmitted over the Internet because available bandwidth changes permanently. To solve this problem we developed our own adaptive video streaming infrastructure. We have presented our adaptive streaming on the Gstreamer 2010 conference. There is presentation video and slides available.

In addition to the video stream, driver console also receives data collected from on-board sensors. It makes it possible, for example, to display robots current location on the map which is downloaded from the openstreetmap.org also in real-time.

The following picture illustrates how the driver "cockpit" looks like:

This is real screenshot (not a photoshop :-) ). Driver console application is written using OpenGL and works on Linux and Windows (should also work on Mac but I did not test it). The whole 3D model is made in Blender and exported in to the standard .DAE (COLLADA) format. So the project also contains rather evolved COLLADA visualisation library which supports animation and some other advanced features. Every time the new video frame is received and decoded to the raw RGB data block, the corresponding OpenGL texture is updated in 3D scene. The middle panel is then used as such textured surface. Panels to the left and right are to visualize additional information such as for example map with current location or to constantly scroll unreadable cool looking logs :-) .

I want to say special thanks to Sungreen (aka. Nikolay :-) )! He is kindly respond to my call for help on blender-3d.ru site and created different 3D cockpit models. If you are interested, you can take a look at this link. The site is in Russian but on the 2-nd page there are several examples of the alternative cockpit models.

All the software for this project (including recipes for OpenEmbedded/Angstrom to build Linux image for flash card) are available on github.com. We are currently in process of migrating from gitorious.org to github, so if some pieces are missing on github, chances are that they could be found in old repository.

Also, for some additional information about the project I would suggest to check our Blog.

I would really appreciate any comments about the project!

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

Real-time enough? About PWMs and shaky servos

2011-09-05T19:36:00.000+02:00

We are currently working on the next incarnation of our software and hardware platform for vehicles controlled over the Internet. Among others, we decide to investigate the way to control our robot without micro-controllers. In fact, BeagleBoard has a lot of GPIOs available. In addition, three hardware PWM generators and I2C bus should be enough at least to control the tank with camera, compass, GPS, and sonar. We believe that BeagleBoard is also powerful enough to accomplish all these tasks (including DSP-based real time video encoding) without additional micro controllers.If this claim is true, than it could be an advantage from the price, space and complexity point of view.

Hardware setup
There is a lot of devices can be controlled using Pulse Width Modulation (PWM). The very typical example is the standard servo motor. Also, it is very typical to use PWMs to talk to motor controllers which in turn control DC motors. That is why we decide to test the real-time performance of our whole system by implementing PWM generator using one of the available GPIO pins. Standard PWM signal (what we need to generate) is represented in the following picture:

Pulses has 20ms distance (50Hz) and servo position is encoded by the pulse width with ~0.7ms to ~2.0ms meaning left and right most positions respectively (actual values typically vary a little depending on the servo model and vendor).

From the hardware point of view, level shifting is necessary because GPIO pins on BeagleBoard have 1.8V output but we need 5V to control the servo. For these purposes we used TI's TXS0108E voltage-level translator with corresponding SSOP to DIP adapter (a little bit difficult to solder but is doable if you did not drink a lot of bear the day before :-) ).

Software setup
Pins on the output connector on BeagleBoard can be configured to play different roles. This configuration process is called PinMuxing. For me personally it was and still a little bit confusing where and how to do it. There are in general three places where you can do it: in u-boot, in kernel and in your application with direct memory access over /dev/mem.

The first two ways will give you file system interface with /sys/class/gpio/gpio* entries in file system. Standard read() and write() functions could be used with these files to control GPIO direction and output.

The third way will require to map the physical memory using /dev/mem and then read/write to special addresses to control the output of the GPIO pin. To make it work properly, the kernel config option CONFIG_OMAP_RESET_CLOCKS, which is enabled in the default BeagleBoard defconfigs (and is also enabled in the kernel used by Angstrom distribution we are using) should be switched off. This is a kernel power saving feature and somehow it leads to the kernel oops in this scenario.

Let's trigger GPIO
The obvious and straightforward approach to generate PWM signals mentioned above could be described with the following pseudo-code:

  for(;;) {

    set_gpio_output_to_high();

    sleep(something_between_0.7_and_2.0ms);

    set_gpio_outout_to_low();

sleep(20ms);}
This loop can be actually implemented even as a shell script. As an alternative, it is possible to use read() and write() function in C to implement set_gpio_output_to_high() and set_gpio_output_to_low(). What is important however, is to ensure that the pulse width remains exactly the same as long as you want the servo to stay in certain position. This is the place where the problem arise - Linux is not a real-time system and that is why there is no guarantee that sleep() function will return exactly after requested amount of time. Context switching between kernel and user space (as happens when using kernel functions such as read() and write()) may also add to the unpredictable timing. As a result the servos will become "nervous". They will shake periodically instead of staying in the desired position.

Problem
In order to achieve stable servo positioning two sources of the undeterministic timing behavior should be eliminated:

Unpredictable delays caused by context switching between kernel and user space.
Unpredictable thread sleep time with standard sleep() function

Solution 1 - eliminating context switches
Inspired by this and this postings, we decide to go the way with direct memory access made from our application to trigger the GPIO. As a result, we write very simple test program which implements PWM generation loop as following:
for(;;) {
      //set_data_out has offset 0x94
      gpio[OFFSET(0x6094)]=0x40000000;
      sleep(up_period);
      //clear_data_out has offset 0x90
      gpio[OFFSET(0x6090)]=0x40000000;
      sleep(20 * 1000); }
Here we replace the call to write() kernel function with direct memory access to trigger GPIO158. This solution should solve the problem with undeterministic timing resulted by context switching. So let us see what we can achieve with this implementation. To make this post not so boring, instead of showing typical oscilloscope measurements, we decide to place a video of our current testing vehicle where the video camera is mounted on the servo we are controlling with our PWM generation application.

The following video fragment illustrates the servo behavior when running the PWM generation with direct memory writes. No other load was put on the system (~99% system idle).

Initially, the servo is turned away from the camera. After a couple of seconds, we run the test application which rotates the servo towards the camera to illustrate that we are actually controlling the servo. What can be clearly observed is that servo is shaking. In addition, there is a typical noise from the servo could be heard which is also a sign that the position is constantly changing.

The situation become much worth if we put considerable load on the system. In this case we run another process in background which creates about 98% system load (~1% system idle shown by top).

In fact, those results were easy to predict. It is also obvious that, the version with write() function (not speaking about corresponding shell script) would show even worth results. That is why we decide even not to test this configuration.

Solution 2 - predictable timing with Xenomai
So now it becomes clear that we need deterministic sleep() function behavior to achieve stable servo positioning. There are two options we were consideing: PREEMPT_RT kernel patches and Xenomai. Speeking about preempt_rt, we did not manage to find such configuration of the Linux kernel used by Angstrom/OpenEmbedded which works on BeagleBoard and has preempt_rt patches for it.

Installing Xenomai was also not a straightforward task. There were problems applying patch to the kernel we were using. However, we were able to solve them. In addition, according to our understanding, Xenomai is considered as "more real-time" then preempt_rt because it runs whole Linux kernel as one of it's task in parallel with other real-time tasks and can preempt it almost at any point if necessary. So we decide to continue our experiments with Xenomai. At the time of writing, we were using kernel version 2.6.35.9 with Xenomai version 2.5.6.

After getting Xenomai running, we rewrite the test PWM generation application. The core generation loop was running as a real-time thread with high priority (99). We also replace sleep() function invocations with corresponding Xenomai functions:
for(;;) {
      //set_data_out has offset 0x94
      gpio[OFFSET(0x6094)]=0x40000000;
      rt_task_sleep(up_period);
      //clear_data_out has offset 0x90
      gpio[OFFSET(0x6090)]=0x40000000;
      rt_task_wait_period(NULL); }

Test results were much better! Actually, without system load, servos controlled by Xenomai application was perfectly stable. That is why there is no video for this case - there is nothing to see :-) because servo just remains at the defined position. No shaking, no noise.

As the next test, we put the system under heavy load either by running xeno-test program in background (puts 100% load on the system) or running our own vehicle control software which performs real-time video compression using DSP-enabled h264 codec and communicates intensively over the network with vehicle operator software (puts about 60% load on the system). Under these conditions, unfortunately, servo starts shaking again as illustrated by the following video fragment:

Nevertheless, this behavior is better then the one without Xenomai and in fact the best we were able to achieve so far. According to our (non formal) observations, what is typically causing increasing shaking is extensive console IO (connected with SSH over the USB WLan adapter on the BeagleBoard) or in general network activity.

Running the Xenomai's latency application in parallel with our test program reveals the latency of around 40 microseconds. Based on what we read in Internet, 40 microseconds is considered "normal/OK" latency for Linux/Xenomai running on ARM at 600MHz. Taking in account typical pulse width of about 1 millisecond, 40 microseconds is about 4%. Is this 4% in fact what we actually see on the video? If yes, does it mean that even with Xenomai it is not possible to control servos reliably even under moderate system load?!

We would really appreciate any feedback if someone familiar with the matter can provide some thoughts about the way to improve the situation. We are somehow do not want to accept the fact that Linux running on such a powerful system such as BeagleBoard can not be enough real-time to generate 1-2ms wide pulses at 50Hz with latency considerably less then 40us? We are really hope that there are some tweaks can be made to achieve much better real-time performance.

Software availability
Our whole software (including Xenomai-based control module) is open-source and available online here.

The relevant Xenomai-based servo control sources could be found here.

The whole source code repository contains the OE layer for Angstrom to generate image with all patches and dependencies to run our on-board software. This software, among others, illustrates how to control servo motors with direct memory access and Xenomai, how to communicate over I2C not using Linux kernel (for better real-time), how to work with I2C sensors such as compass and sonar, how to receive location from GPS sensor, how to compress video in real-time using DSP-enabled codecs with Gstreamer, how to send video and other sensor data over the network in real-time and much more.
At the client (driver) side we are using OpenGL and high performance GLSL-based rendering to display 3D driver console with live video applied as a texture to the defined surface, displaying map for received GPS position in real-time using openstreetmap.org and much more.
We are currently working hard on documenting all the software and hardware we have made so far. There is already some documentation for the project available here and here.

We would be very glad to receive any comments regarding question about Linux/Xenomai real-time performance stated in this post above, our project in general and of course, contributions are also very welcome.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

Stereo video stream from on-board cameras

2010-12-15T02:25:00.000+01:00

After completing our BeagleBoard based car, we decide to build the quad-copter which also could be controlled over the Internet. In addition, we set as our goal to implement adaptive live stereo video streaming from two on-board cameras based on our previous work presented at Gstreamer conference 2010.

PandaBoard folks from TI offered the opportunity to win the PandaBoard at Gstreamer conference. To participate, it was necessary to submit description of the intended project. We did participate and were very glad to get notification that we will receive the PandaBoard for our stereo video streaming project (the board is already arrived). However, hardware accelerated video encoders are not yet released by TI. That is why we decide to make an iteration using BeagleBoard xM to check some new software ideas and make more integrated (more compact, less weight) hardware design as a preparation for the quad-copter development.

For these purposes we develop small (approx. 200×140mm) vehicle based on DFRobot's RP5 tracked platform. Two standard model motor controllers were used to drive motors. Control signals are standard PWMs used in modeling. On top of the platform we mount plastic box where BeagleBoard xM and level shifters are hidden. And finally on top of the vehicle we mount two Logitech 9000Pro USB cameras. The following pictures illustrate the whole mechanical design.

Complete vehicle: RP5 tracked platform, box with BeagleBoard xM and soldered level shifters, TP-Link WLan N stick and two Logitech 9000Pro USB camera

There is a plenty of space remains within the box. We will need this space later when we will move to PandaBoard.

More detailed view of the BeagleBoard xM inside

In the "basement": NiMh battery, two standard PWM motor controllers and power regulator module.

We are quite happy with overall design. However it could not be considered as a final yet. There are two problems:

Very unstable camera mounts. In fact we did not decide yet which cameras we will use in the future. Current 9000Pro has great optic, UVC compliant and as a result works out of the box on the BeagleBoard with Angstrom. However, it is rather large and is very inconvenient to mount. That is why we are also considering the option to use much smaller analog cameras with corresponding USB grabber.
Only two motors could be currently controlled directly from BeagleBoard. The reason is that only two hardware PWM generators are available on the expansion slot. Since our final goal is to build quad-copter, it is not an issue because we will be using I2C to communicate with motor controllers instead of PWM. However, more sensors will be necessary and not all of them are I2C devices (such as for example gyro and accelerometers). So most probably we will use the Arduino Nano micro-controller board to interface with sensors. As a side effect, it would be very easy to add more PWM channels to control additional motors and servos from Arduino for this particular vehicle.

We are currently working on these two issues and will post an update soon.

Now to software part of the project. This vehicle is currently running Angstrom with modified and extended version of our control applications. Similar to our previous vehicle we are using Angstrom Linux distribution, Gstreamer to work with video data, ZeroC Ice for communication and OpenGL for user interface. As a result, we can control the vehicle over the Internet and there is solution for firewall/NAT traversal problem and video adaptation for changing networking conditions.

As mentioned at the very beginning of this post, our goal was to provide stereo vision for vehicle drivers. We are mixing left and right frame from two cameras in one large frame, compress it using Gstreamer and corresponding DSP-accelerated h264 encoder. Then, video is transmitted using Ice middleware to the client computer where we use OpenCV computer vision library to generate anaglyph stereo image. The result of this process is
illustrated on the following picture.

Driver cockpit with anaglyph stereo video generated from two on-board cameras. Corresponding glasses (red/cyan) are required to see stereo effect.

It might be possible to generate anaglyph image directly on BeagleBoard but we decide against it and instead transmit two whole left and right frames to the client. There are several reasons for it. First, using anaglyphs to achieve stereo effects leads to the partial lost of color information and as a result rather poor image quality. The more sophysticated approaches to display stereo images such as for example polarized glasses are available. They can display both images without losses. To utilize this approach it is necessary to transmit complete left and right frames to the visualization application (and this is what we are doing). Second reason is limited computation power available on on-board computer. As a result it is not feasible to implement computation intensive algorithms (such as for example real-time depth map calculation) on on-board computer. Instead it might be beneficial to run such algorithms on more powerful driver computer or even on high performance cluster (with GPUs, SPEs, FPGA or whatever might be most appropriate). For this purposes it is again necessary to transmit complete left and right frame to the driver's computer. Moreover, some widely-used computer vision libraries such as OpenCV are highly optimized for x86 platform and pretty slow on BeagleBoard. That is why, we decide against doing image processing on on-board computer and instead perform such calculation on driver's computer.

This is our very first experiments with stereo vision and the code base is not as clean and stable as we want it to be. There are also some issues with camera calibration and positioning. That is why the stereo appearance of the image above is not as good as it could be. We are currently working very actively on it and will push the update to our git repository very soon.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

Embedded Linux and GStreamer Conference 2010

2010-11-08T15:51:00.006+01:00

In the last week of October we participated in the Embedded Linux Conference Europe 2010 (ELC-E) and presented at GStreamer Conference 2010. Both conferences took place in the Cambridge, UK and boy, atmosphere in the town is just amazing, you feel history and ideas at every corner. But when you get back to the conference, it's just one level higher. There you see the folks working on real world problems and getting them solved. What was interesting: TI OMAP processors, Beagleboard and Pandaboard were pretty hot topic on both conferences. Introduction of OMAP4 processor and related presentation on enabling GStreamer support for this processor were enlightening.

On the ELC-E we took part in the Case Study demo with our remotely controlled car and the feedback was great. We drove the car several times during couple of hours of case study and showed all inner electronics. Electronic components and plenty of wires always get attention :)

Getting ready to storm

Everyone could try steering wheel and gas pedal worked great. Unfortunately, we could not allow everyone drive the car, there were way too many people in the room and wireless connection was patchy.

During GStreamer Conference we presented our solution to the adaptive video streaming using Ice and GStreamer. Generally, adaptive video streaming was one of the major topic on the conference. Several different approaches were introduced, but most of them are in early development stage. Our answer to the problem with using TCP buffer for monitoring was the only one with working example.

Presentation of adaptive video streaming

The presentation with slides and demo videos are available in pdf or as a zip bundle with screencasts.

Generally, the conference, the atmosphere and the people were simply great, highly recommended experience.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

Support for joysticks (wheels) is implemented

2010-10-13T23:28:00.000+02:00

We just pushed the new version of our cockpit and vehicle application to our project repository at gitorious.
The main new feature in the cockpit application is support for joysticks. In fact, it also means support for steering wheels with pedals devices since they are "joysticks" from the OS point of view. We tested it with Logitech Formula Force RX wheel pictured below.

In addition, we add support for rear-view camera in the capturing and encoding pipeline which runs on the vehicle (BeagleBoard). However, we did not mechanically mount the second camera yet, that is why, by default, the rear view is generated by the videotestsrc element with "snow" pattern. But the tested working pipeline is already in repository and can be found in the car configuration file. The following picture illustrates how it is shown by the cockpit application.

Please note the rectangle with "snow" patter in the top left corner. This is the place where the video from rear-view camera will be shown as soon as we mount the second camera to the vehicle.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

New mechanical design

2010-10-13T23:05:00.000+02:00

We finally found the hardware design (mechanical and electrical) which we are going to stick with for some time. The main problem was power supply which was affected by the noise from motors. Using SparkFun's DC/DC converter breakout board and logic level converter we manage to isolate the noise from the main board. Here is how it looks like now:

As you can see on the first picture, we are not using TrainerBoard for now. Instead we decide to generate two PWM signals to control steering and acceleration directly from BeagleBoard using GPIO pins available on expansion connector. Level converter is used to convert +1.8V to +5V required for standard RC controller and servo.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

The first version of adaptive video streaming is implemented

2010-09-18T18:13:00.000+02:00

We just pushed the update to our repository at gitorious. This is our first implementation of the adaptive video streaming. To adapt to changing network conditions, we are tracking the size of output queue and adjust the frame size dynamically if the queue exceed certain size.
This functionality uses scaler element within gstreamer encoding pipeline. Scaler is located between video source element and encoding element. If we detect the Quality Of Service (QoS) condition we need to react (queue size is too large) then we pause the pipeline, adjust corresponding caps to instruct the scaler to reduce or enlarge frame size and then unpause the pipeline again. The implementation of this functionality could be found in
src/vehicle/VideoSenderThread.cpp file. In particular, VideoSenderThread::setFrameSize() function makes pipeline adjustments mentioned above. Starting at line 300 there is a piece of code which illustrates how we react on observed QoS conditions.

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

We will be talking about our project at Gstreamer conference 2010

2010-08-12T11:31:00.000+02:00

Our submission for Gstreamer conference 2010 is accepted and we will be giving a presentation at 16.30 talking about video streaming related aspects in our BeagleBoard and Gstreamer based project.

Title:
Using ICE middleware with GStreamer to implement real-time QoS-aware video streaming for remotely controlled vehicle.
Abstract:
Typically, remotely controlled vehicles are equipped with video camera to let the remote driver see surrounding environment. To enable precise control of the vehicle over the Internet, it is necessary to assure low video latency and ability to adapt to the changing network conditions. In addition, low power consumption is required to increase the battery-based operation time. In this paper we: a) outline our embedded control system based on BeagleBoard open hardware and Ångström m Linux distribution; b) provide requirements, motivation and describe how we build real-time quality of service (QoS) aware video streaming solution with ICE open-source middleware and GStreamer on this platform; c) we contrast and compare our solution with RTP/RTCP-based streaming alternatives and d) provide performance measurements for our adaptation algorithms.

In addition, we would be glad to make live demo where participants can drive our model car (approx. 30x40 cm size).

Content on this site is licensed under a Creative Commons Attribution 3.0 License .

Sources are published on Gitorious

2010-08-10T00:09:00.000+02:00

Veter project is the set of applications required to control remote vehicles over the Internet to organize races. Vehicles are equipped with on-board embedded computer (like for example BeagleBoard) connected to video camera and WLan adapter. The cockpit application let the driver see the video stream from on-board camera in real-time and control the vehicle over the Internet.

Veter is a hobby project. We are working on it since about three years. A couple of days ago we decide to open our sources and publish the first version on Gitorious: http://gitorious.org/veter/. The documentation is not complete yet but I am working on it. You can take a look on overview.pdf for more detailed project overview. Please make sure to check README if you are going to try to compile it yourself.

From the technology prospective, Veter project is based on the Internet Communication Engine (ICE), GStreamer and OpenGL. It could run on Windows and Linux. Currently, we are using BeagleBoard and TrainerBoard to control slightly modified RC car. The driver cockpit application is running on PC (Windows or Linux) and uses sophisticated visualization technique where decoded video is placed as a texture on the 3D model of the cockpit designed with Blender (the cockpit does not looks very attractive now because it was made by me :-) we are desperately need 3D artist who can help)

Motor and steering connected to the AVR-s GPIO. AVR is a slave on I2C bus from BeagleBoard.

The whole sandwich: BeagleBoard, Trainer (on the opposite site), USB HUB with camera and WLan adapter

Content on this site is licensed under a Creative Commons Attribution 3.0 License .