You got to be kidding me!

Winexe: To run code on windows machines from Linux

noreply@blogger.com (Priya Bhat) — Wed, 13 Apr 2011 19:15:00 +0000

References:
The instructions on this post are a summary of information collected from this link. And I made some changes to it as well

These are instructions of how to get winexe working on Ubuntu 10.04.1 LTS system

To begin with make sure your system has the required libraries:

sudo apt-get install build-essential autoconf checkinstall

You will also need the python development libraries and header files:

sudo aptitude install python python-all python-dev python-all-dev python-setuptools

Now get the source code of winexe using svn. And apply the first patch:

svn co http://dev.zenoss.org/svn/trunk/wmi/Samba/source cd source wget https://gist.github.com/raw/843062/5bb87c4fa13688f65ca8b1e54fc42676aee42e5a/fix_winexe_service.diff patch -p0 -i fix_winexe_service.diff

Now there is bug in the source/winexe/service.c file:
replace file

#define NT_STATUS_SERVICE_DOES_NOT_EXIST NT_STATUS(0xc0000424)

with

#define NT_STATUS_SERVICE_DOES_NOT_EXIST NT_STATUS(0x00000424)

Save,close and you are ready to compile:

cd source/ ./autogen.sh ./configure make proto bin/winexe sudo cp bin/winexe /usr/local/bin/

and now you can test that that winexe is working correctly by doing:

winexe -W WORKGROUP -U Administrator%SecretPassword //192.168.xxx.xxx 'cmd.exe'

Compiling C/C++ code from Matlab on Ubuntu Linux machine

noreply@blogger.com (Priya Bhat) — Thu, 13 Jan 2011 20:38:00 +0000

It seems that there is much easier way of compiling and running C/C++ from your Matlab application, than the one I was trying before.

Reproducing the instructions on this blog here:

My version of gcc (4.4) is too high for Matlab R2009b (4.3), and Matlab chokes on it. The solution is as follows:

Install gcc-4.3:

sudo apt-get install gcc-4.3 sudo apt-get install g++-4.3

In matlab, run mex -setup and select the GCC compiler.(Choose option 2). Next, at the command line:

sudo gedit ~/.matlab/R2009b/mexopts.sh

Replace all instances of CC=’gcc’ with CC=’gcc-4.3′, and al instances of CXX='g++' with CXX='g++-4.3'.
And restart Matlab !

Compile gcc from souce on Ubuntu 10.04.1 LTS

noreply@blogger.com (Priya Bhat) — Thu, 13 Jan 2011 19:31:00 +0000

To run C/C++ code using the mex utility of Matlab 7.11.0 (R2010b I needed to compile a different version of GCC than what was already installed on my machine. (I had the gcc version 4.4.1-4ubuntu9 and the currently supported version with mex is 4.3.4)

So I downloaded gcc-4.3 release from here.
Untarred the file: tar -jxvf gcc-4.3.4.tar.bz2

And now when I tried to use the ./configure from inside the gcc-4.3.4 folder, the system complained that it could not find two necessary API's namely GMP and MPFR.

The issue can be resolved by doing the following. We need to (1) download the GMP and MPFR libraries separately , (2) place them inside the GCC folder and then, (3) tell the configure script to look for it there.

tar -jxf gcc-4.3.2.tar.bz2 tar -jxf mpfr-2.3.2.tar.bz2 mv mpfr-2.3.2 gcc-4.3.2/mpfr tar -jxf gmp-4.2.4.tar.bz2 mv gmp-4.2.4 gcc-4.3.2/gmp cd gcc-4.3.2 ./configure --with-gmp-include=$(pwd)/gmp --with-gmp-lib=$(pwd)/gmp/.libs make make install

Hopefully, that should solve the issues !

Disable the graphics on your machine

noreply@blogger.com (Priya Bhat) — Wed, 13 Oct 2010 18:54:00 +0000

Just a quick note about how to disable the graphical user interface on your machine safely and quickly. This may be useful if you are having issues with your video card causing your system to crash. By disabling the graphics you could get command line interface to your machine.. and hopefully determine the exact problem to get your GUI to work.

mv /etc/init/gdm.conf /etc/init/gdm.disabled

This works on Ubuntu 10.04 LTS.
Remember that this would disable the graphics on all your boots. Not just the the one from which you make the change. And of course run the reverse mv command and reboot your machine to start your graphics again.

Removing entries from the known_hosts

noreply@blogger.com (Priya Bhat) — Wed, 29 Sep 2010 15:59:00 +0000

One often encounters the following error message, particularly in situations where the machine owning the target IP address has changed:

user@host-machine:~$ ssh user@target_IP @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that the RSA host key has just been changed. The fingerprint for the RSA key sent by the remote host is XX:XX:XX. Please contact your system administrator. Add correct host key in /home/user/.ssh/known_hosts to get rid of this message. Offending key in /home/user/.ssh/known_hosts:4 RSA host key for target_IP has changed and you have requested strict checking. Host key verification failed.

The issue can be resolved by using the ssh-keygen command with the -R option to remove and update the known_hosts file. Simply do:

user@host-machine:~$ ssh-keygen -R target_IP -f /home/user/.ssh/known_hosts /home/user/.ssh/known_hosts updated. Original contents retained as /home/user/.ssh/known_hosts.old user@host-machine:~$ ssh user@target_IP The authenticity of host 'target_IP (target_IP)' can't be established. RSA key fingerprint is XX:XX:XX. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'target_IP' (RSA) to the list of known hosts. user@target_IP's password: Linux XenOpen 2.6.32.21-xen #2 SMP Mon Sep 27 10:23:06 EDT 2010 x86_64 GNU/Linux Ubuntu 10.04.1 LTS

Starting new domains on Xen 4.0.1 from command line

noreply@blogger.com (Priya Bhat) — Wed, 29 Sep 2010 15:10:00 +0000

So, I have issues with my installation of Xen4.0.1 as the graphics driver keeps crashes during boot up, resulting in a hung machine. For now, I am using a work around suggested by my colleagues David Wolinsky and Pierre St. Juste, namely to disable the graphics display module (using "mv /etc/init/gdm.conf /etc/init/gdm.disabled") and hence I only have command line access to my domain-0 at this point.

At this point, Domain-0 boots up just fine, and xm and xend seem to be working. I now need to start Virtual domains using only the command line. (Later I will explore ways to do the same thing using APIs like Virt-Manager, or its console variant Virt-install)

Creating an Image

The first step is to create an image that will hold the user-domain (domU) virtual disk. This can be just a raw zero-filled file and so we can use the dd command here:

dd if=/dev/zero of=/var/lib/xen/images/domain1.img oflag=direct bs=1M seek=2047 count=1

The above command will create /var/lib/xen/images/domain1.img file of 2048MB although the actual data blocks are allocated in a lazy fashion. To reserve all the data blocks right away, get rid of the seek option from above. So do:

dd if=/dev/zero of=/var/lib/xen/images/domain1.img oflag=direct bs=1M count=2048

This will avoid data block allocation problems if the volume that holds the image is full. It is important to check that the image has the correct security context (permissions), otherwise access to the virtual disk will be denied to the user domain system. You can check this as follows:

ls -l /var/lib/xen/images/domain1.img -rw-r--r-- 1 root root 2147483648 2010-10-13 10:23 /var/lib/xen/images/domain1.img

Preparing a Xen configuration file for the installation
Xen uses a configuration file per domain. The configuration for the domains is different because we have to provide installation kernels, initial ram-disk and possibly some boot parameters. The domainU installation initrd image and kernel for my machine architecture (64-bit x86) can be downloaded from this page. You can put them in some sensible directory and name them appropriately.

root@Xen-Open:~# mkdir /home/user/x86_64_domU_images root@Xen-Open:~# cd /home/user/x86_64_domU_images root@Xen-Open:~/x86_64_domU_images# wget http://archive.ubuntu.com/ubuntu/dists/lucid/main/installer-amd64/current/images/netboot/xen/initrd.gz root@Xen-Open:~/x86_64_domU_images# wget http://archive.ubuntu.com/ubuntu/dists/lucid/main/installer-amd64/current/images/netboot/xen/vmlinuz root@Xen-Open:~/x86_64_domU_images# ls initrd.gz vmlinuz root@Xen-Open:~/x86_64_domU_images# gunzip -c initrd.gz > initrd.img root@Xen-Open:~/x86_64_domU_images# ls initrd.gz initrd.img vmlinuz

In this example, the kernel and the initrd image were named /home/user/x86_64_domU_images/vmlinuz and /home/user/x86_64_domU_images/initrd.img respectively. With the images in place, we can now create the installation configuration file named /etc/xen/domain1.cfg

kernel = "/home/user/x86_64_domU_images/vmlinuz" ramdisk = "/home/user/x86_64_domU_images/initrd.img" name = "domain1" memory = "256" disk = [ 'file://var/lib/xen/images/disk.img,xvda,w', ] vif = [ '' ] dhcp="dhcp" netmask="255.255.240.0" gateway="10.5.144.1" root="/dev/xvda ro" vcpus=1 extra="4" on_reboot = 'destroy' on_crash = 'destroy'

Couple of points to note:

The netmask and gateway values to be used can be determined by using the route command on your control domain. If you encounter problems in getting the network to start, you could try using methods suggested here.

The [ 'file://var/lib/xen/images/disk.img,xvda,w', ] line here does not work if you use "tap:aio" instead of "file". Also for some reason, if you replace "xvda" with "xvda1" or something else, it does not work either. If you do any of those things you will get the following error:
Error message: Error informing the kernel about modifications to partition /dev/xvda1p1 -- Invalid argument. This means Linux won't know about any changes you made to /dev/xvda1p1 until you reboot -- so you shouldn't mount it or use it in any way before rebooting.

Once your config file is ready, you can start creating a domain with:

sudo xm create -c /etc/xen/domain1.cfg

And now, to access your domain, do:

sudo xm domain1

This will open an interactive installation window. Everything is self explanatory in general. If you get an error to the effect that the Ubuntu repository cannot be reached, check your control domain network and here is the fix.

Once your domain is properly created it will ask permission to reboot and will automatically get destroyed in the process because of the last two lines in the configuration command line, namely:

on_reboot = 'destroy' on_crash = 'destroy'

So now you need to change your configuration file to the following:

name ="domain1" memory ="256" disk =[ 'file://var/lib/xen/images/disk.img,xvda,w', ] vif=[ '' ] dhcp="dhcp" netmask="255.255.240.0" gateway="10.5.144.1" vcpus=1 on_reboot="restart" on_crash="restart" bootloader="/usr/bin/pygrub"

And now just use the xm create command to create your new domain. Finally in order to automatically start and stop your domains when domain-0 starts, move the location of the configuration file to /etc/xen/auto.

At this point, it is a good idea to create a copy of your virtual machine image. To create new virtual machines using the same image all you have to do is to change the configuration file (name, location of the disk image etc) to point to the copy and start the new machines... waaa la !

References:

How to install Xen4.0 on Ubuntu 10.04 LTS

noreply@blogger.com (Priya Bhat) — Mon, 13 Sep 2010 15:26:00 +0000

This post provides documentation about how to get Xen4.0 (the opensouce version) up and running on an Ubuntu 10.04 LTS (Long Term Support) 64-bit machine.

Start of with a fresh new installation of Ubuntu 10.04 LTS. Install the required packages first:

apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml libx11-dev

On a 64-bit machine, you will also need this additional package:

apt-get install gcc-multilib

The next step is to get Xen4.0.1

cd /usr/src/ wget http://bits.xensource.com/oss-xen/release/4.0.1/xen-4.0.1.tar.gz

Un-tar and make:

tar xf xen-4.0.1.tar.gz cd xen-4.0.1 make xen make tools make stubdom

At this point you should have the Xen hypervisor/tools binaries in "dist/" directory ready for installation. Now run:

make install-xen make install-tools make install-stubdom

The next step is to download and compile a dom0 kernel, and to update the grub file. We can do this using the following commands:

make prep-kernels

This will create a build-linux-2.6-pvops_x86_64 directory in /usr/src/ which has by a .config file in it. You need to modify the .config file to suit your requirements. You can have a look at mine here. (Source Boris Derzhavets).
You could use make menuconfig from inside the build-linux-2.6-pvops_x86_64 to make alterations to your .config file.

cd build-linux-2.6-pvops_x86_64 make menuconfig make chmod g-s /usr/src/ -R make deb-pkg dpkg -i ../linux-image-2.6.32.21-xen_*.deb depmod 2.6.32.21-xen update-initramfs -c -k 2.6.32.21-xen

At this point enable Xend at boot:

update-rc.d xend defaults 20 21 update-rc.d xendomains defaults 21 20

When I typed in the last two lines, I got warning messages saying:

update-rc.d: warning: xend start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (3 4 5) update-rc.d: warning: xend stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 1 2 6)

Apparently these messages are benign and should not cause any issues in the working of Xen Daemon. Now create the Grub2 entry for your new kernel:

vim /etc/grub.d/40_custom

Following is what I entered to in the 40_custom file. My installation still has issues because the graphics driver keeps crashing, and I think that the grub file has something to do with it.

#!/bin/sh exec tail -n +3 $0 # This file provides an easy way to add custom menu entries. Simply type the # menu entries you want to add after this comment. Be careful not to change # the 'exec tail' line above. menuentry 'Xen 4.0.1, Ubuntu 10.4 kernel 2.6.32.21' { recordfail insmod ext2 set root='(hd0,1)' multiboot (hd0,1)/boot/xen-4.0.1.gz dummy=dummy dom0_mem=832M module (hd0,1)/boot/vmlinuz-2.6.32.21-xen dummy=dummy root=/dev/sda1 nopat ro nomodeset console=tty0 earlyprintk=xen module (hd0,1)/boot/initrd.img-2.6.32.21-xen }

Update grub and reboot... And choose the correct grub option from the boot-up menu.

update-grub reboot

Compile glxgears from source on Ubuntu 9.10

noreply@blogger.com (Priya Bhat) — Fri, 10 Sep 2010 15:53:00 +0000

I decided to modify the source code of glxgears to overcome a peculiar problem. This application would not write the FPS information to file (if redirected) unless it is shutdown.

I checked the source code, glxgears.c and found the problem was simply that there was no fflush statement after the output printf:

static void event_loop(Display *dpy, Window win) { .... if (t - t0 >= 5.0) { GLfloat seconds = t - t0; GLfloat fps = frames / seconds; printf("%d frames in %3.1f seconds = %6.3f FPS\n",frames, seconds, fps); //no flushing! t0 = t; frames = 0; }}} }

And so, the solution seemed simple: add the line of code that would force the output buffer to flush (fflush(stdout);) after printf and re-compile the source code.

To be able to compile glxgears.c, however one needs to install many other libraries and packages. Here are the steps that I followed to get the whole thing working.

glxgears is part of the Mesa Library. I downloaded th latest source code (MesaLib-7.8.2.tar.gz)

Start by untarring this file:
tar -xvf MesaLib-7.8.2.tar.gz
Enter the directory Mesa-7.8.2 and type: ./configure --with-driver=xlib. The configuration code will now start spitting out the libraries that it needs (and which your machine does not have). Following are the libraries that I needed on my machine:

libdrm:
sudo wget http://dri.freedesktop.org/libdrm/libdrm-2.4.21.tar.gz tar -xvf libdrm-2.4.21.tar.gz cd libdrm-2.4.21/ ./configure make make install
xorg-macros:
wget http://mirrors.kernel.org/ubuntu/pool/main/x/xutils-dev/xutils-dev_7.5+4_i386.deb sudo dpkg -r xutils-dev sudo dpkg -i xutils-dev_7.5+4_i386.deb ls /usr/share/pkgconfig/
dri2proto:
wget http://xorg.freedesktop.org/releases/individual/proto/dri2proto-2.3.tar.gz tar -xvf dri2proto-2.3.tar.gz cd dri2proto-2.3/ ./configure make make install
glproto:
wget http://xorg.freedesktop.org/releases/individual/proto/glproto-1.4.12.tar.gz tar -xvf glproto-1.4.12.tar.gz cd glproto-1.4.12/ ./configure make
xxf86vm:
sudo apt-get install libxxf86vm-dev
xdamage:
sudo apt-get install libxdamage-dev

After all the dependencies are satisfied, ./configure --with-driver=xlib will complete and create the make file.

Follow it up with: make. Once you modify the Mesa-7.8.2/progs/xdemos/glxgears.c file, call ./configure --with-driver=xlib and make. The new glxgears executable will be created in the same directory as the souce code.

Altering the behavior of CTRL-C

noreply@blogger.com (Priya Bhat) — Tue, 07 Sep 2010 12:22:00 +0000

Short piece of code that might come in handy. One can use the "trap" procedure to alter the behavior of Ctrl-C which is often used to interrupt execution. One can use the altered behavior to delete back-ground processes or remove temporary files that might otherwise keep lying around.. So here is the src code which you can add anywhere in your script.

# trap ctrl-c and call ctrl_c() trap 'ctrl_c' 2 function ctrl_c() { pkill temp_processes #run clean up code here ... }

Writing the output of running glxgears to file

noreply@blogger.com (Priya Bhat) — Fri, 03 Sep 2010 14:59:00 +0000

glxgears is a API that people frequently use to test if their graphical user interface works, and though it not a standardized benchmark of how good your GUI is, it does (at least, theoretically) give you some idea of the frames per second that are being rendered on your screen.

When you type the command glxgears in your terminal you should see an animation of rotating gears in the foreground and in the background, the terminal outputs every 5 or so seconds the frames per second that are being used in the rendering...

So now, my problem was simply to get the output of the terminal, that is the FPS values that are getting printed after roughly 5 seconds into a file.

The most obvious solution, that is:

glxgears >> out.txt

does not work. Apparently, glxgears does not flush the values if the output is redirected to file.

I was unable to "solve" the problem (that would require changing the source code of glxgears to make it not only print the output values but also flush it). For now I got something to work which is alright for me. I manually kill the glxgears every 5 seconds, store the output, and then restart another glxgears process.

This is not so trivial actually. If you kill the glxgears using something like

killall glxgears

no FPS values are returned or written to file. Instead you need to shutdown glxgears by telling x-server to withdraw its window resource!

Obtaining GUI access to Xen Domains Using TightVNC

noreply@blogger.com (Priya Bhat) — Thu, 02 Sep 2010 23:05:00 +0000

This is a step by step tutorial on how to install a Xen Domain on Xen 3.2 (kernel version 2.6.24-24-xen) and how to get graphical user interface working on it.

I am using a rather ancient version of the Xen kernel because I am short of time at the moment and this is something I have worked with before. But hopefully, I should be able to get the same thing done on a more recent Xen kernel soon.

So to install a new domain from the command line, do:

sudo xen-create-image --hostname=xen_7 --size=1Gb --swap=256Mb --ide --ip=10.5.155.7 --netmask=255.255.240.0 --gateway=10.5.159.255 --force --dir=/home/xen --memory=256Mb --arch=i386 --kernel=/boot/vmlinuz-2.6.24-16-xen --initrd=/boot/initrd.img-2.6.24-16-xen --install-method=debootstrap --dist=hardy --mirror=http://archive.ubuntu.com/ubuntu/

Most of the parameters are self-explanatory. I am using static IP address (10.5.155.7) on my VM. The kernel (vmlinuz-2.6.24-16-xen) and the initial ram disk (initrd.img-2.6.24-16-xen) should be present in your boot directory. Ususally the gateway parameter and the netmask parameter would be the same for your Domain-0 and user domains. To see how much memory is available for the new domain that you are creating use the df or free commands before hand.

Once the new domain has been created you will see a new configuration file in the /etc/xen/ directory. This file needs to be edited a little bit as follows:

# Configuration file for the Xen instance xen_7, created # by xen-tools 3.8 on Thu Sep 2 19:12:55 2010. # # # Kernel + memory size # kernel = '/boot/vmlinuz-2.6.24-16-xen' ramdisk = '/boot/initrd.img-2.6.24-16-xen' memory = '256' vcpu = '2' # # Disk device(s). # root = '/dev/hda2 ro' disk = [ 'tap:aio:/home/xen/domains/xen_7/swap.img,hda1,w', 'tap:aio:/home/xen/domains/xen_7/disk.img,hda2,w',

The text in red shows the text that has been changed/added. The vcpu parameter can be used if your want your domain to have more than one virtual processor. Later when your machine has booted you can check that this is indeed the case from the /proc/cpuinfo file.

So now we are all set to boot up our machine. In the Domain-0 terminal, type:

sudo xm create /etc/xen/xen_7.cfg

You could monitor the booting up and working of your new domain from Domain-0 using xentop and

sudo xm list

To access your machine, use:

sudo xm console xen_7

Login as root and set your new password using passwd command. If the network on your new domain is not functioning for some reason, look at this post for possible ways to correct the problem.

So at this point you have a functioning domain that you can access via the terminal. But now we want to get GUI access using tightVNC. To do this, a truck load of software needs to be installed. I have used the dpkg --get-selections option to generate a list of all the software that I needed on my machine. Use the dpkg --set-selection command followed by dselect (Select the "install" option on the interface opened by dselect) to install all the packages in the list.

Now the client/remote machine from which you want to access the graphical user interface of your domains needs to have VNCviewer ( sudo apt-get install vncviewer) installed.

On the host (guest domain) terminal, do:

Xvfb :0 -screen 0 800x600x16 & export DISPLAY=:0 x11vnc -display :0& xterm&

You can check that the Xvfb and xterm processes have started from the list of running processes. (ps uax)

On the client machine, do vncviewer 10.5.155.6:0 . You should see the GUI interface at this point. To test that the graphics are working, try:

glxgears

Well, that's it! You should see an animation of a bunch of gears rotating on your screen.... Which means that you have a GUI access to your machine!

Issuing commands to VLC and collecting output to file

noreply@blogger.com (Priya Bhat) — Wed, 01 Sep 2010 13:39:00 +0000

So, I wanted to play a video using VLC Media Player and collect statistics like lost frames and bit rate into a file every x seconds while the video is running. Seems pretty straight forward considering that the stats can be viewed from the VLC GUI. So it should be a straight forward matter to get them into a file, right? Wrong !!

I tried several approaches including using the rc interface. Pierre gave me a piece of code that solves the problem on his machine:

while true; do echo stats; sleep 2; done | vlc -I rc filename > out.txt

Here, the filename is the name of the file that needs to be played and the number following sleep is the time between two reads of the statistics. As you can see, all this piece of code does is to issue (using pipe |) stats command to the rc interface (denoted by the -I option) every 2 seconds and store the output into out.txt file. For some reason the same code did not work on my machine and I kept getting the error message:

user@machine:~$ while true; do echo stats; sleep 2; done | vlc -I rc 350animation2.mpg > out.txt VLC media player 1.0.2 Goldeneye [0x8731750] main interface error: no suitable interface module [0x8685140] main libvlc error: interface "default" initialization failed

I am using VLC media player 1.0.2 and my OS is Ubuntu Karmic Koala.If anyone has a clue as to what could be causing the problem, please send me a comment. Between, I was able to redirect the output stats to a file using the rc interface. For that I started VLC using vlc --intf rc filename >output.txt and then in the interface that opens, I issued the stats command. What I could not do was to automate the process of issuing the stats command every x seconds.

While trying to resolve the issue, I stumbled upon this "Not a blog" where the author shows how to issue commands to VLC from the command line using netcat and UNIX sockets. I tried to extrapolate from the code and tweak it to channel the collection of stats but it did not work. My super naive attempts basically were along the lines of writing a bash script that starts the video and then runs in an infinite loop trying to collect stats and sleep, as shown below:

echo play 350animation2.mpg while true doecho stats sleep 2 done

Then I would redirect the input and output to the VLC interface. But the same error message persisted. David Wolinsky came to the rescue. Here is the script that he come up with:

#!/bin/bash set +o nounset LC_ALL=C ; LANG=C ; export LC_ALL LANG version >/dev/null 2>&1 && version "=o" $(_eat $0 $1) set -o nounset set -m run() { pid=$1 sleep 4 while true; do echo -n "stats" | nc -U vlc.sock if [[ ! $(ps uax | grep -v grep | grep $pid) ]]; then return fi sleep 2 done } vlc -I rc --rc-unix vlc.sock 350animation2.mpg &> out & pid=$! run $pid & fg %1

As you can see, this code uses the UNIX socket approach as well. After the VLC process is started, the PID is stored and used to start the run script on the top. The output gets stored in the out file. The fg (fore ground) command in the end is apparently necessary.

Thanks to Pierre, David and Andrabr for help with this problem!

How to get Virtual Manager to work for a Xen host

noreply@blogger.com (Priya Bhat) — Thu, 22 Jul 2010 17:43:00 +0000

So, continuing with the theme of Xen, Virtual Machine Manager or virt-manager for short, is a piece of software that I decided to check out because I wanted graphical access to my virtual machines. A more formal description as given on their website is:

"Virt-manager is a desktop user interface for managing virtual machines. It presents a summary view of running domains, their live performance & resource utilization statistics. The detailed view graphs performance & utilization over time. Wizards enable the creation of new domains, and configuration & adjustment of a domain's resource allocation & virtual hardware. An embedded VNC client viewer presents a full graphical console to the guest domain"

You need to download virt-manager on the machine from which you wish to access your virtual machines. I am using a machine running Ubuntu 10.04 which I will call the client machine in the description below.

My Virtual Machines (VMs) were created on top of Xen 3.2 hypervisor, Earlier I had some issues getting the network up and running on my VMs, the resolution of which I have described in an earlier post. My hypervisor machine which I will call my host machine is running kernel 2.6.24-24-xen.

So here is the procedure that I followed to get virt-manager to work:

I used the Synaptic Package Manager (System->Administration->Synaptic Package Manager to download virt-manager and all the dependencies on my client machine. Alternatively, one could also do sudo apt-get install virt-manager

On my host machine, I downloaded and installed libvirt-bin and ssh-askpass
sudo apt-get install libvirt-bin sudo apt-get install ssh-askpass

Now to access the virt-manager console from my client machine I used Application->System Tools->Virtual Machine Manager

On my host machine, I started the libvirtd daemon:
sudo libvirtd start

At this point I tried to connect to my host machine using File->Add Connection...; I chose my specific hypervisor (Xen) and Connection as Remote Tunnel over SSH (I am not sure that this is the one that I was supposed to use, but it did work, eventually!)
At this point, I got the following error on a pop-up on my client machine and on the console of my host machine:
libvir: Xen Daemon error :internal error failed to connect to xend
After some effort, the following fix worked:
1. Open the xend configuration file
  sudo vim /etc/xen/xend-config.sxp
2. In my previous post I had talked about adding a temporary bridge between physical network device and the virtual TAP device. I had to let the xend daemon become aware of this. So under the line:
  (network-script network-bridge)
  I added:
  (network-script 'network-bridge netdev=tmpbridge')
  (To recall what my network device is called, I did a ifconfig. It was the one whose description contains the domain-0's IP.
3. I was not done with this file yet. I needed to add the following line somewhere in the file:
  (xend-unix-server yes)
4. I saved and closed the file. Restarted the host machine's network (sudo /etc/init.d/networking restart), restarted the xend daemon (sudo xend restart), and finally restarted the libvirtd daemon( sudo libvirtd start)
Now when I tried to add the connection using the setting mentioned above, it went through, asked me for a password and I would see a table giving the details of the CPU utilization and some other statistics of my VMs. Following is a screen shot of what I can see so far:

No network on Xen user domain

noreply@blogger.com (Priya Bhat) — Thu, 22 Jul 2010 14:11:00 +0000

Today I installed Xen 3.2 for my work and created a guest VM using the this link as a guide. However I found that the network on my VM didn't work. The problem as Pierre explained was that the link between my physical network device and the virtual TAP device was not set up.

This was in spite of the fact that I re-started the xend daemon on my machine and used the static IP address method that was recommended in the blog mentioned above.So it seems that the xm create and xend scripts are not doing what they are supposed to do!

David Wolinsky came up with the following fix:
On the console of your control domain, do the following:

$ brctl addif tmpbridge eth0 $ ifup tmpbridge $ ifconfig eth0 0.0.0.0 promisc $ brctl show $ dhclient tmpbridge

Now when you do an ifconfig on you domain-0, the tmpbridge interface should show up with the IP that was originally assigned to eth0.

And since eth0 is set to "promiscuous" mode, it will accept all network packages coming to it.

Now using the route command find the default gateway used by your domain-0. The following terminal snapshot is provided to clarify which IP I am talking about:

user@Domain0:~$ route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.5.144.0 * 255.255.240.0 U 0 0 0 tmpbridge default 10.5.144.1 0.0.0.0 UG 0 0 0 tmpbridge

In the last line, the default gateway is highlighted in red. Make a note of this IP.
Now open the console of your user domain and check the route:

root@UserDomain:~# route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.5.144.0 * 255.255.240.0 U 0 0 0 eth0

As you can see, in my case the default gateway was not set. To do so use the route add default gw command and the IP we obtained from the default gateway of domain-0.

root@UserDomain:~# route add default gw 10.5.144.1

Verify using route, and then try pinning something:

root@UserDomain:~# route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.5.144.0 * 255.255.240.0 U 0 0 0 eth0 default 10.5.144.1 0.0.0.0 UG 0 0 0 eth0 root@UserDomain:~# ping google.com PING google.com (72.14.253.104) 56(84) bytes of data. 64 bytes from mia04s03-in-f104.1e100.net (72.14.253.104): icmp_seq=1 ttl=56 time=15.5 ms 64 bytes from mia04s03-in-f104.1e100.net (72.14.253.104): icmp_seq=2 ttl=56 time=14.0 ms 64 bytes from mia04s03-in-f104.1e100.net (72.14.253.104): icmp_seq=3 ttl=56 time=13.9 ms

Dadaaa! Many thanks to Pierre and David!
As a side note, if you are adding more domains, you do not need to repeat all the commands on domain-0, as its interface is already in the "promiscuous" mode. Only run the dhclient again. You will however need to set up the default gateway in every new domain that you create.

Disable Sudo password prompts on Ubuntu

noreply@blogger.com (Priya Bhat) — Tue, 25 May 2010 14:57:00 +0000

A simple trick to disable the sudo password prompts on Ubuntu (I have tested this on Ubuntu 9.10 Karmic Koala). A word of caution, be aware that by doing this you are sacrificing the security that sudo provides.

You basically need to edit the /etc/sudoers file. This file cannot be changed by any user. You need to sign in as root and then use the visudo command to start editing the file.

user@machine:~$ sudo su
[sudo] password for user: 
root@machine:/home/user# visudo

In this file, right at the bottom you will find a line:

%admin ALL=(ALL) ALL

Replace it with:

%admin ALL=NOPASSWD: ALL

This will disable the requirement to enter the password every time you run a sudo command. Save and close (ESC, :wq, ENTER)

So you can now run a command like sudo reboot without having to enter the password:

user@machine:~$ sudo reboot

Broadcast message from user@machine
 (/dev/pts/0) at 11:17 ...

The system is going down for reboot NOW!

Again, be careful and know what you are doing. Also, it might be a good idea to make a copy of your /etc/sudoers file before your make any changes to it.

One way to use this trick could be in a scenario where you want to run a sudo command through a script and do not want to use the -S option of sudo (-S option allows you to provide the password from the script itself; this can be dangerous if someone gets access to your script)

Bash chit-sheet

noreply@blogger.com (Priya Bhat) — Thu, 20 May 2010 15:17:00 +0000

Some simple tricks that can make bash scripting a breeze:

To truncate a float to int:
INT=${FLOAT/\.*}

To split a string (of 2 words) into its components:
FIRST_WORD=${STRING%% *},SECOND_WORD=${STRING#* }

To do simple floating point arithmetic, use bc:
AVERAGE=$(echo "scale=2; ($FLOAT1 + $FLOAT2 ) / 2" | bc )

Conditional if statement:
if [[ $INT1 -gt $INT2 ]]; then echo greater; fi

To check the number of arguments for a bash script and to issue an error message if the number of arguments if insufficient, use the following code snippet at the top of the script:
EXPECTED_ARGS=2 E_BADARGS=65 if [ $# -ne $EXPECTED_ARGS ]; then echo "Usage: ./script_name arg1 arg2" echo " Example: ./sum 1 2" exit $E_BADARGS fi

To find the process ID of a running process, use pgrep:
pgrep -fl PROCESS_NAME

To kill a process using the process name instead of the process ID, use:
kill $(ps -ef |grep PROCESS_NAME | grep -v grep | awk '{print $2}')

Find files that contain a text string:
grep -lir "text to find" *
The -l switch outputs only the names of files in which the text occurs (instead of each line containing the text), the -i switch ignores the case, and the -r descends into subdirectories.

Clock source and NTP control

noreply@blogger.com (Priya Bhat) — Wed, 19 May 2010 13:44:00 +0000

To view the available clocksource options for your machine, look into the /sys/devices/system/clocksource/clocksource0 directory. This directory also shows the current clock source that your machine is using. You will need sudo privileges to access this information:

user@machine:/sys/devices/system/clocksource/clocksource0$ sudo more available_clocksource
tsc hpet acpi_pm pit jiffies
user@machine:/sys/devices/system/clocksource/clocksource0$ sudo more current_clocksource
hpet

Usually, your machine will pick the "best" clock source from the available options at start up, but if you want to force the selection of a particular clock source, you can do this by including the desired option in the configuration file that the system uses at the time of start-up, namely /boot/grub/menu.lst

user@machine:/boot/grub$ sudo vim menu.lst

Add the clock source option at the end of the line declaring the kernel path, as shown below:

title           Ubuntu 8.04.3 LTS, kernel 2.6.24-26-generic
root            (hd0,0)
kernel          /boot/vmlinuz-2.6.24-26-generic root=UUID=cea24a64-f038-469c-b716-226ab0da2f93 ro quiet splash clocksource=tsc
initrd          /boot/initrd.img-2.6.24-26-generic
quiet

Save and restart your machine. The desired clock source should show up in the /sys/devices/system/clocksource/clocksource0/current_clocksource file.

To start/stop or restart the ntp demon on your machine from the command line, do:

user@machine$ sudo /etc/init.d/ntp stop
[sudo] password for user: 
 * Stopping NTP server ntpd                                                                                                                           [ OK ]

Add commands to bash

noreply@blogger.com (Priya Bhat) — Thu, 06 May 2010 17:50:00 +0000

This post is about adding user defined commands to bash so that you can run applications only using the command rather than specifying the whole path of the application executable.

The code provided here has been tested on Ubuntu 9.10 (Karmic Koala).

There are two ways to accomplish the mission, the first is to put your application is the default path that bash already looks into while trying to execute your command, and the second is to add another location as a path that bash must check.

For the first option, run $PATH in the terminal. Your terminal output would look something like

bash: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

If you put your scripts/executables in any of the above locations, bash will find it and execute it without you having to specify the whole path.

For the second option, you need to edit the ~/.bashrc file and add the location of the executable to the PATH locations. Suppose this location is /home/pbhat/bin/. Concatenate the following line to the end of ~/.bashrc file.

PATH=$PATH:/home/pbhat/bin

export PATH

Save and close the file and now run the following command in the terminal:

source ~.bashrc

Unless you run this command the updated path will not show up.

Inflate multiple files with one command

noreply@blogger.com (Priya Bhat) — Thu, 06 May 2010 16:01:00 +0000

If you have many multiple archives which have been compressed using different encryptions (tar, tar.gz or zip) in one directory, you can use the script provided in this post to inflate them into the directories with the same name as the original archive, that is:

file1.zip will get inflated to file1/

file2.tar will get inflated to file2/

file3.tar.gz will get inflated to file3/

So here is the script:

#!/bin/bash

cd $1

for i in $(ls)
do 
 dir_name=$(echo $i | sed "s/\([^.]*\)[.].*/\1/")

 if [[ "$i" =~ "zip" ]]; then
  mkdir -p $dir_name 
  cd $dir_name; 
  unzip $i
  cd ../
 elif [[ "$i" =~ "tar.gz" ]]; then
  mkdir -p $dir_name
  cd $dir_name
  tar -xzvf ../$i
  cd ../
 elif [[ "$i" =~ "tar" ]]; then
  mkdir -p $dir_name
  cd $dir_name
  tar -xvf ../$i
  cd ../
 fi 
done

The argument to this code is the full path of the directory that contains the archived files. Notice that the script searches for the strings, "tar", "zip" and "tar.gz" in the filenames. If you have other files in the directory which contain these strings in their names, they can get overwritten!

You can save the script using a name of your choice (I have named it open.sh, as indicated in the terminal output shown below)

terminal$ ls directory_path/
file1.tar  file2.zip  file3.tar.gz
terminal$ ./open.sh directory_path/
terminal$ ls directory_path/
file1  file1.tar  file2  file2.zip  file3  file3.tar.gz

I would like to thank Girish Venkatasubramanian for advice on this script.

Setting up automatic back-ups

noreply@blogger.com (Priya Bhat) — Thu, 06 May 2010 14:42:00 +0000

To set up automatic back-ups from your machine to a remoteserver use rsync and cron/anacron commands that now come pre-installed on your Linux machine.

Here is the reference from the Linux gazette that I used to set up my backups. Since my machine is not necessarily turned on 24 hours a day, I used anacron in stead of cron to schedule my backups.

To set-up a new job for anacron, edit the /etc/anacrontab file

1 20 backup rsync -r -e ssh --delete /home/username/thesis username@remoteserver:backups/thesis/thesis/

For the above command to run you should have the public/private key for password-less login to the remoteserver set-up before hand.

On the remoteserver, you must set up cron-jobs to store timely snapshots of the backups. This would help if your current versions are corrupted and you would like to roll back to a previous version.

Here is the code that must be added to your cron file (edit it using crontab -e ) to do that (root access not required):


# Back up mail files with snapshots of 6, 4, 3, 2, 1 months and 3, 2, 1 weeks
# Order 4m->6m, 3m->4m, 2m->3m, 1m->2m, 3w->1m, 2w->3w, 1w->2w, mirror->1w

# At 3am on the 1st of Jan,Mar,May,Jul,Sep,Nov copy the 4m to the 6m
00 03 1 1,3,5,7,9,11 * cp -f /backups/thesis/backup/4month.tar.gz /backups/thesis/backup/6month.tar.gz

# At 3.02am on the 1st of every month move the 3m to the 4m (and continue for other months)
02 03 1 * * cp -f /backups/thesis/backup/3month.tar.gz /backups/thesis/backup/4month.tar.gz
04 03 1 * * cp -f /backups/thesis/backup/2month.tar.gz /backups/thesis/backup/3month.tar.gz
06 03 1 * * cp -f /backups/thesis/backup/1month.tar.gz /backups/thesis/backup/2month.tar.gz
08 03 1 * * cp -f /backups/thesis/backup/3week.tar.gz  /backups/thesis/backup/1month.tar.gz

# And then every Sunday take care of the weekly snapshots and the archiving of the mirror
10 03 * * 0 cp -f /backups/thesis/backup/2week.tar.gz  /backups/thesis/backup/3week.tar.gz
12 03 * * 0 cp -f /backups/thesis/backup/1week.tar.gz  /backups/thesis/backup/2week.tar.gz
14 03 * * 0 rm -f /backups/thesis/backup/1week.tar.gz
16 03 * * 0 tar zcf /backups/thesis/backup/1week.tar.gz /backups/thesis/thesis/*

Translating from Names to IPs

noreply@blogger.com (Priya Bhat) — Wed, 21 Apr 2010 13:07:00 +0000

To store the translation from a machine name to its IP go to /etc/hosts under Ubuntu Linux. You would need to sudo to change the file. Add stuff like:

192.X.X.X Xen1

Doing this will make it easier to ssh, sftp, etc. as you don't need to remember the IP addresses of the machines and more easily memorizable mnemonics can be used instead

For example now you could do:

ssh xen1

and your machine would know where you want to go !

The tale of three Operating Systems

noreply@blogger.com (Priya Bhat) — Fri, 06 Feb 2009 18:09:00 +0000

I have no qualms in admitting that I am pretty much a one-OS person. Having grown up with Windows, I have struggled, tripped, crawled and somehow managed to survive in this hopelessly zesty world of research where Linux has come to become a religion...

You can just say that for a person who is so less inclined to find "discovering new commands" as a pleasure giving exercise, I have chosen the wrong profession...

But here I am... with a (primitive) 30 GB hard disk machine which can boot into 3 Operating systems namely Windows, Ubuntu and Xen, (I need all three to compare performance).... cyphering through the long list of configuration files.... and wondering if the solution to my whole research problem isn't simply for the whole world to accept one single OS that is just good enough...

How much simpler would that be...

So what am I PhDing in?

noreply@blogger.com (Priya Bhat) — Thu, 21 Aug 2008 19:53:00 +0000

Ahhh Academia! Here I am again.... Almost 15 months after leaving Robot Vision Laboratory, Purdue University and 2 months after leaving Read-Ink Technologies Pvt. Limited, Bangalore here I am at Advanced Computing and Informations Systems Laboratory, University of Florida!

And, btw my welcome-party here includes a surprise performance by Fay!

My friend Mridul once narrated a conversation that he had with his adviser regarding when is it too late for a person to think about doing a PhD. The conversation boiled down to this: "if you can name even three areas/topics/questions that intrigue you and which you want to grasp, you qualify!"

Hmmm... on the question of inquisitiveness, 'll, I ain't sure 'bout how hungry for knowledge I'm.... but 'sure 'm Mal-knowledge-ed!!

So to come to the question raised in the topic of this post and which so many people have asked me, the answer is "I don't know yet"...

Its a whole new universe out here and for me the journey has just begun!

Last Day at Read-Ink …

noreply@blogger.com (Priya Bhat) — Wed, 18 Jun 2008 11:03:00 +0000

Exactly a year ago, on this very day I walked into a white, three-storied building in Indiranagar, Bangalore-the office to a start-up called Read-Ink Technologies Private Limited. That was the beginning of my first job…. And today, I am bidding farewell to this place and all the wonderful people I met here.

A lot changed over the last year; personally speaking, I got married, had a laptop blast in my house, realized a new local maximum of my ignorance, got banned, reached a new high in fitness, traveled, trekked, met some amazing people, made a few friends for life and lost a few dear ones to death…. To sum it up, it has been an eventful year!

But as I leave today I am not thinking about all the events that transpired over the last year or about the eventualities that I might have to face in the days to come… instead, my brain is working like a camera capturing humdrum moments of a ‘normal’ day here… things that will remain the same tomorrow as they were before I came…

Maybe at some time in the future I will chalk out the balance sheets for how ‘useful’ this last year has been. But as of today, such algebra is beyond me….

In my life’s book another chapter is closing.... tomorrow there will be no office, no bantering over coffee, no peeping over the cubicle, no meetings, no walks, no ‘goodh foodh’….and no second lunches…

Some how, the knowledge that we are going to leave soon does not stop us from making ourselves at home…. And that applies to life just the same.

Tomorrow will be a new chapter… beginning of new journeys…

But today, I feel like the narrator just gave the verdict… “Priya is dead! Priya is Mafia!!”

Damn! I have to sit out for the rest of the game… AGAIN!

MATLAB vs. C++? And the winner is….

noreply@blogger.com (Priya Bhat) — Wed, 04 Jun 2008 11:03:00 +0000

….C++, of course! And, for the simple reason that MATLAB IS EVIL!

To all you pseudo-programmers, Simulink-flaunting ‘engineers’ and the like who swear by the ‘inbuilt functionality’ and ‘user-friendliness’ of MATLAB, I would just like to say… people, get a life! Its not that you don’t KNOW how stupid this application is- after all, you are the ones who sit there wiggling your toes while MATLAB takes minutes to run something C++ would do in micro-seconds; and you are the ones who get stymied, trying to make MATLAB understand even most simple Data Structures - then why not accept it? Why not, for once and for all, agree to the fact that C++ is superior and stop going ya-ya-ya about the ‘cool’ things about MATLAB!

People often tell me that MATLAB works better for engineers because of its inbuilt libraries and as a proof (by demonstration) they may type ‘14382^11.32’ on their command windows and lo-and-behold MATLAB tells you the answer! And then their argument may go something to the effect that to do the same thing with C++, you would need to write a function, maybe include a couple of libraries, debug, compile and run … too much headache for something MATLAB does in ONE line! Well, that is a fair argument…. only if you think that that is engineering… namely calculating random numbers to the power of random numbers…!

Honestly, that is anything but engineering! Engineering is about conceptualizing and it takes power to design elegant structures to translate those concepts to code… A good data structure is the difference between an understandable, well-organized, effectual program and useless, exasperating gobbledygook. But that is something MATLAB just does not understand. The ‘arrays’, ‘cell arrays’ and ‘structure’ that it provides address the problem of stucturing data with the same success rate as Bush is addressing the issue of ending the Iraq War!

Engineering also needs checks and while I admit that C/C++ has its own shortcomings when it comes to security issues (memory leaks, garbage collection problems, etc.), MATLAB is just outright scary. For example, there are no type-checks as a variable can be redefined from an integer to a character, to an array, to a structure within the span of a single function! Further, MATLAB passes parameters only by value… which is totally sinful if you want to pass large arrays and structures… just think of how much time and space you are consuming!

With C++ and a little more patience you can write code that is not only more efficient, it also better structured, more-understandable at any later date and therefore more reusable… I guess that’s why the latest (2008a) version of MATLAB had introduced object-oriented concepts and parameter passing by reference… But that’s something I am yet to get my hands dirty with….