Chathura Sarathchandra

Openstack: OpenvSwitch Kernel panic

noreply@blogger.com (Chathura Sarathchandra) — Thu, 22 May 2014 14:39:00 +0100

When starting openvswtich-swtich or quantum-plugin-openvswitch-agent with enable_tunneling = True the network node kernel panics.

The kernel version linux 3.5.0-41-generic
Openvswitch version 1.4.6

Solution

Upgrade to Openvswitch version to 1.10.2 from the Havana repositories.

add-apt-repository -y cloud-archive:Havana && apt-get update && apt-get install openvswitch-switch

Enabling Virtio drivers in kernel for running androidx86 on Openstack

noreply@blogger.com (Chathura Sarathchandra) — Tue, 1 Apr 2014 23:06:00 +0100

Guest operating systems running on virtualised systems needs to cooperate with the underlying hypervisors when using virtualised resources. Virtio is a set of standard for disk and network virtualisation that is required to be installed on instances that runs on Openstack. The default Androidx86 kernel does not come with these modules installed. Also, you have to edit the source code of Androidx86 OS to detect the virtualised block devices. Otherwise you will see a screen with "Detecting Android-x86... (continuous dots :s)".

This is what you will see in androidx86 debug mode:

If you do not want to compile the source and set it up yourself. I have already created the image for you. Download it from here.

1. First, to compile the OS you have to initialise the build environment. Follow instructions here.

2. Download and install the repo client using instructions here.

3. Download Androidx86 source from here.

4. Alter the kernel defconfig files in "kernel/arch/x86/configs/android-x86_defconfig"

and "kernel/arch/x86/configs/android-x86_64_defconfig" by adding following lines.

CONFIG_VIRT_DRIVERS=Y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_MMIO=m
CONFIG_VIRTIO_BALLOON=m
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=m
CONFIG_VIRTIO_RING=m
CONFIG_VIRTIO_CONSOLE=m
CONFIG_HW_RANDOM_VIRTIO=m

5. To be able to detect virtual block devices, alter the "bootable/newinstaller/initrd/init" file.

In line 124, Change

for device in ${ROOT:-/dev/sr* /dev/[hs]d[a-z]* /dev/mmcblk*}; do

for device in ${ROOT:-/dev/sr* /dev/[hsv]d[a-z]* /dev/mmcblk*}; do

6. Insert following lines to device/generic/x86/init.sh

#Force dhcp on eth0 interface.
netcfg eth0 dhcp
#Start SSH daemon at startup.
start-ssh

7. Add follwing packages to PRODUCT_PACKAGES in device/generic/x86/packages.mk.

ssh-keygen
sshd_config
start-ssh

8. Then compile.

9. Install androidx86 OS onto a disk storage (dvi, vmdk, qcow2) .

10. To fetch the nova keypair, create fetchsshkeys script with following lines in /data/local/.

#!/system/bin/sh

# Fetch public key using HTTP
cd /data
wget http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key 
cat /data/openssh-key > /data/ssh/authorized_keys

chmod 0600 /data/ssh/authorized_keys
restorecon /data/ssh/authorized_keys

rm /data/openssh-key

11. Make the script executable.

chmod 755 /data/local/fetchsshkeys

12. Call the script in /etc/init.sh file.

#Fetch ssh keys from openstack metadata service.
/data/local/fetchsshkeys

13. Upload the disk to Glance.

It should now detect /dev/vda1.

14. Create a virtual machine using the uploaded image. Your VM should get a private ip. (Make sure to specify a keypair)

15. You should be able to ssh into the androidx86 VM now.

16. To get get GUI, add "nomodeset" to grub.

Thanks to everyone helped me on this issue from Androidx86 mailing list.

Openstack Grizzly slow API and timeouts

noreply@blogger.com (Chathura Sarathchandra) — Wed, 12 Mar 2014 11:40:00 GMT

I have come across this issue a while ago on our Openstack testbed that as the amount of API calls increase, the delay when you do API calls increased. Then I discovered that Openstack does not clean the expired tokens automatically when you are using a SQL backend.

https://bugs.launchpad.net/ubuntu/+source/keystone/+bug/1032633

To delete the expired tokens I have created a script so that I can automate this, by creating a cron job.

You can find the script here on my github. Download the script and change the password to your mysql password.

Then create a cron job to periodically execute this script.

0 0 * * * /path/to/keystone_flush 2>&1 >> /path/to/log/keystone_flush.log

Above cron job will run ones everyday. You may change the timing of the job as you desire.

However this problem is solved in Openstack Havana release.

https://blueprints.launchpad.net/keystone/+spec/keystone-manage-token-flush

Openstack Folsom cant ping instance/VM

noreply@blogger.com (Chathura Sarathchandra) — Wed, 29 Jan 2014 14:02:00 GMT

The most annoying bug that I came across when setting up Openstack Folsom was that, after creating the a virtual machine it was not able to reach the metadata service and I was not able to reach the virtual machine.

After days of debugging, I found out that it was not only me that was having the same issue. Finally it was reported as a known bug to the openstack developers.

https://bugs.launchpad.net/neutron/+bug/1091605

The problem seems to be that OpenvSwitch devices have been brought back up after after rebooting. Hence before I upgraded to Grizzily from Folsom, I created the following script and had it running at the boot-up in the network node.

 #!/bin/sh  
   
 #  
 # This script fixes the following bug.  
 # Has to run after reboot.  
 # Bug: https://bugs.launchpad.net/quantum/+bug/1091605  
 #  
 #  
 # Chathura S. Magurawalage  
   
 ((  
 #Check for root permissions  
 # Make sure only root can run our script  
 if [ "$(id -u)" != "0" ]; then  
   echo "This script must be run as root"  
   exit 1  
 fi  
   
 BRIDGE=${BRIDGE:-br0}  
   
 while getopts b:hv option  
 do   
   case "${option}"  
   in  
     b) BRIDGE=${OPTARG};;  
      v) set -x;;  
     h) cat <<EOF   
 Usage: $0 [-b bridge_name]  
   
 Add -v for verbose mode, -h to display this message.  
 EOF  
 exit 0  
 ;;  
      \?) echo "Use -h for help"  
        exit 1;;  
   esac  
 done  
   
 ovs-vsctl del-br br-int  
 ovs-vsctl del-br br-ex  
 ovs-vsctl add-br br-int  
 ovs-vsctl add-br br-ex  
 ovs-vsctl add-port br-ex $BRIDGE  
 ip link set up br-ex  
   
 service quantum-plugin-openvswitch-agent restart  
 service quantum-dhcp-agent restart  
 service quantum-l3-agent restart  
   
 )) 2>&1 | tee $0.log

Ubuntu Kernel panic when creating Openstack instance

noreply@blogger.com (Chathura Sarathchandra) — Wed, 29 Jan 2014 13:49:00 GMT

Currently I am using Openstack Folsom, nova version 2013.1.4, Installed on Ubuntu 12.04. I upgraded my compute and network nodes's kernel to "Linux 3.2.0-58-generic". Since then, when I try to create an instance on a the compute nodes or when I restart the network node, the systems goes to Kernel panic.

The only fix for this issue I found, is to downgrade the kernel from "Linux 3.2.0-58-generic" to "Linux 3.2.0-55-generic" on all compute nodes and network node to keep things consistent. The controller nodes does not seem to be affected by this kernel bug.

Also make sure to install the header if the corresponding kernel, if you are installing the kernel. Or if you have already got "Linux 3.2.0-55-generic" kernel installed, then just change it to the default kernel in grub records.

First find out the index of the kernel in the grub records.

grep menuentry /boot/grub2/grub.cfg

Then the fist answer of the following question should help when editing the grub config.

http://askubuntu.com/questions/216398/set-older-kernel-as-default-grub-entry

Openstack Grizzily Error "FATAL: Module openvswitch_mod not found" when start openvswitch or at startup

noreply@blogger.com (Chathura Sarathchandra) — Wed, 29 Jan 2014 13:26:00 GMT

I came across this error on Openstack Grizzily installed on Ubuntu 12.04 LTS, when I upgrade or degrade the linux kernel. So its very likely that you will get this error after running a "dist-upgrade" on your system.

The cause of this problem seem to be that when a different kernel is used, the openvswitch modules have not been built with it.

Error:
FATAL: Module openvswitch_mod not found.
* Inserting openvswitch module
* not removing bridge module because bridges exist (virbr0)
invoke-rc.d: initscript openvswitch-switch, action "load-kmod" failed.

Solution:

Copy the following script to a file. Then save.

 #!/bin/bash  
   
 sudo apt-get install linux-headers-$(uname -r)  
   
 depmod  
 modprobe openvswitch  
   
 sudo apt-get remove openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent  
   
 sudo apt-get install openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent

Then make it make the file executable.

Go to the file directory then execute following command in terminal:

sudo chmod +x filename

Finally execute the file.

./filename

Openstack Folsom Error "FATAL: Module openvswitch_mod not found" when start openvswitch or at startup

noreply@blogger.com (Chathura Sarathchandra) — Wed, 29 Jan 2014 13:17:00 GMT

I came across this error on Openstack Folsom installed on Ubuntu 12.04 LTS, when I upgrade or degrade the linux kernel. So its very likely that you will get this error after running a "dist-upgrade" on your system.

The cause of this problem seem to be that when a different kernel is used, the openvswitch modules have not been built with it.

Error:

FATAL: Module openvswitch_mod not found.
* Inserting openvswitch module
* not removing bridge module because bridges exist (virbr0)
invoke-rc.d: initscript openvswitch-switch, action "load-kmod" failed.

Solution:

Copy the following script to a file. Then save.

 #!/bin/bash  
   
 sudo apt-get install linux-headers-$(uname -r)  
   
 sudo apt-get remove quantum-plugin-openvswitch openvswitch-switch quantum-plugin-openvswitch-agent openvswitch-datapath-dkms openvswitch-common quantum-common python-quantum  
   
 apt-get install openvswitch-switch  
 mkdir -p /etc/quantum/  
 apt-get install quantum-plugin-openvswitch-agent quantum-dhcp-agent quantum-l3-agent quantum-server  
   
 depmod  
 modprobe openvswitch

Then make it make the file executable.

Go to the file directory then execute following command in terminal:

sudo chmod +x filename

Finally execute the file.

./filename

No sound on Ubuntu 12.04 after upgrading from 11.10

noreply@blogger.com (Chathura Sarathchandra) — Sat, 23 Jun 2012 15:54:00 +0100

I have recently upgraded my Ubuntu 11.10 distribution to Ubuntu 12.04 LTS on my HP pavillion dv6 laptop.

This is the output from 'aplay -l' command of my system.

card 0: Intel [HDA Intel], device 0: STAC92xx Analog [STAC92xx Analog]

  Subdevices: 0/1

  Subdevice #0: subdevice #0

card 0: Intel [HDA Intel], device 1: STAC92xx Digital [STAC92xx Digital]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

card 2: HDMI [HDA ATI HDMI], device 3: HDMI 0 [HDMI 0]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

After rebooting I noticed that I had no sound. After trying many things, to resolve this problem I had to add the following two lines to the "alsa-base.conf" file.

alias snd-card-0 snd-hda-intel

options snd-hda-intel model=auto

If this does not work then replace "auto" with the following keywords one at a time. Save the file and restart your computer each time you test with one configuration.

ref                    - Reference board

mic-ref            -Reference board with power management for ports

dell-s14           -Dell laptop

hp                     -HP laptops with (inverted) mute-LED

hp-dv7-4000  -HP dv-7 4000

auto                  - BIOS setup (default)

Reference :- ubuntu 12.04 no sound vostro 3300 i5
Chathura

Lunux/Ubuntu not booting up after the kernel update?

noreply@blogger.com (Chathura Sarathchandra) — Sat, 21 Apr 2012 14:07:00 +0100

After I have upgraded to 2.6.38-9-generic kernel I got the following message and I have tried many thing but this is how I have solved the problem.

Gave up waiting for root device. Common problems:
-Boot args (cat /proc/cmdline)
-Check rootdelay= (did the system wait long enough?)
-Check root= (did the system wait for the right device?)
-Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/disk/by-uuid/310147a9-f21c-4e7c-aeb0-18f31073da58 does not exist. Dropping to a shell!

BusyBox v1.10.2 (Ubuntu 1:1.10.2-2ubuntu7) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

Boot your computer and when the grub loader appears, select the linux version you use (to the rescue mode) and press 'e' to edit it.

Then remove the the UUID= part of the line from root=UUID=

Replace it with your current partition, so in my case it looks like

root=/dev/sda5

press +x to boot up with the changed settings

Then after you have logged in install fglrx module.

sudo apt-get install fglrx

Then restart.

Now this time select your linux version but this time not the rescue mode.

Then replace the UUID how you did earlier.

Done.

To permanently apply the settings to grub, you will have to edit the grub.cfg file

sudo vi /boot/grub/grub.cfg

Intelligent Agents in a simulated environment

noreply@blogger.com (Chathura Sarathchandra) — Fri, 20 Apr 2012 22:31:00 +0100

The following video and the description is about a Agent simulation that I have designed and implemented. I would like to give my gratitude to the "Intelligent Agents" module director

Dr. Sam Steel at the University Of Essex for his supervision throughout the module.

The simulation shows an "Agent-world" represented by a grid of squares. Agents survive in this world by eating plants. There are four types of plants that occur in various quantities. Agents are able to consume a combination of plants and gain more energy. The energy levels decreases as they live and as they roam to other squares in the gird. Creatures can carry a limited amount of plants so then can eat when it is needed. If the creatures can create two combinations at once then the most energetic combinations of plants will be eaten first.

In the “Agent world” there could be maximum of two types creatures in the world.

The agent types:

SimpleCreature: Less intelligent creature.

IntelCreature: Most intelligent creature.

SimpleCreature:

A simple creature will be born in the agent world with a limited amount of energy. The creatures move around the world in random directions in a consecutive manner, for random amounts of times, with an intention to feed to survive. A simple creature can carry a limited amount of plants. This creature eats plants when it's energy goes down less than it's energy limit. (When it feels hungry) The creature is able to consume compounds(CP)of plants (P) of varied types (t) that they carry, and gain more energy than it would if it eats them individually. Such that:

CP = {Pt₁, Pt₂, Pt₃, Pt₄}

Creatures can only see the plants in the same square as it is. When the world has been resized, eventually the creatures will follow back in to the world. Each individual creature has got a unique ID. This phenomenon has been recorded on the video.

The simulated environment in which the creatures act:
The “Agent world” is a grid of squares. The plants and the creatures will be in this grid.

IntelCreature:

Only an IntelCreature can look around. These creatures can only see only one square around ahead, as well as the square it is interpolated. They can also see the the plants as well as the creatures in its range of vision.

The creature can remember the creatures and the plants in the squares that it can see (This memory is limited). Although the vision is perfectly accurate only within its local square. Just after it has been born, IntelCreature will be in an Observation mode, so that it will observe the world by capriciously observing the world until it finds more than one plant. Then it will start going after the plants intentionally that it sees around. A creature may decide to go after plants that are more therapeutic than the others. Such plants may be the plants that could complete a combination.

If two creatures meet together in one square they start talking to each other, asking about the plants that the other creature carry. Then the first creature will propose to the other creature for a shared combination, with the amount that it would like to share. Such that:

Assume:
combo1 = P1 + P2 + P2
Ag₁:Plants = {P1, P3, P3, P1}
Ag₂:Plants = {P2, P2, P3}

Then:

Ag1: Ag2.requestCombo(<share_amount>, <plant>)

If the other creature agrees then they will consume the plants and share the energy amount. When they share the plant combinations, they will get the double the amount of the normal combination, hence it is more desirable for the creatures to share. The creatures can decide to be partners with another one, and go for other plants that one of the creatures have seen. Furthermore when the creatures are in the same square, they may request other creatures to be parters, a creature will request to be a partner of another creature, if the plant the second creature is going after, interests the first creature. If the second creature agrees to be partners, then it will make a leader and partner relationship between each other. The first creature that made the request will be the partner and the other will be the leader. After the leader gets the plant, the energy amount will be shared. Finally the partnership will end and the two creatures will go their own ways. In the situations where the plants that are subject to the partners engrossment have been taken by some other creature, the partnership will immediately end. When the partnership has been made, the colour of the red circle, will change to a light yellow colour if it is a partner, and the colour of the leader's circle will change in to dark yellow. This is demonstrated in the figure blow.

A creature can make partnership with another one, only if the both of the creatures are neither a partner or a leader. If the creatures do not carry any plants and it can not remember any plants, it goes back to the Observation mode to collect information about the world(Grid). If the creature can not carry any more plants, it will stop travelling to save the amount of energy that it will loose by going to other squares.

What are plants?:

There are four types of plants in the world.
Green plants, Red plants, Yellow plants, and Magenta plants.
The plants are represented in small squares with the colour of their names.
Each plant holds different energy levels.
A plant set consists of a set of random types of plants.
The plant sets appears in the “Agent world” in arbitrary places.
The amount of the plants that will be put in to the grid is variable.
In the world there will be a constant amount of plants at a time.
The size of the “Agent world” is variable.
The number of types of plants are in the world are variable.
The energy of creatures will be consumed until it dies. The energy consumption speed is variable.
If the creature moves to a different square, more energy will be consumed than if it stays in the same square.

The log output of the software which show

what a particular simple creature does
what a particular creature that looks around does
the interaction of a two creatures who meet

The above description about the "AgentWorld" is just a brief introduction about the project. Accordingly there are many other theoretical and technical details that I have not introduced to you in this article. Although if you are interested, I am happy to supply you with further details on this project.

Bicycle Actuators Artist Edition playing cards

noreply@blogger.com (Chathura Sarathchandra) — Wed, 18 Apr 2012 22:20:00 +0100

The new arrival to my playing card collection. Bicycle Actuators Artist Edition deck has been designed by Lance T. Miller. This is a great piece of art work that was hand drawn by the designer. Only 2500 decks have been printed and all of the decks have been individually numbered.

For more information follow the link to the kickstarter project.

Lance was grateful enough to sign my deck :)

Content Based Image Retrieval using cross correlation co-efficiency

noreply@blogger.com (Chathura Sarathchandra) — Wed, 4 Apr 2012 23:21:00 +0100

Have you ever searched google for images? I am sure everyone who is reading this article has done this before more than once. Wouldn't it be nice if you could search images for the similar images, for a given image? So for an example, if you want to search for an image of a particular object, by supplying a probe image (a image which to find similar images of) to the search engine, you can search for images that are similar to the given image.

Content-based image retrieval (CBIR) or Query by image content (QBIC), is an active area in Computer Vision research. Comparing images using cross-correlation efficiency was similar to the state of the art for CBIR in the early 1990s. Comparing the colours in the probe image and other test images, is one conservative way of doing this. Histogram is a good way of discriminating the colours in an image.

One way of doing this is explained below. There are many ways of doing this although this is not necessarily the right or the best way of doing it.

Split the R,G,B channels of the probe image
Split the R,G,B channels of the "test image" that is subject to comparison.
Calculate the histograms of the R,G,B channels of each images.
Calculate the cross correlation of the R, G, B channels of the probe image and the "test image" histograms. Such that,

r1 = correlation(img1.R,img2.R);
r2 = correlation(img1.G,img2.G);
r3 = correlation(img1.B,img2.B);

5. Finally multiply the results all together ( r1 x r2 x r3 )

The result (r) of a cross correlation calculation lies between -1 and +1. If the value is +1 in average the two images are identical. If it is -1 then the images are identical but the image looks like a photographic negative version of the probe image. If r=0 then the images are on average very unlike each other.

People who are interested in Computer Vision are likely to be familiar with the OpenCV library. The following code snippet (incomplete) is an example for calculating the cross correlation co-efficiency using the OpenCV library in C.

  //split img1 in to RGB channels and store them in seperate images

  cvSplit(img1, img1R, img1G, img1B, NULL);

 

  //calculate histograms of the the image

  CvHistogram *h1R = calHistogram(img1R, bins, ranges);

  CvHistogram *h1G = calHistogram(img1G, bins, ranges);

  CvHistogram *h1B = calHistogram(img1B, bins, ranges);



    //split the image two in to RGB channels and store them

    //in seperate images

    cvSplit(img2, img2B, img2G, img2R,0);

   

    //calculate histograms of the RGB channel images of the "img2"

    CvHistogram *h2R = calHistogram(img2R, bins, ranges);

    CvHistogram *h2G = calHistogram(img2G, bins, ranges);

    CvHistogram *h2B = calHistogram(img2B, bins, ranges);

      

    //Calculate the correlation coefficient of RGB images, of img1 and img2

    rR = cvCompareHist(h1R,h2R,CV_COMP_CORREL);

    rG = cvCompareHist(h1G,h2G,CV_COMP_CORREL);

    rB = cvCompareHist(h1B,h2B,CV_COMP_CORREL);

    

    //Multiply the results together

    tempR= rR*rG*rB;

The performance:

Some performance evaluation has been done on this program using a test harness called 'Fact'.


Error rates calculated from mycbir 



tests   TP         TN       FP        FN   accuracy  recall precision specificity       class 



8        0         0         8         0     0.00     0.00    0.00        0.00   box 



8        0         0         8         0     0.00     0.00    0.00        0.00   cockeral 



8        6         0         2         0     0.75     1.00    0.75        0.00   dog 



9        7         0         2         0     0.78     1.00    0.78        0.00   horus 



8        7         0         1         0     0.88     1.00    0.88        0.00  ornament 



10        8         0        2         0     0.80     1.00    0.80        0.00   penguin 



8        1         0         7         0     0.12     1.00    0.12        0.00   reindeer 



9        2         0         7         0     0.22     1.00    0.22        0.00   tortoise 



68       31        0        37         0     0.46     1.00    0.46        0.00          overall

TP = True Positives
TN= True Negatives
FP = False Positives
FN= False Negatives

In the matrix above it shows the amount of test cases for the each picture in the “test” column, True Positives, True Negatives, False Positives and False Negatives in the next four columns. Then the application calculates the accuracy, recall, precision and specificity as follows, and displays in the next four columns. The last “class” column show the type of picture that it has carried out the test on,

Precision is the number of relevant pictures that are retrieved, divided by the total number of of pictures retrieved. Recall is the number of relevant pictures retrieved divided by the total existing relevant pictures. The specificity and the accuracy is calculated as below

TP TN

sensitivity = recall = ------------------ specificity = -----------------

TP + TN FP + TN

As you can see above, the application has failed to find the True Positives for the “box” and “cockeral” pictures. So in contrast it has found False Positives in all cases. The best case is the tests on the ornament pictures as it has found 7/8 TPs and 1/8 FPs. Overall this algorithm has done 68 test cases and have found 31 True Positives and 37 False Positives, but no True Negatives or False Negatives.

The matrix below, is the class confusion matrix.


Confusion matrix calculated from mycbir 



expected 



actual    box    cockeral         dog    horus   ornament  penguin reindeer tortoise 



box        0        0              0        0        0        0        0        1 



dog        0        0              6        0        0        0        0        0 



horus      0        0              1        7        0        0        0        4 



ornament   1        1              0        0        7        2        7        0 



penguin    1        0              0        0        1        8        0        0 



reindeer   5        7              0        1        0        0        1        2 



tortoise   1        0              1        1        0        0        0        2

The columns and rows indicates that, the classes in the rows titles were obtained when the classes in the column title should have been found (Rows as obtained results, columns as expected result). The numbers indicates the number of times it has been obtained. So as an example, as the worst case: in the “box” column,

ornament =1, penguin = 1, reindeer = 5, tortoise = 1 has been obtained when “box” pictures should have been found. But it has not obtained any “box” pictures as the value of the “box” raw is 0.

As I have mentioned earlier the best results has been obtained on the tests with the “ornament” picture. It has obtained the correct “ornament” pictures 7 times and a penguin picture just one time.

Conclusion:

Cross correlation is not the best Content based Image Retrieval technique to date, though it helps us to understand how the histograms of two images can vary or can be complementary. But it is no doubt that analysing just the colours in images are a good way of characterising them though it is hard to compare two images using just this factor.

The record breaking Lego Christmas tree

noreply@blogger.com (Chathura Sarathchandra) — Tue, 20 Dec 2011 10:37:00 GMT

I am on my way to Derby and just saw this amazing Lego structure in the London st.Pancras train station.

Install and configure trac on Linux/Ubuntu 11.10

noreply@blogger.com (Chathura Sarathchandra) — Fri, 16 Dec 2011 18:52:00 GMT

In this blog post I am going to explain about how to install 'trac' on linux. 'trac' is a nice project management tool which is written in python, that can be used to track your projects. And it is open-source :D (Win)

I am using 'trac' with 'git' revision control system, for my project. In my next blog post I will be explain how to use 'trac' along with 'git', using the 'GitPlugin'

Ok, now if you follow the following instructions hopefully you will have a working 'trac' site.

Install 'trac'

Make sure you have the latest version of python installed, otherwise apt will install an older version of 'trac'

      sudo apt-get install python python-babel trac

Yes, apt may ask you to install few dependencies including apache2. So please install all of them.

'trac' uses SQLite as its default data store. So if you have not already got it installed. Then do the following

     sudo apt-get install sqlite3

It is possible to get 'trac' working with other database systems. If you are interested, click on the following URL.
http://trac.edgewall.org/

Upgrade python packages

There are two ways you can do this

With easy_install

 easy_install Babel

 easy_install Trac

or

With 'pip'

 pip install --upgrade Babel

 pip install --upgrade Trac

If you have not got pip installed, install pip as follows. Please skip this step if you were successfully able to do the previous step.

 sudo apt-get install python-pip

Now you have successfully installed all the software that you need to run 'trac'.

Configuration

Change the directory locations as desired.

Create a directory for 'trac'
Then change it's access permissions.

sudo mkdir -p /var/local/trac && chown www-data: /var/local/trac

Initialise the 'trac' environment

 sudo trac-admin /var/local/trac initenv

Now create the configuration file using your favourite text editor. I use emacs.

 sudo emacs /etc/apache2/sites-available/trac

Paste the following code in to it.

ServerName trac.local

SetHandler mod_python

PythonInterpreter main_interpreter

PythonHandler trac.web.modpython_frontend

PythonOption TracEnv /var/local/trac

PythonOption TracEnvParentDir /var/local/trac

PythonOption TracUriRoot /

PythonOption TracEnv /var/local/trac

# PythonOption TracEnvIndexTemplate /var/local/trac/templates/index-template.html

PythonOption TracLocale en_US.UTF8

PythonOption PYTHON_EGG_CACHE /tmp

Order allow,deny

Allow from all

</Location>

AuthType Basic

AuthName "myproject"

AuthUserFile /var/local/trac/.htpasswd

Require valid-user

</Location>

</VirtualHost>

Now you should have a working 'trac' instance. So check if it works.

      sudo tracd -p 8080 /var/local/trac

[-p] flag specifies the port that this particular 'trac' instance belongs to.

Note: the flag [-p] is same as [-port]

And then go to:

http://localhost:8080/

Now you should see the 'trac' instance running.

Adding Authentication

Basic Authorisation

In this case we are going to authorise the 'trac' site with a .htpasswd file.

You have to have 'fcrypt' package installed to decode '.htpasswd'

Creating the '.htpasswd'

 sudo htpasswd -c /var/local/trac/.htpasswd admin

To add more users:

 sudo htpasswd -c /var/local/trac/.htpasswd admin

'htpasswd' creates the flat-file with the username and password that you are given.
[-c] = create the password file in the given path.

Start tracd:

 sudo -p 8080 --basic-auth="projectdirectory,path/to/the/.htpasswd,mname" /path/to/the/environment/directory

In my case, it is

 sudo -p 8080 --basic-auth="trac,/var/local/trac/.htpasswd, admin" path/to/the/environment/directory

Digest Authentication

'htdigest' will be used to create the digest file.

 sudo htdigest -c /var/local/trac/.htdigest admin admin

Now start 'trac' with 'tracd':

 sudo -p 8080 --auth="projectdirectory,path/to/the/.htpasswd, admin" /path/to/the/environment/directory

So it will be,

 sudo -p 8080 --auth="trac,/var/local/trac/.htpasswd, admin" /var/local/trac

If you have just one project in 'trac', then use the [-s] flag with the 'tracd', so it will skip the environment list when it starts.
eg: sudo tracd -s ...........

There we go, we are done :D

Trouble Shooting Errors

If you get following error. It is most likely because of permission issues. So make sure you have given root access when you start your 'trac' instance.

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/trac/web/api.py", line 440, in send_error
    data, 'text/html')
  File "/usr/lib/python2.7/dist-packages/trac/web/chrome.py", line 827, in render_template
    message = req.session.pop('chrome.%s.%d' % (type_, i))
  File "/usr/lib/python2.7/dist-packages/trac/web/api.py", line 216, in __getattr__
    value = self.callbacks[name](self)
  File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 300, in _get_session
    return Session(self.env, req)
  File "/usr/lib/python2.7/dist-packages/trac/web/session.py", line 198, in __init__
    self.get_session(sid)
  File "/usr/lib/python2.7/dist-packages/trac/web/session.py", line 219, in get_session
    super(Session, self).get_session(sid, authenticated)
  File "/usr/lib/python2.7/dist-packages/trac/web/session.py", line 61, in get_session
    db = self.env.get_db_cnx()
  File "/usr/lib/python2.7/dist-packages/trac/env.py", line 328, in get_db_cnx
    return get_read_db(self)
  File "/usr/lib/python2.7/dist-packages/trac/db/api.py", line 90, in get_read_db
    return _transaction_local.db or DatabaseManager(env).get_connection()
  File "/usr/lib/python2.7/dist-packages/trac/db/api.py", line 152, in get_connection
    return self._cnx_pool.get_cnx(self.timeout or None)
  File "/usr/lib/python2.7/dist-packages/trac/db/pool.py", line 226, in get_cnx
    return _backend.get_cnx(self._connector, self._kwargs, timeout)
  File "/usr/lib/python2.7/dist-packages/trac/db/pool.py", line 146, in get_cnx
    raise TimeoutError(errmsg)
TimeoutError: Unable to get database connection within 0 seconds. (TracError(<babel.support.LazyProxy object at 0x22e47d0>,))

If you need more information, please go to http://trac.edgewall.org.

Bicycle new fan back (white) Limited edition playing cards deck

noreply@blogger.com (Chathura Sarathchandra) — Wed, 17 Aug 2011 02:32:00 +0100

This is the new member of my card pack collection. It is Bicycle fan back limited edition white deck, gold sealed, by Zenneth Kok. These are really rare, as there are less than 1000 copies that have been produced, and I have got one of them. The new fan back black and white decks will be officially released in September. There are only 5000 copies of black decks that have been produced.

I bought my deck from Zenneth and there were very limited quantity of both type of decks available. You have to buy at least one of his Metal Jackets along with the decks as it is not available to buy on its own. Personally I do not think they are still available to buy from Zenneth as he only had a limited amount of stock.You get two gaff cards with the deck, they are a double backer and a 0 heart card.

I am not going to open this deck and I am just going add this to my collection. Collecting card decks is what I do apart from being a computer scientist. I enjoy the art in the cards and I enjoy doing flourishes.

I hope you enjoyed the pictures.

Chathura

Set up Alpine to work with Gmail

noreply@blogger.com (Chathura Sarathchandra) — Sat, 2 Jul 2011 02:07:00 +0100

To set up Alpine with Gmail, You have you to have Alpine installed on your computer. To set up Gmail, click here to go to my previous blog post which explains how to install it.

If you have got Alpine installed then, next

To receive emails from your gmail account:

Go to the Main Menu >> Setup >> collectionLists >> Add.

Nickname : [Your Nickname]
Server : imap.gmail.com/ssl/user=username@gmail.com
Path :
View :

Save your settings

You can add as many accounts you want if you press E and exit, then do the same.

Then you need to change few settings on it.

On Main menu > Set up > Config, find the "Advanced user preferences", Then mark "Save will not delete"

Then scroll down and find Pruning Rule and select "don't rename, don't delete"

To send emails using your gmail account :

Go to Main menu >> Rules and then Add a new rule.

Add a Nickname
In the Current Folder Type section, Select "Specific"
Then select "Folder List" and press "T" and select the specific account folder(Inbox)
Scroll down and find the section where it says "Action Begin here"
Insert (Add) your email address to "Set Form" field
Then to the "use SMTP" server section Add "smtp.gmail.com/tls/user=" (Without the quotes)
Next Scroll down and find the "Compose Use" value and then select "With Confirmation"
Then Exit
You need to Add a new rule per each account

Now you are ready to go!

Read/Receive and Send emails on the linux command line - Alpine/pine

noreply@blogger.com (Chathura Sarathchandra) — Tue, 28 Jun 2011 17:23:00 +0100

I like to use my command line to do most of the work rather than using GUIs. So I wanted to be able read and send emails using the linux command line. After done much research I have found this free application called "Alpine" developed at the University Of Washington, which the earlier versions of it was called "Pine".

To download you can go to the official website by clicking the the URL below
http://www.washington.edu/alpine/

or if you are a Debian/Ubuntu user you can simply type in the command below

sudo apt-get install alpine

Currently I am using Alpine with my gmail account. I may be do a post about how to configure gmail on Alpine

Enjoy

Secure your Gmail/Google Account

noreply@blogger.com (Chathura Sarathchandra) — Mon, 27 Jun 2011 17:58:00 +0100

Are you confidant that your email account is secure anymore? You can never make anything 100% secure but, there are things you can do to make it more secure than it is.

Gmail has introduced 2-step verification which protects your gmail account little bit more. I recommend that everyone who read this blog, should enable the feature.

To enable: Go to your Google account page and then click on the
"Using 2-step verification" And then follow the instructions

Make sure you read the instructions carefully and print out or keep the verification codes safely, which you will need to log in to your account when you haven't got your phone to get a verification code. Never loose them.

The other thing you could do to protect your Gmail account is that to always use HTTPS. To enable this feature.

Log in to your gmail account
Click on the Options symbol
Click on the General tab
Then click on the radio button which says "always use https", under "Browser Connection"
Save Changes

Surf Save!

Start Dradis in BackTrack 5

noreply@blogger.com (Chathura Sarathchandra) — Thu, 23 Jun 2011 22:02:00 +0100

Back Track 5 comes with Dradis pre loaded already
But you might not be familiar with the way you start Dradis in BackTrack 5

To start Dradis:

Go to the dradis/server directory

root@bt:~# cd /pentest/misc/dradis/server

Then run start.sh script in the dradis directory

root@bt:/pentest/misc/dradis/server# /pentest/misc/dradis/start.sh

No Active Driver when you type in db_driver or db_connect?

noreply@blogger.com (Chathura Sarathchandra) — Thu, 23 Jun 2011 19:24:00 +0100

Do you get the following error message when you type in db_driver on Metasploit console on BlackTrack5?

[*] No Active Driver
[*]        Available:

[*]     DB Support: Enable the mysql driver with the following command:
[*]                 $ gem install mysql
[*]     This gem requires mysqlclient headers, which can be installed on Ubuntu with:
[*]                 $ sudo apt-get install libmysqlclient-dev

[*]     DB Support: Enable the postgresql driver with the following command:
[*]                   * This requires libpq-dev and a build environment
[*]                 $ gem install postgres
[*]                 $ gem install pg # is an alternative that may work

msf >

And you have tried installing mysql driver but didn't solve the problem ?

This happens because when you start msfconsole by /pentest/exploits/framework3/msfconsole, it does not set the environment variables properly

So all you have to do is, start msfconsole using BT5 menu or by using /usr/local/bin/msfconsole

The "Anonymous" threats continues against Sony (Update)

noreply@blogger.com (Chathura Sarathchandra) — Thu, 14 Apr 2011 06:31:00 +0100

The old story Is that Sony has filed a restraining order early this year, against George Hotz A.K.A GeoHot as he has violated copyrights by jail breaking the Sony's play station 3 which allowed him to install custom software on it. Now an independent organisation called 'Anonymous' has threatened Sony by putting up a video on youtube stating their demands (See below). Then this group began to take down Sony website including the Sony main website with DOS (Denial Of Service) attacks. They call this Anti-Sony operation the "Operation Payback".

Sony has demanded popular social media networks such as Youtube and Facebook, to hand over the IP addresses of people who has viewd GeoHot's videos.

Their argument was that you should be able to do what ever you want with the product you bought with your own money. More future, they ask, if its legal to jail break your phones and install unsigned software applications, why not jail break PS3?

In the last video they ask the public to return Sony products if you have bought any and support by join their force, by organising protests against Sony. Anonymous claims

"GeoHot has taken a settlement with sony. The case has been dropped. In the eyes of the law, the case is closed, for anonymous it is just beginning."

Is this just a start of a massive catastrophic cyber war? would it cause many other war's to start?

Problem With psexec (Solution)

noreply@blogger.com (Chathura Sarathchandra) — Wed, 13 Apr 2011 09:10:00 +0100

I do my pentests on a Windows XP sp3 box and I had this problem, that I could not get psexec to work as I kept getting this error message.

[*] Started reverse handler on :4444
[*] Connecting to the server...
[*] Authenticating to :445|WORKGRO[*] Started reverse handler on 192.168.56.3:4444
[*] Connecting to the server...
[*] Authenticating to :445|WORKGROUP as user ''...

[-] FAILED! The remote host has only provided us with Guest privileges. Please make sure that the correct username and password have been provided. Windows XP systems that are not part of a domain will only provide Guest privileges to network logins by default.

[*] Exploit completed, but no session was created.UP as user ''...

[-] FAILED! The remote host has only provided us with Guest privileges. Please make sure that the correct username and password have been provided. Windows XP systems that are not part of a domain will only provide Guest privileges to network logins by default.

[*] Exploit completed, but no session was created.

That was because if your Windows XP Professional based computer is not in a domain, by default all users trying to log in to it over the network, are forced to use the Gust Account. To find out more information go to http://www.windowsnetworking.com/articles_tutorials/wxpsimsh.html

So the solution is to

Go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
And change the ForceGuest value to 0

Done!

Text Encryption Application

noreply@blogger.com (Chathura Sarathchandra) — Tue, 12 Apr 2011 01:46:00 +0100

Under the Computer Security module we did one last assignment at the end of the term, the assignment was to implement the substitute cypher and demonstrate how it can be broken by using Frequency Analysis. So I have written a code to encrypt some text using a key, which the user inputs to the application (Which the application assumes that the key is unique), and to Break the encryption using frequency analysis, using Java. It was required to leave the spaces and non alphabetic characters, and maintain the cases in the cipher text (which makes the cypher text weaker).

I have added another extra feature for the cipher breaking code called The "Advanced Frequency Analysis". Which means that, In frequency analysis it just analyses the frequencies of single characters. But if you turn on this feature if will analyse two and three character combinations as well as single characters, which makes the code breaker more strong.

You will probably wonder that why would I want to add extra code to make the encryption weak. As an example this program passes spaces, non alphabetical characters through to the cipher text. It is because that they are requirements of the assignments so students can write about it on the report and give explanations about how to make it more stronger.

Find the screen shots and examples of the program below.

Encryption

Plain Text:

Data centres use vast amounts of electricity to run their computer equipment and also to keep it cool.

Environmental group Greenpeace has estimated that their total global energy use will have reached 2 trillion kw/h by 2020.

Key:

qazxswedcvfrtgbnhyujmkiolp

Cipher Text:

Hjtj koetfor gro bjrt jwdgetr dl osoktfakaty td fge tqoaf kdwcgtof ovgacwoet jei jsrd td xooc at kdds.

Yebafdewoetjs pfdgc Jfooecojko qjr ortawjtoi tqjt tqoaf tdtjs psdmjs oeofpy gro nass qjbo fojkqoi 2 tfassade xn/q my 2020.

Decryption

Advanced Frequency Analysis = off

Decrypted plain text:

Kata lestier dre wart amodstr oq eheltinlntg to ids tueni lomc1dtei ezdnc1mest asf ahro to veec1 nt looh. Jswniosmestah yiodc1 Xieesc1eale uar ertnmatef tuat tueni totah yhopah eseiyg dre bnhh uawe iealuef 2 tinhhnos vb/u pg 2020.

Advanced Frequency Analysis = on

Decrypted plain text:

Kata lestier the wart amodstr in eheltinlntg of for tueni lomc1dtei ezdnc1mest and ahro of veec1 to looh. Jswniosmestah yiodc1 Xieesc1eale his ertnmatef tuat tueni totah yhopah eseiyg the bnhh uawe iealuef 2 tinhhnos vb/u it 2020.

My Example is just a simple demonstration. As you can see its hard to analyse and decrypt just by using an application/ Algorithm, without having any human help. But This algorithm works better as the plain text gets larger, as it gets more data to analyse so the accuracy of the analysis gets better.

My Tetris

noreply@blogger.com (Chathura Sarathchandra) — Mon, 11 Apr 2011 04:10:00 +0100

'Tetris' is one of the most popular games of all time. For the last project of Application Programming module at the university, I got to develop this small Tetris game applet. As you can see its very basic and simple as I added more than requirements of the assignment, to the application itself. It would have had more functionalities but I moved on from it to the next assignment so I can finish my assignments and get it everything done way before the deadline. I will add more features to it if I get much time in the future, even though I tried ones and then I got busy with something else and then moved on.

To control the blocks You can use either keyboard or mouse or both.

Controls:

Mouse Controls:

Right-Mouse Button- Move right
Left-Mouse Button - Move Left

Key Board Controls:

Right Arrow key- Move Right
Left Arrow key - Move Left
'R' key - Rotate

If I develop it more further I will definitely post it on here. Like I have said it earlier its a very simple version of Tetris. But it plays according do all Tetris rules.

Youtube releases new HTML5 embed code

noreply@blogger.com (Chathura Sarathchandra) — Sat, 24 Jul 2010 06:57:00 +0100

Google is testing new HTML5 embed code that plays the videos using either flash or HTML 5 player and it uses <iframe> tag. It is still under construction but developers want feed back form current users so the beta version of it is now available.

It won’t play the video on HTML 5, if the video has got Google ads or if you haven’t set it up yet. If you are on a mobile device, it won’t play if it won’t take you to a separate player or if it won’t play the video in-line. But if your not eligible to view the video in HTML 5 it will play the video using the flash player. But again it depend on the preferences and where you are viewing the video.

To use this service you have to join the YouTube HTML 5 beta version and you must be using a browser which supports HTML5.

Supported browsers:

Google Chrome

Opera

Firefox that support the WebM format

Apple Safari 4.0 or late

-Chathura Mazz-