Ani's Blog

Before you start coding a web application. You must read this.

2014-11-24T15:25:00.002+05:30

How Browsers Work: Behind the scenes of modern web browsers

An excellent resource for you to know before you code.

Visit http://www.html5rocks.com/en/tutorials/internals/howbrowserswork/ for in-depth tutorial.

Nagios - How it works, the workflow!

2013-12-12T15:41:00.002+05:30

While working on a Nagios setup, its often difficult to explain how it works. I beleive in principle of "A picture says thousand word". Hence, started searching the internet, but didn't found any which is simple and easy to understand.

So created my own. For all those Nagios lovers, here's my version of it.

Hope this helps many people, like me!

- Ani

AWS for newbie

2013-09-16T05:56:00.001+05:30

If you are new to amazon's web services aka. Amazon cloud you must go and visit slideshare,search for aws. There are end number of documents and presentations to help you to start with. But before heading to slideshare, please read all the official whitepapers.

EnStratus: A Service to Manage Your Cloud Infrastructure by @alexwilliams http://t.co/duL1uGbugl

2013-09-15T20:36:00.001+05:30

EnStratus: A Service to Manage Your Cloud Infrastructure by @alexwilliams http://t.co/duL1uGbugl

— Ani (@animeshdas) September 15, 2013

via Twitter https://twitter.com/animeshdas

September 15, 2013 at 08:29PM

Distributing Your Blog Content: The Best Auto-Posting Services http://t.co/vaRJQfqdPn

2013-09-15T19:32:00.001+05:30

Distributing Your Blog Content: The Best Auto-Posting Services http://t.co/vaRJQfqdPn

— Ani (@animeshdas) September 15, 2013

via Twitter https://twitter.com/animeshdas

September 15, 2013 at 07:32PM

Checking and detecting host Virtualization

2013-05-20T12:50:00.001+05:30

How to check if the host is running as Virtual Machine (VM) or Physical Machine? and more importantly which Virtualization technology?

So here it is how to acheive that. Run the following command as superuser (root) :

dmidecode -s system-manufacturer

the output should be something

"Redhat"

"VMWare"

"XEN"

"Citrix"

then run

dmidecode | grep -i "vm"

it should show result like

Product Name: KVM

(which means running KVM Virtualization)

or

Product Name: HVM domU

(which means running XEN Virtualization)

then run

dmesg | grep -i virtual

Different virtual host will throw different output, similar to :

VMWARE

VMware vmxnet virtual NIC driver
Vendor: VMware Model: Virtual disk Rev: 1.0
hda: VMware Virtual IDE CDROM Drive, ATAPI CD/DVD-ROM drive

Qemu or KVM

CPU: AMD QEMU Virtual CPU version 0.9.1 stepping 03
skipping (on a virtual guest)

or sometime

[ 0.000000] Booting paravirtualized kernel on KVM

Microsoft VirtualPC:

# dmesg | grep -i virtual
hda: Virtual HD, ATA DISK drive
hdc: Virtual CD, ATAPI CD/DVD-ROM drive

Xen :

# dmesg | grep -i xen
Xen virtual console successfully installed as tty1

Virtuozzo :

# dmesg
(returns no output)

# cat /var/log/dmesg
(returns no output)

# ls -al /proc/vz
veinfo veinfo_redir veredir vestat vzaquota vzdata

On longer-running systems, you may need to grep /var/log/dmesg instead.

If that doesn't produce anything useful, try using dmidecode to look at the BIOS information. Frequently, there will be at least one component identifying itself as virtualized:

VMWare:

     # dmidecode | egrep -i 'manufacturer|product'
     Manufacturer: VMware, Inc.
     Product Name: VMware Virtual Platform

Microsoft VirtualPC:

     # dmidecode | egrep -i 'manufacturer|product'
     Manufacturer: Microsoft Corporation
     Product Name: Virtual Machine

QEMU or KVM:

# dmidecode | egrep -i 'vendor'
Vendor: QEMU

Virtuozzo:

# dmidecode
/dev/mem: Permission denied

Xen:

# dmidecode | grep -i domU
Product Name: HVM domU

You should just examine the output of dmidecode directly rather than trying to grep as above, in case the output changes. QEMU, for example, doesn't report the vendor in all versions.

Next, check disk devices for identification as virtualized:

VMWare:

    # cat /proc/ide/hd*/model
    VMware Virtual IDE CDROM Drive
    # cat /proc/scsi/scsi
    Attached devices:
    Host: scsi0 Channel: 00 Id: 00 Lun: 00
      Vendor: VMware   Model: Virtual disk     Rev: 1.0
      Type:   Direct-Access                    ANSI SCSI revision: 02

Microsoft VirtualPC:

    # cat /proc/ide/hd*/model
    Virtual HD
    Virtual CD

QEMU, KVM, or Xen:

    # cat /proc/ide/hd*/model
    QEMU HARDDISK
    QEMU DVD-ROM

Virtuozzo:

# ls -al /dev/vzfs
b-----x--- 1 root root 0, 19 2009-04-06 15:04 /dev/vzfs

The following bash script can also be used.


#!/bin/bash -
# @configure_input@
# Copyright (C) 2008-2011 Red Hat Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

# 'virt-what' tries to detect the type of virtualization being
# used (or none at all if we're running on bare-metal).  It prints
# out one of more lines each being a 'fact' about the virtualization.
#
# Please see also the manual page virt-what(1).
# This script should be run as root.
#
# The following resources were useful in writing this script:
# . http://www.dmo.ca/blog/20080530151107

VERSION="@VERSION@"

function fail {
    echo "virt-what: $1"
    exit 1
}

function usage {
    echo "virt-what [options]"
    echo "Options:"
    echo "  --help          Display this help"
    echo "  --version       Display version and exit"
    exit 0
}

# Handle the command line arguments, if any.

TEMP=`getopt -o v --long help --long version --long test-root: -n 'virt-what' -- "$@"`
if [ $? != 0 ]; then exit 1; fi
eval set -- "$TEMP"

while true; do
    case "$1" in
    --help) usage ;;
        --test-root)
            # Deliberately undocumented: used for 'make check'.
            root="$2"
            shift 2
            ;;
    -v|--version) echo $VERSION; exit 0 ;;
    --) shift; break ;;
    *) fail "internal error ($1)" ;;
    esac
done

# Add /sbin and /usr/sbin to the path so we can find system
# binaries like dmicode.
# Add /usr/libexec to the path so we can find the helper binary.
prefix=@prefix@
exec_prefix=@exec_prefix@
PATH=$root@libexecdir@:$root/sbin:$root/usr/sbin:$PATH

# Check we're running as root.

uid=`id -u`
if [ "$uid" != 0 ]; then
    fail "this script must be run as root"
fi

# Many fullvirt hypervisors give an indication through CPUID.  Use the
# helper program to get this information.

cpuid=`virt-what-cpuid-helper`

# Check for various products in the BIOS information.
# Note that dmidecode doesn't exist on non-PC architectures.  On these,
# this will return an error which is ignored (error message redirected
# into $dmi variable).

dmi=`LANG=C dmidecode 2>&1`

# Architecture.
# Note for the purpose of testing, we only call uname with -p option.

arch=`uname -p`

# Check for VMware.
# cpuid check added by Chetan Loke.

if [ "$cpuid" = "VMwareVMware" ]; then
    echo vmware
elif echo "$dmi" | grep -q 'Manufacturer: VMware'; then
    echo vmware
fi

# Check for Hyper-V.
# http://blogs.msdn.com/b/sqlosteam/archive/2010/10/30/is-this-real-the-metaphysics-of-hardware-virtualization.aspx
if [ "$cpuid" = "Microsoft Hv" ]; then
    echo hyperv
fi

# Check for VirtualPC.
# The negative check for cpuid is to distinguish this from Hyper-V
# which also has the same manufacturer string in the SM-BIOS data.
if [ "$cpuid" != "Microsoft Hv" ] &&
    echo "$dmi" | grep -q 'Manufacturer: Microsoft Corporation'; then
    echo virtualpc
fi

# Check for VirtualBox.
# Added by Laurent Léonard.
if echo "$dmi" | grep -q 'Manufacturer: innotek GmbH'; then
    echo virtualbox
fi

# Check for OpenVZ / Virtuozzo.
# Added by Evgeniy Sokolov.
# /proc/vz - always exists if OpenVZ kernel is running (inside and outside
# container)
# /proc/bc - exists on node, but not inside container.

if [ -d $root/proc/vz -a ! -d $root/proc/bc ]; then
    echo openvz
fi

# Check for Linux-VServer
if cat $root/proc/self/status | grep -q "VxID: [0-9]*"; then
    echo linux_vserver
fi

# Check for UML.
# Added by Laurent Léonard.
if grep -q 'UML' $root/proc/cpuinfo; then
    echo uml
fi

# Check for IBM PowerVM Lx86 Linux/x86 emulator.
if grep -q '^vendor_id.*PowerVM Lx86' $root/proc/cpuinfo; then
    echo powervm_lx86
fi

# Check for Hitachi Virtualization Manager (HVM) Virtage logical partitioning.
if echo "$dmi" | grep -q 'Manufacturer.*HITACHI' &&
   echo "$dmi" | grep -q 'Product.*HVM LPAR'; then
    echo virtage
fi

# Check for IBM SystemZ.
if grep -q '^vendor_id.*IBM/S390' $root/proc/cpuinfo; then
    echo ibm_systemz
    if [ -f $root/proc/sysinfo ]; then
        if grep -q 'VM.*Control Program.*z/VM' $root/proc/sysinfo; then
            echo ibm_systemz-zvm
        elif grep -q '^LPAR' $root/proc/sysinfo; then
            echo ibm_systemz-lpar
        else
            # This is unlikely to be correct.
            echo ibm_systemz-direct
        fi
    fi
fi

# Check for Parallels.
if echo "$dmi" | grep -q 'Vendor: Parallels'; then
    echo parallels
    skip_qemu_kvm=1
fi

# Check for Xen.

if [ "$cpuid" = "XenVMMXenVMM" ]; then
    echo xen; echo xen-hvm
    skip_qemu_kvm=1
elif [ -f $root/proc/xen/capabilities ]; then
    echo xen
    if grep -q "control_d" $root/proc/xen/capabilities; then
        echo xen-dom0
    else
        echo xen-domU
    fi
    skip_qemu_kvm=1
elif [ -f $root/sys/hypervisor/type ] &&
    grep -q "xen" $root/sys/hypervisor/type; then
    # Ordinary kernel with pv_ops.  There does not seem to be
    # enough information at present to tell whether this is dom0
    # or domU.  XXX
    echo xen
elif [ "$arch" = "ia64" ]; then
    if [ -d $root/sys/bus/xen -a ! -d $root/sys/bus/xen-backend ]; then
        # PV-on-HVM drivers installed in a Xen guest.
        echo xen
        echo xen-hvm
    else
        # There is no virt leaf on IA64 HVM.  This is a last-ditch
        # attempt to detect something is virtualized by using a
        # timing attack.
        virt-what-ia64-xen-rdtsc-test > /dev/null 2>&1
        case "$?" in
            0) ;; # not virtual
            1) # Could be some sort of virt, or could just be a bit slow.
                echo virt
        esac
    fi
fi

# Check for QEMU/KVM.
#
# Parallels exports KVMKVMKVM leaf, so skip this test if we've already
# seen that it's Parallels.  Xen uses QEMU as the device model, so
# skip this test if we know it is Xen.

if [ ! "$skip_qemu_kvm" ]; then
    if [ "$cpuid" = "KVMKVMKVM" ]; then
    echo kvm
    else
        # XXX This is known to fail for qemu with the explicit -cpu
        # option, since /proc/cpuinfo will not contain the QEMU
        # string.  The long term fix for this would be to export
        # another CPUID leaf for non-accelerated qemu.
        if grep -q 'QEMU' $root/proc/cpuinfo; then
        echo qemu
    fi
    fi
fi

How to connect to a remote X Windows just like sitting in front of it.

2013-01-22T18:09:00.001+05:30

I have used as Zenity to create a GUI application for Xephyr. Xephyr is a much better implementation than Xnest. Those who dont know what is Xephyr or Xnest, please search google. Also remember not to use my procedure of using Xephyr connecting to a system using internet as there is no encryption of any kind are used here. It has serious security implication. This procedure is better suited for intranet use.

First of all I am not gonna tell you how to do, this and that. You need to be good in linux and a seasoned linux enthusiast like me. Here I will just share a bash script which is actually an attempt to create a GUI interface for Xephyr.

So first ensure X-windows is running in the system that you want to connect to. Lets call it server. Also ensure that you can actually login directly to your server's graphical session. Now enable XDMCP in it.

Now save the following code in a file /usr/bin/xephyr-gui

#!/bin/bash

reschoice=(FALSE "640x480" FALSE "800x600" FALSE "1024x768" FALSE "1152x864" FALSE "1280x960" FALSE "1400x1050" TRUE "1600x960")
resolution=$(zenity --title "Resolution" --height 300 --list --text "Select a resolution from the list below" --radiolist --column "Pick" --column "Resolution" ${reschoice[@]})

return_value=$?
case $return_value in

1)

echo -e '\E[31m'"Canceled"
exit ;;

0)

display=$(zenity --title="Gtk-Xephyr" --scale --text "Pick a Display Number\n[0 is default and used]" --min-value=1 --max-value=100 --value=1 --step 2)

return_value=$?
case $return_value in

1)

echo -e '\E[31m'"Canceled"
exit ;;

0)

connectto=$(zenity --title="Gtk-Xephyr" --entry --text "Please enter IP/hostname X server is running to connect." --entry-text "localhost")

return_value=$?
case $return_value in

1)

echo -e '\E[31m'"Canceled"
exit ;;

0)

gksudo lsof | tee >(zenity --title "Please wait" --text "Connecting..." --progress --pulsate --no-cancel --auto-close) > /dev/null
/usr/bin/Xephyr -ac -br -reset -terminate -screen $resolution -title $connectto -query $connectto 2> /dev/null :$display &

exit ;;

esac
esac
esac

exit 0

Save the file and exit. Now make it executable.

chmod +x /usr/bin/xephyr-gui

Now you have a GUI to work with Xephyr. Enjoy.

How to resize KVM or Xen Virtual Disk Image

2012-06-01T19:05:00.001+05:30

Without much humdrum, let me jump into the how-to part. But just before that a small note. When i mentioned disk image, I meant device image and not partition image.

There is a difference between disk image and partition image. Even if the file is something.img, you need to ensure that it is a disk and not a partition image. Disk image is where the partition reside. It is a container for partitions. In simple term device holds the partion within it just like chocholate box holds chocolate cubes in it. Thus depending upon the size/capacity of the box or device inner volume, you can hold partition or chocoloate of upto its maximum permitted size/volume and not beyond that. So please ensure you are actually resizing the disk, using the methods below, and then the partition. In the tutorial below resizing the partition is just a part of the entire process. If you just want to expand the device image, you can skip partition expanding process or if you to expand the partition only without or with a already large device, skip the device expanding part. My tutorial below uses basic Linux tools and syntaxes that will work in any version and flavour of Linux that support KVM or Xen based virtualization. I have not tried this with any other virtualization technology, but with little changes it should work for others too.

For the purpose of this tutorial, following are the specs and assumptions used as example.

Host OS : Installed, configured and running CentOS or RHEL 5.6 as OS with KVM or Xen as virtualization technology.
Guest VM OS : Installed, configured and running CentOS or RHEL 5.6 as OS.
VM Disk(Device) Image Name : 60GB.img
VM Disk(Device) Size : 60GB (maximum allowed size)
Root ( / ) partition : The / partition of this machine was assigned to the virtual disk of size 60GB. So it is utilizing the maximum allowed size of the device.
Resize requirement : around 100GB

RESIZE THE DISK(DEVICE)

1. Create new image of size 40GB for the disk(device). 40GB because we are going to expand the 60 GB to 100GB, so we need additional 40GB

qemu-img create -f raw 40GB.img 40G

2. Now create a new disk(device) image by concatenating 60GB.img + 40GB.img to a new file 100GB.img. Remember to concatenate 60GB.img first, with 40GB and not vise-versa.

cat 60GB.img 40GB.img > 100GB.img

The disk(device) is now resized to 100GB. Now attach 100GB.img as your new primary device(removing the old 60GB.img) in your VM configuration and boot your guest.

RESIZE(ONLINE) THE PARTITION

1. After the VM finishes booting, login to it and issue the following command.

df -h // See all mounted partition related information.

fdisk -l // See all available disk(device) related information.

Note down the partition name and number, and device name from the above command. e.g. /dev/sda3 is the partion I want to expand, thus /dev/sda is the device.

2. Now we will do the actual resizing using fdisk. First we will ask fdisk to load and modify the device information into its cache. And then apply it into the disk by flushing its cache.

fdisk /dev/sda // Choose the device, to load information of, in its cache.

3. Then, ask fdisk to delete the partition information in its cache.

Command (m for help): p // Print partition information
Command (m for help): d // Delete partition
Partition number (1-4): 3 // Partition number to delete. It should be the partition that you want to expand.

above, 3 is the partion number of /dev/sda3 I want to expand. In case of a single partition it will be selected automatically by fdisk. Dont save the partion at this stage else you will lose data.

Then, ask fdisk to recreate the parition in fdisk cache.

Command (m for help): n // new partition
Command action
e extended
p primary partition (1-4)
p // partition type is primary
Partition number (1-4): 3 // same partition number as given above.

keep pressing return, to select all default values for all other, including cylinder start and end to utilize the entire available capacity of the device. Otherwise give end value of cylinder after calculating the size using cylinder value. Google it for that with keyword "howto calculate disk size from cylinder value"

Now, review the changes made in fdisk's cache.

Command (m for help): p // print partition information to review the changes made in fdisk's cache.

Finally flush the changes from fdisk cache to disk.

Command (m for help): w // Flush the cache to disk parmanently.

See example image below for the entire step 3.

Ignore any error messages as above

4. Now reboot your VM once again. Then once the VM finishes booting, login to it.

5. Run the following commands

resize2fs /dev/sda3 // Online resizing of the partition /dev/hda1

6. Now reboot your VM for the last time. One the VM finishes booting, you have resized partion inside your resized disk(device). Resized disk(device) only in case you have resized that too, to make space for your new resized partition.

df -h

Isn't that simple?

Cheers!
Ani

Downloadable Ebook : Build your own private cloud using Ubuntu Eucalyptus Enterprise Cloud computing platform.

2012-02-10T00:43:00.002+05:30

There was many requests to publish one of my previous article in a form of a downloadable EBook. Keeping that in mind I am happy to publish an Ebook in pdf format for offline use. Please let me know if this effort has helped you in anyway.

Download it here. | The original article was published here.

Android vs Meego (more precisely Motorola Xoom vs Nokia N9)

2011-08-10T22:22:00.002+05:30

Let me first address the fact that I work for such a company which is a leader in mobile devices. When it comes to mobile phone people still talk about them despite of so many rumors in the market and declining market share. But my post here today is not influenced by that fact anyway, neither I want to pretend something. I'm honestly writing my experience with these 2 devices and their OS'es from my extensive daily use.

I know comparing XOOM with N9 is just like comparing a Whale with a River Shark. Believe me I don't have another Android hand set of similar kind to compare. But I must tell you that despite of the price and HW spec that XOOM has, it still falling behind N9. Here it is how.

I had the opportunity to try N9 as a part of a Nokia internal program for testing. I'm using the device for almost 3 months now. Last Sunday I bought Motorola XOOM by spending almost 40K INR for my personal reason. But after using it almost for a week, I realized its not worth the price I paid for.

User Experience

Chiliproject Init script

2011-07-05T22:32:00.002+05:30

Dear Friend,

This is my init.d script for automatic startup and shutdown of chiliproject (mongrel) using RVM and REE env during system bootup. You can use/modify this script to suite your need for your chiliproject/redmine installation.

#!/usr/bin/env bash
#
# Chilliproject init script written by Animesh Das <jobs4ani at gmail dot com>
#
# chkconfig: - 16 84
# description: Init Script for Chiliproject
#
# processname: mongrel

# source function library
. /etc/rc.d/init.d/functions

# Get network config
. /etc/sysconfig/network

[ "${NETWORKING}" = "no" ] && exit 0

# App Settings
APP_USER=chili
APP_USER_HOME=/home/$APP_USER
APP_NAME=Chiliproject
APP_ROOT=$APP_USER_HOME/chiliproject
APP_PIDFILE=$APP_ROOT/log/chiliproject.pid

# Daemon Settings
DAEMON_USER=$APP_USER
DAEMON_GROUP=$APP_USER
DAEMON_NAME=mongrel
DAEMON_CMD=mongrel_rails
DAEMON_ENV=production
DAEMON_HOME=$APP_ROOT
DAEMON_LOGFILE=$DAEMON_HOME/log/$DAEMON_NAME.log
DAEMON_PIDFILE=$DAEMON_HOME/log/$DAEMON_NAME.pid
DAEMON_LOCKFILE=$DAEMON_HOME/tmp/$DAEMON_NAME.lock
DAEMON_ARGS="-e $DAEMON_ENV -d -p 9090 -l $DAEMON_LOGFILE -P $DAEMON_PIDFILE -c $DAEMON_HOME --user $DAEMON_USER --group $DAEMON_GROUP"

# Set the environment
source $APP_USER_HOME/.rvmrc && [[ -s $APP_USER_HOME/.rvm/scripts/rvm ]] && source $APP_USER_HOME/.rvm/scripts/rvm
if [[ -s "$APP_USER_HOME/.rvm/environments/ree-1.8.7-2011.03" ]]
then
source "$APP_USER_HOME/.rvm/environments/ree-1.8.7-2011.03"
#exec mongrel_rails "$@"
else
echo "ERROR: Missing RVM environment file: '$APP_USER_HOME/.rvm/environments/ree-1.8.7-2011.03'" >&2
exit 1
fi

# Ruby config
MONGREL_BIN=$(type -P $DAEMON_CMD)

start() {
echo -n $"Starting ${APP_NAME}: "

cd $DAEMON_HOME
cat /dev/null > $DAEMON_LOGFILE
sleep 2
daemon $MONGREL_BIN start $DAEMON_ARGS
sleep 2
RETVAL=$?
if [ $RETVAL -eq 0 ]; then
touch $DAEMON_LOCKFILE
fi
echo
}

stop() {
echo -n $"Stopping ${APP_NAME}: "
sleep 2
cd $DAEMON_HOME
killproc -p $DAEMON_PIDFILE
RETVAL=$?
[ $RETVAL -eq 0 ] && /bin/rm -f $DAEMON_LOCKFILE $DAEMON_PIDFILE
echo
}

case "$1" in
start)
start
;;
stop)
stop
;;
restart)
stop
start
sleep 3
;;
status)
status -p $DAEMON_PIDFILE $DAEMON_NAME
;;

*)
echo "Usage: $SCRIPTNAME {start|stop|restart|status}" >&2
exit 3
;;
esac

Build your own private cloud using Ubuntu Eucalyptus Enterprise Cloud computing platform.

2010-08-21T03:13:00.002+05:30

1.0 Introduction

In this Hands-on Labs session, we’re going to use the Linux Ubuntu 10.04 LTS (Lucid Lynx) release. Ubuntu 10.04, Lucid Lynx, is the platform of choice for anybody who intends to build and deploy large-scale infrastructure, whether you're trying to build the next Facebook, or the next Google, or the next eBay. If you want to start on [Amazon's cloud-platform] EC2 and migrate to your own managed private cloud, Ubuntu 10.04 is going to be the platform for you. Lucid Lynx will be the third long-term support (LTS) release by Ubuntu. LTS releases arrive bi-annually and are supported for three years on desktops and five years for servers.

Ubuntu 10.04 LTS Server Edition: Lean, fast and powerful – Ubuntu Server delivers services reliably, predictably and economically - and easily integrates with your existing infrastructure.
Ubuntu is an operating system built by a worldwide team of expert developers. It contains all the applications you need: a web browser, office suite, media apps, instant messaging and much more. Ubuntu is an open-source alternative to Windows and Office.

Ubuntu Enterprise Cloud (UEC): The UEC allows you to build your own private cloud. The Ubuntu Cloud Planet is a window into the world, work and lives of those that work on making Ubuntu the best cloud platform there is. With Ubuntu Enterprise Cloud you can bring the same self-service capability into your data center using the same tools and APIs used on Amazon EC2.

With Ubuntu, you can build the perfect cloud environment for your business whether private, with Ubuntu Enterprise Cloud (UEC), or public, with Ubuntu Server on Amazon EC2. Because UEC uses the same machine images as Amazon EC2, you can create a hybrid cloud or migrate between public and private easily at any time.

Private clouds offer immediacy and elasticity in your own IT infrastructure. Using Ubuntu Enterprise Cloud, you can experience the benefits of cloud computing behind your firewall. Deploy workloads and have them running immediately. Grow or shrink computing capacity to meet the needs of your application.

Solution:

In this Hands-on Lab session, you’ll learn how to setup virtual network on VMware (you may also use any other virtual machines like MS VirtualPC, Linux Xen, or VirtualBox from Sun). In this lab session, you’ll learn to Build your Own Private Cloud using Ubuntu 10.04 Eucalyptus Enterprise Cloud Computing Platform.

Part 1: Install & Configure Ubuntu 10.04 Enterprise Cloud Server

Step 1: Getting Started & Hardware Pre-requisites
In this Hands-on Labs, it’s assumed that your target computer is connected to the internet. After installing the cluster controller, log into your machine and ensure you perform software updates to bring your systems up-to-date. In the second part, you will install the node controller. For minimum hardware requirements check here.

Step 2: Install Ubuntu 10.04 LTS Enterprise Cloud

Hope over to http://www.ubuntu.com/getubuntu/download-server website and download the Linux Ubuntu 10.04 64-bit ISO which at the time of writing this lab manual was "Ubuntu-10.04-server-amd64.iso"

Once you have downloaded the Ubuntu ISO specific to your distribution, you have the option burning it into CD or just by using the ISO package to install it from your virtual machine, in our case VMware.

Fire-up a new virtual machine and perform the initial configuration and setup to use ISO package.

Start the virtual machine, and you should be able to see the first Ubuntu installation screen.

From Fig. 1, select the language desired and hit Enter key.

From Fig. 2, select the option desired and then hit the Enter key to commence installation. In our case we have selected to Install Ubuntu Enterprise Cloud option.

I am a working professional. So I have to spent most of my time, in office, works, R&D, and learning new technologies. Infact I do not find enough time to spend with my family, specially my one year old daughter. So you can understand how severe time constraint I am in, specially spending time in writing such big blog post. So, unless it helps others, there is no point of composing, formatting such big article which will take hours.

Now do you really want me to post this complete article? If yes, post your comments and let me know why you want this article to be posted and how it can help you. I promise I will post rest of the content.

Thank you so much to all who have find this post useful and forced me to update it with rest of the content. Thanks again for your response. Please accept my apology for my late and delayed resposne. Now as I've mentioned above, I was extremely busy and due to lack of time it is difficult to post such a huge content. Anyway without wasting any further time here goes the rest of this content…..

7. From Fig. 3, Choose language screen, select English option, and hit the Enter key.

Fig. 3

8. From Fig. 4, Choose language screen, Choose a country, select United States option, and hit Enter

Fig. 4

9. From the Ubuntu installer main menu screen, Fig. 5, accept the default selected <No> and hit the

Enter key to continue.

Fig. 5

10. From the Ubuntu installer main menu screen, Fig. 6, Origin of keyboard, select USA and hit the Enter key to continue.

Fig. 6

11. From the Ubuntu installer main menu screen, Fig. 7, Keyboard layout, select USA and hit the

Enter key to continue.

Fig. 7

12. From Fig. 8, the Detecting hardware to find CD-ROM drives screen, wait for the system complete the process.

Fig. 8: System Detecting hardware to find CD-ROM drives

13. From Fig. 9, the Loading additional components screen, wait for the system complete the process.

Fig. 9: System Loading additional components

14. From Fig. 10, the Configure the network screen, enter desired the Hostname: "cloudoscarcloud.com", change as desired, and hit the Tab key to select <Continue> and hit the Enter key to continue.

Fig. 10

15. From Fig. 11, the Select cloud installation mode screen, leave the cloud control address blank, hit the Tab key to select <Continue> and hit the Enter key to continue.

Fig. 11

16. From Fig. 12, the Select cloud installation mode screen, accept the selected default option, hit the

Tab key to select <Continue> and hit the Enter key to continue.

Fig. 12

17. From Fig. 13, the setting up the clock screen, wait for the system to complete the process.

Fig. 13

18. From Fig. 14, the Configure the clock screen, accept the default selection for the time zone, or change as desired, and then hit the Enter key to continue.

Fig. 14

19. From Fig. 15, the Starting up the partitioner screen, wait for the system to complete the process.

Fig. 15

20. From Fig. 16, the Partition disks screen, accept the default selection, or change as desired, and then hit the Enter key to continue.

Fig. 16

21. From Fig. 17, the Partition disks screen, accept the default selection, or change as desired, and then hit the Enter key to continue.

Fig. 17

22. From Fig. 18, the Partition disks screen, select <Yes>, or change as desired, and then hit the Enter

key to continue.

Fig. 18

23. From Fig. 19, the Partition disks screen, accept the default selection of 21.2GB, or change as desired, and then hit the Tab key to select <Continue> and hit the Enter key to continue.

Fig. 19

24. From Fig. 20, the Partition disks screen, read the warning and then select <Yes>, or change as desired, and then hit the Enter key to continue.

Fig. 20

25. From Fig. 21, the Partition disks screen, wait for the system to complete the process.

Fig. 21

26. From Fig. 22, the Installing the base system screen, wait for the system to complete the process.

Fig. 22

27. From Fig. 23, the Setup users and passwords screen, enter the full name of the user: "Cloud

Admin", or change as desired, and then select <Continue> and hit the Enter key to continue.

Fig. 23

28. From Fig. 24, the Setup users and passwords screen, enter the username of the user: "cadmin", or change as desired, and then select <Continue> and hit the Enter key to continue.

Fig. 24

29. From Fig. 25a, the Setup users and passwords screen; enter the password of the user "cadmin", and then select <Continue> and hit the Enter key to continue.

Fig. 25a Note: Reenter the password when prompted.

30. From Fig. 25b, the Setup users and passwords screen; accepted the selection <No> not to encrypt your home directory, or change as desired, and then hit the Enter key to continue.

Fig. 25b

31. From Fig. 26, the Configuring apt screen, wait for the system to complete the process.

Fig. 26

32. From Fig. 27, the Configuring the package manager screen; leave HTTP proxy blank, hit the Tab

key to select <Continue> and hit the Enter key to continue.

Fig. 27

33. From Fig. 28, the Configuring apt screen, the system will continue with configuring apt, wait for the system to complete the process.

Fig. 28

34. From Fig. 29, the Select and install software screen, select the desired update option, and hit the

Enter key to continue.

Fig. 29

35. From Fig. 30, the Select and install software screen, wait for the system to complete the process.

Fig. 30

36. From Fig. 31, the Postfix Configuration screen; under System mail name: "cloud.oscarcloud.com", or change as desired, and then hit the Tab key to select <Continue> and hit the Enter key to continue.

Fig. 31

37. From Fig. 32, the Configuration eucalyptus-cc screen; accept the default Eucalyptus cluster name: "cluster1", change as desired, and then hit the Enter key to continue.

Fig. 32

38. From Fig. 33, the Configuration eucalyptus-cc screen; leave the IP address range, enter:

"192.168.83.30-192.168.83.50", or change as desired, and then hit the Tab key to select

<Continue> and hit the Enter key to continue.

Fig. 33

39. From Fig. 34, the Select and install software screen, the system will continue with the installation process, wait for the system to complete the process.

Fig. 34

40. From Fig. 35, the Installing GRUB boot loader screen, wait for the system to complete the process.

Fig. 35

41. From Fig. 36, the Installing GRUB boot loader screen, accept the default selection to install GRUB

boot loader, or change as desired, and hit the Enter key to continue.

Fig. 36

42. From Fig. 37, the Finishing the installation screen, wait for the system to complete the process.

Fig. 37

43. From Fig. 38, the Finish installation screen; Installation complete. Remove the installation

CD, and hit the Enter key to restart the system.

Fig. 38

44. From Fig. 39, you can observer the system startup process.

Fig. 39: Ubuntu cloud server startup process.

45. From Fig. 40, enter your username and password, same as the one entered during the installation stage, and hit enter.

Fig. 40: Ubuntu cloud server with successful login.

46. To check you server IP address, if the Ifconfig command, as shown in Fig. 41.

Fig. 41: Checking server IP address

47. We’re done with this section.

Step 3: Update Ubuntu 10.04 Operating Systems

It is useful to update your system with the latest components and system patches. The first command below asks Ubuntu to update its database of available packages, and the second command installs the latest packages based on your current configuration. We run the upgrade command twice to ensure that any packages that may have post-upgrade dependencies also have an opportunity to be upgraded.

1. From the command line, enter the following commands, one at a time.

$ sudo apt-get update

$ sudo apt-get -u upgrade

Note 1: The sudo command is used to run privileged operations on the Ubuntu platform. The first time you run the command, it will ask for your password. Subsequently, it will not ask for your password again for a short period of time (typically 15 minutes).

Note 2: apt-get is the program Ubuntu uses for managing the system’s packages. When used to manipulate the core packages of the operating system, it needs to be run in conjunction with the sudo command.

2. We’re done with this section

3. You’re now ready to begin any other application installation as desired!

Part 2: Ubuntu Linux Node Installation

Node installation under the Ubuntu 10.04 install is as straight forward as the Cloud controller.

1. At the start of the node installation, the option "Install Ubuntu Enterprise Cloud" should be selected just like the Cloud controller installation. However, in the Select Cloud Installation Mode screen (Fig.

42), select "Node controller" as the install method.

Fig. 42: Cloud installation mode.

Note 1: Node’s hostname use: "node01.oscarcloud.com", you may change as desired, see Fig.

43.

Fig. 43

Note 2: The rest of the installation remains the same as installing the Cluster controller in Part 1.

2. When done with installation and rebooted the node back, use Ifconfig command to check network devices installed on the node controller, as shown in Fig. 44.

Fig. 44

3. Next, use the ping command to test that you can connect to the Cloud controller, as show in Fig. 45.

Fig. 45

4. We’re done with this section.

Part 3: Cloud provisioning from Ubuntu

Step 1: Logging into Ubuntu Cloud Admin console

In this section, I will show you how install the node controller. Once the Cluster controller and nodes are installed the cloud is managed via the cluster controller's IP address. The system can be managed via the

Web interface by connecting on port 8443. For example, assuming the cluster controller's IP address is

192.168.83.236, the URL http://192.168.83.236:8443 can be used to manage the cluster.

1. To access the Ubuntu Cloud Admin console, fire-up another virtual machine, in our case it’s WinXP PC, and then use your favorite browser and enter the URL: http://192.168.83.236:8443. You will be prompted for Security Alert window as shown in Fig.46. Click Yes to accept the security certificate.

Fig. 46

2. The Web login will be shown at this point as shown in Fig. 47. The default username "admin" with the password "admin" can be used for the initial login.

Fig. 47: Enterprise cloud login.

3. Once you are logged in, Eucalyptus will prompt you for the administrator's new password as well as email address and the cluster host IP address, as shown in Fig. 48. Change as desired and then click the Submit button to update the changes. (Note: we have changed to Firefox browser)

Fig. 48: First time configuration for Ubuntu Enterprise Cloud.

4. When you hit the Submit button, from Fig. 48 above, you’ll be brought to the screen shown in Fig. 49.

Notice at the moment there is only one user, who is the server administrator.

Fig. 49: Ubuntu Enterprise Cloud admin console

5. Before we do anything, we need to register Walrus, this can be found on the Configuration tab. Enter under Walrus host: 192.168.83.236, which is the IP of the cloud controller, then click the Register Walrus button.

Fig. 50

Note: The integration of UEC goes beyond the installation. The Eucalyptus Web interface supports the concept of a Store, and Extras. The Store and Extras provide similar function. They are repositories and links to the Internet for operating systems supported within the cloud. The Images tab

is initially empty, as shown in Fig. 51.

Fig. 51: Ubuntu Enterprise Cloud admin console – Images tab

Step 2: A closer look at Ubuntu enterprise cloud

The Web-based management console is made up of seven tabs: Credentials, Images, Store, Users, Configuration, Services, and Extras. The major advantage of the console over the command line is that it gives a large view of the whole environment in one location, as shown in Fig. 51.

• Credential tab: The Credentials pane contains two fundamental configuration parameters: the admin account information, and the credentials information. The admin account information is the basic authentication and authorization information needed to access the Web interface; whereas the credential information is the authentication (keys) needed to access cloud infrastructure such as private cloud resources managed by the enterprise or public ones such as Amazon's cloud computing services.

• Images tab: - Virtual machines within Eucalyptus are made up of images, and these "base" images are made up of three discrete pieces: a kernel, a ram disk, and an OS image. These images are stored under the Images tab. This images tab integrates tightly with the Store tab which provides administrators a set of four Images available for download and quick provisioning, see Fig. 52.

• Users tab: - Eucalyptus also supports additional users, which are provisioned under the Users tab.

• Configuration tab: - The Configuration tab is the major tab and is used to provision the settings for the cloud, including its IP, DNS, storage, and cluster information. This is really for changing the parameters, as the Ubuntu integrated install process provisions most of these out of the box.

• Services tab: -The Services tab is a bit misleading. It is not to start and stop services, but rather a

'help" tab. It provides links to documentation, training, and community Web sites.

Fig. 52: Store tab in Enterprise Cloud console showing images that are ready for installing.

• Extras tab: - Finally, the Extras tab contains other Linux versions as well as older Ubuntu images that will also function with the platform. The Extra tabs, also contains client tools, see Fig. 53.

Fig. 53: Extras tab in Enterprise Cloud console showing Eucalyptus-certified Images that are ready for installing.

Note: if you scroll down you’ll also see the Eucalyptus-compatible Tools which you can also install, see Fig. 54.

Fig. 54

Step 3: Set up your cloud

In this section I’ll show you how to download and install tools necessary to help you manage your cloud.

1. Download and install a tool to allow you to start and stop instances, see Fig. 54 located in the Extras tab. You can use either Elastic Fox or Euca2ools for free. Euca2ools is open source and comes with Ubuntu Server

2. Click euca2ools link from Fig. 54 to install it access the package and the guide on how to install it.

Step 4: Install an image from the store

The following is by far the simplest way to install an image. However, advanced users may be interested in learning how to Bundle their own image.

1. The simplest way to add an image to UEC is to install it from the Image Store on the UEC web interface from the Store tab, see Fig.52 above.

2. From the available images under All Images heading, locate the desired image and then click on the Install button to install it the desired image, in our case: Ubuntu 9.10 – Karmic Koala (amd64), see Fig. 55.

Fig. 55: From the UEC Store tab, click Install button to install Ubuntu 9.10 –Karmic Koala (amd64)

Note: It will take several minutes to download & complete installation, so take time-off for some coffee.

3. Once the image has been downloaded and installed, you can click on "How to run?" that will be displayed below the image button to view the command to execute to instantiate (start) this image.

The image will also appear on the list given on the Image tab, see Fig. 56.

Fig. 56: Click the ”How to run?” link.

Note: When you click "How to run?", a window will open as shown in Fig. 57. Click the Close

button when done.

Fig. 57: Information on how to run: Ubuntu 9.10 –Karmic Koala (amd64)

4. Now if you click the Image tab, should notice that we have some images, as shown in Fig. 58.

Fig. 58: Image tab, now showing some images.

5. We’re done with this section.

6. Take your time to some Google search to help you learn how to extend the knowledge that you have learned in this lab session.

Note: Shutdown the systems in this order: first the node controller followed by the cloud controller. Use the following command to shutdown: "sudo shutdown now -h".

To power up the cloud system, fire-up the cloud controller first followed by the node controller.

Step 5; Summary

The Ubuntu integrated install of Eucalyptus and the Web-based enterprise console is a welcome addition to cloud computing. It makes a consistent and quick deployment of the technology much more attainable by many enterprise shops.

09-Feb-2012, UPDATED: Those who are interested to read this article/manual offline, can download it in PDF format from this link.

References

1. Installing Ubuntu 9_10 and the Eucalyptus enterprise cloud computing platform http://www.canonical.com/projects/landscape/cloud-management

2. Landscape Cloud Management Canonical https://help.ubuntu.com/community/UEC/CDInstall

3. Private cloud Ubuntu - http://www.ubuntu.com/cloud/private-steps

Microsoft walking to open source

2009-11-20T04:06:00.002+05:30

Back in July, Microsoft announced it was making .NET available under its Community Promise, which in theory allowed free software developers to use the technology without fear of patent lawsuits. Not surprisingly, many free software geeks were unconvinced by the promise (after all, what's a promise compared to an actual open licence?), but now Microsoft has taken things to the next level by releasing the .NET Micro Framework under the Apache 2.0 licence. Yes, you read that correctly: a sizeable chunk of .NET is about to go open source.

Head to head comparison between Subversion, Git and Mercurial

2009-11-20T01:51:00.002+05:30

[scribd id=22758516 key=key-1ekb6845ynffe0mxdypi]

Malaysia and Open Source

2009-11-19T04:09:00.002+05:30

More than 70 percent of Malaysian government offices are running open source software, according to figures released by the country's Open Source Competency Centre.

The centre was established as part of the 2004 Malaysian Public Sector OSS Master Plan, to guide and co-ordinate the implementation of OSS in the public sector.

The latest OSS adoption figures, released on 24 July, show that 521 of the country's 724 public sector agencies (72 per cent) have adopted OSS. This is a significant increase from 354 agencies (49 percent) in 2008 and 163 (22.5 per cent) in 2007.

Malaysia is certainly raising the bar in terms of open source adoption and leadership!

Data recovery from corrupted ext2/ext3 filesystem having bad superblock

2009-11-18T21:28:00.002+05:30

NOTE : I do not take any responsibilty of any damage to your disk or data while trying my method or any of my commands stated in this article. YOU HAVE BEEN WARNED!!!

Let's say our corrupted filesystem is at partition /dev/sdb3 of ext3 type. We will mount the partition under /mnt/sdb3, so create the directory structure if you dont have it already.

Also, create the following directory structure to keep backup data.
```
mkdir /sdb3-backup
mkdir /sdb3-backup/image
mkdir /sdb3-backup/copy
```
Note that ext3 filesystem is same as ext2, with only addition of journal. So our entire technique will use ext2 filesystem if even our corrupted filesystem is ext3 type. Because our aim is to recover data not journal recovery (which is unrecoverable as far as I know). So be carefull while you issue any of my commands, unless explicitly told dont add any ext3 filesystem type in any of our command. Use all my command as it is written below.
Before applying this technique be sure that your superblock is corrupt only not the entire disk or something else.
USE THE INSTRUCTION BELOW TO CHECK IT

A. Try to mount the disk read only using
```
mount -t ext2 /dev/sdb3 /mnt/sdb3 -o ro
```
OR
```
mount -t ext2 -o ro,errors=recover,
errors=continue /dev/sdb3 /mnt/sdb3
```
Check the message you are getting. it must be something like below :
```
mount: wrong fs type, bad option,
bad superblock on /dev/sdb3,
or too many mounted file systems
```
B. Now try "dmesg | more" to actually verify if the superblock is really damaged. We will see a line like below :
```
EXT2-fs: unable to read superblock
```
Now we will first start to recover the data by mounting it then we will try to correct the file system.
Before you proceed further, I'd recommend taking an image, by executing :
```
dd if=/dev/sdb3 of=/sdb3-backup/image/backup
```
and then using the loop device to work on that. If the partition is bigger than 2 gig you will probably need to compress it alternatively by executing
```
dd if=/dev/sdb3 | gzip > /sdb3-backup/image/backup.gz
```
Note: If compressing the loop device doesn't work and you have to work on the bare hardware and using an alternate superblock failed too, a last dirty fix could be
```
mke2fs -S /sdb3-backup/image/backup
```
Or if the dd result is too big use
```
mke2fs -S /dev/sdb3
```
This will rewrite the superblocks, you will then need to run e2fsck to clean up. This did the trick for me on a drive that got a whole bunch of bad sectors that I had to get data off.

Hmm, if I'd only have another HDD to save the data, I’m would be happy ; to do
```
dd if=/dev/hda2 of=/dev/xyz
e2fsck -b 32768 /dev/xyz
```
e2fsck restored the first superblock and my problems are gone...but unfortunately I do not have another HDD which can hold this big chunk of data

Linux file system writes backup of superblock in different locations. Find the backup superblocks using

mke2fs -n /dev/sdb3

The above command will give you a result like below

mke2fs 1.35 (28-Feb-2004)
Filesystem label=OS type: LinuxBlock size=4096 (log=2)
Fragment size=4096 (log=2)14385152 inodes, 28754341
blocks1437717 blocks (5.00%) reserved <span class="kwrd">for</span> the
super userFirst data block=0878 block groups32768
blocks per group, 32768 fragments per group16384
inodes per groupSuperblock backups stored on
blocks:32768, 98304, 163840, 229376, 294912, 819200,
884736, 1605632, 2654208,4096000, 7962624, 11239424,
20480000, 23887872

Now please note the superblock backup blocks number at the last line and the block size(which is multiple of 1K=1024b). In our case the block size is 4096 that mean 4K (4096/1024=4)
While the block number which must be given to mount is based on the blocksize which is installed on the hard disk (4k in my case), the block number which must be given to mount is calculated on a 1k-block-basis, so I had to multiply 32768*4.

Now we will try to mount the corrupted filesystem partition. We have to tell mount command to use backup superblocks so we will calculate the first backup superblock (if it is OK) 32768*4 = 131072. Now mount with the command below :
```
mount -t ext2 -o sb=131072 /dev/sdb3 /mnt/sdb3
```
Now imagine the backup at position 32768 was damaged too . . . then you just try again with the backup stored at position 98304, and 163840, and 229376 etc. etc. until you find an undamaged backup ( there are several backups so if at least one of those five is okay. BINGO! )

Use the formula below to calculate the backup superblock location :
```
Superblock backups stored on blocks location
* Block size in K(i.e.Block size/1024) =
Nmount -t ext2 -o sb=N /dev/sdb3 /mnt/sdb3
(replace N with the result of the calculation)
```
If you successfully able to mount the partition, go to the mounted partion and copy data using
```
cd /mnt/sdb3
cp -R /mnt/sdb3/data /sdb3-backup/copy/.
```
If any of the above steps fails. You need to consult a experienced data recovery experts.

In 90% case of damaged superblock this technique of data recovery works. Hope the above technique will help you too. Please post your result if you use the technique in this article.
And at last I would like to say "ALWAYS BACKUP YOUR DATA IN MULTIPLE / DISK OR MEDIA" so if one fails you can restore from others.

Derived from my original article publish in My TechBlog

Cloud computing to drive open source

2009-11-18T03:57:00.002+05:30

With the cloud computing wave poised to reach the world market in the next 12 to 18 months, open source software and coding techniques are about to hit the big time.

That’s because open source software and the methodologies that accompany it have already been proven to be the chosen route for the vast majority of companies aiming to capitalise on the cloud phenomenon.

For evidence of this, you need look no further than the route companies such as Amazon, Google and Rackspace have taken in building out the massive datacentres they plan to begin selling capacity on in the coming years.

Without fail in each of these examples, open source is either at the core or forms a vital component of what’s on offer. And as cloud computing becomes more a prominent topic, so open source will find greater traction in the market.

The reasons open source is a popular route are not difficult to find. Since the cloud computing players are extremely technically proficient, they have sufficient skill in-house to capitalise on the more open nature of open source – and in doing so, can build a far lower cost solution than what would be on offer from a proprietary technology.

These solutions’ open nature furthermore allow cloud providers to mould and form tools to their own needs, changing and adapting underlying technology rapidly so that extra performance can be eked out of a solution.

Incidentally, cloud companies using open source technologies gain the useful side-effect of adhering to the open standards that the majority of open source solutions subscribe to. This proves to be a great benefit down the line when it comes to integrating disparate line of business systems or solutions providing specific functionality to a business silo.

A number of companies are wondering when exactly cloud computing will hit South Africa, since the topic is becoming an important part of the planning most enterprises in the more developed US or European markets are doing today.

Because of the bandwidth limitations we face locally and despite the arrival of new undersea cables, cloud computing will take on a different form in South Africa to markets where bandwidth is ubiquitously available.

My personal belief is that South African companies will become involved with cloud computing from an internal perspective, building clouds that exist inside their datacentres, but function similarly to clouds located at service providers’ offsite datacentres.

It stands to reason that these customers will need to look at open source technologies just like their outsourced peers, since the level of scalability, customisability and control is just not there in the proprietary world.

For that reason I can’t see why open source won’t go from strength to strength locally over the coming years.

A brief list of commercial grade enterprise open source solution

2009-11-18T00:57:00.002+05:30

[scribd id=22651625 key=key-f8f6yv7quw1q4r4cqd9]

Another hole in SSL

2009-11-17T04:07:00.002+05:30

Discussed in Slashdot quite few days back becomes true. Some researchers had claimed was too theoretical to worry about, has now been demonstrated by exploit. The attack description is available on securegoose.org.

The exploit by Anil Kurmus is significant because it successfully targeted the so-called SSL renegotiation bug to steal Twitter login credentials that passed through encrypted data streams. When the flaw surfaced last week, many researchers dismissed it as an esoteric curiosity with little practical effect.

For one thing, the critics said, the protocol bug was hard to exploit. And for another, they said, even when it could be targeted, it achieved extremely limited results. The skepticism was understandable: While attackers could inject a small amount of text at the beginning of an authenticated SSL session, they were unable to read encrypted data that flowed between the two parties.

Project management lessons from the FreeDOS Project

2009-11-16T04:03:00.002+05:30

A lot of people seem to think that open source is a magic solution to project management and that open source projects will automatically attract a large and healthy community of contributors and users who will improve the software. This, of course, is not the case. In fact, creating a successful open source project is a really major and difficult effort. You have to deliver an initial promise that people find interesting, attract other people, then facilitate and lead the community, etc. You just have to look at all the failed projects on Source

Forge that never delivered any code to see that "open source" is not a guarantee for success.

Even though project management is a key element of every open source project, there are only few resources about this topic. That's why I always enjoy reading about the experience from open source project leaders. Jim Hall, the founder of the FreeDOS project, recently posted a series of four articles which I find particularly interesting.

Here are links to the articles along with a quick summary:

Free and Open Source Software: the first article just describes what free software and open source are all about. It also looks at Raymond's model and compares the cathedral and the bazaar style of development.
Open source software in the real world: this article summarizes a number of key lessons learned: every project needs a clear purpose (i.e. you have to solve an actual problem); initial users of the software should be recruited as developers; releases are important; and every project needs an active coordinator or maintainer.
Cultivating Open Source Software: you need a web site; making the source code availability in an easy way is important; documentation is often hard to write but is vital; you need a bug tracking system and responding to bug reports is important to attract good feedback.
Transitions in an open source software project: finally, when you need to hand over the project, make sure to communicate openly, arrange for your replacement and stick around to ensure a successful hand over.

I really like these articles from Jim Hall since they contain a lot of great insights that apply to other projects, so I suggest you check them out!

(Originally published on FOSSBazaar)

What's waiting in upcoming KDE 4.4

2009-11-13T04:05:00.002+05:30

If you're wondering what the folks over at KDE have been cooking up for the next major release, KDE 4.4, well, quite a bit as it turns out. In a lengthy interview, KDE core developer and spokesperson for the project Sebastian Kugler details the myriad changes that are coming with the 4.4 release — the fifth major release since KDE 4.0 debuted to much criticism nearly two years ago. The project has closed about 18,000 bugs over the past six months and the pace of development is snowballing. The 'heavy-lifting' in libraries and frameworks for 4.0 is now starting to pay off. Perhaps the biggest change is in the development of a semantic desktop. According to Kugler, 'If you tag an image in your image viewer, the tag becomes visible in your desktop search. That's how it should be, right?' There is also a picture gallery of KDE 4.4 (svn) screenshots so you can see what it will look like.

100 Million-Core Supercomputers Coming By 2018

2009-11-05T04:04:00.002+05:30

As amazing as today's supercomputing systems are, they remain primitive and current designs soak up too much power, space and money. And as big as they are today, supercomputers aren't big enough — a key topic for some of the estimated 11,000 people now gathering in Portland, Ore. for the 22nd annual supercomputing conference, SC09, will be the next performance goal: an exascale system. Today, supercomputers are well short of an exascale. The world's fastest system at Oak Ridge National Laboratory, according to the just released Top500 list, is a Cray XT5 system, which has 224,256 processing cores from six-core Opteron chips made by Advanced Micro Devices Inc. (AMD). The Jaguar is capable of a peak performance of 2.3 petaflops. But Jaguar's record is just a blip, a fleeting benchmark. The US Department of Energy has already begun holding workshops on building a system that's 1,000 times more powerful — an exascale system, said Buddy Bland, project director at the Oak Ridge Leadership Computing Facility that includes Jaguar. The exascale systems will be needed for high-resolution climate models, bio energy products and smart grid development as well as fusion energy design. The latter project is now under way in France: the International Thermonuclear Experimental Reactor, which the US is co-developing. They're expected to arrive in 2018 — in line with Moore's Law — which helps to explain the roughly 10-year development period. But the problems involved in reaching exaflop scale go well beyond Moore's Law.