Clogeny Technology - Blog

Citrix's CloudStack 3.0 (Acton release) - Advanced Zone Setup

noreply@blogger.com (Clogeny Technology) — Tue, 13 Mar 2012 08:51:00 +0000

CloudStack 3.0, a product by Cloud.com (recently acquired by Citrix), implements IaaS (Infrastructure as a service) style private clouds. It is an open source product for the deployment and management of large networks of virtual machines to create a scalable cloud computing platform.

Recently, in one of our projects we needed a Cloudstack 3.0 setup with advanced mode networking options. Cloudstack has two options of networking setup viz basic and advanced. Setting up Cloudstack in basic mode is pretty simple! But, for setting up Cloudstack with advance mode one needs a clear understanding of internals of networking in advanced mode. This post will cover most of the concepts required along with step by step guide of advanced mode Cloudstack setup.

CloudStack 3.0 basic mode does not have VLAN enabled i.e. all instances created under basic mode have a public IP assigned. Users outside the platform can access these instances directly using the public IP associated with those instances. Also, basic mode doesn’t provide flexibility in networking. Setting up advanced mode provides such flexibility. In advanced mode, admins can customize networking options such as VPN access, load balancing, firewall and port forwarding for particular account. Virtual routers (setup using VLANs, refer diagram) in advance mode serves as abstraction for all the instances associated with that account and helps in routing the network traffic.

Lets first discuss the common terminology used in CloudStack setup -

Common terminologies:

Hosts:

Hosts are basic physical blocks of the Cloudstack platform. A host can be single XenServer or ESX server. The number of guest virtual machines that can be hosted on Cloudstack can be determined by number of hosts and capacity of each host. Hosts are not visible to the end user. An end user cannot determine which Host their guest has been assigned to.

Clusters:

Clusters are second level of physical scaling in Cloudstack platform. Cluster is a group of hosts (for Ex. XenServer or VSphere) that have same hypervisor type and share the primary storage. Size of cluster is limited by underlying hypervisor type. In a particular cluster, instances can migrate from one host to another.

Clusters can be Cloud Managed or Externally Managed. For Ex. ‘Xenserver’ based clusters are ‘Cloudmanaged’ however, for ‘VMWare’ based clusters are managed by vCenter servers.

Pods:

Pod is a collection of different types of clusters. It is a third level of physical scaling in CloudStack platform. The pod may contain only one cluster or multiple clusters with different base hypervisor type. Each node in pod shares the secondary storage and network. A Pod is frequently mapped to a single rack with a layer-2 switch.

Zones:

Zones are fourth level of physical scaling in Cloudstack platform. Zones can also provide physical isolation from other zones. Users have option to choose zone while deploying their guest VMs. Admins can setup private zones which is only accessible to specific domain.

Primary storage:

Guest VM’s root disks and other additional data disks are stored on primary storage. Instances in a particular cluster have same primary storage for them. Speed of primary storage directly impact guest VM performance.

Root volumes are created when guest VM is created on Cloudstack platform. Additional data disks can be added later or they can be added while guest VM deployment. Root disks are deleted when guest VM is destroyed. but, data disks are not deleted from primary storage when VM is destroyed which is a main difference between two types.

Secondary Storage:

Secondary storage is used for storing templates, ISO images and snapshots on cloudstack platform. Submissions to secondary storage go through the Secondary Storage VM (System VM). The Secondary Storage VM can retrieve templates and ISO images from URLs.

Each zone can have multiple secondary storage devices of type NFS added to them.

Different types of Networks in CloudStack platform

Before we start discussing the setup & configurations for advance mode, lets first understand the types of network used in Cloudstack platform. CloudStack has three networks - Public, Private and Link local network.

Every account has a internal network associated with it called a “link local network”. All guest VMs associated with the account communicate on this network and guest VMs on two different clusters (for Ex. XenServer and VSphere) communicate on a private network. Through public network, other users can access the CloudStack resources. [refer diagram below]

Public network

Users of Internet can access Cloudstack resources through Public network. For every account source NAT is created and a public IP is allocated to that network. Resources belonging to this account, if public, are accessible through the public IP address. Multiple public IPs can be assigned to single resource.

Private network

Two instances on different hosts communicate with each other using private network. For Ex. The client has instances on two different hypervisors- ‘Xenserver’ and ‘VMWare’ and if these instances want to communicate in between them then they can communicate on private network.

Link local network

The Management Server automatically creates a Virtual Router for each guest virtual network. A virtual router is a special virtual machine that runs on the Hosts. This router communicates with other resources of the account on link local network as shown in diagram above.

We will now discuss how to setup Advance zone with VLANs-

Setting Up Zone in Advanced mode

Advanced mode provides the most flexibility in allowing administrators to provide custom network offerings such as providing Firewall, VPN, or Load Balancer support as well as enabling direct vs virtual networking. Advanced mode of CloudStack setup can be setup in two modes - with VLANs or with Security groups.

Step 1: Login to admin console.

Step 2: Navigate to menu: System > Physical Resources from let side bar menues

Step 3: Select Add zone and choose ‘Advanced’ mode. Click next.

Step 4: Enter name and other details provided by your network provider.

Step 5: In CloudStack 3.0, for every NIC present on management server, a separate physical network is should be created. If the management server has only one NIC, click next else add additional physical networks.

Step 6: CloudStack’s resources like virtual machines are accessible to outside world through domain router. Every domain router is associated with a public IP which is picked from pool of IP addresses provided at time of zone setup. Add appropriate public IP range for CloudStack setup.

NOTE: If VLAN ID is specified, all the CloudStack requests will be tagged with that VLAN ID. For communication with CloudStack’s resources, requests should be tagged with VLAN ID specified. If VLAN ID is not specified, outside resources can communicate with CloudStack without any restrictions.

Step 7: The next step is to add pod specific settings. The reserved System IPs are used for creation of private network.

Step 8: Enter the VLAN Range which will be used by CloudStack for it’s internal communication. A separate VLAN range should be specified for every physical network created in step 5.

Step 9: Add Cluster, host configurations with primary and secondary storage servers. Click next. CloudStack will create physical network, pods and clusters inside new zone. System VMs will come up after successfully adding host, primary and secondary storage in the zone. Wait for their state to change to ‘Running’ state.

Step 10: Check status of BUILTIN templates from templates menu. If they are in downloading state, wait till the downloading is complete.

That’s it! You have setup a Cloudstack 3.0 with advanced networking options and now you can spawn your first VM instance.

Clogeny sponsors CloudCamp Pune, India 2011

noreply@blogger.com (Clogeny Technology) — Tue, 15 Mar 2011 09:14:00 +0000

CloudCamp is an unconference where early adopters of Cloud Computing technologies exchange ideas. With the rapid change occurring in the industry, we need a place we can meet to share our experiences, challenges and solutions. CloudCamp, encourages End users, IT professionals and vendors to participate and share their thoughts in several open discussions. Cloudcamp organizes there premier events in cloud computing across the world .

Cloudcamp now comes to Pune,India !! Yes there is a Cloudcamp un-conference being held in Pune on 5th Feb 2011. We highly encourage everyone even remotely interesting in cloud computing to attend this event. Register for this event now as the seats are filling quickly!!!

While CloudCamp has been sponsored by the likes Amazon, Google, IBM, salesforce.com and Microsoft globally, they have found local sponsors like Persistent Systems and Trillion IT Solutions as well.

Clogeny now joins this elite list by sponsoring the CloudCamp event being held in Pune,India on 5th Feb 2011 !!!

Booting A Custom Kernel in EC2

noreply@blogger.com (Clogeny Technology) — Tue, 15 Mar 2011 09:12:00 +0000

Amazon EC2 has recently released PV-GRUB loader supported kernels that allow one to boot their
kernels. This PV-GRUB loader simply chain-boots the kernel provided in the associated AMI (Amazon Machine Image). This results in your instance running the kernel in the AMI instead of the kernel specified in the boot process. This is hugely helpful feature for folks who want load their own customized kernels into EC2's virtual machines. This article talks about we successfully booted a customized(extra patches) RHEL/CENTOS 5.5 kernel on EC2:

Pre-requisites:
1. Amazon EC2 account (obviously!)
2. Knowledge of running EC2 instances/bundling EC2 images using ElasticFox or command line ec2 tools.
3. The kernel you want to build.
4. The patches you want to apply.

NOTE: You'll might think that, wow! with the feature i can boot any damn kernel in the world.
Well chances are you can but what Amazon says is that they are 100% sure certain these kernels (mentioned below) definitely boot, rest need to test their luck:

• Fedora 8‐9 Xen kernels
• Fedora 13 (2.6.33.6‐147 and higher)
• SLES/openSUSE 10.x, 11.0, 11.1 Xen
• SLES/openSUSE 11.x EC2 Variant
• Ubuntu EC2 Variant kernels
• RHEL 5.x kernels
• RHEL 6.x kernels
• CentOS 5.x kernels

Now we needed to boot a custom RHEL/CENTOS 5.5 (which is supported above).
Here are the steps we followed:

1. Find the right centos 5.5 AMI: That’s easy! We used ami-8737deee.

2. Choosing the right kernel (AKI):
Now this is a very important step. Choose the right kernel. If you run this AMI with a default settings you get a VM booted with 2.6.21-based kernel. The RHEL5 kernel is 2.6.18-ish based. The problem then is finding the right kernel configuration that ensures that your kernel first compiles and then boots. Since it is highly recommended that you'll choose an kernel (aki-xxxxxx) that is close to the kernel you want to actually boot. In this case we chose, aki-2a42a043 which is 2.6.18 based EC2 Kernel

3. Run the instance with this AMI/AKI configuration either through ElasticFox or the command line.
4. Download your kernel, apply the patches, build the configuration

   1: wget linux-2.6.18-194.17.1.el5.tar.bz2 #RHEL5 kernel   2: tar -xf linux-2.6.18-194.17.1.el5.tar.bz2    3: cd linux-2.6.18-194.17.1.el5.tar.bz2    4:  ln -s patches /patches   5: quilt push –a # quilt is useful for pushing a series of patches, else you can use normal patch    6: command   7: cp /proc/config.gz .# This ensures the right configuration is used to build the    8: kernel   9: gunzip config.gz  10: mv config .config  11: make oldconfig  12: make -j4 && make modules && make install 

5. Build your ramdisk

   1: mkinitrd --builtin=ehci-hcd --builtin=ohci-hcd --builtin=uhci-hcd /boot/initrd-    2:   # The builtin option is required to tell mkinitrd that these modules are    3: built-in the kernel itself.

6. Create a /boot/grub/menu.lst with the following content:

   1: cat /boot/grub/menu.list   2: default 0    3: timeout 3    4:     title EC2    5:     root (hd0)    6:     kernel /boot/vmlinuz- root=/dev/sda1    7:     initrd /boot/initrd- 

7. Bundle the image, upload and register the image.

While bundling the image, we need to choose the right PV-grub kernel with which we need we want to load the kernel. That information is available here . We choose the aki-4e7d9527 for booting a 64-bit non-EBS kernel. You choose what you want as per your need.

   1: ec2-bundle-vol -r x86_64 -d /mnt –e /mnt,/root/.ssh -p  --kernel aki-4e7d9527    2: ec2-upload-bundle –b     3: ec2-register /

8. Voila! We are done!

ec2-register provides you with the AMI ID. Launch another instance with this AMI and you should see
your new kernel (with your patches/modules) loaded.

Potential problems:

ec2-bundle-image complains about lack of loopback devices.
Solution: Load a loopback device

   1: insmod /lib/modules/2.6.18-xenU-ec2-v1.0/kernel/drivers/block/loop.ko 

2. The PV-GRUB cannot find the kernel to load.
Solution: Ensure that in the menu.lst root=/dev/sda1 and root (hd0) is set correctly.

3.    Chances are your kernel does not compile at all or fails to boot for mysterious reasons
          Solution: Well, there isn’t a silver bullet for all such problems. We faced multiple compilation issues
          which we fixed on our own.

Incase you cannot get through these issues and need help respond to the comment sections of the blog or
feel free to mail me at chirag at clogeny dot com

References: