Gerard Braad

Editing documentation using Jekyll and Gollum

2012-11-04T13:16:00.000+01:00

Currently migrating some technical material and my blog to the static site generator Jekyll. This way I can add articles using a simple text editor to write the markdown (or other supported format) and easily publish it using git. However, I also wanted to have the ability to author pages from a wiki tool like Gollum. So I looked into it and came up with a simple solution: 'generate a list of available pages for the index.html'.

An example is published on http://github.com/gbraad/articles. You can still edit pages using http://github.com/gbraad/articles/wiki/_pages and the generated pages are published at http://gbraad.github.com/articles/. The following problems exist: you have to keep the project-wiki in sync with the project repository when using GitHub (not really needed if you use Gollum locally), manual step needed for publishing of the gh-pages branch (not a big deal if you control the post-receive on your own server) and no directory nesting is possible for articles (do not really miss this since I use a repository per document).

There is still room for some improvement, but for now it allows me to easily work on teaching material and ePUB documents.

Simple Bugzilla client

2011-04-23T12:17:00.008+01:00

At the moment I work as an IT Consultant on a QA-related project in Beijing at an outsourced partner. As you might know, the Internet connectivity in China is not optimal and this caused some issues on our end. Since we have to do a lot of cross-checking, we needed a way to keep track of remotely reported bugs and store these in a local instance of a previously in-house developed cross-check tracker. I started to develop a simple client to pull information from the remote Bugzilla instance which is then stored locally. This solved most of the issues we encountered with the latency and improved the productivity of our team.

XML-RPC client
The local tracker is built in PHP, so at first a small prototype was built to see if it was possible to call Bugzilla's XML-RPC interface. To ease the development I chose to use the standard ZendFramework library and the XML-RPC client they offer, Zend_XmlRpc_Client. The biggest issue I encountered was the way authentication was implemented on the remote side, but the actual client was simple and very easy to implement. The prototype was first done in a very sequential way... and eventually after a little bit of refactoring this is what remained:

// Add the AutoLoader
require_once('Zend/Loader/Autoloader.php');
Zend_Loader_Autoloader::getInstance();

class BugzillaClient
{
  // Initialize the client
  protected $rpcClient = null;
  protected $basUrl = null;

  public function __construct($baseUrl, $username, $password)
  {
       $this->baseUrl = $baseUrl;    // Store internal for later use
       $this->rpcClient = new Zend_XmlRpc_Client($baseUrl . "/xmlrpc.cgi");

       $httpClient = $this->rpcClient->getHttpClient();
       $httpClient->setCookieJar();  // Needed to retain user cookie
       $httpClient->setAuth($username, $password, Zend_Http_Client::AUTH_BASIC);

       // Login request (XMLRPC)
       $response = $this->rpcClient->call('User.login', array(array(
           'login'    => $username,
           'password' => $password,
           'remember' => 1
       )));
  }

  public function __destruct()
  {
       // Logout request (XMLRPC)
       $response = $this->rpcClient->call('User.logout', array());
  }

  public function Get($id)
  {
       $returnValue = array();

       // Get request (XMLRPC)
       $response = $this->rpcClient->call('Bug.get', array(array('ids' => $id)));
       $index = 0; // Only retrieving one bug, so don't bother with index now

       $returnValue['id'] = $id;
       // Construct a remote URL
       $returnValue['url'] = $this->baseUrl . "/show_bug.cgi?id=" . $id;
       // Collect some basic information about the bug
       $returnValue['summary'] = $response['bugs'][$index]['summary'];
       $returnValue['status'] = $response['bugs'][$index]['status'];
       $returnValue['resolution'] = $response['bugs'][$index]['resolution'];

       // Comments request (XMLRPC)
       $response = $this->rpcClient->call('Bug.comments', array(array('ids' => $id)));
       // Extract the first comment from the bug and store as description
       $returnValue['description'] = $response['bugs'][$id]['comments'][0]['text'];

       return $returnValue;
   }
}

This code provides a constructor which does the login and a 'Get' function to query the remote instance.

Example
To use this client you just need to include this module and create a new BugzillaClient instance.

require_once('config.php');          // Contains the baseUrl and credentials
require_once('BugzillaClient.php');  // This is the client code

$bzClient = new BugzillaClient($bzBaseUrl, $bzUsername, $bzPassword);
$bug = $bzClient->Get($id);

echo $bug['summary']);

After this the summary will be printed.

Future work
The last time I did some actual development with PHP was during PHP 3 and the move to PHP 4. At that moment I was not so happy with the Object Orientation model that the language offered. Luckily this improved a lot and hope this code testifies of that... as I have seen some horrific spaghetti code during these several days.

Using this solution does not provide a complete up-to-date picture of the bug, but this is not a problem to us. Besides, triggering a script which iterates over all local bugs and queries the remote state solved this issue. Some more features will be implemented, like searching for bugs and pushing comments back. This basic implementation code has been posted to public git repositories, so fork it and start pushing your improvements.

Repositories

How-to setup dual-booting MeeGo 1.1 on the N900

2010-11-03T14:25:00.009+01:00

The current image of MeeGo is for development purpose. Do NOT try to use it for daily use. You can make a phonecall or send a message, but I already had some issues with not being able to pick up phonecalls which I receive. It should be used as a means for developers to get familiar to the MeeGo environment for use on a handset. Also see the note at the end of this article.

For this how-to I use Fedora 13, but tmost of the steps also apply to any other Linux distribution. Be sure you have downloaded the correct MeeGo images before you begin and that you have a MicroSD card of 2G or more. I used a 4G MicroSD card from SanDisk.

The kernel root filesystem you download have to download from: http://repo.meego.com/MeeGo/releases/1.1/handset/images/meego-handset-armv7l-n900/

You need the following two files:
meego-handset-armv7l-n900-1.1-mmcblk0p.raw.bz2 (521M)
meego-handset-armv7l-n900-1.1-vmlinuz-2.6.35.3-10.3-n900 (1.5M)

N900
Enable the extras-devel repo on your device and install from app-manager: uboot-pr13
or from the terminal with:

 # root
$ apt-get install uboot-pr13

Be sure you have upgraded to PR1.3 before doing this. This is all that's needed on the mobile handset.

Workstation
On Fedora you have to install the uboot-tools to create a bootable kernel image. For writing the disk image I also included pv to have some progress indication. Install these tools with:

$ yum install uboot-tools pv cfdisk

Connect your N900 in Mass Storage mode with the MicroSD card inserted or use a multicard reader. As mentioned before, use a 2G or more MicroSD card to dump the raw image to. You have to find the correct device to use from e.g. the dmesg, replace sdX with the correct device. The command for this is:

$ bzcat meego-handset-armv7l-n900-1.1-mmcblk0p.raw.bz2 | pv | sudo dd bs=4096 of=/dev/sdX

Note: This may take some time, especially on the class of SD card you are using. I used just a class 2 4GB from SanDisk. Also be sure it is not mounted during this action. Some distributions use auto-mounting for Removable Media.

To make the MicroSD card usable for dual-booting you have to add a third partition. This might be a little more tricky if you haven't done this often. That is why you can use cfdisk.

$ cfdisk /dev/sdX

And add a new partition in the last freespace you see. The first is only 1.3MB, which is TOO small to contain the kernel image. Define it as a primary partition of 4MB (or full size). Take note of the device name. It should be sdX3.

Now create a FAT filesystem on the newly created partition. This is where your kernel needs to be placed.

$ mkfs.vfat /dev/sdX3

Now you need to prepare the kernel image for use on the SD card.

$ mkimage -A arm -O linux -T kernel -C none -a 80008000 -e 80008000 -n meego-handset-armv7l-n900-1.1-vmlinuz-2.6.35.3-10.3-n900 -d meego-handset-armv7l-n900-1.1-vmlinuz-2.6.35.3-10.3-n900 uImage

You would now have a file called uImage which you need to place in the FAT filesystem on /dev/sdX3. If you have difficulty creating the file, you can download it from here. Be sure you rename it to uImage. Temporarily mount it or reinsert the SD card so auto-mount can deal with it. Copy the uImage file to the root and unmount.

Now you could use this MicroSD to boot MeeGo 1.1. Try it out by powering off your N900 phone, remove the backcover and place the SD card. Be sure you have placed the backcover before you power on the device. If you don't do so, the root filesystem can not be mounted and the device will show a kernel panic.

External links

Note
I do not take any responsibility in you damaging your device or loosing data during the process. As I had mentioned, it is intended for development use and therefore I hope you have some basic understanding of what you are doing. If you have any question, please post them in the MeeGo Handset forum.

Fedora visits China (Beijing/Shanghai)

2010-08-02T13:45:00.002+01:00

A lot of things have happened in China relating to Fedora; our MIPS port gained attention from Loongson and we made Fedora 13 available on our servers. Although it has not been mirrored yet to secondary.fedoraproject.org, you can already try it out. The other thing is about what Kaio and I have been busy with for the last half year, the Chinese Fedora Community!

It seems our efforts have not been unnoticed and the Fedora Community Architecture team sent Mel Chua to China. Yesterday she arrived in Beijing and we immediately started to discuss about our community and future activities and how to gain a more widespread acceptance of Open Source in China.

One of the things we immediately agreed upon was, luckily her explanation was less morbid, to 'Raptor-proof our community'. Our community is fragile when it comes to responsibilities and the load carried upon those shoulders. For this to succeed we would like you to propose people or yourself for a role within the community.

It would be ideal if some of you would step forward and become a regional ambassador who can represent key regions in China, like Shanghai, Hong Kong, etc. The workload for this is low and when we can balance it out it would even be minimal. You would take care of distributing Fedora swag, like t-shirts in your area.

This Raptor issue also troubles our MIPS port. Currently we have three members and of this only the lead role can be performed by someone else. We have a nice group of people around us, but we are always searching for new members. If you want to help with documenting or other activities, just get in touch. No, you don't need to be Chinese :-P. If you have a Yeeloong netbook and got tired or Mandriva, help us test and build third-party packages.

We would really like to know about Open Source activities happening all over China or maybe you have suggestions. If you want to discuss this with us, join #fedora-zh and mailinglist. Mel's agenda for this week is still open, so if you want to meet her in Shanghai, this is your chance.

Thanks to all the members of the Chinese Fedora Community!

Installing SELinux on Linode (CentOS profile)

2010-06-22T07:42:00.003+01:00

Recently I signed up for a different VPS hosting partner as the one I was using had a dreadful latency when dealing with international connections. The new VPS at Linode I am using has good ping times, the control panel often does correctly what it should do, but the out-of-the-box CentOS profile was in my opinion not what I am used to.

RHEL and Fedora sys-admins know about SELinux and the pains and pleasures it can induce. If you are new to SELinux, be sure to read a little bit of background: SELinux for dummies and Dan Walsh's Blog. Or watch these videos recorded by Scott Dowdle: "SELinux demystified", "Intro to SELinux" Part 1, Part 2.

It seems Linode does not provide SELinux support with their provided kernel. I do not know what their motivation is. Maybe they want to relieve their support of troubling questions (to which I would say: permissive or enforcing=0) or maybe they just don't see the need. However after reading several reports on the web I am sure people want this functionality, but have been troubled by out-of-date resources about recompilation of the kernel to get the needed SELinux support and paravirtualization operations (pv_ops), setting up pv-grub, etc.

Well, things are now much simpler as RHEL/CentOS provides the needed pv_ops kernels. The only missing part is to setup grub and get SELinux going. In several steps I will explain what needs to be done.

Instructions
For these instructions I created a standard CentOS 5.5 (64bit) profile VPS using Linode's Dashboard option 'Deploy a distribution'. Start the node and from the console or ssh perform these steps.

1. Install grub
Grub is not installed on the base system. You will need to install the package, create some symlinks and an empty configuration file which you will need later.

# yum install grub
# mkdir /boot/grub
# cd /boot/grub/
# touch grub.conf
# ln -s ../boot/grub/grub.conf /etc/grub.conf
# ln -s ./grub.conf menu.lst

2. Install kernel with pv_ops
You will need to install a Xen enabled kernel which are provided by CentOS. But for some reason the xennet and xenblk are provided as modules and fail during startup. To solve this issue we create a new initrd which can load these. Be sure to specify your kernel version correctly. You will also need this later for Grub.

# yum install kernel-xen
# mkinitrd -v -f --with=ext3 --with=xennet --with=xenblk /boot/initrd-2.6.18-194.3.1.el5xen-custom.img 2.6.18-194.3.1.el5xen
# echo "devpts    /dev/pts    devpts    defaults  0 0" > /etc/fstab

3. Edit grub.conf
To enable the new kernel, you need to add this to our empty grub.conf. Be sure to verify your version.

# vi /etc/grub.conf

The contents of the grub.conf are:

#boot=/dev/vxda
default=0
timeout=1
hiddenmenu
title CentOS
root (hd0)
kernel /boot/vmlinuz-2.6.18-194.3.1.el5xen ro root=/dev/xvda
initrd /boot/initrd-2.6.18-194.3.1.el5xen-custom.img
title CentOS (enforcing=0)
root (hd0)
kernel /boot/vmlinuz-2.6.18-194.3.1.el5xen ro root=/dev/xvda enforcing=0
initrd /boot/initrd-2.6.18-194.3.1.el5xen-custom.img

The above configuration will automatically boot to the default option which is a pv_ops kernel which has SELinux enabled. The second boot option is handy of you encounter some issues during the next step. If you are uncertain, you can uncomment the following lines, but this means the boot will be interrupted by grub and needs you to select the boot option from the lish console.

#default=0
#timeout=1

4. Install SELinux
Installing SELinux now is a breeze.

# yum install libselinux selinux-policy selinux-policy-targeted
# genhomedircon
# touch /.autorelabel
# shutdown -h now

Now you only need to set your node to use pv-grub to boot from the configuration profile. This is briefly explained in this article on the Linode Library. Boot up your node and allow some time for the relabeling. It might be wise to verify the progress from the lish console. If everything went well, the filesystem should have been been relabeled after a while and the system should operate 'as normal' ;-). You can verify the state of SELinux from ssh using the command sestatus.

There is however still one issue I have not resolved, as soon as the console is handled by mingetty, it seems the lish console does not properly handle this. It might be an issue to use xvc0 instead, but I see this as a minor problem and rather have SELinux enabled.If you have a fix, please let me know so I can improve these instructions.

Chinese Fedora Community and Fedora-MIPS Port

2010-05-08T21:14:00.007+01:00

Currently I'm getting ready to move to Beijing (北京), China (中国) in June. While I am still looking for employment I have also been busy with assisting the Chinese Fedora Community (FZUG/中文用户组). As part of some our activities I will give presentations about Fedora and the community at the BeijingLUG. It is a growing community... but we are still looking for new users, ambassadors and contributors. We would also really like to work together with other other user groups in the APAC. If you are Chinese, in China or interested, drop by in our IRC channel #fedora-zh on freenode or get in touch with me (吉拉德, gbraad), Caius Chance (kaio) or Yuan Yijun (袁乙钧, bbbush). We are looking forward to seeing you!

Part of building the Fedora brand, some friends of the Chinese community and I started to port Fedora to the MIPS architecture. MIPS is an architecture that gained a lot of attention recently with the release of the Loongson processor. This processor is Chinese made and ended up in several netbooks and mini-pc's. While they were only available on the Chinese market, they are now being sold in America and Europe, e.g. by the Freedom Included initiative.

Currently we have chosen to target the Loongson 2F, but in time we will also support other MIPS-compatible processors. The primary goal of this project is to provide support for MIPS as a secondary architecture in Fedora. Just like the ARM port, our secondary goal is to enable derivative distributions based on the Fedora package collection and repository that are more suitably optimized for embedded and mobile use-cases. For this we want to work together with the Mini SIG as we can both benefit from this.

We now have a test build available of Fedora 12 (as n32 mipsel) which should be able to run on Qemu or MIPS-based hardware, like the Fuloong. While we are in the process of fixing some remaining build and technical issues and submitting the patches, you are invited to help us test this release. The Fuloong is currently the platform we build and test on ourselves. Be warned, expect to do some DIY especially since no documentation is available yet... and your mileage may vary.

The team currently consists of me (project-lead), FaiWong (technical lead, lazyfai) and XinZhen (co-lead, lonestar). If you want to contribute or donate hardware, please get in touch with us. More information can be found on the Fedora-MIPS wiki page, or just drop by in our IRC channel #fedora-mips on freenode.

Migrating from VMware to KVM

2009-12-14T00:22:00.005+01:00

A lot of reports on the internet talk about the reduction of costs when you invest in virtualization. This is something I will certainly not deny after many years of experience with VMware Workstation, VMware Fusion, VMware GSX (now VMware Server) and VMware ESX and having implemented several high-availability setups using ESX. Even at home I had several whitebox servers with Virtual Center... until I migrated away to KVM. In this article I will explain the reason of the migration and how to perform it with an example. It is written towards people who currently run VMware GSX or VMware Server on Linux.

Motivation
It is undeniable that virtualization provide you with a lot of benefits; consolidation of servers will save you money. Maintaining your virtualized servers becomes a breeze, since they are all accessible from a single front-end. But implementing your environment on VMware comes at a price... even for the free version VMware Server; maintenance. You will still need to monitor your virtualization servers for performance and distribute your virtual machines among your servers or resource pools according to load. Although there are tools available to assist you, it is not the biggest issue.

VMware Server requires an additional operating system like Linux (or Windows). This introduces an added maintenance dependency; kernel modules which VMware uses need to be recompiled when the kernel gets updated. If a security exploit is found in the kernel, the virtualization host can be compromised and therefore jeopardize the continuity of your business. To prevent this, you update the kernel, but on reboot you notice that VMware does not start properly. On the command line you will need to perform a vmware-config.pl which build the modules. Unfortunately, your kernel headers are newer and either vmmon (virtual machine monitor) or vmnet (network component) does not build properly. Because of this, your whole infrastructure is offline until VMware provides new components or the release of the community patch package 'vmware-any-any'.

Users of VMware ESX will not have this issue, since the operating system and modules are provided in the form of the VMware's vmkernel. If a security update is needed, you only need to install an update. The previously mentioned issue has happened to me often enough and I I wanted to have the same solution as ESX, but with the convenience of using Un*x/Linux experience for maintenance without the dependency of a third-party product. Since RHEL 5.4 this is provided as an enterprise supported solution; KVM, the Kernel-based Virtual Machine, once developed by Qumranet and now owned by RedHat. And by using CentOS this is available to everyone without too much trouble.

There are some differences between VMware and KVM. KVM is provided as part of the kernel, so any Linux distribution can provide it as part of their offering. KVM supports different formats for a virtual hard disk, while VMware only uses their own VMDK format. VMware can run in a full virtualized mode and is therefore suitable for older hardware. KVM needs a processor that provides VT-x (Intel) or AMD-V (AMD) to operate. VMware can run x86_64 guests on a i686 host, while KVM can not.

Because of this last reason, I had to reinstall CentOS on one of my systems, since it still ran an i686 version of CentOS 5.4 (on a Core2Duo E6600). This provides a good overview of what needs to be done to run KVM on CentOS.

Note: since KVM uses qemu to emulate any hardware you can also choose to use qemu without KVM as the hypervisor, but instead use the kqemu kernel module.

Installation

You will need to download an installation disc of CentOS 5.4. Prefer to use the x86_64 version; CentOS-5.4-x86_64-bin-DVD.iso. During the installation you can keep the default settings and when asked for the 'additional tasks' you will perform, do not select the 'Virtualization' option. After the installation finished, you will need to update the system to the latest updates.

From the reinstall I noticed a minor error in the CentOS 5.4 x86_64 installation. Due to installing openssl.i686 it also installed a lot of other standard i386-arch libraries. You can find out if you also have those packages installed as described in a earlier post. If you also want to remove those packages, perform the following command line option before continuing.

$ rpm -e `rpm -qa --qf '%{name}.%{ARCH}\n' |grep i386` openssl.i686

To update your system:

$ yum update

If your system is up-to-date, continue to install the needed packages for KVM and Virt-Manager.

$ yum install bridge-utils kvm kvm-qemu virt-manager virt-viewer python-virtinst

In later versions of CentOS it might suffice to issue the command "yum groupinstall 'Virtualization'", but currently this installs a Xen enabled kernel. According to the used processor, you can test to load the needed kernel module and start the libvirt daemon.

$ modprobe kvm
$ modprobe kvm-intel (or kvm-amd)
$ service libvirtd start

I normally use a different management workstation. And as an example I use a Fedora 12 installation to connect to the virtualization host (beibei). First from a tunneled connection to the host (with X11 forwarding).

$ ssh -X root@beibei
root@beibei's password: **************
# virt-manager

This will start the Virtual Machine Manager from which you can create new instances. You can of course use a Gnome desktop from the CentOS system to manage your virtual machines, but it pays off to try a seperate management workstation. This will allow you to try additional features, like live migrations between KVM virtualization servers.

This is all it needs to install KVM on a CentOS 5.4 installation. For an earlier version of CentOS, it is best to consult the wiki article about KVM.

Network settings
A default VMware Server installation will bridge the network card you have. You will probably want to have the same configuration for your KVM environment. For this you need standard Linux bridging utilities.

$ yum install bridge-utils

You only need to edit (or create) some network-scripts to make a bridge work.

$ vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
 HWADDR=00:1a:2b:3c:4d:5e
ONBOOT=yes
BRIDGE=br0

And create the bridge

$ vi /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0
 BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Bridge

When you start the network again, you will have a br0 and eth0 device.

$ service network restart

You will now be able to select a bridged network card inside the KVM virtual machine configuration. If you have more network card, create a bridge per network card.

Converting the virtual disk files
For the migration part I choose a virtual machine who once began it's life as a physical machine; forum.survion.net. It is a Linux installation once based on FC3, got migrated to CentOS... and then ran as a VMware virtual machine.

The VMware virtual machine consists of one or more hard disk files, with the extension .vmdk and a description file, with the extension .vmx. A listing of server-forum.vmwarevm looks like:

[root@beibei server-forum.vmwarevm]# ls -al
total 29405144
drwxr-xr-x  2 root users        4096 Dec 12 20:53 .
drwxr-xr-x 21 root root         4096 Dec  9 17:32 ..
-rwxr-xr-x  1 root vmware       8684 Dec  7 21:09 nvram
-rw-------  1 root vmware          0 Jan  3  2009 Red Hat Linux.vmsd
-rwxr-xr-x  1 root vmware       2369 Dec  7 21:09 Red Hat Linux.vmx
-rw-r--r--  1 root vmware        268 Jan  3  2009 Red Hat Linux.vmxf
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:22 storage-f001.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:20 storage-f002.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:21 storage-f003.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:20 storage-f004.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:22 storage-f005.vmdk
-rw-r--r--  1 root vmware 2147221504 Aug 30 14:45 storage-f006.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:22 storage-f007.vmdk
-rw-r--r--  1 root vmware 2147221504 Oct 25 19:18 storage-f008.vmdk
-rw-r--r--  1 root vmware 2147221504 Nov 30 04:05 storage-f009.vmdk
-rw-r--r--  1 root vmware 2147221504 Nov 30 04:04 storage-f010.vmdk
-rw-r--r--  1 root vmware    2621440 Jul 23  2008 storage-f011.vmdk
-rw-r--r--  1 root vmware        749 Dec  7 21:11 storage.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:22 system-f001.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:22 system-f002.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:22 system-f003.vmdk
-rw-r--r--  1 root vmware 2147221504 Dec  7 22:22 system-f004.vmdk
-rw-r--r--  1 root vmware   17659904 Jul 23  2008 system-f005.vmdk
-rw-r--r--  1 root vmware        616 Dec  7 21:11 system.vmdk

As you can see, there are two hard disks (system and storage) and several other files. The hard disk as shown here is a fully allocated file which is divided up in seperate 2Gb files (handy for use on a FAT32 filesystem for portability). We have to convert it back into a monolithic file for use on KVM. Luckily KVM understands the monolithic VMDK file format, so we will choose this for interoperability. To convert the file you can use a VMware provided tool called vmware-vdiskmanager (includes with VMware products).

$ vmware-vdiskmanager -r system.vmdk -t 0 forum-system.img
$ vmware-vdiskmanager -r storage.vmdk -t 0 forum-storage.img

After this you will have two files that need to be copied to the in the images folder of libvirt on the virtualization host (/var/lib/libvirt/images/). You can do this using scp, ftp or mount an NFS export. From here we can create a virtual machine to connect these disk files to. Connect to the virtualization host and start the Virtual Machine Manager.

Create the Virtual Machine
Select localhost and create a new virtual machine using the context menu. Give it a unique name that identifies it easily for you, as I did: server-forum. If KVM is installed properly, you can only select "Full virtualized' from the Virtualization Method. Select the CPU architecture that fits your former VMware virtual machine and as the hypervisor: kvm.

On the 'Installation Method' page choose 'Network boot (PXE)' so you don't have to select a installation disk image.

On the 'Storage' page you deselect the 'Allocate entire virtual disk now' since we will not use the create virtual hard disk image. Beware; don't let the size become 0, since this way libvirt will not create the disk file and will fail to create the virtual machine.

On the 'Network' page you will probably want to select the bridged network card. This gives you the same functionality as VMware did as default option. On the 'Memory and CPU Allocation page' you can adjust the settings which are most close to your former virtual machine. After the summary the virtual machine will start automatically. Immediately issue a 'Force Off' as Shut down option. Now select the 'Hardware' tab.

Remove the 'Disk hda' and add new hardware of the type 'Storage' and select the system file from the storage location. If you have selected the file, you will see the correct size. Do the same of the other disks you have.

Be sure to change the boot device in the Boot options of this VM. Else it will try to start using Network (PXE) instead of the virtual hard disk.

Now everything should be ok to start the virtual machine.

Issues you might encounter
The conversion will probably not be without problems. For instance, I had a common issue with the kernel of the virtual machine. Due to a kernel panic it did not start properly. Fortunately tools are available to perform actions of the virtual hard disk files. One of those tools is guestfish. On CentOS and Fedora you can install this tool using:

$ yum install guestfish

For instance, I had to download a newer kernel and try to install this inside the virtual machine. The following commands show what I did:

$ guestfish
fs# add /var/lib/libvirt/images/forum-system.img
fs# run
Could not open '/dev/kqemu' - QEMU acceleration layer not activated: No such file or directory
fs# mount /dev/sda2 /
fs# mount /dev/sda1 /boot
fs# upload /tmp/downloaded.rpm /tmp/kernel.rpm
fs# command "rpm -ivh /tmp/kernel.rpm"
fs# vi /boot/grub/grub.conf

These commands will add the forum-system.img to a qemu instance. Inside the qemu instance you will then mount the mountpoints, upload the kernel RPM from the local path to the tmp inside the filesystem. From here nothing special... and then it will issue a command inside qemu to install the kernel package. This command will run inside the filesystem as if it performed the command from the commandline. Finally I verify the grub configuration and change it if necessary. From here I was able to start the virtual machine without any errors.

Note: If you convert Windows virtual machines you might encounter more issues. Those conversion steps are outside the scope of this document, but are not impossible to perform.

Fedora 12
The latest release of Fedora includes many improvements which are virtualization related. If you want to try KVM on a workstation, you should definitely try Fedora 12.

Below here you can see virt-manager. The interface has changed a little. As you can see from the screenshot, it is connected to the local machine and two remote hosts. The console is shown from the remote host (using ssh and vnc). For this no additional configuration was needed.

It is easy to monitor the performance of a virtual machine using the Virtual Machine Manager. Here you can see the server-forum instance with an overview of CPU/Memory usage and Disk/Network activity.

Just a few days ago SPICE was released. This is an alternative to vnc to view the console of a virtual machine. It is target towards the use of a Virtual Desktop. More information about this can be found on the SPICE website. A how-to for Fedora 12 is available.

Maemo 5 SDK on Fedora 12

2009-11-23T00:24:00.005+01:00

It is possible to install the Maemo 5 SDK (final) on Fedora 12. It only takes two minor edits to the installer script. You first need to make sure you have installed Xephyr. You can do so with the command:

$ yum install Xephyr

since the installer is not able to do so for other distributions than Ubuntu and debian. You can download the GUI installer script from Forum Nokia: Maemo 5 SDK.

In the file maemo-sdk-install-wizard_5.0.py you can change line 129 to

SB_PATH="/opt/scratchbox"

It is just optional. But to me this location feels more appropriate then '/scratchbox'. On my system it is linked to another location with other hardware and software development tools.

The current version of the script seems to fail on Fedora because of not being able to install scratchbox due to a missing path.

Change line 2311 to:

exec_cmd(sb_installer_fn + opt + "-s " + SB_PATH)

Change line 2351 to:

cmd = "%s -d -m %s -s %s" % (sdk_installer_fn, self.__sdk_inst_m_opt_arg, SB_PATH)

This includes the Scratchbox path during the command invocation. You can then install the SDK by running the script. It will handle the download of PyQT and sip itself.

After the install you can start Xephyr. However you can not use the -kb option:

$ Xephyr :2 -host-cursor -screen 800x480x16 -dpi 96 -ac &

The first start of af-sb-init.sh failed for me with a coredump and several segmentation faults. try to close the scratchbox environment and try again.

Note: I haven't tried it with SELinux as enforcing since I currently run my workstation as permissive. Discussion is possible on the Maemo developer's forum posting.

聚集：Fedora 12 正式发布 from LinuxTOY

2009-11-17T23:25:00.008+01:00

以技术创新驰名的 Fedora Project 今日发布了代号 Constantine 的 Fedora 12 ，口号 Unite，取意“聚集开源软件智慧”。

新特性

性能优化：在 32 位平台上全部软件包针对 i686 架构重新编译，并对 Intel Atom 处理器进行性能调优。
NetworkManager ：改善了对于宽带、蓝牙和 IPv6 的连接配置过程。配合 PolicyKit ，网络配置只需要点击鼠标即可轻松完成。
下一代 Theora 编码支持：Fedora 12 集成了最新的开源视频编解码器 Theora 1.1 版本（详情见本站此文）。
更好的显卡支持：针对 Nvidia Nouveau 驱动和 AMD R600/700 系列的 KMS 支持，同时为 Intel 显卡引入 DisplayPort 支持。
虚拟化支持：改善了 KVM 性能，并提供虚拟机磁盘镜像监控工具 guestfish。
自动臭虫汇报系统和 SELinux：新的崩溃收集程序 abrt 只需要点击几下鼠标即可将遇到的软件或 SELinux 提交至 Bugzilla，方便开发者修复。
新的 Dracut initrd 启动系统：并行、以事件为单位的 Dracut 系统进一步加快启动速度。
PackageKit 插件：当用户尝试运行一个包含在尚未安装的软件包中的命令时，PackageKit 可以自动提示安装相应软件包。
按需蓝牙服务：蓝牙服务会根据需要自动启动和停止。
Moblin 桌面环境：增加了 Moblin 桌面环境软件包组以及Fedora Moblin Spin 。
PulseAudio ：增添了 UPnP 和 DLNA 支持，可以直接将音频流发送至 PS3 等设备。同时改善了热拔插和蓝牙设备支持。
更加安全：降低了部分系统文件和进程的运行访问权限，避免使用 root 权限访问。同时为 SELinux 增添了沙箱支持。
Broadcom 固件：默认包含了 Open Broadcom 固件，对部分 Broadcom 无线网卡提供 out-of-the-box 支持。
混合 Live 镜像：现在只需要使用 dd 即可创建光盘或者 USB Live 镜像，不过还是推荐使用 Livecd-tools 以获得诸如保留空间等功能。
更好的摄像头支持：改善了部分摄像头的成象质量，尤其是很多廉价摄像头。
GNOME 2.28：使用 Gnote 替代了 Tomboy 成为默认便签，用 Empathy 替代了 Pidgin 成为默认即时通讯客户端。
GNOME-shell 预览：尽管 GNOME 3.0 延期了，但是依然可以在 Fedora 12 中提前体验下 GNOME Shell 的。
KDE 4.3：升级版的 Air 主题，可配置的快捷键支持，新的窗口管理器特效以及更好的红外线遥控支持。
开发者：Eclipse Galileo, Perl 6, PHP 5.3, Netbeans 6.7.1
管理员：GFS2 集群 Samba 管理
X.org 1.7.1：包含支持多指点设备支持的 XI2。

英文原文及详细贡献者
官方镜像下载
完整功能列表

Thanks 黑日白月, Orginal article

Get to know a Fedora Ambassador or User

2009-11-07T16:01:00.005+01:00

Name: Gerard Braad
IRC-Nick: gbraad
IRC-Channels: #fedora-ambassadors, #maemo, #beijinglug on freenode (seldom)
Location: Apeldoorn, NL / Beijing, CHN
Fedora Ambassador: Netherlands
FAS username: gbraad

Nokia N900; the geek-device!

2009-10-24T16:00:00.013+01:00

During the Maemo Summit 2009 I was given a Nokia N900 for evaluation and development. For about three weeks now I have been able to use the device and got used to it in such a way that I seldom check the internet on my notebook or desktop. Here you will find my thoughts on using this 'pre-production' model.

Services I often use, like Identi.ca and Twitter, are consolidated using my Google Mail account. Normally I would quickly browse the web to check my mail and check the messages I received while I was offline. During a working day, I tended to use my other Nokia phone for tethering to my N810. This way I was mostly always online (Bluetooth sometimes wouldn't re-establish until a power down of both devices)... I was able to receive VoIP calls (SIP) on both my phone and internet tablet. Most of the time I would still use my notebook to have a full web experience or respond to my emails... and for calendar-keeping? I gave up on that... Symbian's calendar isn't comparable to the Palm calendar (or Agendus!). I have tried to use the N810 as a PDA, but the GPE tools didn't work well for me... I missed a good synchronization and integration: then came the N900.

Near desktop experience
The N900 is very powerful! That needs to be said. It is powered by a Texas Instruments OMAP3430. This processor is comparable to the BeagleBoards OMAP3530. The ARM Cortex-A8 core runs at 600Mhz and has the same PowerVR SGX 530 GPU. Only the DSP core is a little slower with 430MHz (TMS320C64x). The device gives a feeling of having more than enough power. When you start a game, like Bounce, you can still see it being active in the task-switching overview; true multi-tasking.

The OMAP-series processor can provide a full desktop experience, like the BeagleBoard can do with for instance Fedora and Ubuntu. However, this is also where the N900 falls a little short. Instead of providing a mini-HDMI connector it only has a composite out. After some tweaks in the OS you can connect a Bluetooth keyboard and mouse... and using the correct cables even USB. But you can't connect a larger screen... this way it is not a convergence device which could replace a desktop computer. Compare this to the the Creative Zii which I have for development; it provides HDMI-out and can playback a HD video in 1080p (but it can't do the desktop experience at the moment). The composite image quality is in my opinion too blurry. For games it works well, but viewing photos or a video felt a little disappointing.

Battery time and use
Lately I have seen a lot of news about the Motorola Droid. This device is also powered by the OMAP3430 and would be thinner than a iPhone 3GS. I am curious to see how they deal with battery time... since that battery must be small! The N900 is quite large and the capacity of the BL-5J battery is less than the one in the N810. The battery is empty after half a day of extensive use; GPS always on, always online (using 3G or lower depending on the location in NL, WLAN at home), FM transmitter on when I go to work (and back home) while playing music of course.

Luckily you can charge the device using the MicroUSB connector and at work I always have to do so. During this 'extensive' use I seldom made a phonecall! For me the N900 is primarily a mobile computer and a phone second. The GSM functionality provides me with the cellular data service I was missing the N810. Most of my phonecalls were done using VoIP (over 3G) and were short. I did notice the device became VERY HOT during a VoIP (over WLAN) of about 30 minutes.

Internet
If you want the full Internet in a device, the N900 is what you want. It has a Firefox browser which loads pages fast and plays Flash videos from YouTube without any hiccup. Photos you take with the camera can be immediately uploaded to remote services, like Flickr and Ovi. GPS information can be set as a status message for your IM. Maemo uses the telepahy framework for messaging and video/voice calls. Out-of-the-box it provides support for SIP, Jabber/XMPP (and Google Talk, Ovi), Skype and GSM (Voice calls, SMS). What I really like is the way conversations are kept or started. Last messages are all easily viewed from the Conversations application.

All contacts are aggregated from the IM accounts I use. This immediately fills the gap of not having a phonebook service for my SIP accounts. Contacts from different sources can be merged into a single contact entry. If you want to call a contact from the phone application you can easily see if he is online using Google Talk or Skype and even initiate such a call.

You can send a message from either the contacts or conversations (or even desktop shortcut) and it will open a separate window in which you can see the chat. I haven't felt the need to install Pidgin, unlike what I did for the N810. By installing the telepathy-extras package you can also integrate MSN, Yahoo, ICQ, IRC and more.

Closing thoughts
Maemo 5 includes a simple calendar which for the moment works quite well for me. Although you can not sync it to a remote service, it works very intuitive. Notes, calendar entries and your contacts can be synchronized using the Nokia PC Suite or SyncML compatible client.

The camera in the device is 5 mega-pixels and can really make very good quality photos. Just take a look at the Photojourney with N900 from tigert.

As a game device the N900 could be a very good competition against the iPhone. It easily plays games like Quake3. Just connect it to the TV and play using the keyboard and the accelerometer! It is almost like having an embedded Wiimote. While the N810 still had a D-pad, this was dropped on the N900. The cursor keys feel more natural for typing emails... but for gaming it feels like you miss some control. Also the screen does not allow multitouch... so no on-screen joysticks for now.

Although the device has a complete open development model it currently lacks commercial support of big game companies. I have to admit, this is still where the iPod Touch and iPhone excels... it has a lot of available games. This will be addressed in the next release of Maemo (codenamed Harmattan). It allows a security model which will probably attract a lot more commercial development.

Last week Nokia postponed the release of the N900 to allow more feedback from developers and evaluators. It also gives developers more opportunity to get familiar to the N900. As an example, most applications were installed in the / (root) filesystem, while large datafiles should have been installed to the user directory (/home/user). I had suffered from an almost complete device lock-up due to system memory being full. This should not happen to a none experienced (Linux) users. So I really think Nokia does a good thing here... They really want to give new users of Maemo a good experience.

So if you are searching for a game console rather buy a PSP Go or even iPhone. But if you bought a Macbook with the idea of having Un*x under the hood (and a terminal) you should buy a N900. Even the Android falls short in this area. If you want a phone with mobile computer ambitions (being powerful and useful) buy a N97.

For me the iPhone is a good sheep-phone, while the N900 is the real geek-phone (/device)!

Take note, I have been using a 'pre-production' model. The full production model will likely provided a much better experience.

Fedora 11 on coLinux (fixes)

2009-09-21T14:50:00.004+01:00

Recently HenryN (of coLinux) made some minor changes to the Fedora 11 image for coLinux. The new image is available from the coLinux forum at SourceForge or directly from [FedoraPeople] or [MegaUpload] (70.3 MB).

Thanks!

Note: the original image has been replaced by this fixed image. If needed the original image is available upon request. However it should be considered obsolete.

Beijing LUG meeting (8th of September 2009)

2009-09-09T04:22:00.006+01:00

Last night I attended a meeting of the Beijing Linux User Group. It was for me the first time I could attend it during my time in Beijing and it is certainly something I will do more often. Every 2nd Tuesday of the month they will have a group meeting in which they will have speakers about a subject. I was lucky to listen to Coly Li and Ulrich Drepper.

Coly Li is a kernel developer who supervised a Google Summer of Code student, YuEr, who ported openSUSE to MIPS. YuEr showed what he had done, the results and the errors still left. He was able to start a X11 session using FVWM on a Gdium (MIPS-based smartbook). For this he had used a cross-platform RPM build environment targeting MIPS and qemu. Amazing results for such a short period he was able to work on it.

Ulrich Drepper gave the same talk as he did at the RedHat Summit in Chicago about 'Acceleration Through Stream Computing'. In this talk he explains how to speed up applications by using vectorization and the vector extensions as provided by SSE. Using templates and partial specialization in C++ it is possible to speed up applications without a lot of changes. The specialized critical code could be altered by experts and just a rebuild would suffice. He explains this for instance with a simple example of a vector scaling and translation... the result speaks for themselves. It was also apparent that Ulrich is not a promoter of CUDA (or even CTM) either. In my opinion they result in vendor-lockin. The alternative to look for is OpenCL...

Overall it was a nice evening. I talked to a talented bunch of 'folklore dancers' wearing T-Shirts. If you ever have the chance to be in Beijing, stop by... you will not regret it.

Accidently installed i386 packages on x86_64?

2009-08-25T21:45:00.009+01:00

If you have a CentOS installation (or Fedora) running as x86_64 architecture, be sure to install your packages with yum using the following command:

$ yum install [packagename].x86_64

This way you will specify the architecture to use. If you do not specify the architecture you might end up installing both the x86_64 and the i386 packages with all it's dependencies.

If you have already done so in error, you can easily query for these packages using the following command:

$ rpm -qa --qf '%{name}.%{ARCH}\n' |grep i386

this command returns all i386 packages. You can pipe it directly into rpm -e or choose to uninstall them by hand.

FreeSWITCH packages for CentOS 5.3

2009-08-23T11:52:00.004+01:00

Recently I built and packaged the 1.0.4 release of FreeSWITCH for CentOS 5.3 as i386 and x86_64. They are hosted on a temporary location http://files.spotsnel.net/freeswitch/. Eventually they will move to a repository for use with yum.

Fedora 11 on coLinux

2009-07-26T19:37:00.013+01:00

After some minor issues I managed to install Fedora 11 on coLinux. Normally I would perform a manual installation by creating a basic rpm environment, but with Fedora 11 I encountered problems during the installation; first the version of squashfs was different. This was easily fixed with a dump from Fedora 10... but second, rpm packages were all of a sudden not available halfway a rpm transaction. Since it occurred more than once, I settled for using qemu again.

The file colinux-fedora11.exe (82MB) is published on [fedorapeople], [dropbox] and [files]. It does have some minor issues during startup, but besides this it is operational. The installation was performed as a basic installation; so it does not contain any graphical environment, no additional network servers, no updates, etc. This way the download is small. The only added packages are wget, mc and my kernel modules RPM.

All you need to run it is: coLinux, winpcap and the archive. The file 'run.bat' would start the cooperative environment. Any tweaks you would like to make can be placed in the 'run.txt' file. The user is 'root' without a password. After you start it, do set a password for this user.

Personally I use the environment for Fedora Electronic Lab which coexists with Cygwin. Not all the tools I use for electronics are available on Linux... yet!

Note: I have used version 0.7.3 of coLinux because the current stable release 0.7.4 suffers from a problem with the tap device.

Note 21-09-09: The original image has been replaced by the updated image from HenryN. IF needed the original image is available upon request. It should however be considered as obsolete.

Linux distributions for ARM

2009-05-01T14:59:00.018+01:00

Recently there is a lot of attention for the ARM port of Linux. This is of course related to the netbooks and specialized distributions like Maemo for the Nokia Internet Tablets and Android from Google. Earlier I wrote a short post about the Hasty Hippogriff release of the Mojo Project. This Nokia-sponsored project ported Ubuntu's Hardy Heron to the ARM architecture. It allowed a full desktop distribution to be run on hardware like BeagleBoard. Luckily the major distributions also have increased their support for ARM. openSUSE, Debian and Ubuntu officially announced their support for the ARM architecture in their next release. This will allow companies to use a distribution like openSUSE as a thin client solution or for hardware developers to use Debian as their base system.

The distributions I have tried are Fedora 10, Debian 5.0 (Lenny) and Ubuntu 9.04. Debian 5.0 (Lenny) and Ubuntu 9.04 (Jaunty Jackalope) are now released. Debian provides downloads and bootstrap tools in their distro and lots of documentation. Despite the announcement it seems Ubuntu only has a release candidate?! So the release schedule seemed to not apply for the ARM architecture. Fedora however did not official announce any ARM support, but has a wiki page which describe how to use their distribution. openSUSE recently released a testing milestone for their 11.2 release, but I was unable to find the packages necessary to make a base system. I do have to make a note here: the openSUSE Build Service (OBS) provides ARM support for building packages for Fedora 10, Debian 5.0 and Ubuntu 9.04.

Below you can find the images I have created or used to test the distributions and the information I have used. These images are all base systems to get you started. They can serve as a starting point to create a build environment.

Fedora 10
Fedora does not provide official downloads, but all the files are provided on the wiki. To ease the deployment from a Windows system I created a usable root image which you can download from dropbox or vipeers and the kernel (dropbox). The archive can be unpacked using a tool like Winrar.

With the following command will start the qemu environment without networking:


qemu-system-arm.exe -M versatilepb -kernel zImage-versatile-2.6.24-rc7.armv5tel -hda rootfs-f10.img -append "root=/dev/sda"

You can login with username: root, without a password.

Ubuntu 9.04
Canonical only provides a Release Candidate for their ARM port of Jaunty Jackalope. You can find a the downloads from the cdimage server, such as a general image and an image for the NSLU2. I however built a new rootfs using the information provided on the wiki and the build-arm-rootfs script. This script will use debootstrap and qemu to produce a usable tarball or qemu image. I used the script to create a tarball and dumped this in a mounted image (the same as done for the fedora image).

You can download the root image from dropbox or vipeers and the kernel (dropbox).

To start the environment:


qemu-system-arm.exe -M versatilepb -kernel vmlinuz-2.6.28-11-versatile -hda rootfs-jaunty.img -append "root=/dev/sda"

To login use the username: ubuntu and the password: ubuntu.

Debian 5.0
The ARM support for Ubuntu is no miracle, as their base distribution Debian has support for this architecture. The installation guide can be found on the debian website. Using the debootstrap tool you can easily create your own rootfs. On the webpage of Aurélien Jarno (aurel32) you can find a complete article which describe the process of getting debian running on an emulated ARM using qemu on Linux. The root images and kernel he uses can be found here.

To start the environment:


qemu-system-arm.exe -M versatilepb -kernel vmlinuz-2.6.26-2-versatile -hda debian_lenny_arm_small.qcow -append "root=/dev/sda1"

To login using the username: user and password: user. The password for root is root.

Notes
Read the How to use Network if you want to enable networking for the qemu environment on Windows.

CentOS 5.3 image for coLinux

2009-04-03T20:11:00.006+01:00

A few days ago CentOS 5.3 was finally released. Just like what I have done with the previous version, I created a coLinux environment. This installation is a base system performed using the previously described manual method. Updated instructions are included in the published archive.

For now you can find the files here:
colinux-centos53.exe [VIPeers] [dropbox] (98mb)

It is configured to use DHCP on the network connections eth0 (slirp) and eth1 (bridged). The user account is 'root' without a password. After login, you should change the password. During startup you will see some complaints about the kernel modules. To solve this you can install my kernel modules RPM or use the tarball provided by coLinux.

Using Mono 2.4 on coLinux to ease .NET development

2009-03-31T16:00:00.002+01:00

Currently I deal a lot with .NET development in my daily work. Most of the development is done on Windows. When you need to develop cross-platform .NET you would like to use Mono. Mono is binary compatible with .NET and allows .NET applications to run on Linux distributions. If you would develop from Visual Studio you want to easily test the binary directly in a mono session on Linux!

To test software both platforms in a convenient way mostly virtualization is chosen. Of course you can use VMware or VirtualBox when you use Windows, but you can also use coLinux. coLinux is a port of the Linux kernel that allows it to run cooperatively alongside another operating system on a single machine. My choice of distribution to use on top of coLinux is the open version of SUSE, both supported by Novell.

This description does expect you to have understanding of a Linux environment and administrative rights on your Windows system.

Installation (Windows side)
First of all you would need to coLinux installer. This file is available from the coLinux website. The downloads section will point you to the latest stable, as of writing v0.7.3. On the website of Henry Nestler you can also find development and daily builds for 0.8.0. This installation is pretty straightforward, just keep the default settings. When the installer asks you 'Hardware installation' you need to press 'Continue anyways', this will allow the TAP network driver to be installed.

To use a desktop environment from coLinux you need to install a X server. The more difficult way would be to use Cygwin/X... this way you will have a complete GNU system running on top of Windows to provide a Linux-like experience. The fastest and easy way is to use a tool like Xming. The installer you need is the Xming or Xming-portablePuTTY (Mesa is not needed). Also, this installation is quite straightforward. After the installation start the X server from the start menu (Programs → Xming → Xming).

In the posting 'openSUSE 11.1 on coLinux' I described a way to make a base installation of openSUSE for use with the coLinux environment. The file 'colinux-opensuse-111.exe' (113Mb) [VIPeers] [dropbox] contains all you need to run the openSUSE environment. As you can see from the size, it contains only a base system. After you have extracted the file, you will still need to perform some additional installations to make the Mono development work.

Before you start the Mono installation inside the coLinux environment, you need to reserve a directory for your development files or some other shared directory. I choose to use 'D:\Development'. In the run.txt file in the openSUSE directory, you can add a line which says:


cofs1="D:\Development"

This will later be mounted inside the Linux environment.

Installation (Linux side)
Now you can start the environment from the 'runonce.bat'. This one time start is needed to finalize the installation (describe in the 'openSUSE on coLinux' posting as the post-install). You will see something similar to:

Logon using the credentials: 'root' and the password 'password'. Immediately change the password using


passwd

Enable the network using the command:


for i in 0 1
do
cat > /etc/sysconfig/network/ifcfg-eth$i << END BOOTPROTO='dhcp' STARTMODE='auto' USERCONTROL='no' END done

Add a mount point to your shared directory


mkdir /media/Development
vi /etc/fstab

and add the following line to this file


cofs1    /media/Development  cofs     defaults        0 0

To enable the shared directory you need to mount it


mount /media/Development

You can now shutdown...


shutdown -hn now

... the system will halt and closes the console. From now on you are able to start the environment using the normal 'run.bat'. Do so and log on using your new password.

Your network should now use eth0 as a serial line and eth1 as a bridged network using the device called 'Local Area Connection'. If your Windows network device is called differently, please change it in the 'run.txt' file. IP addresses are assigned using DHCP. The device eth0 will probably be assigned the address 10.0.2.15/24. For device eth1 this will depend on your local network settings. I will just assume that eth1 is configured and provides the Internet connection.

To allow packages to be installed you would need to add the standard repositories to this installation.


zypper ar http://download.opensuse.org/distribution/11.1/repo/oss/ openSUSE111
zypper ar http://download.opensuse.org/distribution/11.1/repo/non-oss/ openSUSE111_NonOSS 
zypper ar http://download.opensuse.org/update/11.1/ openSUSE-11.1-Updates

Enabling the User Interface
You can use a full-blown desktop environment as GNOME, but since coLinux does not provide a framebuffer device it is not advisable at the moment. We will use the Xming as the X server for our desktop. The X protocol is a client/server model for user interfaces. To keep the installation small, we use a very lightweight Desktop Environment on top of X, namely LXDE.

Install the LXDE packages using:


OCICLI http://download.opensuse.org/repositories/home:/swyear/openSUSE_11.1/LXDE-desktop.ymp

This will start the One-Click-Install. You will need to agree to some questions and lean back... this might take some time.

If this finished, you can test if the installation finished correctly. You will need to set the DISPLAY parameter to specify where the X server is running. Now let's start the appearance settings...


export DISPLAY=10.0.2.2:0.0
lxappearance

and choose a theme that suits your current Windows theme.

This means your desktop environment works.

If you would receive a 'Gtk-WARNING **: cannot open display:' this means your connection might be blocked for some reason. Check your settings or disable a firewall/virus scanner.

Mono installation
Mono will provide a Linux/Unix system with support to run .NET applications. Originally this description was written to use Mono 2.2 which was provided from the openSUSE repository. To install these packages issue a simple


zypper in -t pattern devel_mono

would be sufficient to install a development environment. To start the Monodevelop you can start this using the command:


export DISPLAY=10.0.2.2:0.0
monodevelop &

Since the 30th of March a newer release of Mono (v2.4) is available. To install you will need to specify the mono repository and upgrade the previous installation.


zypper addrepo http://ftp.novell.com/pub/mono/download-stable/openSUSE_11.1 mono-stable
zypper refresh --repo mono-stable
zypper dist-upgrade --repo mono-stable

You can now start Monodevelop 2.0 and use managed debugging!

That's it for now... In later posts I will automate the startup and show how to do development using this environment.

Have a lot of fun...

Notes
If you think VMware provides a better solution you can make use of the files that the Mono Project provide in their download section. Either download the generated VMware image of the LiveCD.

If you want to use GNOME as your desktop environment you are advised to remove the gnome-screensaver package and it's dependent, since it can cause issues during use. If you want to experiment with the LXDE environment, you are advised to use the lxpanel and pcmanfm. In a later post these will be explained in more detail.

openSUSE 11.1 on coLinux

2009-03-20T18:56:00.037+01:00

To ease cross-platform .NET development, I have installed openSUSE on coLinux. This allows me to develop from Windows and also test and debug the binary on Linux. Advantages are: No need to dual boot and a much better performance than using VMware or other virtualization solution. In this article you can find the base system and I will describe how I did the base installation.

Take note, these instructions do NOT provide any details about the .NET environment. Those are published on my .NET blog.

The files to install and run the environment 'colinux-opensuse-11.1.exe' can be found at [VIPeers], [dropbox] (113Mb). This file contains the base system.img, a swap image and start commands. If you don't want to install it yourself, you can at least immediately start to use openSUSE on coLinux. Take note, network has not been configured yet (see notes). The user account is 'root' with the password 'password'.

Preparation
The installation closely follows the instructions I wrote for CentOS. You will need a clean system.img which you can find in the colinux-centos52-base.rar. Copy the Debian businnesscard init ramdisk to install_initrd.gz. And of course you need to download the openSUSE 11.1 DVD from http://www.opensuse.org/. This file will be about 4.25GB in size.

You can perform a RPM installation, but since openSUSE provides an image-based installation I will use this for the base of the coLinux environment. My main purpose was to keep the installation easy to perform (and if possible small in size). Copy the files base-i386.tar.lzma, base-meta-i386.tar.lzma and common-base-i386.tar.lzma from the openSUSE 11.1 DVD. It is located in the directory images. Since the Debian initrd does not contain the lzma binary, so you will need to decompress the archives with a tool like 7-Zip. The resulting files will be standard tape-archives. Place them in a directory called 'images'. The install.bat will map it to cofs1.

Installation
If you have the install files from the opensuse-11.1.exe file, the images, debian's install_initrd.gz and the system.img from the centos52-base.rar, you are ready to go.

Start the install.bat. You will see a debian installation screen... toggle with ALT-F2 to a console and press Enter.

Please press Enter to activate this console.

From here you can perform the following commands. The line which says 'tar -C /mnt/linux -xf $i' will unpack the image to the system.img. Just be sure to have some coffee ready since it might take some time.


mkswap /dev/cobd7
swapon /dev/cobd7

mkdir /mnt/linux
mkdir /mnt/win
mount /dev/cobd0 /mnt/linux
mount -t cofs cofs0 /mnt/win

mkdir /mnt/images
mount -t cofs cofs1 /mnt/images
cd /mnt/images

for i in *.tar
do
tar -C /mnt/linux -xf $i
done

mkdir -p /mnt/linux/media/cdrom
mount /dev/cobd1 /mnt/linux/media/cdrom

chroot /mnt/linux
cd /media/cdrom/suse/i586

rpm -ivh openSUSE-release-[0-9]*.rpm \
openSUSE-release-dvd-[0-9]*.rpm

echo password | passwd - stdin root 

cat > /etc/fstab << END
/dev/cobd0 / ext3 acl,user_xattr 1 1
/dev/cobd7 swap swap defaults 0 0
proc /proc proc defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs noauto 0 0
debugfs /sys/kernel/debug debugfs noauto 0 0
END

exit

After this you can halt the system. And start it for the first time.

Post-install
To finish the installation you need to start it with the coLinux initrd once. The script runonce.bat will take care of that... after you have run it, openSUSE will start properly. Log on and halt again. After this you can start it without a initrd being specified. The script run.bat will start the final system.

It might give some startup errors... nothing serious. I didn't configure the network inside the base system. This can be done using yast or perform the following command:


for i in 0 1
do
cat > /etc/sysconfig/network/ifcfg-eth$i << END
BOOTPROTO='dhcp'
BROADCAST=''
ETHTOOL_OPTIONS=''
IPADDR=''
MTU=''
NAME=''
NETMASK=''
NETWORK=''
REMOTE_IPADDR=''
STARTMODE='auto'
USERCONTROL='no'
END
done

This enables both eth0 and eth1 to use DHCP. Kernel modules can be installed with the same RPM I have provided earlier. Just use the following commands:


zypper in http://gbraad.fedorapeople.org/files/kernel-modules-2.6-co0.7.3.i386.rpm

Alternative installation
If you want to perform a RPM-based installation you will still need to install the base-i386.tar image. I tried without, but ended up with a lot of errors during installation. The can find the installation instructions in the file instructions-rpm.txt which is included in the archive. Eventually I have not further tested this alternative installation...

Notes
I have tried to only use the files provided by openSUSE, but somehow they seem to consume too much time. The common filesystem did not provide all the tools I needed (similar to the stage2.img from redhat). If you know another way to install it, let me know. Qemu can also install it, but it consumes a lot more time in my opinion; install, dd, etc.

For the moment it is not possible to perform an image installation without the openSUSE DVD installation media. I do hope they will provide a single CD image installer or provide the images from their 11.1 download site. If it is possible, please enlighten me?!

If you want to install additional software, you will need to add the openSUSE repositories. This can easily be done using zypper.


zypper ar http://download.opensuse.org/distribution/11.1/repo/oss/ openSUSE111
zypper ar http://download.opensuse.org/distribution/11.1/repo/non-oss/ openSUSE111_NonOSS 
zypper ar http://download.opensuse.org/update/11.1/ openSUSE-11.1-Updates

Now you can install Gnome by using a command like:


zypper in -t pattern gnome

More information about installing software can be found on openSUSE Tutorials.


Have a lot of fun...

Manually installing CentOS 5.2 on coLinux

2009-03-16T18:58:00.018+01:00

In the previous post I published a minimal CentOS 5.2 environment for use with coLinux. As I described in the posting, I did a manual installation since I was unable to use Qemu to start the CentOS 5.2 installation process. Although I received a comment that someone else was able to install CentOS, I am still unable to do so... even with the latest build of Qemu (currently 0.9.1) on Windows. Instead of trying to convert a VMware image or using an update process, I wanted to only use coLinux. In this posting I will try to describe what I did.

Requirements
To ease the installation I have created a base which can assist you with installing. It contains the files you need as images (formatted as Ext3). It is available as centos52-base.rar [dropbox] (35kb).

This file contains what is needed to install the CentOS 5.2 image for
use with coLinux:

system.img 4Gb system image, formatted as Ext3

swap512m.img 512Mb swap location

Batch and parameters file needed to start coLinux

install.bat, install.txt for the installation process (needed once)

run.bat, run.txt to start the environment after install

instructions.txt file containing the commands to perform

Extract the base archive and place the CentOS 5.2 (i386) installation media in the same directory. Mirrors are provided on the download page of CentOS. If you do not want to download the DVD file, you can also choose to only download CD 1of6. For this you need to change the install.txt parameters file. After the initial install, you can then rely on YUM to install all additional packages.

I used a custom init ramdisk to create the images and perform the installation. Since this file is still rough, I suggest you to download this business card iso from debian. The businesscard ISO is about 35Mb.

With a tool like winrar (or mount it using a tool like Daemon Tools), you can extract the file /install.386/initrd.gz. If you prefer to only download the initrd.gz. Save this file as install_initrd.gz in the directory where you extracted this base directory. This ramdisk will be used to start a system to perform commands from which will start the RPM install.

After installing coLinux to the standard location you can run the install.bat and start the installation process. It will start the start kernel with the install_initrd.gz as ramdisk. The terminal of choice for this process was a NT terminal, since copy-n-paste was easier to perform. When the installation process interface is shown, you should toggle with ALT+F2 to a console.

Please press Enter to activate this console.

When you do so, you will see a BusyBox prompt where you can perform the instructions as shown in the instructions.txt.

Instructions
You will now perform the instructions needed to do the actual install. We will create a basic environment to perform an RPM installation of the base packages. After this you can use YUM to download and/or install additional packages. To create
the basic layout:


mkswap /dev/cobd7
swapon /dev/cobd7
mkdir /mnt/linux
mkdir /mnt/win
mkdir /mnt/stage2
mount /dev/cobd0 /mnt/linux
mkdir -p /mnt/linux/media/cdrom
mount -t cofs cofs0 /mnt/win
mount /dev/cobd1 /mnt/linux/media/cdrom

This will create some swapspace, a mountpoint /mnt/linux which points to the block device cobd0 (system.img), /mnt/win will be used to point to the coLinux program files folder and /mnt/stage2 will later be used to mount the CentOS stage2 image file. Block device cobd1 is used to mount the DVD image to the mountpoint /mnt/linux/media/cdrom. This will later be used to install the base packages from.

The stage2 image file uses the squashfs filesystem. CoLinux provides kernel modules to use this filesystem.


tar xzvf /mnt/win/vmlinux-modules.tar.gz lib/modules/2.6.22.18-co-0.7.3/kernel/fs/squashfs/squashfs.ko
insmod /lib/modules/2.6.22.18-co-0.7.3/kernel/fs/squashfs/squashfs.ko
mount -t squashfs -o loop /mnt/linux/media/cdrom/images/stage2.img /mnt/stage2

The contents then need to be copied to the /mnt/linux to form the basis of our installation process.


cp -a /mnt/stage2/ /mnt/linux
umount /mnt/stage2
mv /mnt/linux/stage2 /mnt/linux/base

This might take some time...

The system needs to have some directories which will form the basis. The following command will create this.


for n in bin dev proc lib usr tmp etc
do
  mkdir -p /mnt/linux/$n
done
touch /mnt/linux/etc/fstab /mnt/linux/etc/mtab
mkdir -p /mnt/linux/var/lib/rpm

The location /var/lib/rpm is needed to store the RPM database files.

Pre-installation
RPM will need some libraries to run. These will be copied to the appropriate location using the following commands:


cd /mnt/linux/lib
ln -s ../base/lib/* .
rm /mnt/linux/lib/udev
rm /mnt/linux/lib/bdevid
mkdir /mnt/linux/usr/lib
cp -a /mnt/linux/base/usr/lib/librpm-* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/librpmio-* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/librpmdb-* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/libpopt.* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/libsqlite3.* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/libelf* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/libbeecrypt.* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/libz.* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/libbz2.* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/libstdc++.* /mnt/linux/usr/lib/
cp -a /mnt/linux/base/usr/lib/rpm /mnt/linux/usr/lib/

All necessary libraries are now available to pivot the root to the /mnt/linux location.


chroot /mnt/linux /base/usr/bin/sh
/base/usr/bin/rpm --initdb

This will create a fresh RPM database which the installation process can use.

Installation
You are now ready to perform the installation.


cd /media/cdrom/CentOS
/base/usr/bin/rpm -ivh setup-*.rpm

The following packages need to be installed at once. These will form a base installation of CentOS.


/base/usr/bin/rpm -ivh \
audit-libs-[0-9]*.rpm \
audit-libs-python-[0-9]*.rpm \
authconfig-[0-9]*.rpm \
basesystem-[0-9]*.rpm \
bash-[0-9]*.rpm \
beecrypt-[0-9]*.rpm \
bzip2-libs-[0-9]*.rpm \
centos-release-[0-9]*.rpm \
centos-release-notes-[0-9]*.rpm \
checkpolicy-[0-9]*.rpm \
chkconfig-[0-9]*.rpm \
coreutils-[0-9]*.rpm \
cpio-[0-9]*.rpm \
cracklib-[0-9]*.rpm \
cracklib-dicts-[0-9]*.rpm \
cryptsetup-luks-[0-9]*.rpm \
cyrus-sasl-lib-[0-9]*.rpm \
db4-[0-9]*.rpm \
dbus-[0-9]*.rpm \
dbus-glib-[0-9]*.rpm \
device-mapper-[0-9]*.rpm \
device-mapper-event-[0-9]*.rpm \
device-mapper-multipath-[0-9]*.rpm \
dhclient-[0-9]*.rpm \
dhcpv6-client-[0-9]*.rpm \
diffutils-[0-9]*.rpm \
dmidecode-[0-9]*.rpm \
dmraid-[0-9]*.rpm \
e2fsprogs-[0-9]*.rpm \
e2fsprogs-libs-[0-9]*.rpm \
ecryptfs-utils-[0-9]*.rpm \
ed-[0-9]*.rpm \
elfutils-libelf-[0-9]*.rpm \
ethtool-[0-9]*.rpm \
expat-[0-9]*.rpm \
file-[0-9]*.rpm \
filesystem-[0-9]*.rpm \
findutils-[0-9]*.rpm \
gawk-[0-9]*.rpm \
gdbm-[0-9]*.rpm \
glib2-[0-9]*.rpm \
glibc-[0-9]*.i686.rpm \
glibc-common-[0-9]*.rpm \
gnu-efi-[0-9]*.rpm \
grep-[0-9]*.rpm \
grub-[0-9]*.rpm \
gzip-[0-9]*.rpm \
hal-[0-9]*.rpm \
hdparm-[0-9]*.rpm \
hwdata-[0-9]*.rpm \
info-[0-9]*.rpm \
initscripts-[0-9]*.rpm \
iproute-[0-9]*.rpm \
iptables-[0-9]*.rpm \
iptables-ipv6-[0-9]*.rpm \
iputils-[0-9]*.rpm \
kbd-[0-9]*.rpm \
kernel-[0-9]*.rpm \
keyutils-libs-[0-9]*.rpm \
kpartx-[0-9]*.rpm \
krb5-libs-[0-9]*.rpm \
kudzu-[0-9]*.rpm \
less-[0-9]*.rpm \
libacl-[0-9]*.rpm \
libattr-[0-9]*.rpm \
libcap-[0-9]*.rpm \
libgcc-[0-9]*.rpm \
libgcrypt-[0-9]*.rpm \
libgpg-error-[0-9]*.rpm \
libhugetlbfs-[0-9]*.rpm \
libselinux-[0-9]*.rpm \
libselinux-python-[0-9]*.rpm \
libsemanage-[0-9]*.rpm \
libsepol-[0-9]*.rpm \
libstdc++-[0-9]*.rpm \
libsysfs-[0-9]*.rpm \
libtermcap-[0-9]*.rpm \
libusb-[0-9]*.rpm \
libuser-[0-9]*.rpm \
libvolume_id-[0-9]*.rpm \
libxml2-[0-9]*.rpm \
libxml2-python-[0-9]*.rpm \
lvm2-[0-9]*.rpm \
m2crypto-[0-9]*.rpm \
MAKEDEV-[0-9]*.rpm \
mcstrans-[0-9]*.rpm \
mingetty-[0-9]*.rpm \
mkinitrd-[0-9]*.rpm \
mktemp-[0-9]*.rpm \
module-init-tools-[0-9]*.rpm \
nash-[0-9]*.rpm \
ncurses-[0-9]*.rpm \
net-tools-[0-9]*.rpm \
newt-[0-9]*.rpm \
nspr-[0-9]*.rpm \
nss-[0-9]*.rpm \
openldap-[0-9]*.rpm \
openssh-[0-9]*.rpm \
openssh-clients-[0-9]*.rpm \
openssh-server-[0-9]*.rpm \
openssl-[0-9]*.i686.rpm \
pam-[0-9]*.rpm \
passwd-[0-9]*.rpm \
pciutils-[0-9]*.rpm \
pcre-[0-9]*.rpm \
pm-utils-[0-9]*.rpm \
policycoreutils-[0-9]*.rpm \
popt-[0-9]*.rpm \
prelink-[0-9]*.rpm \
procps-[0-9]*.rpm \
psmisc-[0-9]*.rpm \
python-[0-9]*.rpm \
python-elementtree-[0-9]*.rpm \
python-iniparse-[0-9]*.rpm \
python-sqlite-[0-9]*.rpm \
python-urlgrabber-[0-9]*.rpm \
readline-[0-9]*.rpm \
redhat-logos-[0-9]*.rpm \
rhpl-[0-9]*.rpm \
rootfiles-[0-9]*.rpm \
rpm-[0-9]*.rpm \
rpm-libs-[0-9]*.rpm \
rpm-python-[0-9]*.rpm \
sed-[0-9]*.rpm \
selinux-policy-[0-9]*.rpm \
selinux-policy-targeted-[0-9]*.rpm \
setools-[0-9]*.rpm \
setserial-[0-9]*.rpm \
shadow-utils-[0-9]*.rpm \
slang-[0-9]*.rpm \
sqlite-[0-9]*.rpm \
sysfsutils-[0-9]*.rpm \
sysklogd-[0-9]*.rpm \
system-config-securitylevel-tui-[0-9]*.rpm \
SysVinit-[0-9]*.rpm \
tar-[0-9]*.rpm \
tcl-[0-9]*.rpm \
tcp_wrappers-[0-9]*.rpm \
termcap-[0-9]*.rpm \
tzdata-[0-9]*.rpm \
udev-[0-9]*.rpm \
udftools-[0-9]*.rpm \
usermode-[0-9]*.rpm \
util-linux-[0-9]*.rpm \
vim-minimal-[0-9]*.rpm \
wireless-tools-[0-9]*.rpm \
yum-[0-9]*.rpm \
yum-metadata-parser-[0-9]*.rpm \
zlib-[0-9]*.rpm

If you perform the command, the following will show indicating that the installation process started.

This will again take some time... and will eventually result in the following.

If no error occurred you can start with the post-installation to make the system usable from coLinux.

Post-installation
We will now perform some post-installation commands to finish the installation. Create the device nodes in the device directory. These will be used by the filesystem table to mount the filesystem. Do not forget to first exit out of the chroot environment.


exit
mknod -m 666 /mnt/linux/dev/null c 1 3
for i in 0 1 2 3 4 5 6 7 8 9 10
do
  mknod -m 660 /mnt/linux/dev/cobd${i} b 117 ${i}
done
chown 0:6 /mnt/linux/dev/cobd*

To have the filesystems mounted on startup you need to have the fstab file being created. Just overwrite the current file:


cat > /mnt/linux/etc/fstab << END
/dev/cobd0   /          ext3     defaults        1 1
/dev/cobd7   swap       swap     defaults        0 0
none         /proc      proc     defaults        0 0
none         /dev/shm   tmpfs    defaults        0 0
none         /dev/pts   devpts   gid=5,mode=620  0 0
none         /sys       sysfs    defaults        0 0
END

Write some basic information to the hosts file.


cat > /mnt/linux/etc/hosts << END
127.0.0.1 localhost localhost.localdomain
END

The following will create the shared memory and the pseudo terminal devices, turns off the hardware daemon and allows you to give an initial password for the root user.


chroot /mnt/linux /sbin/MAKEDEV con
chroot /mnt/linux /bin/bash -c "/bin/mkdir /dev/{shm,pts}"
chroot /mnt/linux /bin/bash -c "/bin/chmod a+rwxt /dev/shm"
chroot /mnt/linux /sbin/MAKEDEV generic
chroot /mnt/linux /sbin/chkconfig haldaemon off
chroot /mnt/linux /usr/sbin/authconfig --enableshadow --update
chroot /mnt/linux /usr/bin/passwd root

To have the network start, you need to create the following files. This way your system will be configured with a basic hostname and two network devices using DHCP.


cat > /mnt/linux/etc/sysconfig/network << END
NETWORKING=yes
HOSTNAME=localhost.localdomain
END


cat > /mnt/linux/etc/sysconfig/network-scripts/ifcfg-eth0 << END
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Ethernet
END


cat > /mnt/linux/etc/sysconfig/network-scripts/ifcfg-eth0 << END
DEVICE=eth1
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Ethernet
END

Cleanup
You can now remove the base directory we had copied from the stage2 image.


rm -rf /mnt/linux/base
halt

After this you can halt the system and start the environment using the run.bat file. If everything went well your system would look like this!

During startup you will notice some minor error which are related to the kernel dependencies file being missing. To solve this, you can extract the kernel modules using the /dev/cofs0 mapping. Although I prefer to use RPMs, since they are easier to cleanup and upgrade when needed.


yum install wget
wget http://gbraad.fedorapeople.org/files/kernel-modules-2.6-co0.7.3.i386.rpm
rpm -ivh kernel-modules-2.6-co0.7.3.i386.rpm

Enjoy using it... If you have suggestions or stories about your use, leave a comment or send me an email.

CentOS 5.2 image for coLinux

2009-03-13T20:43:00.013+01:00

Although soon we hope to have the newer 5.3 release of CentOS, I still decided to create a coLinux image of the current release. The problem with this release is that the kernel has issues when run on Qemu as can be seen in this bug report. This issues is known upstream and is solvable by installing 5.1 and then update it beyond the problematic kernel... or you could just wait until CentOS 5.3 is released.

To make CentOS 5.2 usable in coLinux you could install 5.1 in this way using Qemu and update it. I just took a different approach: use RPM. I used a modified initrd image and installed a basic RPM installation (kinda similar to the FC3-CentOS frankenupgrade). More details about how the installation was performed will follow.

For now you can find the files here:
colinux-centos52.exe [VIPeers] [dropbox] (82mb)

The files contain an image with a minimal installation which you can use on coLinux 0.7.3. The startup is not error-free... but you can ignore the failures for now, since I will update the images to reflect the Fedora images. Your suggestions are of course welcome.

It is configured to use DHCP on the network connections eth0 and eth1. The user account is 'root' without a password.

Edit: repacked the archive to be self-extracting and kernel modules (co0.7.3) have been installed by RPM.

libvirt for RHEL5/CentOS 5

2009-02-01T23:04:00.005+01:00

Just built and packaged the libvirt library version 0.6.0 for use with Red Hat Enterprise Linux 5 and CentOS 5 (or any other EL5 compatible system). This library supports the following virtualization systems: Xen, KVM, Qemu and OpenVZ.

Files are located at http://files.gbraad.nl/packages/el5/.

These packages have been signed with my packaging key, which is available here (mirror). More information about this library can be found on the libvirt homepage.

The Wizard of Yum: Upgrade from FC3 to CentOS 5.2

2009-01-11T13:45:00.011+01:00

If you have a production server, you would like to stay on the yellow brick road of upgrades for the operating system and installed software. However, it might happen you forget to support a server... and get 'scared' of performing updates: "If it ain't broken, don't fix it"! This mentality is not always correct, since errors you don't have at the moment, might still be a problem later on. The further away you get from the updates, the more difficult it will become to perform an upgrade to a later version of a distribution.

Why to write about it
The motivation to describe my experience of upgrading is because I hear a lot of people still complain about RPM and the dependency issues. Most of them have never used it and call the apt system the best solution. In my opinion, the key to success for apt is the reliance on a single repository. Debian people seldomly use dpkg -i to install package, since they use apt-get install to resolve dependencies. YUM solves the issue for RPM systems in a similar fashion. Instead of using rpm -i you can install packages using yum install, which resolves necessary dependencies from the configured repositories.

Note: With Ubuntu and Maemo I have noticed that more repositories get introduced which cause the same problems for apt-get. Dependency resolving might not work, since packages and depencies get fragmented and sometimes even unavailable or conflicting. The Red Hat community recently merged several repositories (RPM Fusion and RPM Forge) which does solve a lot of issues.

Setup
I was faced with the upgrade problem with my own personal (web)server for quite some time. I have no idea what I originally installed on it, but it has always been a Red Hat Linux system. It has been upgraded many times until it ran Fedora Core 3. At least the oldest file on the server was from January 2000. Since the server ran FC3, there was a problem. Fedora Core 3 was released in November 2004 it therefore only had updates available until about 2006.

Last year the machine was taken apart and virtualized to run on a newer VMware environment. I did update the system quite often... but never upgrade the distribution again. I had strayed from the yellow brick road and had no clear upgrade path any more. To solve this issue I could do to things:

Install a new server and migrate the files
Upgrade to a new distribution and hope this would start

No matter what option I would choose, I wanted to have similar functionality. Currently it runs:

Webserver:
Apache HTTPd (with various modules) → Zope
Mail infrastructure:
Postfix → Amavisd-new → Cyrus-imapd
Database:
PostgreSQL

I wanted to upgrade (option 2) as I had always done. Because of this, my options for upgrade were kinda limited: Fedora Core 4? Fedora Core had no support either... and the support for Fedora 8 would also expire soon (as it did this week). On the internet I had seen people mention it would be impossible to perform an upgrade from FC3 to a later version, like FC9 or they strongly advice others not to do so...

Hmmm, so instead I copied my 'sites', 'imap', 'zope' and database dumps to an NFS. Did I have to start a new installation?!??? All other servers I installed allready ran CentOS, I would also make this a CentOS installation. Well, considering CentOS is derived from RHEL (RedHat Enterprise Linux) it is in fact a Fedora installation. If you look at the release details:

RHEL-4 is based on FC3
RHEL-5 is based on FC6

You can upgrade from RHEL-4 to RHEL-5, this would mean like you jump from FC3 to FC6. CentOS 5.2 is a RedHat derivative from RHEL-5.2... So I wanted to upgrade my FC3 to CentOS 5.2. I know this would be difficult, but in my opinion possible. The key to success would be YUM, the chosen applications and the self-contained installs. The structure I had laid out was simple

Amavisd-new ran its own Perl installation from /opt/activestate/perl/
Zope
ran from a Python installation from /opt/activestate/python/
datafiles are in /home/zope/
Cyrus was built from source
installed in /opt/cyrus/
datafiles are in /home/cyrus/, totalling 7GB.
Each website has its own vhost-[domainname].conf for Apache and a separate directory in /home/sites/[domainname]/[hostname]/web/ which contains the files. Several websites, subdomains and databases, totalling 4GB of data.

Description
The following description are the actions I took, but it is NOT a howto for guaranteed success. It might work for you, although you could also end up with a non-booting system. Only perform this installation if you have enough knowledge of a Linux system and have backups available if it goes wrong.

On VMware I made a snapshot of the VM and opened the console, logged in on the 2nd and 3rd console as root. The main installation I would perform from two SSH connections (one as a backup). The tool I would use for the upgrade is YUM. YUM is a tool which resolves dependencies from repositories you have specified.

As root I created a 'frankenstein' directory with the following files from a CentOS mirror.
centos-release-5-2.el5.centos.i386.rpm beecrypt-4.1.2-10.1.1.i386.rpm centos-release-notes-5.2-2.i386.rpm db4-4.3.29-9.fc6.i386.rpm db4-devel-4.3.29-9.fc6.i386.rpm db4-utils-4.3.29-9.fc6.i386.rpm elfutils-0.125-3.el5.i386.rpm elfutils-libelf-0.125-3.el5.i386.rpm elfutils-libelf-devel-0.125-3.el5.i386.rpm elfutils-libelf-devel-static-0.125-3.el5.i386.rpm elfutils-libs-0.125-3.el5.i386.rpm glib2-2.12.3-2.fc6.i386.rpm glib2-devel-2.12.3-2.fc6.i386.rpm glibc-2.5-24.i386.rpm glibc-common-2.5-24.i386.rpm glibc-devel-2.5-24.i386.rpm glibc-headers-2.5-24.i386.rpm keyutils-1.2-1.el5.i386.rpm keyutils-libs-1.2-1.el5.i386.rpm krb5-devel-1.6.1-25.el5.i386.rpm krb5-libs-1.6.1-25.el5.i386.rpm krb5-workstation-1.6.1-25.el5.i386.rpm libselinux-1.33.4-5.el5.i386.rpm libselinux-devel-1.33.4-5.el5.i386.rpm libsepol-1.15.2-1.el5.i386.rpm libsepol-devel-1.15.2-1.el5.i386.rpm libxml2-python-2.6.26-2.1.2.1.i386.rpm m2crypto-0.16-6.el5.2.i386.rpm mcstrans-0.2.7-1.el5.i386.rpm openssl-0.9.8b-10.el5.i386.rpm popt-1.10.2-48.el5.i386.rpm python-2.4.3-21.el5.i386.rpm python-devel-2.4.3-21.el5.i386.rpm python-elementtree-1.2.6-5.i386.rpm python-ldap-2.2.0-2.1.i386.rpm python-sqlite-1.1.7-1.2.1.i386.rpm python-urlgrabber-3.1.0-2.noarch.rpm readline-5.1-1.1.i386.rpm readline-devel-5.1-1.1.i386.rpm rpm-4.4.2-48.el5.i386.rpm rpm-build-4.4.2-48.el5.i386.rpm rpm-devel-4.4.2-48.el5.i386.rpm rpm-libs-4.4.2-48.el5.i386.rpm rpm-python-4.4.2-48.el5.i386.rpm sqlite-3.3.6-2.i386.rpm sqlite-devel-3.3.6-2.i386.rpm yum-3.2.8-9.el5.centos.1.noarch.rpm yum-metadata-parser-1.1.2-2.el5.i386.rpm

The centos-release file tell your YUM to which repository it should refer for an upgrade. Your version number may differ, but this would generally give you a working YUM installation from a CentOS system. Install all the packages at once.

$ rpm -Uvh *.rpm --nodeps.

This is what my console showed after the installation. Since the berkely database version has been upgraded, you need to remove the previous RPM database and rebuild it.

$ rm -f /var/lib/rpm/__db.* $ rpm --rebuilddb

From this moment on, you could upgrade the system. The command

$ yum upgradewill show you what issues you need to satisfy for the dependencies. These you can install using:

$ yum install [packagename]

Before I upgraded, I disabled several repositories in my /etc/yum.repos.d/ directory. You can do this by adding enabled=0 to a repo definition. Before continuing the upgrade I stopped unnecessary services and disabled SELinux for now. Depending on the use of your installation you may need to also remove some packages. I had to do so.

$ rpm -qa

shows you all the installed packages. For my installation I removed some Novell branded packages for Mono and most of the X11 packages. After this remove the installed kernels. You might to iterate these steps (rpm -qa and rpm -e or yum remove) several times before you can perform the upgrade.

$ yum upgrade

starts the actual upgrade... be careful during the installation, since some services might not function as expected. Do not abort it. This would render your system not usable.

After the yum upgrade has been performed, you need to check if you have a kernel configured in /boot/grub/grub.conf (/etc/grub.conf). I had to install the kernel myself due to architecture conflicts?!

$ rpm -ivh --ignorearch kernel-2.6.18-92.el5.1686.rpm
Even after this installation you would need to check the grub configuration. Reboot the system and when it starts, you would have a CentOS system.

After this you can cleanup your system.

$ rpm -qa > /tmp/rpmlist $ cat /tmp/rpmlist | grep fc3 $ cat /tmp/rpmlist | grep FC3
Remove any remaining package which refer to your previous installation. Note: it is normal to find references to fc6 packages for your CentOS installation.

After the reboot, I noticed that most of the applications still worked as they should. Most of the time went into getting Apache and Zope running well together. Now a week later, I removed the VMware snapshot, even consolidated two harddisc files into one. Cyrus-impad is now makes use of the official release package (using my own config file) and upgraded amavisd-new to the RPMforge release. Only zope is running the old installation...

Conclusion
Always plan your system for future use. A sever has a lifespan which may exceed your initial thoughts. Try to compose the server as seperate responsibilities (mail, web, etc) as this would make upgrade easier. Try to stay close to the release schedule of patches and updates for your system... try to avoid building from source or compiling your own kernels as this would make management more difficult later on. And when a distribution gets the end-of-life status, try to move away from it immediately.

As you can see, it would still be possible to upgrade to CentOS 5.2 from Fedora Core 3... but this is NOT advised or a common practice. For me it shows how well thought the components and layout were, since this is also the setup I had used for SpotlightMedia and my previous employer.

In my opinion it shows how well the YUM dependency resolving and upgrading works. The wonderful wizard of Yum gave me a working CentOS system!

Installation of OpenVZ on CentOS 5.2

2008-12-30T16:32:00.010+01:00

OS virtualization allows you to better scale if you need a large amount of similar software installations and you do not want to deploy more than one physical (or even virtual) machine. It creates isolated containers in which your environment run. In a way it is comparable to a jailed environment, but it provides better isolation, security and management. Hardware virtualization, for products as VMware, XEN and KVM, has a different approach in which you run a completely emulated distribution (often paravirualized) on top of your host system. OS and Hardware virtualization can complement each other.

The installation of OpenVZ containers on CentOS (or RHEL5) is very simple. The following steps were performed on a default installation of CentOS 5.2. In my case I use a virtual machine in a VMware environment.

The OpenVZ installation


$ setup

First you need to disable SELinux and the firewall.


$ cd /etc/yum.repos.d
$ wget http://download.openvz.org/openvz.repo
$ rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ
$ yum -y install ovzkernel
$ vi /etc/sysctl.conf

In this file change the following lines:


net.ipv4.ip_forward = 1
kernel.sysrq = 1
net.ipv4.conf.default.proxy_arp = 0

In this same file add the following lines:


net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0

Install the OpenVZ service and have it start at boot:


$ yum install vzctl vzquota
$ chkconfig --add vz
$ reboot

Container configuration
After this, you need to create or download an OS template. For keeping this post easy, I use a precreated template:


$ cd /vz/template/cache
$ wget http://mirror.proserve.nl/openvz/contrib/template/precreated/centos-5-i386-default.tar.gz

Now you can create your first OS container from this template. The information we need is an IP address, hostname and nameservers.


$ vzctl create 101 --ostemplate centos-5-i386-default --conf vps.basic --ipadd 10.73.11.150 --hostname c1.survion.net
$ vzctl set 101 --name c1 --nameserver "10.73.11.1 10.73.11.2 10.73.11.3" --diskspace 10G:10G --save
$ vzctl set 101 --userpasswd root:password --save

Each container instance is represented by an CTID. In the previous steps we used id 101 the identify the container. The configured information can now be found in the file /etc/vz/conf/101.conf. Be sure to use your own secret password for the last command.

You can now start this container and list processes


$ vzctl start 101
$ vzctl exec 101 ps ax

You can also issue an enter command:


$ vzctl enter 101

You would then authenticate as root inside the container. On the console you can perform commands as if you would on a normal machine.

That's it! You now have a CentOS container running on top of your CentOS installation.

Links and additional information
Information on how to create a template can be found on http://wiki.openvz.org/OS_template_cache_preparation

OS Templates can be found at:

If you download the minimal template you can not enter the container. This is because the container does not have a console configured. You could still issue commands using the 'vzctl exec' command and be able to configure the console, but YMMV.

You can read more detailed information about the installation on the following websites:

Edit: An interesting experiment performed by Scott Dowdle can be found on his blog called 'How many containers?'.