Vanderkussen.org - Have you tried turning it off and on again?

RHEV setup

2011-09-02T14:07:00+02:00

This blog post comes a little late because I did this RHEV setup at our company more than 6 months ago and it has been living in the drafts folder for some time now. Now with RHEV 3.0 Beta released I tought it’s time to publish this.

About a year and a half ago we started looking at alternatives for our VMWare ESXi setup because we wanted to add hypervisor nodes to our 2 existing nodes running VMWare ESXi. We also wanted the ability to live migrate vm’s between the nodes. At the same time Red Hat released RHEV 2.1 and being a Red Hat partner we decided to evaulate it.

We extended our existing setup with 2 Supermicro servers and a Supermicro SATA disk based SAN box configured as an iSCSI target providing around 8TB of usable storage.

Migration

To migrate our existing VM’s running on VMWare we used the virt-v2v tool that converts and moves VMWare machines to RHEV. This procedure can be scripted so you can define a set of VM’s you want to migrate in one go. Unfortunate these VM’s need to be powerd down. I noticed that if your vmdk folders/files are scattered around on you storage including differend folder names, the virt-v2v tool in some cases bails out. In our case I could understand why the tool refused to migrate some machines (it was quite a mess).

Hypervisors

You have 2 options to install the hypervisor nodes :

RHEV-H : stripped RHEL with a 100MB foorprint that provides enough to function as a hypervisor node.
RHEL : a default RHEL install you can configure yourself.

We created a custom profile on our Kickstart server so we could easily deploy hypervisors nodes based on a standard RHEL. By using a standard RHEL you can install additional packages later on which is not the case with a RHEV-H based install.

Once installed you can then add this node from within the manager interface to your cluster. Once added it will automatically install the necessary packages and becomes active in the cluster.

Storage

After adding hypervisor nodes you need to create “Storage Domains” based on either NFS, FC or iSCSI. Besides Storage Domains you also need to define an ISO domain to stock your installation images. If you want to migrate VM’s from VMWare or other RHEV clusters you need to create an Export Domain.

In each cluster one hypervisor node automatically gets the SPM (Storage Pool Manager) role defined. This host keeps track of where storage is assigned to. As soon as this host is put in maintenance or becomes unavailable another host in the cluster will take over the SPM role.

VM’s can use Preallocated disks (RAW) or Thin Provisioning (QCOW). For best performance Preallocated is recommended.

conclusion

We have been running this setup for more than a year now and haven’t had any real issues with it. We actually filed 2 support cases which have been resolved in newer releases of RHEV. At the moment we run around 100 VM’s and although I haven’t run any benchmarks yet, I see no real difference with our VMWare setup using FC storage. Although the product still has some drawbacks I believe it has a solid base to build on and already has some nice features like Live Migration, Load Balancing, Thin provisioning,..

Cons

RHEV-M (manager) runs on Windows
RHEV-M can only be accessed via IE (will probably change in 3.1)
Storage part is quite confusing at first.
API only accesible via Powershell
no live snapshots

In a few weeks I’ll probably start testing RHEV 3.0 which now runs on Linux on JBOSS. This makes me think if JBOSS clustering will work to get RHEV-M working in a HA setup.

Detect MTU size when using Jumbo Frames

2011-06-26T21:58:00+02:00

Recently I’ve setup an iSCSI target based on RHEL6 + tgt. After adding Logical Volumes to a target in the tgtd config file, the iSCSI target was discoverable and ready for use.

After testing this setup for a few days I wanted to tune the network traffic by enabeling Jumbo Frames. If you search on the interwebz you’ll most likely find information about adding “MTU=9000” ( for RHEL based clones) to the config file of the network interface.

The problem with Jumbo Frames is that when setting the mtu to high, you get fragmentation. Changing your mtu to 9000 will probably lead to fragmentation. If you don’t know this it can be quite hard to troubleshoot because you can still use ssh, ping the target etc.. but the iSCSI targets will keep failing.

You can easily check this with good old ping. Running this:

ping -M do -s 9000 <target_ip>

-M : MTU discovery strategy. “do” means “prohibit fragmentation”
-s : here you can specify the packet size

Gave me the following result :

From 10.0.0.13 icmp_seq=1 Frag needed and DF set (mtu = 9000)

Lower the packet size until you get a normal ping reply. This is the value you can use as your mtu size in your network card’s config file.

ping -M do -s 8900 <target_ip>

Switched to Jekyll

2011-06-11T14:07:00+02:00

It has been a while since I last blogged about a “decent” topic and actually it’s been a while blogging about anything. The reason is the lack of time and also some lazyness. But that should change now, and the first step I took was migrating my blog from Drupal to a Jekyll generated website. Not that Drupal is bad or anything, but it’s quite overkill and somehow felt not really productive while creating content.

So how did I end up with Jekyll?

Because I like using plain text files for writing (I use Latex quite a lot) I started looking for a blogging tool that used plain text files to store it’s content instead of a database. PyBloxsom, Blosxom came to mind, but then Jekyll popped up in one of my search results and immediatly liked it because it generates static content you can upload to any webserver. No more php, python, perl, Mysql or updating needed. However, you do need Ruby on the machine that does the generation.. One “drawback” of a static website is commenting and for a moment I was planning on dropping comments on my blog but went for Disqus which I actually quite like.

Now I have my blog stored in a git repository that rsyncs the static content to my webserver when I push my changes. As simple as that.

I really like the thought of using Markdown and vim to write my blogposts from now on (and of course the geeky factor of all this). The only thing left is improving the layout and sanatizing the setup a bit more

I'll be at LOAD (Linux Open Administrator Days)

2011-04-13T00:00:00+02:00

Getting DropBox to work with SELinux

2010-11-21T00:00:00+01:00

Recently Serge mentioned DropBox to me, and I remembered creating an account once but haven’t used or installed it in the last 2 years.

These days you also get lot more free space with your DropBox so I decided to start using it again.

So I started installing DropBox using the rpm from their website, but got an SELinux warning. Setroubleshootd perfectly explains what’s going on and the solution is trivial.


[root@localhost ~]# semanage fcontext -a -t execmem_exec_t '/home/vincent/.dropbox-dist/dropbox'
[root@localhost ~]# restorecon -vvF '/home/vincent/.dropbox-dist/dropbox'
restorecon reset /home/vincent/.dropbox-dist/dropbox context unconfined_u:object_r:user_home_t:s0->system_u:object_r:execmem_exec_t:s0

RHCE

2010-10-29T00:00:00+02:00

So today I went for the second time to sit the RHCE exam. This time the results were better then earlier.

RHCT components score: 100.0 RHCE components score: 100.0

RHCE certificate number : 805010290454578

The instructor mentioned that this was probably one of the last exams based on RHEL5.

Anyways, I’m glad I made it this time…

Fedora 14 Release party

2010-10-21T00:00:00+02:00

The date for the Belgium Fedora Release Party has been set. A bigger (as in “print this and hang it up in your office”) file has been attached.

Large image

LOAD dinner

2010-08-11T00:00:00+02:00

Yesterday evening we had a dinner with most of the LOAD organizers to catch up and have a nice get together. On the other hand we wanted to discuss some things regarding LOAD.

One of them was if we all wanted to organise a second edition of LOAD, and i can already tell you there’ll be a second edition of LOAD. For now that’s the only thing that’s certain. Date, location, talks, … are still undecided although the location will probably be the same.

We will soon archive the current website and start posting updates regarding the next edition.

We hope to see you all at the next edition of LOAD.

Fedora 13 Release Party @ hackerspace Ghent

2010-05-30T00:00:00+02:00

This time the Fedora 13 Release Party took place in the Hackerspace in Ghent, called WhiteSpace. As i arrived in the street where the Hackerspace is located i noticed someone who was also at the previous Release party.

A few minutes later biertie arrived with a big Fedora banner and signs to hang up so people would find their way to the HackerSpace (quite handy since the venue was like a small labyrinth).

Next thing was putting the PXE boot server i prepared in place so people could install Fedora 13 on their machine. After PXE booting some laptops to see if it stil worked we were good to go. Bert also created USB sticks with Fedora for some people.

The last day of Puppet Camp Europe was also taking place in Ghent and a lot of people came over to the Fedora Release Party and the HackerSpace became quite crowded.

bert ordered pizza’s with the Fedora budget he had so we wouldn’t starve. Drinks were provided by the HackerSpace for for very reasonable prices. Club Mate anyone?

After the food bert gave a quick presentation about the new stuff in Fedora 13. Dag Wieers also showed up and was aked to give a lightening talk about dstat . In the end his talk lasted more than one hour. He showed us a nice demo of Dstat’s features and talked with real passion about it, so thanks for your talk Dag!

After all this it was time for some chit chat….

Thanks everyone for being there and see you all at the next Release Party!

Ow yeah, thanks Kris for bringing me a Puppet Camp T-Shirt! I would also like to thank the people from HackerSpace Ghent for using their infrastructure to host the event. If you’re a geek living near Ghent, join them!

I’ve seen people take pictures, so if you read this put links to them in the comments please…thanks.

Fedora 13 Release Party

2010-05-16T00:00:00+02:00

With imminent release of Fedora 13, there’s also a release party scheduled. fedora 13 release party flyer

Failed RHCE

2010-05-03T00:00:00+02:00

Last Friday I took the RHCE exam. Unfortunate i didn’t pass the RHCE part of the exam. These where the numbers I received :

RHCT components score: 100.0% RHCE components score: 62.5%

All I needed was an extra 7.5% to pass the RHCE part of the exam (70% is the required minimum). If I had some extra time to get everything finished I might have made it. You really have no time to look up stuff if you’re stuck somewhere.

Anyway,next time we’ll get there….

Linux Open Administration Days

2010-04-12T00:00:00+02:00

Loadays has ended and I found it was good.

During the week of LOAD we did the wireless network setup using Linksys WRT/WAP54* devices. We actually expected problems, but beyond some little hickups the wireless worked quite well (Gryp’s megamachine had probably something to do with it :) ).

On Saterday we setup the projectors and power outlets so the event could start. After some time the first speakers showed up for breakfast provided by the school where LOAD took place.

Saturday evening we had a pizza party with a big pile of pizza’s and some beers

I didn’t have a real schedule planned but I attended some talks like FusionInventory, Large MySQL setups, Bacula, Observer, Selinux. I skipped most of the configuration management tools, but attended the last part of the puppet talk which was quite speedy :).

There was only one talk that needed to be cancelled, Pieter Colpaert couldn’t make it because of a train crash.

From the comments we got from people it seems that they would like to see a second edition of LOAD which is a good thing..

All in all things went well and I would like to thank some people.

First of all the people from the school who came working during their weekend and provided a lot of stuff like internet access, infrastructure, food, drinks…

Then the people who joined the crew the last few days. Tosh</a> for arranging the hotel, dim0</a> for doing the taxi part and Gh0sty for helping out with infrastructure and stuff.

And of course the other people who helped organizing LOAD, speakers and visitors.

If i forgot someone please forgive me…

LOAD countdown

2010-03-31T00:00:00+02:00

Just 10 days left until LOAD !

After a few weeks of planning, meeting and arranging stuff it’s almost there, the Linux Open Administration Days.

This week the schedule got finalized and it looks rather good. There’s a lot of differend stuff like open spaces, tutorials, regular talks,…

On Saturday evening there’s a social/beer event not far from the event venue (it also has some pizza’s involved).

I hope i’ll see you there !

R.I.P. Remco

2010-03-21T00:00:00+01:00

Our Rottweiler passed away a few weeks ago. So I’m posting this as a way to remember him.Remco, you’ll be missed. I’m sure they’ve spaghetti sauce in dog heaven.

My blog is running on Drupal

2010-02-11T00:00:00+01:00

I moved my Wordpress Blog to Drupal. Not that Wordpress is bad, but I’ve to create some websites and I’m planning on using Drupal for them. So this is a good way to get the hang of it.

Event advertisement

2010-01-24T00:00:00+01:00

I made some advertisement @work.

Dust cleaning on Thinkpad

2010-01-15T00:00:00+01:00

Fan started to make some strange noises. After some dust removing all is fine agian.

Routed Xen setup

2010-01-13T00:00:00+01:00

Yesterday i needed to setup a Xen machine where only the MAC address of the host (dom0) was allowed by the firewall. Because of this, a bridged setup could not be used since all domU’s will be using their own MAC address.

So i needed a routed setup instead. That way the host (dom0) will be used as a “router” for the domU’s and the firewall would only see the MAC of the dom0 host when communicating.

Configuration is quite easy. Edit /etc/xen/xend-config.sxp and comment out the bridge related stuff. Then uncomment or add the following lines to enable routing:

(network-script network-route)
(vif-script     vif-route)

Then in /etc/sysctl.conf make sure you’ve these 2 lines to enable routing. The Proxy arp is needed if you have domU’s on differend networks.

net.ipv4.ip_forward = 1
net.ipv4.conf.all.proxy_arp = 1

Restart xend, reboot the domU’s and you’re done (you can also just reboot the host).

LOAD logo

2009-12-23T00:00:00+01:00

I created a logo for the LOAD event. This is what i have for now, but I’ll probably make some other designs to. If you’ve comments or suggestions, please don’t hesitate to put them here.

F13 : I voted

2009-11-30T00:00:00+01:00

I voted for Gloriana.

Fedora 12 release party

2009-11-29T00:00:00+01:00

Yesterday was the Fedora 12 release party in Antwerp (Belgium). It was held at the Don Bosco school which had a good infrastructure for this event. A nice room, drinks, food, network, … So, thanks to them for providing this in their spare time !

I came a little earlier because I needed to setup a pxe server so people could get Fedora 12 installed on their machines if they wanted. After a while more people showed up (thanks Kris for being on time :) ) to listen to Bert and Bart's talk about the new Fedora and the Fedora community here in Belgium.

Later on Martin Langhoff from the OLPC project came in to talk a little about the OLPC, which I think is nice little device and has some cool features for the kids to play with.

This was the first Fedora Release party here and all in all it was a nice event and I hope we can do this on a regular basis so we can make the Belgium Fedora community grow.

Fedora 12 Release Party

2009-11-18T00:00:00+01:00

Fedora 12 has been released, and for this there’s a release party being held (probably the first Fedora release party in Belgium).

This will be a relative small event where the focus lies on getting together. You’ll also have the opportunity to install Fedora 12 on your machine.

More info on the Fedora Wiki.

RHEL SSL certificate error

2009-10-30T00:00:00+01:00

Yesterday i had a machine that wouldn’t register with the red Hat network. It gave me the following error : “The SSL certificate failed verification”. After some searching i noticed that the machine was still living in 2001, so all i had to do was set the time right.

Auto create home dirs with Samba

2009-10-20T00:00:00+02:00

If you’re using Samba to provide network shares to your users, it might be usefull to autmatically create the their homedir when they access it the first time. This can be done by adding this line to the homes directive : root preexec = /usr/local/sbin/mkhomedir.sh %U

This is the mkhomedir.sh script itself :

#!/bin/bash

if [ ! -e /storage_users/DOMAIN/$1 ]; then
mkdir /storage_users/DOMAIN/$1
chown $1:"DOMAIN+Domain Admins" /storage_users/DOMAIN/$1
fi
exit 0

Thesis : anti spam by MTA selection

2009-10-05T00:00:00+02:00

I’m publishing the Thesis that I and my fellow student Joost Ringoot made for our Graduate training in evening school. It might be useful to someone.

Projectwerk_2008_AntiSpam

Slow DNS resolving

2009-09-22T00:00:00+02:00

The other day one of our internal DNS needed to be shut down for maintenance. During that time people complained about slow responsiveness of the applications running on our RHEL machines. Because the DNS maintenance was the only change we made at that time we started looking in that direction.

As it turned out, it seems that Linux is trying to contact the first DNS server in the resolv.conf file for every DNS query. The second one is queried after x number of seconds

I added the following options to /etc/resolv.conf to make better use of the second nameserver in the list.

options rotate
options timeout:1

Keep VPN Connection alive

2009-08-07T00:00:00+02:00

Recently i had a VPN connection with OpenSwan that didn’t stay alive. Adding the following option to the config fixed it

dpdaction=restart

Wordpress on LightTPD

2009-08-02T00:00:00+02:00

I’m running this blog on LightTPD now. It runs a lot faster now on my 64MB VPS.

64MB VPS and wordpress

2009-07-28T00:00:00+02:00

I switched from Typo to Wordpre

64MB VPS

2009-07-28T00:00:00+02:00

I switched from Typo to Wordpress . I thought it would run a little faster on the 64MB VPS i use on my server. But WP uses MySQL as a backend and that doesn’t really help.

Apache is also quite a memory hog on systems with low memory, but with some Apache and MySQL tuning my blog isn’t going OOM anymore :).

I guess Apache and MySQL are a little to heavy on resources. I’m going to tune some more, but if that doesn’t help much I’m switching to LightTPD instead of Apache.

Sorry if thing run a little slow for the moment….

Typo with Phusion Passenger

2009-07-22T00:00:00+02:00

I’m running this blog now with the Phusion Passenger mod for Apache.

Normally this Apache directive should be enough to start your rails app.

<VirtualHost *:80>
servername mydomain.tld
documentroot /var/www/html/typo/public
</VirtualHost>

I had to change it to the following though to get it to work.

<VirtualHost *:80>
servername mydomain.tld
PassengerEnabled on
documentroot /var/www/html/typo/public
</VirtualHost>

An item with the same key has already been added

2009-07-08T00:00:00+02:00

When you try to export a virtual machine and get the following error : "An item with the same key has already been added'. There is probably a snapshot attached to the Virtual Machine.

You can't export the VM with snapshots you've got to remove them first

Installing Fedora 11

2009-06-13T00:00:00+02:00

During the installation of Fedora 11 on my laptop, the installation process did a suggestion for a hostname.

I swear i didn't enter anything myself ....

speedtest meme

2009-06-12T00:00:00+02:00

VMWare vSphere

2009-05-24T00:00:00+02:00

Today i attended a VMWare vSphere event. I was waiting for the fist one who mentioned the "Cloud" word, as this is the buzz word these days. I must say, this didn't take long :)

There was a whole presentation of the new VMWare vSphere product that they claim is "the best platform for building cloud infrastructures".
During this presentation i could see the resemblance with the older (but still used) mainframe technologie, and how we're going back to this sort of infrastructure. Virtual Desktop Infrastructure (VDI) is the next step in virtualization.

Although the "Cloud" is mostly considered a hype, it has a lot of possibilities and advantages.

Cisco is also getting on the Cloud boat with their Nexus 1000v. It's a Virtual Switch for VMWare, but with a full blown IOS, so network management can be done by the networking guys. The switch can span multiple ESX machines.

I'm curious on what Red Hat will be doing later on this year with RHEL 5.4 and KVM. I hope their oVirt technology becomes stable soon, because VMWare's management tools are really good.

Nikon D70

2009-05-24T00:00:00+02:00

I bought a second hand D70 to dive into some photography.

The difference between a regular compact device we've been using until now and this is quite ... well different.

Virtualbox for Leonidas

2009-04-28T00:00:00+02:00

Sun has released VirtualBox 2.2.2. It has a Fedora11 (Leonidas) package…always nice to see.

Typo 5.3

2009-04-18T00:00:00+02:00

On my way upgrading typo, al lost all my data (i must stop this habit).