STOP! MAKE SURE YOU HAVE FOLLOWED THE “Load Balancing with LVS-TUN Part 1″ ARTICLE PRIOR TO FOLLOWING THIS ARTICLE

The following document has all the configuration files needed to properly setup the Web Node aspect of lvs-tun. This is a continuation of the lvs-tun Load Balancing article I created previously.

I have gone through and commented the config files so that there is an explanation behind many of the important variables that are needed in order to get LVS-TUN working properly. There are several steps that need to be completed which I’ve listed below. Please follow them in order.

1. Prerequisites
2. The Tunnel Device
3. sysctl Parameters
4. iptables
5. Next Steps

Prerequisites
This document assumes that…
• You have already setup your load balancers using the lvs-tun Load Balancer article.
• You have installed and configured apache.
• You are running CentOS 5.5 or greater. This has not been tested on any other Linux Distribution at this time.
• You have already shared the ‘shared ip’ from within your hosting provider to all web nodes.

The Tunnel Device
We will now want to create the tunl0 (note, it’s spelled with a lowercase l, and the number 0) device on your Web nodes. On each node, we want to create a new file called ifcfg-tunl0 on all of your web nodes, and paste the following config inside ::


DEVICE=tunl0
TYPE=ipip
# IPADDR is the shared Public IP
IPADDR=x.x.x.x
NETMASK=255.255.255.255
ONBOOT=yes
Once that is complete, simply turn the device on by running ::
ifup tunl0


You will also want to test that the interface comes up upon reboot. You can do this by running the following commands ::


ifdown tunl0
service network restart


This should re-enable the tunl0 device. If it did not work, please double check that your configuration is correct and matches the configuration above.

sysctl Parameters
The next step is going to be adding some parameters to your /etc/sysctl.conf file. These parameters allow the tunnel to function and pass traffic to your web nodes.
The first step here will be to rename your current /etc/sysctl.conf file to /etc/sysctl.conf.BAK. Next, create a new sysctl.conf file with the following configuration ::


# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.tunl0.arp_ignore = 1
net.ipv4.conf.tunl0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.ip_local_port_range = 1024 61000
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.eth0.rp_filter = 0
net.ipv4.conf.eth1.rp_filter = 0
net.ipv4.conf.tunl0.rp_filter = 0


# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename
# Useful for debugging multi-threaded applications
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

Once this is completed, run the following command ::


sysctl -p


iptables
Finally, you will want to setup iptables to secure your server. For this, we will want to edit the /etc/sysconfig/iptables file. Delete everything that is in there, and replace it with the following rules ::


*filter
:INPUT DROP [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -j ACCEPT --proto 4
-A INPUT -i lo -j ACCEPT
-A INPUT -d [IP_OF_LB] -i tunl0 -j ACCEPT
-A INPUT -d [IP_OF_SECOND_LB] -i tunl0 -j ACCEPT
-A INPUT -d 127.0.0.0/255.0.0.0 ! -i lo -j REJECT --reject-with icmp-port-unreachable
-A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p icmp -j ACCEPT
COMMIT


Finally, we will restart iptables by running ::

service iptables restart


Next Steps
If everything was done correctly, you should be able to go back to the load balancer that currently has the shared IP attached, and run the ipvsadm command. The output should look something like this ::


IP Virtual Server version 1.2.1 (size=XXXX)

-> 10.x.x.x:http Tunnel 10 0 0


-> 10.x.x.x:http Tunnel 10 0 0

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP x-x-x-x.static.cloud- wrr

If your setup is working properly, you should now be able to browse to the Shared IP address and view the default document.
You now have a working lvs-tun setup!

The following blog post has all the configuration files needed to properly setup the load balancer aspect of lvs-tun. I have gone through and commented the config files so that there is an explanation behind many of the important variables that are needed in order to get LVS-TUN working properly. There are several steps that need to be completed which I’ve listed below. Please follow them in order.

*Please note, I will be adding the lvs-tun web node article next – I’m just working on the formatting.

Table of Contents
1. Prerequisites
2. Shared IP
3. Add Repositories
4. Installing Pulse
5. The lvs.cf file
6. sysctl parameters
7. iptables
8. Start the Service
9. Next Steps

Prerequisites

• This document assumes you have already built your load balancers and your web nodes, and you have at least 3 servers.
• This document also assumes you are running CentOS 5.5/RHEL 5.5 or greater. This has not been tested on any other Linux Distribution at this time.

Shared IP

First, you will want to add a shared IP Address without config to the primary load balancer. The IP address should then be shared with the load balancer and ALL of the web servers. I recommend getting all of the web servers standard packages (apache, php, etc.) setup PRIOR to setting up the load balancer. We will cover the lvs-tun web node configuration in a different article linked at the bottom of this page.

Add Repositories
You will want to add the EPEL and IUS repositories to your servers. You can do so by running the following commands ::

rpm -Uvh http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/epel-release-1-1.ius.el5.noarch.rpm
rpm -Uvh http://dl.iuscommunity.org/pub/ius/stable/Redhat/5/x86_64/ius-release-1.0-6.ius.el5.noarch.rpm
yum -y update


Installing Pulse

Next, you will want to install pirahna. While we will not actually use pirahna, it contains the service (pulse) necessary to make lvs-tun work. You will also want to install the standard installation packages at this time (sysstat, ntpd, etc). You will do this by running the following command ::


yum -y install piranha sysstat ntp telnet curl


Once completed, enable the service to start at boot. Note, though we installed pirahna, we are actually using the ‘pulse’ service. Pirahna itself is not used at all. ::


chkconfig pulse on
chkconfig ntpd on
chkconfig sysstat on


It’s been a while! I’ve been super busy with work. A quick update ::

• I’ve been studying for the first of my RHCA exams. Hopefully I’ll be able to start taking these soon.
• I have had 2 different positions at work in the last 6 months. I was part of our Professional Services team where I was building custom configurations for customers, and then I moved on to a more undefined role. I can’t talk a lot about the new role at this time, but I can say that it is technical with some Business Analysis thrown in for good measure.
• My grandfather passed away. It’s been rough.
• I quit using twitter – I’m 100% on facebook now

That’s mostly it. I know it’s a short post, but I’ll update this blog soon. I’ve got a lot of builds and configurations I’ve done, and I’ve written a few technical posts that I need to edit and post. Until next time!

• I just became the mayor of Valero on @foursquare! http://4sq.com/4AGIWG #

• I just became the mayor of Valero on @foursquare! http://4sq.com/4AGIWG #

• Just got accepted into the @socialscope beta. So far, it rocks! Wish u could disable pub time line and only get list updates tho. #
• Just got a pneumonia shot. I didn't even know they made those! #
• Wrapping up work for the day. #
• You know that feeling you get when you really make someone's day? Yea. I get that daily. Pay it forward. It feels great! #
• Woke up exhausted today – ended up working REALLY late on a build. Interesting stuff. I really like being able to hack away at new things! #
• I just ousted @ajhansentx as the mayor of Chick-fil-a on @foursquare! http://4sq.com/6ikdXJ #
• I just ousted alex c. as the mayor of Shell on @foursquare! http://4sq.com/agunKU #
• I just became the mayor of Taco Cabana on @foursquare! http://4sq.com/beDeLL #

• I just ousted alex c. as the mayor of Shell on @foursquare! http://4sq.com/agunKU #
• I just became the mayor of Taco Cabana on @foursquare! http://4sq.com/beDeLL #

• Woke up exhausted today – ended up working REALLY late on a build. Interesting stuff. I really like being able to hack away at new things! #
• I just ousted @ajhansentx as the mayor of Chick-fil-a on @foursquare! http://4sq.com/6ikdXJ #

• Wrapping up work for the day. #
• You know that feeling you get when you really make someone's day? Yea. I get that daily. Pay it forward. It feels great! #

• Just got a pneumonia shot. I didn't even know they made those! #