Me in IT - Articles

Access thepiratebay.org from The Netherlands

Robert de Bock — Tue, 17 Jan 2012 13:22:10 +0000

Long story short: Visit The Pirate Bay through Me in IT Consultancy instead of typing it in the URL directly and you'll be able to download torrents again, because from The Netherlands, using Ziggo or XS4All, it's going to be difficult to access thepiratebay.org.

Short story long: With Apache, mod_proxy and mod_proxy_html you can make other website available through a Location on another website. If the web server is in a different region, chances are you'll be able to visit "blocked" websites.

To technically make this work, I used these ingredients:

CentOS - I used 5, but 6 should work too.
Apache - Just install it with "yum install httpd".
mod_proxy_html - I created an SRC RPM for mod_proxy_html.
configuration files - See below.

The configuration looks like this:

<VirtualHost *.80>
...
 ProxyRequests off

 ProxyPass /thepiratebay.org/ http://thepiratebay.org/
 ProxyPass /static.thepiratebay.org/ http://static.thepiratebay.org/
 ProxyPass /rss.thepiratebay.org/ http://rss.thepiratebay.org/
 ProxyPass /torrents.thepiratebay.org http://torrents.thepiratebay.org

 <Location /thepiratebay.org/>
  ProxyPassReverse /
  ProxyHTMLEnable On
  ProxyHTMLURLMap http://thepiratebay.org /thepiratebay.org
  ProxyHTMLURLMap http://static.thepiratebay.org /static.thepiratebay.org
  ProxyHTMLURLMap http://rss.thepiratebay.org /rss.thepiratebay.org
  ProxyHTMLURLMap //static.thepiratebay.org /static.thepiratebay.org
  ProxyHTMLURLMap http://torrents.thepiratebay.org /torrents.thepiratebay.org
  ProxyHTMLURLMap / /thepiratebay.org/
  RequestHeader unset Accept-Encoding
 </Location>

 <Location /static.thepiratebay.org/>
  ProxyPassReverse /
  ProxyHTMLEnable On
  ProxyHTMLURLMap http://thepiratebay.org /thepiratebay.org
  ProxyHTMLURLMap http://static.thepiratebay.org /static.thepiratebay.org
  ProxyHTMLURLMap http://rss.thepiratebay.org /rss.thepiratebay.org
  ProxyHTMLURLMap //static.thepiratebay.org /static.thepiratebay.org
  ProxyHTMLURLMap http://torrents.thepiratebay.org /torrents.thepiratebay.org
  ProxyHTMLURLMap / /static.thepiratebay.org/
  RequestHeader unset Accept-Encoding
 </Location>

 <Location /rss.thepiratebay.org/>
  ProxyPassReverse /
  ProxyHTMLEnable On
  ProxyHTMLURLMap http://thepiratebay.org /thepiratebay.org
  ProxyHTMLURLMap http://static.thepiratebay.org /static.thepiratebay.org
  ProxyHTMLURLMap http://rss.thepiratebay.org /rss.thepiratebay.org
  ProxyHTMLURLMap //static.thepiratebay.org /static.thepiratebay.org
  ProxyHTMLURLMap http://torrents.thepiratebay.org /torrents.thepiratebay.org
  ProxyHTMLURLMap / /rss.thepiratebay.org/
  RequestHeader unset Accept-Encoding
 </Location>

 <Location /torrents.thepiratebay.org/>
  ProxyPassReverse /
  ProxyHTMLEnable On
  ProxyHTMLURLMap http://thepiratebay.org /thepiratebay.org
  ProxyHTMLURLMap http://static.thepiratebay.org /static.thepiratebay.org
  ProxyHTMLURLMap http://rss.thepiratebay.org /rss.thepiratebay.org
  ProxyHTMLURLMap http://torrents.thepiratebay.org /torrents.thepiratebay.org
  ProxyHTMLURLMap //static.thepiratebay.org /static.thepiratebay.org
  ProxyHTMLURLMap / /rss.thepiratebay.org/
  RequestHeader unset Accept-Encoding
 </Location>
...
</VirtualHost>

Zabbix triggers with "flap-detection" and a grace period.

Robert de Bock — Wed, 09 Nov 2011 10:03:26 +0000

Monitoring an environment with some monitoring system gives control, so it's pretty important. But it can be a challenge to setup a monitoring system; it should not alert too fast, but also not too slow.

Nagios uses "flap detection" to prevent many ERROR's and OK's being sent right after each other. Zabbix calls this "hysteresis". Zabbix's hysteresis is rather difficult to understand, so I'd like to share some triggers that I have setup for Zabbix that implement both flap detection/hysteresis and grace.

Grace can be defined like this: "When a value is higher (or lower) then a threshold, make sure it's a little lower (or higher) as the threshold that caused the trigger to alert, before recovering a trigger." I know; it's not easy to understand... Let's look at some examples.

Thresholds that should be above a certain value

With values that need to be below a threshold, like cpu load, number of users logged in or number of processes running:

({TRIGGER.VALUE}=0&{TEMPLATE:CHECK[ITEM].min(300)>ALERTVALUE)|({TRIGGER.VALUE}=1&{TEMPLATE:CHECK[ITEM].max(300)<RECOVERYVALUE)

Just to clarify the different part of the trigger:

{TRIGGER.VALUE} makes sure the first part (before the |) is evaluated when there is no alert, the part after | indicates the trigger should be on/in alert.
.min(300) makes sure the values are minimally as high as ALERTVALUE for 300 seconds.
The last part (after the |) makes sure the trigger recovers when the measured value is lower than RECOVERYVALUE for 300 seconds.

For example CPU load with an ALERTVALUE of 5 and a RECOVERYVALUE of 4:

({TRIGGER.VALUE}=0&{Template_Linux:system.cpu.load[,avg1].min(300)}>5)|({TRIGGER.VALUE}=1&{Template_Linux:system.cpu.load[,avg1].max(300)}<4)

Thresholds that should be below a certain value

With values that need to be above a threshold, like percentage diskspace free, number of inodes free or number of httpd processes running:

({TRIGGER.VALUE}=0&{TEMPLATE:CHECK[ITEM].max(300)<ALERTVALUE)|({TRIGGER.VALUE}=1&{TEMPLATE:CHECK[ITEM].min(300)>RECOVERYVALUE)

For example disk space of /var free in percent with an ALERTVALUE of 10 and a RECOVERYVALUE of 11:

({TRIGGER.VALUE}=0&{Template_Linux:vfs.fs.size[/var,pfree].max(300)}<10)|({TRIGGER.VALUE}=1&{Template_Linux:vfs.fs.size[/var,pfree].min(300)}>11)

These rather complex triggers will prevent spikes of load or diskusage to cause an alert, but the drawback it that you might miss certain interesting spikes too. Overall my opinion is that a monitoring system should not drive people crazy because alerts will be ignored when too many are received.

Examples for batch on linux

Robert de Bock — Wed, 26 Oct 2011 11:25:58 +0000

Linux has a few ways to schedule jobs to be executed. I am sure most are familiar with crontab and at, but batch is lesser known.

"batch" can be used to: (from the man-page on batch)
executes commands when system load levels permit; in other words, when the load average drops below 0.8, or the value specified in the invocation of atrun.

So:

crontab is used for periodic scheduling.
at is used for executing something once at a specific time.
batch can be used to execute commands when your system has resourses.

You can also combine crontab and batch. Imagine you need to run a sequence of command in a specific order every hour. crontab does not guarantee one command is finished when it executs the next command.
batch can be used from crontab like so:

crontab -l
* * * * /usr/bin/batch now /usr/local/bin/prepare-something.sh
* * * * /usr/bin/batch now /usr/local/bin/process-something.sh
* * * * /usr/bin/batch now /usr/local/bin/report-something.sh

This batches these three commands in a specific order, one after the other, when the systemload is not too high.

One specific situation where I use this; Drupal needs to run a program (cron.php) every hour. crontab would be perfect for that, but when the load is too high, it's not a problem that this program is executed a little later. This is what I have setup:

* * * * /usr/bin/batch now /usr/bin/wget -o /dev/null -O /dev/null http://1.example.com/cron.php
* * * * /usr/bin/batch now /usr/bin/wget -o /dev/null -O /dev/null http://2.example.com/cron.php
* * * * /usr/bin/batch now /usr/bin/wget -o /dev/null -O /dev/null http://3.example.com/cron.php

This ensures that every hour cron.php is ran, but not if the systemload is too high (0.8 or more). One disadvantage of this solution; when your system is overloaded for a long period of time, these batch jobs pile up, then when the load drops below 0.8, all batched commands will be executed. Happily Drupals cron.php will not consume that much resources when it's ran twice.

Release scheme for RPM based Linux distributions

Robert de Bock — Tue, 27 Sep 2011 06:58:46 +0000

It can be rather confusing what the differences and similarities are on Fedora, Red Hat Enterprise Linux and CentOS. Especially with different versions. This article explains what release schedule and relations the various RPM based Linux distributions have.

Fedora is a Red Hat sponsored community project. Fedora is release approximately every 6 month. Fedora "supports" (supplies updates) for 13 month only. Clearly this is a development distribution.

Red Hat picks up a Fedora version and adds a few patches and call that "Red Hat Enterprise Linux". Red Hat supports that version for quite some time. Red Hat releases more conservatively; every 2 years. Red Hat supports a release for about 5 years after releasing, making this distribution much more "enterprise".

Fedora - Red Hat release relationship
Fedora release	Red Hat release
Fedora Core 3	Red Hat Enterprise Linux 4
Fedora Core 6	Red Hat Enterprise Linux 5
Fedora Core 13	Red Hat Enterprise Linux 6

CentOS picks up the source code that Red Hat published for Red Hat Enterprise Linux. The CentOS community patches the artwork and very few other things. CentOS "supports" (provides updates) for as long as Red Hat supplies updates to Red Hat Enterprise Linux.
Interesting to know; once you have choosen to use a certain main version of CentOS, you'll automatically update to the most recent child-version of that release when using "yum update". So; if you install "CentOS 5.0" and run "yum update", you will automatically have "CentOS 5.7". (at the time of this writing.)

So; this makes this image:

Apache Tomcat 7 spec file RPM

Robert de Bock — Thu, 14 Jul 2011 14:14:56 +0000

I tried to find an RPM for Apache Tomcat version 7, but could not find one. You can use this one, it requires the source code, downloadable from Apache Tomcat's website, under "Source Code Distributions".

This SPEC file creates an RPM "apache-tomcat" that installs in /opt/tomcat, and the default web applications (apache-tomcat-manager, apache-tomcat-ROOT, apache-tomcat-docs, apache-tomcat-examples, apache-tomcat-host-manager). An init-script is included at the bottom, that needs to be available in the SOURCES directory, named "apache-tomcat-iniscript".

Downloads:
SOURCE:
Apache Tomcat SRC rpm

x86_64:
Apache Tomcat 86_64 rpm
Apache Tomcat ROOT application 86_64 rpm
Apache Tomcat docs 86_64 rpm
Apache Tomcat example application 86_64 rpm
Apache Tomcat host manager application 86_64 rpm
Apache Tomcat manager application 86_64 rpm

So far it's been fine, but any comments would be appreciated.

apache-tomcat.spec:

Name: apache-tomcat
Version: 7.0.20
Release: 1
Summary: Open source software implementation of the Java Servlet and JavaServer Pages technologies.
Group: Productivity/Networking/Web/Servers
License: Apache Software License.
Url: http://tomcat.apache.org
Source: %{name}-%{version}-src.tar.gz

BuildRoot: %{_tmppath}/%{name}-%{version}-build
BuildRequires: ant
BuildRequires: ant-trax
Requires: java-1.6.0-openjdk
BuildArch: x86_64

%description
Apache Tomcat is an open source software implementation of the Java Servlet and JavaServer Pages technologies. The Java Servlet and JavaServer Pages specifications are developed under the Java Community Process.

%package manager
Summary: The management web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description manager
The management web application of Apache Tomcat.

%package ROOT
Summary: The ROOT web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description ROOT
The ROOT web application of Apache Tomcat.

%package docs
Summary: The docs web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description docs
The docs web application of Apache Tomcat.

%package examples
Summary: The examples web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description examples
The examples web application of Apache Tomcat.

%package host-manager
Summary: The host-manager web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description host-manager
The host-manager web application of Apache Tomcat.

%prep

%setup -q -n %{name}-%{version}-src
# This tells ant to install software in a specific directory.
cat << EOF >> build.properties
base.path=%{buildroot}/opt/apache-tomcat
EOF

%build
ant

%install
rm -Rf %{buildroot}
mkdir -p %{buildroot}/opt/apache-tomcat
mkdir -p %{buildroot}/opt/apache-tomcat/pid
mkdir -p %{buildroot}/opt/apache-tomcat/webapps
mkdir -p %{buildroot}/etc/init.d/
mkdir -p %{buildroot}/var/run/apache-tomcat
%{__cp} -Rip ./output/build/{bin,conf,lib,logs,temp,webapps} %{buildroot}/opt/apache-tomcat
%{__cp} %{_sourcedir}/apache-tomcat-initscript %{buildroot}/etc/init.d/apache-tomcat

%clean
rm -rf %{buildroot}

%pre
getent group tomcat > /dev/null || groupadd -r tomcat
getent passwd tomcat > /dev/null || useradd -r -g tomcat tomcat

%post
chkconfig --add %{name}

%preun
if [ "$1" = "0" ] ; then
 service %{name} stop > /dev/null 2>&1
 chkconfig --del %{name}
fi

%files
%defattr(-,tomcat,tomcat,-)
%dir /opt/apache-tomcat
%config /opt/apache-tomcat/conf/*
/opt/apache-tomcat/bin
/opt/apache-tomcat/lib
/opt/apache-tomcat/logs
/opt/apache-tomcat/temp
/opt/apache-tomcat/pid
%dir /opt/apache-tomcat/webapps
/var/run/apache-tomcat
%attr(0755,root,root) /etc/init.d/apache-tomcat

%files manager
/opt/apache-tomcat/webapps/manager

%files ROOT
/opt/apache-tomcat/webapps/ROOT

%files docs
/opt/apache-tomcat/webapps/docs

%files examples
/opt/apache-tomcat/webapps/examples

%files host-manager
/opt/apache-tomcat/webapps/host-manager

%changelog
* Fri Aug 19 2011 - robert (at) meinit.nl
- Updated to apache tomcat 7.0.20
- Split (example) applications into their own RPM.
* Mon Jul 4 2011 - robert (at) meinit.nl
- Initial release.

apache-tomcat-initscript:

#!/bin/sh
#
# apache-tomcat
#
# chkconfig: - 85 15
# description: Jakarta Tomcat Java Servlets and JSP server
# processname: java
# pidfile: /var/run/apache-tomcat/pid

. /etc/rc.d/init.d/functions

# Set Tomcat environment.
USER=tomcat
LOCKFILE=/var/lock/apache-tomcat
export BASEDIR=/opt/apache-tomcat
export TOMCAT_HOME=$BASEDIR
export CATALINA_PID=/var/run/apache-tomcat/pid
export CATALINA_OPTS="-DHOME=$BASEDIR/home -Xmx512m -Djava.awt.headless=true"

case "$1" in
  start)
        echo -n "Starting apache-tomcat: "
        status -p $CATALINA_PID apache-tomcat > /dev/null && failure || (su -p -s /bin/sh $USER -c "$TOMCAT_HOME/bin/catalina.sh start" > /dev/null && (touch $LOCKFILE ; success))
        echo
        ;;
  stop)
        echo -n "Shutting down apache-tomcat: "
        status -p $CATALINA_PID apache-tomcat > /dev/null && su -p -s /bin/sh $USER -c "$TOMCAT_HOME/bin/catalina.sh stop" > /dev/null && (rm -f $LOCKFILE ; success) || failure
        echo
        ;;
  restart)
        $0 stop
        $0 start
        ;;
  condrestart)
       [ -e $LOCKFILE ] && $0 restart
       ;;
  status)
        status -p $CATALINA_PID apache-tomcat
        ;;
  *)
        echo "Usage: $0 {start|stop|restart|condrestart|status}"
        exit 1
        ;;
esac

service status somedaemon - status: unrecognized service

Robert de Bock — Fri, 08 Jul 2011 09:35:55 +0000

I created an initscript for a daemon recently it works well for starting and stopping, but "service status somedaemon" would return "status: unrecognized service"

After a bit of googling I found that the top few lines of the initscript are critical. It should be a describe here:

#!/bin/sh
#
# somedaemon
#
# chkconfig: - 85 15
# description: Some daemon
# processname: somedaemon
# pidfile: /var/run/somedaemon.pid

Here is what these lines do.

#!/bin/sh - This is a shell script.
# somedaemon - This is the name of the service used with for "service status somedaemon". If this is missing, you'll see: "status: unrecognized service".
chkconfig: - 85 15 - This tells chkconfig three things, firstly (the dash) at what runlevel to start it. A dash means default. Secondly at what priority to start and lastly at what priority to stop.
description: Some daemon - Not used.
processname: somedaemon - Not used.
pidfile: /var/run/somedaemon.pid - Used in /etc/rc.d/init.d/functions to determine if the process is running or not.

Linux interview questions

Robert de Bock — Fri, 10 Jun 2011 07:35:31 +0000

From time to time you might need to interview somebody for a Linux role. It's hard to think of good questions; you don't want to scare somebody with your questions, but you do want to know if the person is knowledgable.

Questions for a junior.

"People that use your Linux server complain it's slow. What tools would you use to check resource usage?"
top, sar, netstat -an, iostat, free, df.

"You discover a disk is full on your Linux server. What do you use to discover where the bigest files/directories are?"

To create a short report on the largest directories: du -sk * | sort -nr | head -n10
You might use "find" to find large files: find / -size +100M

"A system needs more disk space. You want to add a partition /var/log. You add a drive to the machine, it becomes /dev/sdb. What actions do you take to use this disk?"
Depending if you use LVM or not. If you don't use LVM:

"fdisk /dev/sdb" to add one partition so /dev/sdb1 becomes available.
"mkfs /dev/sdb1" to create a filesystem on it.
"mount /dev/sdb1 /var/log". Actually the data needs to be copied into the new /var/log.

If you do use LVM:

"fdisk /dev/sdb" to add one partition so /dev/sdb1 becomes available.
"pvcreate /dev/sdb1" to make it an LVM device.
"vgextend SomeVolumeGroup /dev/sdb1" to extend an existing Volume Group.
"lvcreate -L 1G -n SomeLogicalVolumeName SomeVolumeGroup" to create a new Logical Volume.
"mkfs /dev/SomeVolumeGroup/SomeLogicalVolumeName" to create a filesystem on it.
"mount /dev/SomeVolumeGroup/SomeLogicalVolumeName /var/log". Actually the data needs to be copied into the new /var/log.

Questions for a medior.

"People that use your Linux server complain it's slow. You've seen the disk usage is high. What can you do to improve this?"

Move away services, devide them over different server.
Add more memory so disk caching can be used better.
Look into the application, why is it reading/writing so much.
User faster disks.

"What happens in relation to DNS, SMTP and IMAP when I send send an email to example@example.com?"
My computer is likely to be configure to send the email to a mail server on port 25, SMTP protocol. That mailserver will query the DNS for the MX records of example.com. The mailserver that show up with the lowest priority will be contacted to deliver the email. That mailserver at example.com can accept the email and put in into the imap folder for the user or alias of example@example.com.

"You need to setup 100 Red Hat Enterprise Linux systems. If you don't want to walk around and eject and insert a boot CD 100 times, what options would you have?"
Kickstarting would help out. Install one machine as you like it, save /root/anaconda-ks.cfg to a webserver. Setup a PXE (DHCP, TFTP, HTTP, DNS) environment and use that kickstart file to install the rest.

Questions for a senior.

"You have destroyed /etc/pam.d/system-auth and can't login anymore. Another machine has a propper version of /etc/pam.d/system-auth. How would you fix that broken machine?"
The machine needs to be booted in single user mode so you don't get a login prompt. After that here are some option:

Start network, use "nc" to get and replace that file.
Mount a CD that has the package to fix the broken file.
Manually repair it.

"You have installed apache, php and mysql and a webapplication such as Drupal. The webapplication tries to send emails to an external mailserver but fails. What could be the cause when these items have been verified:

The web application is correctly configured to use the external mailserver.
It's possible to connect to the mailserver on the specified port from the command line.
The logs of the mailserver don't record anything when the web application tries to send email."
IPtables allows connections out to port 25.

SELinux could be blocking apache from using port 25 on an external system. The logfile /var/log/messages might inform you about it. To fix it issue "setsebool -P httpd_can_network_connect=1".

"What determines the load (w, uptime) of a system?"
The number of processes that are waiting for execution. These processes are in the run queue. Processes could be waiting for io, network or memory allocation.

Drupal on Amazon's Elastic Compute Cloud (EC2)

Robert de Bock — Tue, 25 Jan 2011 08:32:46 +0000

Hosting Drupal site in the Amazon EC2 cloud is not difficult. Here is a recipe I have used, first attempt was a Fedora 14 EC2 ami, but Fedora 14 comes with php 5.3, which can't be combined with Drupal 5.x. If you only have Drupal 6 (or Drupal 7) sites to host, you can use Fedora 14. If you want to use a "small" instance, please read this bug about readdir64_r. The fix for that bug is easy:

echo "hwcap 1 nosegneg" > /etc/ld.so.conf.d/libc6-xen.conf

Let's continue with Drupal on CentOS. Rightscale provides perfect CentOS amis that can be used on Amazons EC2 platform. If you install one, these are the steps I took to make it Drupal 5, Drupal 6 and Drupal 7 ready:

# Update the software.
yum -y update

# Set the timezone for this machine.
cp /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime

# This images came with 10Gb of ESB storage, I added another 64 Gb volume, here it's called "/dev/sdc".
# Use LVM to be ready to grow in the future.
pvcreate /dev/sdc
vgcreate vg0 /dev/sdc
lvcreate vg0 -L 32G -n var-www
lvcreate vg0 -L 1G -n var-lib-mysql
lvcreate vg0 -L 2G -n root

# Put filesystems on the logical volumes.
mke2fs -j /dev/vg0/var-www
mke2fs -j /dev/vg0/var-lib-mysql
mke2fs -j /dev/vg0/root

# Add the mountpoints to fstab.
echo "/dev/vg0/var-www /var/www/virtualhosts ext3 defaults 0 0" >> /etc/fstab
echo "/dev/vg0/var-lib-mysql /var/lib/mysql ext3 defaults 0 0" >> /etc/fstab
echo "/dev/vg0/root /root ext3 defaults 0 0" >> /etc/fstab

# Create the mountpoints.
mkdir /var/www /var/lib/mysql /root

# Mount all mounpoints in /etc/fstab.
mount -a

# Install the webserver.
yum -y install httpd
service httpd start
chkconfig httpd on

# Install the database server.
yum -y install mysql-server
service mysqld start
chkconfig mysqld on
/usr/bin/mysqladmin -u root password 'YourPassWord'

# Install PHP and all required Drupal php modules.
yum -y install php php-mysql php-mcrypt php-xml php-mbstring php-gd

# Add a single new file in the ESB root filesystem, that includes configurations from /var/www/conf.d
# Using this trick allows you to easily remount the volume on another host in case of troubles.
echo 'Include /var/www/conf.d/*.conf' >> /etc/httpd/conf.d/virtualhosts.conf
mkdir /var/www/conf.d

# Rightscale CentOS images comes with postfix and sendmail. Postfix is enabled, but sendmail is fine for me.
# First erase postfix.
yum -y erase postfix

# Now reinstall sendmail to fix a few permissions.
yum -y reinstall sendmail
service sendmail start
chkconfig on sendmail

# Reboot the box to make sure it's working properly.
reboot

Fedora 14 to be released

Robert de Bock — Wed, 27 Oct 2010 05:52:11 +0000

So it's not so long before Fedora 14 is released. Lets take a look at some features Fedora 14 will bring that look interesting to me.

An Amazon EC2 Image (AIM) for Fedora will be released! Finally move on from that old Fedora 8 image that was available on Amazon EC2.
For Desktop Virtualization, Spice is released. This is a KVM feature that enable super fast access to virtualized desktop systesm, Windows, Linux or any other.
Perl, Python and Ruby will be updated, nice but not very important to me.
OpenSCAP can ensure security compliance. Should be helpful to many customers.

All in all, a pretty good release although it's kind of hard to improve an already good distribution.

Many of the changes done in Fedora, will end up in RHEL. Current estimate is that RHEL 7 will be based on Fedora 16 (earliest) up to Fedora 19. (latest)

Moving a single Drupal instalation into a multisite environment.

Robert de Bock — Mon, 18 Oct 2010 08:12:07 +0000

If you'd like to move a single installation of Drupal into a multisite environment, use these steps, and replace example.com for your websites name. In this example, Drupal is installed in /var/www/drupal/ .

1. Change the webserver configuration.

This one is obvious, the website was first pointing to an individual installation of Drupal, it needs to be directed to the multi-site installation of Drupal.

2. Copy templates, settings.php and files into the multisite environment.

mkdir -p /var/www/drupal/sites/example.com/
cd /var/www/example.com
cp -Rip html/sites/all/themes html/sites/default/settings.php html/sites/default/files/ /var/www/drupal/sites/example.com/

3. Update the MySQL database with the new paths.

If you have used files (including images) on your website, the path needs to be updated. Earlier files were located on "sites/default/files/", but this will become "sites/example.com/files"

mysql -u root -pPaSsWoRd
USE examplecom;
UPDATE files SET filepath=REPLACE (filepath, 'default', 'example.com');
UPDATE files SET filepath=REPLACE ('filepath', 'images', 'sites/example.com/images') WHERE filepath REGEXP '^images.*';
QUIT;

4. Change the location of the icons for the selected theme.

Go to Administer - Site building - Themes - Your Theme "configure" and change the path to reflect the right one. Mostly this means changing the word "files/" to "sites/example.com/files/".

5. Restart the web server and clean up the old environment.

For Apache, that would be:

apachectl configtest
apachectl restart

Check the website, everything should work, maybe you have to reselect your template to make it look better. If all works well, remove the old code.

rm -R /var/www/example.com

Shrinking a filesystem with LVM

Robert de Bock — Mon, 02 Aug 2010 13:31:25 +0000

After an installation you might find some file systems are too large, they are almost empty. When you want to use that space for another file system, here are the steps you can take:
Imagine /opt is now 10 Gb, but 1 Gb would be sufficient.

Check if the file system is in use. Using lsof /opt you will get a list of processes that currently use /opt. Stop these processes.
Find out what device is used for /opt with df -h /opt or mount. In my example, I found /dev/mapper/VolGroup/opt hold files on /opt.
Unmount the filesystem, using umount /opt
Resize the filesystem using resize2fs /dev/mapper/VolGroup/opt 1G. This frees the "right" part of the disk that LVM will un-allocate in a moment. All data from the file system is on the "left hand side".
Run lvreduce -L 1G /dev/mapper/VolGroup-opt to shrink the logical volume. (It might warn you that you need to run e2fsck -f /dev/mapper/VolGroup-opt before you can continue.
Remount the filesystem with a command as mount /opt.

For /opt or any other filesystem that can easily be freed from open file handles, the above procedure works fine, but for "busy" filesystems, like /, /var, /usr, and so on, you'd have boot the machine without mounting filesystems. One way to do this is using the installation CD and starting up the "rescue" environment.

Forwarding one port to another

Robert de Bock — Tue, 13 Jul 2010 07:28:53 +0000

Here is a very simple stick to forward a TCP port from your local workstation to another host. Can be easy to use for debugging purposes:

mkfifo pipe ; cat pipe | nc -l 8080 | nc google.com 80 > pipe ; rm pipe

What this one does:
1) Create a fifo (First in First out) file. This is a very simple type of file, you can put stuff in there with an output redirect (>) and get stuff out there with cat for example. It acts as a temporary buffer.
2) Open that newly created pipe. Anything that gets in, will be printed. (and forwarded in this example to "nc")
3) Open a listening port on your local workstation, listening on port 8080.
4) Open a connection to google.com, on port 80.
5) Send al the output to the earlier created pipe.
6) Remove the pipe when done.

Have a look the netcat homepage, it's a great tool!

Creating an RPM of some binary

Robert de Bock — Tue, 15 Jun 2010 16:53:50 +0000

We've covered this topic before in this story about creating an RPM from a shell script, but this information might help you better understand how to create an RPM.

So; you've found a piece of software that has no RPM? (Or; your manager tells you to install a piece of software that the development department created.)

Normally you'd use ./configure ; make ; make install, here is how to put that all in an RPM.

Prepare your rpm building environment: (DO THIS AS A USER!)

$ sudo yum install rpm-build
$ mkdir -p RPMBUILD/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
$ echo "%_topdir /home/username/RPMBUILD" >> .rpmmacros

Now copy the software into that newly create structure.

$ cp software.tar.gz RPMBUILD/SOURCES/

And now create a "spec file" for the software. This basically explains rpmbuild how to make the software and what to put in the RPM. This is the most "tweakable" step and might require quite some time to get right. Put this into /home/username/RPMBUILD/SPECS/software.spec:

Name: software
Version: 0.23
Release: 1
Summary: Custom software to run enterprise servers.

Group: Applications/Internet
License: GPLv2
URL: http://meinit.nl/
Source0: %{name}-%{version}.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root

%description
This software runs all enterprise software as a daemon. It's been developed by Me in IT consultancy.

%prep
%setup -q

%build
make

%install
mkdir -p $RPM_BUILD_ROOT/usr/local/bin
install software $RPM_BUILD_ROOT/usr/local/bin/software

%files
%defattr(-,root,root)
%doc README
/usr/local/bin/software

%clean
rm -rf $RPM_BUILD_ROOT

%changelog
* Tue Jun 15 2010 Robert de Bock <robert@meinit.nl> - 0.23-1
- Initial build

Good to know; the %install refers to the temporary environment that rpm will create when building this RPM. The %files section refers to what will end up in the RPM. They should correspond; you can't %install a whole bunch of files and only include a few in the $files part. (rpmbuild will display the missing files.

The group can be any line out of /usr/share/doc/rpm-*/GROUPS

So; you are prepared, run this command to so if you got everything correct:

$ rpmbuild -ba software.spec

When it finally builds, you'll find the rpm in /home/username/RPMBUILD/RPMS/$arch/software-0.23-1.$arch.rpm

Setting up iSCSI (target/server and initiator/client) on RHEL

Robert de Bock — Mon, 19 Apr 2010 18:53:27 +0000

It's quite easy to setup an iSCSI environment on Red Hat Enterprise Linux. Try this easy setup to get a better understanding of iSCSI.

Ingredients:

Two (virtual) machines, a server 172.16.0.1 and a client 172.16.0.2.
Access to the "RHEL Cluster-Storage" channel on Red Hat Network.

N.B. SELinux must be disabled when using this recipe, iptables tcp port 3260 must be opened on the server.

On the server execute these commands to setup a 100 Mb iSCSI target. This target can later be mounted on the client(s).

# yum install scsi-target-utils
# cat /etc/tgt/targets.conf
<target iqn.2010-04.nl.meinit:node1.target1>
	backing-store /iscsi1.img
	initiator-address 172.16.0.2
</target>
# dd if=/dev/zero of=/iscsi1.img bs=1024 count=102400
# chkconfig tgtd on
# service tgtd start

Now on (all) client(s) follow these steps. (Please pay attention that only one client was give access in the configuration example above; 172.16.0.2.)

# yum install iscsi-initiator-utils

Start iscsi daemon.

# service iscsi start

To see what IQNs are available, run:

# iscsiadm -m discovery -t sendtargets -p 172.16.0.1

The result is a list of IQN(s) available. This discovery is a mandatory step of connecting to the iSCSI target.

# iscsiadm -m node -T iqn.2010-04.nl.meinit:node1.target1 -p 172.16.0.1 -l

If that all works, you have new SCSI devices available, check dmesg and start iscsi at boot time:

# chkconfig iscsi on

In this example the iSCSI target does not have a filesystem. Create it on the client and mount it at boot time:

# fdisk /dev/sda
# mkfs.ext3 /dev/sda1
# echo "/dev/sda1 /mnt ext3 defaults,_netdev 0 0" >> /etc/fstab

You are done, but these commands are quite useful when connecting to an unknown iSCSI device.

To see more about the IQN:

# iscsiadm -m node -T iqn.2010-04.nl.meinit:node1.target1 -p 172.16.0.1

Using "recording" in VI instead of being annoyed

Robert de Bock — Tue, 02 Feb 2010 09:25:43 +0000

Here is a trick you can use in VI if you need to repeat an action multiple times. Imagine you have this file:

hello
Workd
hello
Workd
hello
Workd

You want to change it to read "Hello world!" Here is what you can do in VI:

Go to the first occurrence of what you would like to have changed.
Enter "q" to initiate a recording.
Enter a letter or digit to save the recording under, like "a" or "1".
Do the actions you would like to repeat.
Enter "q" again to stop and save the recording.
Go to the line where you would like to start running the saved recording/macro.
Enter "@" followed by the letter of digit you have saved the recording under, like "a" or "1".

So, now you have learned what this (annoying) "recording" option can be used for!

Debugging an SSL connection

Fred Clausen — Thu, 28 Jan 2010 10:42:43 +0000

Imagine you would like to debug an SSL connection to a box, for example HTTPS. Using telnet or netcat, you are not able to do so, but openssl makes this possible, here is how:

First; connect to a machine:

$ openssl s_client -connect mail.google.com:443

You will be given the certificate details:

CONNECTED(00000003)
depth=2 /C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
verify return:1
depth=1 /C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
verify return:1
depth=0 /C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
  i:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
 1 s:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
  i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDIjCCAougAwIBAgIQHxn23jXdY6FCkYrVLMCrEjANBgkqhkiG9w0BAQUFADBM
MQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkg
THRkLjEWMBQGA1UEAxMNVGhhd3RlIFNHQyBDQTAeFw0wOTEyMTgwMDAwMDBaFw0x
MTEyMTgyMzU5NTlaMGkxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh
MRYwFAYDVQQHFA1Nb3VudGFpbiBWaWV3MRMwEQYDVQQKFApHb29nbGUgSW5jMRgw
FgYDVQQDFA9tYWlsLmdvb2dsZS5jb20wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJ
AoGBANknyBHye+RFyUa2Y3WDsXd+F0GJgDjxRSegPNnoqABL2QfQut7t9CymrNwn
E+wMwaaZF0LmjSfSgRSwS4L6ssXQuyBZYiijlrVh9nbBbUbS/brGDz3RyXeaWDP2
BnYyrVFfKV9u+BKLrebFCDmzQ0OpW5Ed1+PPUd91WY6NgKtTAgMBAAGjgecwgeQw
DAYDVR0TAQH/BAIwADA2BgNVHR8ELzAtMCugKaAnhiVodHRwOi8vY3JsLnRoYXd0
ZS5jb20vVGhhd3RlU0dDQ0EuY3JsMCgGA1UdJQQhMB8GCCsGAQUFBwMBBggrBgEF
BQcDAgYJYIZIAYb4QgQBMHIGCCsGAQUFBwEBBGYwZDAiBggrBgEFBQcwAYYWaHR0
cDovL29jc3AudGhhd3RlLmNvbTA+BggrBgEFBQcwAoYyaHR0cDovL3d3dy50aGF3
dGUuY29tL3JlcG9zaXRvcnkvVGhhd3RlX1NHQ19DQS5jcnQwDQYJKoZIhvcNAQEF
BQADgYEAicju7fexy+yRP2drx57Tcqo+BElR1CiHNZ1nhPmS9QSZaudDA8jy25IP
VWvjEgaq13Hro0Hg32ZNVK53qcXwjWtnCAReojvNwj6/x1Ciq5B6D7E6eiYDSfXJ
8/a2vR5IbgY89nq+wuHaA6vspH6vNR848xO3z1PQ7BrIjnYQ1A0=
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
issuer=/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
---
No client certificate CA names sent
---
SSL handshake has read 1778 bytes and written 343 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
   Protocol  : TLSv1
   Cipher    : AES256-SHA
   Session-ID: 53530BBF94619E255B7956A18D9B9F26241B2A1BF16F30C18C73C88A60200E5F
   Session-ID-ctx:
   Master-Key:
B40F4E1D533F88AF9248E6576CA4E4CFC2C4BD092816DB7EF9D4FE650EA62B4CAC1F23C36892866B40E3502E67D52CF1
   Key-Arg   : None
   Krb5 Principal: None
   Start Time: 1264674178
   Timeout   : 300 (sec)
   Verify return code: 0 (ok)
---

Now you can enter commands in plain text, just as you would using telnet or netcat:

HEAD / HTTP/1.0

Here is what you get:

HTTP/1.0 302 Found
Cache-Control: private
Location: http://www.google.com
Content-Type: text/html; charset=UTF-8
Content-Length: 218
Date: Thu, 28 Jan 2010 10:23:05 GMT
Server: GFE/2.0
X-XSS-Protection: 0

read:errno=0

An easy trick that can also be used to connect to SSL-ed STMP connections, IMAPS, POP3S, etc.

Making an RPM for a shell script.

Robert de Bock — Tue, 12 Jan 2010 10:21:28 +0000

So, you have written an enterprise quality shell script and would like to deploy it on serveral Red Hat based machines? Creating an RPM will make this easy to do. Here are the steps required.

1. Install rpmbuild so you may start to build your own RPMs.
2. Package your shell script into a tar.gz file and move that to /usr/src/redhat/SOURCES/

# tar -cvzf shell-script-0.1.tar.gz shell-script-0.1
# mv shell-script-0.1.tar.gz /usr/src/redhat/SOURCES/

3. Create a .spec file that describes where everything is.

# cat /usr/src/redhat/SPECS/shell-script.spec
Summary: The do it all script. (Enterprise quality)
Name: shell-script
Version: 0.1
Release: 1
URL:     http://meinit.nl
License: GPL
Group: Applications/Internet
BuildRoot: %{_tmppath}/%{name}-root
Requires: bash
Source0: shell-script-%{version}.tar.gz
BuildArch: noarch

%description
A shell script.

%prep
%setup

%build

%install
rm -rf ${RPM_BUILD_ROOT}
mkdir -p ${RPM_BUILD_ROOT}/usr/bin
install -m 755 shell-script.sh ${RPM_BUILD_ROOT}%{_bindir}

%clean
rm -rf ${RPM_BUILD_ROOT}

%files
%defattr(-,root,root)
%attr(755,root,root) %{_bindir}/shell-script.sh

%changelog
* Tue Jan 12 2010 Robert de Bock <robert@meinit.nl>
- Uberscript!

3. Build it!

# rpmbuild --bb /usr/src/redhat/SPECS/shell-script.spec

4. Install it!

# rpm -Uvh /usr/src/redhat/RPMS/noarch/shell-script-0.1.1.noarch.rpm

Ranges in shell scripts

Robert de Bock — Tue, 15 Dec 2009 07:23:33 +0000

I have been admiring the people who know how to use ranges in shell scripts. These people are faster and more fluent on the Linux command line than anybody without knowledge of ranges could be.

Here are some ranges or patters that you could use.

A sequence of characters, in this case 1 to 10, printed on one line.

$ echo "file{1..9}"
file1 file2 file3 file4 file5 file6 file7 file8 file9
$ echo file{s..z}
files filet fileu filev filew filex filey filez

The order of the range can be found in the man-page for "ascii".

A pattern that describes either a range, or a separated pattern.

$ echo file{1,2,4}
file1 file2 file4

Additional information can be found in the man page of bash, under "Brace Expansion".

Restore hidden files with Apple Mac OS X Time Machine

Robert de Bock — Thu, 03 Dec 2009 09:40:34 +0000

Apple's Time Machine works great, but restoring hidden files (files that start with a dot, like .ssh, .bashrc or .Trash) is difficult, but possible!

Time machine uses the settings as used by the Finder. So first step is to change Finders behaviour, to show hidden files. Execute this command (as a regular user) from within the Terminal.

$ defaults write com.apple.finder AppleShowAllFiles TRUE
$ killall Finder

Now you should be able to see extra files in the finder, like this:

Now start Time Machine and scroll back to the date you were sure a file existed.

Restore it and to hide all these (annoying) hidden files, revert to original Finder settings:

$ defaults write com.apple.finder AppleShowAllFiles FALSE
$ killall Finder

Nagios time check using SNMP

Robert de Bock — Wed, 25 Nov 2009 08:26:21 +0000

When you would like to retrieve the remotely configured time using SNMP and compare it to see how accurate the time is, here is a script to help you out.

This setup does not specifically require NTP to be running on the hosts that are checked, it just requires that the time is correct. Virtual machines for example are advised to have the appropriate "tools" installed to synchronize time. NTP is not desirable for virtual machines.

(Parts of the script are borrowed from http://spielwiese.la-evento.com/xelasblog/archives/27-SNMP-hrSystemDate.0-Datumsformat-anpassen.html)

This is the graph that is created:

The script:

#!/bin/sh

# Nagios plugin to report time difference as received via SNMP compared to the local time.
# Make sure the machine this script runs on (poller/nagios host) is using NTP.

usage() {
 # This function is called when a user enters impossible values.
 echo "Usage: $0 -H HOSTADDRESS [-C COMMUNITY] [-w WARNING] [-c CRITICAL] [-v VERSION]"
 echo
 echo " -H HOSTADDRESS"
 echo "     The host to check, either IP address or a resolvable hostname."
 echo " -C COMMUNITY"
 echo "     The SNMP community to use, defaults to public."
 echo " -v VERSION"
 echo "     The SNMTP version to use, defaults to 2c."
 echo " -w WARNING"
 echo "     The amount of seconds from where warnings start. Defaults to 60."
 echo " -c CRITICAL"
 echo "     The amount of seconds from where criticals start. Defaults to 120."
 exit 3
}

readargs() {
 # This function reads what options and arguments were given on the
 # command line.
 while [ "$#" -gt 0 ] ; do
  case "$1" in
   -H)
    if [ "$2" ] ; then
     host="$2"
     shift ; shift
    else
     echo "Missing a value for $1."
     echo
     shift
     usage
    fi
   ;;
   -C)
    if [ "$2" ] ; then
     community="$2"
     shift ; shift
    else
     echo "Missing a value for $1."
     echo
     shift
     usage
    fi
   ;;
   -w)
    if [ "$2" ] ; then
     warning="$2"
     shift ; shift
    else
     echo "Missing a value for $1."
     echo
     shift
     usage
    fi
   ;;
   -c)
    if [ "$2" ] ; then
     critical="$2"
     shift ; shift
    else
     echo "Missing a value for $1."
     echo
     shift
     usage
    fi
   ;;
   -v)
    if [ "$2" ] ; then
     version="$2"
     shift ; shift
    else
     echo "Missing a value for $1."
     echo
     shift
     usage
    fi
   ;;
   *)
    echo "Unknown option $1."
    echo
    shift
    usage
   ;;
  esac
done
}

checkvariables() {
 # This function checks if all collected input is correct.
 if [ ! "$host" ] ; then
  echo "Please specify a hostname or IP address."
  echo
  usage
 fi
 if [ ! "$community" ] ; then
  # The public community is used when a user did not enter a community.
  community="public"
 fi
 if [ ! "$version" ] ; then
  # Version 2c is used when a user did not enter a version.
  version="2c"
 fi
 if [ ! "$critical" ] ; then
  critical="120"
 fi
 if [ ! "$warning" ] ; then
  warning="60"
 fi
}

getandprintresults() {
 # This converts the date retreived from snmp to a unix time stamp.
 rdatestring=$( snmpget -v $version -c $community $host HOST-RESOURCES-MIB::hrSystemDate.0 | gawk '{print $NF}' )

 if [ ! "$rdatestring" ] ; then
  echo "Time difference could not be calculated; no time received."
  exit 3
 fi

 rdate=$( echo $rdatestring | gawk -F',' '{print $1}' )
 rtime=$( echo $rdatestring | gawk -F',' '{print $2}' | gawk -F'.' '{print $1}' )
 cldate=$( echo $rdate | gawk -F'-' '{printf("%4i",$1)}; {printf("%02i",$2)}; {printf("%02i",$3)};' )
 cltime=$( echo $rtime | gawk -F':' '{printf("%02i",$1)}; {printf("%02i",$2)}; {printf(" %02i",$3)};' )
 rdate_s=$( date -d "$cldate $cltime sec" +%s )
 ldate_s=$(date +'%s')
 
 # If the calculated difference is negative, make it positive again for comparison.
 difference=$(($rdate_s - $ldate_s))
 if [ "$difference" -lt 0 ] ; then
  positivedifference=$(($difference*-1))
 else
  positivedifference=$difference
 fi
 
 if [ "$positivedifference" -gt "$critical" ] ; then
  echo "Time difference is more than $critical seconds: $difference|diff=$difference"
  exit 2
 fi
 
 if [ "$positivedifference" -gt "$warning" ] ; then
  echo "Time difference is more than $warning seconds: $difference|diff=$difference"
  exit 1
 fi
 
 echo "Time difference is less than $warning seconds: $difference|diff=$difference"
 exit 0
 }

# The calls to the different functions.
readargs "$@"
checkvariables
getandprintresults

To implement it in Nagios, add these sniplets to nagios.cfg. (or any other applicable nagios file.)

The service for a group.

define service{
        hostgroup_name                  Servertype_Linux
        service_description             time
        _SERVICE_ID                     1856
        use                             SNMP-time
}

The service template.

define service{
        name                            SNMP-time
        service_description             time
        use                             generic-service
        check_command                   check_snmp_time!$_HOSTSNMPCOMMUNITY$!120!60
        max_check_attempts                      30
        normal_check_interval           5
        retry_check_interval            1
        notification_interval           0
        register                                0
}

The command.

define command{
        command_name                    check_snmp_time
        command_line                    $USER1$/check_snmp_time -H $HOSTADDRESS$ -C $ARG1$ -c $ARG2$ -w $ARG3$
}

Optimize only fragmented tables in MySQL

Robert de Bock — Tue, 17 Nov 2009 15:38:36 +0000

When you are using MySQL, you will (likely) have tables that can be fragmented. In MySQL terms this is called "OPTIMIZE".

You could simply OPTIMIZE every table in every database, but during an OPTIMIZE, the tables are locked, so writing is not possible.

To minimize the time that MySQL will be locked (and results cannot be written), here is a script that checks fragmentation of every table of every database. Only if a table is fragmented, the table is OPTIMIZED.

#!/bin/sh

echo -n "MySQL username: " ; read username
echo -n "MySQL password: " ; stty -echo ; read password ; stty echo ; echo

mysql -u $username -p"$password" -NBe "SHOW DATABASES;" | grep -v 'lost+found' | while read database ; do
 mysql -u $username -p"$password" -NBe "SHOW TABLE STATUS;" $database | while read name engine version rowformat rows avgrowlength datalength maxdatalength indexlength datafree autoincrement createtime updatetime checktime collation checksum createoptions comment ; do
  if [ "$datafree" -gt 0 ] ; then
   fragmentation=$(($datafree * 100 / $datalength))
   echo "$database.$name is $fragmentation% fragmented."
   mysql -u "$username" -p"$password" -NBe "OPTIMIZE TABLE $name;" "$database"
  fi
 done
done

Result will look something like this:

MySQL username: root
MySQL password:
...
database.cache_filter is 19% fragmented.
meinit.cache_filter	optimize	status	OK
database.cache_page is 35% fragmented.
meinit.cache_page	optimize	status	OK
...

You may comment out that line with OPTIMIZE TABLE in it, if you are just interested in seeing the fragmentation.

Failover on OpenBSD is so easy to setup using carp!

Robert de Bock — Wed, 11 Nov 2009 13:16:42 +0000

I am not the first (and last) to write about carp, the failover/vip/floating-IP solution OpenBSD is using. Many articles describe this topic including a very complete answer to a frequently asked question about carp.

If you are not familiar with IP failover situations; in case of carp/pulse/HSRP/VIP, an IP "floats" between different machines. One machine actually answers request to received packets, so this is an solution that knows of a MASTER of ACTIVE node .

A CARP interface (which is not physical) is bound to a physical interface. The physical interface advertises statuses so other CARP interfaces know about each other.

You can bind almost any service to a CARP interface, some examples are:

DNS
HTTP
NTP
Proxy/Squid

Services that store data/stadia locally are not very suitable for a CARP solution. Examples are: DHCP (because leases are stored localy), MySQL/PostgreSQL (because data is stored on a physical local storage) and SSH (because you can never be sure what machine you are connecting to.

Here is how to set it up. On both boxes add a file /etc/hostname.carp0 with this content:

inet 192.168.1.123 255.255.0 192.168.1.255 vhid 1 pass SeCrEt carpdev em0

Remember to activate the interface like this: (All your network cards will be (re-) configured!)

# sh /etc/netstart

In this case, 192.168.1.123 is the floating IP address and em0 is the physical device that carp0 is running on. Be aware that the other server's carpdev should be connected to the same LAN.

Now that this is done, you may access services on the newly created CARP device's IP address. You may also specifically bind applications to only the CARP device.

You may check the status using ifconfig: (Please not the "carp: MASTER" part, it tells you this machine is the master, all others are "BACKUP".)

# ifconfig carp0
carp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        lladdr 00:00:5e:00:01:02
        priority: 0
        carp: MASTER carpdev em0 vhid 1 advbase 1 advskew 0
        groups: carp
        inet6 fe80::200:5eff:fe00:102%carp0 prefixlen 64 scopeid 0x5
        inet 192.168.1.123 netmask 0xffffff00 broadcast 192.168.1.255

One limitation I found; you can not run dhclient on a carp interface, you will need to assign an IP address to the carp device. Please be aware that this would be a very odd setup; DHCP in a failover interface...

Your Soekris OpenBSD as a OpenVPN appliance

Robert de Bock — Wed, 21 Oct 2009 12:50:28 +0000

I have an existing network at home, but would like to be able to connect to it using a VPN every now and then. This enables me to access the fileserver, printer and so on.

My network contains an Apple Time Capsule as a nat router, an ethernet modem provided by my cable company Ziggo and devices such as laptops, that use the network.

A Soekris box I had lying around meets all requirements perfectly for a VPN-server. Here is how to set it up.

1. Forward UDP port 1194 from your router to your soekris box.

This one is easy enough, on Apple Mac OS X and a Time Capsule (or Airport Express) open AirPort Utility on your Mac, select the Time Capsule, click Manual Setup.
Go to Internet - NAT
Select the box "Enable NAT Port Mapping Protocol" and click on "Configure Port Mappings..."

Click on the "+" to add a portmapping. OpenVPN uses UDP port 1194, so map it from the "Public UDP Port(s)" to the "Private UDP Port(s)" on the "Private IP Address" of your soekris box. Fill in "OpenVPN" in the next "Description" field.

Finish your router configuration by pressing "Update". N.B. The network connection will be gone for a minute or two.

2. Install OpenVPN

I assume OpenBSD is already running on your Soekris box, otherwise check out how to install your soekris box with OpenBSD.
Add the package "openvpn". A dependency "lzo" will be added automatically.

3. Configure OpenVPN

Create a directory /etc/openvpn/keys:

soekris # mkdir -p /etc/openvpn/keys

And create the file /etc/openvpn.conf with this content:

port 1194
proto udp
dev tun0
ca /etc/openvpn/keys/ca.crt
cert /etc/openvpn/keys/server.crt
key /etc/openvpn/keys/server.key                                  
dh /etc/openvpn/keys/dh1024.pem
# This is the network that lives on the tun0 device.
# My regular network uses 10.0.1.0/24, so using
# 10.0.2.0/24 seems pretty logical.
server 10.0.2.0 255.255.255.0
ifconfig-pool-persist ipp.txt
# When clients connect, tell them that 10.0.1.0/24 can
# be reached through this tunnel. (You may also set this on the,
# client instead of "broadcasting" this...
push "route 10.0.1.0 255.255.255.0"
client-to-client
keepalive 10 120
comp-lzo
user _openvpn
group _openvpn
persist-key
persist-tun
status openvpn-status.log
verb 3

4. Create OpenSSL certificates

This is quite an abstract step. It boils down to this: on the server you will create a certificate authority (ca) key and certificate, also you will create a key and certificate for each client connecting and sign them using your newly create certificate authority. The certificate from the certificate authority (ca.crt) and client (client1.crt) and the key for the client (client1.key) will be distributed to all clients. That's a mouth full, but here is how to do it in steps:

soekris # cp -Rip /usr/local/share/example/openvpn/easy-rsa /etc/openvpn
soekris # cd /etc/openvpn/easy-rsa/2.0
soekris # cat vars
export EASY_RSA="`pwd`"
export OPENSSL="openssl"
export PKCS11TOOL="pkcs11-tool"
export GREP="grep"
export KEY_CONFIG="/etc/openvpn/easy-rsa/2.0/openssl.cnf"
export KEY_DIR="/etc/openvpn/keys"
echo NOTE: If you run ./clean-all, I will be doing a rm -rf on $KEY_DIR
export PKCS11_MODULE_PATH="dummy"
export PKCS11_PIN="dummy"
export KEY_SIZE=1024
export CA_EXPIRE=3650
export KEY_EXPIRE=3650
export KEY_COUNTRY="NL"
export KEY_PROVINCE="UT"
export KEY_CITY="Utrecht"
export KEY_ORG="Me in It Consultancy"
export KEY_EMAIL="robert@meinit.nl"

N.b. Please change the KEY_ values to match your personal settings.

Now execute these steps, as stolen from The OpenVPN homepage.

soekris # . vars
soekris # ./clean-all
soekris # ./build-ca
soekris # ./build-key-server server
soekris # ./build-key client1
soekris # ./build-key client2
soekris # ./build-key client3
soekris # ./build-dh

Once again; send the newly created file /etc/openvpn/keys/ca.crt, /etc/openvpn/keys/client1.crt and /etc/openvpn/keys/client1.key to the machine using the vpn connection.

5. Configure the OpenBSD Packet Filter

This step enables client to reach your local network using network address translation. At the bare minimum, add this rule to your pf configuration in /etc/pf.conf

nat pass on sis0 from !(sis0) to any -> (sis0)

sis0 is a physical interface that connects the Soekris box to my local area network.

Also, make sure the packet filter is enabled and is using your pf.cofn

soekris # pfctl -e
soekris # pfclt -f /etc/pf.conf

And finally make sure it works after a reboot:

soekris # echo "ps=yes" >> /etc/rc.conf.local

6. Start OpenVPN on the server

Wow, almost there, let's start the software:

soekris # /usr/local/sbin/openvpn --config /etc/openvpn/server.conf --key /etc/openvpn/keys/server.key

Some debugging information will scroll down your screen.

7. Make sure OpenVPN starts at boot time

Add these lines to your /etc/rc.local.

# Add your local startup actions here.

echo " openvpn"
/usr/local/sbin/openvpn --config /etc/openvpn/server.conf --key /etc/openvpn/keys/server.key >> /var/log/openvpn.output &

echo '.'

8. Configure the client(s)

I use Mac OS X to connect to OpenVPN. You will have to install some extra software, your choices are:

Tunnelblick - (Free) Tunnelblick is a ready-to-use graphical OpenVPN client for Mac OS X
Viscosity - is an OpenVPN client for Mac, providing a rich Cocoa graphical user interface for creating, editing, and controlling VPN connections.

For now I am using the trail version of Viscosity because it looks great. Check out the screenshots below.

Managing a Linux Virtual Server

Robert de Bock — Mon, 31 Aug 2009 12:03:09 +0000

When you have setup an LVS you will need to administer it. Here are the tools you can use.

Discover what machine is the master.

# ipvsadm

A list of active services and it's IP-addresses will be printed on the active master.

Or, check /var/log/messages for a line like this:

pulse[$pid]: STARTING PULSE AS BACKUP

This clearly shows you the machine is a backup.

Failover from one machine to another.

You could simple reboot the active machine. Otherwise, stop the service pulse for a moment on the active server. The backup will discover this and configure the floating IP.
On the active machine, issue:

# /etc/init.d/pulse stop
# sleep 60
# /etc/init.d/pulse start

Add/delete a Virtual Service or Real Server.

Use the piranha web interface, located on port 3636 of either one of the load balancers. Remember to copy /etc/sysconfig/ha/lvs.cf to the backup machine as well.
After you have altered the configuration, restart pulse on the active machine. (Be aware; this makes services unavailable for a couple of seconds.

# ipvsadm
[services are printed]
# /etc/init.d/pulse restart
# ipvsadm
[services should be printed in a couple of seconds.]

Making a loadbalancer with CentOS using Linux Virtual Server

Robert de Bock — Mon, 24 Aug 2009 10:06:54 +0000

When you are new to all terminology Red Hat is using, it can be challenging to understand what actions to take to create a simple load-balancer with Linux. Here is some information to get you started.

Read more on the CentOS Virtual Server Administration pages.

Terminology:

piranha This is a package that provides a configuration interface for setting up LVS.
ipvs The name of the module that allows a load-balancing on Linux.
ipvsadm A package (and a command) that makes administrating ipvs possible. Be aware, you either user piranha-gui or /etc/sysconfig/ipvsadm to configure ipvs.
LVS Linux Virtual Server - the project name of all tools used in this document.
pulse A service (/etc/init.d/pulse) that runs on both active and backup machine and opens a port to make checking possible. piranha-gui configures everything, pulse actually activates all configurations.
nanny A process started by pulse to monitor the realservers.
nat Network Address Translation. A common combination with LVS. In case NAT is uses, the loadbalancer/director/LVS accepts traffic on one the VIP and sends traffic to the realservers. Be aware, the current implementation of ipvsadm does not properly use Source NAT, it currently does not rewrite the source address.
direct routing A method of routing available in LVS. Traffic is received on the VIP, sent through to the realservers. The realservers receive the traffic as if it was sent to the VIP, so the VIP must be configured on the realservers. To avoid arp-caches to register the wrong MAC-address-IP-address combination, a few "tricks" must be done.
wlc Weighted Least Connections, an algorithm to balance load to realservers.
VIP Virtual IP. The IP-address the service is configured on.
RIP Real server IP. The IP-address of a real-server.
realserver The server providing the actual service. This can be Linux, Windows, whatever.

Here is an overview of a possible setup:

Steps to finish before eternal succes:

Install piranha-gui

# yum install piranha-gui

(modify 172.16.0.0/24 to whatever network you are using on the realserver network.)

Configure services

# chkconfig httpd on
# chkconfig piranha-gui on
# chkconfig pulse on
# sed -i 's/net.ipv4.ip_forward = 0/net.ipv4.ip_forward = 1/' /etc/sysctl.conf
# echo "*nat
:PREROUTING ACCEPT [46:3346]
:POSTROUTING ACCEPT [431:32444]
:OUTPUT ACCEPT [431:32534]
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
*filter
:INPUT ACCEPT [4186:1094786]
:FORWARD ACCEPT [729:111992]
:OUTPUT ACCEPT [4266:388099]
-A FORWARD -i eth1 -j ACCEPT
COMMIT" > /etc/sysconfig/iptables
# chkconfig iptables on
# sed -i 's/SELINUX=enabled/SELINUX=disabled/' /etc/sysconfig/selinux

Configure via webinterface

Execute these steps to be able to use the piranha-gui web interface:

# service httpd start
# piranha-passwd
# service piranha-gui start 

Open a browser and open your servers IP-address, port 3636. URL looks something like this: http://192.168.202.50:3636

Synchronize /etc/sysconfig/ha/lvs.cf

You will need to have the piranha configuration the same on both machines. Here are some hints. Do this on both machines, just switch the IP-address.

# ssh-keygen
# scp .ssh/id_rsa* 192.168.202.110:./.ssh/
# cp .ssh/id_rsa.pub .ssh/authorized_keys

Now you should be able to connect to both machines without using a password.

# cat update-lvs.cf 
#!/bin/sh

copiedserialno=$(grep serial_no /tmp/lvs.cf | awk '{ print $NF }')
runningserialno=$(grep serial_no /etc/sysconfig/ha/lvs.cf | awk '{ print $NF }')

if [ "$copiedserialno" -gt "$runningserialno" ] ; then
 mv /tmp/lvs.cf /etc/sysconfig/ha/lvs.cf
fi
# crontab -l
* * * * * /usr/bin/scp /etc/sysconfig/ha/lvs.cf 192.168.1.46:/tmp > /dev/null
* * * * * /root/update-lvs.cf

Reboot the server.

To activate all changes, simply reboot the load-balancer.

Really minimal HOWTO for loadbalancing on CentOS 5.3 using ipvsadm

Robert de Bock — Thu, 20 Aug 2009 11:04:50 +0000

There are quite a few howto's for LVS, but all of them are quite extensive. To be honest; you'll need to read them at some point, but for now let's try to make a very minimal howto for setting up LVS.

Step 1: Install and configure a few settings.

Configure the director/loadbalancer to have two NIC's. One side on a routable network, the other side connected to the machine running the services, called realservers.

# chkconfig ipvsadm on
# sed -i 's/net.ipv4.ip_forward = 0/net.ipv4.ip_forward = 1/' /etc/sysctl.conf
# sysctl -p

If you want your realservers to be able to use the internet, execute these lines on the director. Replace YOURREALSERVERLAN for the network address of the network where the real servers are located, for example. 192.168.1.0

# iptables -A POSTROURING -s YOURREALSERVERSLAN/24 -j MASQUERADE
# service iptables save

Step 2: Tell IPVS how to configure itself for HTTP.

Fill in the blanks for PUBLICIP and REALSERVERIP. If you would like to add more servers to this virtual server, just repeast the last line a few times, changing the REALSERVERIP every time.

# echo "-A -t PUBLICIP:80" > /etc/sysconfig/ipvsadm
# echo "-a -t PUBLICIP:80 -r REALSERVERIP -m" >> /etc/sysconfig/ipvsadm
# service ipvsadm start

Step 3: Test it.

From a machine other then the redirector and/or the realserver, visit the ipaddress of your virtual ip.

N.B. I have spent quite some time trying to access the loadbalancer from the loadbalancer; this does not work.

Zabbix on OpenBSD

Robert de Bock — Fri, 14 Aug 2009 05:55:36 +0000

Installing and using the monitoring tool Zabbix on OpenBSD is quite simple. Take just these steps to get started.

Step 1: Install a few packages.

Use pkg_add to add these packages: (Versions could change over time.)

curl-7.19.3
gettext-0.17p0
jpeg-6bp3
libiconv-1.12
libidn-1.11
libxml-2.6.32p2
net-snmp-5.4.2.1p1
php5-core-5.2.8p0
php5-gd-5.2.8-no_x11
php5-pgsql-5.2.8
png-1.2.33
postgresql-client-8.3.6
postgresql-server-8.3.6
t1lib-5.1.0p1

Follow all hints the package manager tells you.

Step 2: Configure some items.

Make sure the apache daemons is started at boot time. (/etc/rc.conf.local)

Modify PHP to allow longer execution times and set the timezone:

$ grep max_execution_time /var/www/conf/php.ini
max_execution_time = 300
$ grep date.timezone /var/www/conf/php.ini
date.timezone = Europe/Amsterdam
$ sudo pkill httpd
$ sudo /usr/sbin/httpd

Step 3: Compile and install Zabbix.

Get the latest release of Zabbix, untar it and use these options to configure it:

./configure --enable-server --with-pgsql --with-net-snmp --with-libcurl --enable-agent

Use "make install" to install all items. The binaries will be placed in /usr/local.

Import database schemes as described in the Zabbix documentation, chapter 2.4.3: "Zabbix Server"

Create /etc/zabbix/zabbix_agentd.conf and /etc/zabbix/zabbix_server.conf by copying them from the untarred zabbix release:

# mkdir /etc/zabbix
# cp zabbix-1.6.5/misc/conf/zabbix_agentd.conf /etc/zabbix
# cp zabbix-1.6.5/misc/conf/zabbix_server.conf /etc/zabbix

Set DBName DBUser and DBPassword in /etc/zabbix/zabbix_server.conf.

Step 4: Automatically start Zabbix items.

$ cat /etc/rc.local
#       $OpenBSD: rc.local,v 1.39 2006/07/28 20:19:46 sturm Exp $

# Site-specific startup actions, daemons, and other things which
# can be done AFTER your system goes into securemode.  For actions
# which should be done BEFORE your system has gone into securemode
# please see /etc/rc.securelevel.

echo -n 'starting local daemons:'

# Add your local startup actions here.

if [ -x /usr/local/sbin/zabbix_agentd ] ; then
 echo -n ' zabbix_agentd'
 /usr/local/sbin/zabbix_agentd
fi

if [ -x /usr/local/sbin/zabbix_server ] ; then
 echo -n ' zabbix_server'
 /usr/local/sbin/zabbix_server
fi

echo '.'

Step 5: Install the webfrontend.

You are practically done, now copy the php files and visit your zabbix installation:

# cp -Rip zabbix-1.6.5/frontends/php/* /var/www/htdocs/zabbix/

That's it, not extremely difficult!

Bash completion for Mac OS X

Robert de Bock — Fri, 10 Jul 2009 07:20:20 +0000

If your are using the Terminal application of your Apple computer running Mac OS X, try bash programmable completion. It allow you to use the TAB key more often, for example in scp: (If you are using ssh-keys.)

$ scp shell01:/etc/pa
/etc/pam.d/         /etc/pam_smb.conf   /etc/passwd 
/etc/pam_pkcs11/    /etc/pango/         /etc/passwd- 
$ scp shell01:/etc/pa

The steps to start using this great utility are these:

Download the bash programmable completion .tar.gz file.
In a Terminal on your Apple, untar it. (tar -xvf bash-completion-20060301.tar
Copy the bash_completion shell-script to /etc (sudo cp bash_completion/bash_completion /etc)
Add the bash complation to your login script. (echo ". /etc/bash_completion" >> ~/.profile)
Start a new terminal and see the result!

Using DynDNS for iPhone OS 3.0 without crontab but with launchd

Robert de Bock — Fri, 03 Jul 2009 11:10:08 +0000

Apple iPhone OS 3.0 does not have crontab anymore. You are supposed to use launchd's facilities to execute something at a scheduled interval. Here is an example of a simple script to update the IP-address at DynDNS.org.

The script /var/mobile/update-dyndns.org contains:

#!/bin/sh

host="your-host-name.dyndns.org"
user="your-username"
pass="your-password"

oldip=$(cat /tmp/ip)
wget -o /dev/null -O - http://ip.serverchief.com/ > /tmp/ip
ip=$(cat /tmp/ip)

if [ "$oldip" != "$ip" ] ; then
 echo -n "$(date) "
 echo $(/usr/local/bin/wget -O - "http://$user:$pass@members.dyndns.org/nic/update?hostname=$host&wildcard=NOCHG&bacakmx=NOCHG"  2> /dev/null)
fi

The file /var/LaunchDaemons/org.dyndns.update.plist contains:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>  
	<key>Label</key>
	<string>org.dyndns.update</string>
	<key>ProgramArguments</key>
	<array> 
		<string>~/mobile/dyndns-update.sh</string>
	</array>
	<key>StartCalendarInterval</key>
	<dict>  
		<key>Minute</key>
		<integer>3</integer>
	</dict>
</dict>
</plist>

And execute:

# launchctl load org.dyndns.update.plist

Now your IP will be update every 3-rd minute. Have fun!

A shell script to measure network throughput on Linux machines.

Robert de Bock — Tue, 30 Jun 2009 11:58:53 +0000

Here is a shell script to see how many (kilo-, mega-, giga-, terra-) bytes pass a network interface. The output looks like this:

$ ./network-traffic.sh --help
Usage: ./network-traffic.sh [-i INTERFACE] [-s INTERVAL] [-c COUNT]

-i INTERFACE
    The interface to monitor, default is eth0.
-s INTERVAL
    The time to wait in seconds between measurements, default is 3 seconds.
-c COUNT
    The number of times to measure, default is 10 times.
$ ./network-traffic.sh         
Monitoring eth0 every 3 seconds. (RXbyte total = 706 Mb TXbytes total = 1 Gb)
RXbytes = 104 b	TXbytes = 194 b
RXbytes = 80 b	TXbytes = 188 b
RXbytes = 52 b	TXbytes = 146 b
RXbytes = 689 b	TXbytes = 8 Kb
RXbytes = 52 b	TXbytes = 146 b
RXbytes = 52 b	TXbytes = 146 b
RXbytes = 52 b	TXbytes = 146 b
RXbytes = 52 b	TXbytes = 146 b
RXbytes = 4 Kb	TXbytes = 4 Kb
RXbytes = 716 b	TXbytes = 5 Kb

Here is the script:

#!/bin/sh

usage(){
 echo "Usage: $0 [-i INTERFACE] [-s INTERVAL] [-c COUNT]"
 echo
 echo "-i INTERFACE"
 echo "    The interface to monitor, default is eth0."
 echo "-s INTERVAL"
 echo "    The time to wait in seconds between measurements, default is 3 seconds."
 echo "-c COUNT"
 echo "    The number of times to measure, default is 10 times."
 exit 3
}

readargs(){
 while [ "$#" -gt 0 ] ; do
  case "$1" in
   -i)
    if [ "$2" ] ; then
     interface="$2"
     shift ; shift
    else
     echo "Missing a value for $1."
     echo
     shift
     usage
    fi
   ;;
   -s)
    if [ "$2" ] ; then
     sleep="$2"
     shift ; shift
    else
     echo "Missing a value for $1."
     echo
     shift
     usage
    fi
   ;;
   -c)
    if [ "$2" ] ; then
     counter="$2"
     shift ; shift
    else
     echo "Missing a value for $1."
     echo
     shift
     usage
    fi
   ;;
   *)
    echo "Unknown option $1."
    echo
    shift
    usage
   ;;
  esac
 done
}

checkargs(){
 if [ ! "$interface" ] ; then
  interface="eth0"
 fi
 if [ ! "$sleep" ] ; then
  sleep="3"
 fi
 if [ ! "$counter" ] ; then
  counter="10"
 fi
}

printrxbytes(){
 /sbin/ifconfig "$interface" | grep "RX bytes" | cut -d: -f2 | awk '{ print $1 }'
}

printtxbytes(){
 /sbin/ifconfig "$interface" | grep "TX bytes" | cut -d: -f3 | awk '{ print $1 }'
}

bytestohumanreadable(){
 multiplier="0"
 number="$1"
 while [ "$number" -ge 1024 ] ; do
  multiplier=$(($multiplier+1))
  number=$(($number/1024))
 done
 case "$multiplier" in
  1)
   echo "$number Kb"
  ;;
  2)
   echo "$number Mb"
  ;;
  3)
   echo "$number Gb"
  ;;
  4)
   echo "$number Tb"
  ;;
  *)
   echo "$1 b"
  ;;
 esac
}
  
printresults(){
 while [ "$counter" -ge 0 ] ; do
  counter=$(($counter - 1))
  if [ "$rxbytes" ] ; then
   oldrxbytes="$rxbytes"
   oldtxbytes="$txbytes"
  fi
  rxbytes=$(printrxbytes)
  txbytes=$(printtxbytes)
  if [ "$oldrxbytes" -a "$rxbytes" -a "$oldtxbytes" -a "$txbytes" ] ; then
   echo "RXbytes = $(bytestohumanreadable $(($rxbytes - $oldrxbytes)))	TXbytes = $(bytestohumanreadable $(($txbytes - $oldtxbytes)))"
  else
   echo "Monitoring $interface every $sleep seconds. (RXbyte total = $(bytestohumanreadable $rxbytes) TXbytes total = $(bytestohumanreadable $txbytes))"
  fi 
  sleep "$sleep"
 done
}

readargs "$@"
checkargs
printresults