Welcome to DigLinux, OpenSource Knowledge Platform

Configure OpenLDAP Client for PAM Authentication in Ubnutu

2011-11-20T04:52:00.001+05:00

Previously in article OpenLDAP Setup I showed how to setup and configure basic OpenLDAP server with basic directory structure. By using that OpenLDAP structure we can implement centralized PAM Authentication. This is very useful when maintaining large amount of servers.

To configure Ubuntu as OpenLDAP client for PAM Auth.

Basic Assumptions:

We will be using root account to install and configure OpenLDAP client for PAM Auth.
We will be using "diglinux.com" as our Domain, as previously setup in OpenLDAP Server.
Admin user for managing OpenLDAP structure will be "Admin".
Password for Admin user will be "diglinux".
OpenLDAP Server IP address is 192.168.1.1
Internet is properly configured and working on the machine that we will be configuring OpenLDAP.

Installing OpenLDAP Client and Required Services

root@diglinux.com:~# apt-get install nss-updatedb libnss-ldap nscd libpam-ldap libpam-mkhomedir auth-client-config ldap-utils

All packaged play a different role in configuring OpenLDAP PAM Authentication. During installation of these packages, you will be asked some basic questions.

-----------------------------------------------------------------

1. LDAP server Uniform Resource Identifier:

-> ldap://192.168.1.1

2. Distinguished name of the search base:

-> dc=diglinux,dc=com

3. LDAP version to use:

-> Version 3

4. Make local root Database admin:

-> Yes

5. Does the LDAP database require login?

-> Yes

6. LDAP account for root:

-> cn=Admin,dc=diglinux,dc=com

7. LDAP root account password:

-> diglinux

8. Unprivileged database user:

-> cn=Admin,dc=diglinux,dc=com

9. Password for database login account:

-> diglinux

-----------------------------------------------------------------

Remember to change the settings as per the settings of your server, these settings we implemented on our OpenLDAP setup guide.

Once above settings are done you will need to edit file "/etc/ldap.conf"

Find

#bind_policy hard

and replace it with

bind_policy soft

Un-comment parameter

pam_password crypt

Save and exit this file.

Now execute this command, this command will add "LDAP" in /etc/nsswitch.conf , which tells where to look for credentials first.

root@diglinux.com:~# auth-client-config -t nss -p lac_ldap

Now we need to update the PAM Auth DB. For this execute this command

root@diglinux.com:~# pam-auth-update

Make sure you select "LDAP Authentication".

Final step is to restart some services.

root@diglinux.com:~# /etc/init.d/libnss-ldap restart && /etc/init.d/nscd restart

If everything goes without error, then you can authenticate PAM using OpenLDAP. To verify, create a user in OpenLDAP. Once user is created, issue below command to verify whether that user has been picked up by Linux or not

root@diglinux.com:~# getent passwd

This will show you a list of users present in system.

Till here you have a working OpenLDAP PAM Authentication. However an issue that can come is that user will not have their own home directory which is defined in OpenLDAP. PAM does provide a plugin that can dynamically create user home directory if not found.

We previously did the installation of this plugin, we only need to configure it. And it is very easy to configure.

For this we will need to create a plain text file "/usr/share/pam-configs/my_mkhomedir". Copy and Paste below lines in this file and then save and exit

#------------------------------------

Name: activate mkhomedir

Default: yes

Priority: 900

Session-Type: Additional

Session:

required pam_mkhomedir.so umask=0022 skel=/etc/skel

#------------------------------------

Do not copy the "----" lines. Once done now we have to update PAM. This will be done using below command.

root@diglinux.com:~# pam-auth-update

Here you will now see "activate mkhomedir", make sure this is checked.

If everything goes without errors, you will have a 100% working OpenLDAP PAM Authentication.

DigLinux on G+

2011-11-08T11:35:00.001+05:00

DigLinux on G+on

DigLinux has officially launched its Google+ Page, DigLinux will now regularly be updated. And we hope to bring on more authors to this platform.

OpenLDAP Setup on Ubunut 10.04.2

2011-11-03T09:55:00.000+05:00

In this article I will be covering how to install and configure basic OpenLDAP. Many people have asked me regarding this and yes there are many tutorials available on internet. No one is bound to use or follow this article.

I have tried to make things as easy as possible in covering all aspects of this subject.

Lets Start...

Basic Assumptions:

We will be using freshly installed Ubuntu Server 10.04.2 LTS.
We will be using root account to install and configure OpenLDAP and related services.
We will be using "diglinux.com" as our Domain.
Admin user for managing OpenLDAP structure will be "Admin".
Password for Admin user will be "diglinux".
Internet is properly configured and working on the machine that we will be configuring OpenLDAP.

Step1: Installing OpenLDAP

root@diglinux.com:~# apt-get update

root@diglinux.com:~# apt-get install slapd ldap-utils

Package "slapd" is used to install OpenLDAP on Ubuntu and package "ldap-utils" installs some useful scripts to play around with LDAP without restarting LDAP server.

Step2: Importing Basic LDAP Schema

root@diglinux.com:~# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/cosine.ldif
root@diglinux.com:~# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/inetorgperson.ldif
root@diglinux.com:~# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/nis.ldif
root@diglinux.com:~# ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/schema/misc.ldif

By default OpenLDAP comes with some basic predefined data types, by importing these schemas we add more data types are more often used.

Step3: Creating Basic Backend OpenLDAP Structure

Here we will be creating the basic back-end structure for out OpenLDAP Server.
First we will create a file, you can name it what ever you want, here I will name it as "diglinux.backend.ldif"

root@diglinux.com:~# vim diglinux.backend.ldif

Now copy and paste below lines in this file. Make sure that you replace "diglinux" with your domain and password with the password that you want to set.
--------------------------------------------------------------------------------------------------

# Load dynamic backend modules
dn: cn=module{0},cn=config
objectClass: olcModuleList
cn: module{0}
olcModulePath: /usr/lib/ldap
olcModuleLoad: {0}back_hdb

# Create directory database
dn: olcDatabase={1}hdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcHdbConfig
olcDatabase: {1}hdb
olcDbDirectory: /var/lib/ldap
olcSuffix: dc=diglinux,dc=com
olcRootDN: cn=Admin,dc=diglinux,dc=com
olcRootPW: diglinux
olcAccess: {0}to attrs=userPassword,shadowLastChange by dn="cn=Admin,dc=diglinux,dc=com" write by anonymous auth by self write by * none
olcAccess: {1}to dn.base="" by * read
olcAccess: {2}to * by dn="cn=Admin,dc=diglinux,dc=com" write by * read
olcLastMod: TRUE
olcDbCheckpoint: 512 30
olcDbConfig: {0}set_cachesize 0 2097152 0
olcDbConfig: {1}set_lk_max_objects 1500
olcDbConfig: {2}set_lk_max_locks 1500
olcDbConfig: {3}set_lk_max_lockers 1500
olcDbIndex: uid pres,eq
olcDbIndex: cn,sn,mail pres,eq,approx,sub
olcDbIndex: objectClass eq
################################
# Modifications
################################

dn: cn=config
changetype: modify

dn: olcDatabase={-1}frontend,cn=config
changetype: modify
delete: olcAccess

dn: olcDatabase={0}config,cn=config
changetype: modify
add: olcRootDN
olcRootDN: cn=Admin,cn=config

dn: olcDatabase={0}config,cn=config
changetype: modify
add: olcRootPW
olcRootPW: diglinux

dn: olcDatabase={0}config,cn=config
changetype: modify
delete: olcAccess

--------------------------------------------------------------------------------------------------

Once this file is created, you will need to import this in LDAP to create basic structure and add LDAP Admin User.

root@diglinux.com:~# ldapadd -Y EXTERNAL -H ldapi:/// -f diglinux.backend.ldif

Step4: Creating Basic OU Structure

Here we will be creating two basic OU's. This is pretty much like default in OpenLDAP, almost all tutorials on internet plays around with the same OU's.
You can change them as per your needs, but remember whatever you create here, you should have a good idea what you are doing.

We will create a file, you can name it what ever you want, here I will name it as "diglinux.ou.ldif"

Now copy and paste below lines in this file. Make sure that you replace "diglinux" with your domain and password with the password that you want to set.
--------------------------------------------------------------------------------------------------

# Tree root

dn: dc=diglinux,dc=com

objectClass: dcObject

objectclass: organization

o: diglinux

dc: diglinux

description: Tree root

# Creating Admin User

dn: cn=Admin,dc=diglinux,dc=com

objectClass: simpleSecurityObject

objectClass: organizationalRole

cn: admin

userPassword: diglinux

description: LDAP administrator

# Creating an OU named "people"

dn: ou=people,dc=diglinux,dc=com

ou: people

objectClass: organizationalUnit

objectClass: top

# Creating an OU named "groups"

dn: ou=groups,dc=diglinux,dc=com

ou: groups

objectClass: organizationalUnit

objectClass: top

#Adding a test user name "diglinux"

dn: uid=diglinux,ou=people,dc=diglinux,dc=com

objectClass: inetOrgPerson

objectClass: posixAccount

objectClass: shadowAccount

uid: diglinux

sn: com

givenName: DigLinux

cn: DigLinux

displayName: DigLinux

uidNumber: 1002

gidNumber: 1000

userPassword: diglinux

gecos: DigLinux

loginShell: /bin/bash

homeDirectory: /home/diglinux

shadowExpire: -1

shadowFlag: 0

shadowWarning: 7

shadowMin: 8

shadowMax: 999999

shadowLastChange: 10877

mail: info@diglinux.com

postalCode: 31000

l: Karachi

o: diglinux

mobile: +92 123 1234567

homePhone: +92 123 1234567

title: System Administrator

postalAddress:

initials: DL

--------------------------------------------------------------------------------------------------

Make sure that you make required changes in this file. Once all changes are done, save and exit.

Now its time to import these settings.

root@diglinux.com:~# ldapadd -x -D "cn=Admin,dc=diglinux,dc=com" -W -f diglinux.backend.ldif

Once you execute this command, this will ask for a password. And in our backend file we set the password as "diglinux", if you have changed the password, you will need to enter the same password here.

Now if everything goes fine and without any errors, you should have a running OpenLDAP server with basic structure.

You can verify this by using JXplorer a free windows based utility for managing OpenLDAP. This application can be downloaded from http://www.jxplorer.org/

Below is the snapshot of JXplorer connection settings to connect to LDAP server.

JXplorer Connection Settings

Mikrotik PHP PPP User Management System

2011-09-12T18:52:00.000+05:00

I have uploaded a simple PHP based Mikrotik PPP User Management System.

Please visit http://wiki.mikrotik.com/wiki/Basic_php_ppp_scripts for complete instructions.

You can request for more features.

Mikrotik Bash Script

2011-09-11T07:47:00.001+05:00

I have created a Wiki article on Mikrotik site and thought that I should share it with you all. Please follow the link below to view the article.

http://wiki.mikrotik.com/wiki/Useful_Bash_Scripts

Please feel free to provide your important feedback

MySQL Startup Script

2011-09-09T16:16:00.000+05:00

This is a MySQL start up script. I created this script when I faced issues in Ubuntu 10.04 LTS MySQL start up script.

This is a very handy small script which can be used not only for MySQL but for almost all services.

#! /bin/bash

JOB=/usr/sbin/mysqld

case $1 in

start)

$JOB &> /dev/null &

ps aux | grep mysqld | grep -v grep | awk '{print $2}' > /tmp/mysqld.pid

;;

stop)

kill -9 $(cat /tmp/mysqld.pid)

rm -f /tmp/mysqld.pid

;;

restart)

stop

start

;;

echo Oops! bad options

;;

esac

exit 0

Mikrotik script to create and email backup

2011-09-07T04:44:00.000+05:00

A small script I created after going through many scripts available at the Mikrotik wiki. This is a simple script that creates a backup file of the Mikrotik RouterOS and emails the file to any email address specified in the script.

:global file ([/system identity get name] . [:pick [/system clock get date ] 7 11] . [:pick [/system clock get date ] 0 3] . [:pick [/system clock get date ] 4 6] . ".backup")
/system backup save name=$file
/tool e-mail set server=smtp.example.com
/tool e-mail set from="backup@example.com"
/tool e-mail send to="reports@example.com" subject="Mikrotik RouterOS Firewall Backup" body="Attached is the backup file of Mikrotik RouterOS" file=$file

A very useful Disk Size Monitoring script

2011-09-07T04:27:00.000+05:00

This is a very useful and small script that I created quiet some time back when I was creating custom monitoring plugins for one my my clients.
Still I use this script to perform Disk Size Monitoring at many places that I am monitoring.

For this script to work properly, a small linux command line utility known as "mutt" needs to be installed. It is a very handy command line e-mail client. Normally this comes by default with linux, but just in case this is not present, simply use the native package manager (yum for redhat/centos/fedora and apt-get for debian/ubuntu) to install this utility.

Now for the script. It is a bit messed up.

[root@crystalnetworks.org home]# cat disksize.sh

#!/bin/bash
#
# Author : Mudasir Mirza

#Un-comment the following line to run the script in debug mode
#set -x

# Absolute path to CSV file that will be generated by script
OUTPUT="/tmp/size.csv"

rm -f $OUTPUT
NUM_COL=`df -h | head -n1 | awk '{ print NF }'`
for (( i=1; i<=$NUM_COL; i++ ))
do
COM[$i]=`df -h | head -n1 | awk -v "awkvar=$i" '{print $awkvar}'`
done
FLD=`echo ${COM[@]} | sed 's/\ /,/g'`
echo $FLD > $OUTPUT
P=`df -h | tail -n+2 | wc -l`
for (( i=1; i<=$P; i++ ))
do
CMD=`df -h | tail -n+2 | sed "${i}!d"`
CMDA=`echo $CMD | sed 's/\ /,/g'`
echo $CMDA >> $OUTPUT
done
echo | mutt -a $OUTPUT -s "Daily Disk Size Email" monitoring@example.com
###########################################################
# End of Scirpt

Brief VMware Intro

2011-08-31T05:06:00.000+05:00

For last 4 to 5 years I have worked on VMware Virtualization Platform and Linux based Virtual Hosts. In my very short experience, I found out some very important things that can help any System Admin who is managing VMware Infrastructure. I have also written some Perl and Bash based scripts to automate some the tasks that help me out in managing backups of VMs and keep an eye on some of the very important aspects.

Here I will share those scripts and those aspects with you all, and hope that anyone who faces the same situation can easily resolve it.

The main aspects are listed below, more to come very soon.

Disaster Recovery and Backups
When I started work on VMware, the main thing or concern of my client and employer was the backups of Virtual Machines and Data that is being saved on those virtual machines.
I will very soon be sharing a complete method of how to make this a very easy task to manage and be on top of everything. I have written few custom scripts to verify everything, and using the ALL TIME FAMOUS GhettoVCB scripts to take backups of VMs.

Service Console
Service Console is a port / console through which you are directly connected to the VMware Server. This is the most important part of VMware which needs to be configured very well.
One more thing that I have learned after my little experience is that, one should always have a secondary / backup service console.
I will also be explaining this in my coming posts, how to configure service console and how to make backup service consoles.

VM Recovery from Backup After Crash
This is something that initially created allot of issues for me, but as time passed and after going through allot of documents and communities this became very clear to me. In fact now I think this is the most easiest task that can be completed in few minutes.

Creating and Managing Datastores
What is a Datastore? Datastore is just like another partition or your can also take it as a special mount point, where all your data is kept in VMware. Datasore can be on the local hard disk or over network or every a USB device connected. Inside a normal Datastore, you will find Virtial Machines, VM Disks, VM Configuration files and any other files that you want to place in the Datacenter ( such as a separate folder for ISO files and some scripts, etc).

ESX Host behind VPN and Firewall
This is something that I have found to be very very useful and very secure method. I have managed many VMware ESX hosts, and believe me I think this is the most secure method to put your ESX host over internet. In this method what happens is that, your main ESX host is actually not on Public IP but is only accessible through a Local IP and that is also only when you are connected to a VPN Server that is running on one of the Virtual Machines. I usually use an OpenVPN server for this.

MySQL DB Backup Script

2011-07-27T15:41:00.000+05:00

I created this script for a project of mine that I was working on about 3 years back. This script takes MySQL backups based on the database names in a file.

You can modify and use this script as per your needs.

root@crystalnetworks:~# cat /root/scripts/dbackup.sh

******************************************************************************************************

#!/bin/bash

bk_dir="/backups/mysql" # dir where backups will be takes

dir=`date +%Y-%m-%d`

tm=`date +%H%M%S`

bk_date=`date +%F`

bk_time=`date +%H-%M-%S`

db_list_file="db_list" # list of DB dynamically created using MySQL

bk_log_file="backup.log" # while creating backups, messages will be logged

## DB Credentials

db_user="root"

db_pass="root"

## Creating Necessary directories

if [ ! -d $bk_dir/$bk_date ]; then

mkdir $bk_dir/$bk_date

if [ ! -d $bk_dir/$bk_date/$bk_time ]; then

mkdir $bk_dir/$bk_date/$bk_time

mysql -u $db_user -p$db_pass -Bse 'show databases' | grep -vE "information_schema|mysql" > $bk_dir/$db_list_file

cat $bk_dir/$db_list_file |while read line

mysqldump -u $db_user -p$db_pass --database $line --default-character-set=utf8 > $bk_dir/$bk_date/$bk_time/$line.sql

if [ "$?" -eq 0 ]; then

echo "backup sucessfull database: $line" >> $bk_dir/$bk_date/$bk_time/$bk_log_file

elif [ "$?" -eq 1 ]; then

echo "error during backup database: $line" >> $bk_dir/$bk_date/$bk_time/$bk_log_file

done

cd $bk_dir/$bk_date/$bk_time; gzip *.sql

echo "Compressed all SQL files successfuly" >> $bk_dir/$bk_date/$bk_time/$bk_log_file

******************************************************************************************************

PHP Web Spider / Hammer

2011-07-26T17:03:00.000+05:00

Recently I was working on a PHP script with simple tasks that were needed to be performed. Now as I read more about the PHP Treading function, I thought of creating a web-spider or you can say web-site load testing script based on PHP.
Currently this script is under heavy modification and performance tuning, but I thought it would be great to share my first draft with you people.
This script is heavily documented, you can just go through this script and understand the purpose of each and every line.

root@crystalnetworks:~# cat hammer.php
#!/usr/bin/php -q
<?php

##################################################################
# Author : Mudasir Mirza
# Date : 25-July-2011
# E-Mail : mudasirmirza@gmail.com
# Purpose : PHP based web spider, to put load on web-site.
# Limitations : HTTPS is not properly supported yet, Only one url can be given as input.
# Defaults : Default -c = 1 , Default -s = 300
# Note : -c , -s and -u are only supported arguments.
# License : Under GNU / GPL, if you want to re-distribute this script, please include the name of author.
##################################################################

# Set time limit to 0, to make sure script does not get any php timeout errors
set_time_limit(0);

# PHP Class taken from internet for Spidering the url provided
class wSpider
{
var $ch; # going to used to hold our cURL instance
var $html; # used to hold resultant html data
var $binary; # used for binary transfers
var $uri; # used to hold the url to be downloaded
function wSpider()
{
$this->html = "";
$this->binary = 0;
$this->uri = "";
}
function fetchPage($uri)
{
$this->uri = $uri;
$agent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"; # Defining user agent that will be shown to remote URL
if (isset($this->uri)) {
$this->ch = curl_init (); # open a cURL instance
curl_setopt($this->ch,CURLOPT_RETURNTRANSFER, 1); # tell cURL to return the data
curl_setopt($this->ch,CURLOPT_USERAGENT, $agent); # Setting User agent for the curl session, defined earlier
curl_setopt($this->ch,CURLOPT_URL, $this->uri); # set the URL to download
curl_setopt($this->ch,CURLOPT_FOLLOWLOCATION, true); # Follow any redirects
curl_setopt($this->ch,CURLOPT_BINARYTRANSFER, $this->binary); # tells cURL if the data is binary data or not
curl_setopt($this->ch,CURLOPT_VERBOSE, false); # Set verbose, this writes error to STDERR (standard Error)
curl_setopt($this->ch,CURLOPT_TIMEOUT, 1000); # Setting timeout for curl
curl_setopt($this->ch,CURLOPT_SSL_VERIFYPEER, true); # Setting curl to accept any SSL certficate blindly
$this->html = curl_exec($this->ch); # pulls the webpage from the internet
curl_close ($this->ch); # closes the connection
}
}
}

# Defining arguments that will be taken as input from user
$opts = getopt('c:s:u:');

# A foreach loop to go through each argument and set respective paratemers, using switch statement with foreach loop
foreach (array_keys($opts) as $opt) switch ($opt) {
case 'c': # What to do with the input argument "c", and the input value of it.
$concurrent = $opts['c']; # Setting the value of argumetn -c in variable $concurrent, number of process to start
break; # go out of switch statement for case "c"
case 's': # What to do with input argument "s", and the input value of if.
$proc_time = $opts['s']; # Setting the value of argument -s in variable $proc_time, this the process time.
break; # go out of switch statement for case "-s"
case 'u': # What to do with the input argument "u", and the input value of it.
if(!empty($opts['u'])){ # Checking if the argument "u" is not empty
$url = $opts['u']; # If argument "u is not empty, then set $url as the variable holding the value or argument "u"
} else { # if argument "u" is foung empty then
$url = ""; # Set the $url variable as an empty variable (nul)
} # End of if
break; # go out of switch statement for case "-u"
} # End of foreach loop

# Some basic validation

# Checking if input argument "-c" has any value or not, $concurrent is defined above which contains value of -c
if(empty($concurrent)){
# if $concurrent if found empty (null), print to screen below text
echo "\nNo Concurrent session defined, using default\n";
# Setting default value to $concurrent, default is 1
$concurrent = '1';
}

# Checking if input parameter "-s" has any value or not, $proc_time is defined above which contains value of -s
if(empty($proc_time)){
# if $proc_time is found empty (null), print to screen below text
echo "No Process time is defined, using default\n\n";
# setting default value to $proc_time, defaut 300
$proc_time = '300';
}

# Checking if input parameter "-u" has any valu or not, if no URL is defined script will not run. $url defined above
if(empty($url)){
# IF url is found empty (null), then below text will print on screen, this is the script usage text.
echo "Please specifiy URL\n";
echo "Usage ./blast.php [options] [url]\n";
echo "Example: ./blast.php -c 10 -s 100 -u http://www.yahoo.com\n";
echo "Options:\n";
echo "-c\tConcurrent processes to run\n";
echo "-s\tTime for each process in Sec\n";
echo "-u\tURL to hit with each process\n\n";
exit(1); # script will exit without performing any further tasks, with exit status code 1
}

# Print to screen the settings defined by user as input arguments.
echo "Concurrent processes set to $concurrent\n";
echo "Process session time set to $proc_time sec\n";
echo "URL To Spider is set to $url\n\n";

# Setting $child with value of $concurrent (concurrent processes to start)
$child = $concurrent;

# using for loop, with maximum iterations equals to the concurrent number processes that this script needs to start
for ($i=1; $i<=$child; $i++)
{
$pid = pcntl_fork(); # $pid is set to the process id of the main process using pcntl_fork() function
if ($pid == -1) # if $pid == -1, which means for some reason main process is unable to start child processes.
{
die("could not fork\n");
}
else if ($pid) # if $pid = 1, this means the process is the parent process, which is used to fork children processes.
{
exit(0); # exit the parent process after giving birth to children processes
}
else # if $pid not equals 1 or -1 then do the following
{
$cpid = pcntl_fork(); # $cpid will start child processes and will contain there pids using function pcntl_fork()
if ($cpid == -1) # if $cpid is = -1, then child process can not be reached, print on screen and die.
{
die("could not fork in child process\n");
}
if (!$cpid) # if $cpid != 1, then this is the child process.
{
# Here we define what to do with child process.
# A for loop will going to start with maximum iterations equals to the argument "-s" seconds defined by input.
# each iteration will call the Visit function which uses curl to load the web-site
# after the function Visit is called, the loop will sleep for 1sec, thats how i made the loop to iterate for the
# amount of time in seconds defined by input

$start_date = strtotime(date('Y-m-d H:i:s')); # Start time to calculate Proper Process running time
for ($k=1; $k<=$proc_time; $k++) { # Start of for loop, iterations = $proc_time
$end_date = strtotime(date('Y-m-d H:i:s')); # End time to check for process running time
$seconds = $start_date - $end_date; # Caclucate Seconds from start_date and end_date
if($seconds > $proc_time){ # if running time is greater then provided argument -s
exit(0); # then exit the process
}
$mySpider = new wSpider(); # creates a new instance of the wSpider
$mySpider->fetchPage($url); # fetches the home page of url provided by the user as input argument
sleep(1); # sleep for 1 second in the iteration of loop
}
exit(0); # Now exiting the child process properly, after the iterations of loop for given time are complete.
}
}
}
# End of Script
?>

########################################

This script should be used only for testing purposes, and the person using this script will be responsible for any outcomes related to System and / or Legal.

OpenVPN on Ubuntu 10.04.2

2011-07-26T16:45:00.000+05:00

This article is specially designed to help Linux Beginners in setting up OpenVPN on Ubuntu 10.04.2 platform. Server configuration and certificate generation steps on all Linux Distributions are almost the same. Installation of OpenVPN can differ from Distribution to Distribution.

Lets Start with OpenVPN !!!

Basic Assumptions.

We will be working on a freshly installed Ubuntu 10.04.2 LTS Server.
We will use root account for this article.
Two Ethernet cards are attached. (eth0 connected to internet and eth1 connected to local network with IP range 192.168.100.0/24. eth1 has IP Address 192.168.100.1/24)
Server is directly connected to Internet and is not behind any firewall. If server is placed behind firewall then port 1194 (tcp/udp based on your config) should be forwarded to OpenVPN Server.
We will not use same certificate for multiple clients.
We will not apply any restriction / access policy based on OpenVPN Certificates.
We will use 172.16.0.0/24 as our VPN IP Range.

Step 1 : (Setting root password and updating APTITUDE Repo)

mudasir@crystalnetworks:~# sudo passwd root

root@crystalnetworks:~# apt-get update

Step 2 : (Installing OpenVPN Using apt-get)

root@crystalnetworks:~# apt-get install openvpn

The above command takes care of installing OpenVPN on Ubuntu 10.04.2, now the main thing which is left is configuring the server and creating certificates for server and clients.

Step 3 : (Creating Certificates for Server)

We need to copy easy-rsa tools so that we can create certificates.

root@crystalnetworks:~# cp -fr /usr/share/doc/openvpn/examples/easy-rsa/ /etc/openvpn/

Above command will copy easy-rsa tools to /etc/openvpn directory. Now we will create certificates.

root@crystalnetworks:~# cd /etc/openvpn/easy-rsa/2.0/

Edit "vars" file and set some basic parameters which are present at the bottom of the file. Below given are the parameters which you should change in this file.

export KEY_COUNTRY="PK"
export KEY_PROVINCE="SN"
export KEY_CITY="Karachi"
export KEY_ORG="Crystal Networks"
export KEY_EMAIL=" support@crystalnetworks.org"

Set these parameters to your liking and then save and exit the file. (Output is not shown of below commands)

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0# source ./vars

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0#./clean-all

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0# ./build-dh

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0# ./build-ca

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0# ./build-key-server Server

This completes the generation of Certificates for our Server.

We can use the Server Certificate from its current location, however I normally copy these Certificates to /etc/openvpn directory. All generated certificates are present in "/etc/openvpn/easy-rsa/2.0/keys/". Now I will copy the Server related Certificates to /etc/openvpn/

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0# cp keys/{ca.crt,Server.crt,Server.key,dh1024.pem} /etc/openvpn/

Step 4: (Configuration file for OpenVPN Server)

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0# cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz /etc/openvpn/

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0# cd /etc/openvpn

root@crystalnetworks:/etc/openvpn/easy-rsa/2.0# gzip -d server.conf.gz

Now we have a raw server configuration file with default settings. We need to change settings as per our requirement. The settings that we will change in the config file as as follows.

Set "proto" to "tcp".
Make sure that "dev tun" is enabled and "dev tap" is disabled.
Set "ca", and give the path to "ca.crt" file ( example: "ca ca.crt" , because we copied the ca.crt to /etc/openvpn directory)
Set "cert", and give path to "Server.crt"file ( example: "cert Server.crt" , because we copied the Server.crt to /etc/openvpn directory)
Set"key", and give path to "Server.key" file ( example: "key Server.key" , because we copied the Server.key to /etc/openvpn directory)
Set "dh", and give path to "dh1024.pem" file (example: "dh dh1024.pem" , because we copied the dh1024.pem file to /etc/openvpn directory)
Enable "Server 172.16.0.0 255.255.255.0" line. This is the network IP Range that will be used for VPN Connection. This is also mentioned in Basic Assumptions part.
Set "push 192.168.100.0 255.255.255.0" . This dynamically adds routes entry in the OS of client that this range with this subnet should be routed through the VPN. This range is also mentioned in Basic Assumptions part.
Enable "log" and "log-append" lines and do not change any thing.

Now we will need to restart our OpenVPN Server.

root@crystalnetworks:# /etc/init.d/openvpn restart

After restarting OpenVPN Server, to confirm whether the server has properly started or not we can either check log files or just issue "ifconfig" and if you see an interface like tun0 then the service has started successfully.

Step 5: (Setting Firewall Rules)

Now we need to set firewall rules to allow proper communication between remote client and office machines.

IP Range of VPN is 172.16.0.0/24
IP Range of local network is 192.168.100.0/24
eth0 is connected to WAN

root@crystalnetworks:# iptables -t nat -A POSTROUTING -d 192.168.100.0/24 -s 172.16.0.0/24 -j SNAT --to-source 192.168.100.1
root@crystalnetworks:# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

root@crystalnetworks:# echo 1 > /proc/sys/net/ipv4/ip_forward

To make these rules persistent you can simple copy these rules to /etc/rc.local which is knows as the startup script for Linux.

Well that is all from OpenVPN Server side. I will very soon be updating another article related to Client config of OpenVPN and also in near future I will be updating another article related to integration of OpenVPN with OpenLDAP.

Linux HeartBeat IP Fail-over

2011-07-16T04:27:00.000+05:00

-------------------------------------------------------------------------------
TITLE : Heartbeat fail-over of IP addresses
OS LEVEL : Linux (Ubuntu)
AUTHOR : Mudasir Mirza
-------------------------------------------------------------------------------

Few years back I was working as System Administrator and had to implement High-Availability. I googled allot and came across the HeartBeat Daemon which provides this functionality in the most flexible ways that one can think of.

In this post I will guide you all in configuring HeartBeat as an IP Fail-over daemon, which can provide you high-availability between servers running synced data / service.

Pre-requisites:

Basic install of Ubuntu 10.04.2 LTS.
Internet connectivity for installing heartbeat.

Assumptions:

For Host 1 (i.e. named as Dell )

Host 1 = Dell
Interface connected on Host 1 = eth0 (192.168.0.5/24)
Hostname = dell.intranet.local
cat /etc/hosts

#/etc/hosts should have below entries
192.168.0.4 inbox
192.168.0.5 dell

For Host 2 (i.e. named as INBOX )

Host 2 = INBOX
Interface connected on Host 2 = eth0 (192.168.0.4/24)
Hostname = inboxess.intranet.local
cat /etc/hosts

#/etc/hosts should have below entries
192.168.0.4 inbox
192.168.0.5 dell

Service IP: Service IP is the IP that is used to provide the service provided by the server to users. This is basically the IP that will automatically shift from Host 1 to Host 2 on Fail-over.

Server IP = 192.168.0.9/24

NOTE: Where ever you see "DELL#>" this means shell prompt on Host 1 (i.e. named as Dell) and whenever you see "INBOX#>" this means shell prompt on Host 2 (i.e. named as INBOX)

I will configure both servers simultaneously, because configuration on both servers are very identical, except for some minor changed.

Now Lets start installing and configuring HearBeat.

---------------
Step 1: Updating the aptitude package repository
---------------
DELL#> apt-get update
INBOX#> apt-get update

---------------
Step 2: Installing Heartbeat Daemon
---------------
DELL#> apt-get install heartbeat
INBOX#> apt-get install heartbeat

---------------
Step 3: Configure Heartbeat.
---------------
# This file ha.cf contains the configuration of your heartbeat daemon. Here we define which interface to listen, and which method to use. The path of log file and the fail-over time.

DELL#> vim /etc/ha.d/ha.cf

debugfile /var/log/ha-debug # error and debug logfile
logfile /var/log/ha-log # general log file for daemon
logfacility local0 # method for logging, this is from syslog
keepalive 2 # sets the interval between heartbeat packets, 2sec
deadtime 10 # how quickly Heartbeat should decide the node is dead, 10sec
warntime 5 # how quickly Heartbeat should issue a "late heartbeat" warning.
initdead 60 # set the time that it takes to declare a cluster node dead
ucast eth0 192.168.0.4 # communicate over a UDP unicast communications link
auto_failback on # will a resource will automatically fail back to its "primary" node
node dell # primary node ( host 1 named as Dell)
node inbox # secondary node ( host 2 named as inbox)

INBOX#> vim /etc/ha.d/ha.cf

debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 10
warntime 5
initdead 60
ucast eth0 192.168.0.5
auto_failback on
node dell
node inbox

---------------
Step 4: Configure Auth keys for secure authentication of nodes with each other
---------------

The file /etc/ha.d/authkeys contains the keys that are used by server to authenticate each other in starting the heartbeat server between servers.

NOTE: Heartbeat will not start if permissions of this file is not set to "0600".

DELL#> vim /etc/ha.d/authkeys

auth 1
1 sha1 ess1234

DELL#> chmod 0600 /etc/ha.d/authkeys

INBOX#> vim /etc/ha.d/authkeys

auth 1
1 sha1 ess1234

INBOX#> chmod 0600 /etc/ha.d/authkeys

---------------
Step 5: Configure the heartbeat rule.
---------------

The file /etc/ha.d/harecources contains all the rules related to heartbeat. Which server to monitor or which IP to fail-over.

DELL#> vim /etc/ha.d/haresources

dell 192.168.0.9/24/eth0/192.168.0.255 ARP

When I was implementing this, I came across a small problem. When the IP used to shift from primary node to secondary, the communication between client and server used to get disconnected.

After doing some research I found that the issue was related to the ARP entry.

To resolve the issue, I created a small custom script that will run each time the IP is shifted to any node, and will broadcast the new IP and MAC address of the new node to the whole network using "arping"

Directory "/etc/ha.d/resource.d/" contains all the scripts that can be used by heartbeat daemon and yes you can write or customize the given script.

DELL#> vim /etc/ha.d/resource.d/ARP

#!/bin/bash
case "$1" in
start)
arping -f -q -c 5 -w 5 -I eth0 -U 192.168.0.9
;;
stop)
# Stop commands go here
;;
status)
# Status commands go here
;;
esac

DELL#> chmod +x /etc/ha.d/resource.d/ARP

Same will be done on Host 2 named as INBOX.

INBOX#> vim /etc/ha.d/haresources

dell 192.168.0.9/24/eth0/192.168.0.255 ARP

Here also we need the same script as defined in Host 1 (i.e. named as Dell)

INBOX#> vim /etc/ha.d/resource.d/ARP

#!/bin/bash
case "$1" in
start)
arping -f -q -c 5 -w 5 -I eth0 -U 192.168.0.9
;;
stop)
# Stop commands go here
;;
status)
# Status commands go here
;;
esac

INBOX#> chmod +x /etc/ha.d/resource.d/ARP

The heartbeat daemon is now configured. Now we need to restart our servers, and make sure that your primary server boots up before your secondary server, otherwise secondary server will make it self the primary server.

---------------
Step 6: Restarting the servers.
---------------

DELL#> /etc/init.d/heartbeat restart

Wait 1 Min after that restart the service on INBOX

INBOX#> /etc/init.d/heartbeat restart

Please feel free to provide your feedback. More contributions from my side will be coming very soon.