Akhil's Blog

3 Amazing facts about SElinux.

2015-02-25T22:39:00.000+05:30

What is SELinux?

SELinux, Security-Enhanced linux , is an additional method to protect your system.Presuming we want to allow remote anonymous access to a web server,we must open the ports through firewall. However,that means that malicious people can try to crack into the system through a security exploit and, if they compromise the web server process, gain its permissions: the permissions to the apache user and the apache group. That user/group has read access to things like the document root (/var/www/html),as well as write access to /tmp,/var/tmp and any other files/directories that are world writable.

SELinux is a a set of security rules that determine which process can access which files, directories, ports, etc. Every file, process, directory and ports has a special security label called Selinux contexts.

A context is simply a name that is used by the SELinux policy to determine whether or not a process can access a file, directory or post.

SELinux labels have several contexts, but we will discuss only one context with web server: The type context. Type context names usually end with_t.The type context for the web server is httpd_t. The type context for files and directories normally found in /var/www/html is httpd_sys_content_t. The type contexts for files and directories normally found in /tmp and /var/tmp is tmp_t.The t type context for web server ports is http_port_t.

There is a rule in the policy that permits Apache (the web server process running as httpd_t) to access files and directories with a context normally found in /var/www/html and other web server directories (httpd_sys_content_t). There is no allow rule in the policy for files normally found in /tmp and /var/tmp, so access is not permitted. With SELinux enabled a malicious user could not access the /tmp directory, let alone write files to it. SELinux even has rules for remote filesystems such as NFS and CIFS, although all files on these filesystems are labeled with the same context.

SELinux MODES

Enforcing Mode

Premissive Mode

Disabled

Enforcing Mode:In enforcing mode, SELinux actively access to the web server attempting to read files with tmp_t type context. In enforcing mode , SELinux both logs and protects.

Permissive Mode:Permissive mode is often used to troubleshoot issues. In permissive mode, SELinux allows all interactions, even if there is an explicit rule mentioned and it logs all of the denied interactions. This mode can be used to determine if you are having an SELinux issue. NO reboot is required to go from enforcing to permissive or back again.

Disabled:A third mode,disabled, completely disables SELinux. You must reboot to disable SELinux entirely, or to get from disabled mode to enforcing or permissive.

Enable And Disable SELinux Mode

There are two ways of Enable and Disable SElinux on linux:

1. Make the changes in /etc/sysconfig/selinux file.

# This file controls the state for selinux on the system.

# selinux= can take one of these three values:

# enforcing – SELinux security policy is enforced.

# permissive – SELinux prints warnings instead of enforcing.

# disabled – No SELinux policy is loaded.

SELINUX=enforcing

# SELINUXTYPE= can take one of these two values:

# targeted – Targeted processes are protected,

# mls – Multi Level Security protection.

SELINUXTYPE=targeted

use /etc/sysconfig/selinux to change the default SELinux mode at boot time. In the example above, it is set to enforcing mode.

2. Changed the Mode at command line.

To make the SELinux mode change without restart you can use the two commands:-

getenforce

setenforce

To check the current status of SELinux on the system use getenforce

[root@serverx ~]# getenforce

Enforcing

It show's the current SELinux Status is Enforcing.

To change the current Status of SELinux on the system use Setenforce

[root@serverx ~]# setenforce

usage: setenforce [enforcing | permissive | 1 | 0 ]

You can choose either enforcing or permissive mode but you can't disable it with setenforce.

[root@serverx ~]# setenforce 0

[root@serverx ~]# getenforce

Permissive

SElinux Status changed to Premissive

[root@serverx ~]# setenforce 1

[root@serverx ~]# getenforce

Enforcing

SElinux Status changed to Enforcing

[root@serverx ~]# setenforce permissive

[root@serverx ~]# getenforce

permissive

SElinux Status changed to Premissive

[root@serverx ~]# setenforce enforcing

[root@serverx ~]# getenforce

Enforcing

SElinux Status changed to Enforcing

Above Example shows you the different usage of setenforce command to change the modes of SELinux.

How to Install Mongodb on Ubuntu

2015-01-20T19:36:00.001+05:30

Install Mongodb On Ubuntu

In this Article we will learn how to install mongodb on Ubuntu.If you want to install mongodb on other systems i.e fedora,redhat & centos please refer my last article.So lets start with install mongodb Ubuntu systems.

What is Mongodb?

MongoDB is the leading NoSQL database, designed for how we build and run applications today.MongoDB empowers organizations to be agile and scalable. It helps them enable new types of applications, improve customer experience, accelerate time to market and reduce total cost of ownership (TCO). MongoDB is named from "huMONGOus" meaning "extremely large".

MongoDB is a general purpose, open-source database. MongoDB features:

Document data model with dynamic schemas
Full, flexible index support and rich queries
Auto-Sharding for horizontal scalability
Built-in replication for high availability
Text search
Advanced security
Aggregation Framework and MapReduce
Large media storage with GridFS

After looking what is mongodb.let's gets started with installation of mongodb on Ubuntu..

Install Mongodb Ubuntu System

Ubuntu has its own Mongodb Packages but we will be using the offical Mongodb package available on the mongodb website as they are more up-to-date.

First Import the public key used by the package management system i.e. dpkg and apt which can be done with the following command.

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10

Create the repo file or list file for the mongodb using the following command

echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list

Now it's time to reload the database of package management which can be done with following command

sudo apt-get update

Now it's Time to Install the Mongodb package which can be done

sudo apt-get install -y mongodb-org

This will install the latest stable release of mongodb on ubuntu if you want to install a specific release of mongodb on ubuntu,Please use the following command.

sudo apt-get install -y mongodb-org=2.6.1 mongodb-org-server=2.6.1 mongodb-org-shell=2.6.1 mongodb-org-mongos=2.6.1 mongodb-org-tools=2.6.1

The above command will install the mongodb version 2.6.1,you can replace the version number with your requirement.

If you have successfully executed all the above steps.You will be able to install mongodb on ubuntu.

MysqlDump - A program to take Mysql Backup

2015-01-15T20:18:00.000+05:30

In This article we will learn What is Mysqldump? how it can be used with multiple options like add-drop-tables,d,e,w,x,f,no-tablespaces.Lets start with what is Mysqldump.

What is Mysqldump?

Mysqldump client is a utility that performs backup .It can be used to backup a database or a collection of databases for backup or transfer to another MySQL server. The dump typically contains SQL statements to create the table, populate it, or both. However, Mysqldump can also be used to generate files in CSV, other delimited text, or XML format.

Mysqldump requires at least the SELECT privilege for dumped tables, SHOW VIEW for dumped views, TRIGGER for dumped triggers, and LOCK TABLES if the --single-transaction option is not used.

You can use Mysqldump in three ways:-

db_name [tbl_name ...]
--databases db_name ...
--all-databases

let's see how to use all the three ways of Mysqldump:-

db_name [tbl_name ...] : This option should be used,if you are taking backup of a single database with multiple tables.

shell> mysqldump [options] db_name [tbl_name ...]

--databases db_name ... : This option should be used,If we have multiple databases to backup.

shell> mysqldump [options] --databases db_name ...

--all-databases: As the name suggest,This options should be used if you need to take backup of complete database.

shell> mysqldump [options] --all-databases

Now we will se what lies in the [ options ],Mysqldump supports multiples options which can be used but we will be discussing only few of those options here.The options which are going to discuss are given below:-

--add-drop-table

-d

-e

-w

-x

-f

--no-tablespaces

--add-drop-table: Add a DROP TABLE statement before each CREATE TABLE statement.So when every you take a backup of database or a table,It will always drop the existing table before creating the new table.It will help you in case having the same schema design.

-d or --no-data: if this option is used,No content from the table will be dumped.This option will only dump the structure of the table or tables

-e or --extended-insert: Use multiple-row INSERT syntax that include several VALUES lists. This results in a smaller dump file and speeds up inserts when the file is reloaded.

-w or --where: Dump only selected records. Quotes are mandatory.

-x or --xml: Dump the database/databases in the well formed XML.

-f or --force: it will continue to take the dump of the table even if there is any error.

--no-tablespaces: If the option no-tablespaces is used, it will take not take the dump of table structure,only the content will be backup from the tables.

Amazon S3 - Simple Storage Service

2015-01-02T23:00:00.004+05:30

What is Amazon S3?

Amazon S3 is a repository for Internet data. Amazon S3 provides access to reliable, fast, and inexpensive data storage. It is designed to make web-scale computing easy by enabling you to store and retrieve any amount of data, at any time, from within Amazon EC2 or anywhere on the web.

Amazon S3 stores data objects redundantly on multiple devices across multiple facilities and allows concurrent read or write access to these data objects by many separate clients or application threads. You can use the redundant data stored in Amazon S3 to recover quickly and reliably from instance or application failures.

How to Start With Amazon S3 - Simple Storage Service

Follow the simple six steps to successfully Use a Amazon S3 :-

Sign-up for Amazon S3
Create Bucket
Add An Object to the Bucket
View Object
Move Object
Delete Object and Bucket

Sign-up for Amazon S3

Go to http://aws.amazon.com/s3 and click Sign Up
Follow the on-screen instructions.

Create Bucket for Amazon S3

Sign into the AWS Management Console at https://console.aws.amazon.com/s3.
Click Create Bucket and dialog box will pop up.
In the Create a Bucket dialog box, enter a bucket name in the bucket Name box .After you create a bucket, you cannot change its name. The bucket name is also visible in the URL that points to the objects stored in the bucket. Ensure that the bucket name you choose is appropriate.
In the Region box, select a region.You can choose a region to optimize latency, minimize costs, or address regulatory requirements.
Click Create.

When Amazon S3 successfully creates your bucket, the console displays your empty bucket in the Buckets panel.

Add An Object To The Bucket for Amazon S3

click the name of bucket that you want to upload an object to and then click Upload.
In the Upload - Select Files wizard, if you want to upload an entire folder, you must click Enable Enhanced Uploader to install the necessary Java applet. You only need to do this once per console session.
Click Add Files.
- If you enabled the advanced uploader in step 2, you see a Java dialog box titled Select files and folders to upload, as shown.
- If not, you see the File Upload dialog box associated with your operating system.
Now Select the file that you want to upload and then click Open.
Click Start Upload

View Object from Amazon S3

In the Objects and Folders list, right-click the object or objects that you want to open or download, then click Open or Download as appropriate.
If you are downloading the object, specify where you want to save it. The procedure for saving the object depends on the browser and operating system that you are using.

So far we have seen how to create an Amazon S3 bucket,Add an object in the bucket and how to view the same object from the bucket.In the next article we will see how to move object in bucket and how to delete an object and bucket from Amazon S3.Thanks!!

Setup Mysql Replication in Master Slave Mode

2014-12-24T16:09:00.004+05:30

In this tutorial we will learn what is Mysql Replication,How to setup mysql replication.

Let's start with what is MySQL replication is.

MySQL replication

MySQL replication is a process that allows you to easily maintain multiple copies of a MySQL data by having them copied automatically from a master to a slave database. This can helpful for many reasons :

Scale-out solutions
Data Security
Analytics
data distribution
Backup

we will Setup a very simple Mysql Replication with One Master and One slave.let's take two machines for achieving the same. lets take two machines for setting up the Mysql Replication

Machine One:10.33.76.78- Master Database

Machine Two:10.22.43.46- Slave Database

Setup

In this article i am using ubuntu to setup the replication,So first thing which needs to be done for replication is installation of mysql on the system

apt-get install mysql-server mysql-client

Once you are done with the installation part,lets start the real fun of replication.

Step One—Setup the Master Database

On Master database we need to make couple changes in the configuration file on Mysql.

The first step is to find the section that looks like this, binding the server to the local host:

bind-address = 127.0.0.1

Replace the standard IP address with the IP address of server.

bind-address = 0.0.0.0

Next we need to un-comment the line which has server-id:

server-id = 1

Now,Search for the line that refers to log_bin and uncomment the same,I will make the bin logs on default location but you can change the location of bin logs as well.

log_bin = /var/log/mysql/mysql-bin.log

Next option is used if you want to have Mysql Replication on the basis of databases i.e if you have multiple databases and you want Replication for only selective database

binlog_do_db = newdatabase

We are done with all the configuration changes on Master and now it's time to restart the mysql.

service mysql restart

We need to grant privileges to the slave. The command should be in this format:

GRANT REPLICATION SLAVE ON *.* TO 'slave_user'@'%' IDENTIFIED BY 'password';

'slave_user' : Denotes the name of user used for replication,It can be anything.

'%' : Denotes that with above user and password,Any host can become slave of this master,I would recommend that instead of % you should use the ip-address of the slave host.

'password': it should be combination of character numeric.

Last thing to do :

FLUSH PRIVILEGES;

Now,We need to note down the position of master and take the backup of databases for which we want to setup the Mysql Replication,Which can be done with following sequence of commands:-

USE newdatabase;

FLUSH TABLES WITH READ LOCK;

mysql> SHOW MASTER STATUS;
+------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 | 107 | | |
+------------------+----------+--------------+------------------+
1 row in set (0.00 sec)

The position noted above will be helpful while setting up the Slave.

export your database using mysqldump in the new window (make sure you are typing this command in the bash shell, not in MySQL).

mysqldump -u root -p --opt newdatabase > newdatabase.sql

Now, returning to your your original window, unlock the databases (making them writeable again). Finish up by exiting the shell.

UNLOCK TABLES;

QUIT;

Step Two—Configure the Slave Database

On Slave Machine,open up the MySQL shell and create the new database that you will be replicating from the master (then exit):

CREATE DATABASE newdatabase;

EXIT;

After creating the database, Now Import that you previously exported from the master database.

mysql -u root -p newdatabase < /path/to/newdatabase.sql

Now we need to configure the slave configuration :

make sure that your have the following Four criteria appropriately filled out:

server-id = 2

relay-log = /var/log/mysql/mysql-relay-bin.log

log_bin = /var/log/mysql/mysql-bin.log

binlog_do_db = newdatabases

Server-id needs to be unique as and we have already used server-id = 1 on master,lets setup server-id=2 here on slave.

You might not find the rest of three values in my.cnf by default,So if the values are missing please add it and restart the mysql.

service mysql restart

Let's enable the replication from within the MySQL shell.Open up the the MySQL shell and type in the details :

CHANGE MASTER TO MASTER_HOST='ip-address of master',MASTER_USER='slave_user', MASTER_PASSWORD='slave_password', MASTER_LOG_FILE='', MASTER_LOG_POS= ;

This command accomplishes several things at the same time:

It designates the current server as the slave of our master server.

It provides the server the correct login credentials

Last of all, it lets the slave server know where to start replicating from; the master log file and log position come from the numbers we wrote down previously.

With that—you have configured a master and slave server.

Activate the slave server:

START SLAVE;

You be able to see the details of the slave replication by typing in this command. The \G rearranges the text to make it more readable.

SHOW SLAVE STATUS\G

Show slave status will give you complete details which you have entered while setting up the slave.like master server,user name and etc.You can test t

he replication by simplying createing a table in the database for which you have enabled the replication and see if the same table exists on slave.If

you table is present on the slave machine as well then your Mysql Replication is working fine,Else investagte further to check what has went worng.

Install MongoDB on Fedora, Red Hat Enterprise and CentOS

2014-12-13T11:16:00.000+05:30

In this Tutorial we will learn how to Install MongoDB on different Flavors on Linux i.e fedora, Redhat Enterprise and CentOS.

Considerations

For production deployments, always run MongoDB on 64-bit systems.

The default /etc/mongodb.conf configuration file supplied by the 2.6 series packages has bind_ip`set to 127.0.0.1 by default. Modify this setting as needed for your environment before initializing a replica set.

Changed in version 2.6: The package structure and names have changed as of version 2.6. For instructions on installation of an older release, please refer to the documentation for the appropriate version.

Install Mongodb

Lets begin with the installation of Mongodb with YUM.

Step 1: Add the Mongodb repo in package management system (YUM) in fedora.

Create a /etc/yum.repos.d/mongodb.repo file which should have the following configuration information for the MongoDB repository:

[mongodb]

name=MongoDB Repository

baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/

gpgcheck=0

enabled=1

The Above configuration will work with 64-bit system,if You are running a 32-bit configuration please use the given below Confiugration:

[mongodb]

name=MongoDB Repository

baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/i686/

gpgcheck=0

enabled=1

Step 2: Once you have successfully done the changes in the package management system (YUM),It's time to install the Mongodb on the system i.e fedora, Red Hat Enterprise and CentOS :

To install the latest stable version of MongoDB, simply issue the following command:

sudo yum install -y mongodb-org

if you want to use a specific version of mongodb,You have to specify each component package individually and append the version number to the package name,An example is shown below:

sudo yum install -y mongodb-org-2.6.1 mongodb-org-server-2.6.1 mongodb-org-shell-2.6.1 mongodb-org-mongos-2.6.1 mongodb-org-tools-2.6.1

If you have completed the above two steps successfully,You have installed mongodb in fedora or Redhat Enterprise .Thanks!!

DRBD and Heartbeat

2014-12-09T14:23:00.000+05:30

DRBD And HeartBeat

About three months ago, I was planning a new server setup that would run our new portal as well as e-mail, databases, DNS and so forth. One of the most important goals was to create a redundant solution, so that if one of the servers failed, it wouldn't affect company operation.
I looked through a lot of the redundant solutions available for Linux at the time, and with most of them, I had trouble getting all the services we needed to run redundantly. After all, there is a very big difference in functionality between a Sendmail dæmon and a PostgreSQL dæmon.
In the end, though, I did find one solution that worked very well for our needs. It involves setting up a disk mirror between machines using the software DRBD and a high-availability monitor on those machines using Heartbeat.
DRBD mirrors a partition between two machines allowing only one of them to mount it at a time. Heartbeat then monitors the machines, and if it detects that one of the machines has died, it takes control by mounting the mirrored disk and starting all the services the other machine is running.
In this tutorial, I show you how to set up a redundant Sendmail system, because once you do that, you will be able to set up almost any service you need. We assume that your master server is called server1 and has an IP address of 192.168.1.1, and your slave server is called server2 and has an IP address of 192.168.1.2.
And, because you don't want to have to access your mail server on any of these addresses in case they are down, we will give it a virtual address of 192.168.1.5. You can, of course, change this to whatever address you want in the Heartbeat configuration.

How It Works

-------------------------

This high-availability solution works by replicating a disk partition in a master/slave mode. The server that is running as a master has full read/write access to that partition; whereas the server running as slave has absolutely no access to the partition but silently replicates all changes made by the master server.
Because of this, all the processes that need to access the replicated partition must be running on the master server. If the master server fails, the Heartbeat dæmon running on the slave server will tell DRBD that it is now the master, mount the replicated partition, and then start all the processes that have data stored on the replicated partition.

How to Get It Running

-------------------------------------
The first step for running a redundant system is having two machines ready to try it out. They don't need to have identical specs, but they should meet the following requirements:

Enough free space on both machines to create an equal-sized partition on each of them.
The same versions of the dæmons you want to run across both machines.
A network card with crossover cable or a hub/switch.
An optional serial port and serial port crossover cable for additional monitoring.

You also should think carefully about which services you want running on both machines, as this will affect the amount of hard disk you will need to dedicate to replication across them and how you will store the configuration and data files of these services.
It's very important that you have enough space on this shared partition, because it will be the main data storage location for all of these services. So, if you are going to be storing a large Sendmail spool or a database, you should make sure it has more than enough space to run for a long time before having to repartition and reconfigure DRBD for a larger disk size.

Setting Up the Basics on Your Servers

----------------------------------------------------------------
Once you've made sure your machines are ready, you can go ahead and create an equal-sized partition on both machines. At this stage, you do not need to create a filesystem on that partition, because you will do that only once it is running mirrored over DRBD.
For my servers, I have one DRBD replicated drive that looks like this on my partition tables:

/dev/sda5      7916    8853   7534453+  83  Linux

Note: type fdisk -l at your command prompt to view a listing of your partitions in a format similar to that shown here. Also, in my case, the partition table is identical on both redundant machines.
The next step after partitioning is getting the packages for Heartbeat version 1.2+ and DRBD version 0.8+ installed and the DRBD kernel module compiled. If you can get these prepackaged for your distribution, it will probably be easier, but if not, you can download them from www.linux-ha.org/DownloadSoftware and www.drbd.org/download.html.
Now, go to your /etc/hosts file and add a couple lines, one for your primary and another for your secondary redundant server. Call one server1, the other server2, and finally, call one mail, and set the IP addresses appropriately. It should look something like this:

192.168.1.1    server1
192.168.1.2    server2
192.168.1.5    mail

Finally, on both your master and slave server, make a folder called /replicated, and add the following line to the /etc/fstab file:

/dev/drbd0    /replicated   ext3   noauto    0   0

Configuring DRBD

--------------------------------

After you've done that, you have to set up DRBD before moving forward with Heartbeat. In my setup, the configuration file is /etc/drbd.conf, but that can change depending on distribution and compile time options, so try to find the file and open it now so you can follow along. If you can't find it, simply create one called /etc/drbd.conf.
Listing 1 is my configuration file. I go over it line by line and add explanations as comments that begin with the # character.

Listing 1. /etc/drbd.conf

# Each resource is a configuration section for a
# mirrored disk.
# The drbd0 is the name we will use to refer
# to this disk when starting or stopping it.

resource drbd0 {
  protocol C;
  handlers {
    pri-on-incon-degr "echo 'DRBD: primary requested but inconsistent!'
    ↪| wall; /etc/init.d/heartbeat stop"; #"halt -f";
    pri-lost-after-sb "echo 'DRBD: primary requested but lost!'
    ↪| wall; /etc/init.d/heartbeat stop"; #"halt -f";
  }

  startup {
    degr-wfc-timeout 120;    # 2 minutes.
  }

  disk {
    on-io-error   detach;
  }
# These are the network settings that worked best for me.
# If you want to play around with them, go
# ahead, but take a look in the man pages of drbd.conf
# and drbdadm to see what each does.

  net {
    timeout 120;
    connect-int 20;
    ping-int 20;
    max-buffers     2048;
    max-epoch-size  2048;
    ko-count 30;

# Remember to change this shared-secret on both the master
# and slave machines.

    cram-hmac-alg "sha1";
    shared-secret "FooFunFactory";
  }

  syncer {
    rate 10M;
    al-extents 257;
  }

# This next block defines the settings for the server
# labeled as server1.  This label should be in your
# /etc/hosts file and point to a valid host.

  on server1 {

# The following device will be created automatically by
# the drbd kernel module when the DRBD
# partition is in master mode and ready to write.
# If you have more than one DRBD resource, name
# this device drbd1, drbd2 and so forth.

    device     /dev/drbd0

# Put the partition device name you've prepared here.

    disk      /dev/sda5;

# Now put the IP address of the primary server here.
# Note: you will need to use a unique port number for
# each resource.

    address   192.168.1.3:7788;
    meta-disk  internal;
  }

# This next block is identical to that of server1 but with
# the appropriate settings of the server called
# server2 in our /etc/hosts file.

  on server2 {
    device    /dev/drbd0;
    disk      /dev/sda5;
    address   192.168.1.2:7788;
    meta-disk internal;
  }
}

Now, let's test it by starting the DRBD driver to see if everything works as it should. On your command line on both servers type:

drbdadm create-md drbd0; /etc/init.d/drbd restart; cat /proc/drbd

If all goes well, the output of the last command should look something like this:

0: cs:Connected st:Secondary/Secondary ds:Inconsistent/Inconsistent r---
   ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
       resync: used:0/7 hits:0 misses:0 starving:0 dirty:0 changed:0
       act_log: used:0/257 hits:0 misses:0 starving:0 dirty:0 changed:0

Note: you always can find information about the DRBD status by typing:

cat /proc/drbd

Now, type the following command on the master system:

drbdadm -- --overwrite-data-of-peer primary drbd0; cat /proc/drbd

The output should look something like this:

0: cs:SyncSource st:Primary/Secondary ds:UpToDate/Inconsistent r---
   ns:65216 nr:0 dw:0 dr:65408 al:0 bm:3 lo:0 pe:7 ua:6 ap:0
       [>...................] sync'ed:  2.3% (3083548/3148572)K
       finish: 0:04:43 speed: 10,836 (10,836) K/sec
       resync: used:1/7 hits:4072 misses:4 starving:0 dirty:0 changed:4
       act_log: used:0/257 hits:0 misses:0 starving:0 dirty:0 changed:0

This means it is syncing your disks from the master computer that is set as the primary one to the slave computer that is set as secondary.
Next, create the filesystem by typing the following on the master system:

mkfs.ext3 /dev/drbd0

Once that is done, on the master computer, go ahead and mount the drive /dev/drbd0 on the /replicated directory we created for it. We'll have to mount it manually for now until we set up Heartbeat.

Preparing Your Services

------------------------------------------
An important part of any redundant solution is properly preparing your services so that when the master machine fails, the slave machine can take over and run those services seamlessly. To do that, you have to move not only the data to the replicated DRBD disk, but also move the configuration files.
Let me show you how I've got Sendmail set up to handle the mail and store it on the replicated drives. I use Sendmail for this example as it is one step more complicated than the other services, because even if the machine is running in slave mode, it may need to send e-mail notifications from internal applications, and if Sendmail can't access the configuration files, it won't be able to do this.
On the master machine, first make sure Sendmail is installed but stopped. Then create an etc directory on your /replicated drive. After that, copy your /etc/mail directory into the /replicated/etc and create a symlink from /replicated/etc/mail to /etc/mail.
Next, make a var directory on the /replicated drive, and copy /var/mail, /var/spool/mqueue and any other mail data folders into that directory. Then, of course, create the appropriate symlinks so that the new folders are accessible from their previous locations.
Your /replicated directory structure should now look something like:

/replicated/etc/mail
/replicated/var/mail
/replicated/var/spool/mqueue
/replicated/var/spool/mqueue-client
/replicated/var/spool/mail

And, on your main drive, those folders should be symlinks and look something like:

/etc/mail -> /replicated/etc/mail
/var/mail -> /replicated/var/mail
/var/spool/mqueue -> /replicated/var/spool/mqueue
/var/spool/mqueue-client -> /replicated/var/spool/mqueue-client
/var/spool/mail -> /replicated/var/spool/mail

Now, start Sendmail again and give it a try. If all is working well, you've successfully finished the first part of the setup.
The next part is to make sure it runs, even on the slave. The trick we use is copying the Sendmail binary onto the mounted /replicated drive and putting a symlink to the binary ssmtp on the unmounted /replicated folder.
First, make sure you have ssmtp installed and configured on your system. Next, make a directory /replicated/usr/sbin, and copy /usr/sbin/sendmail to that directory. Then, symlink from /usr/sbin/sendmail back to /replicated/usr/sbin/sendmail.
Once that's done, shut down Sendmail and unmount the /replicated drive. Then, on both the master and slave computers, create a folder /replicated/usr/sbin and a symlink from /usr/sbin/ssmtp to /replicated/usr/sbin/sendmail.
After setting up Sendmail, setting up other services like Apache and PostgreSQL will seem like a breeze. Just remember to put all their data and configuration files on the /replicated drive and to create the appropriate symlinks.

Heartbeat is designed to monitor your servers, and if your master server fails, it will start up all the services on the slave server, turning it into the master. To configure it, we need to specify which servers it should monitor and which services it should start when one fails.
Let's configure the services first. We'll take a look at the Sendmail we configured previously, because the other services are configured the same way. First, go to the directory /etc/heartbeat/resource.d. This directory holds all the startup scripts for the services Heartbeat will start up.
Now add a symlink from /etc/init.d/sendmail to /etc/heartbeat/resource.d.
Note: keep in mind that these paths may vary depending on your Linux distribution.
With that done, set up Heartbeat to start up services automatically on the master computer, and turn the slave to the master if it fails. Listing 2 shows the file that does that, and in it, you can see we have only one line, which has different resources to be started on the given server, separated by spaces.

Listing 2. /etc/heartbeat/haresources

server1 IPaddr::192.168.1.5/24 datadisk::drbd0 sendmail

The first command, server1, defines which server should be the default master of these services; the second one, IPaddr::192.168.1.5/24, tells Heartbeat to configure this as an additional IP address on the master server with the given netmask. Next, with datadisk::drbd0 we tell Heartbeat to mount this drive automatically on the master, and after this, we can enter the names of all the services we want to start up—in this case, we put sendmail.
Note: these names should be the same as the filename for their startup script in /etc/heartbeat/resource.d.
Next, let's configure the /etc/heartbeat/ha.cf file (Listing 3). The main things you would want to change in it are the hostnames of the master/slave machine at the bottom, and the deadtime and initdead. These specify how many seconds of silence should be allowed from the other machine before assuming it's dead and taking over.
If you set this too low, you might have false positives, and unless you've got a system called STONITH in place, which will kill the other machine if it thinks it's already dead, you can have all kinds of problems. I set mine at two minutes; it's what has worked best for me, but feel free to experiment.
Also keep in mind the following two points: for the serial connection to work, you need to plug in a crossover serial cable between the machines, and if you don't use a crossover network cable between the machines but instead go through a hub where you have other Heartbeat nodes, you have to change the udpport for each master/slave node set, or your log file will get filled with warning messages.

Listing 3. /etc/heartbeat/ha.cf

debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility     local0
keepalive 2
deadtime 120
initdead 120
serial  /dev/ttyS1
baud 9600
udpport 694
udp     eth0
nice_failback on
node server1
node server2

Now, all that's left to do is start your Heartbeat on both the master and slave server by typing:

/etc/init.d/heartbeat start

Once you've got that up and running, it's time to test it. You can do that by stopping Heartbeat on the master server and watching to see whether the slave server becomes the master. Then, of course, you might want to try it by completely powering down the master server or any other disconnection tests.

Congratulations on setting up your redundant server system! And, remember, Heartbeat and DRBD are fairly flexible, and you can put together some complex solutions, including having one server being a master of one DRBD partition and a slave of another. Take some time, play around with them and see what you can discover.

-->

Centralized Syslog Server

2014-12-09T14:21:00.000+05:30

A centralized syslog server was one of the first true SysAdmin tasks that I was given as a Linux Administrator way back. My boss at the time wanted to pull in log files from various appliances and have me use regexp to search them for certain key words. At the time Linux was still in its infancy, and I had just been dabbling with it in my free time. So, I jumped at the chance to introduce Linux to the company that I had worked for. Did it work? You bet it did! What this post is going to cover is not only how to setup a centralized syslog-ng server, but why you would go about setting one up in the first place.
So what is syslog? Syslog is used in Linux to log system messages (huh, another easy to guess name). Syslog-ng is just a rewrite of the original syslog, that was developed in 1998. Syslog-ng is still being actively developed (as of 2010) by BalaBit IT Security and comes with many more features, including better TCP handling, TLS encryption of messages, and sending messages to a database among other things. Some distributions allow you to install either syslog, rsyslog or syslog-ng. For this article, I'll be focusing on syslog-ng as this is more up to date, and if the reader wishes, can be 'supported' via the company that owns the syslog-ng software by going with their enterprise edition version at a later date.
Now that you've got an overview of syslog-ng, let's talk about just why you would use a centralized syslog-ng server. I am sure there are more than the two reasons that I will bring up, but I can think of at least 2 of them off the top of my head. The first is for security purposes. If you have your routers, firewalls, switches, Linux servers and/or other hardware pointing to a SECURED centralized syslog-ng server, when someone does attempt to attack one of the above devices log files can be safely off-site in a secure location. If syslog files are kept on the device this gives an attacker the ability to clean up their tracks. Granted, they can disable the ability to send log files to an external syslog-ng server, but any and all connections prior to that will be located on the centralized syslog server. The other reason is for convenience. For instance, if you have a server that crashed and is unresponsive, you can check the kernel error logs on your centralized syslog server. If you want to check syslog patterns between various dates over an extended time, regex the log files from the centralized syslog server.
So what do I do? I actually use both approaches at home. Not only do my devices and servers forward all their syslog files to a centralized location, but that location is locked down. The machine in question is a virtual machine with only 1 port open (syslog) and accessible only from the local machine, the syslog files are kept on an external drive. Is it paranoia? Probably a wee bit. But I do know that in my home environment, if my external drive fills up from too many syslog files it won't crash my virtual machine. If somehow something happens to my virtual machine, my host OS won't be affected, if someone does gain access to one of my devices then they can't gain access to my syslog server. Granted if something happened to my host OS then I would have issues with my guest VM's, but we can't always prepare for everything. Okay, I admit it's paranoia in the highest of levels, and for most people this is probably too far.
Before we get started, here's a quick disclaimer. First off, as with all of my previous posts, I do all of my blogpost testing in Debian. In this case I had a virtual machine setup for Debian 6.0.1, thus your mileage may vary. Also I won't be getting into how to properly secure your server, best practices on where to place syslog files, or how to setup anything other than syslog-ng. I leave that up to the reader. This blog post just covers the basics of a centralized syslog-ng server.

Installing and Configuring - Server Side

Installing syslog-ng isn't as hard as it looks especially if you're installing from packages. For Debian: apt-get install syslog-ng, for Redhat: yum install syslog-ng. For those of you that enjoy a good source install: http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.2.4/source/syslog-ng_3.2.4.tar.gz Download, unpackage, configure, make & make install. Once you have syslog-ng installed, we can get to configuring the server side.

Global Options

First thing you need to do is locate your syslog-ng configuration file. The default install (for Debian variants) is '/etc/syslog-ng/syslog-ng.conf'. Before editing any configuration files it is best practice to make a copy of the original configuration file prior to any changes. This is just in case something happens and you need to go back to the original configuration file. I tend to label my original configuration files with .orig (in this case: syslog-ng.conf.orig). Now that you have made a copy of your configuration file, let's open it up with your editor of choice and get started.
long_hostnames(default: off ) - For this post I'm using syslong-ng OSE version 3.1, and I actually can't find long_hostnames in the global configuration guide online. I'll go with long hostnames as a default of off, being fully qualified domain names.
flush_lines(default: 0 ) - Sets the number of lines flushed to a destination at a time. Setting to 0 sends messages as they are received, but keep in mind setting this number higher may increase message latency. This is useful on the client side of syslog-ng. You would keep xx messages on the client before flushing to the destination so that you are not flooding the main syslog-ng server if you have alot of traffic coming from a server.
use_dns(default: no ) - Options: yes, no, persist_only. This one is up to you and your environment. If your syslog-ng is behind a firewall and not accessible to the outside of the world then 'yes' would be appropriate. If accessible to the outside of the world, set to 'no' in order to stop possible DoS attacks. I set mine to 'persist_only' which checks my /etc/hosts file on my syslog-ng server to resolve hostnames, without relying on dns servers.
use_fqdn(default: no ) - Set the Fully Qualified Domain Name, your choice. As a home network I only have one internal domain name. So mine defaults to 'no'. Setting to 'yes' would have your clients hostname show up as: 'hostA.domain.com' instead of 'hostA'
owner(default: root ) - Owner of output files
group(default: adm ) - Group of output files
perm(default: 0640 ) - Permission of output files. Defaults to 640 - Owner Read-Write, Group Read, Other none.
stats_freq(default: 0 ) - Time (in seconds) between two STATS (statistics messages about dropped log messages) Messages. 0 disables STATS messaging.
bad_hostname(default: ^gconfd$ ) - Regex containing hostnames that should not be handled as hostnames..in this case gconfd. If you have more than a handful of servers than I woudl recommend hostnames, unless of course you remember every ip address in your domain..if you, I applaud you.
Now that's it for the 'Default' Global configuration options, but there are many more that you can use. I also use the following:
normalize_hostnames(yes) - This converts all hostnames to lowercase. Some of my devices have uppercase hostnames, and sometimes I get carried away with a new host and Uppercase the first letter of the hostname. This will just lowercase all characters for easier readability.
keep_hostname(yes) - This keeps the hostname if running through a relay or an external server, so that when the host finally reaches the central server the hostname comes with it instead of relying on DNS (or /etc/hosts). If you're using $HOST macro, this should be enabled.
In a bigger and more important environment (read: not soho) I would be setting stats_freq(600) and stats_level(2) in order to retrieve statistics messages from the server. In most soho environments you might be gathering syslog data from 3-5 devices, at which point the odds of actually losing data are pretty slim. In a larger enterprise environment of several hundred devices reporting to centeral syslog servers, enabling statistics allows the sys admin the ability to check on stats and possibly lost messages.
Your global configuration options (if you want it to mirror mine) would look like the following:


options {(off); 
flush_lines(0); 
use_dns(persist_only);
use_fqdn(no); 
owner("root"); 
group("adm"); 
perm(0640); 
stats_freq(0);
bad_hostname("^gconfd$"); 
normalize_hostnames(yes);
keep_hostname(yes); 
};

Setting up Listener

Setting up the listener for syslog-ng is actually only a few lines in the configuration file. A typical listener line looks like this:

source s_net { tcp((ip(127.0.0.1) port(1000) max-connections 5000)); udp (); };

source s_net = Network listener
tcp(ip(127.0.0.1) = Listen on localhost. If you have multiple NIC's, or want to specify an ip to bind this to, change 127.0.0.1 to the ip address of that specific network card
port (1000) = Listen to TCP port 1000
max connections = Allow 5000 simultaneous connections (stops the dreaded 'run away server' syndrome)
udp () = Some devices send their syslog messages via udp, so enable udp if you can't specify tcp and port number.
encrypt(allow) = This could be an entire blog post in itself. Syslog-ng allows for encrypted (TLS, certificate based) syslog messages
Mine for example looks like this:


# Listen on TCP Port 1000 and UDP Port 514, Max 500 Connections source s_net {
tcp(port(1000) max-connections(500)); udp(););

Destination - What goes up must come down. In this case what gets sent out must get put somewhere. Once a message is received from the syslog-ng server it's got to go somewhere. Thus the destination section of the syslog-ng.conf file. As you can see, the default covers your *nix destination for server messages on the local machine. But what about incoming messages? Where do they go? Good question, by default they will send their syslog messages to the subsystem specified in syslog-ng. For instance if it's a message that would be classified as an authentication message (/var/log/auth) then it will dump the message into the syslog-ng's /var/log/auth.log file with the appended information (hostname, date/time, etc).
If that's actually what you want to accomplish, a bunch of servers dumping to the same file as your main server, then I guess the task is complete. But syslog-ng can do so much more than that. If I do much more on server side configuration though I fear this will end up being a chapter in a book. Destinations can be flat files, pipes into other applications, SQL Databases (mysql, MS SQL, Oracle, etc), Remote Log servers, and Terminal Windows. I'll be focusing on flat files and assume you are doing the same for now.
Now the way I setup my centralized syslog server might be different then the way you setup yours. In my case I have a folder that has each hostname and the syslogs from the hostname are located in the folder. For Example: /mount/syslog/macha, /mount/syslog/beag, and so on and so forth. Logrotate takes care of zipping, removing (old files are backed up to a remote server just in case) and cleaning up log files.
My Destination directive looks like this:


destination d_net_auth {
file("/var/log/syslog/remote/$HOSTNAME/auth.log"); }; destination d_net_cron {
file("/var/log/syslog/remote/$HOSTNAME/cron.log"); }; destination d_net_daemon
{ file("/var/log/syslog/remote/$HOSTNAME/daemon.log"); }; destination
d_net_kern { file("/var/log/syslog/remote/$HOSTNAME/kern.log"); }; destination
d_net_lpr { file("/var/log/syslog/remote/$HOSTNAME/lpr.log"); }; destination
d_net_mail { file("/var/log/syslog/remote/$HOSTNAME/mail.log"); }; destination
d_net_syslog { file("/var/log/syslog/remote/$HOSTNAME/syslog.log"); };
destination d_net_user { file("/var/log/syslog/remote/$HOSTNAME/user.log"); };
destination d_net_user { file("/var/log/syslog/remote/$HOSTNAME/uucp.log"); };
destination d_net_debug { file("/var/log/syslog/remote/$HOSTNAME/debug"); };
destination d_net_error { file("/var/log/syslog/remote/$HOSTNAME/error"); };
destination d_net_messages { file("/var/log/syslog/remote/$HOSTNAME/messages");
}; destination d_net_mailinfo {
file("/var/log/syslog/remote/$HOSTNAME/mail/mail.info"); }; destination
d_net_mailwarn { file("/var/log/syslog/remote/$HOSTNAME/mail/mail.warn"); };
destination d_net_mailerr {
file("/var/log/syslog/remote/$HOSTNAME/mail/mail.err"); };

Now in theory, the syslog-ng server is supposed to create the directories necessary for the files to drop into (as specified in the global policies) but sometimes I run into problems where the directories were not created properly and the errors in syslog-ng are reported in /var/log/errors. To alleviate future pain and suffering I tend to create the host and log files as I go, anything I'm missing will end up in /var/log/errors and I can create them later.
For those of you that are veteran syslog-ng users, you might wonder why I split my localhost destination and my remote(off-site clients) destinations when in theory I could have created a d_auth and had my regular localhost filter into a folder as well. The reason behind that was that I wanted to separate my localhost syslog traffic from remote traffic - more configuration lines, but easier on me. Also, I'm not messing with the Linux subsystem when it's out looking for where to put regular log files.
Filtering - The ability for Syslog-NG to filter its messages is what really seperates the 'men from the boys' in the syslog battle. The filtering is what really sets syslog-ng apart. Granted I separate my hosts in folders defined in $HOST variable, but filtering is the real meat and potatoes. With filtering I can (and do) the following: Filter Firewall logs looking for certain key words such as port scans, that get dumped into 1 folder, DDOS attacks that get filtered into another folder. My voip adaptor sends syslog events and I filter based on those messages into individual files instead of a single file. Filtering also allows you to specify multiple hosts to filter based on, and into multiple destinations. Not only that, but you can use regular expressions in filtering.
Filtering expressions are created like: filter { expression; };
is the name you give your filter. contains the function, and boolean operators (and,or,not).
An example for my firewall would be:


filter firewall_ddos_filter { host("10.1.1.1") and match("Denial of Service"
value("MESSAGE")); };

This filter is called 'firewall_ddos_filter, it listens for incoming syslog messages from 10.1.1.1 with a message of 'Denial of Service'. To complete the filter you need a log statement:
log firewall_ddos_filter { source(s_net); filter(firewall_ddos_filter); destination(d_net_firewall_ddos); };
In my above destination I would add a destination for firewall DDOS Attacks, port scanning, etc. This makes it easier to separate log files from servers/devices that do not use the standard *nix logging facilities, or easier for a system admin to filter logs coming out of a firewall (or many firewalls filtered into one log).
If you want to use multiple 'firewall' hosts (as an example) do NOT use just add them in and create a log/filter rule using a boolean operator of 'and'. It will not work, and you beat your head on the desk for many hours to come. Instead, use the 'or' boolean operator as such:
filter firewall_ddos_filter { host("10.1.1.1") or host ("10.1.1.2") and match("Denial of Service" value("MESSAGE")((; };
My 'Default' Filtering directive looks like this (Beautified for this post but they call fit in 'paragraph' form as long as there is a semi-colon seperating each case):


filter f_dbg { level(debug); }; 
filter f_info { level(info); }; 
filter f_notice{ level(notice); }; 
filter f_warn { level(warn); }; 
filter f_err { level(err);  }; 
filter f_crit { level(crit .. emerg); };

filter f_debug { level(debug) and not facility(auth, authpriv, news, mail); };
filter f_error { level(err .. emerg) ; }; 
filter f_messages { level(info,notice,warn) and not facility(auth,authpriv,cron,daemon,mail,news);
};

filter f_auth { facility(auth, authpriv) and not filter(f_debug); }; 
filter f_cron { facility(cron) and not filter(f_debug); }; 
filter f_daemon { facility(daemon) and not filter(f_debug); }; 
filter f_kern { facility(kern) and not filter(f_debug); }; 
filter f_lpr { facility(lpr) and not filter(f_debug);}; 
filter f_local { facility(local0, local1, local3, local4, local5, local6, local7) and not filter(f_debug); }; 
filter f_mail { facility(mail) and not filter(f_debug); }; 
filter f_news { facility(news) and not filter(f_debug); };
filter f_syslog3 { not facility(auth, authpriv, mail) and not filter(f_debug); }; 
filter f_user { facility(user) and not filter(f_debug); }; filter f_uucp { facility(uucp) and not filter(f_debug); };

filter f_cnews { level(notice, err, crit) and facility(news); }; 
filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); };

filter f_ppp { facility(local2) and not filter(f_debug); }; 
filter f_console { level(warn .. emerg); };

Statistics

There's nothing more I enjoy better than some good statistics. When I run any server or service, be it at the house or at work I want to see what my server has processed over time. Beginning with version 3.1, syslog-ng now has a syslog-ng-ctl stats utility which has greatly simplified grabbing log files. Prior to 3.1 to fetch statistic files you would run: echo STATS | nc -U /var/run/syslog-ng.ctl.
Because I'm a regex geek I'm not thrilled with the semi-colons in the output of syslog-ng-ctl stats thus I run: syslog-ng-ctl stats | sed 's|;|\t|g' to clean up the output.
What you have when you type the above command is 6 columns: SourceName, SourceID, SourceInstance, State, Type and Number.
SourceName - The name of the Source, for instance: destination, source, global, center
SourceID - The ID you gave the source (a previous example was firewall_ddos_filter, other examples would be: d_mail, d_net_user, etc)
SourceInstance - The destination of the Source Instance such as a filename, or the name of an application for a program source (sql) or destination
State: - Status of the object: a (Active - Currently active and receiving data), d (Dynamic - Not continuously available) o (Once active but stopped receiving messages such as an orphaned object)
Type - Type of Statistic such as: Processed: Number of Messages that reached their destination Dropped: Number of dropped messages Stored: Number of messaged stored in message Queue waiting to be sent to destination Suppressed (not sent): Number of Suppressed Messaged Stamp: Timestamp of Last message sent. These statistics are reset when the syslog-ng service is reset.
Number: Number of Messages

Log Rotate, Log Rotate, LOG ROTATE

Was that a clear enough message for you? Rotating your message logs will save your butt in the log run. Without rotating your logs your log disk space will just continue to grow and grow eventually filling up your hard drive. Not only will log rotate save space, but it will make searching for log files on specific dates easier than pulling up a 50MB log file that you didn't set into log rotate and searching for a specific date. Depending on your distro, logrotate is located in /etc/logrotate.conf. As this isn't a blogpost on logrotate, I'll leave your configuration up to your imagination and give you an example on how I rotate my log files:
/var/log/remote/*/ { rotate 5 weekly missingok create }
This goes through /var/log/remote/*/ every week and rotates my logs. Logs are rotated for 1 month at which point I have a cronjob that tar-zips my old logs and they are moved off to a backup location where they are kept for another month before being rotated off. In a business environment of course logs would be kept for however long management and legal dictates, but for a home environment I feel 2 months of logs is good enough to troubleshoot any problems that might have come up in that time.

Syslog Client

As each server and device is different in their setup, I won't get too in-depth into this. Syslog communicates on UDP port 514, but as I stated earlier above, I also set the main syslog server to communicate on TCP port 1000 for other devices. This allows the syslog-ng server to listen on two ports, 514 UDP for devices that can't change their ports, and TCP 1000 for servers that you can specify port numbers. Why did I put TCP 1000 and not TCP 514? Because Linux uses tcp 514 for rsh (remote shell) which would have caused some problems with my (and other's) host system. If you plan on running syslog-ng on the outside of the world (and I would assume your setting authentication, and using TLS encryption) then setting a TCP port that's not typical would be your best bet.
1. For devices all you should need to do is tell the device to point to the hostname and make sure either UDP 514 or TCP 1000 is the destination
2. For rsyslog clients add the following line:

For TCP:  *.* @@ipaddress:1000
For UDP:  *.* @ipaddress:514

3. For syslog-ng clients add the following line:

*New syslog Protocol* syslog(host tranport [options];

*old syslog protocol* destination d_tcp { syslog(ip("remoteip")
transport("tcp") port(1000) localport(999)}; };

destination d_udp { syslog(ip("remoteip") transport("udp") port(514)
localport(999)}; };

Conclusion

Well there you have it, a birds eye view of syslog-ng. There is plenty more that you can learn about syslog-ng, as I just went into the basics of getting started. From here you can get into macros, increased filtering, and TLS/Certificate based encryption of syslog messages (which I might cover in a later blog post). By sending your syslog messages to a centralized syslog server, and backing up said syslog server, you can rest assured that your system messages are secure and easy to get to when you need them.

Upgrade MongoDB Replica Set to 2.4

2014-12-09T14:19:00.000+05:30

MongoDB is the leading NoSQL database, designed for how we build and run applications today. MongoDB empowers organizations to be agile and scalable. It helps them enable new types of applications, improve customer experience, accelerate time to market and reduce total cost of ownership (TCO). MongoDB is named from "huMONGOus" meaning "extremely large".

MongoDB is a general purpose, open-source database. MongoDB features:

Document data model with dynamic schemas
Full, flexible index support and rich queries
Auto-Sharding for horizontal scalability
Built-in replication for high availability
Text search
Advanced security
Aggregation Framework and MapReduce
Large media storage with GridFS

After having a overview of what Mongodb is,Lets have look on how to upgrade Mongodb to its latest version on Ubuntu.In the general case, the upgrade from MongoDB 2.2 to 2.4 is a binary-compatible “drop-in” upgrade: shut down the mongod instances and replace them with mongod instances running 2.4. However, before you attempt any upgrade please familiarize yourself with the Recommendations and Checklist

When upgrading, consider the following:

For all deployments using authentication, upgrade the drivers (i.e. client libraries), before upgrading the mongod instance or instances.
To upgrade to 2.4 sharded clusters must upgrade following the meta-data upgrade procedure.
If you’re using 2.2.0 and running with auth enabled, you will need to upgrade first to 2.2.1 and then upgrade to 2.4. See Rolling Upgrade Limitation for 2.2.0 Deployments Running with auth Enabled.
If you have system.users documents (i.e. for auth) that you created before 2.4 you must ensure that there are no duplicate values for the user field in the system.users collection in any database. If you do have documents with duplicate user fields, you must remove them before upgrading.

You can upgrade to 2.4 by performing a “rolling” upgrade of the set by upgrading the members individually while the other members are available to minimize downtime. Use the following procedure:

Upgrade the secondary members of the set one at a time by shutting down the mongod and replacing the 2.2 binary with the 2.4 binary. After upgrading a mongod instance, wait for the member to recover to SECONDARY state before upgrading the next instance. To check the member’s state, issue rs.status() in the mongo shell.

                   Akhil@akhil-3450:~# mongo
                         MongoDB shell version: 2.2.3
                         connecting to: test
                         > use admin;
> db.shutdownServer()

Now Install the Lastest package of Mongod

Issue the following command to import the 10gen public GPG Key:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

Create a /etc/apt/sources.list.d/10gen.list file and include the following line for the 10gen repository.

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

Now issue the following command to reload your repository:

sudo apt-get update

To install a specific release, append the version number to the package name:

sudo apt-get install mongodb-10gen=2.X.X

To install the latest stable version of MongoDB:

sudo apt-get install mongodb-10gen

Once you are done with all the above steps successfully,restart the mongodb and wait for this instances to became secondary and in-sync with the master.

Now continue upgrading the remaining secondary members by repeating the above steps.After you are done with upgrading secondaries to the latest version It's time to upgrade the master to latest version as well.

To upgrade master first we need to switch the master,So that one of the Secondary becomes master and the existing master becomes secondary.

Akhil@akhil-3450:~# mongo

MongoDB shell version: 2.2.3

connecting to: test

> use admin;

> rs.StepDown()

Once the primary has stepped down and another member has assumed PRIMARY state, as observed in the output of rs.status(),Follow the above mentioned steps for upgrading Secondaries in the replica set.

After all the members of replica set are upgraded to required version,you can switch back the primary and you are done with upgrade of mongodb.

Page Caching with Nginx

2014-12-09T14:18:00.000+05:30

Nginx can also be used as a cache server for serving cache static content.In this tutorial we will learn how to do page caching with Nginx.

To start, install the latest stable Nginx avaliable at http://wiki.nginx.org/NginxInstall.

Next edit your nginx.conf and add the proxy_cache_path directive to define a named cache storage. These are independent of servers and locations, and can be reused inside each later on.

...
http {
...

proxy_cache_path /var/cache/nginx keys_zone=anonymous:10m;

proxy_temp_path /tmp/nginx;

}

proxy_cache_path and proxy_temp_path — are used to set default cache folder settings and temporary folder

Next create the directory for the cache:

mkdir -p /var/cache/nginx

Once your are done with above mentioned steps,It time to make the changes in the nginx.conf.Add the following directives in the nginx conf for caching only get and head requests.

location / {

if ($request_method !~ ^(GET|HEAD)$) {

set $nocache 1;

}

proxy_pass http://127.0.0.1:8080;

proxy_cache browse;

proxy_cache_key $request_uri;

proxy_cache_valid 1440m;

proxy_cache_use_stale updating;

proxy_ignore_headers "Set-Cookie";

proxy_cache_bypass $nocache;

proxy_no_cache $nocache;

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

add_header X-Cached $upstream_cache_status;

log_format custom '$remote_addr - $remote_user [$time_local] '

'"$request" $status $body_bytes_sent '

'"$http_referer" "$http_user_agent" nocache:$upstream_cache_status';

access_log /var/log/nginx/microcache.log custom;

}

I only included what you need to add to your virtualhost configuration that you are already using, some more configuration can exist there

proxy_pass sets the address of the proxied server and the URI to which location will be mapped

proxy_cache you pass here the name of your cache server

proxy_cache_key this is important, do not forget to set this variable else content may be passed to other vhosts and that’s not good.

proxy_cache_valid set some validation on the cached files

proxy_cache_use_stale updating tells Nginx when to serve a stale item from the proxy cache.

proxy_ignore_headers Prohibits the processing of the header lines from the proxy server's response.

proxy_cache_bypass set the bypass headers that will invalidate the cache and refresh it

proxy_no_cache Specifies in what cases a response will not be cached

proxy_set_header This directive allows to redefine and to add some request header lines which will be transferred to the proxied server.

Make sure though you have created the cache folders before you reload the configuration.Once you are done with all the above steps,You caching has been setup successfully.To Test the Caching you can simple check the headers with curl.

A simple example with curl is given below:-

akhil@akhil-Dell-System-Vostro-3360:~# curl -I http://www.example.com/search/simple/CD
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 18 Mar 2014 07:40:07 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Location: http://www.example.com/search/simple/CD
Vary: Accept-Encoding

X-Cached: MISS

The X-cached filed shows whether the page is served from cache or not.if it's a MISS then the page is not served from the cache and if it's a HIT then the page has been served from the Cache.Requesting the same page again will change the status of X-Cached.

akhil@akhil-Dell-System-Vostro-3360:~# curl -I http://www.example.com/search/simple/CD
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 18 Mar 2014 07:40:07 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 0
Connection: keep-alive
Location: http://www.example.com/search/simple/CD
Vary: Accept-Encoding

X-Cached: HIT

And that’s all for today, hope it helps someone and will see you again.Don’t forget to comment below and subscribe!

Snort : Network Intrusion Detection System

2014-12-09T14:17:00.000+05:30

snort really isn't hard to use,this article aims to make using snort easier for new users

snort can be configured to run in three modes:

sniffer mode: which simply reads packets off the network

packet logger mode: which logs the packets to disk

Network Intrusion Dectection system mode: The most complex and most configurable.

This article will focus more on the installation and configuration of snort,if you are looking for detail description of snort,I would request you to please go thru the documentation.

Prerequisites

Libpcap-1.2.1-dev
daq-0.6.2
libdnet-1.12
libpcap-1.2.1
bison
flex
PCRE
libc6-dev
g++
gcc

The first thing I like to do is grab all the dependant packages once your are done with the installation of all the dependencies we are ready for the next step.It's time to download tar package of snort and untar the packages.I would like to have all the tar packages at a single place,So I'm going to use edge's structure here:

# cd /opt

# mkdir snortpackage

# cd /opt/snortpackage

Let's get snort. The latest version of snort at the time of writing is 2.9.2.1

Open a web browser and navigate to http://www.snort.org/; download the most recent release in /opt/snortpackages.It's time to untar the Snort package

#tar -xzvf /opt/snortpackage/snort-2.9.2.1.tar.gz

It's time to get snort rules.Change directories into the new snort-2.9.2.1 folder

# cd /opt/snortpackages/snort-2.9.2.1

Open a web browser and navigate to http://www.snort.org/snort-rules/#registered

Scroll down to the "Sourcefire VRT Certified Rules - The Official Snort Rule set (registered user release)" section and download the snort rules.

Untar the Snort Rules

# tar -xzvf /opt/snortpackages/snortrules-snapshot-2911.tar.gz

Now it's time start the installation of snort

# cd /opt/snortpackages/snort-2.9.2.1

Here we will do the make/install

# ./configure -enable-dynamicplugin --with-mysql
# make
# make install

after the above commands are executed successfully.We need to create some folders in /etc for snort to function correctly and copy some files over to them

# mkdir /etc/snort /etc/snort/rules /var/log/snort

Let's move some files.

# cd /opt/snortpackages/snort-2.9.2.1/rules
# cp * /etc/snort/rules/

Let's get the /etc snort files also.

# cd /opt/snortpackages/snort-2.9.2.1/etc
# cp * /etc/snort/

We are done with the installation of snort and now we need to tweak some options to configure snort according to your requirement.

We need to modify the snort.conf file to suite our needs.
Open /etc/snort/snort.conf with your favorite text editor.

# vim /etc/snort/snort.conf

Change "var HOME_NET any" to "var HOME_NET 192.168.1.0/24" (your home network may differ from 192.168.1.0)

Change "var EXTERNAL_NET any" to "var EXTERNAL_NET !$HOME_NET" (this is stating everything except HOME_NET is external)

Change "var RULE_PATE ../rules" to "var RULE_PATH /etc/snort/rules"

Once you are done with changes in the config,you can test those changes with following command:
# snort -c /etc/snort/snort.conf

We are done. Congrats!!!

To start Snort in the terminal type:

# snort -c /etc/snort/snort.conf -A fast -D

This starts snort in a daemon mode.

To make sure it is running you can check with the following command:

# ps aux | grep snort

If it's running you will see an entry similar to snort -c /etc/snort/snort.conf -A fast -D

Creating Jailkit With Chroot Sftp

2011-10-18T15:31:00.000+05:30

A few months back I was given an assignment to create some chroot jails for a group of customers so that they could securely upload files with sftp. The requirement was that the customers needed to be able to upload file, but in a secure and private way. Customer One should not be able to see Customer Two's files, for example. And neither customer should be able to browse the filesystem of the server. I was also asked to define a process whereby our support staff could add new jails as needed.
I've used the chroot command many times, it's a very handy way of fixing servers that can't otherwise be fixed --- like when no one can remember the root password on some box that no one has touched for years. Simply boot the server with a live CD (Knoppix is a personal favorite, I have it loaded on a usb key that I carry with me everywhere), mount the main drive and then issue a command along the lines of:



chroot /mnt/hda1

Suddenly it's like you are root on the local filesystem and not on the live CD. I can then change passwords, run programs, and generally get myself into trouble. :-)

You can do a lot more with chroot than simply use it as part of your system recovery toolkit though. Chroot is _very_ useful on a day-to-day basis for jailing applications and users that should (or need) to be isolated from the rest of the system (and from each other).

To create a jail you create a folder that has a replication of the directory structure of a normal Linux box. The difference is that you only copy in the bare minimum of what you need into the directory structure. The way you find out what libraries and files an application needs is by using ldd or strace. If I can get away with it, I prefer using ldd since its output is much more readable, but there are times when both are needed to track down elusive libraries.
As an example, lets say I want to have a jail that contains the bash shell, just that and nothing more. It's a stupid example, but very easy.
I first need to create the directory that will house the jail:



Akhil@groundzero:~$ mkdir /opt/jail

It's just an empty directory now, so we now need to add some apps to it. Bash lives in the /bin directory so I first recreate the /bin directory and then copy the bash binary into it like so:



Akhil@groundzero:~$ mkdir /opt/jail/bin

Akhil@groundzero:~$ cp /bin/bash /opt/jail/bin/

Then I run ldd on bash to find out what libraries it uses like so:



Akhil@groundzero:~$ ldd /bin/bash

linux-gate.so.1 => (0xffffe000)

libncurses.so.5 => /lib/libncurses.so.5 (0xb7ec2000)

libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7ebe000)

libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7d7d000)

/lib/ld-linux.so.2 (0xb7f1f000)

Akhil@groundzero:~$

I can ignore the linux-gate.so.1 file, it's a virtual object created by the Linux Kernel, but I need the others so I create the directories and then copy them over:



Akhil@groundzero:~$ mkdir -p /opt/jail/lib/tls/i686/cmov

Akhil@groundzero:~$ cp /lib/libncurses.so.5 /opt/jail/lib/

Akhil@groundzero:~$ cp /lib/tls/i686/cmov/libdl.so.2 /opt/jail/lib/tls/i686/cmov/

Akhil@groundzero:~$ cp /lib/tls/i686/cmov/libc.so.6 /opt/jail/lib/tls/i686/cmov/

Akhil@groundzero:~$ cp /lib/ld-linux.so.2 /opt/jail/lib/

Now I can test the jail like so:



Akhil@groundzero:~$ sudo chroot /opt/jail /bin/bash

bash-3.2#

The presence of the bash-3.2# prompt tells me I was successful.
This is a pretty useless jail right now, with only bash. I can 'cd' and 'pwd' and 'exit' and do other bash built-in commands, but I can't 'ls' or 'less' or 'cp' or 'mv' or anything else that is external to bash. Adding more binaries is the same as adding bash: copy the binary in and then ldd it to find its libraries and then copy them in. A lot of programs use the exact same libraries, so often you'll find that copying the binary in to an existing jail is all you need to do since the libraries will already be there.
Once I copy in a few binaries this could turn into a very nice sandbox. However, I had some specific needs when I started out building custom jails, and there are a few gotchas that the above jail doesn't come close to meeting:
1. Users assigned to a jail need to be automatically chrooted into their jail when they login

2. There should be no shell access --- we don't want them to be able to ssh in, they should only be able to use sftp and scp

3. This needs to be done yesterday, and it needs to be repeatable.

Number one is easy enough in theory, just create a false shell whose sole purpose is to chroot you into your jail. Yeah. Easy. Oh, and do it securely with no shortcuts that might open up security holes. Yup, definitely easy.
Number two actually is easy in theory and practice, just copy in the sftp and scp binaries and leave out the ssh binary and any shell binaries.
Number three is the real kicker. I needed a quick, repeatable process. Especially since once things are up and running the plan was to turn everything over to the support group so that they could create new jails as needed.
So what did I do? I went looking for something someone had already built and I found Jailkit.
There are other tools out there for automatically creating jails, so please do some searching and explore all options when choosing your own solution. Some of the things I like about jailkit are:
1. It has many pre-configured collections that you can mix and match to build the perfect jail. I call them jail-sets.
2. If none of the existing jail-sets meets your needs you can customize them or create new ones.
3. Creating multiple jails is easy and each jail can have multiple users assigned to it --- users can't belong to more than one jail though, but that's a good thing IMO.
4. Adding users to jails is very easy, almost to the point of "that's all there is to it?"
5. The fake shell that Jailkit uses to chroot users when they log in checks for potential security risks every time it is run and will disconnect the user if it notices anything bad (like the jail directory being setuid root, for example).
6. The Jailkit website has clear, easy to follow directions that walk you through several potential jail setups and it has a good troubleshooting section to help you when things don't work for some reason or other.
Here's what I did to create the jails on my server:
As far as the actual server is concerned, I didn't do much to it apart from installing a base ubuntu-server OS with the ssh-server option. Oh, and I also installed the build-essential meta-package so that I could compile and install Jailkit.
After downloading the tarball from the Jailkit website, Jailkit is installed from source like so:



Akhil@groundzero:~$ wget http://olivier.sessink.nl/jailkit/jailkit-2.5.tar.gz

Akhil@groundzero:~$ tar -zxvf jailkit-2.5.tar.gz

Akhil@groundzero:~$ cd jailkit-2.5

Akhil@groundzero:~/jailkit-2.5$ ./configure

Akhil@groundzero:~/jailkit-2.5$ make

Akhil@groundzero:~/jailkit-2.5$ sudo make install

The whole unpack-compile-install process is really quick. By default, Jailkit installs its binaries into /usr/sbin/ and the configuration and template files into /etc/jailkit/.
Once installed, creating jails is easy. As I said above, Jailkit comes with several default jail-sets. These are defined in /etc/jailkit/jk_init.ini and they include things like scp, sftp, apache, basicshell, and so on. As far as learning about what jail-sets are available and what each jail-set includes, the best thing I can say is to look through the /etc/jailkit/jk_init.ini file and familiarize yourself with them. This file is also where you can define your own custom sets.
You create a jail using the jk_init script and specifying where you want the jail to be located and what jail-sets you want included in the jail. The example below is what I use to create a jail that has sftp and scp functionality:



Akhil@groundzero:~$ sudo jk_init -v /path/to/jail sftp scp jk_lsh

The jk_lsh jail-set provides a special shell that limits the binaries it will execute via a config file. That way, even if a user uploads their own binary file, they won't be able to execute it.
The jail at this point (assuming nothing went wrong) is technically a valid jail, but there aren't any users in it yet and because this jail has no 'shell', it is pretty useless to try and chroot into it manually to test it. The next order of business therefore is to create some users and add them to the jail. Once a user has been created and added to a jail using the method below, they will always be put in that shell whenever they log in. So you absolutely need to create special jail-only users. Do not put regular users into jails!



Akhil@groundzero:~$ sudo groupadd companyname

Akhil@groundzero:~$ sudo useradd -g companyname -m username

Akhil@groundzero:~$ sudo passwd username

Akhil@groundzero:~$ sudo jk_jailuser -m -j /path/to/jail username

The reason I create a group is because I am creating jails per client company on this server and it made sense to group (no pun intended) all the users that way. I also threw in the passwd command into the example above since the useradd command (on Ubuntu at least) doesn't set a password by default. Things may be different on your Linux distro of choice.
The jk_jailuser utility, when run as in the example above, changes the shell of an existing user to jk_chrootsh in the /etc/passwd file and it changes the home directory to /path/to/jail/./home/username. It also adds the user to a stripped down passwd file located at /path/to/jail/etc/passwd and adds the user's groups to a stripped down group file located at /path/to/jail/etc/group.
If you want to edit the two passwd files yourself you can do so. Just change the user's line in /etc/passwd to something resembling this:



username:x:1010:1010::/path/to/jail/./home/username:/usr/sbin/jk_chrootsh

and change the user's line in /path/to/jail/etc/passwd to something resembling this:



username:x:1010:1010::/home/username:/usr/sbin/jk_lsh

(I'm afraid of messing things up if I do it manually, so I use the jk_jailuser utility whenever possible.)
Now we need to allow the user to execute the scp and sftp programs. Edit the /path/to/jail/etc/jailkit/jk_lsh.ini file and add in some lines like so:



[group companyname]

paths= /usr/bin, /usr/lib/

executables= /usr/bin/scp, /usr/lib/sftp-server, /usr/lib/openssh/sftp-server, /usr/libexec/sftp-server

That's all there is to it. I now have a fully functioning jail and I can test it by connecting to the server using sftp or scp. When I sftp in and look around, I only see the directories below the jail directory, exactly the behavior I wanted.
Should you find later on that the jail needs to have some other binary in it, Jailkit also includes a handy copy command that you can use to copy additional files into jails like so:



Akhil@groundzero:~$ sudo jk_cp /path/to/jail /path/to/command

Very simple, and it handles all of the locating of dependencies, libraries, and whatnot. The only gripe I have about the jk_cp command is that you have to specify jk_cp
instead of doing it the way the cp command works, which is cp . On occasion I have run into issues where Jailkit doesn't copy over everything properly, or it simply misses some libraries. In those cases I had to experiment with copying over additional libraries. Rather than reinvent the wheel, I'll just point you to the Jailkit website, which has a very nice page on troubleshooting jails. (it's in the links)
Once I had jailkit working on my jail server and I had a repeatable process for adding users, my next step was to create two _very_ simple shell scripts for creating jails and creating jailed users. These scripts are used by our support staff, who have limited sudo access to run only these scripts as root along with the passwd command (so that they can change jailed users' passwords, if need be).
The two scripts are called mkjail and jailuser. They're not all that pretty, and they could be considered dangerous if used by the wrong people. But for internal use by a select group of specifically authorized and trained people, they work fine.
With the server set up and the scripts in place and the staff trained my job was done. The server is under constant use and new jails are created whenever they are needed with no intervention needed on my part. Of course, the minute the project was completed I was immediately assigned three more . . . but that's what's great about working in technology, right? It's never boring, that's for sure. :-)
Admittedly, creating an sftp jail is probably not something that everyone will need to do, but jailkit has found its way into my sysadmin bag of tricks. Whenever I need to isolate a user or a process into a sandbox, and a Xen virtual machine is way too much overkill, the first thing I reach for is jailkit.

Enhance Security with Port Knocking

2010-11-13T05:54:00.000+05:30

In the field of IT systems security, concept of” port knocking” is relatively new. However with the passage of time, it is getting popular day by day among system and security administrators.

Port knocking is a method of externally opening ports on a firewall by generating a connection attempt on a set of pre-specified closed ports. Once a correct sequence of connection attempts is received, the firewall rules are dynamically modified to allow the host which sent the connection attempts to connect over specified port (s).
The primary purpose of port knocking is to prevent an attacker from scanning a system for potentially exploitable services by doing a port scan. Until the correct knock sequence is used, the protected ports will appear closed– so attackers won’t be able to conduct an attack on those ports.

More specifically, Port knocking works on the concept that users wishing to attach to a network service must initiate a predetermined sequence of port connections or send a unique string of bytes before the remote client can connect to the eventual service.
For example, suppose that a remote client wants to connect to an FTP server. The administrator configures the port-knocking requirements ahead of time, requiring that connecting remote clients first connect to ports 2000, 4000, and 7107 before connecting to the final destination port, 21 on FTP server.
The administrator tells all legitimate clients about the correct” combination” of knocks to port knocking daemon running on FTP server and hence when they want to connect to FTP service, they simply send these knocks to the server and then start using FTP service.
The question arises, what is the basic advantage of the additional step of sending knocks and then connecting to FTP service? The answer is simple: The FTP service is not always running on the server, it will be started only when the correct port knocks are sent to server, and it will shut down once it receives another predefined sequence of port knocks.
The potential backdoor to business-critical services is only to be opened for a short time, when it’s required. Once the service is no longer needed, it is closed again, mitigating the vulnerability to attack.
One of the primary advantages to using port knocking is that it is platform, service, and application agnostic. Any operating system with the correct client and server software can take advantage of port knocking. If you need help finding a tool, you can find a list of port knocking implementations here. The site lists clients and daemons for pretty much any platform you’d care to use.
I selected knockd, which is considered to be one of the most famous and robust implementation of port knocking mechanism for Linux and UNIX. In this article, I will cover setting up port knocking on a Red Hat Enterprise Linux (RHEL) server, using knockd, a popular open source port knocking tool. Most importantly, I will try to extend the idea of port knocking beyond simple firewall modifications to more complex system administration tasks.
Note that knockd is available for other systems as well, so if you’re using Debian, Ubuntu, Mac OS X, or even Windows, you should be able to follow along with most of the advice herein to secure your system with knockd.

Flaws with Port Knocking

Before we begin, I should note that port knocking has some detractors. Some IT security professionals say that a predefined and fixed sequence of knocks is, in and of itself, a security flaw. To overcome this, some port knocker daemons have been modified to generate a random sequence of knocks, which can be used by clients to issue requests.
It’s also important to remember that port knocking is just one component of a successful security strategy. You’ll need to deploy other security mechanisms so that if an attacker is successful in providing the correct sequence, they are still faced with authentication and other barricades before connecting to a service.

Port Knocking: A Basic Overview

To start, let’s take a look at the basic functionality of a port knock server. knockd is a daemon that runs on a server, passively listening to network traffic. You configure knockd with a sequence of ports, the length of time between connection attempts, the type of packet that will be sent, and the command to be run when the correct sequence is given.
Once knockd” sees” a port sequence it has been configured to recognize, it will run the command it’s been configured to run. Note that you can use TCP, UDP, or a combination of both. Usually the action will be an iptables command, but not always.
So, to implement port knocking, we start with the installation of knockd and run it in the background. (Or foreground, if you wish, but we will usually want to run it in the background.)

Securing A MySQL Database Remote Connections with Port Knocks

Now that we know what port knocking is, let’s put it to use. In this scenario, I have a business-critical MySQL-based application running on RHEL. On occasion, I need to allow remote connections from a DBA who is performing basic database maintenance activities.
However, for security reasons, we don’t want to allow remote database connections at all times or from every IP address. Because we wanted tighter control over remote connections, we decided to explore port knocking so that remote connections would be open for a limited time only and from a specific IP address.
Let’s start with the firewall rule, just in case you’re not already a firewall wizard. To append a rule to one of the” chains,” you’ll use the -A option. The -I parameter tells iptables to insert the rule into a specific position in the chain. This is important because you may want specific rules to be processed first. Make sure you give it a rule number.
Now, to secure MySQL connections to my database server (172.16.2.183), I blocked network traffic on server’s MySQL port (default 3306) coming from all addresses. For this purpose, I executed following command:

iptables -A INPUT -p tcp -s 0/0 -d 172.16.2.183 --dport 3306 -j REJECT

You don’t want to be reissuing the command every time you restart the machine, so you’ll want to save the rule permanently, using iptables-save.

Getting and configuring knockd

The next step is to install the knockd server on the system you want to use it on. You can get the RPM from the RHEL network.
After installing knockd it’s time to customize your configuration. The knockd config file is found at /etc/knockd.conf

[options]
  logfile=/var/log/knockd.log
[DB2clientopen]
        sequence    = 7050,8050,9050
        seq_timeout = 10
        tcpflags    = syn
        command     = /sbin/iptables -I INPUT 1 -p tcp -s 192.168.2.201 
       --sport 1024:65535 -d 172.16.2.183 --dport 3306 -m state 
 --state NEW,ESTABLISHED -j ACCEPT
[DB2clientclose]
        sequence    = 9050,8050,7000
        seq_timeout = 10
        tcpflags    = syn
        command     = /sbin/iptables -D INPUT 1

Let’s take a look at the format. The syntax is very simple, you give knockd option / value pairs, separated by =, and port numbers are separated by commas in the order you want the” knocks” to be received. Don’t forget to specify a logfile, you may need to review it later!
It should be obvious from the knockd.conf example that it has two types of actions that can be executed by the daemon, depending on the sequence it receives.
First, if it receives syn packets to port 7050, 8050, and 9050, knockd will insert the first iptables rule as rule number 1 in the INPUT chain. This will open the MySQL database port, so a remote connection can be made from 192.168.2.201– and only that IP address. It’s a good idea to specify the IP address whenever possible, so that if an attacker tries to connect while the port is open, they will still be denied.
On the other hand, if the server receives a knock sequence of 9050, 8050, and 7000, it will delete the rule so that all remote database connections will be closed down again.
I made sure that MySQL would know what address my DBA would be coming from, so I added my PC’s IP address to the server’s /etc/hosts file and created a test database called test1, and created a user called test1 as well with the appropriate grant privileges.
First, fire up the MySQL client with mysql-u root-p test1 and enter the following commands:

mysql> create user test;
mysql> grant all privileges on *.* to 'test@dbawin' 
identified by 'polanipass' with grant option;

Next, restart knockd as a daemon.

/usr/sbin/knockd -d

It should be noted that, by default, knockd will start listening on eth0. If you need it to run on a different interface, you can configure it to do so using the -i option. For instance, to start knockd as a daemon on wlan0 you’d use /usr/sbin/knockd-i wlan0. If you’re always going to run knockd on a different interface, you can add this to your knockd.conf:

[options]
 interface = wlan0

Knock, Knock, It’s Me!

Now, knockd isn’t very useful without a client, so let’s get a client to talk to knockd. I chose a Windows-based Cygwin client, but you can find a client for just about any client OS at the implementations page mentioned earlier.
To use the Windows client, you open a DOS prompt and run something similar to this command:

C:KNOCKKNOCKWINDOWS>knock.exe 172.16.2.183 9050 8050 7000

Of course, the IP address and ports will vary. Once the” knock” is issued, the knock daemon will execute the iptables command listed under the [DB2clientopen] section of knockd.conf and add the rule in INPUT chain to allow DB2 PC to connect to database running on server.
Now you can connect to your MySQL database with your favorite client and do whatever you need to do. Once you’re finished, it’s time to close the door.
If you send the close knock sequence, in this case a syn packet sent to ports 9050, 8050, and then 7000, the MySQL port will be closed and all connections will be terminated. If you try to reconnect to the server, your MySQL client will time out and you’ll eventually see an access error. This will be the case until you send the proper sequence to re-open the port.
So, now you see how you can use port knocking to increase security for remote MySQL connections. Of course, this is really database (and application) independent, so you can use port knocking to secure any database or application you want to connect to remotely.
If it’s too much hassle to open and close the connection each time you need to connect to the database, it might make more sense to set it up so that the port is open during specific hours. For example, if your database guru works 10 a.m. to 7 p.m., you could set up a script to open the port a bit before 10 a.m., and close the port a bit after 7 p.m.
This is not quite as secure, but it does mean that the port won’t be open 24/7, so it may block some automated and casual (i.e., not targeted) attacks. Also, if the port knocking is coupled with only allowing connections from specific IP addresses or IP address ranges, then you have an additional layer of security.
Performing Other System Administration Tasks with Knocks
But wait, that’s not all! Port knocking can be used to do more than set iptables rules. After configuring knockd to play doorkeeper, I decided to explore the feature and see if I could use it to make my life easier in other ways.
I decided I wanted to be able to restart my system remotely, just by” knocking” in the right sequence. I also configured knockd to kick off my backups to tape, so I don’t even need to log in to start the backup– just send a quick series of packets, and my data is safe another day.
Here’s my /etc/knockd.conf:

[options]
  logfile=/var/log/knockd.log
[systemreboot]
        sequence    = 7050,8050,9050
        seq_timeout = 10
        tcpflags    = syn
        command     = /usr/bin/reboot
[systembackup]
        sequence    = 9050,8050,7000
        seq_timeout = 10
        tcpflags    = syn
        command     =  /usr/bin/tar -cf  /dev/rmt0 /home/root/

You can take this a lot farther, and set it up so that other admins (say, the junior admin who’s reliable but still a bit green) can perform complex actions just by using a knock client, or even just by running a shell script that sends the packets.

Summary

Port knocking is a very useful tool for systems security. It is because of its usefulness and robustness that the number of implementations, and users, are growing rapidly. If you can open a door into a closed black box for to perform some system administration tasks, even without requiring a login to the system, it can be very ideal for many environments.
Finally, it is always a good idea to further secure your systems by changing the knock sequences frequently, or by using random seed generators to create random port knocks.

Svn backup

2010-04-12T14:54:00.000+05:30

Whatever your reason for migrating repository history, using the svnadmin dump and svnadmin load subcommands is straightforward.svnadmin dump will output a range of repository revisions that are formatted using Subversion's custom filesystem dump format. The dump format is printed to the standard output stream, while informative messages are printed to the standard error stream. This allows you to redirect the output stream to a file while watching the status output in your terminal window. For example:
$ svnlook youngest myrepos
26
$ svnadmin dump myrepos > dumpfile
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
…*
Dumped revision 25.
* Dumped revision 26.At the end of the process, you will have a single file (dumpfile in the previous example) that contains all the data stored in your repository in the requested range of revisions. Note that svnadmin dump is reading revision trees from the repository just like any other “reader” process would (e.g., svn checkout), so it's safe to run this command at any time.The other subcommand in the pair, svnadmin load, parses the standard input stream as a Subversion repository dump file and effectively
replays those dumped revisions into the target repository for that operation. It also gives informative feedback, this time
using the standard output stream:
$ svnadmin load newrepos <><<<>>>
<<<>>>
<<<>------- Committed new rev 25 (loaded from original rev 25) >>>
<<<>------- Committed new rev 26 (loaded from original rev 26) >>>The result of a load is new revisions added to a repository—the same thing you get by making commits against that repository from a regular Subversion client. Just as in a commit, you can use hook programs to perform actions before and after each of the commits made during a load process. By passing the --use-pre-commit-hook and --use-post-commit-hook options to svnadmin load, you can instruct Subversion to execute the pre-commit and post-commit hook programs, respectively, for each
loaded revision. You might use these, for example, to ensure that loaded revisions pass through the same validation steps that regular commits pass through. Of course, you should use these options with care—if your post-commit hook sends emails to a mailing list for each new commit, you might not want to spew hundreds or thousands of commit emails in rapid succession at that list! You can read more about the use of hook scripts in the section called “Implementing Repository Hooks”.
Note that because svnadmin uses standard input and output streams for the repository dump and load processes, people who are feeling especially saucy can try things such as this (perhaps even using different versions of svnadmin on each side of the pipe):
$ svnadmin create newrepos
$ svnadmin dump oldrepos svnadmin load newreposBy default, the dump file will be quite large—much larger than the repository itself. That's because by default every version of every file is expressed as a full text in the dump file. This is the fastest and simplest behavior, and it's nice if you're piping the dump data directly into some other process (such as a compression program, filtering program, or loading process). But if you're creating a dump file for longer-term storage, you'll likely want to save disk space by using the --deltas option. With this option, successive revisions of files will be output as compressed, binary differences—just as file revisions are stored in a repository.
This option is slower, but it results in a dump file much closer in size to the original repository.
We mentioned previously that svnadmin dump outputs a range of revisions. Use the --revision (-r) option to specify a single revision, or a range of revisions, to dump. If you omit this option, all the existing repository revisions will be dumped.
$ svnadmin dump myrepos -r 23 > rev-23.dumpfile
$ svnadmin dump myrepos -r 100:200 > revs-100-200.dumpfile
As Subversion dumps each new revision, it outputs only enough information to allow a future loader to re-create that revision based on the previous one. In other words, for any given revision in the dump file, only the items that were changed in that revision will appear in the dump. The only exception to this rule is the first revision that is dumped with the current svnadmin dump command.
By default, Subversion will not express the first dumped revision as merely differences to be applied to the previous revision. For one thing, there is no previous revision in the dump file! And second, Subversion cannot know the state of the repository into which the dump data will be loaded (if it ever is). To ensure that the output of each execution of svnadmin dump is self-sufficient,
the first dumped revision is, by default, a full representation of every directory, file, and property in that revision of the repository. However, you can change this default behavior. If you add the --incremental option when you dump your repository, svnadmin will compare the first dumped revision against the previous revision in the repository—the same way it treats every other revision that gets dumped. It will then output the first revision exactly as it does the rest of the revisions in the dump range—mentioning only the changes that occurred in that revision. The benefit of this is that you can create several small dump files that can be loaded in succession, instead of one large one, like so:$ svnadmin dump myrepos -r 0:1000 > dumpfile1
$ svnadmin dump myrepos -r 1001:2000 --incremental > dumpfile2
$ svnadmin dump myrepos -r 2001:3000 --incremental > dumpfile3
These dump files could be loaded into a new repository with the following command sequence:
$ svnadmin load newrepos <>

$ svnadmin load newrepos <>Another neat trick you can perform with this --incremental option involves appending to an existing dump file a new range of dumped revisions. For example, you might have a post-commit hook that simply appends the repository dump of the single revision that triggered the hook. Or you might have a script that runs nightly to append dump file data for all the revisions that were added to the repository since the last time the script ran. Used like this, svnadmin dump can be one way to back up changes to your repository over time in case of a system crash or some other catastrophic event.
The dump format can also be used to merge the contents of several different repositories into a single repository. By using the --parent-dir option of svnadmin load, you can specify a new virtual root directory for the load process. That means if you have dump files for three repositories—say calc-dumpfile, cal-dumpfile, and ss-dumpfile—you can first create a new repository to hold them all:
$ svnadmin create /var/svn/projects
$Then, make new directories in the repository that will encapsulate the contents of each of the three previous repositories:
$ svn mkdir -m "Initial project roots" \
file:///var/svn/projects/calc \
file:///var/svn/projects/calendar \
file:///var/svn/projects/spreadsheet
Committed revision 1.
$Lastly, load the individual dump files into their respective locations in the new repository:
$ svnadmin load /var/svn/projects --parent-dir calc <>

…$ svnadmin load /var/svn/projects --parent-dir calendar <>

…$ svnadmin load /var/svn/projects --parent-dir spreadsheet <> We'll mention one final way to use the Subversion repository dump format—conversion from a different storage mechanism or version control system altogether. Because the dump file format is, for the most part, human-readable, it should be relatively easy to describe generic sets of changes—each of which should be treated as a new revision—using this file format. In fact, the cvs2svn utility (see the section called “Converting a Repository from CVS to Subversion”) uses the dump format to represent the contents of a CVS repository so that those contents can be copied into a Subversion repository.

Five Steps to Secure Your Linux System

2010-03-27T10:16:00.000+05:30

There’s no excuse to run insecure systems on your network. Your data’s integrity (and your job) depend on your ability to keep those systems running correctly and securely for your co-workers and customers. Shown here are five simple ways to make your systems less vulnerable to compromise.

Account Locking

Account locking for multiple failed tries puts extra burden on the system administrators but it also puts some responsibility on the user to remember his passwords. Additionally, locking allows the administrator to track the accounts that have potential hack attempts against them and to notify those users to use very strong passwords.
Typically, a system will drop your connection after three unsuccessful attempts to login but you may reconnect and try again. By allowing an infinite number of failed attempts, you’re compromising your system’s security. Smart system administrators can take the following measure to stop this threat: Account lockout after a set number of attempts. My preference is to set that limit to three.
Add the following lines to your system’s /etc/pam.d/system-auth file.

auth    required   /lib/security/$ISA/pam_tally.so onerr=fail no_magic_root
account required   /lib/security/$ISA/pam_tally.so per_user deny=3 no_magic_root reset

Your distribution might not include the system-auth file but instead uses the /etc/pam.d/login file for these entries.

Cron Restriction

On multiuser systems, you should restrict cron and at to root only. If other users must have access to scheduling, add them individually to the /etc/cron.allow and /etc/at.allow files. If you choose to create these files and add user accounts into them, you also need to create /etc/cron.deny and /etc/at.deny files. You can leave them empty but they need to exist. Don’t create an empty /etc/cron.deny unless you add entries to the /etc/cron.allow because doing so allows global access to cron. Same goes for at.
To use the allow files, create them in the /etc directory and add one user per line to the file. The root user should have an entry in both allow files. Doing this restricts cron to the root user only.
As the system administrator, you can allow or deny cron and at usage based upon the user’s knowledge and responsibility levels.

Deny, Deny, Deny

“Deny everything” sounds eerily Presidential doesn’t it? But for system security and certain political indiscretions, it’s the right answer. System security experts recommend denying all services for all hosts using an all encompassing deny rule in the /etc/hosts.deny file. The following simple entry (ALL: ALL) gives you the security blanket you need.

#
# hosts.deny    This file describes the names of the hosts which are
#               *not* allowed to use the local INET services, as decided
#               by the '/usr/sbin/tcpd' server.
#
# The portmap line is redundant, but it is left to remind you that
# the new secure portmap uses hosts.deny and hosts.allow.  In particular
# you should know that NFS uses portmap!

ALL: ALL

Edit the /etc/hosts.allow file and insert your network addresses (192.168.1., for example) where you and your users connect from before you logout or you’ll have to login via the console to correct the problem. Insert entries similar to the following to allow access for an entire network, single host or domain. You can add as many exceptions as you need. The /etc/hosts.allow file takes precedence over the /etc/hosts.deny to process your exceptions.

Deny SSH by Root

Removing the root user’s ability to SSH provides indirect system security. Logging in as root to a system removes your ability to see who ran privileged commands on your systems. All users should SSH to a system using their standard user accounts and then issue su or sudo commands for proper tracking via system logs.
Open the /etc/ssh/sshd_config file with your favorite editor and change PermitRootLogin yes to PermitRootLogin no and restart the ssh service to accept the change.

Change the Default Port

While changing the default SSH port (22) will have limited effectiveness in a full port sweep, it will thwart those who focus on specific or traditional service ports. Some sources suggest changing the default port to a number greater than 1024, for example: 2022, 9922 or something more random, such as 2345. If you’re going to use this method as one of your strategies, I suggest that you use a port that doesn’t include the number 22.
Edit your /etc/ssh/sshd_config and change the “Port” parameter to your preferred port number. Uncomment the Port line too. Restart the sshd service when you’re finished and inform your users of the change. Update any applicable firewall rules to reflect the change too.
System security is important and is a constant battle. You have to maintain patch levels, updates and constantly plug newly discovered security holes in system services. As long as there are black hat wearing malcontents lurking the Net looking for victims, you’ll have a job keeping those wannabe perpetrators at bay.

Akhil's Blog

3 Amazing facts about SElinux.

What is SELinux?

SELinux MODES

Enforcing Mode Premissive Mode Disabled

Enable And Disable SELinux Mode

1. Make the changes in /etc/sysconfig/selinux file.

2. Changed the Mode at command line.

How to Install Mongodb on Ubuntu

Install Mongodb On Ubuntu

What is Mongodb?

Install Mongodb Ubuntu System

MysqlDump - A program to take Mysql Backup

What is Mysqldump?

--add-drop-table -d -e -w -x -f --no-tablespaces

Amazon S3 - Simple Storage Service

What is Amazon S3?

How to Start With Amazon S3 - Simple Storage Service

Sign-up for Amazon S3

Create Bucket for Amazon S3

Add An Object To The Bucket for Amazon S3

View Object from Amazon S3

Setup Mysql Replication in Master Slave Mode

MySQL replication

Setup

Step One—Setup the Master Database

Step Two—Configure the Slave Database

Install MongoDB on Fedora, Red Hat Enterprise and CentOS

Considerations

For production deployments, always run MongoDB on 64-bit systems.

Install Mongodb

DRBD and Heartbeat

How It Works

How to Get It Running

Setting Up the Basics on Your Servers

Configuring DRBD

Preparing Your Services

Centralized Syslog Server

Installing and Configuring - Server Side

Global Options

Setting up Listener

Statistics

Log Rotate, Log Rotate, LOG ROTATE

Syslog Client

Conclusion

Upgrade MongoDB Replica Set to 2.4

Page Caching with Nginx

Snort : Network Intrusion Detection System

We are done. Congrats!!!

Creating Jailkit With Chroot Sftp

Enhance Security with Port Knocking

Port Knocking: A Basic Overview

Svn backup

Five Steps to Secure Your Linux System

Enforcing Mode

Premissive Mode

Disabled

--add-drop-table

-d

-e

-w

-x

-f

--no-tablespaces