Linux Help

TCP trace route in windows using tracetcp

noreply@blogger.com (Randeep) — Mon, 01 Apr 2013 18:54:00 +0000

Trace route is a nice tool to find where in network resides the actual problem. Windows tracert uses ICMP. ICMP traces can often be inaccurate. To get a more realistic picture of what is going on we will use TCP based trace route. For this in windows based systems we can use tracetcp tool. In this post we will see how to install tracetcp in windows 7 and use it.

You can download a TCP trace route utility for Windows from the following location:
http://tracetcp.sourceforge.net/

You will need to download and install WinPcap first to be able to use tracetcp from the command line. You can download WinPcap from here:
http://www.winpcap.org/install/default.htm

Installing Wincap:

Click next:

Click agree and complete the installation.

Now download tracetcp, unzip it and copy it to your executable's location.

Now open the command prompt and run tracetcp command.

Syntax:
#tracetcp host

Thats it :)

Amazon AMI Copy example with snapshots

noreply@blogger.com (Randeep) — Mon, 25 Mar 2013 07:42:00 +0000

Amazon recently announced a new feature to copy the AMIs over regions. It makes replication very easy process. You have to create Linux AMI or Windows AMI of your instances. Then copy it to other regions in case if the current region goes down you can run your site/application from other regions. I have explained how to create AMIs of Linux and windows in previous posts. Also the replication of AMI. But this feature makes it very easy.

In this example we will show how to copy an AMI from North Virginia(US-East) to Oregon(US-West):

Here We have an AMI in North Virginia

Right click on it and click copy

Choose the Destination Region and give a name for new AMI. And click yes copy. Copying is initiated. It will take20-30 minutes for copying the AMI.

Copying is initiated. It will take20-30 minutes for copying the AMI.

You can go to Oregon dashboard and check the status. While processing it will be in Pending status. After finishing it will be Available.

Don't just finish and run. Always check its working by launching one instance and make sure you terminate the test instance after testing.

configuring tftp server in Ubuntu

noreply@blogger.com (Randeep) — Tue, 18 Dec 2012 07:12:00 +0000

We use tftp server for network boots. Loading images etc. In this post we will see how to set up a tftp server in Ubuntu.

Install xinetd:
tftp doesn't have a daemon of its own. So it uses xinetd for service control. It is not installed in Ubuntu by default.
apt-get install xinetd

Now install tftpd:
apt-get install tftpd

Create tftp content repository directory
We have to create a directory or use an existing directory to keep the images available through tftp.
mkdir /tftpboot

Create tftp xinetd configuration file:
#vi /etc/xinet.d/tftp
service tftp
{
disable = no
socket_type = dgram
protocol = udp
wait = yes
user = root
server = /usr/sbin/in.tftpd
server_args = -s /tftpboot
per_source = 11
csp = 100 2
}

Now the configuration is done. We can restart the xinetd to enabling tftp server.
#service xinetd restart
or
#sudo service xinetd restart

Thats it. Enjoy. Don't forget to set enough env variables on client while accessing tftp service.

benchmarking with http_load

noreply@blogger.com (Randeep) — Sat, 03 Nov 2012 17:59:00 +0000

There are a lot tools are there to benchmark a webserver. One of the useful tool is http_load. It is very simple to use. First you have to download and install http_load. Then test the webserver with urls and different options. With http_load you can run multiple http fetches in parallel to test the performance. It gives you a rough idea of how many bytes a server can serve in a time period.
Install the http_load tool:
wget http://www.acme.com/software/http_load/http_load-12mar2006.tar.gz
tar xvzf http_load-12mar2006.tar.gz
cd http_load-12mar2006
make
make install #if required

Now create a file of urls you want to test.
vi urls.txt
# cat urls.txt
http://www.linuxhelp.in/
http://www.linuxhelp.in/index.html

Now start the test.
./http_load -rate 5 -seconds 5 urls.txt
./http_load -rate 500 -seconds 10 urls.txt
Some points to remember:
parallel – number of requests to open at a time
rate – number of requests to open per second

jitter – vary the rate by about 10%

fetches – quit after this many fetches
seconds – seconds to run the program
Example:
[root@work http_load-12mar2006]# ./http_load -rate 500 -seconds 10 urls.txt
4629 fetches, 435 max parallel, 1.56025e+08 bytes, in 10.0193 seconds
33706 mean bytes/connection
462.007 fetches/sec, 1.55724e+07 bytes/sec
msecs/connect: 271.956 mean, 4062.13 max, 1.98 min
msecs/first-response: 159.822 mean, 5099.23 max, 2.672 min
HTTP response codes:
code 200 -- 4629
[root@work]#

No space left on device: mod_jk: could not create jk_log_lock

noreply@blogger.com (Randeep) — Mon, 15 Oct 2012 11:01:00 +0000

I got this error while working on Apache.
[crit] (28)No space left on device: mod_jk: could not create jk_log_lock
Configuration Failed

It was due to the semaphore limitation.
[root@host ~]# ipcs -su
------ Semaphore Status --------
used arrays = 127
allocated semaphores = 127
[root@host~]#

[root@host~]# ipcs -s
------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 4292608 nobody 600 1
0x00000000 4325377 nobody 600 1
0x00000000 4358146 nobody 600 1
0x00000000 4390915 nobody 600 1
0x00000000 4423684 nobody 600 1
0x00000000 4456453 nobody 600 1

Apache was running as nobody. In your case it may be as Apache itself.
I had to kill all the semaphores and restart Apache.
You can either kill one by one or use some script if a lot of semaphores are there.

To kill one by one:
[root@host~]# ipcrm -s

By script:
[root@host~]# for i in `ipcs -s | grep nobody | cut -d " " -f2`;do ipcrm -s $i; done
Now restart Apache. It should work.

Installing SSL certificate on Apache with Tomcat

noreply@blogger.com (Randeep) — Thu, 11 Oct 2012 07:41:00 +0000

I have an Apache tomcat stack on which Apache is front proxy and tomcat is serving contents. How to install SSL on this scenario? Whether to install ssl on Apache or Install ssl on Tomcat? In my case Apache and tomcat is connected using mode_jk. I installed ssl on apache. Here is the steps how I did. Comment if you know some better methods. Operating system used is Centos 5.4, Apache version is httpd-2.2.3-65.el5.centos and Tomcat tested on 5 and 6.

Check here for Installing and configuring Apache with tomcat using mod_jk
Generating key and csr:
yum install mod_ssl openssl

Key:
openssl genrsa -out www.yourdomain.com.key 2048

csr:
openssl req -new -key www.yourdomain.com.key -out www.yourdomain.com.csr
You have to give

Country Name (2 letter code) [GB]:IN
State or Province Name (full name) [Berkshire]:Karnataka
Locality Name (eg, city) [Newbury]:Bangalore
Organization Name (eg, company) [My Company Ltd]:Some Company Private Limited
Organizational Unit Name (eg, section) []:IT
Common Name (eg, your name or your server's hostname) []:www.yourdomain.com
Email Address []:someone@yourdomain.com
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []: [You can skip by pressing enter]
An optional company name []: [You can skip by pressing enter]

Now you have generated csr.
Now buy ssl for this csr from Godaddy,bigrock,geotrust or whomever you trust. Don't forget to keep the key safe.

Now when the crt is issued do the following.
Save the ssl and ca certificates
Add ssl.conf to /etc/httpd/conf.d/ (It will be there if you install mod_ssl)
Include ssl.conf in your httpd.conf

And give the following in ssl.conf

SSLCertificateFile /etc/pki/tls/certs/www.yourdomain.com.crt
SSLCertificateKeyFile /etc/pki/tls/private/www.yourdomain.com.key
SSLCACertificateFile /etc/pki/tls/private/www.yourdomain.com.ca.crt

Restart apache. You are done.
To redirect all the http to https you can use the following rewrite rules.

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

url redirection in apache using proxypass

noreply@blogger.com (Randeep) — Sun, 07 Oct 2012 17:55:00 +0000

Url redirection in Apache webserver.
Here is a small example of url redirection in Apache using proxypass. I used this when I was using Apache as proxy to Apache tomcat using mod_jk.

Example:
you want to forward
www.yourdomain.come/abc to www.yourdomain.com/linux/commands/abc

Here is how you can do it. Open httpd.conf in your favorite editor. Add the following lines.
ProxyPass /abc http://localhost/linux/commands/abc
ProxyPassReverse /abc http://localhost/linux/commands/abc

But while using above solution you may have to give trailing "/" to correct it use the following rewrite rule.
RewriteEngine on
RewriteRule ^/abc$ /abc/ [R]

But for using the above solution you have to load modules mod_proxy and mod_rewrite.
Load the modules, add the rules, restart Apache You are done. Please suggest if you know any better ways to do it. Remember that all my webapps is in tomcat and Apache is just front end.

Configuring multiple domains or sub-domains in tomcat

noreply@blogger.com (Randeep) — Wed, 05 Sep 2012 18:09:00 +0000

We all know how to create multiple domains in Apache. By adding virtual host etries. But how to configure multiple domains in tomcat? We can do this by adding multiple Host tags in server.xml. Its very simple. See the example below. Edit server.xml under conf directory. Suppose you want to setup three domains domain1.com, domain2.com and domain3.com. Every domain points to same ip address in the server.

<Engine defaultHost="arnid" name="Catalina">
<Host name="domain1.com" appBase="/usr/tomcat/apache-tomcat-6.0.26/webapps/domain_root1">
unpackWARs="true" autoDeploy="true"
<Alias>www.domain1.com</Alias>
<Context path="" docBase="."/>
</Host>
<Host name="domain2.com" appBase="/usr/tomcat/apache-tomcat-6.0.26/webapps/domain_root2">
unpackWARs="true" autoDeploy="true"
<Alias>www.domain2.com</Alias>
<Context path="" docBase="."/>
</Host>
<Host name="domain3.com" appBase="/usr/tomcat/apache-tomcat-6.0.26/webapps/domain_root3">
unpackWARs="true" autoDeploy="true"
<Alias>www.domain3.com</Alias>
<Context path="" docBase="."/>
</Host>
<Engine

Now restart tomcat and everything should be working fine.

Creating multiple user login in Amazon ec2

noreply@blogger.com (Randeep) — Sat, 11 Aug 2012 07:40:00 +0000

We know we can login in Amazon ec2 linux instances with our .ppk/.pem keys. But it is restricted to a single root user. How to create more normal users and let them also login to the instance? of course they cant use root users key. so we have to create new login keys for them. This post is also applicable for normal systems. We will see how to create key based authentication for normal users.

We will add a normal user randeep and set password for him.
[root@server ~]# adduser randeep
[root@server ~]# passwd randeep
Changing password for user randeep.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.

Now we will create a ssh key for him and set the permissions.
[root@server ~]# cd /home/randeep/
[root@server randeep]# mkdir .ssh
[root@server randeep]# ls -a
. .. .bash_logout .bash_profile .bashrc .mozilla .ssh
[root@server randeep]# chmod 700 .ssh
[root@server randeep]# ssh-keygen -b 1024 -f randeep -t dsa
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in randeep.
Your public key has been saved in randeep.pub.
The key fingerprint is:
82:ed:75:a3:6e:03:f4:bf:49:e3:39:2f:bb:8f:66:f6 root@server.lap.work
[root@server randeep]# ls -a
. .. .bash_logout .bash_profile .bashrc .mozilla randeep randeep.pub .ssh
[root@server randeep]# cat randeep.pub > .ssh/authorized_keys
[root@server randeep]# chmod 600 .ssh/authorized_keys
[root@server randeep]# chown -R randeep:randeep .ssh

Now checking the login from the command line itself
[root@server randeep]# ssh -i randeep randeep@localhost
The authenticity of host 'localhost (127.0.0.1)' can't be established.
RSA key fingerprint is 63:6d:4a:08:b4:b4:19:3c:d0:58:f3:60:8a:ec:7a:a0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
[randeep@server ~]$

Now you can transfer this key to any windows machine and convert the key to .ppk using puttygen for login through putty of scp.
Install puttygen and load the key.

Loading key for creating .ppk key

create a private key of the loaded key.

Click yes and proceed to create .ppk key

thats it. now you can use the key for login.

if you want to disable password authentication and restrict only to key authentication,
open ssh configuration file /etc/ssh/sshd_config
edit as
PasswordAuthentication no
save it and restart ssh
[root@server randeep]# /etc/init.d/sshd restart
Stopping sshd: [ OK ]
Starting sshd: [ OK ]
[root@server randeep]#

password protected directory in tomcat

noreply@blogger.com (Randeep) — Sat, 11 Aug 2012 06:37:00 +0000

How to protect a web directory with a password? if we are using Apache, we can do it easily with .htaccess. It will prompt user for credentials while entering the directory. But how to protect a directory with password in tomcat web server? In this post we will discuss how to do it with tomcat Realms. This example was tested in tomcat 7 and tomcat 6.

Steps :
1) Add user, password and role in conf/tomcat-users.xml
2) In the webapps/examples/WEB-INF/web.xml specify role, method and urls.
3) Restart Tomcat and check.

Step 1:
in vi conf/tomcat-users.xml

<tomcat-users>
<role rolename="webadmin"/> //webadmin is the rolename of the users who can access the application
<user username="randeep" password="randeep" roles="webadmin"/>
</tomcat-users>

Step 2:

<security-constraint>
<display-name>Example Security Constraint</display-name>
<web-resource-collection>
<web-resource-name>application</web-resource-name>
<url-pattern>/*</url-pattern> //applicable toall urls in the application
<http-method>DELETE</http-method>
<http-method>GET</http-method>
<http-method>POST</http-method>
<http-method>PUT</http-method>
</web-resource-collection>
<auth-constraint>
<role-name>webadmin</role-name>
</auth-constraint>
</security-constraint>

<login-config>
<auth-method>BASIC</auth-method> //Authentication type
<realm-name>application</realm-name>
</login-config>

Step 3:
Restart tomcat
/etc/init.d/tomcat restart
or
bin/shutdown.sh
bin/startup.sh

Now goto browser and check. You can see it prompt for credentials as this.

password protected directory in tomcat

setting nameserver ip addresses on android

noreply@blogger.com (Randeep) — Wed, 04 Jul 2012 17:38:00 +0000

Almost all the android devices will be using dhcp service to get ip address and nameservers. But how to set custom nameservers on android devices? In this article we will see how. Is resolv.conf is there in android? if not? how?

From the command prompt :
run the following command to set the nameservers. We will use google's public nameserver ips 8.8.8.8 and 8.8.4.4 in this example.

#setprop net.dns1 8.8.8.8
#setprop net.dns2 8.8.4.4

NOTE:
in some post you can see like "setprop net.eth0.dns1" which is deprecated now.

You can also make this variable permanent by adding to default.prop file
vi default.prop
net.dns1 8.8.8.8
net.dns2 8.8.4.4

In mobile devices you can set the ips as follows:
Settings > Wireless & network settings > Wi-Fi settings > Advanced > Use static IP

The dhcp uses the following scipt in android devices. you can modify the script to use google's public dns also.
“/system/etc/dhcpcd/dhcpcd-hooks/20-dns.conf”

You can modify the script as
-----------------------------------------------------------------
# Set net.<iface>.dnsN properties that contain the

# DNS server addresses given by the DHCP server.
set_dns_props()
{
    case "${new_domain_name_servers}" in
    "")   return 0;;
    esac
    count=1
    for i in 1 2 3 4; do
        setprop dhcp.${interface}.dns${i} ""
    done    
    count=1
    for dnsaddr in ${new_domain_name_servers}; do
        setprop dhcp.${interface}.dns${count} ${dnsaddr}
        count=$(($count + 1))
    done 
 setprop dhcp.eth0.dns1 8.8.8.8
 setprop dhcp.eth0.dns2 8.8.8.4
}
unset_dns_props()
{
    for i in 1 2 3 4; do
        setprop dhcp.${interface}.dns${i} ""
    done
}
case "${reason}" in
BOUND|INFORM|REBIND|REBOOT|RENEW|TIMEOUT)       set_dns_props;;
EXPIRE|FAIL|IPV4LL|RELEASE|STOP)                unset_dns_props;;
esac

-----------------------------------------------------------------

Creating and dropping indexes in MySQL table

noreply@blogger.com (Randeep) — Sat, 30 Jun 2012 12:30:00 +0000

Benchmarking MySQL with mysqlslap

noreply@blogger.com (Randeep) — Sat, 30 Jun 2012 11:04:00 +0000

I have been checking with different tools for benchmarking a mysql server. I went through a lot of blogs and manuals and decided to use a tool named mysqlslap. In this post we will discuss how to install mysqlslap and use it. We are using Centos Linux 5.4 to test this.

Luckily mysqlslap comes with mysql-client rpm itself. With the versions 5.1.4 and above. So you can either install mysql with yum or rpm. rpms are available on mysql.com for download.

I have tested it with MySQL-client-5.5.25-1.rhel5.i386.rpm.
Just install it as
#rpm -ivh MySQL-client-5.5.25-1.rhel5.i386.rpm
Then you will get the command "mysqlslap"

Testing :
Command :
[root@mysql ~]# mysqlslap -v -h hostname -u root -ppassword --auto-generate-sql
Output :
Benchmark
Average number of seconds to run all queries: 0.002 seconds
Minimum number of seconds to run all queries: 0.002 seconds
Maximum number of seconds to run all queries: 0.002  seconds
Number of clients running queries: 1
Average number of queries per client: 0

The –auto-generate-sql argument tells mysqlslap to automatically generate and execute SQL statements. The results shows that MySQL took 0.002  seconds to execute a single sql statement.
The –auto-generate-sql argument  creates a table, executes an INSERT query and saves dummy data to it, executes a SELECT query to retrieve the dummy data, and then drops the table. You can see behind-the-scenes action by adding the -v option. You can use multiple v's to get more detailed output.
Eg:
[root@mysql ~]# mysqlslap -vvv -h hostname -u root --auto-generate-sql -ppassword

Suppose you can to test the mysql with a 100 concurrent users each executing a single query. It can be done with the following command.
[root@mysql ~]# mysqlslap -vvv -h hostname -u root -ppassword --auto-generate-sql --concurrency=100 --number-of-queries=1
Benchmark
Average number of seconds to run all queries: 0.055 seconds
Minimum number of seconds to run all queries: 0.055 seconds
Maximum number of seconds to run all queries:  0.055 seconds
Number of clients running queries: 100
Average number of queries per client: 1

Using the number of Queries:
If you want to test with 100 concurrent users and 10,000 queries,
[root@mysql ~]# mysqlslap -vvv -h hostname -u root -ppassword --auto-generate-sql --concurrency=100 --number-of-queries=10000

Benchmark
Average number of seconds to run all queries: 0.450 seconds
Minimum number of seconds to run all queries: 0.450  seconds
Maximum number of seconds to run all queries: 0.450   seconds
Number of clients running queries: 100
Average number of queries per client: 100

Using iterations:
You can tell mysqlslap to repeat the query more than once using the iterations argument.
[root@mysql ~]# mysqlslap -vvv -h hostname -u root -ppassword --auto-generate-sql --concurrency=100 --number-of-queries=10000 --iterations=5

Using custom queries.
By default mysqlslap will use its own schema and queries to test. Suppose you want to test a mysql server where exists a database and schema, you can then specify the database and custom query in the mysqlslap command as,

[root@mysql ~]# mysqlslap -vv -h hostname -u root -ppassword  --create-schema=DATABASE_NAME --query="SELECT * from table_name;"  --concurrency=100  --number-of-queries=100

Comparing the Engines:
You know the performance varies when using different engines. You can compare the performance by engine using the argument --engine.

Using the engine innodb
[root@mysql ~]# mysqlslap -vvv -h hostname -u root  -ppassword --auto-generate-sql --concurrency=100 --number-of-queries=1000 --engine=innodb

Using the engine myisam
[root@mysql ~]# mysqlslap -vvv -h hostname -u root  -ppassword --auto-generate-sql --concurrency=100 --number-of-queries=1000 --engine=myisam

Thats it. Try it and comment your experiences.

importing csv file to MySQL database table

noreply@blogger.com (Randeep) — Thu, 28 Jun 2012 15:05:00 +0000

Sometimes we need to get the MySQL tables into an Excel sheet. How this can be done? We can dump the MySQL tables into a csv format file. It is explained in our previous post Dumping MySQL table into CSV file. But how to import a csv file to a mysql database table? We will discuss it in this post.

Login to your MySQL server.
[root@database ~]# mysql -p
Enter password:

Use the desired database
mysql> use database_name

For importing the csv file "viewership.csv" from the directory "/root/databases" into the table "table_name"

mysql> load data local infile '/root/databases/viewership.csv' into table table_name fields terminated by ',' enclosed by '"' lines terminated by '\n';

Thats it. Now the csv file is loaded into the table.

Things to consider in video streaming

noreply@blogger.com (Randeep) — Thu, 07 Jun 2012 18:22:00 +0000

Video streaming is gaining momentum. Nowadays nobody wants to download videos and watch. Why should we download videos if we can watch them online? But nobody can enjoy a video if it's quality is bad or if it periodically pauses for buffering etc. So every system administrators must be aware of some aspects while streaming videos. In this post we will discuss main points in streaming videos. The main aspects of video streaming are resolution, encoding, frame rate, bit rate, data rate, aspect ratio and data transfer rate.

First of all lets see some video definitions :
Standard Definition :
These are the common standard definition resolutions.
480p 720x480 (Normal)
576p 720x576 (PAL)

High Definition :
720p 1280x720

Full HD :
1080p 1920x1080

Before streaming a raw video we have to encode it. Here is my suggestions for encoding a video.
Video format :
MPEG-4 (.mp4)

Video encoding:
H.264

Audio encoding:
AAC LC

Protocol :
RTMP, or RTSP

Resolution:
Before encoding we have to decide some parameters for encoding the video. Of course, the first one is resolution of the video, here we have a few options
1080p
720p
480p
360p
YouTube gives 4 different resolutions which users can select manually. 1080p, 720p, 480p and 360p (Full screen not supported).

Frame rate :
Frame rate is one of the important parameter affecting the quality of the picture. Some commonly used are 25fps(frame per second), 29.97 and 30. I recomment 30 fps.

Bit rate / Data rate
At which rate the data should be encoded. We will discuss this later in this post.

Adaptive or By user selection.
Adaptive bit rate streaming is one of the latest tecchnologies in video streaming. In this video will be encoded in many bitrates and the customer will be able to select the most suitable bitrate video depending upon his network capabilities. This can be done either automatically or manually. YouTube gives option for users to select the quality they want. Normally 1080p, 720p, 480p, 360p and 240p

Audio
Sample rate 96khz or 48khz
Channels: Stereo or Stereo + 5.1

Bits Per Pixel:
Bits per pixel is one of the most important thing affecting the picture quality. It can be calculated using the following formulae.

bits per pixel = (bit rate)/(height x width x frame rate)
For the low motion, talking head videos the BPP range of .1 to .15 will be fine. But for the high motion videos like car race of football match .15 to .20 is required. But in my experience it has been see that if the source file is good, converted files with BPP with range even 0.7 - 0.15 does well.

For example a video encoded with 5000kbps bit rate with 1280*720 resolution @30fps
will have a bit/pixel rate of
(5000*1000)/(1280*720*30) = 0.18

Aspect ratio :
Old telivisions were using the 4:3 aspect ration. But almost all the televisions and monitors available now is using 16:9. So the later one is preferable.

Data transfer size :
Data transfer size is the most important one. It is the data trasnferred between server and clients pc while watching a video. It can be calculated using the fowllowing formulae.

Storage size (in megabytes) = length (in seconds) × bit rate (in bit/s) / (8 × 1024 × 1024)

Lets see how much data will be trasferred or bandwidth usage for watching a video for one hours.
Playing video with bit rate 8,000 kbps
3433Mb = (3600*8000*1000)/(8*1024*1024) 3.4 G/Hour

Playing video with bit rate 5,000 kbps
2145Mb = (3600*5000*1000)/(8*1024*1024) 2.2 G/Hour

Playing video with bit rate 2,500 kbps
1072Mb = (3600*2500*1000)/(8*1024*1024) 1.0 G/Hour

Playing video with bit rate 2,000 kbps
1072Mb = (3600*2000*1000)/(8*1024*1024) 860Mb/Hour

The data transfer will be same irrespective of frame(XxY) size. For different frame sizes with same bitrate, quality will differ. (Big frames will have less quality with same bitrate).

Thats all for now. Kindly share your experiences, opinions and views. For streaming videos using Amazon Cloudfront, you can read my guest post on renjusblog at streaming videos using clodfront

Dumping MySQL table into CSV file

noreply@blogger.com (Randeep) — Thu, 31 May 2012 12:16:00 +0000

Sometimes we need to get the MySQL tables into an Excel sheet. How this can be done? We can dump the MySQL tables into a csv format file. Lets see how to do it.

Login to your MySQL server.

[root@database ~]# mysql -p
Enter password:

Use the desired database
mysql> use database_name

Now for dumping the data of the table named "table_name" execute the command, the csv file will be save in /tmp. You can specify the destination in the command.

mysql> SELECT * INTO OUTFILE '/tmp/table_name.csv' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n' FROM table_name;
Query OK, 5000 rows affected (0.09 sec)
mysql>

If you need only first 250 RAWs, you can specify the limit as follows.

mysql> SELECT * INTO OUTFILE '/tmp/table_name.csv' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n' FROM table_name LIMIT 250;
Query OK, 250 rows affected (0.09 sec)
mysql>

Thats it. You can get the csv file to your windows system and can open with Microsoft Excel.

Confiugring daily backup of amazon centos instance

noreply@blogger.com (Randeep) — Mon, 21 May 2012 14:24:00 +0000

Configuring daily backup of amazon linux centos instance to amazon s3 storage

How do you backup your amazon ec2 (Amazon elastic cloud) Linux instance? Did you buy any costly backup sofwares to do the daily backup task? Here we will discuss how to backup your amazon ec2 Linux instance to amazon s3 bucket storage using a bash script scheduled with cron on a daily basis.

Requirements :
Amazon ec2 (Amazon elastic cloud) Linux instance.
Amazon s3 storage bucket for storing the backups.
s3cmd command line tools

Before we go to the code we will discuss what our daily backup script does.
Script takes the backup of following directories and databases
1) /usr/share/tomcat5/webapps/company
Databases
1) company
All the log files will be save in /scripts/logs directory

------------------------------------------------

#!/bin/bash
### Creating the daily backup directory
mkdir /scripts/server_name-`date +%F`
cd /scripts/server_name-`date +%F`
### Taking the backup of directories
tar -cpvzf company.tar.gz /usr/share/tomcat5/webapps/company 2>> /scripts/logs/tar.log 1> /dev/null
if [ $? == 0 ]
then
echo -ne "company.tar.gz is created\n" >> /scripts/logs/production_daily`date +%F`.log
else
echo -ne "ALERT: company.tar.gz creation has FAILED\n" >> /scripts/logs/production_daily`date +%F`.log
fi
### Uploading the directory backups to s3 bucket "bucket_name"
s3cmd put company.tar.gz s3://bucket_name/`date +%F`/company.tar.gz 2>> /scripts/logs/s3cmd.log 1> /dev/null
if [ $? == 0 ]
then
echo -ne "company.tar.gz is uploaded to amazon s3\n" >> /scripts/logs/production_daily`date +%F`.log
else
echo -ne "ALERT: company.tar.gz uploading to amazon s3 has FAILED\n" >> /scripts/logs/production_daily`date +%F`.log
fi
### Taking backup of databases
mysqldump company > company.`date +%F`.sql 2>> /scripts/logs/mysql.log
if [ $? == 0 ]
then
echo -ne "company.`date +%F`.sql is created\n" >> /scripts/logs/production_daily`date +%F`.log
else
echo -ne "ALERT: company.`date +%F`.sql creation has FAILED\n" >> /scripts/logs/production_daily`date +%F`.log
fi
### Uploading the database backups to s3 bucket bucket_name
s3cmd put company.`date +%F`.sql s3://bucket_name/`date +%F`/company.`date +%F`.sql 2>> /scripts/logs/s3cmd.log 1> /dev/null
if [ $? == 0 ]
then
echo -ne "company.`date +%F`.sql is uploaded to amazon s3\n" >> /scripts/logs/production_daily`date +%F`.log
else
echo -ne "ALERT: company.`date +%F`.sql uploading to amazon s3 has FAILED\n" >> /scripts/logs/production_daily`date +%F`.log
fi
### Mailing the status
mail -s "Daily backup report on `date +%F` of Production server" admin@domain.com < /scripts/logs/production_daily`date +%F`.log
### Clearing the logs and temporary data
cd ~
rm -rf /scripts/server_name-production-`date +%F`

------------------------------------------------
Give the script executable permissions:
#chmod u+x ec2_backup_script.sh

Install the crontab entry :
#crontab -e
30 15 * * * sh /scripts/ec2_backup_script.sh

Thats it.

Configuring daily backup of amazon RDS server

noreply@blogger.com (Randeep) — Mon, 21 May 2012 14:00:00 +0000

Configuring daily backup of amazon RDS server to amazon s3 storage

How do you backup your amazon RDS (Amazon Relational Database Service) MySQL server? Did you buy any costly backup managers to do the daily backup task? Here we will discuss how to backup your amazon rds MySQL databases to amazon s3 bucket storage using a bash script scheduled with cron on a daily basis.

Requirements :
Amazon RDS (Amazon Relational Database Service) MySQL instance.
Amazon s3 storage bucket for storing the backups.
Amazon Elastic Cloud (ec2) Linux (centos) instance.
s3cmd command line tools

Before we go to the code we will discuss what our daily backup script does.
1) First it creates a file containing list of existing databases
2) Take the dump each and every databases in the file
3) Creates an compressed archive (tar.gz) of all the database dumps
4) Uploads the archive to the s3 bucket.
5) Deletes the 8 days old backup from the s3 bucket. (Why do we need backups older than one week. Remember we have to pay for the storage.)
6) Sends the admins email notification regarding the status/errors etc.

The script is as follows :
You have to save the script in a file(.sh), set proper executable permissions and install a crontab entry.

---------------------------------------------------

#!/bin/bash
### Creating the file list_of_databases.txt which contains the list of databases.
mysql -u root -pPASSWORD -h RDS_Public_DNS_URL -e "show databases" | awk '{print $1}' > /scripts/logs/list_of_databases.txt
sed -i '1d' /scripts/logs/list_of_databases.txt
mkdir /scripts/RDS_backups-`date +%F`
### Taking backup ###
for i in `cat /scripts/logs/list_of_databases.txt`
do
mysqldump -u root -h RDS_Public_DNS_URL -pPASSWORD --lock-tables=false $i > /scripts/RDS_backups-`date +%F`\/$i.sql 2>> /scripts/logs/rds_backup_log.`date +%F`.txt
if [ $? == 0 ]
then
echo -ne "Backup of $i of RDS server has been completed\n" >> /scripts/logs/rds_backup_log.`date +%F`.txt
else
echo -ne "ALERT: Backup of $i of RDS server has FAILED\n" >> /scripts/logs/rds_backup_log.`date +%F`.txt
fi
done
### command to archive and upload to s3 bucket ###
tar cvzf /scripts/RDS_backups-`date +%F`.tar.gz /scripts/RDS_backups-`date +%F`/ &> /scripts/logs/tar.log
s3cmd put /scripts/RDS_backups-`date +%F`.tar.gz s3://s3_bucket_name/`date +%F`/RDS_backups-`date +%F`.tar.gz &> /scripts/logs/s3cmd.log
if [ $? == 0 ]
then
echo -ne "RDS_backups-`date +%F`.tar.gz has been uploaded to s3_bucket_name\n" >> /scripts/logs/rds_backup_log.`date +%F`.txt
else
echo -ne "Uploading RDS_backups-`date +%F`.tar.gz to s3_bucket_name has FAILED\n" >> /scripts/logs/rds_backup_log.`date +%F`.txt
fi
### Deleting the logs that are older than one week
jan=31
feb=28
march=31
april=30
may=31
june=30
july=31
aug=31
sep=30
oct=31
nov=30
dec=31
yearmonth=`date +%F | cut -f1,2 -d "-"`
current_date=`date +%F | cut -f3 -d "-"`
current_month=`date +%F | cut -f2 -d "-"`
old_month=`echo "$current_month - 01" | bc`
year=`date +%F | cut -f1 -d "-"`
if [ $current_date -lt 9 ]
then
diff=`echo "8 - $current_date" | bc`
case $old_month in
1)
x=`echo "$jan - $diff" | bc`
old_date=$year-0$old_month-$x ;;
2)
x=`echo "$feb - $diff" | bc`
old_date=$year-0$old_month-$x ;;
3)
x=`echo "$march - $diff" | bc`
old_date=$year-0$old_month-$x ;;
4)
x=`echo "$april - $diff" | bc`
old_date=$year-0$old_month-$x ;;
5)
x=`echo "$may - $diff" | bc`
old_date=$year-0$old_month-$x ;;
6)
x=`echo "$june - $diff" | bc`
old_date=$year-0$old_month-$x ;;
7)
x=`echo "$july - $diff" | bc`
old_date=$year-0$old_month-$x ;;
8)
x=`echo "$aug - $diff" | bc`
old_date=$year-0$old_month-$x ;;
9)
x=`echo "$sep - $diff" | bc`
old_date=$year-0$old_month-$x ;;
10)
x=`echo "$oct - $diff" | bc`
old_date=$year-$old_month-$x ;;
11)
x=`echo "$nov - $diff" | bc`
old_date=$year-$old_month-$x ;;
12)
x=`echo "$dec - $diff" | bc`
old_date=$year-$old_month-$x ;;
esac
else
x=`echo "$current_date - 8" | bc`
### suppose the date is 15, 15 -8 will become 7 but we need 07 so adding a 0 in the front
if [ $x -lt 10 ]
then
x=0$x
fi
old_date=$yearmonth-$x
fi
s3cmd del --recursive s3://s3_bucket_name/$old_date 2>> /scripts/logs/s3cmd.log 1> /dev/null
if [ $? == 0 ]
then
echo -ne "Backup of the day $old_date is deleted\n" >> /scripts/logs/rds_backup_log.`date +%F`.txt
else
echo -ne "ALERT: Backup of the day $old_date does not exists\n" >> /scripts/logs/rds_backup_log.`date +%F`.txt
fi
### Notifying the admin ###
mail -s "Daily backup report `date +%F` of RDS server" admin@domain.com < /scripts/logs/rds_backup_log.`date +%F`.txt
### Cleare temporary logs and data ###
> /scripts/logs/list_of_databases.txt
rm -rf /scripts/RDS_backups-`date +%F`
rm -rf /scripts/RDS_backups-`date +%F`.tar.gz

---------------------------------------------------
Give the script executable permissions:
#chmod u+x filename.sh

Install the crontab entry :
#crontab -e
30 15 * * * sh /scripts/RDS_backup_script.sh

Thats it.

Amazon Elastic Load balancing with autoscaling

noreply@blogger.com (Randeep) — Thu, 17 May 2012 18:50:00 +0000

Suppose you have a website. linxhelp.in And suppose sometimes you get a traffic which your system cannot handle. What you will do? You will create multiple instances of your system and will configure a load balancer. But you have to keep both instances running even if there is not much traffic. We will discuss how to eliminate this resource wastage using amazon elastic load balancer and amazon autoscaling. For using this, your instances should be configured in amazon elastic cloud or ec2.

In this example we will use the domain linuxhelp.in and our test instances are in centos linux. For doing this you will need amazon web services account.

Objective :
Create new instance based on the AMI created beforehand when CPU load increases beyond 80% and distribute the load between the old and newly created instances.
Terminate the instance when the CPU load falls below 40% thus making only one instance running while the load is less.

Requirements :
Amazon web services account
Amazon elastic cloud EC2
Amazon Elastic Load Balancer
Amazon Detailed Cloudwatch
Amazon AutoScaling
Amazon Cloudwatch command line tools
Amazon AutoScaling command line tools
Amazon ELB command line tools
DNS configuration access

Steps :
ELB:
Create an Amazon Elastic Load balancer
Configure health Check

AutoScaling:
Create a Launch configuration
Create a Auto Scaling Group

Cloudwatch:
Create Policy to Scale up the instances.
Create Alarm on Average CPUUtilization with threshold > 80 which triggers scale up policy.
Create Policy to Scale down the instances.
Create Alarm on Average CPUUtilization with threshold < 40 which triggers scale down policy.

DNS:
Point the domain to ELB

Now we will see every steps in detail:
First we need to create an amazon Elastic load balancer. We can do this using the following command. In this we need to specify the port, protocol and the availability zones.
elb-create-lb MyELB --headers --listener "lb-port=80,instance-port=80,protocol=http" --availability-zones us-east-1a,us-east-1d

Then we need to specify the health check details to determine which instance is healthy or not.
elb-configure-healthcheck MyELB –target “HTTP:80/” –interval 60 –timeout 5 –unhealthy-threshold 3 –healthy-threshold 5

We can check the status of the elastic load balancer as
[root@Server ~]# elb-describe-lbs MyELB -C cert-id.pem -K pk-id.pem
LOAD_BALANCER MyELB MyELB-2114043205.us-east-1.elb.amazonaws.com 2012-05-12T05:36:37.180Z

Autoscaling:
Creating the launch configuration:
Launch configuration guides aws to which AMI to be used for spawning a new instance, instance type, Security group etc.

[root@Server ~]# as-create-launch-config MyLaunchC --image-id ami-12345 --instance-type m1.small --group Basics --key cdnkey -C cert-id.pem -K pk-id.pem
OK-Created launch config
[root@Server ~]#

Creating autoscaling group:
We have to create a AutoScaling group for the instances to run. In this we can specify minimum and maximum instances, availability zones, and ELB name to associate with.

[root@Server ~]# as-create-auto-scaling-group MyScalingGroup --launch-configuration MyLaunchC --availability-zones us-east-1a,us-east-1d --min-size 1 --max-size 2 --load-balancers MyELB -C cert-id.pem -K pk-id.pem
OK-Created AutoScalingGroup
[root@Server ~]#

Describing the autoscaling group status :
We can check the status of the autoscaling groups with following command. To check how many instances are running, their health etc.
[root@Server ~]# as-describe-auto-scaling-groups --headers -C cert-id.pem -K pk-id.pem
AUTO-SCALING-GROUP GROUP-NAME LAUNCH-CONFIG AVAILABILITY-ZONES LOAD-BALANCERS MIN-SIZE MAX-SIZE DESIRED-CAPACITY
AUTO-SCALING-GROUP MyScalingGroup MyLaunchC us-east-1a,us-east-1d MyELB 1 2 1
INSTANCE INSTANCE-ID AVAILABILITY-ZONE STATE STATUS LAUNCH-CONFIG
INSTANCE i-8df54deb us-east-1d InService Healthy MyLaunchC
[root@Server ~]#

Creating the scale up policy :
In this we set a policy / action to be executed to increase the number of instances running(Response to the alaram when cpu utilization > 80)
By executing the command we get a policy arn which is to be noted.
[root@Server ~]# as-put-scaling-policy MyScaleUpPolicy --auto-scaling-group MyScalingGroup --adjustment=1 --type ChangeInCapacity --cooldown 300 -C cert-id.pem -K pk-id.pem
arn:aws:autoscaling:us-east-1:MyAwsID:scalingPolicy:arnid:autoScalingGroupName/MyScalingGroup:policyName/MyScaleUpPolicy
[root@Server ~]#

Creating the high CPU Utilization alarm :
Here we create an alarm when the average cpu utilization of the instance increase more than 80%. In the command we have to specify the policy arn (Policy to increase the number of instance by 1) to be called when the alarm occurs.
[root@Server ~]# mon-put-metric-alarm MyHighCPUAlarm --comparison-operator GreaterThanThreshold --evaluation-periods 3 --metric-name CPUUtilization --namespace "AWS/EC2" --period 60 --statistic Average --threshold 80 --alarm-actions arn:aws:autoscaling:us-east-1:MyAwsID:scalingPolicy:arnid:autoScalingGroupName/MyScalingGroup:policyName/MyScaleUpPolicy --dimensions "AutoScalingGroupName=MyScalingGroup" -C cert-id.pem -K pk-id.pem
OK-Created Alarm
[root@Server ~]#

Creating the scale down policy :
In this we set a policy / action to be executed to decrease the number of instances running ( Response to the alaram when cpu utilization < 40%).By executing the command we get a policy arn which is to be noted.
[root@Server ~]# as-put-scaling-policy MyScaleDownPolicy --auto-scaling-group MyScalingGroup --adjustment=-1 --type ChangeInCapacity --cooldown 300 -C cert-id.pem -K pk-id.pem
arn:aws:autoscaling:us-east-1:MyAwsID:scalingPolicy:arnid:autoScalingGroupName/MyScalingGroup:policyName/MyScaleDownPolicy
[root@Server ~]#

Creating the low CPU Utilization alarm :
Here we create an alarm when the average cpu utilization of the instance falls below 40%. In the command we have to specify the policy arn (Policy to decrease the number of instance by 1) to be called when the alarm occurs.
[root@Server ~]# mon-put-metric-alarm MyLowCPUAlarm --comparison-operator LessThanThreshold --evaluation-periods 3 --metric-name CPUUtilization --namespace "AWS/EC2" --period 60 --statistic Average --threshold 40 --alarm-actions arn:aws:autoscaling:us-east-1:MyAwsID:scalingPolicy:arnid:autoScalingGroupName/MyScalingGroup:policyName/MyScaleDownPolicy --dimensions "AutoScalingGroupName=MyScalingGroup" -C cert-id.pem -K pk-id.pem
OK-Created Alarm
[root@Server ~]#

Testing:
First we will increase the cpu load and check whether new instance is created or not when the cpu utilization becomes more than 80%.
For increasing the CPU Load:
Run the following command from multiple terminals:
#echo "9999999^999999" | bc

Check the CPU usage using the following command:
#top -c

Check whether new instances are spawned using the following command or from web intercface of aws.
#as-describe-auto-scaling-groups MyScalingGroup --headers -C cert-KEY.pem -K pk-KEY.pem
Now kill all the processes started for increasing the load and check whether one of the instance is terminated.
Point the domain name to the Public DNS of Elastic Load Balancer and check whether the site is loading from the browser.
We have to create CNAME record in Godaddy.com (Or your registrar) account for this. For example,

CNAME:
create a cname record for "elb" points to "MyELB-2114043205.us-east-1.elb.amazonaws.com"

Please login as the ec2-user user

noreply@blogger.com (Randeep) — Mon, 30 Apr 2012 12:13:00 +0000

When login to yous amazon ec2 instance via putty or command line it is possible that you get this error.

Authenticating with public key "imported-openssh-key"
Please login as the ec2-user user rather than root user.

Then login as user "ec2-user" instead of root user and if you want to run commands as root user, use sudo command or switch the user to root

eg:
#sudo vi /etc/dhcpd.conf

or
#sudo su -
Then you'll become root user.

installing s3cmd in ubuntu

noreply@blogger.com (Randeep) — Fri, 27 Apr 2012 08:11:00 +0000

s3cmd is a command line tool for uploading, downloading and managing file and directories with amazon simple storage s3. It is very useful when running scripts and scheduling scripts with cron. First you have to install s3cmd package which is available from s3tools.org. In this post we discuss how to install and configure s3cmd in ubuntu or debian systems.

Import S3tools signing key:
wget -O- -q http://s3tools.org/repo/deb-all/stable/s3tools.key | sudo apt-key add -

Add the repo to sources.list:
sudo wget -O /etc/apt/sources.list.d/s3tools.list http://s3tools.org/repo/deb-all/stable/s3tools.list

Refresh package cache and install the newest s3cmd:
sudo apt-get update && sudo apt-get install s3cmd

Now configure it. It will ask your access key, secret key and encryption key(just hit enter if you don't want).
s3cmd --configure

Now you will be able to list your buckets in your amazon s3 storage using the following command.
s3cmd ls

See this post for installing s3cmd in centos and s3cmd examples also.

sms notification using nagios

noreply@blogger.com (Randeep) — Thu, 26 Apr 2012 19:17:00 +0000

We have discussed how to install and configure nagios monitoring system and configuring nrpe with nagios in previous posts. But it sends only email notifications. What if we are not able to access mail? Internet is not working? Or forgot to check the mails. So it is always better to have a better notification system. Notifications via SMS is an alternative. We can know if a service or host is down by sms even if dont check mail. But how to enable notifications via sms? What are the requirements? Which files needs to modified? How to configure sms gateway? We discuss all these things in this post.

Requirements :
To set up sms alerts notification with nagios we need a few things.
1. Existing Nagios implementation: -We need a working Nagios implementation with email alerts are already
configured.
2. SMS gateway account and API details: For this testing purpose we have used sms gateway provided by
clickatell.com. We have used their HTTP API for this. We need following credentials to use this.

● Username of clickatell.com account
● Client ID
● API ID
● Password
● Mobile numbers

Configuration :
We have to edit mainly two configuration files and include the fields required for configuring alerts via sms.

1. Commands.cfg
We have to add the commands for
● Host notification via sms
● Service notification via sms
Only via mail will be defined by default.

2. Contacts.cfg
We have to add the commands defined in commands.cfg for notifications via sms in
● Service notification commands
● Host notification commands
● Mobile numbers
● Notification periods
● Notification options

Commands.cfg
Add the following lines to the commands.cfg file in order to send sms notifactions via HTTP API of
clickatell.com. The fields USERNAME,PASSWORD and API_ID needs to be replaced with
original ones.
# host-notify-by-sms
define command{
command_name host-notify-by-sms
command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type:
$NOTIFICATIONTYPE$\nHost: $HOSTNAME$ \nState: $HOSTSTATE$\nAddress:
$HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | wget "http:
//api.clickatell.com/http/sendmsg?
user=USERNAME&password=PASSWORD&api_id=API_ID&to=$CONTACTPAGER$&text='**
$NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ @ $LONGDATETIME$ **'"
}

# service-notify-by-sms
define command{
command_name service-notify-by-sms
command_line /usr/bin/printf "%b" "***** Nagios *****\n\nNotification Type:
$NOTIFICATIONTYPE$\nHost: $HOSTNAME$ \nState: $HOSTSTATE$\nAddress:
$HOSTADDRESS$\nInfo: $HOSTOUTPUT$\n\nDate/Time: $LONGDATETIME$\n" | wget "http:
//api.clickatell.com/http/sendmsg?
user=USERNAME&password=PASSWORD&api_id=API_ID&to=$CONTACTPAGER$&text='**
$NOTIFICATIONTYPE$ Service Alert: $SERVICEDESC$ on $HOSTNAME$ State:
$SERVICESTATE$ @ $LONGDATETIME$ **'"
}

Contacts.cfg
The following lines need to be added in contacts.cfg
service_notification_options w,u,c,r,f,s #statuses –warning/critical/fapping/restarted etc.
host_notification_options d,u,r,f,s
service_notification_commands notify-service-by-email service-notify-by-sms #mail and sms
host_notification_commands notify-host-by-email host-notify-by-sms
pager 91xxxxxxxxxx #mobile number
service_notification_period 24x7 #Notifications will be sent 24x7
host_notification_period 24x7

IF you want to get notified to multiple numbers, define multiple contacts.

Testing :
We can test the Gateway using a simple command from the command line
#Wget http://api.clickatell.com/http/sendmsg?
user=USERNAME&password=PASSWORD&api_id=API_ID53&to=MOBILE_NUMBER&t
ext=Message

We can test the alerts working by stopping a service such as mysql and checking the
mobile for the sms. If the service is working properly we will get the following lines in Nagios log file.

#tail -f /usr/local/nagios/var/nagios.log
[1335345427] SERVICE ALERT: nagios-server;MYSQL;CRITICAL;SOFT;3;Connection
refused
[1335345487] SERVICE ALERT: nagios-server;MYSQL;CRITICAL;HARD;4;Connection
refused
[1335345487] SERVICE NOTIFICATION: nagiosadmin;nagios-server;MYSQL;CRITICAL;
service-notify-by-sms;Connection refused
[1335345488] SERVICE NOTIFICATION: nagiosadmin;nagios-server;MYSQL;CRITICAL;
notify-service-by-email;Connection refused

We will get the sms’ with original content only after we buy credit. Else we will get a test
success notification from clickatell.com only.

checking cpu architecture in linux

noreply@blogger.com (Randeep) — Thu, 26 Apr 2012 12:50:00 +0000

You may have to check which is the architecture of your linux system 32 bit or 64 bit. Here there is one thing needs to keep in mind. Whether you are checking the version of installed os kernel or the architecture of underlying cpu. It is possible that Operating system is 32 but cpu has 64 bit support. This post explains how to find or check the architecture of the Linux Operating system and cpu.

The following commands will only show version of installed Linux kernel (OS).

#uname -a
#arch

Examples:

[root@test1 ~]# uname -a
Linux test1.lukup.internal 2.6.18-92.el5 #1 SMP Tue Jun 10 18:49:47 EDT 2008 i686 i686 i386 GNU/Linux
[root@test1 ~]# arch
i686
[root@test1 ~]#

i386/i586/i686 all means its a 32 bit os.
x86_64 means it is a 64 bit os.

If you want to know the architecture of the cpu. You have to check the cpu specifications. See the following commands

The following command will give all the information about cpu
cat /proc/cpuinfo

Example:
[root@test1 ~]# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Pentium(R) Dual-Core CPU E5200 @ 2.50GHz
stepping : 10
cpu MHz : 2499.684
cache size : 2048 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni
bogomips : 5004.21

[root@test1 ~]#

To check your system is 32 bit or 64 bit, check the flags field.
#grep flags /proc/cpuinfo

IM or im flag means Long mode cpu - 64 bit CPU
Real mode 16 bit CPU
Protected Mode is 32-bit CPU

In Debian Ubuntu systems you can use the following commands
#lscpu
#cat /proc/cpuinfo |grep ” lm”

Dumping mysql database schema only

noreply@blogger.com (Randeep) — Mon, 23 Apr 2012 09:13:00 +0000

If you want to dump only schema of a database,

Execute the following command.
#mysqldump -u root -pPASSWORD -d -h Host_name_or_Ip_Address database_name > database_name.sql
It will dump the schema to the file database_name.sql

If you just want it on the console only, then
#mysqldump -u root -pPASSWORD -d -h Host_name_or_Ip_Address database_name

replicating an amazon instance to different zones or regions

noreply@blogger.com (Randeep) — Wed, 18 Apr 2012 17:33:00 +0000

We have to replicate an amazon ec2 instance running in one zone or region to other zones for load balancing and high availability. Or you may want to migrate your instance to other region for low latency. We can do this by ec2-migrate-bundle command. First we have to create an image. See this post for creating image of amazon linux instance. Then we have to we create a s3 bucket in the destination zone and migrate the image to the destination bucket. Then we can register ami based on that and create instances.

Steps:
1) Determining the zone to create the new instance.
2) Creating Amazon s3 bucket in the new zone.
3) Migrating the bundle
4) Registering the AMI
5) Creating new instance based on the AMI
6) Testing the AMI

Requirements:
We need the following files and keys for during the replication.
Private Key File: pk-PRIVATEKEY.pem
X.509 Certificate File: cert-X509CERT.pem
ACCESS-KEY-ID (S3)
SECRET-ACCESS-KEY (S3)
Existing instance stored AMI
Security Group and key pair in destination zone.

Determining the zone :
First we have to determine Amazon zone we are going to use. There are a few zones. We can get the zone details using the command ec2-describe-regions.

Example:
[root@hostname ~]# ec2-describe-regions -K pk-PRIVATEKEY.pem -C cert-X509CERT.pem
REGION eu-west-1 ec2.eu-west-1.amazonaws.com
REGION sa-east-1 ec2.sa-east-1.amazonaws.com
REGION us-east-1 ec2.us-east-1.amazonaws.com
REGION ap-northeast-1 ec2.ap-northeast-1.amazonaws.com
REGION us-west-2 ec2.us-west-2.amazonaws.com
REGION us-west-1 ec2.us-west-1.amazonaws.com
REGION ap-southeast-1 ec2.ap-southeast-1.amazonaws.com
[root@hostname ~]#

Creating Amazon s3 bucket in the required zone :
Once the zone is decided we have to get the AMI of the instance available in the zone. For that we have to,
1) Create an amazon s3 bucket in the region
2) Migrate the bundle to the newly created bucket
3) Register the AMI

Migrating the bundle :
While migrating the bundle we need to understand a few things. Same kernel and ramdisk may not be available in the destination zone. You may have to either specify them explicitly or let the Amazon decide which is best.

Syntax:
ec2-migrate-bundle -k pk-key.pem -c cert-cert.pem -a Access-Key-Id -s Secret-Access-Key --bucket source_bucket --destination-bucket destination_bucket --manifest NAME.ami.manifest.xml --region ZONECode
ZONE-Code – is important because based on this ec2-migrate tool will decide the best kernel and ramdisk in the destination zone.

Registering the AMI :
We have to register the AMI with Amazon. Then only we can create instances based on that AMI.
Syntax:
ec2-register -K pk-PRIVATEKEY.pem -C cert-X509CERT.pem -n name_of_the_AMI Full_path_of_manifest.xml_in_s3_bucket --region zone-id
Example:
[root@hostname_of_instace image]# ec2-register -K pk-ABCDEFGHIJKLMN.pem -C cert-ABCDEFGHIJKLMN.pem -n nagios bucket_name/images/nagios.ami.manifest.xml --region us-west-2
IMAGE ami-wwwwww
[root@hostname_of_instace image]#

It will give the AMI ID as the output. In this case,
IMAGE ami-wwwwww
--region zone-id is important. Else it will create the instance in default zone (virginia)

Creating new instance based on the AMI
Syntax:
ec2-run-instances -K pk-ABCDEFGHIJKLMN.pem -C cert-ABCDEFGHIJKLMN.pem -g Basics -k cdnkey ami-ID --region us-west-2
-g is for the Security group. We have to specify which security group we are using.
-k is for key type. We have to specify which key type we are using.
We have to have the key pair and security group prior to instance launch.
Last field is the AMI-ID based on the instance will be created.

Testing the AMI:
After launching the new instance we must check a few things:
Check the following things are same for original and new instances:
Disk usage
Services running
Accessibility of services such as ssh, http, tomcat and mysql
Ensuring mysql database is up-to-date.
All the important data are present.

Linux Help

TCP trace route in windows using tracetcp

Amazon AMI Copy example with snapshots

configuring tftp server in Ubuntu

Recommended Reading

benchmarking with http_load

Recommended Reading

No space left on device: mod_jk: could not create jk_log_lock

Recommended Reading

Installing SSL certificate on Apache with Tomcat

Recommended Reading

url redirection in apache using proxypass

Recommended Reading

Configuring multiple domains or sub-domains in tomcat

Recommended Reading

Creating multiple user login in Amazon ec2

Recommended Reading

password protected directory in tomcat

Recommended Reading

setting nameserver ip addresses on android

Recommended Reading

Creating and dropping indexes in MySQL table

Recommended Reading

Benchmarking MySQL with mysqlslap

Recommended Reading

importing csv file to MySQL database table

Recommended Reading

Things to consider in video streaming

Recommended Reading

Dumping MySQL table into CSV file

Recommended Reading

Confiugring daily backup of amazon centos instance

Recommended Reading

Configuring daily backup of amazon RDS server

Recommended Reading

Amazon Elastic Load balancing with autoscaling

Recommended Reading

Please login as the ec2-user user

installing s3cmd in ubuntu

sms notification using nagios

checking cpu architecture in linux

Dumping mysql database schema only

Recommended Reading

replicating an amazon instance to different zones or regions

Recommended Reading