I'm beginning to think Linux Really!

Read files line by line using simple script.

2014-01-12T07:17:00.000-08:00

2. Read files line by line using simple script.

[saikrishna.b@gai-1339 Documents]$ cat testfile
This is a sample script.
To test
the file is read line by line.
Hope this works.
Lets c.
Done

[saikrishna.b@gai-1339 Documents]$ cat line.sh
#!/bin/bash
filename=$1
count=0
cat $filename | while read a
do
let count++
echo " $count $a "
done

[saikrishna.b@gai-1339 Documents]$ sh line.sh testfile
1 This is a sample script.
2 To test
3 the file is read line by line.
4 Hope this works.
5 Lets c.
6 Done

[saikrishna.b@gai-1339 Documents]$ cat line.sh
#!/bin/bash
filename=$1
count=0
cat $filename | while read a
do
let count++
echo " $count $a "
done
echo -e "\n Total $count lines read" #Add this line to see how many line read is working.

[saikrishna.b@gai-1339 Documents]$ sh line.sh testfile
1 This is a sample script.
2 To test
3 the file is read line by line.
4 Hope this works.
5 Lets c.
6 Done

Total 0 lines read

Instead use the below syntax to count the lines.

[saikrishna.b@gai-1339 Documents]$ cat line.sh
#!/bin/bash
filename=$1
count=0
while read a
do
let count++
echo " $count $a "
done < $filename
echo -e "\n Total $count lines read"

[saikrishna.b@gai-1339 Documents]$ sh line.sh testfile
1 This is a sample script.
2 To test
3 the file is read line by line.
4 Hope this works.
5 Lets c.
6 Done

Total 6 lines read

shellscript to sort numbers in ascending, descending order with the arguments supplied.

2013-12-14T04:38:00.000-08:00

shellscript to sort numbers in ascending, descending order with the arguments supplied.

cat sort.sh
if [ $# -eq 0 ]
then
echo "Enter valid Argument "
exit
fi
for i in $*
do
echo $i >> tmpfile
done
echo "Your sorted data are below"
sort -nr tmpfile
echo "Script Ends"

output:

sh sort.sh 3 4 5 6 7
Your sorted data are below
7
6
5
4
3

Script Ends

Ascending order use sort -n

sh sort.sh 3 4 5 6 7

Your sorted data are below
3
4
5
6
7
Script Ends

bash: locate: command not found

2013-12-14T04:35:00.000-08:00

bash: locate: command not found

1. yum install mlocate*
2. locate filename

locate: can not open `/var/lib/mlocate/mlocate.db': No such file or directory

3. updatedb

4. locate filename.will display the file location.

Configure Logs for LVM in linux

2013-11-06T00:37:00.000-08:00

Log all LVM commands

[root@test ~]# cp /etc/lvm/lvm.conf /etc/lvm/lvm.conf_backup
[root@test ~]# vi /etc/lvm/lvm.conf

log {

# Controls the messages sent to stdout or stderr.
# There are three levels of verbosity, 3 being the most verbose.
verbose = 0

# Should we send log messages through syslog?
# 1 is yes; 0 is no.
syslog = 1

# Should we log error and debug messages to a file?
# By default there is no log file.
#file = "/var/log/lvm2.log"

# Should we overwrite the log file each tisme the program is run?
# By default we append.
overwrite = 0

# What level of log messages should we send to the log file and/or syslog?
# There are 6 syslog-like log levels currently in use - 2 to 7 inclusive.
# 7 is the most verbose (LOG_DEBUG).
level = 0

# Format of output messages
# Whether or not (1 or 0) to indent messages according to their severity
indent = 1

# Whether or not (1 or 0) to display the command name on each line output
command_names = 0

# A prefix to use before the message text (but after the command name,
# if selected). Default is two spaces, so you can see/grep the severity
# of each message.
prefix = " "

# To make the messages look similar to the original LVM tools use:
# indent = 0
# command_names = 1
# prefix = " -- "

# Set this if you want log messages during activation.
# Don't use this in low memory situations (can deadlock).
# activation = 0
}

2 modifications to be done

Uncomment the line # file = “/var/log/lvm2.log

Change the level = 0 to a value between 2 and 7.

What level of log messages should we send to the log file and/or syslog?

# There are 6 syslog-like log levels currently in use - 2 to 7 inclusive.
# 7 is the most verbose (LOG_DEBUG).

Enter level=7

save and exit.

Just test it with the lvm command and view the logs

This is not the right syntax, just showed here for example.

[root@test ~]# lvcreate VG1
Please specify either size or extents
Run `lvcreate --help' for more information.

[root@test ~]# tail -f /var/log/lvm2.log
activate/activate.c:471 Getting target version for striped
ioctl/libdm-iface.c:1687 dm versions OF [16384] (*1)
lvcreate.c:363 Please specify either size or extents
lvcreate.c:768 <backtrace>
lvmcmdline.c:623 Run `lvcreate --help' for more information.
lvmcmdline.c:1123 Completed: lvcreate VG1
cache/lvmcache.c:1602 Wiping internal VG cache
metadata/vg.c:74 Freeing VG #orphans_lvm1 at 0x1f09d90.
metadata/vg.c:74 Freeing VG #orphans_pool at 0x1f0e600.
metadata/vg.c:74 Freeing VG #orphans_lvm2 at 0x1f13110.

Restore removed LVM partition

2013-11-06T00:34:00.000-08:00

[root@test ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 20G 18G 698M 97% /
/dev/mapper/VG1-backup
49G 197M 46G 1% /backup
/dev/mapper/VG1-downloads
50G 52M 47G 1% /downloads
/dev/mapper/VG1-home 49G 181M 46G 1% /home
tmpfs 979M 0 979M 0% /dev/shm
/dev/mapper/VG1-public
59G 641M 55G 2% /public

[root@test ~]# lvremove /dev/VG1/public
Can't remove open logical volume "public"

[root@test ~]# dmsetup info -c /dev/VG1/public
Name Maj Min Stat Open Targ Event UUID
VG1-public 253 7 L--w 1 2 0 LVM-8FQ0SbqlR7hlFHpG2rnxUQ2WvafgEa1yMvxZcKGIIj6Rippe9VdxO4MptcgKaO1O

Device mapper maintains entry for logical drives, so first we need to delete it from there. Then it can't restrict us while removing logical volumes which are in open status.
rom above table you can see Open = 1 which means the logical volume testlv is open status, so we need to delete entry for this logical volume from dmsetup table then only we can delete any logical volume with open status. let's do it

[root@test /]# dmsetup remove --force /dev/VG1/backup
device-mapper: remove ioctl failed: Device or resource busy
Command failed

[root@test ~]# umount /public/
[root@test ~]# lvremove /dev/VG1/public
Do you really want to remove active logical volume public? [y/n]: y
Logical volume "public" successfully removed
[root@test ~]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
backup VG1 -wi-ao 50.00G
downloads VG1 -wi-ao 54.00G
home VG1 -wi-ao 50.00G
images VG1 -wi-a- 50.00G
stripe VG1 -wi-a- 128.00M
[root@test ~]# vgcfgrestore --list VG1

File: /etc/lvm/archive/VG1_00011-1959345004.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/snap1'
Backup Time: Wed Sep 11 20:23:22 2013

File: /etc/lvm/archive/VG1_00012-1817284231.vg
VG name: VG1
Description: Created *before* executing 'lvcreate -s --name backup-snap --size 1G /dev/VG1/backup'
Backup Time: Wed Sep 11 20:48:21 2013

File: /etc/lvm/archive/VG1_00013-1680048821.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/backup-snap'
Backup Time: Mon Sep 16 09:13:22 2013

File: /etc/lvm/archive/VG1_00014-2084989037.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/dbba'
Backup Time: Mon Sep 16 09:13:47 2013

File: /etc/lvm/archive/VG1_00015-1654653445.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/public_snap'
Backup Time: Mon Sep 16 09:14:14 2013

File: /etc/lvm/archive/VG1_00016-1108210628.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/test1'
Backup Time: Mon Sep 16 09:14:25 2013

File: /etc/lvm/archive/VG1_00017-319369726.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/home1'
Backup Time: Mon Sep 16 09:14:35 2013

File: /etc/lvm/archive/VG1_00018-1088715738.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/home2'
Backup Time: Mon Sep 16 09:14:45 2013

File: /etc/lvm/archive/VG1_00019-950166769.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/dbbackup'
Backup Time: Mon Sep 16 09:15:03 2013

File: /etc/lvm/archive/VG1_00020-722332065.vg
VG name: VG1
Description: Created *before* executing 'lvresize -L -1G /dev/VG1/public'
Backup Time: Mon Sep 16 09:20:38 2013

File: /etc/lvm/archive/VG1_00021-227400998.vg
VG name: VG1
Description: Created *before* executing 'lvresize -L +1G /dev/VG1/public'
Backup Time: Mon Sep 16 09:21:18 2013

File: /etc/lvm/archive/VG1_00022-271935578.vg
VG name: VG1
Description: Created *before* executing 'vgsplit VG1 test /dev/sda5'
Backup Time: Mon Sep 16 10:01:04 2013

File: /etc/lvm/archive/VG1_00023-196107041.vg
VG name: VG1
Description: Created *before* executing 'lvremove /dev/VG1/public'
Backup Time: Tue Sep 17 09:35:07 2013

File: /etc/lvm/backup/VG1
VG name: VG1
Description: Created *after* executing 'lvremove /dev/VG1/public'
Backup Time: Tue Sep 17 09:35:07 2013

[root@test ~]# vgcfgrestore -f /etc/lvm/archive/VG1_00023-196107041.vg
Please specify a *single* volume group to restore.

You also need to specify the volume group name

[root@test ~]# vgcfgrestore -f /etc/lvm/archive/VG1_00023-196107041.vg VG1
Restored volume group VG1
[root@test ~]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
backup VG1 -wi-ao 50.00G
downloads VG1 -wi-ao 54.00G
home VG1 -wi-ao 50.00G
images VG1 -wi-a- 50.00G
public VG1 -wi--- 60.00G
stripe VG1 -wi-a- 128.00M

Logical volume public is present now.

lvdisplay

--- Logical volume ---
LV Name /dev/VG1/public
VG Name VG1
LV UUID MvxZcK-GIIj-6Rip-pe9V-dxO4-Mptc-gKaO1O
LV Write Access read/write
LV Status NOT available
LV Size 60.00 GB
Current LE 1920
Segments 2
Allocation inherit
Read ahead sectors auto

The status is not available. Make it active

[root@test ~]# lvchange -a y /dev/VG1/public

[root@test ~]# lvdisplay
--- Logical volume ---
LV Name /dev/VG1/public
VG Name VG1
LV UUID MvxZcK-GIIj-6Rip-pe9V-dxO4-Mptc-gKaO1O
LV Write Access read/write
LV Status available
# open 0
LV Size 60.00 GB
Current LE 1920
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1

Now it is in available state, mount and see the data is present or not.

[root@test ~]# mount /dev/VG1/public /public

[root@test ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 20G 18G 699M 97% /
/dev/mapper/VG1-backup
49G 197M 46G 1% /backup
/dev/mapper/VG1-downloads
50G 52M 47G 1% /downloads
/dev/mapper/VG1-home 49G 181M 46G 1% /home
tmpfs 979M 0 979M 0% /dev/shm
/dev/mapper/VG1-public
59G 641M 55G 2% /public
[root@test public]# ls
testdir
[root@test public]# cd testdir/
[root@test testdir]# ls
1 11 14 17 2 22 25 28 30 33 36 39 41 44 47 5 52 55 58 60 63 66 69 71 74 77 8 82 85 88 90 93 96 99
10 12 15 18 20 23 26 29 31 34 37 4 42 45 48 50 53 56 59 61 64 67 7 72 75 78 80 83 86 89 91 94 97
100 13 16 19 21 24 27 3 32 35 38 40 43 46 49 51 54 57 6 62 65 68 70 73 76 79 81 84 87 9 92 95 98

the data are present.

Configure Swapiness in Linux

2013-11-06T00:32:00.000-08:00

Setting swapiness means that virtual swap memory is not used until the physical memory gets full. To avoid this you can set swapiness value.

Linux system comes with a default of 60, meaning that the swap file will be used fairly often if the memory usage is around half of my RAM.

You can check your system's swappiness value by running:

[root@test ~]# cat /proc/sys/vm/swappiness
60

So If i have 4 GB of RAM, and I like to turn that down to 10 or 15. The swap file will then only be used when my RAM usage is around 85 or 90 percent.

To change the system swappiness value

vim /etc/sysctl.conf as root. Then, change or add this line to the file:

vm.swappiness = 10

Reboot for the change to take effect

You can also change the value while your system is still running

sysctl vm.swappiness=10

You can also clear your swap by running swapoff -a and then swapon -a as root instead of rebooting to achieve the same effect.

To calculate your swap Formula

free -m (total) / 100 = A

A * 10

[root@test ~]# free -m
total used free shared buffers cached
Mem: 3950 2262 1687 0 407 952
-/+ buffers/cache: 903 3047
Swap: 1953 0 1953

so total is 3950 / 100 = 39.5 * 10 = 395

It means that when 10 % 395 MB of ram left, then it start using swapiness.

RPM Usage in Linux

2013-10-05T08:09:00.000-07:00

1.How to install RPM in linux ?

#rpm –ivh packagename // To install a RPM we need to use -i option.

2. How to remove a RPM ?

#rpm -e packagename // To remove a RPM file we need to use -e option.

3.How to Upgrade a RPM ?

#rpm -Uvh packagename // To upgrade a RPM we need to use -U option.

4.How to check installed RPM?

#rpm -qa packagename // To list or find a installed RPM we need to use -q option.

5. How to List all files of an installed RPM package

#rpm -ql packagename // To list all files of installed rpm we need to use -ql ( query list) option.

6.How to Install a RPM Package Without Dependencies

#rpm -ivh --nodeps packagename // To install a RPM without any dependencies we need to use --nodeps option.

7.How to check dependencies of RPM Package before Installing

#rpm -qpR packagename // To check the dependeny for a RPM we need to use -qpR option.

-q : Query a package
-p : List capabilities this package provides.
-R: List capabilities on which this package depends.

8. How to Check an RPM Signature for a Package?

#rpm --checksig packagename.rpm // We need to use the option --checksig for signature verification.

9. How to List Recently Installed RPM Packages

#rpm -qa --last // To list recently installed RPM we need to use --last option

10. How to Query a file that belongs which RPM Package

#rpm -qf /etc/passwd // To find a file owned by which package

11.Get the Information of RPM Package Before Installing

#rpm -qip packagename

12. How to Verify a RPM Package

#rpm -Vp // To verify the installed RPM

13.How to List all Imported RPM GPG keys

#rpm -qa gpg-pubkey*

14. How to Remove an RPM Package Without Dependencies

#rpm -ev --nodeps packagename // same --nodeps option we need to use

Backup datbases and tables in Mysql

2013-10-05T08:07:00.001-07:00

Backup all databases uncompressed from the command line (not from within MySQL):

backup: mysqldump -u root -pmypass --all-databases > alldatabases.sql

restore full: mysql -u username -pmypass < alldatabases.sql (no space in between -p and mypass)

restore single: mysql -u username -pmypass mydb < mydb.sql (no space in between -p and mypass)

Backup all databases compressed from the command line (not from within MySQL):

With bzip2: mysqldump --all-databases | bzip2 -c > databasebackup.sql.bz2 (use bunzip2 to uncompress)

With gzip: mysqldump --all-databases | gzip > databasebackup.sql.gz (use gunzip to uncompress)

Mini script to backup all databases and tables using gzip:

#!/bin/sh
date=`date -I`
mysqldump --all-databases | gzip > /var/backup/dbbackup-$date.sql.gz

Backup a specific database only:
mysqldump -u username -pmypass databasename > backupfile.sql

Backup database structure only
mysqldump --no-data --databases databasename > structurebackup.sql

Backup a specific database and specific tables within that database only:
mysqldump --add-drop-table -u username -pmypass databasename table_1 table_2 > databasebackup.sql

The syntax for the command to issue is:
mysqldump -u [username] -p[password] [databasename] [table1 table2 ....] > backupfilename.sql

Backing up only the database structure of specific databases, not the actual data:
mysqldump --no-data --databases db1 db2 db3 > structurebackup.sql

Reset Mysql root password

2013-10-05T08:04:00.000-07:00

Reset root password in Mysql

Method 1:

mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> update user set password=password('') where user='root';
Query OK, 3 rows affected (0.00 sec)
Rows matched: 3 Changed: 3 Warnings: 0

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

[root@test ~]# mysqladmin -u root -p'global!23' password 'linklinux'

[root@test ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 17
Server version: 5.1.69 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

[root@gai-1347 ~]# mysqladmin -u root -p'linklinux' password ''

Empty password

Method 2:

Stop mysql

[root@test ~]# mysqld_safe --skip-grant-tables &
[1] 8747
130810 11:38:36 mysqld_safe Logging to '/var/log/mysqld.log'.
130810 11:38:36 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql

[root@test ~]# mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.1.69 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> update user set password=password('linklinux') where user='root';
Query OK, 3 rows affected (0.00 sec)
Rows matched: 3 Changed: 3 Warnings: 0

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> \q
Bye

MySQL Administrative commands

2013-10-05T08:00:00.001-07:00

MySQL Administration

This is just basics of maintaining a database including creation of databases, creation of users, granting of priveleges to users, changing passwords, and other activities from within the mysql CLI front end to MySQL.
Entering and Exiting the mysql Program

[sai@test ~]$ mysql
ERROR 1045: Access denied for user: 'sai@localhost' (Using password: NO)

OOPs! User sai is apparently password protected, so we must run mysql to query for a password, and then type in the password.

[sai@test ~]$ mysql -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 41 to server version: 4.0.18

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> quit
Bye

In the preceding, you ran mysql with the -p option so that it would query for the password, you typed in the password, and then you exited the mysql program by typing quit.

By the way, if the account has no password (this is sometimes an installation default), you would just press Enter when prompted for the password.

Perhaps if you want to log in as root instead of sai. This would probably be the case if you wanted to add a database or any other high responsibility action.

Here's how you log in as root:

[sai@test ~]$ mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 42 to server version: 4.0.18

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> quit
Bye

The -u root tells mysql to log you in as root user, and as previously discussed, the -p tells mysql to query for a password.

In mysql, when you see a prompt like this:

mysql>

you are in the mysql program. For the rest of this section, most examples will begin and end in mysql.

As a preventive measure

There may be more than one root account, and you must password protect all of them.

mysql> use mysql
Database changed
mysql> select host, user, password from user;
+-----------------------+-------+------------------+
| host | user | password |
+-----------------------+-------+------------------+
| localhost | root | 2aa0e9e91dbd7cf2 |
| localhost.localdomain | root | 2aa0e9e91dbd7cf2 |
| localhost | | |
| localhost.localdomain | | |
| localhost | sai | 2aa0e9e91dbd7cf2 |
+-----------------------+-------+------------------+
5 rows in set (0.00 sec)

mysql>

As you can see, there's a root account at localhost and another to localhost.localdomain. Both must be password protected. In reality, all accounts should be password protected.

Exploring Your MySQL Installation

From within mysql you can find quite a bit of information concerning your installation.

As what user are you logged into mysql? What databases exist? What tables exist in a database? What is the structure of a table? What users exist?

Because different operations require different priveleges, to save time we'll perform all these actions logged into mysql as root.

Let's start with finding out your username within mysql:

mysql> select user();
+----------------+
| user() |
+----------------+
| root@localhost |
+----------------+
1 row in set (0.00 sec)

mysql>

Now let's list all the users authorized to log into mysql:

mysql> use mysql
Database changed
mysql> select host, user, password from user;
+-----------------------+-------+------------------+
| host | user | password |
+-----------------------+-------+------------------+
| localhost | root | 2aa0e9e91dbd7cf2 |
| localhost.localdomain | root | 2aa0e9e91dbd7cf2 |
| localhost | | |
| localhost.localdomain | | |
| localhost | sai | 2aa0e9e91dbd7cf2 |
+-----------------------+-------+------------------+
5 rows in set (0.00 sec)

mysql>

Notice there are two root logons -- one at localhost and one at localhost.localdomain.

They can have different passwords, or one of them can even have no password. Therefore, You'd Better check to make sure they're both password protected.

DO NOT forget the password(s). If you lose all administrative passwords, you lose control of your mysql installation.

User Maintenance

Before discussing user maintenance, it's necessary to understand what properties exist for each user. To do that, we'll use mysql -e option to execute SQL commands and push the output to stdout.

Watch this:

[sai@test ~]$ mysql -u root -p -e "use mysql;describe user;" | cut -f 1
Enter password:
Field
Host
User
password
Select_priv
Insert_priv
Update_priv
Delete_priv
Create_priv
Drop_priv
Reload_priv
Shutdown_priv
Process_priv
File_priv
Grant_priv
References_priv
Index_priv
Alter_priv
Show_db_priv
Super_priv
Create_tmp_table_priv
Lock_tables_priv
Execute_priv
Repl_slave_priv
Repl_client_priv
ssl_type
ssl_cipher
x509_issuer
x509_subject
max_questions
max_updates
max_connections

The word "Field" is a heading, not a piece of information about a user. After that, the first three fields are the user's host, username and password.

The password is encoded. If you hadn't included the cut command, you'd have seen that the same username can exist in multiple hosts, even if both hosts refer to the same physical machine.

That's why it's vital to MAKE SURE TO password protect ALL users. The next several fields after password are priveleges that can be granted to the user, or not.
There's also a way to grant ALL privileges to a user. But from a security point of view this is very dangerous, as ordinary looking users can be turned into backdoors.

I'd suggest always granting and revoking specific privileges. Here is a list of the privileges that MySQL users can have, and what those privileges allow them to do:

USER FIELD PRIVILEGE FUNCTION

Select_priv Select Ability to use the select command. In other words, ability to read.

Insert_priv Insert Ability to insert new data -- insert a new row.

Update_priv Update Ability to change existing data -- change contents of a row.

Delete_priv Delete Ability to delete rows of existing data.

Create_priv Create Ability to create a new table.

Drop_priv Drop Ability to drop a table.

Reload_priv Reload

Shutdown_priv Shutdown

Process_priv Process

File_priv File

Grant_priv Grant Ability to grant and revoke privileges to others.

References_priv References

Index_priv Index Ability to create new indexes or drop indexes.

Alter_priv Alter Ability to change the structure of a table.

Show_db_priv

Super_priv

Create_tmp_table_priv Ability to create temporary tables.

Lock_tables_priv Ability to lock tables.

Execute_priv

Repl_slave_priv

Repl_client_priv

The root user, or any user given sufficient privileges, can create new users with the grant command:

mysql> grant select on test2.* to myuid@localhost identified by 'mypassword';
Query OK, 0 rows affected (0.00 sec)

mysql> select host, user, password from mysql.user;
+-----------------------+-------+------------------+
| host | user | password |
+-----------------------+-------+------------------+
| localhost | root | 7939f45f28d0aa41 |
| localhost.localdomain | root | 2aa0e9e91dbd7cf2 |
| localhost | | |
| localhost.localdomain | | |
| localhost | sai | 2aa0e9e91dbd7cf2 |
| localhost | myuid | 162eebfb6477e5d3 |
+-----------------------+-------+------------------+
6 rows in set (0.00 sec)

mysql>

In the preceding, we grant one privilege, select, on every table in the test2 database (test2.*), to user myuid at host localhost (myuid@localhost), giving that user the password "mypassword". We then query table user in the mysql database (mysql.user) in order to see whether user myuid has been created. Indeed he has.

Granting select privilege is insufficient for any user using any app that modies data. Let's give myuid more privileges in the test2 database:

mysql> grant Insert, Update, Delete, Create on test2.* to myuid@localhost;
Query OK, 0 rows affected (0.00 sec)

mysql>

Now user myuid can not only select, but can insert rows, update rows, delete rows, and create tables and databases. Conspicuously missing is the ability to drop tables and databases -- that can be more dangerous, in the hands of a careless but not malicious user, than some of the other abilities.

Privileges granted by the grant command are not kept in the mysql.user table, but instead in the mysql.db table. You can see results by issuing this command from the operating system:

[sai@test ~]$ mysql -u root -p -e 'select * from mysql.db where host="localhost" and user="myuid";' > temp.txt
Enter password:

The results look like this:

Host Db User Select_priv Insert_priv Update_priv Delete_priv Create_priv Drop_priv Grant_priv References_priv Index_priv Alter_priv Create_tmp_table_priv Lock_tables_priv
localhost test2 myuid Y Y Y Y Y N N N N N N N

You can revoke privileges like this:

mysql> revoke Delete, Create on test2.* from myuid@localhost;
Query OK, 0 rows affected (0.00 sec)

mysql>

If you redo the select on mysql.db, you'll see that those two privileges are now marked N.

To actually delete a user, you use a SQL statement to delete him from the mysql.users table after revoking all his privileges.

When deleting the user from mysql.user, if you forget the where clause, or any of its tests, especially the test on column user, you will delete too many users -- possibly all users, in which case you'll have no way to operate the database. BE VERY CAREFUL!

Now that you understand the potential landmines, here's how you delete a user:

mysql> revoke all on test2.* from myuid@localhost;
Query OK, 0 rows affected (0.00 sec)

mysql> delete from mysql.user where user='myuid' and host='localhost';
Query OK, 1 row affected (0.00 sec)

mysql>

Exploring Your Databases

You can refer to a table in two ways: By name after connecting to its database with a use statement, and with the databasename.tablename syntax.
The former is much more common in applications, but the latter is often used in database administration, especially when you must access the system database (mysql) in order to perform work on a different database.

The first item for exploration is to find all databases:

mysql> show databases;
+-------------------+
| Database |
+-------------------+
| depot_development |
| depot_production |
| depot_test |
| mysql |
| test |
| test2 |
+-------------------+
6 rows in set (0.00 sec)

mysql>

In the preceding, you went into the mysql program, determined what databases existed. Now let's explore the test database:

mysql> use test;
Database changed
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| dogs |
| people |
+----------------+
2 rows in set (0.00 sec)

mysql>

So the database test has two tables, dogs and people. Let's examine the columns in each of those tables:

mysql> show columns from dogs;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| owner | varchar(20) | | PRI | | |
| name | varchar(20) | | PRI | | |
+-------+-------------+------+-----+---------+-------+
2 rows in set (0.00 sec)

mysql> show columns from people;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| lname | varchar(20) | | PRI | | |
| fname | varchar(20) | | | | |
| mname | varchar(16) | YES | | NULL | |
+-------+-------------+------+-----+---------+-------+
3 rows in set (0.00 sec)

mysql>

Another way to get the same information is with the describe command:

mysql> describe dogs;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| owner | varchar(20) | | PRI | | |
| name | varchar(20) | | PRI | | |
+-------+-------------+------+-----+---------+-------+
2 rows in set (0.00 sec)

mysql> describe people;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| lname | varchar(20) | | PRI | | |
| fname | varchar(20) | | | | |
| mname | varchar(16) | YES | | NULL | |
+-------+-------------+------+-----+---------+-------+
3 rows in set (0.00 sec)

mysql> \q

Date command with examples

2013-09-23T04:46:00.000-07:00

Date command is most essentially used in scripting for date wise backups and in other scenarios.                                                                
                                                                     
                                                                     
                                             
    date +"%Y/%m/%d %H:%M"

is correct and 

    date +%Y/%m/%d %H:%M  

is not.

Please note that if you date conversion template contains spaces you need to put it in single or double quotes

If  you are OK with two year year representation you can use %y instead of %Y. There is also a macro %D that provides date in  mm/dd/yy format, so we can rewrite the previous example as

     date +"%D %H:%M"

Here  are most important build-in macros:

        %% a literal %
        %a locale's abbreviated weekday name (Sun..Sat)
        %A locale's full weekday name, variable length (Sunday..Saturday) %b locale's abbreviated month name (Jan..Dec)
        %B locale's full month name, variable length (January..December)
        %c locale's date and time (Sat Nov 04 12:02:33 EST 1989)
        %d day of month (01..31)
        %D date (mm/dd/yy)
        %e day of month, blank padded ( 1..31)
        %H hour (00..23)
        %j day of year (001..366)
        %m month (01..12)
        %M minute (00..59)
        %n a newline
        %p locale's AM or PM
        %r time, 12-hour (hh:mm:ss [AP]M)
        %S second (00..60)
        %t a horizontal tab
        %T time, 24-hour (hh:mm:ss)
        %U week number of year with Sunday as first day of week (00..53)
        %V week number of year with Monday as first day of week (01..52)
        %w day of week (0..6); 0 represents Sunday
        %W week number of year with Monday as first day of week (00..53)
        %x locale's date representation (mm/dd/yy)
        %X locale's time representation (%H:%M:%S)
        %y last two digits of year (00..99)
        %Y -- ccyy year

Date also can work not with the current time, but with time of modification of the referenced file

    date -r /etc/passwd +"%y%m%d"

It also can serve as data calculator.
Various options for past days (GNU date only):

date +"%Y%m%d" -d sunday # GNU date
20060709

date +"%Y%m%d" -d last-sunday # GNU date
20060702

date +"%Y%m%d" -d last-week # GNU date
date -v -1m +"%Y%m%d" # BSD date
20060627

date +"%Y%m%d" -d last-month # GNU date
date -v -1w +"%Y%m%d" # BSD date
20060604

date +"%Y%m%d" -d last-year # GNU date
date -v -1y +"%Y%m%d" # BSD date
20050704

date +"%Y%m%d" -d next-week # GNU date
date -v 1w +"%Y%m%d" # BSD date
20060711

date +"%Y%m%d" -d next-month # GNU date
date -v 1m +"%Y%m%d" # BSD date
20060804

date +"%Y%m%d" -d next-year # GNU date
date -v 1y +"%Y%m%d" # BSD date
20070704

To show the time in seconds since 1970-01-01 (Unix epoch):

date +"%s" -d "Fri Apr 24 13:14:39 CDT 2009"
1240596879

To convert Unix epoch time (seconds since 1970-01-01) to a human readable format:

date -d "UTC 1970-01-01 1240596879 secs"
Fri Apr 24 13:14:39 CDT 2009

Or:

date -ud @1000000000
Sun Sep  9 01:46:40 UTC 2001

    Examples

date "+%-m/%-d/%y" 
7/4/06

date "+%Y%m%d"
20060704

    $ date  +"%b %d, %Y"
    Mar 14, 2010

    date -d yesterday +"%Y/%m/%d"
    2010/03/13

Yesterday assigned to variable

DATE=$(date -d yesterday +"%Y%m%d")
echo $DATE
20060704

To assign the time to a variable

START=`date '+%r'`
echo $START
03:06:02 PM
sleep 5
echo $START
03:06:02 PM

To show the time in a different timezone, the TZ environment variable is read, Timezone types is found in /usr/share/zoneinfo

OLDTZ=$TZ
export TZ=GMT; echo "GMT:               `date +\"%F %R (%Z)\"`"
GMT:               2008-10-31 12:30 (GMT)
export TZ=Europe/Stockholm; echo "Stockholm:    `date +\"%F %R (%Z)\"`"
Stockholm:    2008-10-31 13:30 (CET)
export TZ=Asia/Kuala_Lumpur; echo "Kuala Lumpur: `date +\"%F %R (%Z)\"`"
Kuala Lumpur:        2008-10-31 20:30 (MYT)
export TZ=US/Central; echo "Dallas:             `date +\"%F %R (%Z)\"`"
Dallas:             2008-10-31 07:30 (CDT)
export TZ=$OLDTZ

Using date for offset calculation:

date +"%Y%m%d" -d sunday
20060709

date +"%Y%m%d" -d last-sunday
20060702

date +"%Y%m%d" -d last-week
20060627

date +"%Y%m%d" -d last-month
20060604

date +"%Y%m%d" -d last-year
20050704

date +"%Y%m%d" -d next-week
20060711

date +"%Y%m%d" -d next-month
20060804

date +"%Y%m%d" -d next-year
20070704

Cron Format in linux.

2013-09-03T02:58:00.000-07:00

Cron Format

Cron is a simple and powerful way to perform various actions in a timely manner. It also plays a main role in automating tasks.

cron format consists of five fields separated by white spaces:

<Minute> <Hour> <Day_of_the_Month> <Month_of_the_Year> <Day_of_the_Week>

Cron format:

* * * * * followed by command to be executed.
| | | | |
| | | | |
| | | | +---- Day of the Week (range: 1-7, 1 standing for Monday)
| | | +------ Month of the Year (range: 1-12)
| | +-------- Day of the Month (range: 1-31)

| +---------- Hour (range: 0-23)
+------------ Minute (range: 0-59)

Any of these 5 fields may be an asterisk (*). This would mean the entire range of possible values, i.e. each minute, each hour, etc.

Any field may contain a list of values separated by commas, (e.g. 1,3,7) or a range of values (two integers separated by a hyphen, e.g. 1-5).

After the asterisk or values, you can use character / to do repeated action with certain intervals.

For example, you can write "0-23/2" in Hour field to specify that some action should be performed every two hours (it will have the same effect as "0,2,4,6,8,10,12,14,16,18,20,22"); value "*/4" in Minute field means that the action should be performed every 4 minutes, "1-30/3" means the same as "1,4,7,10,13,16,19,22,25,28".

In Month and Day of Week fields, you can use names of months or days of weeks abbreviated to first three letters ("Jan,Feb,...,Dec" or "Mon,Tue,...,Sun") instead of their numeric values.

Examples:

* * * * * Each minute

59 23 31 12 5 One minute before the end of year if the last day of the year is Friday.

(or)

59 23 31 DEC Fri Same as above

00 12 13 7 * Every year, on July 13th at 12:00

0,15,30,45 0,6,12,18 1,15,31 * 1-5 At 00:00, 00:15, 00:30, 00:45, 06:00, 06:15, 06:30,06:45, 12:00, 12:15, 12:30, 12:45, 18:00, 18:15,18:30, 18:45, on 1st, 15th or 31st of each month, but not on weekends

*/15 */6 1,15,31 * 1-5 Same as above

0 12 * * 1-5 * At midday on weekdays

* * * 1,3,5,7,9,11 * Each minute in January, March, May, July, September, and November

1,2,3,5,20-25,30-35,59 23 31 12 * On the last day of year, at 23:01, 23:02, 23:03, 23:05,
23:20, 23:21, 23:22, 23:23, 23:24, 23:25, 23:30,
23:31, 23:32, 23:33, 23:34, 23:35, 23:59

0 9 1-7 * 1 First Monday of each month, at 9 a.m.

0 0 1 * * At midnight, on the first day of each month

* 0-11 * * * Each minute before midday

* * * 1,2,3 * Each minute in January, February or March

* * * Jan,Feb,Mar * Same as above

0 0 * * * Daily at midnight

0 0 * * 3 Each Wednesday at midnight

You can also use special strings like below

@reboot At startup.

@yearly Run once a year

0 0 1 1 * (Same as above)

@annually (same as @yearly)

@monthly Run once a month

0 0 1 * * (same as above)

@weekly Run once a week

0 0 * * 0 (same as above)

@daily Run once a day

0 0 * * * (same as above)

@midnight (same as @daily)

@hourly Run once an hour

0 * * * * (same as above)

you can use

crontab -e to edit and add your cronjobs.

crontab -l to list the cronjobs.

Before pressing the enter button , ensure that you haven't entered crontab -r as it would delete all the cronjobs.

To check the users cronjobs, use the below syntax.

crontab -u username -l

Install postgresql in linux

2013-09-03T02:52:00.000-07:00

Download postgresql software

wget http://ftp.postgresql.org/pub/source/v9.2.3/postgresql-9.2.3.tar.bz2
tar -jxvf postgresql-9.2.3.tar.bz2
cd postgresql-9.2.3

./configure --without-readline
make
make install

[root@machine1 postgresql-9.2.3]# useradd postgres
[root@machine1 postgresql-9.2.3]# mkdir /usr/local/
bin/ etc/ games/ include/ lib/ lib64/ libexec/ pgsql/ sbin/ share/ src/
[root@machine1 postgresql-9.2.3]# mkdir /usr/local/pgsql/data
[root@machine1 postgresql-9.2.3]# chown postgres /usr/local/pgsql/data
[root@machine1 postgresql-9.2.3]# /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.
[root@machine1 postgresql-9.2.3]# su - postgres
[postgres@machine1 ~]$ /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

fixing permissions on existing directory /usr/local/pgsql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 32MB
creating configuration files ... ok
creating template1 database in /usr/local/pgsql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data
or
/usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start

[postgres@machine1 ~]$ /usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data >logfile 2>&1 &
[1] 26097
[postgres@machine1 ~]$ /usr/local/pgsql/bin/createdb test
/usr/local/pgsql/bin/createdb: symbol lookup error: /usr/local/pgsql/bin/createdb: undefined symbol: PQconnectdbParams
[postgres@machine1 ~]$ LD_LIBRARY_PATH=/usr/local/pgsql/lib
[postgres@machine1 ~]$ export LD_LIBRARY_PATH=/usr/local/pgsql/lib
[postgres@machine1 ~]$ /usr/local/pgsql/bin/createdb test
[postgres@machine1 ~]$

create database

[postgres@machine1 ~]$ /usr/local/pgsql/bin/psql test
psql (9.2.3)
Type "help" for help.

test=#
test=#
test=#

change postgres password

[postgres@machine1 ~]$ /usr/local/pgsql/bin/psql test
psql (9.2.3)
Type "help" for help.

test=# ALTER USER postgres WITH ENCRYPTED PASSWORD 'postgres';
ALTER ROLE
test=# \q

Edit the postgre configuration file and add the below lines

[root@machine1 postgresql-9.2.3]# vi /usr/local/pgsql/data/pg_hba.conf

#host all all 127.0.0.1/32 trust
to
host all all 127.0.0.1/32 md5

vi /usr/local/pgsql/data/postgresql.conf

# - Connection Settings -

#listen_addresses = 'localhost' # what IP address(es) to listen on;
to
listen_addresses = '*' # what IP address(es) to listen on;

uncomment
password_encryption = on

The first time you run "service postgresql start", it asks you to initialize a database, run:

service postgresql initdb

To start the postgres service while starting the server, follow the below steps

[root@machine1 ~]# cp -rf /usr/local/src/postgresql-9.2.3/contrib/start-scripts/linux /etc/init.d/postgres
[root@machine1 ~]# chmod 755 /etc/init.d/postgres
[root@machine1 ~]# /etc/init.d/postgres start
Starting PostgreSQL: ok
[root@machine1 ~]# /etc/init.d/postgres restart
Restarting PostgreSQL: ok

To have PostgreSQL start automatically when the computer boots add
symbolic links from the correct /etc/rc*.d/ directories to
/etc/init.d/postgres. If the normal runlevel is 3 then you really only
need to add it to rc3.d:

[root@machine1 ~]# ln -s /etc/init.d/postgres /etc/rc3.d/S85postgres
[root@machine1 ~]# ll /etc/rc3.d/S85postgres
lrwxrwxrwx 1 root root 20 Jun 28 16:25 /etc/rc3.d/S85postgres -> /etc/init.d/postgres

We now successfully installed the postgresql, if you would like to change the port number, do the changes in the below configuration file

/usr/local/pgsql/data/postgresql.conf

Advanced examples with sed command,

2013-08-05T00:35:00.002-07:00

Sed - Stream Editor is a powerful command to modify the files in linux. Sed command is one of most useful commands used in shell scripting. Some of the sed examples

[root@machine2 ~]# cat test.txt
Linux is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.

Search and replace certain patterns in a file

Replaces first occurance of the pattern in file

[root@machine2 ~]# sed 's/Linux/unix/1' test.txt
unix is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.

Replaces second occurance of pattern in the file

[root@machine2 ~]# sed 's/Linux/unix/2' test.txt
Linux is a Unix-like computer operating system. unix is assembled under the model of free and open source software development and distribution.

Even this syntax replaces the first occurance.

[root@machine2 ~]# sed 's/Linux/unix/' test.txt
unix is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.

Replaces all the matching pattern in the file.

[root@machine2 ~]# sed 's/Linux/unix/g' test.txt
unix is a Unix-like computer operating system. unix is assembled under the model of free and open source software development and distribution.

[root@machine2 ~]# cat test.txt
Linux is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Replaces the matching pattern from the second occurances to the last.

[root@machine2 ~]# sed 's/Linux/unix/2g' test.txt
Linux is a Unix-like computer operating system. unix is assembled under the model of free and open source software development and distribution.Love to work with unix.

[root@machine2 ~]# cat > test1.txt
http://google.com
http://yahoo.com

[root@machine2 ~]# cat test1.txt
http://google.com
http://yahoo.com

Replacing a url. Here \ is the delimiter

[root@machine2 ~]# sed 's/http:\/\//www/' test1.txt
wwwgoogle.com
wwwyahoo.com

[root@machine2 ~]# sed 's/http:\/\//www./' test1.txt
www.google.com
www.yahoo.com

We can also use _ as a delimiter

[root@machine2 ~]# sed 's_http://_www._' test1.txt
www.google.com
www.yahoo.com

Here | is the delimiter.

[root@machine2 ~]# sed 's|http:|_www.|' test1.txt
_www.//google.com
_www.//yahoo.com

[root@machine2 ~]# sed 's|http:|www.|' test1.txt
www.//google.com
www.//yahoo.com

[root@machine2 ~]# sed 's|http://|www.|' test1.txt
www.google.com
www.yahoo.com

[root@machine2 ~]# cat test.txt
Linux is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Searches for a specific pattern and add some extra characters, symbols using &

[root@machine2 ~]# sed 's/Linux/{&}/' test.txt
{Linux} is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

[root@machine2 ~]# sed 's/Linux/,&/' test.txt
,Linux is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

[root@machine2 ~]# sed 's/Linux/&,/' test.txt
Linux, is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

To include the same patter once again use &&

only in First occurance.

[root@machine2 ~]# sed 's/Linux/{&&}/' test.txt
{LinuxLinux} is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Includes the extra characters to all the matching pattern.

[root@machine2 ~]# sed 's/Linux/{&}/g' test.txt
{Linux} is a Unix-like computer operating system. {Linux} is assembled under the model of free and open source software development and distribution.Love to work with {Linux}.

Search and replaces the matching pattern and prints line double.

[root@machine2 ~]# sed 's/Linux/unix/p' test.txt
unix is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
unix is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Search, replace and prints the line.

[root@machine2 ~]# sed -n 's/Linux/unix/p' test.txt
unix is a Unix-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Executing multiple sed commands in single line using | symbol and -e option

[root@machine2 ~]# sed 's/Linux/LINUXOS/' test.txt | sed 's/Unix/UnixOS/'
LINUXOS is a UnixOS-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

[root@machine2 ~]# sed -e 's/Linux/LINUXOperatingSystem/' -e 's/Unix/UnixOperatingSystem/' test.txt
LINUXOperatingSystem is a UnixOperatingSystem-like computer operating system. Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

[root@machine2 ~]# cat test.txt
Linux is a Unix-like computer operating system.
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Learning Linux is very easy.

Replaces word Linux with unix in the second line of the file

[root@machine2 ~]# sed '2 s/Linux/unix/' test.txt
Linux is a Unix-like computer operating system.
unix is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Learning Linux is very easy.

Replaces word Linux with unix in the first and second line of the file. We can also specify the line range.

[root@machine2 ~]# sed '1,2 s/Linux/unix/' test.txt
unix is a Unix-like computer operating system.
unix is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Learning Linux is very easy.

Replaces word Linux with unix from first to last line of the file

[root@machine2 ~]# sed '1,$ s/Linux/unix/' test.txt
unix is a Unix-like computer operating system.
unix is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Learning unix is very easy.

Search for a specific pattern and replaces it.

[root@machine2 ~]# sed '/Linux/ s/Linux/centos/' test.txt
centos is a Unix-like computer operating system.
centos is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Learning centos is very easy.

Here, Search for the occurances a matching pattern and replaces it with the given word.

[root@machine2 ~]# sed '/Linux/ s/Unix/Fedora/' test.txt
Linux is a Fedora-like computer operating system.
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Learning Linux is very easy.

In above searches for Unix and replaces with Fedora.

Prints the line twice in a file.

[root@machine2 ~]# sed 'p' test.txt
Linux is a Unix-like computer operating system.
Linux is a Unix-like computer operating system.
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Learning Linux is very easy.
Learning Linux is very easy.

Like grep command fetches and prints the line where the matching string found.

[root@machine2 ~]# sed -n '/free/ p' test.txt
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Prints the line other than the matching string line.

[root@machine2 ~]# sed -n '/free/ !p' test.txt
Linux is a Unix-like computer operating system.
Learning Linux is very easy.

Add a new line after the matching pattern.

[root@machine2 ~]# sed '/Learning/ a "Adding a new line" ' test.txt
Linux is a Unix-like computer operating system.
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
Learning Linux is very easy.
"Adding a new line"

Add a new line before the matching pattern.

[root@machine2 ~]# sed '/Learning/ i "Adding a new line" ' test.txt
Linux is a Unix-like computer operating system.
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
"Adding a new line"
Learning Linux is very easy.

Change the line where the matching string found.

[root@machine2 ~]# sed '/Learning/ c "chaning with a new line" ' test.txt
Linux is a Unix-like computer operating system.
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
"chaning with a new line"

[root@machine2 ~]# sed '/Learning/ c "changing with a new line" ' test.txt
Linux is a Unix-like computer operating system.
Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.
"changing with a new line"

Transfrom with the specific letters like tr command.

[root@machine2 ~]# sed 'y/L/l' test.txt
sed: -e expression #1, char 5: unterminated `y' command

[root@machine2 ~]# sed 'y/L/l/' test.txt
linux is a Unix-like computer operating system.
linux is assembled under the model of free and open source software development and distribution.love to work with linux.
learning linux is very easy

Remove specific lines in a file using sed command.

2013-08-05T00:35:00.001-07:00

sed command examples to delete or remove specific lines in linux
Here file is the file name.

[root@machine1 ~]# cat > file
first
second
third
fourth
fifth
END

[root@machine1 ~]# cat file
first
second
third
fourth
fifth
END

Here 1 is the line number and d is the option to delete first line

[root@machine1~]# sed '1d' file
second
third
fourth
fifth
END

[root@machine1 ~]# cat file
first
second
third
fourth
fifth
END

This will not remove in the orignal file, to remove in original use -i option

[root@machine1 ~]# sed -i '1d' file

[root@machine1 ~]# cat file
second
third
fourth
fifth
END

We also have the option to redirect it to the other file
[root@machine1 ~]# sed '2d' file > file1

To delete a last line in linux.

[root@machine1 ~]# sed '$d' file
second
third
fourth
fifth
[root@machine1 ~]# cat file
second
third
fourth
fifth
END
[root@machine1 ~]# sed -i '$d' file

[root@machine1 ~]# cat file
second
third
fourth
fifth

Delete for specific range. here i delete from 2nd to 4th line

[root@machine1 ~]# sed '2,4d' file
second

[root@machine1 ~]# cat file
second
third
fourth
fifth

Delete except first line

[root@machine1 ~]# sed '1!d' file
second

Delete except last line

[root@machine1 ~]# sed '$!d' file
fifth

Delete except specific line range from 2 to 4

[root@machine1 ~]# sed '2,4!d' file
third
fourth
fifth

Delete first and last line, here ; specify multiple conditions

[root@machine1 ~]# sed '1d;$d' file
third
fourth

Delete blank or empty lines

[root@machine1 ~]# sed '/^$/d' file
second
third
fourth
fifth

Delete line starts with pariticular characters

[root@machine1 ~]# sed '/^f/d' file
second
third

Delete line end with particular character

[root@machine1 ~]# sed '/d$/d' file
fourth
fifth

[root@machine1 ~]# cat file
second
third
fourth
fifth
LINUX
UBUNTU

Delete lines in upper case

[root@machine1 ~]# sed '/^[A-Z]*$/d' file
second
third
fourth
fifth

Delete lines that contain a specific pattern here i delete a line starts with a pattern UBUNTU

[root@machine1 ~]# sed '/UBUNTU/d' file
second
third
fourth
fifth
LINUX

Delete line starting from a pattern till the last line.

[root@machine1 ~]# sed '/fifth/,$d' file
second
third
fourth

Delete last line only if it contains a specific pattern

[root@machine1 ~]# sed '${/UBUNTU/d;}' file
second
third
fourth
fifth
LINUX

prints specific line which contain a pattern fifth

[root@machine1 ~]# cat file
second
third
fourth
fifth
LINUX
UBUNTU

[root@machine1 ~]# sed -n '/fifth/p' file
fifth

To print between 2 patterns

[root@machine1 ~]# sed -n '/fifth/,/UBUNTU/p' file
fifth
LINUX
UBUNTU

Add blank lines using sed command.

2013-08-05T00:35:00.000-07:00

Add a double blank lines after each line

sed G test.txt

Linux is a Unix-like computer operating system.

Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Learning Linux is very easy.

Add triple blank lines after each lines.

[root@machine2 ~]# sed 'G;G' test.txt
Linux is a Unix-like computer operating system.

Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Learning Linux is very easy.

[root@machine2 ~]# sed -i 'G;G' test.txt

[root@machine2 ~]# cat test.txt
Linux is a Unix-like computer operating system.

Linux is assembled under the model of free and open source software development and distribution.Love to work with Linux.

Learning Linux is very easy.

Undo double space.

[root@machine2 ~]# sed 'n;d' test.txt
Linux is a Unix-like computer operating system.

Learning Linux is very easy.

Add a single blank line above the matching pattern.

[root@machine2 ~]# sed '/Redhat/{x;p;x}' test.txt
Linux is a Unix-like computer operating system.

Learning Linux is very easy.

Redhat Enterprise linux

[root@machine2 ~]# cat test.txt
Linux is a Unix-like computer operating system.

Learning Linux is very easy.
Redhat Enterprise linux

Add a blank line below the matching pattern.

[root@machine2 ~]# sed '/Redhat/G' test.txt
Linux is a Unix-like computer operating system.

Learning Linux is very easy.
Redhat Enterprise linux

Add a blank line above and below the matching pattern.

[root@machine2 ~]# sed '/Redhat/{x;p;x;G}' test.txt
Linux is a Unix-like computer operating system.

Learning Linux is very easy.

Redhat Enterprise linux

Number the lines in linux

[root@machine2 ~]# sed = test.txt | sed 'N;s/\n/\t/'
1 Linux is a Unix-like computer operating system.
2
3 Learning Linux is very easy.
4 Redhat Enterprise linux

[root@machine2 ~]# sed = test.txt
1
Linux is a Unix-like computer operating system.
2

3
Learning Linux is very easy.
4
Redhat Enterprise linux

N join the lines.

The 's/\n/\t/' replaces the newline chars with tabs,

1 Linux is a Unix-like computer operating system.
2
3 Learning Linux is very easy.
4 Redhat Enterprise linux

Cound the number of lines in a file

[root@machine2 ~]# sed -n '$=' test.txt
4

Print number of lines in a file.

[root@machine2 ~]# sed 2q test.txt
Linux is a Unix-like computer operating system.

Prints specific line number in a file

Advanced examples with awk command.

2013-08-05T00:34:00.001-07:00

[root@machine2 test]# awk '1;{print ""}' testdoc
total 4.0K

-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing4

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing3

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing6

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing5

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing8

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing7

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing12

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing11

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing13

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing14

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing15

-rw-r--r-- 1 root root 0 Jun 9 17:46 testdoc

[root@machine2 test]# awk 'BEGIN{ORS="\n\n"}' testdoc
ORS- output row separator

[root@machine2 test]# awk 'BEGIN{ORS="\n\n"};1' testdoc
total 4.0K

-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing4

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing3

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing6

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing5

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing8

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing7

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing12

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing11

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing13

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing14

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing15

-rw-r--r-- 1 root root 0 Jun 9 17:46 testdoc

[root@machine2 test]# awk 'NF{print $0 "\n"}' testdoc

triple space a file

[root@machine2 test]# awk '1;{print "\n"}' testdoc
total 4.0K

-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing4

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing3

-rw-r--r-- 1 root root 0 Jun 9 17:43 testing6

Print each line with its line number using thebelow syntax

[root@machine2 test]# awk '{print NR "\t" $0}' testdoc
1 total 4.0K
2 -rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
3 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
4 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing2
5 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing4
6 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing3
7 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing6
8 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
9 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
10 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
11 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
12 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
13 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
14 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
15 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
16 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
17 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing15
18 -rw-r--r-- 1 root root 0 Jun 9 17:46 testdoc
[root@machine2 test]# awk '{print FNR "\t" $0}' testdoc
1 total 4.0K
2 -rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
3 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
4 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing2
5 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing4
6 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing3
7 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing6
8 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
9 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
10 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
11 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
12 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
13 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
14 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
15 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
16 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
17 -rw-r--r-- 1 root root 0 Jun 9 17:43 testing15
18 -rw-r--r-- 1 root root 0 Jun 9 17:46 testdoc

[root@machine2 test]# awk '{printf("%5d : %s\n", NR,$0)}' testdoc
1 : total 4.0K
2 : -rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
3 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
4 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing2
5 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing4
6 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing3
7 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing6
8 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
9 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
10 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
11 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
12 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
13 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
14 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
15 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
16 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
17 : -rw-r--r-- 1 root root 0 Jun 9 17:43 testing15
18 : -rw-r--r-- 1 root root 0 Jun 9 17:46 testdoc

Left align and right align using the above syntax

[root@machine2 test]# awk '{ total = total + NF }; END {print total}' testdoc
155
print the total number of fields ("words") in all lines

[root@machine2 test]# awk '{print $NF}' testdoc
4.0K
for.sh
testing1
testing2
testing4
testing3
testing6
testing5
testing8
testing7
testing9
testing10
testing12
testing11
testing13
testing14
testing15
testdoc

Prints the last filed in every line with above syntax
[root@machine2 test]# awk '{print NF ":" $0 }' testdoc
2:total 4.0K
9:-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing4
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing3
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing6
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
9:-rw-r--r-- 1 root root 0 Jun 9 17:43 testing15
9:-rw-r--r-- 1 root root 0 Jun 9 17:46 testdoc

Prints number of fields followed by a character in each line.

[root@machine2 test]# awk '{test=$NF}; END{print test }' testdoc
testdoc

[root@machine2 test]# awk 'END{print $NF}' testdoc
testdoc

Find the last field of the last line using the above syntaxes

[root@machine2 test]# awk 'NF > 3' testdoc
-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing4
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing3
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing6
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing15
-rw-r--r-- 1 root root 0 Jun 9 17:46 testdoc

Prints number of fields larger than 3

[root@machine2 test]# awk 'NF < 3' testdoc
total 4.0K

Prints number of fields less than 3

[root@machine2 test]# awk 'NR < 4' testdoc
total 4.0K
-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1

[root@machine2 test]# awk 'NR <=4' testdoc
total 4.0K
-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2

Prints number of rows less than 4 and equal to 4 using above syntax.

[root@machine2 test]# awk '/total/' testdoc
total 4.0K
search for a pattern using above syntax.

[root@machine2 test]# awk '!/total/' testdoc
-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing4
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing3
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing6
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing15
-rw-r--r-- 1 root root 0 Jun 9 17:46 testdoc

Prints other than the searching pattern using above syntax.

[root@machine2 test]# awk '/total/; /for/' testdoc
total 4.0K
-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh

search multiple pattern using above syntax.

[root@machine2 test]# awk 'length> 48' testdoc
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing15

print only line above 48 characters through above syntax.

[root@machine2 test]# awk 'length< 15' testdoc
total 4.0K
prints less than 15 characters

[root@machine2 test]# awk 'NR==8,NR==12' testdoc
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10

Prints specific rows from 8 to 12 with the above syntaxes

[root@machine2 test]# awk 'NR==12' testdoc
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10

prints specific line.

awk command with examples

2013-08-05T00:34:00.000-07:00

[root@machine2 test]# for i in `seq 1 1 15`; do touch testing$i; done
[root@machine2 test]# ls
for.sh testing10 testing12 testing14 testing2 testing4 testing6 testing8
testing1 testing11 testing13 testing15 testing3 testing5 testing7 testing9
[root@machine2 test]# ls -lrth
total 4.0K
-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing4
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing3
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing6
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing15

[root@machine2 test]# ls -lrth > testdoc

To fetch only the column type $followed by the column number.

[root@machine2 test]# cat testdoc | awk '{print$1}'
total
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--
-rw-r--r--

[root@machine2 test]# cat testdoc | awk '{print$1,$2}'
total 4.0K
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1
-rw-r--r-- 1

[root@machine2 test]# cat testdoc | awk '{print$1,$5}'
total
-rw-r--r-- 166
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0
-rw-r--r-- 0

[root@machine2 test]# cat testdoc | awk '{print$1,$6}'
total
-rw-r--r-- Jan
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun
-rw-r--r-- Jun

[root@machine2 test]# cat testdoc | awk '{print$1,$9}'
total
-rw-r--r-- for.sh
-rw-r--r-- testing1
-rw-r--r-- testing2
-rw-r--r-- testing4
-rw-r--r-- testing3
-rw-r--r-- testing6
-rw-r--r-- testing5
-rw-r--r-- testing8
-rw-r--r-- testing7
-rw-r--r-- testing9
-rw-r--r-- testing10
-rw-r--r-- testing12
-rw-r--r-- testing11
-rw-r--r-- testing13
-rw-r--r-- testing14
-rw-r--r-- testing15
-rw-r--r-- testdoc

[root@machine2 test]# awk 'BEGIN {total=0} {total=total+$5} END {print total}' testdoc
166

The above syntax will print the sum of 5th column in the testdoc file. Here total is the variable assigned to zero in the begin syntax.
In the action block, the value of 5th column is passed for each row .After all values are processed, it prints the total value in END block.

[root@machine2 test]# awk -f totalscript testdoc
166

[root@machine2 test]# cat totalscript
#!/usr/bin/awk -f
BEGIN {total=0} {total=total+$5} END {print total}

We can use the script in a file and use -f option to run it.

[root@machine2 test]# awk '{if($9 == "testing9") print$0;}' testdoc
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9

[root@machine2 test]# awk 'BEGIN{for(i=1;i<=10;i++) print "square of", i, "is", i*i;}'
square of 1 is 1
square of 2 is 4
square of 3 is 9
square of 4 is 16
square of 5 is 25
square of 6 is 36
square of 7 is 49
square of 8 is 64
square of 9 is 81
square of 10 is 100

[root@machine2 test]# awk -F: '{print$3}' /etc/passwd
0
1
2
3
4
5
6
7
8
10
11
12
13
14
99
81
170
69
32
499
173
68
38
498
89
27
70
29
65534
497
74
72
55
28
65
96
52
496
495
48
50022
75
107
141
14
Here -F is the field separator . We can also use awk 'BEGIN {FS=":"} {print $3}' /etc/passwd

[root@machine2 test]# awk '{print$8,$9}' testdoc

18:53 for.sh
17:43 testing1
17:43 testing2
17:43 testing4
17:43 testing3
17:43 testing6
17:43 testing5
17:43 testing8
17:43 testing7
17:43 testing9
17:43 testing10
17:43 testing12
17:43 testing11
17:43 testing13
17:43 testing14
17:43 testing15
17:46 testdoc
0

[root@machine2 test]# awk 'BEGIN{OFS=":"}{print$8,$9}' testdoc
:
18:53:for.sh
17:43:testing1
17:43:testing2
17:43:testing4
17:43:testing3
17:43:testing6
17:43:testing5
17:43:testing8
17:43:testing7
17:43:testing9
17:43:testing10
17:43:testing12
17:43:testing11
17:43:testing13
17:43:testing14
17:43:testing15
17:46:testdoc

OFS- Output Field separator , we add any character with this option

[root@machine2 test]# ls -l
total 12
-rw-r--r-- 1 root root 166 Jan 25 18:53 for.sh
-rw-r--r-- 1 root root 847 Jun 9 17:46 testdoc
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing1
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing10
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing11
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing12
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing13
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing14
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing15
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing2
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing3
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing4
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing5
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing6
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing7
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing8
-rw-r--r-- 1 root root 0 Jun 9 17:43 testing9
-rw-r--r-- 1 root root 71 Jun 9 17:59 totalscript

[root@machine2 test]# awk '{print NF}' testdoc
2
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9
9

NF - Number of field variables This will print number of columns in each row

[root@machine2 test]# awk '{print NR}' testdoc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
NR- Number of records, prints count of lines in a file or line number

For total line numbers type the below syntax

[root@machine2 test]# awk 'END{print NR}' testdoc
18

to search the contents in awk with out casesensitive IGNORECASE=1

Ethtool - Display or change ethernet card settings

2013-08-05T00:33:00.000-07:00

ethtool - Display or change ethernet card settings

Full duplex -sending and receiving data simultaneously.
half duplex - half duplex, you can either send or receive data at a time (
to find nic card half or full duplex

[root@machine1 ~]# dmesg | grep -i duplex
e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx

[root@machine1 ~]# ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 2
Transceiver: internal
Auto-negotiation: on
MDI-X: off
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000007 (7)
Link detected: yes

[root@machine1 ~]# mii-tool eth0
eth0: negotiated 100baseTx-FD flow-control, link ok

You can change the ethernet setttings using ethtool -s option

ethtool -s eth0 speed 100 duplex full
ethtool -s eth0 speed 10 duplex half

OR

mii-tool -F 100baseTx-HD
mii-tool -F 10baseT-HD

Tomcat start up script.

2013-07-11T06:55:00.002-07:00

1.Ensure there is tomcat user and it has read, write permission to the $CATALINA_HOME/conf and $CATALINA_HOME/logs directories.

2. Also make sure that you have set $JAVA_HOME.

3. Ensure that the tomcat processes are ran by the tomcat user.

4. Save the following scripts as /etc/init.d/tomcat. They will automatically be read and run at boot time.

Check the log files if it does not start properly.

Make a soft link to it from /etc/rc5.d such as:

cd /etc/rc5.d
sudo ln -s /etc//init.d/tomcat S71tomcat

/etc/init.d/tomcat

#!/bin/bash
#
# tomcat
#
# chkconfig:
# description: Start up the Tomcat servlet engine.

# Source function library.
. /etc/init.d/functions

RETVAL=$?
CATALINA_HOME="/usr/local/tomcat/"

case "$1" in
start)
if [ -f $CATALINA_HOME/bin/startup.sh ];
then
echo $"Starting Tomcat"
/bin/su tomcat $CATALINA_HOME/bin/startup.sh
fi
;;
stop)
if [ -f $CATALINA_HOME/bin/shutdown.sh ];
then
echo $"Stopping Tomcat"
/bin/su tomcat $CATALINA_HOME/bin/shutdown.sh
fi
;;
*)
echo $"Usage: $0 {start|stop}"
exit 1
;;
esac

exit $RETVAL

Inspired from http://www.raibledesigns.com/

Apache - Tomcat installation in Linux

2013-07-08T03:27:00.000-07:00

[root@machine1 ~]# echo $JAVA_HOME
/usr/local/jdk1.7.0_17

[root@machine1 ~]# wget http://apache.spinellicreations.com/tomcat/tomcat-7/v7.0.41/src/apache-tomcat-7.0.41-src.tar.gz
--2013-07-06 22:43:04-- http://apache.spinellicreations.com/tomcat/tomcat-7/v7.0.41/src/apache-tomcat-7.0.41-src.tar.gz
Resolving apache.spinellicreations.com... 72.88.94.11
Connecting to apache.spinellicreations.com|72.88.94.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4407748 (4.2M) [application/x-gzip]
Saving to: “apache-tomcat-7.0.41-src.tar.gz”

100%[====================================================================>] 4,407,748 698K/s in 12s

2013-07-06 22:43:24 (356 KB/s) - “apache-tomcat-7.0.41-src.tar.gz” saved [4407748/4407748]

[root@machine1 ~]# tar -zxvf apache-tomcat-7.0.41-src.tar.gz

[root@machine1 ~]# mv apache-tomcat-7.0.41-src /usr/local/

[root@machine1 ~]# cd /usr/local/

create symbolic link in the name tomcat

[root@machine1 local]# ln -s /usr/local/apache-tomcat-7.0.41-src ./tomcat

Run the following command and add it in .bash_profile.

[root@machine1 local]# export CATALINA_HOME=/usr/local/tomcat

[root@machine1 local]# cd

[root@machine1 ~]# vi .bash_profile
# User specific environment and startup programs

export JAVA_HOME=/usr/local/jdk1.7.0_17

PATH=/usr/local/jdk1.7.0_17/bin:$PATH:$HOME/bin

export JAVA_HOME
export CATALINA_HOME=/usr/local/tomcat
export PATH

save and exit

esnure that you also exported java path.

[root@machine1 ~]# $CATALINA_HOME/bin/startup.sh
-bash: /usr/local/tomcat/bin/startup.sh: Permission denied
cd /usr/local/tomcat/bin
chmod 700 *.sh

[root@machine1 bin]# /usr/local/tomcat/bin/startup.sh
Using CATALINA_BASE: /usr/local/tomcat
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat/temp
Using JRE_HOME: /usr/local/jdk1.7.0_17
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar
touch: cannot touch `/usr/local/tomcat/logs/catalina.out': No such file or directory
/usr/local/tomcat/bin/catalina.sh: line 387: /usr/local/tomcat/logs/catalina.out: No such file or directory

[root@machine1 bin]# mkdir /usr/local/tomcat/logs

The problem is that the archive manager didn't create the empty logs folder, even though it was listed in the tar file. I guess it doesn't think empty directories are important. Anyway, just create a logs folder inside the tomcat folder:

mkdir /usr/local/tomcat/logs

Tomcat should startup now. Another option would be to simply use the command line tools to extract the archive.
[root@machine1 bin]# export CATALINA_BASE=/usr/local/tomcat/

[root@machine1 bin]# /usr/local/tomcat/bin/startup.sh
Using CATALINA_BASE: /usr/local/tomcat/
Using CATALINA_HOME: /usr/local/tomcat
Using CATALINA_TMPDIR: /usr/local/tomcat//temp
Using JRE_HOME: /usr/local/jdk1.7.0_17
Using CLASSPATH: /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar

Using CLASSPATH: /usr/local/apache-tomcat-6.0.36/bin/bootstrap.jar

[root@machine1 bin]# /etc/init.d/tomcat start
Starting Tomcat
Using CATALINA_BASE: /usr/local/tomcat/
Using CATALINA_HOME: /usr/local/tomcat/
Using CATALINA_TMPDIR: /usr/local/tomcat//temp
Using JRE_HOME: /usr/local/jdk1.7.0_17
Using CLASSPATH: /usr/local/tomcat//bin/bootstrap.jar

[root@machine1 bin]# /etc/init.d/tomcat stop
Stopping Tomcat
Using CATALINA_BASE: /usr/local/tomcat/
Using CATALINA_HOME: /usr/local/tomcat/
Using CATALINA_TMPDIR: /usr/local/tomcat//temp
Using JRE_HOME: /usr/local/jdk1.7.0_17
Using CLASSPATH: /usr/local/tomcat//bin/bootstrap.jar

[root@machine1 bin]# cat /etc/sysconfig/tomcat
JAVA_HOME="/usr/local/jdk1.7.0_17"
CATALINA_HOME="/usr/local/tomcat"
CATALINA_BASE="/usr/local/tomcat"
You have new mail in /var/spool/mail/root

add the script in /etc/init.d/tomcat

chmod 755 /etc/init.d/tomcat
chkconfig --add tomcat
chkconfig --list tomcat

Type localhost:8080 or your ip address:8080 in your browser to view

Java Installation in linux.

2013-07-08T03:26:00.000-07:00

Check 32 or 64 bit using the below command and download the source accordingly.

[root@machine1 gdp]# getconf LONG_BIT
32
[root@machine1 gdp]# wget jdk-7u17-linux-i586.tar.gz

We can download the java software from oracle website

[root@machine1 gdp]# tar -zxvf jdk-7u17-linux-i586.tar.gz

Edit your .bash_profile and add the path so that it will automatically load when the machine starts.
.bash_profile

export JAVA_HOME=/usr/local/gdp/jdk1.7.0_17
export PATH=$PATH:$JAVA_HOME/bin
export PATH

[root@machine1 ~]# /usr/local/gdp/jdk1.7.0_17/bin/java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) Server VM (build 23.7-b01, mixed mode)

[root@machine1 ~]# source .bash_profile
You have new mail in /var/spool/mail/root

[root@machine1 ~]# java -version
java version "1.7.0_19"
OpenJDK Runtime Environment (rhel-2.3.9.1.el5_9-i386)
OpenJDK Server VM (build 23.7-b01, mixed mode)

Mysql hardening in Linux.

2013-07-03T23:13:00.000-07:00

1. Disable or restrict remote access

Consider whether MySQL will be accessed from the network or only from its own server.

If remote access is used, ensure that only defined hosts can access the server. This is typically done through TCP wrappers, iptables, or any other firewall software or hardware available on the market.

To restrict MySQL from opening a network socket, the following parameter should be added in the [mysqld] section of my.cnf or my.ini:

skip-networking

The file is located in the "/etc/my.cnf" or "/etc/mysql/my.cnf" on Linux.

This line disables the initiation of networking during MySQL startup. Please note that a local connection can still be established to the MySQL server.

Another possible solution is to force MySQL to listen only to the localhost by adding the following line in the [mysqld] section of my.cnf

bind-address=127.0.0.1

You may not be willing to disable network access to your database server if users in your organization connect to the server from their machines or the web server installed on a different machine. In that case, the following restrictive grant syntax should be considered:

mysql> GRANT SELECT, INSERT ON mydb.* TO 'someuser'@'somehost';

2. Disable the use of LOCAL INFILE

The next change is to disable the use of the "LOAD DATA LOCAL INFILE" command, which will help to prevent unauthorized reading from local files. This is especially important when new SQL Injection vulnerabilities in PHP applications are found.

In addition, in certain cases, the "LOCAL INFILE" command can be used to gain access to other files on the operating system, for instance "/etc/passwd", using the following command:

mysql> LOAD DATA LOCAL INFILE '/etc/passwd' INTO TABLE table1

Or even simpler:

mysql> SELECT load_file("/etc/passwd")

To disable the usage of the "LOCAL INFILE" command, the following parameter should be added in the [mysqld] section of the MySQL configuration file.

set-variable=local-infile=0

3. Change root username and password

The default administrator username on the MySQL server is "root". Hackers often attempt to gain access to its permissions. To make this task harder, rename "root" to something else and provide it with a long, complex alphanumeric password.

To rename the administrator’s username, use the rename command in the MySQL console:

mysql> RENAME USER root TO new_user;

The MySQL "RENAME USER" command first appeared in MySQL version 5.0.2. If you use an older version of MySQL, you can use other commands to rename a user:

mysql> use mysql;
mysql> update user set user="new_user" where user="root";
mysql> flush privileges;

To change a user’s password, use the following command-line command:

mysql> SET PASSWORD FOR 'username'@'%hostname' = PASSWORD('newpass');

It is also possible to change the password using the "mysqladmin" utility:

shell> mysqladmin -u username -p password newpass

4. Remove the "test" database

MySQL comes with a "test" database intended as a test space. It can be accessed by the anonymous user, and is therefore used by numerous attacks.

To remove this database, use the drop command as follows:

mysql> drop database test;

Or use the "mysqladmin" command:

shell> mysqladmin -u username -p drop test

5. Remove Anonymous and obsolete accounts

The MySQL database comes with some anonymous users with blank passwords. As a result, anyone can connect to the database To check whether this is the case, do the following:

mysql> select * from mysql.user where user="";

In a secure system, no lines should be echoed back. Another way to do the same:

mysql> SHOW GRANTS FOR ''@'localhost';
mysql> SHOW GRANTS FOR ''@'myhost';

If the grants exist, then anybody can access the database and at least use the default database "test". Check this with:

shell> mysql -u blablabla

To remove the account, execute the following command:

mysql> DROP USER "";

The MySQL "DROP USER" command is supported starting with MySQL version 5.0. If you use an older version of MySQL, you can remove the account as follows:

mysql> use mysql;
mysql> DELETE FROM user WHERE user="";
mysql> flush privileges;

6. Lower system privileges

A very common database security recommendation is to lower the permissions given to various parties. MySQL is no different. Typically, when developers work, they use the system's maximum permission and give less consideration to permission principles than we might expect. This practice can expose the database to significant risk.

* Any new MySQL 5.x installation already installed using the correct security measures.

To protect your database, make sure that the file directory in which the MySQL database is actually stored is owned by the user "mysql" and the group "mysql".

shell>ls -l /var/lib/mysql

In addition, ensure that only the user "mysql" and "root" have access to the directory /var/lib/mysql.

The mysql binaries, which reside under the /usr/bin/ directory, should be owned by "root" or the specific system "mysql" user. Other users should not have write access to these files.

shell>ls -l /usr/bin/my*

7. Lower database privileges

Operating system permissions were fixed in the preceding section. Now let’s talk about database permissions. In most cases, there is an administrator user (the renamed "root") and one or more actual users who coexist in the database. Usually, the "root" has nothing to do with the data in the database; instead, it is used to maintain the server and its tables, to give and revoke permissions, etc.

On the other hand, some user ids are used to access the data, such as the user id assigned to the web server to execute "select\update\insert\delete" queries and to execute stored procedures. In most cases, no other users are necessary; however, only you, as a system administrator can really know your application’s needs.

Only administrator accounts need to be granted the SUPER / PROCESS /FILE privileges and access to the mysql database. Usually, it is a good idea to lower the administrator’s permissions for accessing the data.

Review the privileges of the rest of the users and ensure that these are set appropriately. This can be done using the following steps.

mysql> use mysql;

[Identify users]

mysql> select * from users;

[List grants of all users]

mysql> show grants for ‘root’@’localhost’;

The above statement has to be executed for each user ! Note that only users who really need root privileges should be granted them.

Another interesting privilege is "SHOW DATABASES". By default, the command can be used by everyone having access to the MySQL prompt. They can use it to gather information (e.g., getting database names) before attacking the database by, for instance, stealing the data. To prevent this, it is recommended that you follow the procedures described below.

Add " --skip-show-database" to the startup script of MySQL or add it to the MySQL configuration file
Grant the SHOW DATABASES privilege only to the users you want to use this command
To disable the usage of the "SHOW DATABASES" command, the following parameter should be added in the [mysqld] section of the /etc/my.cnf:

[mysqld]
skip-show-database

8. Enable Logging

If your database server does not execute many queries, it is recommended that you enable transaction logging, by adding the following line to [mysqld] section of the /etc/my.cnf file:

[mysqld]
log =/var/log/mylogfile

This is not recommended for heavy production MySQL servers because it causes high overhead on the server.

In addition, verify that only the "root" and "mysql" ids have access to these logfiles (at least write access).

Error log

Ensure only "root" and "mysql" have access to the logfile "hostname.err". The file is stored in the mysql data directory. This file contains very sensitive information such as passwords, addresses, table names, stored procedure names and code parts. It can be used for information gathering, and in some cases, can provide the attacker with the information needed to exploit the database, the machine on which the database is installed, or the data inside it.

MySQL log

Ensure only "root" and "mysql" have access to the logfile "*logfileXY". The file is stored in the mysql data directory.

9. Change the root directory

A chroot on Linux operating systems is an operation that changes the apparent disk root directory for the current running process and its children. A program that is re-rooted to another directory cannot access or name files outside that directory, and the directory is called a "chroot jail" or (less commonly) a "chroot prison".

By using the chroot environment, the write access of the MYSQL processes (and child processes) can be limited, increasing the security of the server.

Ensure that a dedicated directory exists for the chrooted environment. This should be something like: /chroot/mysql In addition, to make the use of the database administrative tools convenient, the following parameter should be changed in the [client] section of MySQL configuration file:

[client]
socket = /chroot/mysql/tmp/mysql.sock

Thanks to that line of code, there will be no need to supply the mysql, mysqladmin, mysqldump etc. commands with the --socket=/chroot/mysql/tmp/mysql.sock parameter every time these tools are run.

10. Remove History

During the installation procedures, there is a lot of sensitive information that can assist an intruder to assault a database. This information is stored in the server’s history and can be very helpful if something goes wrong during the installation. By analyzing the history files, administrators can figure out what has gone wrong and probably fix things up. However, these files are not needed after installation is complete.

We should remove the content of the MySQL history file (~/.mysql_history), where all executed SQL commands are stored (especially passwords, which are stored as plain text):

cat /dev/null > ~/.mysql_history

or mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!

In order to log into MySQL to secure it, we'll need the current
password for the root user. If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none):
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.

Set root password? [Y/n] n
... skipping.

By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] y
... Success!

Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] y
... Success!

By default, MySQL comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] y
- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] y
... Success!

Cleaning up...

All done! If you've completed all of the above steps, your MySQL
installation should now be secure.

Thanks for using MySQL!

/usr/bin/ld: cannot find -lpython2.7

2013-07-03T00:18:00.003-07:00

/usr/bin/ld: cannot find -lpython2.7
collect2: ld returned 1 exit status

[root@server1 lib]# ln -s /usr/local/python2.7.3/lib/python2.7/config/libpython2.7.a .

[root@server1 lib]# pwd
/usr/lib
rwxrwxrwx 1 root root 58 Jun 27 11:55 libpython2.7.a -> /usr/local/python2.7.3/lib/python2.7/config/libpython2.7.a

NTP error - ntpd dead but pid file exists

2013-07-03T00:15:00.000-07:00

[root@server1~]# ntpq -np
ntpq: read: Connection refused

[root@server1 ~]# /etc/init.d/ntpd status
ntpd dead but pid file exists

[root@server1~]# cat /etc/sysconfig/ntpd
# Drop root to id 'ntp:ntp' by default.
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid"

# Set to 'yes' to sync hw clock after successful ntpdate
SYNC_HWCLOCK=no

# Additional options for ntpdate
NTPDATE_OPTIONS=""

Edit the configuration file and change the contents like below
[root@server1 ~]# vi /etc/sysconfig/ntpd

[root@server1~]# cat /etc/sysconfig/ntpd
# Drop root to id 'ntp:ntp' by default.
#OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid"

# Set to 'yes' to sync hw clock after successful ntpdate
SYNC_HWCLOCK=no

# Additional options for ntpdate
NTPDATE_OPTIONS=""

etc/init.d/ntpd restart
Shutting down ntpd: [FAILED]
Starting ntpd: [ OK ]
[root@server1 ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd:

[root@server1~]# ntpdate -u pool.ntp.org
13 Jun 09:16:22 ntpdate[24783]: adjust time server 69.167.160.102 offset -0.064985 sec

[root@server1 ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@server1 ~]# ntpq -np
remote refid st t when poll reach delay offset jitter
==============================================================================
72.14.183.239 216.218.254.202 2 u 3 64 1 0.001 -38.576 0.001
72.14.183.39 164.244.221.197 2 u 2 64 1 0.001 -37.146 0.001
65.49.70.244 127.67.113.92 2 u 1 64 1 0.001 2.791 0.001
127.127.1.0 .LOCL. 10 l - 64 1 0.000 0.000 0.001