Linux Administration Made Easy

Saving a root owned file as normal user from within Vi/Vim

2011-11-16T11:30:00.003+05:30

This happens lot of times. I login as a normal user and start to edit root owned file in vim / vi text editor. However, I'm not able to save changes due to permission issue (all config files are owned by root). So to save file, i can do :w !sudo tee % where
Where,
:w - Write a file.
!sudo - Call shell sudo command.
tee - The output of write (vim :w) command redirected using tee. The % is nothing but current file name i.e. /etc/apache2/conf.d/mediawiki.conf. In other words tee command is run as root and it takes standard input and write it to a file represented by %. However, this will prompt to reload file again (hit L to load changes in vim itself):

Effective usage of watch

2011-06-05T00:48:00.003+05:30

Sometimes is useful to run over and over again the same command until something happen,
Sure you can use bash history and use up arrow and return over and over again, or perhaps write some line of bash to get an infinite loop that run the same command, but there is a smarter approach to this : watch

watch -d "ls -alrt /tmp/

Options

The most common options for watch are:

-n set the interval in seconds.
-d flag will highlight the differences between successive updates.
-b option causes the command to beep if it has a non-zero exit.
-e cause watch to exit on an error from the program running.

watch -n 1 "mysqladmin --user=root --password=mypassword processlist"

watch 'ps -eo pcpu,pid,user,args | sort -k 1 -n -r | head -10'

Using tar with ssh effectively

2010-04-02T22:37:00.009+05:30

We know that tar could be used to take backup. We may like to transfer backup to some remote box. Also some time,it is needed to transfer file with directory structure in the remote box.
Lets say we have to transfer many files in the remote box while keeping file permissions and directory structure intact.
We may choose to tar the source directory ,compress it and scp to destination box. We will need to create a v big temp file only to be transferred to the destination box and and removed from source box. Once the file is scp'd to destination box, we will extract the tar'd file to create files in the same directory structure.

Steps :-

(In Src box)

1) Tar the directory in the source file ( tar -cvzf tarfilename.tar.gz /path/of/dir )
2) transfer file to destination box (scp tarfilename.tar.gz user@remote-host)
3) Delete the file ( rm tarfilename.tar.gz)

(In Dest Box)

4) Untar the file (tar -xvzf tarfilename.tar.gz)
5) Remote the file (rm tarfilename.tar.gz)

We can accomplish all these steps in a single command.

Tar has the great ability to send data to stdout/stdin using the "-" (dash) as a filename in the command line. So using that we can string together pipes to send the data to a remote server

tar -cvzf - /dir/to/be/transferred/ | ssh remoteuser@remotehost 'tar -C /path/of/dir -xvzf -'

IN NUTSHELL, WE HAVE REPLACED tarfilename.tar.gz WITH -(DASH)

What this does is pretty simple: it creates a compressed tar file of the /dir/to/be/transferred/ and sends it to stdout (-). We catch stdout with the pipe (|) and then call ssh to connect to a remote server where we execute the remote command tar. Using the remote tar we run a decompress extraction of stdin (-). tar -c changes directory

Great way to transfer a bunch of files securely, as well maintain ownership and permissions.
Its the example of PUSH method.
Following example will show the reverse method.

Lets say we have a backup file in remote host and we want to restore it in local box.
(PULL Method)

ssh user@remote-host "cat /path/to/backup/backupfile.tar.bz2" |tar jpxf -

In this command, we are cat'ing zipped file and output is being redirected to local tar using pipe. So local tar command will replace -(dash) with zipped file and will untar it.

Many other scenarios could be found in http://www.lamolabs.org/blog/1766/pushing-pulling-files-around-using-tar-ssh-scp-rsync/

Run the last command as ROOT

2010-03-29T13:50:00.002+05:30

There are many privileged commands which you run as non-prev user only to get error or moy desired output. Then you think like sudo and type the same command again.
We can type sudo !! to execute the last command as PRIV user

How to know the status of all the running services

2010-03-28T18:47:00.002+05:30

There are many commands like netstat -plant,ps -aux but when you want to know all the services which are running presently into your RHEL box, service --status-all command is very handy . It shows all the running services into your box.

Quit from shell without saving into history

2010-03-28T18:44:00.003+05:30

There are many instances when we want to quit from shell without saving any command in history. We might have run by mistake some rookie command and you dont want to disclose it to others.

kill -9 $$ will do the needful as $$ will provide the PID of the current shell.

Keyboard shortcuts for shell

2010-03-27T22:41:00.005+05:30

Some useful and more often used keyboard short cuts in bash shell under Mac

1)CTRL + A Move to the start of the line.

2)CTRL + E Move to the end of the line.( e for END)

3)CTRL + W Delete word backwards

4)CTRL + U Delete whole line backwards.(Delete from the cursor to the beginning of the line,If cursor is at the end of the line, it will delete the whole line) It can also be used to delete whole line if you are at the end of the command and you wish not to execute it.

5)CTRL + k Delete from the cursor to the end of the line.(Delete WHOLE LINE,if the cursor is in the starting,it will delete whole line)
Place the cursor in the right place and press Either U or K to delete rest of the line

6)CTRL + R to Search history. Press escape to verify and enter to execute

7)CTRL + L To clear line leaving the last line on the screen.

Some useful vi commands

2009-10-03T23:56:00.003+05:30

gg -- Start of file
G -- End of File

^ -- Start of Line (Numeric 0 also work)
$ -- End of Line
w -- move forward word by word (Sometime use capital W to move forward,e also works)
b -- move backward word by word
3G -- will move to 3rd line
o(Alphabet) -- Insert into Next line
Page up and Page down works great

* -- Read the string under the cursor and go to the next place it appears.
(For instance, if your cursor was somewhere on the word "bob," the cursor
would move to the next occurrence of "bob" in your file.)

# -- Same as above but in reverse order

u -- undo .. Press successively if required.

U (Capital) -- Undo all changes in the line

typing . (a period) -- will repeat the last command you gave.
For instance, if your last command was dw (delete word), vi will delete another word.

Counts are one of the most powerful and time-saving features of vim.
Any command can be preceded by a number.
The number will tell vim how many times to execute the command. Here are a few examples:

3j will move the cursor down three lines.

10dd will delete ten lines.

y3f" will yank (copy) text from the cursor to the third quotation mark after
the cursor on the current line. Counts are useful to extend the range of a motion in this manner.

F -- Move one screen forward
B -- Move one screen backward

H -- Move the cursor to the top of screen -- Page UP
L -- Move the cursor to the bottom of screen -- Page Down

Nagios Video

2009-07-18T21:35:00.002+05:30

Nagios Introduction

Deployment Scenario

Nagios Monitoring

Just opened binary file in vi !!!!!!!

2009-04-19T14:07:00.003+05:30

By mistake,you have just opened one binary file in vi and screen continues to scroll. You may use CTRL + C to stop that. But now,junk characters are appearing in prompt. You are not able to read anything.Try reset command. It will Reset scrambled screen.

$reset

To test this command try opening some binary file and issue reset command after stopping scrolling by pressing CTRL + C.

How to see what others are doing in real time ?

2009-04-19T12:33:00.005+05:30

There are some situation when you are only interested in knowing what others are doing.
Let say two guys are trying to fix a server and as a supervisor, you are interested in knowing how they are doing in real time.

You may do this using script command. Let say person is logged in and his terminal type is pts/0. Running "w" tells this.

script -f /dev/pts/0

This tool is good for learning from team member's skill.

Why dont you save your command and its output ?

2009-04-19T10:21:00.012+05:30

Lets say you are working on a known issue and you need to document the commands and their result.
You may use script command to capture various commands and its output.

How to use script

$script FILE-NAME

This command will open new shell. Now what ever command you issue,along with its output will be logged into FILE-NAME. Once you are done,press "CTRL-D" and you are out of screen command.

Although this command is very simple but it helps to document things.

lsof can be used in many ways for troubleshooting purposes

2009-04-19T09:51:00.008+05:30

lsof is a very useful command. Is shows various opened files. You may use lsof with specific port,pid or process.When used in correct context,it will save your life in difficult troubleshooting sessions.

Suppose you want to see what are the various services running in you server. I know you will say netstat. But you may also use lsof -i -n.

lsof -i -n

-n for overriding dns resolution
Lets discuss some of other cases too.
-i:portno

lsof -i:389

Port 389 is used by ldap by default.
This will show all the services on this port. It will also show connection status too.

lsof -i:143 -n

-n will give ip address instead of fqdn
-p:pid

lsof -p:1234

There were some situation when some stale processes were hindering new processes to spawn. This switch will display all the files,connections,sockets opened by it.
We can confirm if the particular process is stale(before kill -9)
-c process_name

lsof -c dhcpd

This switch will show all the files(connection status,type) and many other regular info of this process.

what are the various resources a process is utilizing ?

2009-04-19T09:28:00.005+05:30

There was one particular incidence when i am making changes in some conf file but its not taking effect.There were some cases when i am expecting spool directory's path in some other location but postfix is picking up some other location.
In nutshell, we want to see what are the files a particular daemon is using or opening or referring.
Lets be more specific.As i mainly deal with postfix, there are mainly two conf files, master.conf and main.conf. We are interested in knowing what are the files master process is using, i will issue following command to get this info

lsof -c master

it will show all the files,sockets,tcp connections attached to this process.

You may use other processes too to track them down.

Very useful in case you are running out of ideas what next to do if correct thing in correct place( as you think) is not taking place.

I was trying to clear spool directory but master was using different location. I was expecting it to be where it should be(during build i specified) but default postfix installation in OS was forcing different spool location(during booting).

Whole story is too big to mention here. Not in context too.

How to log output of remote ssh session ?

2009-04-19T09:17:00.003+05:30

There are many instances when you are going to ssh to remote server for troubleshooting and data gathering purposes and you want to save those data in your computer.
There is a less frequently but useful "tee" command which could be used to log all output in a remote ssh session. What it will actually do is that it will generate one file which will capture all the commands as well as their output.

ssh user@remote.server.com | tee /path/of/log/file

This command is very useful for troubleshooting purposes.

How to configure BSNL Broabband Connection as truely Always On using Siemens C2110 ADSL Modem?

2008-02-10T14:04:00.000+05:30

i have got my latest broadband connection from Calcutta Telephones(BSNL in Calcutta).They have given Siemens C2110 ADSL modem.This modem requires USB as well as Ethernet card unlike previous modems which worked either on USB or Ethernet. I have configured ADSL modem in ALWAYS ON (by storing UserID/password in Modem) so that i need not to connect from WINDOWS/LINUX.Dialing PPOE client from Windows XP was never been a problem but Linux is a bit shy from USB stuff,so i was a bit worried in Linux environment.Once i have configured modem,i was sure that as soon as Modem gets power it will establish connection irrespective of OS.Modem is working like router in pure PPPoE way unlike dialing from OS where it works as a bridge.THERE IS NO NEED TO CONFIGURE ANY NETWORK CONNECTION IN WINDOWS XP(or installing PPPoE client in Linux).I am so lazy,you know.

HOW TO CONFIGURE SIEMENS C2110 ADSL MODEM:-

Once you have properly connected USB as well Ethernet Card into modem,type http://192.168.1.1 in your browser.Login page will appear.Login as admin and default password is admin.Follow the following steps :-

(I assume internet is not connected in your PC and you have been intimated User Name and Password from BSNL.)

1) Configuration -->Internet

2) PPPoE_0_35 will appear in the right panel.If it does not come automatically,click on add and accept all the default parameters and keep on clicking Next till it ask for user Name and Password.

3)if PPPoE_0_35 appears,click on connect.Accept all the default perameters.Pls ensure that VPI is 0 and VC1 is 35.These values are defaulted too.So nothing to worry.

4)Clicking on Next will bring "Configure Connection Type" screen.Accept the default settings.It should be PPP over Ethernet (PPPoE).Click on Next.

5)If your Broadband settings is thru then, It Will show Some WAN IP Address.Accept the defaults and click on Next.

6)Next screen is "Configure Broadband User Name and Password".Its the only STEP where your intervention is required.Type Broadband User Name and Password provided by BSNL.You can also choose other options too like Always on OR Dial on Demand OR Manually Connect. I will suggest you to choose the default(Always On).Click Next.

7) Next is the summary page.Just have a glance and Click Apply.

In a nutshell,all the default parameters WORK well.You just have to provide User Name and Password and it will be REALLY ALWAYS NO Internet without any PPPoE client.

2007-10-04T12:17:00.000+05:30

Love you ...
but am not your lover.
I care for you...
but I am not from your family.
I am ready to share your pain...
but I am not in your blood relation.
I am.. You’re.......FRIEND!!!!!
True friend scolds like a DAD....
Cares like a MOM....
Teases like a SISTER....
Irritates like a BROTHER...
and finally loves U more than a LOVER.

LDAP Authentication in Apache Directory Protection?

2007-09-19T14:37:00.000+05:30

openLDAP is a open source implementation of Light Weight Directory Access Protocol.it is read optimized which is used to store user information for authentication purpose.

Once we have implemented password protection in Apache there comes the natural question ?

Why to create another password file ???
Cant we utilize our existing authentication mechanism?

Yes,we can.

from some changes in .htaccess, we need not to alter any settings in Apache's httpd.conf.

Following lines have been added in .htaccess to allow LDAP Authentication and allow only certain users.

AuthType Basic
AuthName "Message which will appear in Login Window"
#AuthUserFile /usr/local/apache/passwd/passwd (Not needed anymore)
AuthLDAPURL ldap://linuxadmin.ofb.net:389/ou=people,o=ofb (Fake address)

require user "shailesh.mishra"
require user "tom"
require user "harry"

It is assumed that LDAP server is running in linuxadmin.ofb.net (Fake address) on port 389 with ou=people,o=ofb.

Although all these directive could have been written in httpd.conf itself but its always a good idea not to fiddle with existing settings.

How to implement password protection in sensitive directory in Apache Web Server?

2007-09-19T14:23:00.000+05:30

If any website contains sensitive information or is intended for only a small group of known people, apache provides some standard ways of protecting. Method mentioned below is applicable in Apache 2.x.Location of Apache configuration file may differ.

Basics of password protection in Website

•Make adjustments in /etc/httpd/conf/httpd.conf ( Apache 2.0 in RHEL Destro)
•Make password file (using htpasswd utility)
•Prepare .htaccess file (to provide user names who are authorized to access)

Adjustments in httpd.conf:-

We need to have a server configuration that permits putting authentication directives in these files. This is done with the AllowOverride directive.Following directive have been in added in httpd.conf under :-

AllowOverride AuthConfig

AllowOverride must be “None” in all the other situations. It is good security practice and it also improves Apache performance. In case of virtual hosting (shared hosting with single IP),AllowOverride should in disabled(by equating it None) in main configuration section of Apache and it should be enabled inside .

Make password file :-

You'll need to create a password file. This file should be placed somewhere not accessible from the web. To create the file, use the htpasswd utility that came with Apache. htpasswd stands for HyperText Password.

% htpasswd -c /usr/local/apache/passwd/passwd shailesh.mishra

-c is being used for the first user so that htpasswd utility can create the file. Above command will create one password file named “passwd” in location /usr/local/apache/passwd.

htpasswd will ask you for the password, and then ask you to type it again to confirm it.

To add more users, use only htpasswd (without –c,or else it will again creat another file). Password is encrypted.

Prepare .htaccess file :-

Till now, we have configured httpd to accept user authentication for a particular directory. We have also made password file. But we need to attach this password file so that it can be used for user verification. This will be done with the help of .htaccess file. We need to create one file with the name .htaccess with the following content and store in the directory which needs to be protected.This file must be named as .htaccess as this name is specified in Apache Configuration file httpd.conf)

AuthType Basic
AuthName "Any Message which will be displayed in Login Box"
AuthUserFile /usr/local/apache/passwd/passwd
Require valid-user

If all the steps are properly followed, when user tries to access the password protected site, Login window will appear. User can login by providing user-id and password.

Parameter AuthConfig is used to tell Apache that authentication needs to be implemented.

What is gam server ? How to stop?

2007-09-13T12:27:00.000+05:30

It is a known bug in RHEL.
It hogs all the processors and some time claims 99% of CPU time. This type of behavior is mainly contributed to screen saver. As We are run our server in INIT 3 level,so screen saver is not the culprit in our case.
Gam_server is the running instance name of “gemin” which is by default loaded with RHEL.Its the successor of FAM (File Alteration Module).FAM (or gam_server) is used to decrease CPU cycle by effectively implementing Locking/Unlocking file handles. When used with IMAP , it is claimed that it will ease shared folder implementation.

In our scenario,huge number of gam_servers(sometimes > 350) were running and virtually doing nothing.At the same time, these processes use to fire mails to root which could not be delivered thus giving unnecessary load to Postfix(these mails were lying in Q and postfix tries to resend Q’ed mails after certain intervals)Courier-IMAP has been compiled with FAM module so it was the part and parcel of it. We have not implemented shared folders so I toggled two parameters of imapd.Two parameters IMAP_USELOCKS and IMAP_ENHANCEDIDLE are used to implement shared folders.If you not interested in shared folders(Some webmail client like squirrelmail lets you to implement shared folder ) you can disable these two parameters.

IMAP_USELOCKS=0

IMAP_ENHANCEDIDLE=0

Changed (toggled parameter) is written in RED BOLD. Both these parameters are used in conjunction with Shared Folder Implementation.

After restarting imapd,no gam_server will be spawned.

DDS-4 20/40 GB stores 20 GB data NOT 40 GB

2007-09-12T18:08:00.000+05:30

Today, i came to know that 20/40 GB(150 mm) DDS-4 tape stores roughly around 20 GB data.Hardware or software compression may not be very useful.So beware,DDS-4 will store only 20 GB data not 40 GB as i was thinking.
After seeing from the log that tar command is not sucessful and tape is not getting ejected automatically,i sensed that something was wrong.I can not recall the exact error message but it was something related to less storage available on tape.
As the users data are getting increased,we need to think on this front too.For the time being,i used -M switch in tar command (it allows more than one tape cartridges).
tar -cvMf /dev/st0 /home.Please be aware that software zipping (Z )will not work with multi volume.