bSolutions

Text Processing Unix Commands

2012-10-07T08:52:00.000-07:00

Text Processing Unix Commands

paste COMMAND IN UNIX
$ paste file1 file2 -> combines contents of both files line by line and displays
The '$ paste' command, it will combines file1 and file2 line by line and display on the console.

head UNIX COMMANDS
$ head file1 -> dispays 1st 10 lines of 'file1'
$ head -2 file1 -> displays 1st 2 lines of 'file1'
$ head -2 file1 file2 -> displays 1st 2 lines of 'file1' and 'file2'

head unix command use with ls commad.
$ ls -l -> It will long list all file present in current directory
$ ls -l|head -5 -> It will long list first five files.
That is 'ls -l' output will be input for 'head -5' (pipe '|' is for joining more then one commands.)

tail UNIX COMMANDS
$ tail file1 -> displays last 10 lines of 'file1'
$ tail -3 file1 -> displays last 3 lines of 'file1'
$ tail +3 file1 -> displays all lines from line 3

tail unix command use with ls commad.
$ ls -l -> It will long list all file present in current directory
$ ls -l|tail -5 -> It will long list last five files.
That is 'ls -l' output will be input for 'tail -5' (pipe '|' is for joining more then one commands.)

wc WORD COUNT UNIX COMMANDS
$ wc file1 -> prints no of lines, words and characters in 'file1'
$ wc -l file1 -> prints no of lines in 'file1'
$ wc -w file1 -> prints no of words in 'file1'
$ wc -c file1 -> prints no of characters(includes 'enter button') in 'file1'

cut UNIX COMMANDS
$ cut -c2 file1 -> displays only 2nd character of each line in 'file1'
$ cut -c2-6 file1 -> displays only characters 2-6 of each line in 'file1'
$ cut -d " " -f4 file2 -> displays only field 4 with delimiter " " in 'file2'
$ cut -d " " -file2,4 file2 -> displays fields 2 and 4
$ cut -d " " -file2-6 file2 -> displays fields 2 to 6

sort UNIX COMMANDS
$ sort file1 -> does alphabetical sorting
$ sort -r file1 -> does reverse sorting
$ sort -n file1 -> does numerical sorting
$ sort -u file1 -> removes duplicate data and sorts
$ sort -m file1 file2 -> merges 'file1' and 'file2' (works on sorted files)
$ sort -m file1 file2 -o op -> stores merged data in a file 'op'
$ sort +1 -t ":" file1 -> sorts on 2nd field(delimiter":") in 'file1' field no starts from 0

For any clarification with respect above commands reach me on by comments.

NOTE : All unix commands are case sensitive.

File handling Unix commands

2012-10-02T01:30:00.001-07:00

File handling in unix shell scripts

The following unix shell scripts commands for handling files like creating file, copying file, moving file and listing files.

CREATING FILE IN UNIX
$ touch file1 -> creates file 'file1'(if not exist) or changes time stamp(if already exist). This command will create zero bytes file with name file1
$ vi file1 -> 'vi file1' is VI Editor command for creating file. If you give vi file1, file1 will create and it will open for edit.

USAGE OF UNIX CAT COMMANDS
$ cat file1 -> display contents of 'file1'. It will just display the content of specified file(file1) in non edit mode.
$ cat < file1 -> display contents of 'file1'. It will also behave like above command
$ cat > file1 -> write contents to 'file1'. File will open with edit mode. It will get over write the previous file content
$ cat >> file1 -> append contents to 'file1'. File will open with edit mode. but here we can append over the specified file.
write command " $ cat >> file1" In command line. After editing press CTRL+D to come out.
$ cat file1 file2 -> display contents of 'file1' and file2' one after other

USAGE OF UNIX LS COMMANDS
$ ls -> lists all directories and files in current directory
$ ls d1-> lists all directories and files in target directory
$ ls -a -> lists all including hidden ones
$ ls -r -> lists all in reverse order
$ ls -R -> lists all recursively
$ ls -l -> lists with details ie long list $ ls -i -> lists with i node value
$ ls -F -> lists all with info on format(directory name with ‘/’ as suffix, executable files with ‘*’ as suffix etc.)
$ ls -l file1 -> lists details of file 'file1'

USAGE OF OTHER FILE HANDLING UNIX COMMANDS
$ file file1 file2 -> displays type of file 'file1' and 'file2'
$ more file1 -> page wise display of 'file1'
to view contents of 'file1' page wise(press space bar), line wise(press enter)
$ echo file1 -> It will also create file with zero bytes.

For any clarification with respect above commands reach me on by comments.

NOTE : All unix commands are case sensitive.

Different date formats in Unix

2012-09-19T07:10:00.000-07:00

Date Format:

1 > $a=`date '+%d%m%y_%H%M%S'`

$echo $a

130812_195306

2 >$a=`date '+%d%m%Y_%H%M%S'`

$echo $a

13082012_195409

3 >$a `date '+%d-%m-%Y %H:%M:%S'`

$echo $a

13-08-2012 19:57:13

-->

Deleting empty line in file using unix

2012-09-19T06:49:00.000-07:00

The following unix code for the deleting empty or blank line from a file.

sed '/^$/d' file_name > temp_file
mv temp_file file_name

Here is what happens:

sed '/^$/d' file_name removes all empty lines from the file file_name and outputs the result in the console,
It redirects the output into a temporary file called temp_file,
mv temp_file file_name moves the temporary file temp_file to file_name.

Unix commands for the interview

2012-09-13T08:42:00.001-07:00

Most important basic unix commands will ask in interview. These commands are more sufficient for preparation for interview. For any help regarding these unix commands reach me for putting comments.

File : File handling basic unix commands.

touch f1 -> creates file 'f1'(if not exist) or changes time stamp(if already exist)
cat f1 -> display contents of 'f1'
cat < f1 -> display contents of 'f1'
cat > f1 -> write contents to 'f1'
cat >> f1 -> append contents to 'f1'
cat f1 f2 -> display contents of 'f1' and f2' one after other
file f1 f2 -> displays type of file 'f1' and 'f2'
more f1 -> page wise display of 'f1'
more f1 -> to view contents of 'f1' page wise(press space bar), line wise(press enter)
ls -> lists all directories and files in current directory
ls d1-> lists all directories and files in target directory
ls -a -> lists all including hidden ones
ls -r -> lists all in reverse order
ls -R -> lists all recursively
ls -l -> lists with details ie long list
ls -i -> lists with i node value
ls -F -> lists all with info on format(directory name with ‘/’ as suffix, executable files with ‘*’ as suffix etc.)
ls -l file1 -> lists details of file 'file1'

Renaming : For renaming unix commands
rename f t free.c -> changes the first occurence of 'f' to 't' in free.c(becomes tree.c)
rename .c .cpp *.c -> changes all .c files to .cpp in current directory

Directory : The following unix commands commonly used for directory handling.
mkdir dir1-> creates directory 'dir1'
mkdir path/d1 -> creates 'd1' in given path
mkdir -p d1/d2/d3 -> creates a complete hierarchy of directories
cd dir1 -> change to 'dir1'
cd -> change to home directory
cd ~ -> change to home directory
cd /   -> change to root directory
cd .. -> change to parent directory
rmdir dir1 -> removes 'dir1' if its emptyCopying : List of unix commands for copy from one file to other.
cp f1 f2 -> copy contents of file 'f1' to 'f2'(if not exist creates)
cp f1 path\dir -> makes a copy of 'f1' in given directory
cp -i source target -> copies interactively
cp f1 f2 dir1 -> creates a copy of both files in 'dir1' (last argument should be a diry in case of multiple files)
cp -r dir1 dir2 -> (if dir2 not exist) create 'dir2' and copy contents of 'dir1' to it
cp -r dir1 dir2 -> (if dir2 exists) copies tree rooted at 'dir1' to 'dir2'Moving : List unix commands for moving a file from one directory to anther.
mv f1 f2 -> rename 'f1' with 'f2' (if f2 was existing it will be lost)
mv f1 f2 dir1 -> move 'f1' and 'f2' to 'dir1'
mv dir1 dir2 -> (if dir2 not exist) rename 'dir1' with 'dir2'
mv dir1 dir2 -> (if dir2 exist) moves tree rooted at dir1 to dir2
mv -i source target -> move or rename interactively

Removing : Following unix commands used for the removing file.
rm f1 f2 -> removes given files
rm -r dir1-> removes 'dir1' recursively
rmdir dir1-> removes 'dir1' if empty
rm -i f1 -> removes f1 interactively
rm -f f1 -> remove file forcefully (if it doesnt have a write permission)

Links
ln f1 f2 -> makes a hard link between f1 and f2
ln -s f1 f2 -> makes a soft link between f1 and f2

Meta characters
? -> single character
* -> 0 or more occurance of any character
[xyz] -> matches a single character, either x,y or z
[!xyz] -> matches single characters other than x,y or z
[c-z] -> mathches any single character from c-z

Pipe
cat f1 | more : o/p of 'cat f1' given as i/p to 'more'

Text Processing : unix commands for text processing.
Paste f1 f2 -> combines contents of both files line by line and displays
head f1 -> displays 1st 10 lines of 'f1'
head -2 f1 -> displays 1st 2 lines of 'f1'
head -2 f1 f2 -> displays 1st 2 lines of 'f1' and 'f2'

tail f1 -> displays last 10 lines of 'f1'
tail -3 f1 -> displays last 3 lines of 'f1'
tail +3 f1 -> displays all lines from line 3

wc f1 -> prints no of lines, words and characters in 'f1'
wc -l f1 -> prints no of lines in 'f1'
wc -w f1 -> prints no of words in 'f1'
wc -c f1 -> prints no of characters(includes 'enter button') in 'f1'

cat f2 | tr " " "#" -> change all spaces in 'f2' to #(only for display)
cat f2 | tr -s " " -> squeezes all spaces to single spaces in 'f2'
cat f2 | tr -s " " "#" -> squeezes all spaces to single #
cat f2 | tr '[a-z]' '[A-Z]' -> changes all lower case to uppercase in 'f2'

cut -c2 f1 -> displays only 2nd character of each line in 'f1'
cut -c2-6 f1 -> displays only characters 2-6 of each line in 'f1'
cut -d " " -f4 f2 -> displays only field 4 with delimiter " " in 'f2'
cut -d " " -f2,4 f2 -> displays fields 2 and 4
cut -d " " -f2-6 f2 -> displays fields 2 to 6

sort f1 -> does alphabetical sorting
sort -r f1 -> does reverse sorting
sort -n f1 -> does numerical sorting
sort -u f1 -> removes duplicate data and sorts
sort -m f1 f2 -> merges 'f1' and 'f2' (works on sorted files)
sort -m f1 f2 -o op -> stores merged data in a file 'op'
sort +1 -t ":" f1 -> sorts on 2nd field(delimiter":") in 'f1' field no starts from 0

Pattern matching : Following commands are used for grep (search) something from file
"" or ''
grep 'hello' f1 -> displays lines containing hello
grep -n 'hello' f1 -> displays o/p along with line no
grep -i 'hello' f1 -> does case insensitive search
grep -c 'hello' f1 -> gives count of lines having
grep -v 'hello' f1 -> gives lines which dont contain 'hello'
grep -l 'hello' f1 f2 f3 -> gives files which contain 'hello'

grep '^hi' f1-> displays lines starting with hi (only for alpha numeric)
grep 'hi$' f1 -> displays lines ending with hi
grep '\<hello\>' f1 -> displays lines containing 'hello' as a word (not part of another string)
grep '.' f1 -> lines containing any character
grep '\.' f1 -> lines containing '.'

Uniq :
uniq f1 -> displays data avoiding 'adjacent repeated' lines
uniq -c f1 -> with count of occurance of lines adjacently
uniq -u f1 -> displays only the unique lines
uniq -d f1 -> displays only duplicate lines

Compare : Following unix commands for comparing two files.
cmp f1 f2 - compares and displays occurence of 1st difference (sort both files for desired o/p)
cmp -l f1 f2 -> prints about each difference till end of any of the file
cmp -s f1 f2 -> returns exit code
                      0 : for for same files
                      1 : if files different
                      2 : if any file is inaccessible
                           comm f1 f2 -> prints in 3 columns-
                     1: lines belonging only to f1
                     2: lines belonging only to f2
                     3: lines common to both
                         'f1' and 'f2' need to be sorted

Process related commands : List of below commands for the process related.
ps -> list all processes submitted to kernel
ps -f -> lists with PPID(parent process ID)
ps -e -> lists including user and system processes
ps -l -> long listing
ps -u -> lists processes related to any user( UID   PID PPID C    STIME TTY       TIME COMMAND)

ls -l > f1 & -> to do process in background (will get a jobId and PID)
bg -> see status of background processes
fg %1-> to bring the job to foreground (using jobId)
fg #9506 -> to bring job to foreground(using PID)

nice -> gives current scheduling priority
nice ls -> execute ls with priority value incremented by 10(default)
nice -n5 ls -> proirity value is incremented by 5 and executed

nohup ls & -> execute in background even after logout.

kill -1 PID -> sends hangup signal to the process with PID
kill -2 PID -> interrupt signal
kill -9 %jobId -> terminate the process
kill -15 PID -> software termination signal (default even if no option is given -15 is taken)

df [file] -> file system disk space usage
du [file] -> estimated file space use
df -g -> gives the details of free space

Compression : Following unix commands are used for the compressing file.
gzip f1 -> replaces file with a compressed file with '.gz' extension (with same ownership modes, access time e.t.c)
gzip -d f1.gz -> to unzip
gunzip f1.gz -> to unzip
zcat f1.gz -> to view contents of zipped file
zmore f1.gz -> to view contents of zipped file pagewise

tar -cvf f1f2.tar f1 f2 -> creates an archive file 'f1f2.tar' on 'f1' and 'f2' (extn can be anything or nothing)
               -c : create
               -v : verbose mode(gives details of created archive)
   -f : archive to file
               -x : extact from file
tar -xvf f1f2.tar f1 -> to extract back f1 only (if not specified extract all)

Communication commands :
write user_id -> writes a message to another user who is logged in

mesg y -> gives permision to receive messgaes
mesg n -> denies permission to receive messages
mesg -> shows current status of permission (messages from root cannot be disabled)
wall "Hai" -> broadcasts the message to all users who are logged in
wall -g groupId "Hai" -> broadcasts message to everyone in the group

File Transfer Protocol :
ftp hostname -> gets into ftp session with host
bin -> binary mode

put f1 -> transfer 'f1' from source(user) to destination
get f1 -> gets file from destination to source(user)
mput f1 f2 -> transfers 'f1' 'f2' from source to destination
mput a* -> transfers all files starting with a to destination
mget f1 f2 -> gets 'f1' and 'f2'

bye -> closes ftp session
close -> closes ftp session after giving message "Good Bye"
help -> list of all commands
? -> list of al commands

! cat f1 -> executes command 'cat f1' in local machine prompt and returns to ftp prompt
! -> to go to local machine prompt (can execute any command then)
exit -> to go back to ftp prompt

Security :
passwd -> to change password (It will ask for old password, new password
and confirmation of new password)
chmod 654 f1 -> changes to rw(owner), rx(group), r(others)
chmod ug+rwx f1 -> changes to rwx(owner). rwx(group), existing(others)
chmod u=rwx,g=rx,o=w f1 -> assigns permissions

Find utility :
find ~ -name f1 -print -> searches for all files with name 'f1' downward
from home directory and print search result
find ~ -type d -name dir1 : searches for all directories of name 'dir1'
downward from home directory
    d (directory)
    f (normal file)
    c (special character file)
    b (block special file)

find . -inum 5645 -> searches for file with given inum downward from current directory
find . -links 3 -> searches for files with 3 links
find ~ -perm 744 -> searches for files with permission 744 from home directory
find . -name "f[0-9]" -> searches for filenames with 2 characters starting with 'f' and a digit (if using meta characters, it should be within "")
find . -name "f*" -> searches for file names starting with 'f'
find ~ $ –user vikas –a –name file1 $ -> searches for file1 from
home directory whose owner is vikas
find ~ -size 10c -> searches for files with size '10 bytes'
find ~ -size +15c -> files having size greater than 15 bytes
find ~ -size -10c -> files having size less than 10 bytes
find ~ -size 10 -> having size 10 blocks
find . –name f1 –exec rm –i {} \; -> searches files named f1 and execute 'rm -i' command on them
find . –name f1 –ok rm {} \; -> searches f1 and executes rm on it after confirmation

vi editor : Following commands are commonly used in VI editor.
There are 2 type mode in VI editor
1> COMMAND MODE
2> ESCAPE MODE
3> INSERT MODE

h(left) ( or Left arrow)
j(down) (or Down arrow)
k(up) (or Upper arrow)
l(right) (or Right arrow)
w -> goes to start of next line
b -> goes to start of line upwards (also start of current line)
i -> go to insert mode at current position of cursor
I -> insert mode at start of that line
a -> insert mode at next character position
A -> insert mode at end of that line
o -> insert mode at a new line below current line
O -> insert mode at new line above current line

Esc (button) -> goes to escape mode : Following commands will work in escape mode.

dd -> deletes the current line
dw -> deletes till the next space(rest of the current word) if there is a coming space its also included
cc -> cuts the current line and stores in buffer for pasting
cw -> cuts the rest of the curret word(including space if any)

yy -> copies the current line
yw -> copies the rest of the word
p -> pastes in next cursor position (what was cut or copied)
P -> pastes before the current cursor position
/hello -> searches forward for 1st occurence of 'hello'
?hello -> searches backward for 1st occurence of 'hello'
n -> repeats last search
N -> repeats last search in opposite direction

ESCAPE MODE
:s/hello/houdi -> replaces 1st occurence of 'hello' by 'houdi'
:s/hello/houdi/g -> replaces all occurences in current line
:2,5s/hello/hqoudi/g -> replaces all occurences of 'hello' in lines 2 to 5 with 'houdi'
:.,$s/h/H/g -> replaces all occurances from current line to end of file
:1,$s/h/H/g -> replaces all occurences in all lines
:set showmode
:set nu -> to dispay line numbers
:abbr bjo "Bipin Joseph Odattil" -> creates abbreviation bjo. Future bjo in data will be replaced by the full form
:unabbr bjo -> unabbreviate future bjo won't be replaced
map v :wq -> now 'v' works as ':wq'
enter -> goes to command mode
:q -> end
Esc Shift+g -> goes to the last line
:$ -> goes to the last line

INSERT MODE
esc   -> goes to command mode

List of basic unix commands with examples

2012-09-13T07:15:00.000-07:00

List of basic unix commands:

The following commands used for checking who and are all log in for unix systems.

who : details of who all are logged in

finger : details of who all are logged in with extra information

who am I : details of ourself

whoami : userid of current user

Please check following example screen for the above unix commands.

Fig 1 : Unix example

Some Other basic commands with example

type : to check type of command

alias d=date : d will work like 'date' command

alias : give list of all aliases

pwd : present working directory

man type : manual of 'type' command (for the help.)

If i want to check help for 'ls' command then type man ls, it gives what ls command for.

Unix Calender Basic Commands

2012-09-09T08:05:00.003-07:00

1:- cal : calendar for current month
cal command gives the calender for current month.

>cal

2:- cal 01 2010 : calendar for january, 2010
cal 01 2010 gives the calender for the specific month of year.
>cal 01 2010 in this example, it will show 2010 January month calender

3:- cal 2012 : calendar for year 0013
>cal 2012 gives calender for full year of 2012.

Please check below figure 1 & 2 console for more information has been done from HP AIX.

Figure : 1 It shows whole year calender.

Figure : 2

Unix basic Date commands

2012-09-08T09:39:00.001-07:00

Following are the date commands commonly used in Unix.

date          :date
date +%T    :time only
date +%D    :date only
date +%Y    :year (4 digit)
date +%y    :year (2 digit)
date +%b    :month (3 letter)
date +%B    :month (full)
date +%a    :day (3 letter)
date +%A    :day (full)
date +%m    :month(digit)
date +%M    :minutes

Please check in below console has been done from HP AIX