Unix Training

Too Many Files Open Issue

2011-11-01T00:39:00.000-07:00

While running socket program, we do see "Too Many Files Open Issue. To address this issue,

$ limit

Run limit command with Administrator rights, we will see following details


cputime      unlimited
filesize     unlimited
datasize     unlimited
stacksize    10240 kbytes
coredumpsize 0 kbytes
memoryuse    unlimited
vmemoryuse   unlimited
descriptors  8192 
memorylocked 32 kbytes
maxproc      47264 
openfiles    8192

Open this file /etc/security/limits.conf to change these values. You will get more details from comments section of this file .

In this file, make the changes as below to address this issue.

* soft nofile 65535
* hard nofile 65535

Kernel Parameters

2009-05-22T00:35:00.000-07:00

Name	Description	Reasonable Values
`SHMMAX`	Maximum size of shared memory segment (bytes)	250kB + 8.2kB * `shared_buffers` + 14.2kB * `max_connections` or infinity
`SHMMIN`	Minimum size of shared memory segment (bytes)	1
`SHMALL`	Total amount of shared memory available (bytes or pages)	if bytes, same as `SHMMAX`; if pages, ceil(SHMMAX/PAGE_SIZE)
`SHMSEG`	Maximum number of shared memory segments per process	only 1 segment is needed, but the default is much higher
`SHMMNI`	Maximum number of shared memory segments system-wide	like `SHMSEG` plus room for other applications
`SEMMNI`	Maximum number of semaphore identifiers (that is, sets)	>= ceil(max_connections / 16)
`SEMMNS`	Maximum number of semaphores system-wide	ceil(max_connections / 16) * 17 + room for other applications
`SEMMSL`	Maximum number of semaphores per set	>= 17
`SEMMAP`	Number of entries in semaphore map	see text
`SEMVMX`	Maximum value of semaphore	>= 255 (The default is often 32767, do not change unless asked to.)

We can update directly in this file using root permission /etc/sysctl.conf

TCP Tuning in Linux.

2009-05-21T23:00:00.000-07:00

Command	Description
echo "1" > /proc/sys/net/ipv4/tcp_window_scaling	Activate window scaling according to RFC 1323
echo "1" > /proc/sys/net/ipv4/tcp_timestamps	Activate timestamps according to RFC 1323
echo 4194304 > /proc/sys/net/core/rmem_max	Set maximum size of TCP receive window.
echo 4194304 > /proc/sys/net/core/wmem_max	Set maximum size of TCP transmit window.
echo 4194304 > /proc/sys/net/core/rmem_default	Set default size of TCP receive window.
echo 4194304 > /proc/sys/net/core/wmem_default	Set default size of TCP transmit window.
echo "[wmin] [wstd] [wmax]" > /proc/sys/net/ipv4/tcp_rmem	Set min, default, max receive window. Used by the autotuning function.
echo "[wmin] [wstd] [wmax]" > /proc/sys/net/ipv4/tcp_wmem	Set min, default, max transmit window. Used by the autotuning function.
echo "bmin bdef bmax" > /proc/sys/net/ipv4/tcp_mem	Set maximum total TCP buffer-space allocatable. Used by the autotuning function.
ifconfig eth? txqueuelen 1000	Define length of transmit queue. Replace "?" with actual interface number.

Cron Job

2009-04-22T05:34:00.000-07:00

Create a file with commands whatever you want.
Say for example, I want to move files from /tmp/sps/a*.txt to /home/kdh/abc/ folder, every three minutes once.

Step 1: create a file
vi cr1.sh
mv /tmp/sps/a*.txt /home/kdh/abc/
:wq

Step 2: Create and post a crontab for this command to run for every three minutes
crontab -e
*/3 * * * * /home/kdh/cr1.sh
:wq

Now Cronjob is running, every three minutes.

To display the jobs list :

crontab -l

More details check for man crontab

# Minute   Hour   Day of Month       Month          Day of Week        Command   

# (0-59)  (0-23)     (1-31)    (1-12 or Jan-Dec)  (0-6 or Sun-Sat)               

    0        2          12             *               0,6           ls

Unix Editors - vi, vim

2009-04-15T03:17:00.000-07:00

Editors
-------

Line editors - ed, ex, sed
Screen editors - vi, vim, emacs

vi works with three mode - command, insert, colon / last line mode / esc mode / ex mode

Vi Commands

To open Vi, simply type "vi" at the command prompt in a Unix terminal.
Once Vi editor is running, you can use the following commands.

Moving the Cursor
h One space to the left
j One space to up
k One space down
l One space to the right

Beginning of new line
- Beginning of previous line
^ Beginning of current line
$ End of current line
Forward one space
nG Beginning of line n
b Beginning of current word
w Beginning of next word
e End of current word
Control - e Scroll forward
Control - b Scroll backward
/pattern First occurence of pattern
n Next occurence of pattern
N Previous occurence of pattern
/ Repeats last pattern search

Text Insertion and Deletion
a Appends text to the right of the cursor
A Appends text at the end of the line
i Inserts text to the left of the cursor
I Inserts text at the beginning of the line
o Opens a new line below the current line for text to be inserted
O Opens a new line above the current line for text to be inserted
x Deletes character at cursor position
X Deletes character left of cursor position
dd Deletes current line
dw Deletes current word
d) Deletes rest of sentence
D or d$ Deletes from cursor to end of line
P Puts back text from the previous delete
rx Replaces selected character with x
u Undoes last change
U Restores current line
:R myfile Appends file "myfile" to current file at current cursor position
DEL Overwrites last character during text insertion
ESC Stops text insertion, returns to command mode

Screen Commands
Control - I Reprints current screen
Control - L Exposes one more line at top of screen
Control - E Exposes one more line at bottom of screen
Control - F Pages forward one screen
Control - B Pages back one screen
Control - D Pages down half a screen
Control - U Pages up half a screen

Change Commands
cw Changes characters of current word until escape key is pressed
c$ Changes text up to the end of the line
C or cc Changes remaining text on the current line until escape key is pressed
~ Changes case of the current character
xp Transposes the current and following characters
J Joins current line with the next line
s Deletes current character and enters insertion mode
rx Replaces current character with x
R Replaces the following characters until escape key is pressed
yy Puts current line in a buffer without deleting the line
p Places the line stored in buffer after the current position of the cursor

Other helpful Vi notes:
To create or edit a vi file:

Type "vi myfile" where myfile is the name of the file to be created or edited.

To close a vi file:

ZZ -- Exits vi and saves changes made to the file.
:wq -- Writes changes to current file and quits edit session.
:q! -- Quits edit session and does not save changes.
For complete documentation on Vi, type "man vi" at the Unix command prompt.

To set custom commands for Vi permanently, create a file in home directory .exrc file

set nu -- set line number on/off
abbr os Operating Systems - set abbreviation
map ^W :w^M - map the commands

Disk Usage in Linux or Unix

2009-04-07T02:41:00.000-07:00

To find out folder size

du -s /foldername/

To find out all the subfolder and files size

$du -ah /foldername

CPU Utilization in Unix/Linux

2009-04-01T03:45:00.000-07:00

Top command to find out Linux cpu usage
# top

Display the utilization of each CPU individually using mpstat

# mpstat

# mpstat -P ALL

Comparison of CPU utilization

# sar -u 2 5

To get multiple samples and multiple reports set an output file for the sar command. Run the sar command as a background process using.
# sar -o output.file 12 8 >/dev/null 2>&1 &

Better use nohup command so that you can logout and check back report later on:
# nohup sar -o output.file 12 8 >/dev/null 2>&1 &

All data is captured in binary form and saved to a file (data.file). The data can then be selectively displayed ith the sar command using the -f option.
# sar -f data.file

Task: Find out who is monopolizing or eating the CPUs

ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
ps -eo pcpu,pid,user,args | sort -r -k1 | less

iostat command
You can also use iostat command which report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions

iostat

iostat -xtc 5 3

GUI tools for your laptops/desktops

gnome-system-monitor

Fourth Day

2008-11-12T06:27:00.001-08:00

ed Editor
--
a append
w write
q quit
p print
s search
u undo
d delete
g all the occurence in a line
c change/replace
i insert

a & i works on line-addressing format, can able to spcify only one line number.

$ed
a
heloo this is the data
entered to ed to append
of text
.
w file3
50
q
$

$ ls file3
file3

$ed file3
50
p
of text
1,$p
hel this is the data
entered to ed to append
of text
s/text/TEXT
p
of TEXT
u
p
of text
1,$s/a/A/g
1,$p
hel this is the dAtA
entered to ed to Append
of text
1
hel this is the data
a
*********
.
w
68
1,2p
hel this is the data
*********
c
+++++++
.
1,2p
hel this is the data
+++++++
1,$c
Whole content will be changed
.
1,$p
Whole content will be changed

sed Stream Editor
----
sed [options] 'address Action ... ' [file(s)]

here, address could be line or context based.

Applys the action implicity global to all the lines.

instruction1
instruction2
|
Line--->process-----> output
|
Dynamic Pattern

$sed 's/cow/dog/'
cow
dog
cow1 cow
dog1 cow
^D

$sed 's/cow/dog/g'
cow1 cow
dog1 dog
^D

$sed 's/cow/dog/2'
cow1 cow
cow1 dog
^D

$sed 's/cow/dog/2,3'
cow cow cow
cow dog dog
^D

$sed 's/dog/cow/
> s/cow/horses/'
dog
horses
cow
horses

$sed 's/://g' /etc/passwd removes the : from all the lines of the file
$sed '1,4s/://g' /etc/passwd removes the : from first 4 lines of the file
$sed 's/://g' /etc/passwd /etc/shadow removed : from two files

$sed '1s/cow/dog/' apply only for first line
cow
dog
cow
cow

$sed -e 's/dog/cow/' -e 's/cow/horse/' multiple actions using -e
dog
horse
cow
horse

$sed 's/dog/cow/;s/cow/horse/' multiple actions using ; option
cow
horse
dog
horse

$vi actions
s/dog/cow/
s/cow/horse/
:wq

#multiple actions storing in a file
$sed -f actions
cow
horse
dog
horse

#context based address; if and only if root exist the hello will be changed as bye
$sed '/root/s/hello/bye/'
hello
hello
root hello
root bye
hello root
bye root

#hello will be replace from Line which has root to line which has halt.
$sed '/root/,/halt/s/hello/bye/g'
hello hello
hello hello
this is root line
this is root line
hello hello
bye bye
hello
bye
halt hello
halt bye
hello
hello

#replace until the empty line occurs
$sed '1,/^$/s/hello/bye/g'
hello
bye

hello
hello

#do not apply to first two line replace operation
$sed '1,2!s/hello/bye/g'
hello
hello
hello hello
hello hello
hello
bye

Grouping commands
-----------------
address
{
instructions
}

# print only when pattern matches
$sed 's/cow/dog/gp'
cow
dog
dog

# prints and stores to abc file when pattern matches
$sed 's/cow/dog/gpw abc'
cow
dog
dog

$cat abc
dog

#Store actions result in two different file
$sed 's/hello/bye/w mm
> s/old/new/w mm1'
hello
bye
old
new
$cat mm
bye
$cat mm1
new
$

#store multiple actions result in same file
$sed 's/hello/bye/w mm
s/old/new/w mm'
hello
bye
old
new
$cat mm
bye
new

#if both pattern found two lines will be created
$sed 's/hello/bye/w mm
s/old/new/w mm'
hello old
bye new

$cat mm
bye old
bye new

Replacement part
----------------
& print pattern
\n stored pattern retrieve, here n is number
() stores pattern

#patterns are got remembered and placed using & symbol

$sed 's/director/executive director/'
director
executive director

$sed 's/director/executive &/'
director
executive director

$sed 's/director/& executive &/'
director
director executive director

# operator using $.*$ to select all the characters and saved as pattern
# open and close phrase stores pattern and can be retrieved using \n, here n is number
$cat > text
first:second
third:fourth
$sed 's/$.*$:$.*$/\2\1/' text
secondfirst
fourththird
$sed 's/$.*$:$.*$/\2+\1/' text
second+first
fourth+third

delete command
--------------

$sed '/root/d'
line
line
root
hello
hello

#delete line which ends with bash
$sed '/bash$/d'
hello krishna
hello krishna
hello bash
hello xyz
hello xyz

$sed '/^bash/d' delete line which start with bash
$sed '/^$/d' delete empty lines
$sed '1,/^$/d' delete upto empty lines
sed '/hello/,/bye/d' delete lines including hello and bye lines and found lines in between of this pattern

#appending
$sed 'a\
> ****'
this
this
****
hello
hello
****

#append for second line
$sed '2a\
****'
hello
hello
hello
hello
****

#append at the end of the line
$sed '$a\
****'
a
d
a
g
d
g
****

# append new line for each line
$sed 'a
> ' /etc/passwd

#backwardslash creates new line
$sed '1a\
> ****\
> \
> '
hello
hello
****

Replace
-------
#replace in first line
$sed '1c\
> ****'
hello
****
hj
hj

# replace in all the lines
$sed 'c\
****'
hello
****
h
****

$sed '1c\***'
h
***
hello
hello

NonPrintable characters
-----------------------
# To display non-printable commands
$sed 'l'
^D
^D$
^D
^A
^A$
^A

Transformation
--------------

Character to character transform

$sed 'y/abc/xyz/'
abc
xyz
xyz
xyz
ball
yxll

Printing
--------
sed -n to suppress the printing

$sed 'p'
hello
hello
hello

$sed -n 'p'
hello
hello

#print only the word root containing lines
$sed -n '/root/p' /etc/passwd

# print line numbers use = character in action
$sed '='
line1
1
line1

#two input stream read and line number is not getting resetted for secondstream
$sed '=' - -
hello
1
hello
hello
2
hello

#print totalnumber of line
sed -n '$=' /etc/passwd

#print line containing root
$sed -n '/root/=' /etc/passwd

#print text with line number
$sed -n '/root/=;/root/p' /etc/passwd
1
root:x:0:0:root:/root:/bin/bash
12
operator:x:11:0:operator:/root:/sbin/nologin

Reading
-------
[lineaddress]r file

$sed 'r simple'
hello
hello
hello
welcome

Writing
--------

$sed 'w simple'
hellotyped
hellotyped
$cat simple
hellotyped

sed -n 'w simple' /etc/passwd
$cat simple

#Splitting file - first five line stored into file1 and remaining moved to file2
$sed -n '1,5w file1
>6,$w file2'

quit
----
[line-address]q

# first five lines will be displayed and quit
$sed '5q' /etc/passwd

#first line will be processed
$sed 'q' /etc/passwd
$sed 'q' /etc/passwd /etc/shadow
$sed 'q;=' /etc/passwd

#print first lines of all the files in folder
for i in * ; do sed 'q' $i;done

#remove empty lines which found in between of hello and bye.

$cat > blank
this

hello

hello

end

bye

end
$cat > sedscr
/hello/,/bye/{
/^$/d
}
$sed -f sedscr blank
this

hello
hello
end
bye

end
$

Advanced sed commands
----------------------
working wiht a multiple pattern space (N, D, P)
Using the holds space to preserve the contents of the pattern space and make it available for subsequent comments (H,h,G,g,x).

N creates multiline patternspace
--------------------------------

vi next2
/owner/{
N
s/owner *\n*and *\n*operator *\n*guide/installation guide/
}

vi next3
/owner/{
N
s/ *\n/ /
s/owner and operator guide */installation guide\
/
}

D deletes a portion of the pattern space

vi d3
/^$/{
N
/^\n$/d
}

vi delete.dat
this is followed by one blank line

this is followed by two blank lines

this is followed by three

this is followed by four

the end.
$

$sed -f d3 delete.dat
this is followed by one blank line

this is followed by two blank lines
this is followed by three

this is followed by four
the end.
$

vi d4
/^$/{
N
/^\n$/d
}

$sed -f d4 delete.dat
this is followed by one blank line

this is followed by two blank lines

this is followed by three

this is followed by four

the end.
$

H hold - h or H copy or append contents of PS to Hold Space
G get - g or G - copy or append contents of HS to Pattern Space

$sed 'G' /etc/passwd
$sed 'g' /etc/passwd

$sed '/root/G'
hello
hello
root
root

wow root welcome
wow root welcome

$

$sed '/root/g'
hello
hello
root

$vi ad3
/1/{
h
d
}
/2/{
G
}

$vi ad3.dat
1
2

$sed -f ad3 ad3.dat

awk
---
awk [-options] 'pattern action ...' [file(s)...]

awk is filter.

Used for Report generation,
Some of feature carried from c and shell programming

$awk '/root/' /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
$awk /root/
hroot
hroot
root
root
$

$awk /^root/
hroot
root
root
$

$awk '/^[0-9]+$/'
abcd
12
12
$

$awk '/root/{print }'
a
root
root
$

Field selection
---------------
$0 complete line
$1 first field
$10 tenth field

$awk '{print $1,$2}'
this is line one
this is
$

$awk '{print $0}'
this is line one
this is line one
$

$awk '/root/
> /halt/'
this root
this root
this roo
halt
halt
root halt
root halt
root halt
$

$awk '/root/{print $0}
> /halt/{print}'
this root
this root
halt
halt
root halt
root halt
root halt
$

Default delimiter could be whitespaces like space, tab.

$awk -F ":" '{print $3}' /etc/passwd

awk -F ":" '/root/{print $3}' /etc/passwd

$awk '/[a-z]+/{print "Alphabets"}
/[0-9]+/{print "Numerals"}
/^$/{print "Blank"}'
fds23
Alphabets
Numerals
232
Numerals
sds
Alphabets
$

#multiple field separotor
$awk -F "[:,+]" '/root/{print $3}'
1:2,3:4
root:2+4:5
4
$

$awk -F "[:,+]+" '/root/{print $3}'
root::,2+4:5
4
$

#prints fields with space
awk -F : '{print $1,$3}' /etc/passwd

#string concatenated
awk -F : '{print $1$3}' /etc/passwd

$awk -F : '{print $1+$3}' /etc/passwd
$awk -F : '{print $1"+"$3}' /etc/passwd
$awk -F : '{print $1"\t\t"$3}' /etc/passwd

printf
------

$awk -F : '{printf("%15s%15s\n",$1,$7)}' /etc/passwd

$awk -F : '{printf("%-15s%-15s\n",$1,$7)}' /etc/passwd

$awk -F : '{printf("|%15s||%15s|\n",$1,$7)}' /etc/passwd

#Redirecting to file
$ls -l | awk '{print $5 > "out"}'

#piping
$ls -l | awk '{print $5 | "sort -n "}'
$ls -l | awk '{print $5}' | sort -n

awk script from file
---------------------
$cat > awkscr
/root/{print $1}
/halt/{print $0}
$

$awk -f awkscr
roor is root
roor
halt is fsdf
halt is fsdf
$

Artihmatic operation:

$awk '{print $1 + 5}'
4
9
$awk '{print $1 / 5}'
34
6.8
40
8
$

#Delimiter
$awk -F : '{print $1,$2}' /etc/passwd

#if particular field satisfies the condition then print
$awk '$3 >50'
this is 344
this is 344
this is 50
this is 49

#with logical operators
$awk -F : '$3 >500 && $3 <=510{print $3}' /etc/passwd

$awk -F : '$1 == "root"' /etc/passwd

#regular expressions applied using ~ sumbol
$awk '$3 ~ /^root/'
this is root
this is root
this is bash
$

$awk '$3 !~ /^root/'
this is root
this is bash
this is bash
$

#++ operator usage
$awk '/^$/{print ++x}'

1

2

3
dff

4
$

awk '{print NR}'
line1
1
line2
2
$
$awk '{print NR}' /etc/passwd

# prints the line with number - NR - Number of Record
$awk '{print NR, $0}' /etc/passwd

$awk '{print NR}' /etc/passwd /etc/shadow
#FNR get resetted
$awk '{print FNR}' /etc/passwd /etc/shadow

#NF - number of Fields
$awk '{print NF}' /etc/passwd

#read fourth field using arithmatic calculations
$awk '{print $(2*2)}'
1 2 3 4 5
4

#last field of each record
$awk '{print $(NF)}'
1 2 3 4 5
5
$

Storing and running Awk script:
-------------------------------
$vi awk4
#!/bin/awk -f
{print $0}

$
$chmod +x awk4
$./awk4
hello
hello
$

AWK programming model

BEGIN
MAIN
END

BEGIN used to print the heading/Declaring variables
END used to print the summary/totals
BEGIN and END sections are optional
BEGIN (action)
END (action)

$awk 'BEGIN{print "Welcome to AWK programming"}'
Welcome to AWK programming

$awk 'BEGIN{print "Welcome to AWK programming"}{print}'
Welcome to AWK programming
first
first
second
second

$awk 'BEGIN{print "Welcome to AWK programming"}{print}END{print "Over"}'
Welcome to AWK programming
first
first
second
^D
Over
$

$awk 'END{print "Number of records processed", NR}'
Hello
it is me
look for total number of Records processed
Number of records processed 3
$

#FS - Field Separator
$awk 'BEGIN{FS=":"}{print $3}' /etc/passwd

#OFS - Output Field Separator
$awk 'BEGIN{FS=":";OFS="+"}{print $1,$3}' /etc/passwd

#check current directory holding file size
ls -l | awk 'BEGIN{sum=0}{sum=sum+$5}END{print "total",sum}'

Passing Variables
-----------------

$awk -v var=1 '{print $var}'
hello r u there
hello
wow
wow
$awk -v var=2 '{print $var}'
hello r u there
r
$

Builtin functions in awk
------------------------
$awk '{print int($1)}'
34
34
45
45

$awk '{print sqrt($1)}'
34
5.83095
9
3

$awk '{print length($1)}'
sdsdsd
6
hello
5

$awk '{print length($0)}'
wow this
8
working
7

$awk '{print toupper($0)}'
hello
HELLO

$awk '{print tolower($0)}'
HELLO
hello

$awk '{print substr($1,1,2)}'
this
th
wonderful
wo

$awk '{print substr($1,3,2)}'
1990
90

$awk '{print substr($1,3)}'
1990
90
1990000
90000

#substring in regular expression
$awk 'substr($1,3,2) >45 && substr($1,3,2) < 90'
1940
1946
1946
1980
1980
1990
$

#Split function
$awk '{split($0,arr,":");print arr[1]}'
this:is:find
this
$awk '{split($0,arr,":");print arr[2]}'
19:23:45
23
$awk '{split($0,arr,":");print "hello" arr[2]}'
ram:krishna:moorthy
hellokrishna
$

#system function used to run unix command
$awk 'BEGIN{system("date")}'
Fri Oct 24 16:07:02 IST 2008
$

#displays the line which has more than 65 chracters
$awk 'length >65' /etc/passwd

substituions functions
----------------------
sub()
gsub()
gensub()

$awk '{print sub("root","hello")}'
root
1
this is root root
1
$awk '{print gsub("root","hello")}'
root root
2

$awk '{print gsub("root","hello",$1)}'
hello root
0

$awk '{print gsub("root","hello",$1);print}'
hello
0
hello
root
1
hello
$awk '{gsub("root","hello",$1);print}'
root
hello
$

$awk '{print gensub("root","hello",1)}'
root root
hello root

$awk '{print gensub("root","hello","G")}'
root root
hello hello
$awk '{print gensub("root","hello","g")}'
root root
hello hello

#getline method stores result of previous output's last line
$ awk 'BEGIN{ "date" | getline name ; print name}'
Fri Oct 24 16:20:59 IST 2008
$ awk 'BEGIN{ "date" | getline name ; print "date is "name}'
date is Fri Oct 24 16:21:27 IST 2008
$

#getline method holds input from std in
$awk 'BEGIN{print "Enter your name"
> getline name <"-"
> print name
> }
> '
Enter your name
krishna
krishna
$

$awk 'BEGIN{getline record < "/etc/passwd";print record}'
root:x:0:0:root:/root:/bin/bash

#subdate method to do mail merge of operation
$cat subdate.awk
/@date/{
"date +'%a., %h %d, %y'" | getline today
gsub(/@date/, today)
}
{ print }
$cat subdate.test
I am writing mail on @date to my friend.
this has to reach on @date.
$awk -f subdate.awk subdate.test
I am writing mail on Fri., Oct 24, 08 to my friend.
this has to reach on Fri., Oct 24, 08.
$

$cat awkfor1
#!/bin/awk -f
{
s=0
for (i=1; i<= NF; i++ )
s=s+$i
print s
}
./awkfor1
22
22
2 ee
2
3 44
47
5 6 7
18

do while
--------

cat >awkwhile1
{
total=i=0
do {
++i
total +=$i
}while( total <=100 )
print i, ":", total
}

#after press enter, all the lines of /etc/passwd will be displayed
$awk '{while ((getline < "/etc/passwd") > 0) print }'

#just displays all the lines
$awk 'BEGIN{while ((getline < "/etc/passwd") > 0) print }'

Third Day

2008-11-12T06:26:00.002-08:00

Until
-----

until control command
do
commands
done

#list the numbers reverse, if user enters more than ten
echo Enter the number
read num
until [ $num -le 10 ]
do
echo $num
num=`expr $num - 1`
done
echo "Out of Until loop"

#get ordinary filename from user until then continue
echo enter the filename
read fname
until [ -f $fname ]
do
echo $fname is not an ordinary file
read fname
done
echo $fname is an ordinary file

Write a script to generate the number sequence 20.0 , 19.9, 19.8, 19.7 ... 10.0

num1=20.00
until [ "$num1" = "10.00" ]
do
echo $num1
num1=`echo $num1 - 0.1 | bc`
done

Write a script to check whether the particular user already logged in or not. If not logged in then wait until he logged in . Script has to run in background.

echo Enter user name
read user
sec=0
until [ `who | cut -f1 -d " " | grep $user` ]
# even we can use regular expression to get exactly matchinguser name
do
sec=`expr $sec + 10`
sleep 10
done
echo "waited for ${sec}s"

Case
-----
case value in
choice 1)
commands;;
choice 2)
commands;;
esac

In shell script case statement, do not have dafault key word. *) used for the same and it can present anywhere in case but rest of the choices will not work.

echo -e "Enter the choice:\c"
read ch
case $ch in
1)
ls ;;
2)
who ;;
3)
date;;
*)
echo Default
;;
esac

# we can use regular expression also as choice
echo -e "Enter the choice:\c"
read ch
case $ch in
[lL]*)
ls ;;
[wW]*)
who ;;
3 | 4)
date;;
*)
echo Default;;
esac

#we can use string also for choice
echo -e "Enter the choice:\c"
read ch
case "$ch" in
"ls")
ls ;;
*)
echo Default;;
esac

Control statements
------------------

tput clear
while true
do
echo -e "Enter your choice:\c"
read ch
case "$ch" in
1)
while true
do
echo -e "Enter your number:\c"
read num
if [ $num -le 10 ]
then
echo $num is less than or equal to 10
echo Do you want to continue
read ans
if [ "$ans" = "Y" -o "$ans" = "y" ]
then
continue
else
break
fi
else
echo number is greater than 10
fi
done
;;
2)
echo "default option entered and exiting"
exit
esac
done

tput - screen interface command
tput bold bold
tput smso reverse background
tput rmso normal background
tput reset
tput cup 10 20 expect input from 10 row 20 position

Write a script to display stars like below
*
**
***
****
***
**
*

echo enter number
read num
num1=1
while [ $num1 -le $num ]
do
star=`echo "$star*"`
echo "$star"
num1=`expr $num1 + 1`
done

num1=`expr $num1 - 1`

until [ $num1 -lt 1 ]
do
num1=`expr $num1 - 1`
if [ $num1 -eq 0 ]
then
break
fi

star=`echo "$star" | cut -c1-$num1`
echo "$star"
done

#Multiplication Table upto 10 times for any of the number

1 * 5 =5
.
.
.
10 * 5 = 50

echo Enter number
read num
times=1
until [ $times -eq 11 ]
do

echo "$times * $num = `expr $times \* $num`"
times=`expr $times + 1`
done

trap
----
trap 'echo Interrupted' 2 press CTRL c then "Interrupted" string will be printed

trap - 2 to remove the trap

#Until user say hello don't allow to do something and stop interrupting or quiting

#!/bin/bash
trap "echo you have pressed : Cannot escape" 1 2
echo "Terminal Locked"
read key
while true
do
echo -e "enter your password:\c"
stty -echo
read pw
stty sane
if [ "$pw" = "hello" ]
then
break
else
echo "Wrong! password Illegal user"
fi
done

export
------

copies the variable for all shells which get created in this process
If any change has happened in child shell for this variable will not impacted in parent process. It is quivelent to passing by Value argument.

#demo on export command
#export.sh
echo The value of x is $x
x=100
echo Now the value of x is $x
sleep 1
echo Process ID is $$
echo Your are about to leave this process
sleep 10

$x=10
$echo $$
234

$sh export.sh
The value of x is
Now the value of x is 100
Process ID is 24633
Your are about to leave this process

$export x

$sh export.sh
The value of x is 10
Now the value of x is 100
Process ID is 24644
Your are about to leave this process

Shells and Sub-shells
---------------------
sh filename
sh < filename here filename holds script file

Interpreter Line
----------------
#!/bin/ksh to create/switch temporarily a shell

() brackets used to group the commands in sub-shells

$date ; who ; ls |> out only the output of ls will be stored in out file
$( date ; who ; ls ) |> out the output of date, who and ls will be stored in out file

{} - paranthesis used to group the command and run in same shell

$pwd
root
$( cd /var ; ls )
ls output goes here
$pwd
root

${ cd /var ; ls }
ls output goes here
$pwd
var

Dot commands
------------
. scriptfile runs in current shell

$vi demo1.sh
echo Process id $$

$echo $$
567
$ sh demo1.sh
Process id 8888
$. demo1.sh
Process is 567

.bashrc file -
------------

this file get executed while making login

.bash_profile - current login shells - means this will impact only for currently logged in shells

KORN/BASH
---------

let : computation (KORN/BASH)
----
$let sum=" 3 + 4"
$echo $sum
7

$echo $(date) in KORN/BASH, we can use this format to retrieve variable value

Computation with (( )) two brackets
------------------------------------
Here no need of giving $ for all the variables
x=10
y=10
z=10

sum=$((x+y+z));echo $sum
z=$((z+1));echo $z

The Here Document (<<)
-----------------------

waits for END character to typed by user

$cat << END
>this is
>waiting
>for
>END
this is
waiting
for
$

$cat << END
> this is input
> send on `date +%D`
> to `whoami` user
>END
this is input
send on 10/23/08
to user3 user
$

#Write a program to send the mail using here document

$vi here1.sh
#!/bin/sh
echo this is to test the here document
echo
echo
mail user3 << MARK
Your porgram for printing the invoices has been executed
on `date`. Please check the print queue.
MARK

$sh here1.sh
this is to test the here document

Now mail send to user3

$mail
mail details displayed here

String Operations:
------------------

${#name} used to find the length of the string

${variable%pattern}

$name="oracle"
$echo ${#oracle}
6

$fqdn="www.oracle.co.in" % used to cut from the end of the string , %% from the beginning
$echo ${fqdn%in}
www.oracle.co.
$echo ${fqdn%co.in}
www.oracle.
$echo ${fqdn%.co.in}
www.oracle
$echo ${fqdn%.*}
www.oracle.co
$echo ${fqdn%%.*}
www
$echo ${fqdn%o*}
www.oracle.c
$echo ${fqdn%%o*}
www.
$echo ${fqdn#www}
.oracle.co.in
$echo ${fqdn#www.}
oracle.co.in
$echo ${fqdn#*.}
oracle.co.in
$echo ${fqdn##.}
in

$ip_address="192.168.0.1"
$echo ${ip_address%.*}
192.168.0
$echo ${ip_address##.*}
192.168.0.1
$echo ${ip_address#.*}
192.168.0.1
$echo ${ip_address%%.*}
192

$name="mmm"
$echo ${#name}
3
$expr $name : '.*' using expr string operations
3
$expr length "$name"
3
$

Conditional Parameter substitution

:+ if value found then returns the attached string
:- if value not found then returns the attached string
:= assign the value to the variable if value not found

$found=`ls`
$echo ${found:+"found the values"}
found the values
$found=""
$echo ${found:+"found the values"}

$

$found=""
$echo ${found:-"value not set"}
value not set

$found=""
$echo ${found:="value"}
value
$echo ${found}
value
$found="a"
$echo ${found:="value"}
a

Merging Streams
---------------
1>&2 - send the standard output to the destination of the standard error.>&2
2>&1 - send the standard error to the destination of the standard ouput.>&1

ls -l abcd.tes nonexistingfile.txt 2> errorfile 1>&2

Shell functions
---------------
function_name()
{
commands
}

function will exist for only the shell where it is defined, not visible for subshell.

$test(){
> echo "function defined ..."
> date
> echo Welcome
> }

$test
function defined ...
Thu Oct 23 14:52:22 IST 2008
Welcome

$unset test
$test
$

To bring the functions which are stored in file, available for current shell, will be by running DOT command with filename

. functionfile

$name="marsd"
$expr $name : '...$..$'
sd
$expr $name : '..$..$.'
rs

$vi datef
date_fname() {
set -- `date`
year=`expr $6 : '..$..$'`
echo "$2$3_$year"
}

$. datef
$date_fname
Oct23_08

eval command
------------
Evaluates more than once.

$echo $$
8762
$echo $$$
8762$
$echo $$$$
87628762
$eval echo \$$$ $8762 will be some number based what we set earlier
762

$y=x
$z=y
$echo $$z
8762z
$echo \$$z
$y
$eval echo \$$z
x
$

$ set 1 2 3 4 5 6 7 8 9 10 11 16
$eval echo \${$#}

The exec Statement
------------------
$exec date runs date and logs out

Instead of using std in,out,err,we can create new File descriptor using exec command.

$exec 3>invalidfile
$echo "this goes to invalid file " 1>&3
$cat invalidfile
this goes to invalid file

Redirect output to the file
$exec 1>| out
$ls
$cat out > /dev/tty
list the files

$exec 1> /dev/tty to reset back to the normal

Debuggers
---------

sh -n used for checking errors

set -n set debug enable
set -o noexec remove deubg option

sh -x xtrace
sh -u unbound variable error message at each attempt to use an undeclared variable

POSIX character class
----------------------
The ability of character classes to match characters not in the english alphabet

grep -E "^[[:digit:]]+$"
grep -E "^[[:alpha:]]+$"
grep -E "^[[:cntrl:]]+$"

Second Day

2008-11-12T06:26:00.001-08:00

find
----

find [path] [search criteria]

find displays all the files and displays in standard output
find . -print displays all the files and displays in standard output
find . -name "in*" -print displays the files which are starts with "in"
find . -type f -name "in*" display only files starting with in
find . -type d -name "in*" display only folders starting with in

find /home -user "user1" display files from home folder created by user1
find /home -group "group"
find /home -uid 500 if user name already removed but to search that user created files

find . -links 4 display files which has 4 number of link
find . -links +4 -ls display files which has 4 or more links

find . -perm 755 displays the files which has permission 755
find . -perm 1755
find . -perm 2755
find . -perm 4755

find . -atime -1 within one day accessed files
find . -mtime -1 within one day modified files
find . -c -1

find . -mtime +365 365 day before modified files

ls -lc inode change time stamp - create time
ls -lu modified time

find . $ -name "mm*" -o -type l $ files with mm as starting character or any link files
find . !$ -name "mm*" -o -type l $ files not with mm as starting character or any link files

find -size 45 find files by size

find . -name "mm*" -exec cat {} \; display found files in console
find . -name "mm*" -ok cat {} \; remove found files (ok for interactively do the job)

file
----
Displays the files type whether it is ASCII or other

file * to display current folder's files type
file /bin/cp

shells
------
cat /etc/shells displays the available shells type

old shell - bourne shells
bash - bourne again shells
csh - C shells
ksh- Korn shell
tcsh - TC shell

ps
--
displays the process list

sleep 100 & & symbol used to put particular process in background
fg 1 bring the particular job foreground

^Z to interrupt/suspend

bg 1 to restart interrupted jobs

jobs to display currently available jobs

ps -l list the process

nohup command helps to run background jobs even after we logged out

nohup sort /etc/passwd > out & sort will be stored in out file

ps -el | grep -e "17280" -e "init" to display after reloggin to display running jobs

nice -n 15 sleep 100 & sets 15 as nice value to set priority value

The Shells
----------

internal command
external command
executable files

type cd to display what kind of command
type echo

more etc/passwd holds the which shell alotted for user by default

chsh permanently changes the shell type

Shell Variables
---------------
Named parameters or user defined variables
Positional parameters
Special parameters

Named Parameters
----------------
read name sets the value for name

read name1 name2 sets value for multiple variables
krishna moorthy

set to display user defined variables
unset name1 to unset variables

name1="100" direct assign also possible

echo $SHELL to Know what kind of shell used by current user

Executable order of the command will be check first the given command in builtin and then check current working directory and then check in $PATH.

echo $PS1
[\u@\h \W]\$
PS1='[ \W]$' single quote compulsory to set the PS1 path

echo $IFS
echo "$IFS" | od -bcd

Positional Parameters - used in shell script to get input from command line
---------------------

Maximum 9 positional arguments possible to retrive using $1.. $9 ,to access 10th value ${10}

vi u
echo enter user name
read user
echo welcome $user

sh u

vi u1
echo Welcome $1

sh u1 krishna

To take all the arguments except with $0 (filename of the executed command)

Special Parameters
------------------
$$ pid of the current process
$! pid of the last background process
$# total number of arguments passed in command line
$@ To take all the arguments except with $0 (filename of the executed command)
$* list of all the arguments as similar to $@
$? exit status of the previous command
$- displays the shell options

sh -x u1.sh debug the shell script xtrace mode
sh -v u1.sh debug the shell script verbos mode
sh -n u1.sh debug undefined varibales -it won't run only checking varibales

Test Command
------------
to compare the values or expressions

$read num
10
$echo $num
10
$test $num -eq 10 check using test command
$echo $?
0

$read num
10
$[ $num -eq 20 ] check integer check with square brackets
$echo $?
1

$str1="hello"
$[ "$str1" = "Hello" ] ; echo $? check string comparison
1

$[ -n "$str1" ] ; echo $? check not null
0

$[ -z "$str1" ] ; echo $? check null
1

File operators in Test Command
------------------------------

$test -f file1 ; echo $? file exist and ordinary
$test -d file1 ; echo $? file exist and directory
$test -e file exist
$test -s file2 ;echo $? file exist with some content

relational operators
-------------------
-gt, -lt, -eq, -ge, -le,

Logical operators
-----------------
-a
-o
!

Quotes usage in Unix
--------------------
single quote prints special parameters also as it is
single back quote treats as unix single command
double quote prints the string and special parameters will be replaced

echo 'Today date is date'
echo 'Today date is `date`'
echo Today date is `date`
echo "Today date is `date`"
echo "argument values are $*"

expr
----
makes arithmatic operation

$expr 4 + 5
9

$expr 4 * 5
expr: syntax error

$expr 4 \* 5
20

$expr 5 + 4 \* 2
13

$expr $ 5 + 4 $ \* 2
18

$sum=`expr $ 5 + 4 $ \* 2` ; echo $sum
18

Setting positional variables value using set command
----------------------------------------------------

$set 10 20 30 40
$echo $1
10
$echo $3
30
$echo $#
4
$echo $$
17287
$echo $*
10 20 30 40

$shift shift command used to reposition the values one from the left
$echo $1
20
$echo $#
3

$shift 2
$echo $1
40
$echo $#
1

$set 4 + 5
$echo $1
4

$set `expr 4 + 5`
$echo $1
9

#sets & displays the date to the variable
set `date`
$echo Today is $1
Today is Wed
$echo year is $6
year is 2008
$echo $#
6

IFS
----
Holds the delimeter value

$set `tail -1 /etc/passwd` ; echo $#
1
oldifs="$IFS"
$IFS=":"
$set `tail -1 /etc/passwd` ; echo $#
7

bc - Base Calculations
--
$num1=10.5
$num2=30
$expr $num1 + $num2
expr: non-numeric argument
$echo $num1 + $num2 | bc
40.5

Write a script to accept four numbers(interactive) and calculate average of it.

echo Enter four numbers one by one
read num1
read num2
read num3
read num4
sum=`echo $num1 + $num2 + $num3 + $num4 | bc`
avg=`expr $sum / 4 `
echo $avg

or

read num1 num2 num3 num4
echo `echo $ $num1 + $num2 + $num3 + $num4 $ / 4 | bc`

Display the largest ordinary filename and its size in the current dir.

ls -lSr | grep "^-" | tail -1 | tr -s " " | cut -f5 -d " "

or

ls -lS | grep "^-" | head -1 | tr -s " " | cut -f5 -d " "

Conditional Execution
---------------------
&&
||

$grep "root" /etc/passwd && echo "Pattern Found"
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
Pattern Found
$grep "roottter" /etc/passwd && echo "Pattern Found"

$grep "root" /etc/passwd > /dev/null && echo "Pattern Found" Redirect output to null device
Pattern Found

Program Constructs
------------------
if
for
while
until
case

if statement
------------
if control command
then

fi

if control command
then

else

fi

if control command
then

elif control command
then

else

fi

echo Enter the filename
read fname
if[ -e $fname ]
then
echo $fname exists
else
echo $fname not exists
fi

Write a script to accept two filenames and compare both the files are identical remove one of the file. Remove has to be interactively.

cho Enter two file name
read f1 f2
if `cmp $f1 $f2 > /dev/null`
then
echo both files are identical
rm -i $f1
echo first file got deleted
else
echo files are different
fi

cmp, comm and diff also be used to make compare the files.

Write a script to check whether the user is logged on or not.

echo read user name
read user
if `who | cut -f1 -d " " | grep "\<$user\>" > /dev/null `
then
echo user found
else
echo user not found
fi

for loop
---------

for i in 1 2 3 4 5
do
echo $i
sleep 2
done

for i in `seq 1 5` useing sequence generate numbers
do
echo $i
sleep 2
done

for i in *; do echo $i; done to display the files from the directory

for i in `ls -l | tr -s " " | cut -f 5 -d " " `; do echo $i; done | sort -rn displays size in reverse order

Write a program to count occurence of uniquly the words

vi data1
this is data
hello this is data
data
hello
hello

echo `cat data1 | sort | uniq | wc -w`

Rename all the doc files to txt files

for i in `ls *.doc`
do
name=`basename $i doc` #basename helps to cuts doc
mv $i ${name}txt
done

Remove all the empty files in the current directory only

for i in *
do
if ! test -s $i
then
echo deleted file $i
rm -ir $i
else
echo not deleted file - $i
fi
done

while Loop
----------

#continue the loop until user inputs some character
echo enter the answer
read ans
while [ -z "$ans" ]
do
echo entered nothing
read ans
done
echo some character got entered "$ans"

#display the numbers upto 10 by entered number
echo enter the answer
read num
while [ $num -le 10 ]
do

echo $num
num=`expr $num + 1`
done

df - command used to display the diskspace usage

Write a script to accept a string and reverse the same.

echo enter a string
read str
len=`echo str | wc -c`
#len=`expr $len - 1`

while [ $len -gt 0 ]
do
n1=` echo $str | cut -c $len`
newstr=`echo $newstr$n1`
len=`expr $len - 1`
done
echo $newstr

calender
--------
cal -1 Display single month output. (This is the default.)

cal -3 Display prev/current/next month output.

First Day

2008-11-12T06:12:00.000-08:00

Unix treats all the peripherals as file.

- (hyphen ) is the standard input (keyboard)

> - standard output

< - input from file

File Types:
-----------
drwxrwxrwx

d- directory
l- soft links
c- charcter file - used in keyboard
b- blocks - used in printer

Help in Unix:
-------------

date --help
man date
info date

ls :
----

ls command displays total 108 - this number is total number of memory block get utilized to store data

ls -li
ls -lr reverse listing
ls -R recursive

ls -l file1 file2 >| outfile 2>| error - creates file with out and error

ls -l file1 file2 >> res 2>> error redirects with appending
ls -l file1 file2 1>> res 2>> error redirects with appending

ls [!abc]* other than starts with a,b,c character file will be displayed.

ls -l /dev device folder - stores character files

ls jk gh 2> /dev/null - do not display erro message

kernel version
ls /boot/vm* vmlinuz-2.6.9-55.EL here, 2.6 version, 9-55 patch, EL english

cat
----
cat < file1 > file2 - creates file2
cat file1 - > file3 creates file3 with content of file1 along with standard input
cat >> file2 - appending

Wild Cards:
----------
?- single character
* - multiple character
[] - range of character ex: [0-9] [abc] [!abc]
! - not

alias
-----
alias cls='clear' sets shortcut
unalias cls removes alias

^u clears the prompt
^c kills current prompt
^s stop displaying typing commands
^q to release
^r searches command in reverse

set -o environment settings to control activities like exit using ^D
set nu
set nonu

stty
-----
stty shows terminal type
stty -a shows commands in detail
stty eof \^a eof character changed from ^d to ^a
stty sane revert back to default setting

Hard links
----------
drawbacks 1. can't accessible across the file system , 2 can't create on directories

ln a a.link

426000 -rw-rw-r-- 2 user3 user3 15 Oct 21 11:16 a
426000 -rw-rw-r-- 2 user3 user3 15 Oct 21 11:16 a.link

rm a.link removes the link

ln file1 file2 folder2/ creates file1, file2 links in folder2

ln folder2 /opt/mm - cross device error will be produced.

Softlinks / Symbolic Links
---------------------------
Make use of inode table, same number also shared with the files.

ln -s folder2 folder2soft

ls -il folder2 folder2soft

26010 lrwxrwxrwx 1 user3 user3 7 Oct 21 12:10 folder2soft -> folder2

folder2:
total 16
426001 -rw-rw-r-- 2 user3 user3 6 Oct 21 11:18 file1
426004 -rw-rw-r-- 2 user3 user3 6 Oct 21 11:18 file2

Here first line says size as number of characters of actual folder. here it is 7.

date
----
date +%H
date +%T
date +%D
date +%d-%m-%y
date +%H-%M-%S
date +%d-%b-%y

who
---
whoami show my login name
who who are all there
who am i show my terminal details

Default Folders in unix
-----------------------
bin - normal executable files kept here
sbin - super user executables
dev - device files
var - holds dynamic information - especifically logs
etc - configuration files
home - for user accounts
proc - which holds all the process running on the system
boot - kernal image directory

To display system information
-----------------------------
uname -m i686
uname -s Linux
uname -o GNU/Linux
uname -r 2.6.9-55.EL

tty to show terminal details

who > /dev/pts/1 write in terminal 1 console
who > /dev/tty list tty with login details

Editors
-------
Line editors - ed, ex, sed
Screen editors - vi, vim, emacs

vi works with three mode - command, insert, colon / last line mode / esc mode / ex mode

To set custom commands for Vi permanently, create a file in home directory
.exrc file
set nu
abbr os Operating Systems
map ^W :w^M

Pipes and Filters
-----------------

Types of Commands available in unix - 1. Filters, 2. no std in , but stdout. 3. no stdin and no std out

wc -c -l -w

who | wc -l

head -3 /etc/passwd
tail -5 /etc/passwd

tail +3 /etc/passwd | head -1

cut -c 1 displays first character of each line

cut -c 2-5 displays 2 to 5 character in each line

cut -f 2 displays the second field of the input

cut -f3 -d ":" /etc/passwd

cut -f1,3 -d ":" /etc/passwd

cut -f1 -d ":" /etc/passwd >file1
cut -f7 -d ":" /etc/passwd >file2
paste file1 file2
paste -d "+" file1 file2

Translate sequencing of characters
----------------------------------
tr a A
tr "a-z" "A-Z" translates lowercase to uppercase
tr abc 123 converts a to 1, similarly for bc characters 2 and 3 will be printed
tr "a-z" "0-9" converts a to 1 and similarly remaining characters
tr -d "a-z" removes lowercase characters
tr -d "a-zA-Z" removes all the alphabets

tr -s "a" quising multiple sequence occurence of a character into one character
tr -s "a-z"

ls -l | tr -s " " | cut -f5,9 -d " " | tr -s " " "\t\t" | tr a-z A-Z cut 5,9 th coulumn and display with tab space.

tee
----

Command used to redirect to file and also for next pipe.
ls -l | tr -s " " | cut -f5,9 -d " " | tee filename | tr -s " " "\t\t" | tr a-z A-Z

sort
----

sort -r
sort -n
ls -l | sort -k 5 -n
ls -l | sort -k 5 -n -o outfile sort data stored back in outfilename

unique
------
sort -n | uniq quises adjacent duplicate value

ls -l | tr -s " " | cut -f5 -d " " | sort -n | uniq

uniq -d shows duplicate item
uniq -c shows number of time it got repeated
uniq -u shows only unique items

who | tr -s " " | cut -f1 -d " " | sort | uniq | wc -l

grep
----
grep [options] "pattern" [filenames]

grep -c count the number of lines

grep -e "root" -e "halt" /etc/passwd Specify multiple patterns

cat pat
root
halt

grep -f pat /etc/passwd does same grep command

grep -f pat -e "shutdown" /etc/passwd takes pattern from pat and commandline

Regular Expressions: metacharacters
--------------------
1. Basic regular expressions(BRE)
2. Extented regular expressions(ERE)

BRE:

grep "^root" /etc/passwd ^ - beginning of the line with root
grep "^[aeiou]" /etc/passwd ^ - beginning of the line with a,e,i,o,u
grep "bash$" /etc/passwd $ - end of the line with bash
grep "[0-9]$" line starts with number
grep "^[0-9][0-9]$" anything with only two digit numbers
grep "^$" shows any line is empty
grep "^[^a-z]" other than lower case letter lines will be displayed
grep "^[a^z]" line starts with any of a , ^ , or z will be displayed
grep "a.t" line contains character sequence beginning with a and ends with t, in between any character can present
grep "a\.t" line contains a.t
grep "a[.]t" line contains a.t
grep "[-+/*]" display only arithmatic character containing lines

. (operator) one occurence of any character
? (operator) zero or one occurence of previous expression
* (operator) zero or more occurences of previous regular expression

grep "^aaa*" line starts with minimum of two a characters
grep "^aaa*$" line which has exactly minimum of two aa or more a characters
grep "^aaa.*$" line with minimum of three a characters and any characters with any number of occurences

ERE:
Bar operators |

egrep or grep -E

grep -E "linux|freebsd|unix" - multiple pattern with egrep

egrep "bell lab(s|aratories)" - it matches bell labs and bell labaratories

+ (operator) one or more occurences

egrep "^aaa+$" line starts with aaa and with one or more characters

grep "\" gets the exact word
grep "away\>" gets the word ending with away
grep "^[0-9]\{4\}$" madates the numbers with 4 digit
grep "^[a-z]\{4,\}$" madates atleast four characters
grep "^.\{4,\}$" any of four or more characters
grep "^[a-z]\{4,10\}$" any of the word with minimum of four and maximum of 10 characters

grep -E "(^| )[:/(]?book[s]?[./)/:/?]?" book