Redirecting operators in Linux

 In order to understand about Redirecting operators in Linux we should know how we communicate with a computer. When we are communicating with a computer there should be a way to do it and this is achieved by STDIN (0), STDOUT (1), STDERR (2) file descriptors. There are numbers assigned to these file descriptors as shown below.

STDIN --> 0 
STDOUT  --> 1 
STDERR --> 2

We will see what actually these redirection operators in detail.

STDIN: stands for STandarD INput. By using this we can give an input to the computer to do some task. Whatever device we used to give input to a computer will come under STDIN.

A STDIN can be

A keyboard 
A mouse 
A scanner 
A floppy 
A CD/DVD ROM 
A touch screen 
A barcode reader 
Or a card reader

STDOUT: The abbreviation is StandarD OUTput. By using this we can see the output from a computer. Whatever device we used to get the output from a computer will come under this STDOUT.

A STDOUT can be

A monitor 
A speaker 
Or a printer

STDERR: This is abbreviated as STandarD ERRor. By using this, A computer can communicate with user to give him warning/error etc that something went wrong. A STDERR can be

A monitor 
A printer 
A log file 
A LED indicator 
Or a speaker

Why we require redirecting operators?

 We require redirecting operators in some situations where our standard communication will not meet our requirement. For example sometimes we want to move an error which is popping on a screen to a file for future reference. So at that time we can use these redirecting operators to change the default way of communication between users and computers.

In Linux there are many redirecting operators (File descriptors) as shown below

Basic redirecting operators

 > --Output redirecting operator 
< --Input redirecting operator 
>> --Output appending operator. 
<< --Input appending operator (no significant practical use) 
| --Pipe operator, redirects output of a command to next command.

Advanced redirecting operators

1> --Standard Output redirect operator (same as >) 
1>> --Standard output append redirect operator (same as >>) 
2> --Standard Error redirector 
2>> --Standard error redirector with append 
&> --Standard Output and Error 
tee command --To redirect output to the screen as well as a file 
xargs command --To feed output of command as input files to other command 
- -- (hyphen or Dash) redirection from standard input/output

We will see above operators with some examples on how to use them. We are giving much details on << operator as it is rarely used and 1>, 1>> as these operators are equal to >, >>.

Example1: Redirect output of fdisk command to a file for future reference.

fdisk -l > file1.txt

Example2: Search if Mitch username is present in /etc/passwd file or not

 grep mitch < /etc/passwd

Example3: Use output redirect append operators to redirect fdisk command output once again so that I can see both the previous output and present one in single file.

 fdisk -l >> file1.txt

Note: Not discussing input append redirect operator as its very rarely used in practice.

Example4: How can I count all the files with in present working directory

ls -l | wc -l

Example5: Redirect error of a command to a file and output to the screen.

ls -hrtfd 2> file3.txt

Exampl6: Redirect and append error to a file and output to the screen.

ls -wertdf 2>> file4.txt

Example7: Redirect both output and error to a file

ls -ewrdter &> file5.txt

Note: &> and 2>&1 is one and the same.

tee command: tee(T) command is a special command which will redirect the output to a file as well as screen at same time, please consider it as capital T when it works.

Example8: Redirect fdisk command to a file as well as on the screen at a time.

For example if you want to execute fdisk –l command want to see output of the command then execute one more command to redirect output to a file for future reference. We can do that with following commands

 fdisk –l

fdisk –l > fdskout.txt

But with tee command you no need to execute two command. Below is a single command for the above fdisk example.

 fdisk -l | tee file6.txt

xargs command: xargs command is one more special command which will redirect output of a command and feed it as input to other commands.

Example9: List all the files which contain .sh file extension and within them list all the files which contains bash in them.

ls -l *.sh | grep bash

If I execute above command, this will not give all the files containing bash but will give file name containing bash. For this type of requirements we have to use xargs. Which will consider previous command output as file names and feed them to next command to args as shown below?

ls -l *.sh | xargs grep bash

- (hyphen or dash) Operator: This operator will act as both input as well as output redirect operator. I did not find much usage on using this operator as input redirecting operator. There are numerous examples for using this as output redirect operation such as with tar, cut commands etc. I have taken following examples from tldp.org.

Example10: Search for a string in a file1.txt file and feed that to diff command to see the difference between abc.txt file and this new temporary file (file2.txt) files. Normally we do that as follows..

grep xyz /temp/loc/file1.txt > /temp/loc/file2.txt

Now compare file2.txt with abc file.

diff /temp/loc/abc.txt /temp/loc/file2.txt

This can be done in single command using – operator

grep xyz /temp/loc/file1.txt | diff /temp/loc/abc.txt –

Please show some love by sharing this post on facebook, linkedin etc, if it helped you. Please comment your thoughts on this if you know any other different redirect operators.

History of DNS server

How DNS server evolved?

To understand why we require DNS server we have to go back to initial stages of networking the computers around late 1960’s and early 1970’s where it all started in Universities(especially Barkley University). Initially there is single computer for university due to cost and other factors, then to multiple systems per university. When computers started increasing exponentially in universities at this time people are forced to share documents/printers service etc of one system to other systems for maximizing the resources. This made people to inter network between the systems in labs so there will be effective utilization of resources attached to system. This made many people to remember the servers with numbers. So they started giving numbers to servers in Universities. So initially this worked fine to remember numbers, but when computers started growing faster they felt to give names instead of numbers(As humans prefer names more than numbers for easy remembrance).

Hosts File evolution(/etc/hosts or c:/windows/system32/etc/HOSTS):

To make it easy to communicate with different systems they came up with hosts file concept. Every system in the lab should have hosts file and it should contain name to number conversation table.
So if I say one lab is having 10 machines each machine should have 10 entries for these servers as show below.
Hosts file content:
Labsys1 1
Labsys2 2
Labsys3 3
..
..
Labsys10 10
So becomes easy to remember by users.

Disadvantages of hosts file?

One disadvantage from this hosts file is if we want to add 11th machine in the lab all the hosts files on each system to should be modified to add this host details. Ok, it’s just 10 machines but consider labs having 100’s or thousands of servers. When a new machine is added to the network this machine entry should be added to these thousands of machines which a heavy job for the admin.

What’s the solution?

Go for a centralized monitoring/configuration system where every machine will point their Name quires to a centralized server. This machine should serve all the queries. At this point DNS (Domain Name Server/service) came in to existence.

Disadvantages of initial DNS server?
When DNS server is implemented again it’s a mapping between numbers to host names and vice versa as shown below.
For Berkeley servers:
Server1 123
Lab2 345
Pas23 89

If you see there is no proper naming convention and its bit difficult to find where one server is located in given University.

FQDN is the solution

Researcher came with FQDN (Fully Qualified Domain Name) convention so that’s it’s easy to remember and easy to point where the server located.
FQDN is a hierarchical and tree like structure for naming convention which contain below details.
1) Hostname –Name of the server
2) Domain name – Defines a realm administrative authority or control on a group of machines.
3) Top Level Domains (TLD) – ie .com, .in, co.uk, co.us etc.

Some FAQ:

What are root name servers?
DNS is a hierarchical in structure where if a server is unable to give answer to host requests, it will send this to its superior servers and so on. This will reach to the upper levels to the initial master servers which are called as root name server which can serve all the answer queries if the host exists in internet.

Why there are only 13 root name servers?
This is due to UDP porotocal size which stores DNS information.

How many TOP level and Country specific domains are there?
There are 20 generic TLD’s (gTLD) and 248 country specific TLD’s(ccTLD) are there.

Stay tuned to our next post on “How name resolution works?”

This is a small post on improving your productivity when working with BASH shell. BASH shell is having many inbuilt capabilities such as history, command chaining and BASH shortcut. In this post we will see what are BASH shortcuts and their usage.

Ctrl+a
Bring back the courser to start of the bash prompt


Ctrl+c
Cancel the command before executing it


Ctrl+d
Logout from the Shell


Ctrl+e
Move the courser to end of the command


Ctrl+l
Clear the screen


Ctrl+r
Search the history reverse order


Ctrl+p
go to previous command in history


Ctrl+n
Go to next command in history


Ctrl+s
Suspend terminal output(Useful for long out commands )


Ctrl+z
suspend the command/send command running to background

There are some other shortcuts are there which are not frequently used.

We are very much happy to announce that we started posting on DNS client and Server stuff on our blog for this month(Jan 2011). We already have some minor existing stuff related to DNS and troubleshooting it. Below are quick links to our existing DNS Server/Client stuff

http://www.linuxnix.com/tag/dns-servers

http://www.linuxnix.com/tag/dns

Some of the DNS topics which are going to cover in this month are as follows. Please feel free to add requirements for DNS concepts which are not listed here  in either comments section or DNS Stuff Request Form.

Basics of DNS server

• 1)DNS Evolution
• 2)How Name resolution works?
• 3)How can I Setup DNS Lookup in Linux?
• 4)Linux / UNIX set the DNS from the GUI
• 5)How to set the DNS from the CLI in Linux?
• 6)How can we Find Out What My DNS Servers Address on client side?
• 7)DNS client configuration(Ubuntu/Windows/Redhat/Solaries/AIX/HPUX etc)
• 8)What is NSORDER(DNS search order) and how to set it up in DNS Client?
• 9)Different DNS Records and their usages.
• 10)What is DNS server port number?
• 11)On what protocol DNS wroks?
• 12)What are DNS ROOT servers?

DNS server configuration

• 1)Installing DNS server
• 2)DNS named.conf, Zone files explained
• 3)Upgrading BIND DNS server software
• 4)DNS server implementation
• 5)DNS Master-Slave implementation
• 6)DDNS configuration(Requires DHCP)
• 7)BLDNS server configuration(For Black listing spam Domains from  dsbl.org / spamhaus.org database)
• 8)Round Robin DNS server configuration
• 9)DNS cache server configuration
• 10)How to update named.conf file with out restarting DNS service?
• 11)How to update Zone files with out restarting DNS service?

DNS server troubleshooting

• 1)DNS logging
• 2)Named.conf file error checking
• 3)Zone file error checking
• 4)tcpdump command
• 5)DNS tool – DIG
• 6)DNS tool – nslookup
• 7)DNS tool – host
• 8)DNS tool – dnstop
• 9)DNS tool – fpdns
• 10)How to test or check reverse DNS
• 11)Linux Date Command: Convert Named Stats Dump Date
• 12)How to debug DNS logs using rndc query
• 13)How can i check on what port my DNS is working
• 14)DNS errors

Improving speed of DNS

• 1)DNS response How to
• 2)Disable Dynamic Updates in DNS
• 3)How to Stop Recursion DNS queries Under Linux ?
• 4)How to Force BIND DNS Server to take full advantage of system resources?
• 5)How To Flush Linux BIND DNS Cache?
• 6)nsupdate Dynamic DNS Update Utility for Linux?

Securing DNS

• 1)How to hide out DNS server version
• 2)Find out DNS Server Version With DNS Server Fingeprinting tool
• 3)How can I Find Out If My DNS Server is Free From DNS Cache Poisoning?
• 4)Find the DNS Authority Record for a DNS Domain (SOA)?
• 5)Linux Iptables block or open DNS / bind service port 53
• 6)Cache poisioning issue and how to resolve it
• 7)Configuring DNS in DMZ for security.
• 8)What is DNS spoofing?

Good places to find more info on DNS

• 1)Book review
• 2)Advanced DNS server configurations
• 3)Online tools

Please feel free to contact us if you want to add up something or have a word to say at comments..!

Dear Friends/Readers,

I take this opportunity to convey our sincere thanks to each one of you personally for your support and co-operation for several years. I request you to continue the same support and refer more people to our blog to grow bigger. We are very happy and personally congratulate each one of you for yourr successful career and continuous achievements.

We personally request each one of you to give us your valuable suggestions and feed back for successful growth of our blog to reach new heights in new year. And also request you to send your comments (+’s / -’s) about  service / response, so that I can fix errors and serve you better. You response is highly appreciated.

And once again have a wonderfull and prosperious new year 2012.

Thanks,
A.Surendra Kumar

Founder, Mounix Systems, Hyderabad.
Linux: Fast, friendly, flexible and …. free!
Support Open source.
<*,)}}+<
Only dead fish go with the flow!
http://www.linuxnix.com

What is a sticky Bit and how to set it in Linux?

This is next to SGID in our ongoing File and Folder permissions in Linux. We already discussed about CHMOD, UMASK, CHOWN, CHGRP SGID and SUID File and folder permissions etc in our previous posts. In this post we will see

What is Sticky Bit?

Why we require Sticky Bit?

Where we are going to implement Sticky Bit?

How to implement Sticky Bit in Linux?

What is Sticky Bit?

Sticky Bit is used mainly on folders in order to avoid deletion of a folder and its content by other user though he is having write permissions. If Sticky bit is enabled on a folder, the folder is deleted by only owner of the folder and super user(root). This is a security measure to suppress deletion of critical folders where it is having full permissions by others.

Learn Sticky Bit with examples:

Example: Create a project(A folder) where people will try to dump files for sharing, but they should not delete the files created by other users.

How can I setup Sticky Bit for a Folder?

Sticky Bit can be set in two ways

1) Symbolic way (t,represents sticky bit)

2) Numerical/octal way (1, Sticky Bit bit as value 1)

Use chmod command to set Sticky Bit on Folder: /opt/dump/

Symbolic way:

chmod o+t /opt/dump/
or 
chmod +t /opt/dump/

Let me explain above command we are setting Sticky Bit(+t) to folder /opt/dump by using chmod command.

Numerical way:

chmod 1757 /opt/dump/

Here in 1757, 1 indicates Sticky Bit set, 7 for full permissions for owner, 5 for write and execute permissions for group, and ful permissions for others.

Checking if a folder is set with Sticky Bit or not?

Use ls –l to check if the x in others permissions field is replaced by t or T

For example: /opt/dump/ listing before and after Sticky Bit set

Before Sticky Bit set:

ls -l

total 8

-rwxr-xrwx 1 xyz xyzgroup 148 Dec 22 03:46 /opt/dump/

After Sticky Bit set:

ls -l

total 8

-rwxr-xrwt 1 xyz xyzgroup 148 Dec 22 03:46 /opt/dump/

Some FAQ’s related to Sticky Bit:

Now sticky bit is set, lets check if user “temp” can delete this folder

$ rm -rf /opt/dump

rm: cannot remove `/opt/dump’: Operation not permitted

$ ls -l /opt

total 8

drwxrwxrwt 4 xyz xyzgroup 4096 2012-01-01 17:37 dump

$

if you observe he is unable to delete and even xyz user creted any files in dump folder, temp user can not delete it. But xyz user can crete and delte files in dump folder.

I am seeing “T” ie Capital s in the file permissions, what’s that?

After setting Sticky Bit to a file/folder if you see ‘T’ in the file permission area that indicates that the file/folder does not have executable permissions for others on that particular file/folder.

Sticky bit without Executable permissions:

so if you want executable permissions too, apply executable permissions to the file.

chmod o+x /opt/dump/

output: -rwxr-xrwt 1 xyz xyzgroup 0 Dec 5 11:24 /opt/dump/

Sticky bit with Executable permissions:

you should see a smaller ‘s’ in the executable permission position.

How can I find all the Sticky Bit set files in Linux/Unix.

find / -perm +1000

The above find command will check all the files which is set with Sticky Bit bit(1000).

Can I set Sticky Bit for files?

Yes, but most of the time its not required.

How can I remove Sticky Bit bit on a file/folder?

chmod o-t /opt/dump/

Post your thoughts on this.

What is SGID and how to set SGID in Linux/Unix?

This is next to SUID in our ongoing File and Folder permissions in Linux. We already discussed about CHMOD, UMASK, CHOWN, CHGRP and SUID File and folder permissions etc in our previous posts. In this post we will see

What is SGID?

Why we require SGID?

Where we are going to implement SGID?

How to implement SGID in Linux?

What is SGID?

SGID (Set Group ID up on execution) is a special type of file permissions given to a file/folder. Normally in Linux/Unix when a program runs, it inherits access permissions from the logged in user. SGID is defined as giving temporary permissions to a user to run a program/file with the permissions of the file group permissions to become member of that group to execute the file. In simple words users will get file Group’s permissions when executing a Folder/file/program/command.

SGID is similar to SUID. The difference between both is that SUID assumes owner of the file permissions and SGID assumes group’s permissions when executing a file instead of logged in user inherit permissions.

 Learn SGID with examples:

Example: Linux Group quota implementation

When implementing Linux Group quota for group of people SGID plays an important role in checking the quota timer. SGID bit set on folder is used to change their inherit permissions to group’s permissions to make it as single user who is dumping data. So that group members whoever dumps the data the data will be written with group permissions and inturn quota will be reduced centrally for all the users. For clear understanding of this you have to implement group quota from the above link. Without implementation of SGID the quota will not be effective.

How can I setup SGID for a file?

SGID can be set in two ways

1) Symbolic way (s)

2) Numerical/octal way (2, SGID bit as value 2)

Use chmod command to set SGID on file: file1.txt

Symbolic way:

chmod g+s file1.txt

Let me explain above command we are setting SGID(+s) to group who owns this file.

Numerical way:

chmod 2750 file1.txt

Here in 2750, 2 indicates SGID bitset, 7 for full permissions for owner, 5 for write and execute permissions for group, and no permissions for others.

How can I check if a file is set with SGID bit or not?

Use ls –l to check if the x in group permissions field is replaced by s or S

For example: file1.txt listing before and after SGID set

Before SGID set:

ls -l

total 8

-rwxr--r-- 1 xyz xyzgroup 148 Dec 22 03:46 file1.txt

After SGID set:

ls -l

total 8

-rwxr-sr-- 1 xyz xyzgroup 148 Dec 22 03:46 file1.txt

Some FAQ’s related to SGID:

Where is SUID used?

1) When implementing Linux group disk quota.

I am seeing “S” ie Capital s in the file permissions, what’s that?

After setting SUID or SGID to a file/folder if you see ‘S’ in the file permission area that indicates that the file/folder does not have executable permissions for that user or group on that particular file/folder.

chmod g+s file1.txt

output:
-rwxrwSr-x 1 surendra surendra 0 Dec 27 11:24 file1.txt

so if you want executable permissions too, apply executable permissions to the file.

chmod g+x file1.txt

output:
-rwxrwsr-x 1 surendra surendra 0 Dec 5 11:24 file1.txt

you should see a smaller ‘s’ in the executable permission position.

How can I find all the SGID set files in Linux/Unix.

find / -perm +2000

The above find command will check all the files which is set with SGID bit(2000).

Can I set SGID for folders?

Yes, you can if it’s required (you should remember one thing, that Linux treats everything as a file)

How can I remove SGID bit on a file/folder?

chmod g-s file1.txt

Post your thoughts on this.

What is SUID and how to set SUID in Linux/Unix?

There are some other special permission apart from the normal file permissions read, write and execute. They are SUID, SGID, Sticky Bit, ACL’s and SELinux etc for granular file/folder management by administrator. In this post we will see

1)What’s SUID?
2)How to set SUID?
3)Where to us SUID?

What is SUID and how to set it in Linux?

SUID (Set owner User ID up on execution) is a special type of file permissions given to a file. Normally in Linux/Unix when a program runs, it inherits access permissions from the logged in user. SUID is defined as giving temporary permissions to a user to run a program/file with the permissions of the file owner rather that the user who is running it. In simple words users will get file owner’s permissions as well as their UID and GID when executing a file/program/command.

The above sentence is bit tricky and should be explained in depth with examples.

Learn SUID with examples:

Example1: passwd command

When we try to change our password we will use passwd command which is owned by root as shown below. This passwd command file will try to edit some system config files such as /etc/passwd, /etc/shadow etc when we try to change our password. These files cannot be opened or viewed by normal user only root user will have permissions. So if we try to remove SUID and give full permissions to this passwd command file it cannot open other files such as /etc/shadow file to update the changes and we will get permission denied error or some other error when tried to execute passwd command. So passwd command is set with SUID to give root user permissions to normal user so that it can update /etc/shadow and other files.

Example2: ping command

Similarly if we take ping command, when we have to execute this command internally it should open socket files and open ports in order to send IP packets and receive IP packets to remote server. Normal users don’t have permissions to open socket files and open ports. So SUID bit is set on this file/command so that whoever executes this will get owner (Root user’s) permissions to them when executing this command. So when this command start executing it will inherits root user permissions to this normal user and opens require socket files and ports.

Example3: crontab and at command.

When scheduling the jobs by using crontab or at command it is obious to edit some of the crontab related configuration files located in /etc which are not writable for normal users. So crontab/at commands are set with SUID in-order to write some data.

How can I setup SUID for a file?

SUID can be set in two ways

1) Symbolic way(s, Stands for Set) 2) Numerical/octal way(4)

Use chmod command to set SUID on file: file1.txt

Symbolic way:

chmod u+s file1.txt

Here owner permission execute bit is set to SUID with +s

Numerical way:

chmod 4750 file1.txt

Here in 4750, 4 indicates SUID bitset, 7 for full permissions for owner, 5 for write and execute permissions for group, and no permissions for others.

How can I check if a file is set with SUID bit or not?

Use ls –l to check if the x in owner permissions field is replaced by s or S

For example: file1.txt listing before and after SUID set

Before SUID set:

ls -l

total 8

-rwxr--r-- 1 xyz xyzgroup 148 Dec 22 03:46 file1.txt

After SUID set:

ls -l

total 8

-rwsr--r-- 1 xyz xyzgroup 148 Dec 22 03:46 file1.txt

Some FAQ’s related to SUID:

A) Where is SUID used?

1) Where root login is required to execute some commands/programs/scripts.

2) Where you dont want to give credentials of a perticular user and but want to run some programs as the owner.

3) Where you dont want to use sudo command but want to give execute permission for a file/script etc.

B) I am seeing “S” I.e. Capital “s” in the file permissions, what’s that?

After setting SUID to a file/folder if you see ‘S’ in the file permission area that indicates that the file/folder does not have executable permissions for that user on that particular file/folder.

For example see below example

chmod u+s file1.txt

ls -l
-rwSrwxr-x 1 surendra surendra 0 Dec 27 11:24 file1.txt

If you want to convert this S to s then add executable permissions to this file as show below

chmod u+x file1.txt
ls -l
-rwsrwxr-x 1 surendra surendra 0 Dec 5 11:24 file1.txt

you should see a smaller ‘s’ in the executable permission position now.

SUID with execute permissions:

SUID with out execute permissions:

C) How can I find all the SUID set files in Linux/Unix.

find / -perm +4000

The above find command will check all the files which is set with SUID bit(4000).

D) Can I set SUID for folders?

Yes, you can if its required(you should remember one thing, that Linux treats everything as a file)

E) What is SUID numerical value?

It has the value 4 for SUID.

Please comment your thoughts about SUID usage in your company.

What is UMASK and how to define it in Linux?

UMASK(User Mask or User file creation MASK) is the default permission or base permissions given when a new file(even folder too, as Linux treats everything as files) is created on a Linux machine. Most of the Linux distros give 022(0022) as default UMASK. In other words, It is a system default permissions for newly created files/folders in the machine.

The minimum and maximum UMASK value for a folder is 000 and 777
 The minimum and maximum UMASK value for a file is 000 and 666

 0 --Full permissions(Read, Write, Execute)
 1 --Write and read
 2 --Read and execute
 3 --Read only
 4 --Write and execute
 5 --Write only
 6 --Execute onlyadminadmin
 7 --No permissions

Not(027) = 750

777 AND 750 = 750

777 - 027 = 750

Not(027) = 750

666 AND 750 = 640

110 + 111 = 110(6)
110 + 101 = 100(4)
110 + 000 = 000(0)

umask

0022

umask 027

man umask

info umask

CHGRP(CHange GRouP) is one more command which is useful to change group associated to a file/folder from one group to other in a Linux box. This is sister command to chown which is used to change owner of the file/folder as well as group name associated with that file.

chgrp command syntax:

 chgrp options groupname file/foldername

Learn chgrp with examples:

Example1: Change group name:sales of a file to other group name:hrgroup.

chgrp hrgroup file1

Example2: Give access permissions to a command so that the command can be executed by all users belonging to apache-admins

chgrp apache-admins /etc/init.d/httpd

Example3: Change group ownership all the files located in /var/apache to group:apache

chgrp -R apache /var/apache

Example4:Change group ownership forcefully

chgrp -f apache /var/apache

Difference between chown and chgrp

1) chown command is used to change ownership as well as group name associated to different one, where as chgrp can change only group associated to it.

2) Many people say that regular user only able to use chgrp to change the group if the user belongs to them. But it’s not true a user can use chown and chgrp irrespective to change group to one of their group because chown is located in /bin folder so every can use it with some limited access.

 Usages of chgrp command:

1)Used to change group ownership from one group to other group for a file/folder

2)As a security measure if you want to give permissions to a command to some group you can use this command.