FREE COMPUTER SOLUTIONS

Top 20 OpenSSH Server Best Security Practices

2011-03-19T23:08:00.000+05:30

OpenSSH is the implementation of the SSH protocol. OpenSSH is recommended for remote login, making backups, remote file transfer via scp or sftp, and much more. SSH is perfect to keep confidentiality and integrity for data exchanged between two networks and systems. However, the main advantage is server authentication, through the use of public key cryptography. From time to time there are rumors about OpenSSH zero day exploit. Here are a few things you need to tweak in order to improve OpenSSH server security.

Default Config Files and SSH Port

/etc/ssh/sshd_config - OpenSSH server configuration file.
/etc/ssh/ssh_config - OpenSSH client configuration file.
~/.ssh/ - Users ssh configuration directory.
~/.ssh/authorized_keys or ~/.ssh/authorized_keys - Lists the public keys (RSA or DSA) that can be used to log into the user’s account
/etc/nologin - If this file exists, sshd refuses to let anyone except root log in.
/etc/hosts.allow and /etc/hosts.deny : Access controls lists that should be enforced by tcp-wrappers are defined here.
SSH default port : TCP 22

SSH Session in Action

#1: Disable OpenSSH Server

Workstations and laptop can work without OpenSSH server. If you need not to provide the remote login and file transfer capabilities of SSH, disable and remove the SSHD server. CentOS / RHEL / Fedora Linux user can disable and remove openssh-server with yum command:
# chkconfig sshd off
# yum erase openssh-server
Debian / Ubuntu Linux user can disable and remove the same with apt-get command:
# apt-get remove openssh-server
You may need to update your iptables script to remove ssh exception rule. Under CentOS / RHEL / Fedora edit the files /etc/sysconfig/iptables and /etc/sysconfig/ip6tables. Once done restart iptables service:
# service iptables restart
# service ip6tables restart

#2: Only Use SSH Protocol 2

SSH protocol version 1 (SSH-1) has man-in-the-middle attacks problems and security vulnerabilities. SSH-1 is obsolete and should be avoided at all cost. Open sshd_config file and make sure the following line exists:

Protocol 2

#3: Limit Users' SSH Access

By default all systems user can login via SSH using their password or public key. Sometime you create UNIX / Linux user account for ftp or email purpose. However, those user can login to system using ssh. They will have full access to system tools including compilers and scripting languages such as Perl, Python which can open network ports and do many other fancy things. One of my client has really outdated php script and an attacker was able to create a new account on the system via a php script. However, attacker failed to get into box via ssh because it wasn't in AllowUsers.

Only allow root, vivek and jerry user to use the system via SSH, add the following to sshd_config:

AllowUsers root vivek jerry

Alternatively, you can allow all users to login via SSH but deny only a few users, with the following line:

DenyUsers saroj anjali foo

You can also configure Linux PAM allows or deny login via the sshd server. You can allow list of group name to access or deny access to the ssh.

#4: Configure Idle Log Out Timeout Interval

User can login to server via ssh and you can set an idel timeout interval to avoid unattended ssh session. Open sshd_config and make sure following values are configured:

ClientAliveInterval 300

ClientAliveCountMax 0

You are setting an idle timeout interval in seconds (300 secs = 5 minutes). After this interval has passed, the idle user will be automatically kicked out (read as logged out). See how to automatically log BASH / TCSH / SSH users out after a period of inactivity for more details.

#5: Disable .rhosts Files

Don't read the user's ~/.rhosts and ~/.shosts files. Update sshd_config with the following settings:

IgnoreRhosts yes

SSH can emulate the behavior of the obsolete rsh command, just disable insecure access via RSH.

#6: Disable Host-Based Authentication

To disable host-based authentication, update sshd_config with the following option:

HostbasedAuthentication no

#7: Disable root Login via SSH

There is no need to login as root via ssh over a network. Normal users can use su or sudo (recommended) to gain root level access. This also make sure you get full auditing information about who ran privileged commands on the system via sudo. To disable root login via SSH, update sshd_config with the following line:

PermitRootLogin no

However, bob made excellent point:

Saying "don't login as root" is h******t. It stems from the days when people sniffed the first packets of sessions so logging in as yourself and su-ing decreased the chance an attacker would see the root pw, and decreast the chance you got spoofed as to your telnet host target, You'd get your password spoofed but not root's pw. Gimme a break. this is 2005 - We have ssh, used properly it's secure. used improperly none of this 1989 will make a damn bit of difference. –Bob

#8: Enable a Warning Banner

Set a warning banner by updating sshd_config with the following line:

Banner /etc/issue

Sample /etc/issue file:

----------------------------------------------------------------------------------------------

You are accessing a XYZ Government (XYZG) Information System (IS) that is provided for authorized use only.

By using this IS (which includes any device attached to this IS), you consent to the following conditions:

+ The XYZG routinely intercepts and monitors communications on this IS for purposes including, but not limited to,

penetration testing, COMSEC monitoring, network operations and defense, personnel misconduct (PM),

law enforcement (LE), and counterintelligence (CI) investigations.

+ At any time, the XYZG may inspect and seize data stored on this IS.

+ Communications using, or data stored on, this IS are not private, are subject to routine monitoring,

interception, and search, and may be disclosed or used for any XYZG authorized purpose.

+ This IS includes security measures (e.g., authentication and access controls) to protect XYZG interests--not

for your personal benefit or privacy.

+ Notwithstanding the above, using this IS does not constitute consent to PM, LE or CI investigative searching

or monitoring of the content of privileged communications, or work product, related to personal representation

or services by attorneys, psychotherapists, or clergy, and their assistants. Such communications and work

product are private and confidential. See User Agreement for details.

----------------------------------------------------------------------------------------------

Above is standard sample, consult your legal team for exact user agreement and legal notice details.

#8: Firewall SSH Port # 22

You need to firewall ssh port # 22 by updating iptables or pf firewall configurations. Usually, OpenSSH server must only accept connections from your LAN or other remote WAN sites only.

Netfilter (Iptables) Configuration

Update /etc/sysconfig/iptables (Redhat and friends specific file) to accept connection only from 192.168.1.0/24 and 202.54.1.5/29, enter:

-A RH-Firewall-1-INPUT -s 192.168.1.0/24 -m state --state NEW -p tcp --dport 22 -j ACCEPT

-A RH-Firewall-1-INPUT -s 202.54.1.5/29 -m state --state NEW -p tcp --dport 22 -j ACCEPT

If you've dual stacked sshd with IPv6, edit /etc/sysconfig/ip6tables (Redhat and friends specific file), enter:

-A RH-Firewall-1-INPUT -s ipv6network::/ipv6mask -m tcp -p tcp --dport 22 -j ACCEPT

Replace ipv6network::/ipv6mask with actual IPv6 ranges.

*BSD PF Firewall Configuration

If you are using PF firewall update /etc/pf.conf as follows:

pass in on $ext_if inet proto tcp from {192.168.1.0/24, 202.54.1.5/29} to $ssh_server_ip port ssh flags S/SA synproxy state

#9: Change SSH Port and Limit IP Binding

By default SSH listen to all available interfaces and IP address on the system. Limit ssh port binding and change ssh port (by default brute forcing scripts only try to connects to port # 22). To bind to 192.168.1.5 and 202.54.1.5 IPs and to port 300, add or correct the following line:

Port 300

ListenAddress 192.168.1.5

ListenAddress 202.54.1.5

A better approach to use proactive approaches scripts such as fail2ban or denyhosts (see below).

#10: Use Strong SSH Passwords and Passphrase

It cannot be stressed enough how important it is to use strong user passwords and passphrase for your keys. Brute force attack works because you use dictionary based passwords. You can force users to avoid passwords against a dictionary attack and use john the ripper tool to find out existing weak passwords. Here is a sample random password generator (put in your ~/.bashrc):

genpasswd() {

local l=$1

[ "$l" == "" ] && l=20

tr -dc A-Za-z0-9_ < /dev/urandom | head -c ${l} | xargs

}

Run it:
genpasswd 16
Output:

uw8CnDVMwC6vOKgW

#11: Use Public Key Based Authentication

Use public/private key pair with password protection for the private key. See how to use RSA and DSA key based authentication. Never ever use passphrase free key (passphrase key less) login.

#12: Use Keychain Based Authentication

keychain is a special bash script designed to make key-based authentication incredibly convenient and flexible. It offers various security benefits over passphrase-free keys. See how to setup and use keychain software.

#13: Chroot SSHD (Lock Down Users To Their Home Directories)

By default users are allowed to browse the server directories such as /etc/, /bin and so on. You can protect ssh, using os based chroot or use special tools such as rssh. With the release of OpenSSH 4.8p1 or 4.9p1, you no longer have to rely on third-party hacks such as rssh or complicated chroot(1) setups to lock users to their home directories. See this blog post about new ChrootDirectory directive to lock down users to their home directories.

#14: Use TCP Wrappers

TCP Wrapper is a host-based Networking ACL system, used to filter network access to Internet. OpenSSH does supports TCP wrappers. Just update your /etc/hosts.allow file as follows to allow SSH only from 192.168.1.2 172.16.23.12 :

sshd : 192.168.1.2 172.16.23.12

See this FAQ about setting and using TCP wrappers under Linux / Mac OS X and UNIX like operating systems.

#15: Disable Empty Passwords

You need to explicitly disallow remote login from accounts with empty passwords, update sshd_config with the following line:

PermitEmptyPasswords no

#16: Thwart SSH Crackers (Brute Force Attack)

Brute force is a method of defeating a cryptographic scheme by trying a large number of possibilities using a single or distributed computer network. To prevents brute force attacks against SSH, use the following softwares:

DenyHosts is a Python based security tool for SSH servers. It is intended to prevent brute force attacks on SSH servers by monitoring invalid login attempts in the authentication log and blocking the originating IP addresses.
Explains how to setup DenyHosts under RHEL / Fedora and CentOS Linux.
Fail2ban is a similar program that prevents brute force attacks against SSH.
security/sshguard-pf protect hosts from brute force attacks against ssh and other services using pf.
security/sshguard-ipfw protect hosts from brute force attacks against ssh and other services using ipfw.
security/sshguard-ipfilter protect hosts from brute force attacks against ssh and other services using ipfilter.
security/sshblock block abusive SSH login attempts.
security/sshit checks for SSH/FTP bruteforce and blocks given IPs.
BlockHosts Automatic blocking of abusive IP hosts.
Blacklist Get rid of those bruteforce attempts.
Brute Force Detection A modular shell script for parsing application logs and checking for authentication failures. It does this using a rules system where application specific options are stored including regular expressions for each unique auth format.
IPQ BDB filter May be considered as a fail2ban lite.

#17: Rate-limit Incoming Port # 22 Connections

Both netfilter and pf provides rate-limit option to perform simple throttling on incoming connections on port # 22.

Iptables Example

The following example will drop incoming connections which make more than 5 connection attempts upon port 22 within 60 seconds:

#!/bin/bash

inet_if=eth1

ssh_port=22

$IPT -I INPUT -p tcp --dport ${ssh_port} -i ${inet_if} -m state --state NEW -m recent --set

$IPT -I INPUT -p tcp --dport ${ssh_port} -i ${inet_if} -m state --state NEW -m recent --update --seconds 60 --hitcount 5 -j DROP

Call above script from your iptables scripts. Another config option:

$IPT -A INPUT -i ${inet_if} -p tcp --dport ${ssh_port} -m state --state NEW -m limit --limit 3/min --limit-burst 3 -j ACCEPT

$IPT -A INPUT -i ${inet_if} -p tcp --dport ${ssh_port} -m state --state ESTABLISHED -j ACCEPT

$IPT -A OUTPUT -o ${inet_if} -p tcp --sport ${ssh_port} -m state --state ESTABLISHED -j ACCEPT

# another one line example

# $IPT -A INPUT -i ${inet_if} -m state --state NEW,ESTABLISHED,RELATED -p tcp --dport 22 -m limit --limit 5/minute --limit-burst 5-j ACCEPT

See iptables man page for more details.

*BSD PF Example

The following will limits the maximum number of connections per source to 20 and rate limit the number of connections to 15 in a 5 second span. If anyone breaks our rules add them to our abusive_ips table and block them for making any further connections. Finally, flush keyword kills all states created by the matching rule which originate from the host which exceeds these limits.

sshd_server_ip="202.54.1.5"

table <abusive_ips> persist

block in quick from <abusive_ips>

pass in on $ext_if proto tcp to $sshd_server_ip port ssh flags S/SA keep state (max-src-conn 20, max-src-conn-rate 15/5, overload <abusive_ips> flush)

#18: Use Port Knocking

Port knocking is a method of externally opening ports on a firewall by generating a connection attempt on a set of prespecified closed ports. Once a correct sequence of connection attempts is received, the firewall rules are dynamically modified to allow the host which sent the connection attempts to connect over specific port(s). A sample port Knocking example for ssh using iptables:

$IPT -N stage1

$IPT -A stage1 -m recent --remove --name knock

$IPT -A stage1 -p tcp --dport 3456 -m recent --set --name knock2

$IPT -N stage2

$IPT -A stage2 -m recent --remove --name knock2

$IPT -A stage2 -p tcp --dport 2345 -m recent --set --name heaven

$IPT -N door

$IPT -A door -m recent --rcheck --seconds 5 --name knock2 -j stage2

$IPT -A door -m recent --rcheck --seconds 5 --name knock -j stage1

$IPT -A door -p tcp --dport 1234 -m recent --set --name knock

$IPT -A INPUT -m --state ESTABLISHED,RELATED -j ACCEPT

$IPT -A INPUT -p tcp --dport 22 -m recent --rcheck --seconds 5 --name heaven -j ACCEPT

$IPT -A INPUT -p tcp --syn -j doo

fwknop is an implementation that combines port knocking and passive OS fingerprinting.
Multiple-port knocking Netfilter/IPtables only implementation.

#19: Use Log Analyzer

Read your logs using logwatch or logcheck. These tools make your log reading life easier. It will go through your logs for a given period of time and make a report in the areas that you wish with the detail that you wish. Make sure LogLevel is set to INFO or DEBUG in sshd_config:

LogLevel INFO

#20: Patch OpenSSH and Operating Systems

It is recommended that you use tools such as yum, apt-get, freebsd-update and others to keep systems up to date with the latest security patches.

Other Options

To hide openssh version, you need to update source code and compile openssh again. Make sure following options are enabled in sshd_config:

# Turn on privilege separation

UsePrivilegeSeparation yes

# Prevent the use of insecure home directory and key file permissions

StrictModes yes

# Turn on reverse name checking

VerifyReverseMapping yes

# Do you need port forwarding?

AllowTcpForwarding no

X11Forwarding no

# Specifies whether password authentication is allowed. The default is yes.

PasswordAuthentication no

Verify your sshd_config file before restarting / reloading changes:
# /usr/sbin/sshd -t

Tighter SSH security with two-factor or three-factor (or more) authentication.

References:

The official OpenSSH project.
Forum thread: Failed SSH login attempts and how to avoid brute ssh attacks
man pages sshd_config, ssh_config, tcpd, yum, and apt-get.

If you have a technique or handy software not mentioned here, please share in the comments below to help your fellow readers keep their openssh based server secure.

Top 10 Open Source Web-Based Project Management Software

2011-03-19T22:55:00.000+05:30

This is an user contributed article.

Project management software is not just for managing software based project. It can be used for variety of other tasks too. The web-based software must provide tools for planning, organizing and managing resources to achieve project goals and objectives. A web-based project management software can be accessed through an intranet or WAN / LAN using a web browser. You don't have to install any other software on the system. The software can be easy of use with access control features (multi-user). I use project management software for all of our projects (for e.g. building a new cluster farm) for issue / bug-tracking, calender, gantt charts, email notification and much more.

Obviously I'm not the only user, the following open source software is used by some of the biggest research organizations and companies world wild. For example, NASA's Jet Propulsion Laboratory uses track software or open source project such as lighttpd / phpbb use redmine software to keep track of their projects.

You use the following top 10 software for personal or business use. Keep track of all your projects in one place and finish them successfully on time.

#1: Codendi

Codendi is an open-source collaborative development platform offered by Xerox. From only one interface, it gathers, all the needed tools for software development teams: management and versioning of code, bugs, requirements, documents, reporting, tests etc. It is mainly used for managing software project processes.

Download Codendi

#2: Redmine

Redmine is a flexible project management web application. Written using Ruby on Rails framework, it is cross-platform and cross-database. It includes calendar and gantt charts to aid visual representation of projects and their deadlines.

Download redmine

#3: ProjectPier

ProjectPier is a Free, Open-Source, self-hosted PHP application for managing tasks, projects and teams through an intuitive web interface. ProjectPier will help your organization communicate, collaborate and get things done Its function is similar to commercial groupware/project management products, but allows the freedom and scalability of self-hosting.

Download ProjectPier

#4: Trac

Trac is an open source, web-based project management and bug-tracking tool. Trac allows hyperlinking information between a computer bug database, revision control and wiki content. It also serves as a web interface to a version control system like Subversion, Git, Mercurial, Bazaar and Darcs.

Download Trac

#5: Project HQ

Project HQ is a collaborative open source project management tool, similar to Basecamp and activeCollab. Project HQ is built on open source technologies like Python, Pylons and SQLAlchemy and is fully database independent. Project HQ uses a structured workflow to assist you in managing your projects.

Download Project HQ

#6: Collabtive

Collabtive is a web-based project management software that is being published as Open Source software. The project was started in November 2007. It strives to provide an Open Source alternative to proprietary tools like Basecamp or ActiveCollab.

Download Collabtive

#7: eGroupWare

eGroupWare is a free open source groupware software intended for businesses from small to enterprises. Its primary functions allow users to manage contacts, appointments, projects and to-do lists.

It is used either via its native web-interface, making access platform-independent, or by using different supported groupware clients, such as Kontact, Novell Evolution, or Microsoft Outlook. It can also be used by mobile phone or PDA via SyncML.

Download eGroupWare

#8: KForge

KForge is an open-source (GPL) system for managing software and knowledge projects. It re-uses existing best-of-breed tools such as a versioned storage (subversion), a tracker (trac), and wiki (trac or moinmoin), integrating them with the system’s own facilities (projects, users, permissions etc). KForge also provides a complete web interface for project administration as well a fully-developed plugin system so that new services and features can be easily added.

Download KForge

#9: OpenGoo

It is a complete online solution focused on improving productivity, collaboration, communication and management of your teams. OpenGoo main features include document management, contact management, e-mail, project management, and time management. Text documents and presentations can be created and edited online. Files can be uploaded, organized and shared, independent of file formats.

Download OpenGoo

#10: ClockingIT

ClockingIT is a free Project Management solution, which helps your team stay focused and on top of things.

Download ClockingIT

Ed: The following two paragraphs added by Vivek Gite:

I also use project management software to keep track of how much time I spent per client and project.

My Personal Choice

redmine is my personal choice because I like to use ruby on rails and I often work with small teams. We track networking issues, data center issues, capacity planning, trouble tickets and much more using redmine. I can track multiple projects and its flexible role-based access control make sure only authorized eyes can view the details.

Other FOSS Project Management Software Projects

JotBug
Bugzilla (only bug tracking)
OpenProj (desktop app - replacement for MS-project)

How do you manage your IT / software and other projects? Are you using a better option? Let us know in the comments.

About the author: Rocky Jr., is an engineer with VSNL - a leading ISP / global telecom company and a good friend of nixCraft.

The Novice Guide To Buying A Linux Laptop

2011-03-19T22:35:00.000+05:30

All major laptop (notebook) hardware is supported by Linux. The important things to take into account when looking to buy a Linux powered laptops are as follows to avoid any hardware compatibility problems. Selecting correct specification is important. In this first part, I will cover what to look out for when buying a Linux powered laptop.

Laptop Usage

First, you need to decide and define the purpose of your laptop. Linux laptop can be used for simple office tasks and browsing the Internet or sending e-mails, than a mid-range laptop may be good enough:

Define your budget
Define screen size notebook (13" or 15" or 17")
Define your tasks such as Internet, sys admin, software development, some gaming using Win Vista / 7, dual booting etc.
How many hours of battery life do you need?
Define optical media types - Do you want to watch movies using DVD or Blu ray? Do you want to burn DVDs?
To play games you need a top spec laptop. Please note that Linux has limited support for popular PC gaming titles. So you need to dual boot your laptop with MS-Windows operating systems.
Finally, some people prefer desktop replacements with mobile workstations.

In short, decide how you are going to be using the Linux laptop.

Central Processing Unit (CPU)

CPU is the brain of your laptop. Linux supports all sort of mobile cpus. Linux does supports SMP cpus on servers and it does support portables cpus too. The following are well know mobile cpus:

Intel Atom
Intel Core 2 Duo
Intel Core i5 Duo
Intel Core i7 Duo or Quad
AMD Athlon 64 Mobile
AMD Turion 64 X2

The speed range from 1.3GHz to 2.2GHz or more. The number of cores plays an important role too. Naturally, a quad core CPU should be able to increase the number of calculations. This is useful when you run multiple applications, you will get fast and smooth response from the system. The Intel Core 2 Duo or AMD Turion 64 II (dual core) is pretty popular choice for CPU. My advice is go for 64 bit CPU with 64 bit Linux distro.

Video Card and X Server

This is the most important part in Laptop which is used by X Windows. However, selecting correct Video card is important; otherwise you will end up with 3D hardware accelerated card with 2D Linux driver or just basic display. So make sure your video card with 3D acceleration is supported under Linux. Nvidia has pretty good support with a proprietary driver which can deliver full 3D acceleration with 24 bit color. On other hand ATI drivers are open source. My advice is simple visit the following driver pages and make sure your video card is fully supported including 3D acceleration:

ATI/AMD Linux drivers
Nvidia Linux drivers
Linux Graphics drivers from Intel
Look here at X.org to check, if your graphics card / chipset is supported or not.

Low End Video Cards

The following list summaries low end fully supported Linux video cards:

Intel GMA 4500MHD
Intel 945GM
Intel 950 GMA
Intel X3100 GMA
NVIDIA Quadro NVS 160M (256MB)

Mid to High-End Video Cards

The following list summaries mid-high end priced fully supported Linux dedicated video cards (useful for playing 3D videogames or engineering Linux apps or video / photo editing):

NVIDIA Quadro FX 880M (1024 MB)
NVIDIA Quadro FX 2800M (1024 MB)
NVIDIA Quadro NVS 3100M (256 or 512 MB)
NVIDIA GeForce 9300M GS
NVIDIA GeForce 9300M
Intel GMA X4500 HD
ATI FirePro M7740 (1024 MB)
ATI Radeon HD 3650 (512 MB)
ATI Mobility Radeon HD4650 (1024 MB)

Finally, make sure external X display to your desktop monitor is fully supported. This can come handy when you want to watch movies, play games or do business presentation.

RAM

RAM (Random Access Memory) is another factor which can boost your speed easily, especially if you use RAM intensive apps like virtualization desktop software (e.g., VMWare or VirtualBox), photo editing (e.g., Gimp), video editing etc. 2 to 4 GB should be more than sufficient for most apps. A few high end laptops can support upto 16GB ram and Linux can access more than 4GB ram using PAE or 64bit kernel out of the box.

Hard Disk

Hard disk is used to store all your data and boot into Linux. Hard disks are cheaper, nosier and stores data on moving parts called rotating platters. You can have have 7200rpm drives. Another option is to use SSD ( Solid State Drives ) as storage device which offers the following benefits:

Speed (2 times faster than hard disk)
Lighter and quieter (no moving parts)
Consume less power
Faster booting time.

You can add additional storage using USB or Firewire ports (i.e. external hard disk). My advice is go for SSD.

Optical Disk

Linux does support DVDs, Blu ray and other optical formats. This is useful for playing DVDs or backing up data via DVD/CD burner. DVD burner is fine for most applications. But, if you are interested in playing movies using Blu ray (HD DVD / Blu Ray disks), you will need to get Blue ray optical drive which can read and write DVDs too. However, the AACS 'Digital Rights Management' system in most HD-DVD and all Blu-Ray discs attempts to stop consumers from exercising fair use rights, including - playing purchased Blu-Ray and HD DVD films using Open Source software. To play Blu ray you need the latest version of mplayer and DumpHD to perform the decryption necessary to play the film. If you can afford go for Blu ray. See how to play Blu-Ray and HD DVD video under Ubuntu Linux.

Power Management: Suspend and Hibernate

Linux supports both AMP ( Advanced Power Management ) and ACPI (Advanced Configuration and Power Interface ) which allows you to hibernate the system to a disk partition using swsusp (Software Suspend). It is a suspend-to-disk implementation in the 2.6 series Linux kernel. However, some laptop may give you problem with swsusp. You may also need to compile kernel to include swsusp support. Make sure you get ACPI compliant BIOS and you should be fine with power management. Some time you may need to download a patch from 3rd party or vendor site to enable suspend and hibernate support under Linux.

Wireless 802.11

Another hardware device may not work at all; if you do not pay attention to wireless devices. Most laptops comes with on-board 802.11 (a/b/g/N) wireless cards. Not all card supported so make sure you get Intel Pro series card such as 3945 or Atheros based cards. My advice is use Google to search for your driver or use specialized databases (a more or less complete listing of wireless devices with information about the chipset they are based on and whether or not they are supported in Linux) to search for your laptop card.

Biometric Fingerprint Scanner

Fingerprint scanners are security systems of biometrics. If you work for Government, DoD, police, security industries and, if data security and authentication is your top priority, you need "Biometric Fingerprint Scanner". Every fingerprint is special and different from each other. You can use Linux PAM to hook fingerprint reader with KDM, GDM, sudo, su and many other services. The fingerprint scanners on the following models are known to work:

IBM ThinkPad T and X series selected models
Dell Latitude selected models
HP Pavilion (selected model)

Please refer the following additional pages and make sue your laptop model is supported:

The fprint project aims to plug a gap in the Linux desktop: support for consumer fingerprint reader devices.
Fedora project wiki page describing supported devices and software for fingerprint devices.

Fn key (BlueKey) Support

Fn, or Function, is a modifier key on many keyboards, especially on laptops, used in a compact layout to combine keys which are usually kept separate. It is mainly used for the purpose of changing display or audio settings quickly, such as brightness, contrast, or volume, and is held down in conjunction with the appropriate key to change the settings. These features may be supported under Linux via driver or software. I found most laptop from reputed manufacture (such as Dell, IBM) does supports Fn keys.

Ports

The following are common ports for laptop:

USB - It is well supported under Linux for external hard disk, pen, mouse, keyboard and much more.
FireWire - It may or may not work out of box. In most cases you need to compile the Linux kernel so that you can access Zip drives, hard drives, and CDRW/DVD drives. However, support is limited. See Linux kernel IEEE 1394/ FireWire drivers wiki page for more information.
eSATA -External SATA provides a variant of SATA meant for external connectivity. It allows you to connect external SATA hard drives directly to the SATA bus. You get faster speed as compare to USB or Firewire external hard disks. This may or may not work out of box.

Sound

Most on board sound card should work fine with Linux. You can play all sort of music file such as .mp3, .wav, .ogg and much more. Linux comes with various playback and mp3 players. You can rip audio cds too.

Docking Station (Port Replicator)

A Docking station and port replicator provides a simplified way of “plugging-in” an electronic device such as a laptop computer via common peripherals. You can plug desktop monitor, keyboard, printer, mouse, Palm Pilot and so on. Most docking station should work out of the box. Just confirm this with your vendor.

Ethernet (NIC)

Most NICs are supported but it is a good idea to go through Google and make sure your Ethernet card is supported. Usually, Intel and Broadcom (tg3) Ethernet cards are well supported. See Linux PCI ids database and search for your sound and Ethernet card names. Usually, most drivers are included in running kernel. In rare case you may have to compile the kernel or driver to support your Ethernet card.

Tip: Test Your Laptop With Linux (Try Before You Buy)

Most vendors have a showrooms and shops. Visit nearest shop with Linux Live CD (such as Knoppix or Ubuntu or Fedora ) and ask them to boot laptop using Live CD. See if it works or not; boot into Linux. Make sure you can see X Windows, connects to the Internet via wireless card and so on. If you get 100% result, than just purchase laptop.

Recommend Linux Distribution

Linux is all about choices. Personally, I use Redhat v5.x 64 bit at work and Ubuntu Linux 64 bit at home. My recommendation:

Ubuntu (any flavor) Linux - For the average user, with a strong focus on usability and ease of use and installation.
Fedora Linux - Another popular disro for the average user with rpm based packages. Fedora is also popular among Windows and UNIX sys admins.
OpenSuse Linux - Easy to use and certified on selected HP / IBM laptops.
Debian Linux - For the advanced users, sys admin and academic researchers.
Slackware Linux - One of the oldest distro, preferred by sys admins, BSD lovers and academic researchers.

Linux Laptop Vendors

Now, you know how to choose a laptop, especially you need to pay attention to wireless, video card and Ethernet card. And here is exact model names & number that will be fully compatible to LINUX and vendor website (the list is for information and ready references only; please do your own research before purchasing system):

Dell sales Ubuntu Laptop in USA, Canada, UK, France, Germany, Spain and Latin America. (See official Ubuntu page). Dell India also sales high end Redhat Linux based laptop for business use.
HCL provides Ubuntu / Redhat / Fedora Linux based laptop in India.
Acer India provide Aspire series Linux based laptop in India.
HP Novell / Suse Linux certification and support matrix for HP laptops.
Asus Linux based eeepc netbooks in USA, India, UK, and various other places across the globe.
EmperorLinux provides Linux laptops with full hardware support under Linux in USA.
LinuxCertified provide Linux laptop and support in USA.
Linuxemporium provide Linux laptop and support in UK.
System76 provides Ubuntu Linux laptop and support in USA and Canada.
Zareason provides Ubuntu Linux laptop in USA, but claims to ship to many other places across the globe.

Got a Linux laptop brand you trust for its reliability and support? Add your experience in the comments below.

ss: Display Linux TCP / UDP Network and Socket Information

2011-03-19T22:30:00.000+05:30

The ss command is used to dump socket statistics. It allows showing information similar to netstat command. It can display more TCP and state information than other tools. It is a new, incredibly useful and faster (as compare to netstat) tool for tracking TCP connections and sockets. SS can provide information about:

All TCP sockets.
All UDP sockets.
All established ssh / ftp / http / https connections.
All local processes connected to X server.
All the tcp sockets in state FIN-WAIT-1 and much more.

Most Linux distributions are shipped with ss and many monitoring tools. Being familiar with this tool helps enhance your understand of what's going on in the system sockets and helps you find the possible causes of a performance problem.

Task: Display Sockets Summary

List currently established, closed, orphaned and waiting TCP sockets, enter:
# ss -s
Sample Output:

Total: 734 (kernel 904)

TCP: 1415 (estab 112, closed 1259, orphaned 11, synrecv 0, timewait 1258/0), ports 566

Transport Total IP IPv6

* 904 - -

RAW 0 0 0

UDP 15 12 3

TCP 156 134 22

INET 171 146 25

FRAG 0 0 0

Task: Display All Open Network Ports

# ss -l
Sample Output:

ss -l

Recv-Q Send-Q Local Address:Port Peer Address:Port

0 0 127.0.0.1:smux *:*

0 0 127.0.0.1:10024 *:*

0 0 127.0.0.1:10025 *:*

0 0 *:3306 *:*

0 0 *:http *:*

0 0 *:4949 *:*

0 0 *:domain *:*

0 0 *:ssh *:*

0 0 *:smtp *:*

0 0 127.0.0.1:rndc *:*

0 0 127.0.0.1:6010 *:*

0 0 *:https *:*

0 0 :::34571 :::*

0 0 :::34572 :::*

0 0 :::34573 :::*

0 0 ::1:rndc :::*

Type the following to see process named using open socket:
# ss -pl
Find out who is responsible for opening socket / port # 4949:
# ss -lp | grep 4949
Sample output:

0 0 *:4949 *:* users:(("munin-node",3772,5))

munin-node (PID # 3772) is responsible for opening port # 4949. You can get more information about this process (like memory used, users, current working directory and so on) visiting /proc/3772 directory:
# cd /proc/3772
# ls –l

Task: Display All TCP Sockets

# ss -t –a

Task: Display All UDP Sockets

# ss -u –a

Task: Display All Established SMTP Connections

# ss -o state established '( dport = :smtp or sport = :smtp )'

Task: Display All Established HTTP Connections

# ss -o state established '( dport = :http or sport = :http )'

Task: Find All Local Processes Connected To X Server

# ss -x src /tmp/.X11-unix/*

Task: List All The Tcp Sockets in State FIN-WAIT-1

List all the TCP sockets in state -FIN-WAIT-1 for our httpd to network 202.54.1/24 and look at their timers:
# ss -o state fin-wait-1 '( sport = :http or sport = :https )' dst 202.54.1/24

Search for all account without password and lock them

2011-03-19T22:29:00.000+05:30

For security, reason it is necessary to disable all account(s) with no password and lock them down. Solaris, Linux and FreeBSD provide account locking (unlocking) facility.

Lock Linux user account with the following command:

passwd -l {user-name}

For unlocking the account use:

passwd  -u {user-name}

-l : This option disables an account by changing the password to a value, which matches no possible encrypted value.

Lock FreeBSD user account with the following command:

pw lock {username}

FreeBSD unlocking the account use:

pw unlock {username}

Lock Solaris UNIX user account with the following command:

passwd -l {username}

Lock HP-UX user account with the following command:

passwd -l {username}

For unlocking the HP-UX account you need to edit /etc/passwd file using text editor (or use SAM):

vi /etc/passwd

However, how will you find out account without password? Again, with the help of 'passwd -s' (status) command you can find out all passwordless accounts.

Linux display password status

passwd -S {user-name}

Where,
-S : Display account status information. The status information consists of total seven fields. The second field indicates the status of password using following format:

L : if the user account is locked (L)
NP : Account has no password (NP)
P: Account has a usable password (P)

# passwd -S radmin

radmin P 10/08/2005 0 99999 7 -1

Solaris UNIX display password status

passwd -s {user-name}

Where,
-s : Display account status information using following format:

PS : Account has a usable password
LK : User account is locked
NP : Account has no password

FreeBSD
I have already written about small awk one line approach to find out all passwords less accounts.

Automated Scripting Solution
However, in real life you write a script and execute it from cron job. Here is small script for Linux:

#!/bin/sh

USERS="$(cut -d: -f 1 /etc/passwd)"

for u in $USERS

do

passwd -S $u | grep -Ew "NP" >/dev/null

if [ $? -eq 0 ]; then

passwd -l $u

fi

done

FreeBSD script:

#!/bin/bash

USERS="$(awk -F: 'NF > 1 && $1 !~ /^[#+-]/ && $2=="" {print $0}'

/etc/master.passwd | cut -d: -f1)"

for u in $USERS

do

pw lock $u

done

Sun Solaris script:

#!/bin/sh

USERS=`passwd -sa | grep -w NP | awk '{ print $1 }'`

for u in $USERS

do

passwd -l $u

done

You can easily add email alert support to script so that when ever scripts finds passwordless account(s) it will send an email alert. See the complete working example of script here.

Top 5 Email Client For Linux, Mac OS X, and Windows Users

2011-03-19T22:28:00.000+05:30

Linux comes with various GUI based email client to stay in touch with your friends and family, and share information in newsgroups with other users. The following software is similar to Outlook Express or Windows Live Mail and is used by both home and office user.

Webmail interfaces allow users to access their mail with any standard web browser, from any computer, rather than relying on an e-mail client.

However, e-mail client remains extremely popular in a large corporate environment, small business, home and power users. An e-mail client (also mail user agent (MUA)) is a frontend computer program used to manage e-mail. Mail can be stored on the client, on the server side, or in both places. Standard formats for mailboxes include Maildir and mbox.

The following are top five amazing piece of cross-platform software from various projects to make your life easy with wide variety of plug-ins / add-ons.

#1: Mozilla Thunderbird

It is an e-mail and news cross-platform client software package by Mozilla Foundation. Thunderbird can manage multiple e-mail, newsgroup and RSS accounts and supports multiple identities within accounts. Features like quick search, saved search folders , advanced message filtering, message grouping, and labels help manage and find messages. Just like Firefox, the tons of extensions and themes for this client makes it very secure and flexible to to enhance your productivity.

Fig.01: Mozilla Thunderbird

#2: Claws Mail

Claws Mail is a free, GTK+-based, open source email and news client. It is very light lightweight. Like Firefox , the wide variety of plug-ins for this email client makes it very flexible and secure. Claws Mail runs on Windows, Mac OS X and Unix-like systems such as Linux, BSD, and Solaris.

Fig.02: Claws Mail in Action

#3: Spicebird

Update: Ed - 23/June/2010 ~ Spicebird client is no longer supported on Mac OS X. We will replace this email client with another supported product shortly.

Spicebird is a collaboration client that provides integrated access to email, contacts, calendaring and instant messaging in a single application. It provides easy access to various web services while retaining all the advantages of a desktop application. It is developed by an Indian company called Synovel. It is a free, open source and cross-platform software.

Fig.03: Spicebird in Action (image credit Spicebird project)

#4: Zimbra Collaboration Suite (Open Source Version)

Zimbra is a client and server platform for messaging and collaboration. The web client integrates email, contacts, shared calendar, VoIP, and online document authoring in a rich browser-based interface. This is more like MS-Exchange and Outlook combo. In other words it is compatible with proprietary clients such as Microsoft Outlook and Apple Mail, both through proprietary connectors, as well as the open-source Novell Evolution, so that mail, contacts, and calendar items can be synchronised from these to the ZCS server. Zimbra also provides native two-way sync to many mobile devices such as Nokia Eseries, BlackBerry, Windows Mobile, iPhone with 2.0 software.

Fig.04: Zimbra (credit offical Zimbra website)

#5: Sylpheed

Sylpheed is a free, GTK+-based, open source email and news client. It is very light lightweight. Sylpheed runs on Windows, Mac OS X and Unix-like systems such as Linux, and BSD.

Fig.05: Sylpheed in Action

Comparison of E-mail Clients - Essential Features

The following tables compare general and technical information between e-mail client programs.

Feature	Thunderbird	Claws Mail	Spicebird	Zimbra	Sylpheed
Cross-platform	Y	Y	Y	Y	Y
License	MPL, MPL/GNU GPL/GNU LGPL	GPL	MPL, MPL/GNU GPL/GNU LGPL	MPL (server) and ZPL (client)	GPL/LGPL
Cost	Free	Free	Free	Free	Free
Authentication	Y	Y	Y	Y	Y
SSL and TLS	Y	Y	Y	Y	Y
Image blocking	Y	Y	Y	Y	Y
Junk filtering	Y	Y	Y	Y	Y
Phishing filtering	Y	Y	Y	Y	?
Add-ons	Y	Y	Y	?	N
Thread view	Y	Y	Y	Y	Y
PGP support	Y	Y	Y	?	Y
Label Messages	Y	Y	?	Y	Y
Spell Checking	Y	Y	?	Y	N
Signatures	Y	Y	Y	Y	Y
Scheduled message	Y	?	?	?	?
Message templates	Y	Y	?	Y	Y
Database	mbox	MH, mbox	?	File system	MH
POP3	Y	Y	Y	Y	Y
IMAP4	Y	Y	Y	Y	Y
SMTP	Y	Y	Y	Y	Y
NNTP (News)	Y	Y	Y	Y	Y
RSS Feed	Y	Y	Y	Y	N
LDAP	Y	Y	Y	Y	N
iCalendar	Y	?	Y	Y	N
Paid Support	?	?	?	?	Y

Y = supported; N = not supported; ? = unknown; Privacy feature; Security features; Productivity features; Cross-platform - runs on Mac OS X, Windows and UNIX like operating systems.

Other Email Clients For UNIX Like Operating Systems

SeaMonkey - Mozilla SeaMonkey is an all-in-one Internet application suite that includes an Internet browser, email and newsgroup client, HTML editor, IRC chat, and web development tools. It includes a pop-up blocker, junk mail controls, and a tabbed interface.
Pine (Alpine) - Alpine is a rewrite of the Pine Message System that adds support for Unicode and other features. Alpine is meant to be suitable for both inexperienced email users and the most demanding of power users.
Evolution or Novell Evolution - Evolution provides integrated mail, addressbook and calendaring functionality to users of the GNOME desktop.

Our Recommendations:

Claws Mail - Highly recommended for netbook user due to lightweight usage.
Mozilla Thunderbird - The wide variety of add-on for this email client makes it very flexible, secure and easy to use. Highly recommended for desktop and power users.
Zimbra Collaboration Suite ~ Open Source Edition or Businesses Editon - Highly recommended for business and corporate users due to its support for a broad range of email clients and mobile devices via "over the air" sync.

All of the e-mail client listed above used by me at one point or another. If you know of, or use, another e-mail client that offers better features than those mentioned here, tell us in the comments.

Updated for accuracy!

My 10 UNIX Command Line Mistakes

2011-03-19T22:07:00.000+05:30

Anyone who has never made a mistake has never tried anything new. -- Albert Einstein.

Here are a few mistakes that I made while working at UNIX prompt. Some mistakes caused me a good amount of downtime. Most of these mistakes are from my early days as a UNIX admin.

userdel Command

The file /etc/deluser.conf was configured to remove the home directory (it was done by previous sys admin and it was my first day at work) and mail spool of the user to be removed. I just wanted to remove the user account and I end up deleting everything (note -r was activated via deluser.conf):

userdel foo

Rebooted Solaris Box

On Linux killall command kill processes by name (killall httpd). On Solaris it kill all active processes. As root I killed all process, this was our main Oracle db box:
killall process-name

Destroyed named.conf

I wanted to append a new zone to /var/named/chroot/etc/named.conf file., but end up running:
./mkzone example.com > /var/named/chroot/etc/named.conf

Destroyed Working Backups with Tar and Rsync (personal backups)

I had only one backup copy of my QT project and I just wanted to get a directory called functions. I end up deleting entire backup (note -c switch instead of -x):
cd /mnt/bacupusbharddisk
tar -zcvf project.tar.gz functions
I had no backup. Similarly I end up running rsync command and deleted all new files by overwriting files from backup set (now I’ve switched to rsnapshot)
rsync -av -delete /dest /src
Again, I had no backup.

Deleted Apache DocumentRoot

I had sym links for my web server docroot (/home/httpd/http was symlinked to /www). I forgot about symlink issue. To save disk space, I ran rm -rf on http directory. Luckily, I had full working backup set.

Accidentally Changed Hostname and Triggered False Alarm

Accidentally changed the current hostname (I wanted to see current hostname settings) for one of our cluster node. Within minutes I received an alert message on both mobile and email.
hostname foo.example.com

Public Network Interface Shutdown

I wanted to shutdown VPN interface eth0, but ended up shutting down eth1 while I was logged in via SSH:
ifconfig eth1 down

Firewall Lockdown

I made changes to sshd_config and changed the ssh port number from 22 to 1022, but failed to update firewall rules. After a quick kernel upgrade, I had rebooted the box. I had to call remote data center tech to reset firewall settings. (now I use firewall reset script to avoid lockdowns).

Typing UNIX Commands on Wrong Box

I wanted to shutdown my local Fedora desktop system, but I issued halt on remote server (I was logged into remote box via SSH):
halt
service httpd stop

Wrong CNAME DNS Entry

Created a wrong DNS CNAME entry in example.com zone file. The end result - a few visitors went to /dev/null:
echo 'foo 86400 IN CNAME lb0.example.com' >> example.com && rndc reload

Failed To Update Postfix RBL Configuration

In 2006 ORDB went out of operation. But, I failed to update my Postfix RBL settings. One day ORDB was re-activated and it was returning every IP address queried as being on its blacklist. The end result was a disaster.

Conclusion

All men make mistakes, but only wise men learn from their mistakes -- Winston Churchill.

From all those mistakes I’ve learnt that:

Backup = ( Full + Removable tapes (or media) + Offline + Offsite + Tested )
The clear choice for preserving all data of UNIX file systems is dump, which is only tool that guaranties recovery under all conditions. (see Torture-testing Backup and Archive Programs paper).
Never use rsync with single backup directory. Create a snapshots using rsync or rsnapshots.
Use CVS to store configuration files.
Wait and read command line again before hitting the dam [Enter] key.
Use your well tested perl / shell scripts and open source configuration management software such as puppet, Cfengine or Chef to configure all servers. This also applies to day today jobs such as creating the users and so on.

Mistakes are the inevitable, so did you made any mistakes that have caused some sort of downtime? Please add them into the comments below.

Linux turn OFF password expiration / aging

2011-03-19T22:05:00.000+05:30

/etc/shadow stores actual password in encrypted format for user’s account with additional properties related to user password.

The password expiration information for a user is contained in the last 6 fields. Password expiration for a select user can be disabled by editing the /etc/shadow file

However I recommend using chage command. The chage command changes the number of days between password changes and the date of the last password change. This information is used by the system to determine when a user must change his/her password.

To list current aging type chage command as follows:
# chage -l vivek
Output:

Last password change : May 22, 2007

Password expires : never

Password inactive : never

Account expires : never

Minimum number of days between password change : 0

Maximum number of days between password change : 99999

Number of days of warning before password expires : 7

To disable password aging / expiration for user foo, type command as follows and set:
Minimum Password Age to 0
Maximum Password Age to 99999
Password Inactive to -1
Account Expiration Date to -1
Interactive mode command:
# chage username

# chage -I -1 -m 0 -M 99999 -E -1 username

Updated for accuracy.

Linux Track NFS Directory / Disk I/O Stats

2011-03-19T22:04:00.002+05:30

Q. How do I track NFS client metrics (disk I/O) under Linux operating system?

A. The iostat command is used for monitoring system input/output device loading by observing the time the devices are active in relation to their average transfer rates. The iostat command generates reports that can be used to change system configuration to better balance the input/output load between physical disks. The -n option displays the NFS-directory statistic. Data displayed are valid only with kernels 2.6.17 and newer. This option is exclusive to the -x option:

# iostat -x –n

# iostat –n

Related: How to find out CPU and Disk utilization under Linux using iostat command.

Linux locking an account

2011-03-19T22:03:00.003+05:30

passwd command is used to change user or group accounts password. A normal user may only change the password for his/her own account, the super user (root) may change the password for any account. You can use passwd command for locking or unlocking an account.

Linux locking an account
Syntax:
passwd -l {username}

Lock the account. This option disables an account by changing the password to a value which matches no possible encrypted value.

Example:
Lock user account named vivek. Login as a root user and type following command:

# passwd -l vivek

Linux unlocking an account
Syntax:
passwd -u {username}

Unlock the account. This option re-enables an account by changing the password back to its previous value i.e. to value before using -l option.

Example:
Unlock user account named vivek. Login as a root user and type following command:

# passwd -u vivek

Linux list hardware information

2011-03-19T22:03:00.000+05:30

Q. How do I get detailed information about my Linux server hardware information?

A. You need to use tool called lshw to extract detailed information on the hardware configuration of the machine. It can report exact memory configuration, firmware version, main board configuration, CPU version and speed, cache configuration, bus speed, etc. on DMI-capable x86 or IA-64 systems and on some PowerPC machines.

It currently supports DMI (x86 and IA-64 only), OpenFirmware device tree (PowerPC only), PCI/AGP, CPUID (x86), IDE/ATA/ATAPI, PCMCIA (only tested on x86), SCSI and USB.

List hardware

Type the lshw at a shell prompt:
$ lshw

Email
PDF
Print
Comment/RSS

Linux Limit CPU Usage Per Process

2011-03-19T22:02:00.000+05:30

I don't want background process to eat all my CPU. I know how to find out CPU utilization but how do I limit the cpu usage of a process under Linux operating system? How do I force a process to limit itself to 25% only?

You can use cpulimit program that attempts to limit the cpu usage of a process. Limits are expressed in percentage and not in cpu time. cpulimit does not act on the nice value or other scheduling priority stuff, but on the real cpu usage. Also, it is able to adapt itself to the overall system load, dynamically and quickly.

Install cpulimit

Type the following commands to install latest stable release:
# cd /tmp
# wget 'http://downloads.sourceforge.net/cpulimit/cpulimit-1.1.tar.gz'
# tar -zxvf cpulimit-1.1.tar.gz
# cd cpulimit-1.1
# make
# cp cpulimit /usr/local/sbin/
# rm -rf cpulimit*

A note about Debian / Ubuntu Linux users

Type the following command to install cpulimit:
$ sudo apt-get update
$ sudo apt-get install cpulimit

How do I use cpulimit?

To limit CPU usage of the process called firefox to 30%, enter:
# cpulimit -e firefox -l 30
To limit CPU usage of the process to 30% by using its PID, enter:
# cpulimit -p 1313 -l 30
To find out PID of the process use any of the following:
# ps aux | less
# ps aux | grep firefox
# pgrep -u vivek php-cgi
# pgrep lighttpd
You can also use absolute path name of the executable, enter:
# cpulimit -P /opt/firefox/firebox -l 30
Where,

-p : Process PID.
-e : Process name.
-l : percentage of CPU allowed from 0 to 100.
-P: absolute path name of the executable program file.

Root vs Normal User Account

From the project webpage:
cpulimit should run at least with the same user running the controlled process. But it is much better if you run cpulimit as root, in order to have a higher priority and a more precise control.

A Note About SMP (Multicore / MultiCpu) Systems

Again quoting from the project webpage:

If your machine has one processor you can limit the percentage from 0% to 100%, which means that if you set for example 50%, your process cannot use more than 500 ms of cpu time for each second. But if your machine has four processors, percentage may vary from 0% to 400%, so setting the limit to 200% means to use no more than half of the available power. In any case, the percentage is the same of what you see when you run top.

Related Throttling Utilities

ionice utility - Avoid sudden outburst of backup shell script / program disk I/O.
Limit disk I/O for rsync tool.
Linux nice command: Run Process With Modified Scheduling Priority ( nicenesses )
renice command: Change the Priority of a Already Running Process

Linux: Force Users To Change Their Passwords Upon First Login

2011-03-19T22:01:00.002+05:30

Q. How can I force my Linux users to change their passwords upon the first login under CentOS / Debian Linux?

A. You can use any one of the following command to change user passwords upon the first login:

[a] usermod command - Modify various user account properties including user password expiry information.
Where,

[b] chage command - Change user password expiry information

Task: Use chage command to force users to chage their password upon first login

Use the following syntax:
chage -d 0 {user-name}
# chage -d 0 tom

-d 0 : Set the number of days since January 1st, 1970 when the password was last changed. The date may also be expressed in the format YYYY-MM-DD. By setting it to zero, you are going to force user to change password upon first login.

Linux find the memory used by a program / process using pmap command

2011-03-19T22:00:00.002+05:30

You can find the memory used by a program (process) by looking into /proc directory or using standard command such as ps or top. However, you must calculate all memory usage by hand i.e. add Shared Memory + mapped file + total virtual memory size of the process + Resident Set Size + non-swapped physical memory used by process.

So how do you find the memory used by a process or program under Linux? Use a tool called pmap. It reports the memory map of a process or processes.

pmap examples

To display process mappings, type
$ pmap pid
$ pmap 3724
Output:

3724: /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf

0000000000400000 164K r-x-- /usr/sbin/lighttpd

0000000000629000 12K rw--- /usr/sbin/lighttpd

000000000bb6b000 4240K rw--- [ anon ]

00000035ee600000 104K r-x-- /lib64/ld-2.5.so

00000035ee819000 4K r---- /lib64/ld-2.5.so

00000035ee81a000 4K rw--- /lib64/ld-2.5.so

00000035eea00000 1304K r-x-- /lib64/libc-2.5.so

00000035eeb46000 2048K ----- /lib64/libc-2.5.so

00000035eed46000 16K r---- /lib64/libc-2.5.so

00000035eed4a000 4K rw--- /lib64/libc-2.5.so

00000035eed4b000 20K rw--- [ anon ]

00000035eee00000 8K r-x-- /lib64/libdl-2.5.so

00000035eee02000 2048K ----- /lib64/libdl-2.5.so

.....

....

00002aaaac51e000 4K r---- /lib64/libnss_files-2.5.so

00002aaaac51f000 4K rw--- /lib64/libnss_files-2.5.so

00007fff7143b000 84K rw--- [ stack ]

ffffffffff600000 8192K ----- [ anon ]

total 75180K

The -x option can be used to provide information about the memory allocation and mapping types per mapping. The amount of resident, non-shared anonymous, and locked memory is shown for each mapping:
pmap -x 3526
Output:

3526: -bash

Address Kbytes RSS Anon Locked Mode Mapping

0000000000400000 700 - - - r-x-- bash

00000000006ae000 40 - - - rw--- bash

00000000006b8000 20 - - - rw--- [ anon ]

00000000008b7000 32 - - - rw--- bash

00000000098de000 536 - - - rw--- [ anon ]

00000035ee600000 104 - - - r-x-- ld-2.5.so

00000035ee819000 4 - - - r---- ld-2.5.so

00000035ee81a000 4 - - - rw--- ld-2.5.so

00000035eea00000 1304 - - - r-x-- libc-2.5.so

00000035eeb46000 2048 - - - ----- libc-2.5.so

00000035eed46000 16 - - - r---- libc-2.5.so

00000035eed4a000 4 - - - rw--- libc-2.5.so

00000035eed4b000 20 - - - rw--- [ anon ]

00000035eee00000 8 - - - r-x-- libdl-2.5.so

00000035eee02000 2048 - - - ----- libdl-2.5.so

00000035ef002000 4 - - - r---- libdl-2.5.so

00000035ef003000 4 - - - rw--- libdl-2.5.so

00000035ef600000 12 - - - r-x-- libtermcap.so.2.0.8

00000035ef603000 2044 - - - ----- libtermcap.so.2.0.8

00000035ef802000 4 - - - rw--- libtermcap.so.2.0.8

00002aaaaaaab000 4 - - - rw--- [ anon ]

00002aaaaaaba000 12 - - - rw--- [ anon ]

00002aaaaaabd000 40 - - - r-x-- libnss_files-2.5.so

00002aaaaaac7000 2044 - - - ----- libnss_files-2.5.so

00002aaaaacc6000 4 - - - r---- libnss_files-2.5.so

00002aaaaacc7000 4 - - - rw--- libnss_files-2.5.so

00002aaaaacc8000 55112 - - - r---- locale-archive

00002aaaae29a000 28 - - - r--s- gconv-modules.cache

00002aaaae2a1000 8 - - - rw--- [ anon ]

00007fff9bff4000 92 - - - rw--- [ stack ]

ffffffffff600000 8192 - - - ----- [ anon ]

---------------- ------ ------ ------ ------

total kB 74496 - - -

Linux Find Out Virtual Memory PAGESIZE

2011-03-19T21:59:00.005+05:30

Q. How do I check the size of PAGESIZE under Linux?

A. Most modern operating systems have their main memory divided into pages. It allows better utilization of memory. A page is a fixed length block of main memory, that is contiguous in both physical memory addressing and virtual memory addressing. Kernel swap and allocates memory using pages

To display size of a page in bytes, enter:
$ getconf PAGESIZE

$ getconf PAGE_SIZE
Output:

4096

Linux display system hardware status information gathered from /proc filesystem in easy format

2011-03-19T21:59:00.002+05:30

/proc directory stores lots of system status and hardware information. The proc (mounted at /proc) filesystem is a pseudo-filesystem which is used as an interface to kernel data structures. For example, your CPU related information such as number of CPUs, type of CPU (AMD or Intel) etc can be gathered from /proc/cpuinfo.

However, the output is not always easily readable (especially if you are Linux sys admin). You need to use a command called procinfo which gathers some system data from the /proc directory and prints it nicely formatted on the screen.

It is capable of displaying following Linux status information:

Memory
Bootup (the time the system was booted)
Load average
User (the amount of time spent running jobs in user space)
Nice (the amount of time spent running niced jobs in user space)
System (the amount of time spent running in kernel space)
Idle (the amount of time spent doing nothing)
Uptime (the time that the system has been up).
Page in (the number of disk block paged into core from disk)
Page out (the reverse of the above (What does that mean, anyways?) )
Swap in (the number of memory pages paged in from swapspace)
Swap out (the number of memory pages paged out to swapspace)
context (the total number of context switches since bootup)
disk 1-4 (the number of times your hard disks have been accessed)
Interrupts
Modules (the modules (loadable device drivers) installed on your machine) Character and Block Devices
File Systems

Examples
$ procinfo
Output:

Linux 2.6.5-7.252-bigsmp (geeko@buildhost) (gcc 3.3.3 ) #1 SMP Tue Feb 14 11:11:04 UTC 2006 4CPU [ora10g-host1.xxxx.in]

Memory: Total Used Free Shared Buffers

Mem: 4091932 2327480 1764452 0 209444

Swap: 4194784 4 4194780

Bootup: Fri Mar 10 15:26:44 2006 Load average: 2.00 2.00 2.00 3/108 20202

user : 17:25:52.25 4.5% page in : 0

nice : 3d 7:22:29.54 20.5% page out: 0

system: 0:17:45.90 0.0% swap in : 0

idle : 12d 0:33:54.22 74.7% swap out: 0

uptime: 40d 5:46:29.70 context :621430542

irq 0:3477339909 timer irq 10: 0 ohci_hcd

irq 1: 3237 i8042 irq 12: 9578 i8042

irq 2: 0 cascade [4] irq 14: 6678197 ide0

irq 4: 4 irq 15: 25978305 ide1

irq 8: 2 rtc irq 16: 44294194 eth0

irq 9: 0 acpi

You can find out detailed information with -a flag:

$ procinfo -a

Output:

Linux 2.6.5-7.252-default (geeko@buildhost) (gcc 3.3.3 ) #1 2CPU [suse9ent.nixcraft.com]

Memory: Total Used Free Shared Buffers

Mem: 4125168 4112656 12512 0 276512

Swap: 4200688 32 4200656

Bootup: Mon Apr 10 13:46:48 2006 Load average: 0.76 0.70 0.32 1/105 6641

user : 0:59:24.49 2.2% page in : 0

nice : 0:11:08.41 0.4% page out: 0

system: 0:06:51.10 0.2% swap in : 0

idle : 18d 15:46:46.95 1020.6% swap out: 0

uptime: 9d 8:37:33.35 context : 84375734

irq 0: 0 0 irq 54: 396314 ioc0

irq 28: 1800 cpe_poll irq 55: 30 ioc1

irq 29: 0 cmc_poll irq 56: 1842085 eth1

irq 31: 0 cmc_hndlr irq 57: 18

irq 48: 0 acpi irq232: 0 mca_rdzv

irq 49: 0 ohci_hcd irq238: 0 perfmon

irq 50: 1892 ohci_hcd irq239:1656130975 timer

irq 51: 0 ehci_hcd irq240: 0 mca_wkup

irq 52: 5939450 ide0 irq254: 792697 IPI

irq 53: 404118 eth0

Kernel Command Line:

BOOT_IMAGE=scsi0:\efi\SuSE\vmlinuz root=/dev/sda3 selinux=0 splash=silent elevator=cfq ro

Modules:

147 snd_pcm_oss 240 *snd_pcm 38 *snd_page_alloc 74 *snd_timer

57 *snd_mixer_oss 149 *snd 33 *soundcore 44 thermal

48 *processor 23 fan 28 button 78 usbserial

73 parport_pc 38 lp 104 *parport 700 *ipv6

113 hid 36 joydev 97 sg 98 st

51 sr_mod 93 ide_cd 90 *cdrom 84 ehci_hcd

63 ohci_hcd 35 evdev 244 tg3 63 *af_packet

40 *binfmt_misc 246 *usbcore 122 e100 32 *subfs

19 *nls_utf8 24 *nls_cp437 139 dm_mod 266 *ext3

165 *jbd 30 mptsas 30 mptfc 29 *scsi_transport

29 *mptspi 98 *mptscsih 131 *mptbase 52 *sd_mod

237 *scsi_mod

Character Devices: Block Devices:

1 mem 10 misc 1 ramdisk 71 sd

2 pty 13 input 3 ide0 128 sd

3 ttyp 14 sound 7 loop 129 sd

4 /dev/vc/0 21 sg 8 sd 130 sd

4 tty 29 fb 9 md 131 sd

4 ttyS 116 alsa 11 sr 132 sd

5 /dev/tty 128 ptm 65 sd 133 sd

5 /dev/console 136 pts 66 sd 134 sd

5 /dev/ptmx 180 usb 67 sd 135 sd

6 lp 188 ttyUSB 68 sd 253 device-mapper

7 vcs 254 snsc 69 sd 254 mdp

9 st 70 sd

File Systems:

ext3 [sysfs] [rootfs] [bdev]

[proc] [cpuset] [sockfs] [pfmfs]

[futexfs] [tmpfs] [pipefs] [eventpollfs]

[devpts] ext2 [ramfs] [hugetlbfs]

minix msdos vfat iso9660

[nfs] [nfs4] [mqueue] [rpc_pipefs]

[subfs] [usbfs] [usbdevfs] [binfmt_misc]

Other options

-f : Run procinfo continuously full-screen (update status on screen, the default is 5 seconds, use -n SEC to setup pause)
-Ffile : Redirect output to file (usually a tty). For example procinfo -biDn1 -F/dev/tty5

Linux display information about installed hardware

2011-03-19T21:58:00.003+05:30

You need to use command called lsdev. It gathers information about your computer's installed hardware from the interrupts, ioports and dma files in the /proc directory, thus giving you a quick overview of which hardware uses what I/O addresses and what IRQ and DMA channels.

This command is very useful while troubleshooting various hardware conflict problems.

Task display : CPU Information

$ cat /proc/cpuinfo

Task: Display Free and used Memory Information

$ free -m

Task: Display partition information

# fdisk -l

Task: Display file system disk space usage

# df -H
OR
# df -h

Task: Estimate file space usage

# du

Task: List PCI devices

# lspci

Task: List USB devices

# lsusb

Task display : More Information

$ lsdev
Output:

Device            DMA   IRQ  I/O Ports

------------------------------------------------

0000:00:1d.0                 d800-d81f

0000:00:1d.1                 d000-d01f

0000:00:1d.2                 d400-d41f

0000:00:1f.0                 4000-407f 4080-40bf

0000:00:1f.1                 f000-f00f

0000:00:1f.3                 5000-501f

0000:00:1f.5                 e000-e0ff e400-e43f

0000:01:05.0                   c000-c0ff

8139too                          c000-c0ff

acpi                      9

bttv0                    22

cascade             4

dma                          0080-008f

dma1                         0000-001f

dma2                         00c0-00df

ehci_hcd:usb4            20

eth0                     18

fpu                          00f0-00ff

GPE0_BLK                       4028-402f

i8042                  1 12

ICH4                     21

ide0                     14  01f0-01f7 03f6-03f6   f000-f007

ide1                     15  0170-0177 0376-0376   f008-f00f

Intel                          e000-e0ff   e400-e43f

keyboard                     0060-006f

parport0                  7  0378-037a

PCI                          0cf8-0cff c000-cfff

pic1                         0020-0021

pic2                         00a0-00a1

PM1a_CNT_BLK                   4004-4005

PM1a_EVT_BLK                   4000-4003

PM_TMR                         4008-400b

rtc                       8  0070-0077

serial                       02f8-02ff 03f8-03ff

timer                     0

timer0                       0040-0043

timer1                       0050-0053

uhci_hcd                       d000-d01f   d400-d41f   d800-d81f

uhci_hcd:usb1            16

uhci_hcd:usb2            19

uhci_hcd:usb3            17

vga+                         03c0-03df

Linux display each multiple SMP CPU processors utilization individually

2011-03-19T21:58:00.000+05:30

Q. My AMD sever is quad processor with SMP kernel, How do I display the utilization of each CPU individually?

A. Linux can display each multiple SMP CPU processors utilization individually using mpstat command.

The mpstat command displays output activities for each available processor, processor 0 being the first one. Global average activities among all processors are also reported. The mpstat command can be used both on SMP and UP machines, but in the latter, only global average activities will be printed.

# mpstat -P ALL.
Where, -P cpu | ALL option indicate the processor number for which statistics are to be reported. cpu is the processor number. Note that processor 0 is the first processor. The ALL keyword indicates that statistics are to be reported for all processors.

Output:

Linux 2.6.9-55.ELsmp (server.xyz.com) 06/11/2007

04:13:29 PM CPU %user %nice %system %iowait %irq %soft %idle intr/s

04:13:29 PM all 5.63 0.01 2.33 0.85 0.01 0.10 91.06 2255.03

04:13:29 PM 0 5.74 0.00 2.17 0.88 0.01 0.15 91.05 1170.16

04:13:29 PM 1 5.53 0.01 2.49 0.83 0.01 0.06 91.07 1084.87

Understanding mpstat output

The report generated by the mpstat command has the following format:

CPU: Processor number. The keyword all indicates that statistics are calculated as averages among all processors.
%user: Show the percentage of CPU utilization that occurred while executing at the user level (application).
%nice: Show the percentage of CPU utilization that occurred while executing at the user level with nice priority.
%system: Show the percentage of CPU utilization that occurred while executing at the system level (kernel). Note that this does not include the time spent servicing
interrupts or softirqs.
%iowait: Show the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.
%irq: Show the percentage of time spent by the CPU or CPUs to service interrupts.
%soft: Show the percentage of time spent by the CPU or CPUs to service softirqs. A softirq (software interrupt) is one of up to 32 enumerated software interrupts
which can run on multiple CPUs at once.
%idle: Show the percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request.
intr/s: Show the total number of interrupts received per second by the CPU or CPUs.

Display five reports of global statistics among all processors at two second intervals, enter:

# mpstat 2 5

Display five reports of statistics for all processors at two second intervals, enter:

# mpstat -P ALL 2 5

Linux Check Memory Usage

2011-03-19T21:57:00.000+05:30

How do I check used and free RAM memory usage under Linux operating systems using command line and GUI tools?

Linux comes with different set of commands to check memory usage. The free command displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel. The vmstat command reports information about processes, memory, paging, block IO, traps, and cpu activity. Finally, you can use the top command which provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel.

free command

Display free memory size in MB:

$ free -mOutput:

             total       used       free     shared    buffers     cached

Mem:           750        625        125          0         35        335

-/+ buffers/cache:        254        496

Swap:          956          0        956

Displays a line containing the totals memory in MB:
$ free -t -m
Output:

       total       used       free     shared    buffers     cached

Mem:           750        625        125          0         35        335

-/+ buffers/cache:        253        496

Swap:          956          0        956

Total:        1707        625       1082

vmstat command

Type vmstat command at shell prompt:

$ vmstat
Output:

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa

1  0      0 131620  35432 341496    0    0    42    82  737  1364 15  3 81  1

top command

Type top command at the shell prompt:
$ top
Sample outputs:

Fig.01: top command displaying used memory

To exit from top command type q key. Read man page of free, vmstat, top command for more information.

GNOME Desktop: GUI Tool

The Gnome System Monitor application enables you to display basic system information and monitor system processes, usage of system resources, and file systems. You can also use System Monitor to modify the behavior of your system. You can start System Monitor by visiting System menu > Choose Administration > System Monitor option. Or type the following command at the shell prompt:

gnome-system-monitor

Sample outputs:

Fig.02: Linux See Memory Usage With GUI System Monitor Tool

Linux / UNIX: Restrict Access to A Given Command

2011-03-19T21:54:00.000+05:30

How do I restrict access to a given command for instance /opt/apps/start, to authorized users only under Linux / UNIX / BSD operating system?

You need to use traditional Unix groups concept to enhance security including restricted access to a given command.

Step # 1: Create and Maintain a Group For All Authorized Users

Create a group named appsonly:
# groupadd appsonly
Add all authorized users to appsonly:
# usermod -aG {groupName} {userName}
# usermod -aG appsonly tom
# usermod -aG appsonly jerry
# id jerry
Where,

-a : Add the user to the supplemental group(s) i.e. appends the user to the current supplementary group list.
-G : A list of supplementary groups which the user is also a member of.

Step #2: Restrict Access

Now a group of user had been created. Next, use the chgrp command to change the group of /opt/apps/start to appsonly group:
# chgrp {groupName} {/path/to/command}
# chgrp appsonly /opt/apps/start

Disable the file permission for others

Finally, use the chmod command to change file permission as follows:
# chmod 750 /path/to/command
# chmod 750 /opt/apps/start
You can also apply permissions to directory (this will disable ls command access to others) :
# chgrp appsonly /opt/apps
# chmod 0640 /opt/apps

Step # 3: Test It

su to tom, enter:
# su - tom
$ id
$ /opt/apps/start
$ exit
su to vivek (not a member of appsonly group), enter:
# su - vivek
$ id
$ /opt/apps/start
Sample outputs:

bash: /opt/apps/start: Permission denied

A Note About ACL and SELinux

The access control policies which can be enforced by chmod, chgrp, and usermod commands are limited, and configuring SELinux and fille system ACLs (access control list) is a better and recommend option for large deployments.

How to collect Linux system utilization data into a file

2011-03-19T21:52:00.000+05:30

I’ve already documented the procedure to find out CPU utilization under Linux and UNIX oses using various tools.

However I received a couple of email questions via email:

Howdo I collect Linux system utilization data into a file? How do I email the data or move to another computer?

To be frank you don’t have to do anything data is collected and stored in /var/log/sa location using sa1 and sa2 scripts. It is part of sadc - System activity data collector tools.

The sadc command samples system data a specified number of times ( count ) at a specified interval measured in seconds ( interval ). It writes in binary format to the specified output file.

The sa1 command is a shell procedure variant of the sadc command and handles all of the flags and parameters of that command. The sa1 command collects and store binary data in the /var/log/sa/sadd file, where the dd parameter indicates the current day. The interval and count parameters specify that the record should be written count times at interval seconds.

The sa1 command is designed to be started automatically by the cron command. Type the following command to list files:

# ls /var/log/sa
Output:

sa13 sa14 sa15 sa16 sa17 sa18 sa19 sa20 sa21 sar12 sar13 sar14 sar15 sar16 sar17 sar18 sar19 sar20

How do I copy log files?

You can copy all these logs files using ssh/scp or ftp to another computer. You can run use sar command to read binary raw data files, enter
# sar -f sa13
Output:

Linux 2.6.9-55.ELsmp (monster.labs.nixcraft.in) 05/13/2007

12:00:02 AM CPU %user %nice %system %iowait %idle

12:10:01 AM all 19.75 0.00 2.29 7.49 70.48

12:20:01 AM all 2.28 0.00 0.27 0.06 97.39

12:30:01 AM all 2.58 0.00 0.32 0.07 97.02

12:40:02 AM all 16.05 0.00 3.12 13.48 67.36

12:50:01 AM all 10.78 0.00 1.20 3.61 84.41

01:00:01 AM all 2.42 0.00 0.29 0.30 96.99

01:10:01 AM all 3.70 0.00 0.42 0.42 95.47

01:20:01 AM all 2.47 0.00 0.28 0.14 97.11

01:30:01 AM all 3.20 0.00 0.31 0.15 96.34

01:40:01 AM all 2.46 0.00 0.27 0.07 97.20

01:50:01 AM all 2.40 0.00 0.26 0.10 97.24

....

.....

How much RAM does my Ubuntu / Fedora Linux desktop PC have?

2011-03-14T20:55:00.000+05:30

Here is one of the most frequently asked questions from my mailbag:

Hey I need to know how much ram memory I have in my Ubuntu Linux computer. Under Windows XP I can find out memory by visiting Start > Control Panels > System in control panel. So how do I find out RAM information under Linux PC?

The best way to find out RAM information is open a terminal and type the following command:

$ free -m
OR
$ free -mt

Output:

             total       used       free     shared    buffers     cached

Mem:          2008       1837        170          0         50        963

-/+ buffers/cache:        822       1185

Swap:         1027          0       1027

free command print the total amount of free and used physical and swap memory in the system, as well as the buffers used by the Linux kernel. You can pass following options

-b switch displays the amount of memory in bytes
-k switch (set by default) displays it in kilobytes
-m switch displays it in megabytes.
-t switch displays a line containing the totals.
-o switch disables the display of a "buffer adjusted" line. If the -o option is not specified, free subtracts buffer memory from the used memory and adds it to the free memory reported.
-s switch activates continuous polling delay seconds apart. You may actually specify any floating point number for delay, usleep(3) is used for microsecond resolution delay times.

GUI tool - gnome-system-monitor

The gnome-system-monitor provides an overall view of the resource usage on your system, including memory and CPU allocation. Start gnome-system-monitor from Menus or just type:
$ gnome-system-monitor &

Another Good system monitor - Conky

Conky is a system monitor for X originally based on the torsmo code but kickass :). Run apt-get command to install conky:
$ sudo apt-get install conky
Conky can display just about anything, either on your root desktop or in it’s own window. Conky has many built-in objects, as well as the ability to execute programs and scripts, then display the output from stdout.
To start conky copy default config file and type conky at shell prompt:
$ zcat /usr/share/doc/conky/examples/conkyrc.sample.gz > .conkyrc
$ conky &
OR
$ conky -a left_top &

Read man page of conky for more customization options.

phpSysInfo - PHP script that displays information such as RAM, CPU et all

You can obtained pretty much same information over Internet using webbased php tool called phpSysInfo.

I'd love to hear if anyone has used additional cool GUI tool like conky for this kind of system analysis/monitoring.

How do I find out Linux Resource utilization to detect system bottlenecks?

2011-03-14T20:49:00.000+05:30

Q. How can I find out Linux Resource utilization using vmstat command? How do I get information about high disk I/O and memory usage?

A. vmstat command reports information about processes, memory, paging, block IO, traps, and cpu activity. However, a real advantage of vmstat command output - is to the point and (concise) easy to read/understand. The output of vmstat command use to help identify system bottlenecks. Please note that Linux vmstat does not count itself as a running process.

Here is an output of vmstat command from my enterprise grade system:
$ vmstat -S M
Output:

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa

3  0      0   1963    607   2359    0    0     0     0    0     1 32  0 68  0

Where,

The fist line is nothing but six different categories. The second line gives more information about each category. This second line gives all data you need.
-S M: vmstat lets you choose units (k, K, m, M) default is K (1024 bytes) in the default mode. I am using M since this system has over 4 GB memory. Without -M option it will use K as unit

$ vmstat
Output:

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa

3  0      0 2485120 621952 2415368  0    0     0     0    0     1 32  0 68  0

Field Description For Vm Mode

(a) procs is the process-related fields are:

r: The number of processes waiting for run time.
b: The number of processes in uninterruptible sleep.

(b) memory is the memory-related fields are:

swpd: the amount of virtual memory used.
free: the amount of idle memory.
buff: the amount of memory used as buffers.
cache: the amount of memory used as cache.

(c) swap is swap-related fields are:

si: Amount of memory swapped in from disk (/s).
so: Amount of memory swapped to disk (/s).

(d) io is the I/O-related fields are:

bi: Blocks received from a block device (blocks/s).
bo: Blocks sent to a block device (blocks/s).

(e) system is the system-related fields are:

in: The number of interrupts per second, including the clock.
cs: The number of context switches per second.

(f) cpu is the CPU-related fields are:

These are percentages of total CPU time.

us: Time spent running non-kernel code. (user time, including nice time)
sy: Time spent running kernel code. (system time)
id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
wa: Time spent waiting for IO. Prior to Linux 2.5.41, shown as zero.

As you see the first output produced gives averages data since the last reboot. Additional reports give information on a sampling period of length delay. You need to sample data using delays i.e. collect data by setting intervals. For example collect data every 2 seconds (or collect data every 2 second 5 times only):

$ vmstat -S M 2
OR
$ vmstat -S M 2 5
Output:

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----

r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa

3  0      0   1756    607   2359    0    0     0     0    0     1 32  0 68  0

3  0      0   1756    607   2359    0    0     0     0 1018    65 38  0 62  0

3  0      0   1756    607   2359    0    0     0     0 1011    64 37  0 63  0

3  0      0   1756    607   2359    0    0     0    20 1018    72 37  0 63  0

3  0      0   1756    607   2359    0    0     0     0 1012    64 37  0 62  0

3  0      0   1756    607   2359    0    0     0     0 1011    65 38  0 63  0

3  0      0   1995    607   2359    0    0     0     0 1012    62 35  2 63  0

3  0      0   1731    607   2359    0    0     0     0 1012    64 34  3 62  0

3  0      0   1731    607   2359    0    0     0     0 1013    72 38  0 62  0

3  0      0   1731    607   2359    0    0     0     0 1013    63 37  0 63  0

This is what most system administrators do to identify system bottlenecks. I hope all of you find vmstat data is concise and easy to read.

How do I Find Out Linux CPU Utilization?

2011-03-14T20:47:00.000+05:30

Whenever a Linux system CPU is occupied by a process, it is unavailable for processing other requests. Rest of pending requests must wait till CPU is free. This becomes a bottleneck in the system. Following command will help you to identify CPU utilization, so that you can troubleshoot CPU related performance problems.

Finding CPU utilization is one of the important tasks. Linux comes with various utilities to report CPU utilization. With these commands, you will be able to find out:

* CPU utilization
* Display the utilization of each CPU individually (SMP cpu)
* Find out your system's average CPU utilization since the last reboot etc
* Determine which process is eating the CPU(s)

Old good top command to find out Linux cpu load

The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel.

The top command monitors CPU utilization, process statistics, and memory utilization. The top section contains information related to overall system status - uptime, load average, process counts, CPU status, and utilization statistics for both memory and swap space.

Top command to find out Linux cpu usage

Type the top command:
$ top
Output:

You can see Linux CPU utilization under CPU stats. The task’s share of the elapsed CPU time since the last screen update, expressed as a percentage of total CPU time. In a true SMP environment (multiple CPUS), top will operate in number of CPUs. Please note that you need to type q key to exit the top command display.

The top command produces a frequently-updated list of processes. By default, the processes are ordered by percentage of CPU usage, with only the "top" CPU consumers shown. The top command shows how much processing power and memory are being used, as well as other information about the running processes.

Find Linux CPU utilization using mpstat and other tools

Please note that you need to install special package called sysstat to take advantage of following commands. This package includes system performance tools for Linux (Red Hat Linux / RHEL includes these tools by default).

# apt-get install sysstat
Use up2date command if you are using RHEL:
# up2date sysstat

Display the utilization of each CPU individually using mpstat

If you are using SMP (Multiple CPU) system, use mpstat command to display the utilization of each CPU individually. It report processors related statistics. For example, type command:

# mpstat Output:

Linux 2.6.15.4 (debian)         Thursday 06 April 2006

05:13:05  IST  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s

05:13:05  IST  all   16.52    0.00    2.87    1.09    0.07    0.02    0.00   79.42    830.06

The mpstat command display activities for each available processor, processor 0 being the first one. Global average activities among all processors are also reported. The mpstat command can be used both on SMP and UP machines, but in the latter, only global average activities will be printed.:

# mpstat -P ALL

Output:

Linux 2.6.15.4 (wwwportal1.xxxx.co.in)         Thursday 06 April 2006

05:14:58  IST  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s

05:14:58  IST  all   16.46    0.00    2.88    1.08    0.07    0.02    0.00   79.48    835.96

05:14:58  IST    0   16.46    0.00    2.88    1.08    0.07    0.02    0.00   79.48    835.96

05:14:58  IST    1   15.77    2.70    3.17    2.01    0.05    0.03    0.00   81.44    822.54

Another output from my HP Dual Opteron 64 bit server:# mpstat -P ALLOutput:

Linux 2.6.5-7.252-smp (ora9.xxx.in)   04/07/06

07:44:18     CPU   %user   %nice %system %iowait    %irq   %soft   %idle    intr/s

07:44:18     all    3.01   57.31    0.36    0.13    0.01    0.00   39.19   1063.46

07:44:18       0    5.87   69.47    0.44    0.05    0.01    0.01   24.16    262.11

07:44:18       1    1.79   48.59    0.36    0.23    0.00    0.00   49.02    268.92

07:44:18       2    2.19   42.63    0.28    0.16    0.01    0.00   54.73    260.96

07:44:18       3    2.17   68.56    0.34    0.06    0.03    0.00   28.83    271.47

Report CPU utilization using sar command

You can display today’s CPU activity, with sar command:
# sar
Output:

Linux 2.6.9-42.0.3.ELsmp (dellbox.xyz.co.in)         01/13/2007

12:00:02 AM       CPU     %user     %nice   %system   %iowait     %idle

12:10:01 AM       all      1.05      0.00      0.28      0.04     98.64

12:20:01 AM       all      0.74      0.00      0.34      0.38     98.54

12:30:02 AM       all      1.09      0.00      0.28      0.10     98.53

12:40:01 AM       all      0.76      0.00      0.21      0.03     99.00

12:50:01 AM       all      1.25      0.00      0.32      0.03     98.40

01:00:01 AM       all      0.80      0.00      0.24      0.03     98.92

...

.....

..

04:40:01 AM       all      8.39      0.00     33.17      0.06     58.38

04:50:01 AM       all      8.68      0.00     37.51      0.04     53.78

05:00:01 AM       all      7.10      0.00     30.48      0.04     62.39

05:10:01 AM       all      8.78      0.00     37.74      0.03     53.44

05:20:02 AM       all      8.30      0.00     35.45      0.06     56.18

Average:          all      3.09      0.00      9.14      0.09     87.68

Comparison of CPU utilization

The sar command writes to standard output the contents of selected cumulative activity counters in the operating system. The accounting system, based on the values in the count and interval parameters. For example display comparison of CPU utilization; 2 seconds apart; 5 times, use:

# sar -u 2 5
Output (for each 2 seconds. 5 lines are displayed):

Linux 2.6.9-42.0.3.ELsmp (www1lab2.xyz.ac.in)         01/13/2007

05:33:24 AM       CPU     %user     %nice   %system   %iowait     %idle

05:33:26 AM       all      9.50      0.00     49.00      0.00     41.50

05:33:28 AM       all     16.79      0.00     74.69      0.00      8.52

05:33:30 AM       all     17.21      0.00     80.30      0.00      2.49

05:33:32 AM       all     16.75      0.00     81.00      0.00      2.25

05:33:34 AM       all     14.29      0.00     72.43      0.00     13.28

Average:          all     14.91      0.00     71.49      0.00     13.61

Where,

-u 12 5 : Report CPU utilization. The following values are displayed:

%user: Percentage of CPU utilization that occurred while executing at the user level (application).
%nice: Percentage of CPU utilization that occurred while executing at the user level with nice priority.
%system: Percentage of CPU utilization that occurred while executing at the system level (kernel).
%iowait: Percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.
%idle: Percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request.

To get multiple samples and multiple reports set an output file for the sar command. Run the sar command as a background process using.

# sar -o output.file 12 8 >/dev/null 2>&1 &
Better use nohup command so that you can logout and check back report later on:
# nohup sar -o output.file 12 8 >/dev/null 2>&1 &
All data is captured in binary form and saved to a file (data.file). The data can then be selectively displayed ith the sar command using the -f option.
# sar -f data.file

Task: Find out who is monopolizing or eating the CPUs

Finally, you need to determine which process is monopolizing or eating the CPUs. Following command will displays the top 10 CPU users on the Linux system.

# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10

OR

# ps -eo pcpu,pid,user,args | sort -r -k1 | less

Output:

%CPU   PID USER     COMMAND

  96  2148 vivek    /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Ubuntu 64-bit/Ubuntu 64-bit.vmx -@ ""

0.7  3358 mysql    /usr/libexec/mysqld --defaults-file=/etc/my.cnf --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --socket=/var/lib/mysql/mysql.sock

0.4 29129 lighttpd /usr/bin/php

0.4 29128 lighttpd /usr/bin/php

0.4 29127 lighttpd /usr/bin/php

0.4 29126 lighttpd /usr/bin/php

0.2  2177 vivek    [vmware-rtc]

0.0     9 root     [kacpid]

0.0     8 root     [khelper]

Now you know vmware-vmx process is eating up lots of CPU power. ps command displays every process (-e) with a user-defined format (-o pcpu). First field is pcpu (cpu utilization). It is sorted in reverse order to display top 10 CPU eating process.

iostat command

You can also use iostat command which report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions. It can be use to find out your system's average CPU utilization since the last reboot.

# iostatOutput:

Linux 2.6.15.4 (debian)         Thursday 06 April 2006

avg-cpu:  %user   %nice %system %iowait  %steal   %idle

     16.36    0.00    2.99    1.06    0.00   79.59

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn

hda               0.00         0.00         0.00         16          0

hdb               6.43        85.57       166.74     875340    1705664

hdc               0.03         0.16         0.00       1644          0

sda               0.00         0.00         0.00         24          0

You may want to use following command, which gives you three outputs every 5 seconds (as previous command gives information since the last reboot):$ iostat -xtc 5 3

GUI tools for your laptops/desktops

Above tools/commands are quite useful on remote server. For local system with X GUI installed you can try out gnome-system-monitor. It allows you to view and control the processes running on your system. You can access detailed memory maps, send signals, and terminate the processes.
$ gnome-system-monitor

In addition, the gnome-system-monitor provides an overall view of the resource usage on your system, including memory and CPU allocation.

Get Detailed Information About Particular IP address Connections Using netstat Command

2011-03-14T20:40:00.002+05:30

netstat command and shell pipe feature can be used to dig out more information about particular IP address connection. You can find out total established connections, closing connection, SYN and FIN bits and much more. You can also display summary statistics for each protocol using netstat.

This is useful to find out if your server is under attack or not. You can also list abusive IP address using this method.

# netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n

Output:

1 CLOSE_WAIT

1 established)

1 Foreign

3 FIN_WAIT1

3 LAST_ACK

13 ESTABLISHED

17 LISTEN

154 FIN_WAIT2

327 TIME_WAIT

Dig out more information about a specific ip address:
# netstat -nat |grep {IP-address} | awk '{print $6}' | sort | uniq -c | sort -n

2 LAST_ACK

2 LISTEN

4 FIN_WAIT1

14 ESTABLISHED

91 TIME_WAIT

130 FIN_WAIT2

Busy server can give out more information:

# netstat -nat |grep 202.54.1.10 | awk '{print $6}' | sort | uniq -c | sort -n

Output:

15 CLOSE_WAIT

37 LAST_ACK

64 FIN_WAIT_1

65 FIN_WAIT_2

1251 TIME_WAIT

3597 SYN_SENT

5124 ESTABLISHED

Get List Of All Unique IP Address

To print list of all unique IP address connected to server, enter:
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq
To print total of all unique IP address, enter:
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq | wc -l

Output:

449

Find Out If Box is Under DoS Attack or Not

If you think your Linux box is under attack, print out a list of open connections on your box and sorts them by according to IP address, enter:

# netstat -atun | awk '{print $5}' | cut -d: -f1 | sed -e '/^$/d' |sort | uniq -c | sort -n

Output:

1 10.0.77.52

2 10.1.11.3

4 12.109.42.21

6 12.191.136.3

.....

...

....

13 202.155.209.202

18 208.67.222.222

28 0.0.0.0

233 127.0.0.1

You can simply block all abusive IPs using iptables or just null route them.

Get Live View of TCP Connections

You can use tcptrack command to display the status of TCP connections that it sees on a given network interface. tcptrack monitors their state and displays information such as state, source/destination addresses and bandwidth usage in a sorted, updated list very much like the top command.

Display Summary Statistics for Each Protocol

Simply use netstat -s:
# netstat -s | less
# netstat -t -s | less
# netstat -u -s | less
# netstat -w -s | less
# netstat -s

Output:

Ip:

88354557 total packets received

0 forwarded

0 incoming packets discarded

88104061 incoming packets delivered

96037391 requests sent out

13 outgoing packets dropped

66 fragments dropped after timeout

295 reassemblies required

106 packets reassembled ok

66 packet reassembles failed

34 fragments failed

Icmp:

18108 ICMP messages received

58 input ICMP message failed.

ICMP input histogram:

destination unreachable: 7173

timeout in transit: 472

redirects: 353

echo requests: 10096

28977 ICMP messages sent

0 ICMP messages failed

ICMP output histogram:

destination unreachable: 18881

echo replies: 10096

Tcp:

1202226 active connections openings

2706802 passive connection openings

7394 failed connection attempts

47018 connection resets received

23 connections established

87975383 segments received

95235730 segments send out

681174 segments retransmited

2044 bad segments received.

80805 resets sent

Udp:

92689 packets received

14611 packets to unknown port received.

0 packet receive errors

96755 packets sent

TcpExt:

48452 invalid SYN cookies received

7357 resets received for embryonic SYN_RECV sockets

43 ICMP packets dropped because they were out-of-window

5 ICMP packets dropped because socket was locked

2672073 TCP sockets finished time wait in fast timer

441 time wait sockets recycled by time stamp

368562 delayed acks sent

430 delayed acks further delayed because of locked socket

Quick ack mode was activated 36127 times

32318597 packets directly queued to recvmsg prequeue.

741479256 packets directly received from backlog

1502338990 packets directly received from prequeue

18343750 packets header predicted

10220683 packets header predicted and directly queued to user

17516622 acknowledgments not containing data received

36549771 predicted acknowledgments

102672 times recovered from packet loss due to fast retransmit

Detected reordering 1596 times using reno fast retransmit

Detected reordering 1 times using time stamp

8 congestion windows fully recovered

32 congestion windows partially recovered using Hoe heuristic

19 congestion windows recovered after partial ack

0 TCP data loss events

39951 timeouts after reno fast retransmit

29653 timeouts in loss state

197005 fast retransmits

186937 retransmits in slow start

131433 other TCP timeouts

TCPRenoRecoveryFail: 20217

147 times receiver scheduled too late for direct processing

29010 connections reset due to unexpected data

365 connections reset due to early user close

6979 connections aborted due to timeout

Display Interface Table

You can easily display dropped and total transmitted packets with netstat for eth0:
# netstat --interfaces eth0

Output:

Kernel Interface table

Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg

eth0 1500 0 2040929 0 0 0 3850539 0 0 0 BMRU

FREE COMPUTER SOLUTIONS

Top 20 OpenSSH Server Best Security Practices

Top 10 Open Source Web-Based Project Management Software

The Novice Guide To Buying A Linux Laptop

ss: Display Linux TCP / UDP Network and Socket Information

Search for all account without password and lock them

Lock Linux user account with the following command:

For unlocking the account use:

Lock FreeBSD user account with the following command:

FreeBSD unlocking the account use:

Lock Solaris UNIX user account with the following command:

Lock HP-UX user account with the following command:

Linux display password status

Solaris UNIX display password status

Top 5 Email Client For Linux, Mac OS X, and Windows Users

My 10 UNIX Command Line Mistakes

Linux turn OFF password expiration / aging

Linux Track NFS Directory / Disk I/O Stats

Linux locking an account

Linux list hardware information

Linux Limit CPU Usage Per Process

Install cpulimit

A note about Debian / Ubuntu Linux users

How do I use cpulimit?

Root vs Normal User Account

A Note About SMP (Multicore / MultiCpu) Systems

Related Throttling Utilities

Linux: Force Users To Change Their Passwords Upon First Login

Task: Use chage command to force users to chage their password upon first login

Further readings:

Linux find the memory used by a program / process using pmap command

Linux Find Out Virtual Memory PAGESIZE

Linux display system hardware status information gathered from /proc filesystem in easy format

Linux display information about installed hardware

Task display : CPU Information

Task: Display Free and used Memory Information

Task: Display partition information

Task: Display file system disk space usage

Task: Estimate file space usage

Task: List PCI devices

Task: List USB devices

Task display : More Information

Linux display each multiple SMP CPU processors utilization individually

Linux Check Memory Usage

free command

vmstat command

top command

GNOME Desktop: GUI Tool

Linux / UNIX: Restrict Access to A Given Command

Step # 1: Create and Maintain a Group For All Authorized Users

Step #2: Restrict Access

Disable the file permission for others

Step # 3: Test It

A Note About ACL and SELinux

How to collect Linux system utilization data into a file

How much RAM does my Ubuntu / Fedora Linux desktop PC have?

GUI tool - gnome-system-monitor

Another Good system monitor - Conky

phpSysInfo - PHP script that displays information such as RAM, CPU et all

How do I find out Linux Resource utilization to detect system bottlenecks?

Field Description For Vm Mode

(a) procs is the process-related fields are:

(b) memory is the memory-related fields are:

(c) swap is swap-related fields are:

(d) io is the I/O-related fields are:

(e) system is the system-related fields are:

(f) cpu is the CPU-related fields are:

How do I Find Out Linux CPU Utilization?

Old good top command to find out Linux cpu load

Top command to find out Linux cpu usage

Find Linux CPU utilization using mpstat and other tools

Display the utilization of each CPU individually using mpstat

Report CPU utilization using sar command

Comparison of CPU utilization

Task: Find out who is monopolizing or eating the CPUs

iostat command

GUI tools for your laptops/desktops

Get Detailed Information About Particular IP address Connections Using netstat Command