linuxwave

Ping a list of servers

2012-01-11T16:15:00.003+08:00

To do this, you need to put all the hosts that need to be checked in a file. For example, I put all my hosts in a file called ping_list:

$ cat ping_list
cat.myhost.net
dog.myhost.net
tiger.myhost.net
bird.myhost.net

There are a few ways to ping multiple hostnames, I'll list out what I have tried before:

1. Use nmap

$ nmap -sP -iL ping_list
Failed to resolve given hostname/IP: cat.myhost.net.  Note that you can't use '/mask' AND '1-4,7,100-' style IP ranges
Failed to resolve given hostname/IP: dog.myhost.net.  Note that you can't use '/mask' AND '1-4,7,100-' style IP ranges
Host 192.168.0.99 is up (0.00036s latency).
Host 192.168.0.100 is up (0.00061s latency).

where -sP is for ping test and -iL is for inputting from files.

2. One liner for loop

$ for i in `cat ping_list`; do ping -c1 $i; done
ping: unknown host cat.myhost.net
ping: unknown host dog.myhost.net
PING tiger.myhost.net (192.168.0.99) 56(84) bytes of data.

--- tiger.myhost.net ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

PING tiger.myhost.net (192.168.0.100) 56(84) bytes of data.
 
 --- bird.myhost.net ping statistics ---
 1 packets transmitted, 0 received, 100% packet loss, time 0ms

I believe there are other tools or scripts beside those I listed above, but I always these 2 methods to ping multiple hosts. If you have other tools or script, please leave a comment.

Thanks

Store svn password

2011-12-06T10:49:00.003+08:00

To store your cert and password, all you have to do is:

1. Go to .subversion in your home folder

$ cd ~/.subversion

2. Run below command. This command will add store-auth-creds = yes and store-passwords = yes to your bottom of config file

$ echo -e "store-auth-creds = yes\nstore-passwords = yes" >> config

3. Run any svn command, and choose p for permanent accept certificate, and yes for storing your password.

Now you won't be asked for svn password anymore.

Add file type swap to linux

2011-11-25T15:42:00.003+08:00

To add swap to linux machine:

1. Create a swap file. Size depends on your preference. Let's say we want to create a swap file with 8GB size (1024 x 1024 x 8 = 8388608).

# dd if=/dev/zero of=/swapfile bs=1024 count=8388608

where if is source, of is output file for dd to write to which is /swapfile in this case, bs is read/write byte size at a time and count is number of blocks.

2. Once created, make it a swap file

# mkswap /swapfile

3. Activate your swap file

# swapon /swapfile

4. Check your newly created swap space using free or top

# free -m

# top

5. To make it appear even after reboot, put it into fstab

# echo "swapfile swap swap defaults 0 0" >> /etc/fstab

It will look like this:

# cat /etc/fstab 
...
/swapfile1 swap swap defaults 0 0
...

ssh forward tunnel

2011-11-01T11:28:00.004+08:00

To make this happen, the command is like below:

$ ssh -L 10022:target.local:22 middle.local

where:
-L is for forward tunnel, 10022 is the port at localhost that we want to use, target.local is our target, 22 is the target's port that we want to forward and middle.local is our middleman server.

What the above command do is forwarding port 22 of target.local to port 10022 of localhost by using middle.local as a middleman. Once done, you can access port 22 of target.local just by SSHing into port 10022 in your localhost like below:

$ ssh localhost -p10022

and voila, you will be directed to target.local instead. This technique is useful when you have firewall blocking some ports, and you have server behind the firewall than can access those ports witth openssh installed.

Small script to test perl module

2011-10-25T09:57:00.003+08:00

To test any perl module, just put below lines in a file, make it executable (chmod +x) and run it (./).

#!/usr/bin/perl
use DBD-Pg;
print "No errors!\n"

where DBD-Pg is example of perl module. This script will return "No errors!" if the module is available. If the module is not installed, an error like "Can't locate ...." will appear instead.

Extending LVM logical volume

2011-10-06T17:06:00.005+08:00

To extend the lv (logical volume), the command is lvextend. Please check beforehand that you have free disk space to be extended to.

Check which volume group the partition belongs to:

# df -lh
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/mapper/vg01-db   6.0T  6.0T   20K 100% /var/lib/hwbackup/db
...

# lvdisplay vg01
--- Logical volume ---
  LV Name                /dev/vg01/db
  VG Name                vg01
  LV UUID                2LvWW6-5TIG-ovrl-GUJt-1y3N-M6ME-3LgtUY
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                5.99 TB
  Current LE             1569792
  Segments               5
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     4096
  Block device           253:2

So, our lv belongs to vg01

Now let's check how much space we have for vg01:

# vgdisplay vg01
...
  Alloc PE / Size       4059648 / 15.49 TB
  Free  PE / Size       232788 / 909.33 GB
  VG UUID               iaFnIb-hKjs-q3fQ-yuTy-tKKl-uqSp-EXGyHF
...

So we know that it got 909.33 GB unused space.

Lets do the extension, let's say we want to extend by 300GB. Run a test run first:

# lvextend -t -v -L +300G /dev/mapper/vg01-db
Test mode: Metadata will NOT be updated.
    Finding volume group vg01
  Using stripesize of last segment 512.00 KB
    Test mode: Skipping archiving of volume group.
  Extending logical volume db to 6.28 TB
    Found volume group "vg01"
    Found volume group "vg01"
    Found volume group "vg01"
    Test mode: Skipping volume group backup.
  Logical volume db successfully resized
    Test mode: Wiping internal cache
    Wiping internal VG cache
[root@hitw-gc-backup-1 ~]# lvextend -v -L +300G /dev/mapper/vg01-db
    Finding volume group vg01
  Using stripesize of last segment 512.00 KB
    Archiving volume group "vg01" metadata (seqno 19).
  Extending logical volume db to 6.28 TB
    Found volume group "vg01"
    Found volume group "vg01"
    Loading vg01-db table (253:2)
    Suspending vg01-db (253:2) with device flush
    Found volume group "vg01"
    Resuming vg01-db (253:2)
    Creating volume group backup "/etc/lvm/backup/vg01" (seqno 20).
  Logical volume db successfully resized

where -t is for test mode, -v for verbose and -L is for the size that you want to extend

If we are OK with the test, proceed to the real extension by removing -t:

# lvextend -v -L +300G /dev/mapper/vg01-db
    Finding volume group vg01
  Using stripesize of last segment 512.00 KB
    Archiving volume group "vg01" metadata (seqno 19).
  Extending logical volume db to 6.28 TB
    Found volume group "vg01"
    Found volume group "vg01"
    Loading vg01-db table (253:2)
    Suspending vg01-db (253:2) with device flush
    Found volume group "vg01"
    Resuming vg01-db (253:2)
    Creating volume group backup "/etc/lvm/backup/vg01" (seqno 20).
  Logical volume db successfully resized

Check your new lv size, it is now being extended by 300GB:

# lvdisplay /dev/mapper/vg01-db
...
 LV Size                6.28 TB
...

To make your filesystem aware of the extension, run resize2fs (for ext3 filesystem). Please unmount the partition beofre you run resize2fs

# umount /dev/mapper/vg01-db

# resize2fs /dev/mapper/vg01-db

# mount /dev/mapper/vg01-db /var/lib/hwbackup/db

For xfs, you can resize it on the fly without unmounting it:

# xfs_growfs /var/lib/hwbackup/db

Now you can enjoy your newly extended lv:

# df -lh
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/mapper/vg01-db   6.3T  5.7T  608G  91% /var/lib/hwbackup/db
...

Replacing space with newline

2011-09-29T14:36:00.004+08:00

There are a few ways to achieve that:

1. sed

$ echo "one two three" | sed 's/ /\n/g'
one
two
three

2. awk

$ echo "one two three" | awk '$1=$1' RS= OFS="\n"
one
two
three

3. tr

$ echo "one two three" | tr -s ' ' '\n'
one
two
three

3 ways to do it, have fun

Multiple sed expression in one line

2011-09-27T10:38:00.004+08:00

To do multiple sed expression in one line, you can use -e a few times according to your need. For example:

To subtitute 'little' with 'big' and 'lamb' with 'cow' in "Mary had a little lamb" we can use:

$ echo 'Mary had a little lamb' | sed -e 's/little/big/' -e 's/lamb/cow/'
Mary had a big cow

where -e is for expression flag, 's/little/big/' is the first expression and 's/lamb/cow/' is the second expression

That's all.

Simple http server using python

2011-09-23T10:40:00.003+08:00

This tip is really useful when you need a really quick web server running without too much hassle. Python comes with http server built-in that you can activate in a matter of seconds if you have python already installed. If you haven't install it, you might need an additional few minutes to install it. Below are the steps (this is assuming you are using redhat/fedora linux):

1. open a terminal and install python

# yum install python

2. Once done, browse to any directory that you want to be your document root directory, in this example I'll use /home/myuser

# cd /home/myuser

3. Run the below command to start the simple http server, where 8000 is the port number the server will use

# python -m SimpleHTTPServer 8000
Serving HTTP on 0.0.0.0 port 8000 ...

4. Now open a browser, and type http://192.168.1.2:8000 to access the server, where 192.168.1.2 is your machine's ip address

Download apache directory listings recursively

2011-08-15T10:38:00.003+08:00

To apache directory listings recursively, use wget like below:

$ wget -r -np -nH -R index.html http://filestodownload/

where -r is for recursive retrieving, -np is for no-parent option where wget won't get the parent directory when retrieving recursively, -nH equals to no host diretories where generation of host-prefixed directories will be disabled and -R is to omit index.html.

Extract rpm without installing

2011-08-05T15:42:00.002+08:00

Run below command:

$ rpm2cpio packagename.rpm | cpio -idmv

where rpm2cpio will change rpm file to cpio, -i is for extract, -d is to create leading directories where needed, -m is to preserve modification time and -v is for verbose mode.

Creating a lot of same sized files

2011-07-28T15:32:00.003+08:00

Let's say you need to create 10 files with 100kB size. All you need to do is create a 1MB files and split it into 10 files.

$ dd if=/dev/zero of=myfile bs=1024 count=1024
$ ls -lh
-rw-rw-r--. 1 owner owner 1.0M Jul 28 15:33 myfile

where if is input file, of is output file, bs is byte size and count is the multipler of bs

and then

$ split -b 100k myfile x
$ ls -lh
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xaa
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xab
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xac
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xad
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xae
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xaf
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xag
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xah
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xai
-rw-rw-r--. 1 owner owner 100K Jul 28 15:35 xaj
-rw-rw-r--. 1 owner owner  24K Jul 28 15:35 xak

where -b is byte size, myfile is your file input name and x is your prefix.

Convert time to unix timestamp and vice versa

2011-06-14T11:23:00.005+08:00

To convert current time to unix timestamp

$ date +%s
1308021863

where %s is seconds since 1970-01-01 00:00:00 UTC

To convert a unix timestamp to a readable time/date

$ date -d @1308021863
Mon Jun 13 22:24:23 CDT 2011

where -d is for displaying time described by the string after @

Install Al-Quran extension in openoffice

2011-05-25T17:13:00.004+08:00

This was done in Fedora 13 with openoffice 3.2.0. Please follow below steps to install it:

1. Download the installation files

$ wget http://qioo.googlecode.com/files/QiOO-0.4b.zip

2. Extract it to somewhere

$ unzip QiOO-0.4b.zip
$ ls QiOO-0.4b
CHANGELOG  QiOO-0.4b.oxt  README.txt  ScheherazadeRegOT.ttf

3. Copy ScheherazadeRegOT.ttf to /usr/share/fonts

$ sudo mkdir /usr/share/fonts/ScheherazadeRegOT; sudo cp -apv QiOO-0.4b/ScheherazadeRegOT.ttf /usr/share/fonts/ScheherazadeRegOT

4. Install the extension by opening openoffice writer and go to Tools > Extension Manager > Add, and search for the QiOO-0.4b.oxt plugin.

5. Activate CTL by going to Tools > Option > Language Settings > Languages > Enable For CTL(Complex Text Layout)

6. If all went well, you can see AlQuran menu in your openoffice writer's menubar like below.

That's all.

Cloning virtual machine on vmware esx using vmware-cmd

2011-05-20T16:44:00.010+08:00

Below are the steps:

1. List down all the virtual machines that you have

# /usr/bin/vmware-cmd -l
/vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/server-1/server-1.vmx
/vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/server-2/server-2.vmx
/vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/server-3/server-3.vmx

2. Shutdown the machine that you are going to clone. Let say the machine is server-1

# /usr/bin/vmware-cmd /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/server-1/server-1.vmx stop

3. Use hard stop if you cannot stop it using above command

# /usr/bin/vmware-cmd /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/server-1/server-1.vmx stop hard

4. Make sure the vm is shutdown by viewing the state

# /usr/bin/vmware-cmd /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/server-1/server-1.vmx getstate

5. Copy the whole directory

# cp /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/server-1/ /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/newserver/

6. Once done, rename all the files to new name

# rename server-1 newserver /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/newserver/*

7. Change all occurrence of "server-1" to "newserver" in .vmx, .vmdk and .vmxf

# cd /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/newserver/
# sed -i 's/server-1/newserver/g' {newserver.vmdk,newserver.vmx,newserver.vmxf}

8. Edit newserver.vmx file and remove below lines

ethernet0.generatedAddress = "....."
ethernet0.generatedAddressOffset = "....."
uuid.location = "....."
uuid.bios = "....."

9. Add below line to newserver.vmx

uuid.action = "keep"

10. Register your newly cloned virtual machine. Please use the full path to register as below

# vmware-cmd -s register /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/newserver/newserver.vmx

11. Start your new vm after registration.

# /usr/bin/vmware-cmd /vmfs/volumes/47971de9-34657d96-8ac4-001d09068c0c/newserver/newserver.vmx start

vim comment lines

2011-02-04T11:56:00.003+08:00

In linux, while editing configuration files, we always used # sign for comments. How do we do it to multiple lines in vi easily?

Steps:

1. Edit the files using vi
$ vi files.conf

2. set line number in vi (for us to easily determine what is the line number that we want to edit)
:se nu

3. replace start of lines with # (for example, put # sign in front of line 2 to line 5). The meaning of below command is for line 2 - 5, subtitute (s) start of line (^) with # sign, and press enter
:2,5s/^/#/
<enter>

4. It is done. Save your file (w) and quit (q)
:wq

yum plugin to remove dependencies

2010-12-02T09:45:00.003+08:00

You can remove dependencies using yum-plugin-remove-with-leaves plug-in.

To install:

# yum install yum-plugin-remove-with-leaves

To use:

# yum remove --remove-leaves <package-name>

That's all folks :)

Using markers in vi/vim

2010-11-26T06:18:00.006+08:00

This feature is useful if for example we want to delete a few lines or characters but we are too lazy to count the lines or characters to use dd or dw command. We can set up to 26 marker using lowercase a-z as the marker's name. The main usage of marker is to mark any location in file.

How to use (we put 'a' as the marker's name):

ma - mark current cursor position as marker named 'a'
`a (backquote a) - move to character marked as 'a'
'a (quote a) - move to first non blank character (line) containing marker 'a'
`` (backquote backquote) - move to last operated marker or toggles with last cursor position, if no marker is set, cursor moves to beginning of file (BOF)
'' (quote quote) - move to beginning og line (BOL) operated marker or toggles with BOL of last cursor position, if no marker is set, cursor moves to BOF

Example on using marker:

If you want to indent 5 lines starting from current line: majjjjj>'a
If you want to delete five words using marker: mawwwwwmb`ad`b

That's all friends :)

Edit main menu in fedora

2010-11-13T07:23:00.005+08:00

Starting from fedora 12, I found out that I cannot edit my main menu anymore. This is kind of frustrating since I installed many custom applications, and I would like it to be easily accessible from the main menu. After searching around, i found out that one package is not included in fedora 12 livecd. The package is called alacarte.

$ rpm -qi alacarte
Name        : alacarte                     Relocations: (not relocatable)
Version     : 0.12.4                            Vendor: Fedora Project
Release     : 1.fc12                        Build Date: Tue 22 Sep 2009 08:41:26 AM MYT
Install Date: Sat 13 Nov 2010 07:17:32 AM MYT      Build Host: xenbuilder4.fedora.phx.redhat.com
Group       : Applications/System           Source RPM: alacarte-0.12.4-1.fc12.src.rpm
Size        : 367423                           License: LGPLv2+
Signature   : RSA/8, Thu 01 Oct 2009 03:42:28 AM MYT, Key ID 9d1cc34857bbccba
Packager    : Fedora Project
URL         : http://www.gnome.org
Summary     : Menu editor for the GNOME desktop
Description :
Alacarte is a graphical menu editor that lets you edit, add, and delete
menu entries. It follows the freedesktop.org menu specification and
should work with any desktop environment that uses this specification.

So, just install the alacarte package using yum, and select "System -> Preferences -> Main Menu", and you can now edit your main menu.

# yum install alacarte -y

That's all folks

Xen's domU always depend on dom0's time and date

2010-10-26T11:02:00.006+08:00

By default, the time and date in domU is following dom0. To reset that, below are just what you have to do to decouple the domU's date from dom0

On dom0:

Append xen.independent_wallclock=1 to /etc/sysctl.conf

# echo "xen.independent_wallclock=1"  >> /etc/sysctl.conf

Activate the change

# sysctl -p

Append extra="clocksource=jiffies" to the domU's configuration file.

# echo "extra=\"clocksource=jiffies\"" >> /etc/xen/<domU's name>.cfg

On domU:

Append xen.independent_wallclock=1

# echo "xen.independent_wallclock=1"  >> /etc/sysctl.conf

Activate the change

# sysctl -p

Append jiffies to /sys/devices/system/clocksource/clocksource0/current_clocksource

# echo "jiffies" >> /sys/devices/system/clocksource/clocksource0/current_clocksource

Now you can set your domU's date using "date" command or "ntpdate" to get update from ntp servers.

That's all folks

Using ssh as proxy

2010-10-15T11:51:00.006+08:00

To create ssh tunnel is very easy. Let's say you want to use machine1.example.com as your ssh proxy, your username is foo and you choose port 8080 on your localhost to be your local proxy port. Type below command on your localhost:

$ ssh -D 8080 foo@machine1.example.com

Once connected, set your browser to use Socks5 proxy on port 8080 on your local machine. For firefox, the setting is "Tools -> Options -> Advanced -> Network -> Settings -> Manual proxy configuration" and follow below picture for the proxy setting. Just make sure to keep the ssh connection connected to reap the ssh proxy benefits :)

That's all :)

Cygwin man search problem

2010-10-15T10:19:00.003+08:00

If you are a windows user, and you use cygwin, there is a problem with man page. The search function is a bit weird, where it cannot find something like "-D" in ssh man page. To overcome this, you have to set the LANG environment variable to c. To do this, just add export LANG=c to your .bashrc:

$ echo "export LANG=c" >> .bashrc

That's all folks :)

Forcing user to change password the first time they log in

2010-10-07T14:32:00.005+08:00

If you are creating new user in your machine, and you would like them to change password the first time they login, using command chage is the way. Let's say our user is foo.

# chage -d 0 foo

where -d is for lastday. When we set it to 0, it will ask for password change the first time foo logs in.

That's all folks :)

Installing apache mod_fastcgi(fastcgi module) on CentOS 5

2010-08-11T12:38:00.004+08:00

In the previous post, I have written about installing apache mod_fcgid module to enable fastcgi support on apache. This time, I will write on how to install mod_fastcgi module to enable fastcgi module on apache installed on CentOS 5. Since by the time of this writing, there is no rpm for apache mod_fastcgi, we have to compile the mod_fastcgi module.

1. Install requirements for compilation

# yum install httpd-devel apr apr-devel libtool

2. Download latest mod_fastcgi source code

# cd /opt
# wget http://www.fastcgi.com/dist/mod_fastcgi-current.tar.gz

3. Untar the package.

# tar -xvzf mod_fastcgi-current.tar.gz

4. Install the module. You can find the installation guide on the INSTALL.AP2 file. We have to specify top_dir in the make and make install commands because we install apache2/httpd using yum

# cd mod_fastcgi-2.4.6
# cp Makefile.AP2 Makefile
# make top_dir=/usr/lib/httpd
# make install top_dir=/usr/lib/httpd

5. Add "LoadModule fastcgi_module modules/mod_fastcgi.so" to /etc/httpd/conf/httpd.conf to tell apache to load the new module

6. Restart apache

# /etc/init.d/httpd restart

7. You can assure that the mod is loaded by apache2, by looking at /var/log/httpd/error_log

# grep -i "FastCGI" /var/log/httpd/error_log 

[Wed Aug 11 12:26:27 2010] [notice] FastCGI: process manager initialized (pid 8853)

That's all :)

Installing apache mod_fcgid(fastcgi module) on CentOS 5

2010-08-11T10:16:00.009+08:00

For apache to support FastCGI, you have to install either mod_fastcgi or mod_fcgid. In this example, I will show how to install mod_fcgid on existing apache webserver on centos 5.

First of all, the rpm is available at kbsingh's centos testing repository.To install kbsingh's centos repository:

1. Download kbsingh-CentOS-Extras.repo to /etc/yum.repos.d/

# cd /etc/yum.repos.d
# wget http://centos.karan.org/kbsingh-CentOS-Extras.repo

2. Enable karansingh's testing repo by setting gpgcheck to 0 and enabled to 1 in the [kbs-CentOS-Testing]

# sed -i "s/enabled=0/enabled=1/g" /etc/yum.repos.d/kbsingh-CentOS-Extras.repo

3. Install mod_fcgid

# yum install mod_fcgid

4. Restart apache, and verify whether fcgid_module is available by using the second command below

# /etc/init.d/httpd restart
# httpd -t -D DUMP_MODULES

5. If the module is still not loaded, add "LoadModule fcgid_module modules/mod_fcgid.so" to your /etc/httpd/conf/httpd.conf, and restart apache.

That's all :)