Anoop's Blog

What's new in RHEL6

2011-06-23T23:26:00.000-07:00

Details are provided at http://www.europe.redhat.com/products/rhel/server/details/

Some important ones are:
1. Red Hat Enterprise Linux 6 supports more sockets, more cores, more threads, and more memory.
2. Memory pages with errors can be declared as "poisoned", and will be avoided.
3. The new default file system, ext4, is faster, more robust, and scales to 16TB.
4. Redhat Cluster Nodes can re-enable themselves after failure without administrative intervention using unfencing.
5. iSCSI partitions may be used as either root or boot filesystems.
6. The new System Security Services Daemon (SSSD) provides centralized access to identity and authentication resources, enables caching and offline support.
7. Dracut has been introduced as an replacement of mkinitrd.
7. XEN has been replaced with KVM.

Live upgrade of ZFS root file system

2011-06-22T14:18:00.001-07:00

1. Create a new boot environment to patch or uprade

# lucreate –n newBE

2. Upgrade the new Boot environment

# luupgrade –u –n newBE –s /cdrom

3. Activate the new Boot Environment

# luactivate newBE

4. Boot from new boot environment

# init 6

Live upgrade of Veritas mirrored system

2011-06-22T14:11:00.000-07:00

System had two disks c1t0d0 and c1t1d0, so first break mirror and remove disk c1t1d0 from disk group.

1. # vxplex -g rootdg dis rootvol-02 swapvol-02 home-02

2. # vxdg -g rootdg -k rmdisk rootdg02

3. Create a file autoreg with entry “auto_reg=disable”

4. Modify vxlustart script and mention auto registration disable by putting –k /tmp/autoreg into command running luupgrade otherwise it will fail.

5. ./vxlustart -u 5.10 -d c1t1d0s2 -g rootdg -s /cdrom

6. ./vxlufinish –u 5.10 –g <new root disk group name>

7. shutdown –y –g0 –i6

8. After few reboots system will come up with new Boot environment, mirror the rootdisk and remove old rootdg.

Create users in different Unix Flavors

2011-06-22T14:01:00.000-07:00

In Aix
# mkuser id='<id>' pgrp='<group name>' groups='<secondary group>' su='true' home='/home/mqm' gecos='<Comment>' maxage='0' <user id>

In Solaris/RHEL

#useradd -g <group> -d <home directory> -c <Comment> <user id>

How to add Solaris server to Nagios for monitoring

2011-03-25T13:53:00.000-07:00

1. Create nagios user and group

2. Download Nagios plugin and nrpe daemon from http://www.guntram.de/nagios/ for appropriate solaris version

3. Unzip and untar the files under /usr/local/nagios

4. Create xml file to import nrpe as SMF service or take it from here.
http://unixjournal.org/tag/nrpe/

5. Import xml file into SMF
# svccfg import /var/svc/manifest/network/nrpe.xml

6. Keep the service disabled for time being
# svcs -a|grep nrp
disabled 17:09:11 svc:/network/nrpe:default

7. Create an nrpe file under /lib/svc/method
# more /lib/svc/method/nrpe
#!/sbin/sh
#

LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH
PIDFILE=/var/run/nrpe.pid
NRPE_BIN=/usr/local/nagios/bin/nrpe
CONFIG_FILE=/usr/local/nagios/etc/nrpe.cfg

case $1 in
# SMF arguments (start and restart [really "refresh"])
'start')
$NRPE_BIN -d -c $CONFIG_FILE
;;

'restart')
if [ -f "$PIDFILE" ]; then
/usr/bin/kill -HUP `/usr/bin/cat $PIDFILE`
fi
;;

'stop')
if [ -f "$PIDFILE" ]; then
/usr/bin/kill `/usr/bin/cat $PIDFILE`
fi
;;
*)
echo "Usage: $0 { start | stop | restart }"
exit 1
;;
esac

exit $?
#
8. Start the nrpe service
# svcadm enable nrpe
# svcs -a|grep nrp
online 17:17:31 svc:/network/nrpe:default

9. Now modify /usr/local/nagios/etc/nrpe.cfg file to include what will be monitored on this server.

command[check_local_disk]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /
command[check_local_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_local_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_local_procs]=/usr/local/nagios/libexec/check_procs -w 350 -c 400
command[check_zfs_rpool]=/usr/local/nagios/libexec/check_zfs1 rpool 1
command[check_zfs_datapool]=/usr/local/nagios/libexec/check_zfs1 datapool 1
command[check_fmd]=/usr/local/nagios/libexec/check_fmd
command[check_meta]=/usr/local/nagios/libexec/check_meta

Remember to download check_fmd, check_meta and check_zfs binaries from

http://exchange.nagios.org/directory/Plugins/Uncategorized/Operating-Systems/Solaris

10. Now configuration change has to be made on Nagios server. In the /usr/local/nagios/etc/objects/solaris.cfg file add following

# Define a host for the hostname.domain.org

define host{
use solaris-server host_name hostname.domain.org
alias hostname
address 100.100.100.15
contact_groups Admins
}
###############################################################################
###############################################################################
#
# SERVICE DEFINITIONS
#
###############################################################################
###############################################################################
# Define a service to "ping" the local machine

define service{
use Remote-service ; Name of service template to use
host_name hostname.domain.org
service_description PING
check_command check_ping!100,20%!500,60%
}

# Define a service to check the disk space of the root partition
# on the Remote machine. Warning if < 20% free, critical if # < 10% free space on partition. define service{ use Remote-service ; Name of service template to use host_name hostname.domain.org service_description Root Partition check_command check_nrpe_1arg!check_local_disk } # Define a service to check the number of currently logged in # users on the Remote machine. Warning if > 20 users, critical
# if > 50 users.

define service{
use Remote-service ; Name of service template to use
host_name hostname.domain.org
service_description Current Users
check_command check_nrpe_1arg!check_local_users
}

# Define a service to check the number of currently running procs
# on the Remote machine. Warning if > 250 processes, critical if
# > 400 users.

define service{
use Remote-service ; Name of service template to use
host_name hostname.domain.org
service_description Total Processes
check_command check_nrpe_1arg!check_local_procs
}
# Define a service to check the load on the Remote machine.

define service{
use Remote-service ; Name of service template to use
host_name hostname.domain.org
service_description Current Load
check_command check_nrpe_1arg!check_local_load
}
define service{
use Remote-service ; Name of service template to use
host_name hostname.domain.org
service_description Solaris Fault Manager
check_command check_nrpe_1arg!check_fmd
}
define service{
use Remote-service ; Name of service template to use
host_name hostname.domain.org
service_description Meta Device Status
check_command check_nrpe_1arg!check_meta
}

# Define a service to check SSH on the Remote machine.
# Disable notifications for this service by default, as not all users may have SSH enabled.

define service{
use Remote-service ; Name of service template to use
host_name hostname.domain.org
service_description SSH
check_command check_ssh
}

11. Note the check_nrpe_1arg command used instead of default check_nrpe (since i am not using any arguments. we should have a command by name check_nrpe_1arg in commands.cfg like below.

# # 'check_nrpe_1arg' command definition
define command{
command_name check_nrpe_1arg
command_line /usr/local/nagios/libexec/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}

12. Restart nagios service on Nagios server and in few minutes solaris client will be monitored.

Problems encountered.

1. could not complete ssl handshake

Downloaded nrpe is compiled with ssl support. Make sure that on server command check_nrpe_1arg does not has -n with it as that is used for without ssl support.

2. no output returned from plugin

Try running command from server using command line. for example
#pwd
/usr/local/nagios/libexec
# ./check_nrpe -H hostname.domain.org -c check_meta
OK - No disk failures detected

If you get the above output it means you are using check_nrpe instaed of check_nrpe_1arg. Check_nrpe expects some arguments which we do not require.

How to install nagios server in Unix

2011-03-21T13:34:00.000-07:00

1. Download nagios software and Nagios plugin from http://www.nagios.org/download/

2. Install gcc, libgcc for compilation.

3. Create User nagios and group nagios

4. Install Nagios

# tar –xvf nagios-3.2.3.tar
# cd nagios-3.2.3
# ./configure –with-command-group=nagcmd
# make all
# make install
# make install-config
#make install-commandmode

5. Configure the web interface
# make install-webconf
# htpasswd –c /usr/local/nagios/etc/htpasswd.users nagiosadmin
Enter the password

6. Compile and install nagios plugin
# tar –xvf nagios-plugin-1.4.11.tar
# cd nagios-plugin-1.4.11
# ./configure - -with-nagios-user=nagios - -with-nagios-group=nagios
#make
#make install

7. Start Nagios
#chkconfig - -add nagios
#chkconfig nagios on
# service nagios start

8. Test nagios configuration file
# /usr/local/nagios/bin/nagios –v /usr/local/nagios/etc/nagios.cfg
9. Login to nagios web interface at http://hostname/nagios/ using nagiosadmin account.

Check Ramesh Natrajan's blog at http://www.thegeekstuff.com/2008/05/nagios-30-jumpstart-guide-for-red-hat-overview-installation-and-configuration/

from where i took help to setup my Nagios server

How to take snapshot in M4000 for oracle support

2010-10-29T06:38:00.000-07:00

To take snapshot for Oracle support.
1. Go to XSCF and login using admin account.
2. Connect the USB drive (FAT 32) in the Service Processor
3. XSCF> snapshot -d usb0

Copy the zip file from USB to Sun FTP site.

How to configure XCSF network in M4000

2010-09-28T11:15:00.000-07:00

1. To assign an IP address to XSCF interface
XSCF> setnetwork xscf#1-lan#0 -m 255.255.255.0 192.168.0.10
2. Enable the specified network interface
XSCF> setnetwork -c up
3. XSCF> shownetwork
4. Add route
XSCF> setroute -c add -n -m -g interface
5. XSCF> applynetwork
6. To set Service Processor Host Name and DNS name
XSCF> sethostname xscf#0
7. To set Service Processor's DNS Name Server
XSCF> setnameserver ip_address
8. To enable Service Processor HTTPS service
XSCF> sethttps -c enable
9. To enable Service Processor SSH service
XSCF> setssh -c enable
XSCF> setssh -c genhostkey

How to configure XSCF in M4000

2010-09-28T10:46:00.000-07:00

1. login to XSCF console
2. setpasswordpolicy

How to configure DSCP Network in M4000

2010-09-23T09:12:00.000-07:00

XSCF> setdscp
DSCP network [0.0.0.0 ] > 10.1.1.0

DSCP netmask [255.0.0.0 ] > 255.255.255.0

XSCF address [10.1.1.1 ] > [Enter]
Domain #00 address [10.1.1.2] > [Enter]
.
.
.
Commit these changes to the database? [y|n]:y

XSCF> rebootxscf

How to verify DSCP is configured on M4000

2010-09-23T08:08:00.000-07:00

To Display DSCP information.
XSCF> showdscp

DSCP Configuration:

Network: 192.168.1.0
Netmask: 255.255.255.0

Location      Address
----------   ---------
XSCF         192.168.1.1
Domain #00   192.168.1.2
Domain #01   192.168.1.3
Domain #02   192.168.1.4
Domain #03   192.168.1.5
If it is not configured showdscp output will state this:
XSCF> showdscp

ERROR: DSCP is not configured. Please use setdscp.

First time configuration of M4000

2010-09-03T11:37:00.000-07:00

1. connect laptop to serial port
2. Open Tera Term
3. Set the key on the operator panel to the service position (wrench icon)
4. Turn on the power
5. when login prompt appears enter "default" for the login name
6. Set the key on the operator panel to the Locked position
7. Press the RETURN key
8. Wait for 5 seconds and then set the key on the operator panel to the service position
9. Press the RETURN key
10. Confirm that the XSCF shell prompt is displayed

How to expand an existing metadevice and file system

2010-06-22T14:42:00.000-07:00

1. Attach another slice to existing metadevice

# metattach d0 c1t2d0s2

2. # growfs –M /home2 /dev/md/rdsk/d0

How to change Master Host in SGE

2009-05-10T11:20:00.000-07:00

1. On current master host stop the daemons.
# qconf -km

2. Edit the $SGE_ROOT/$SGE_CELL/common/act_qmaster file according to the following guidelines:

update the act_qmaster file with the output of follwoing command executed on new master host.
# $SGE_ROOT/utilbin/$SGE_ARCH/gethostname

3. On the New Master Host start qmaster
# $SGE_ROOT/$SGE_CELL/common/sgemaster

Configuration of Sun Grid Engine

2009-04-13T13:36:00.000-07:00

1. How to find the name of master host
The location of the master host can migrate between the current master host and one of the shadow master hosts at any time. Therefore, the location of the master host should be transparent to the user.Check below file for current master host.

$SGE_ROOT/$SGE_CELL/common/act_qmaster file.

2. To list all queues
# qconf -sql

3. Each queue is associated with a hostgroup. To List Host group
# qconf -shgrpl

4. To See all execution hosts
# qconf -sel

5. To See all administrative hosts
# qconf -sh

6. To see all submit hosts
#qconf -ss

7. To See all configured resource quota
qconf -srqsl

8. To display configuration of an existing project
# qconf -sprj project-name

9. To display all parallel environment
#qconf -spl

10.To Add a host as administrative host
# qconf -ah hostname

11. To remove a host as administrative host
# qconf -dh hostname

12. To add a host as submit host
# qconf -as hostname

13. To add a new queue
#qconf -aq

14. To add a host group
# qconf -ahgrp host-group-name

15. To add a new project
# qconf -aprj

16. To add a new resource quota
# qconf -arqs

17. To add a new parallel environment
qconf -ap pe-name

Other commands to monitor SGE

qhost — View current status of the available Grid Engine hosts, the queues, and the jobs associated with the queues
qalter and qsub — Submit jobs
qstat — Show the status of Grid Engine jobs and queues
qquota — List each resource quota that is being used at least once or that defines a static limit

How to install Redhat cluster

2009-03-15T08:26:00.000-07:00

How to install Redhat cluster

1. yum groupinstall -y “Cluster Storage” “Clustering”

2. yum install -y iscsi-initiator-utils isns-utils

3. chkconfig iscsi on
chkconfig iscsid on

4. The two “rhel-cluster-nodeX” systems have two NICs, one for production and one for HighAvailability check.
rhel-cluster-node1
192.168.234.201
10.10.10.1
rhel-cluster-node2
192.168.234.202
10.10.10.2

5. Add hostnames of both hosts to /etc/hosts file on all nodes.

6. #lsscsi should show the shared storage device, if not rescan the devices.
echo "- - -" > /sys/class/scsi_host/host0/scan

7. Suppose the device is /dev/sdb
#pvcreate /dev/sdb
#vgcreate vg1 /dev/sdb
lvcreate -l 10239 -n lv0 vg1

8. Create the GFS file system with dlm lock for max 8 hosts
# gfs_mkfs -p lock_dlm -t rhel-cluster:storage1 -j 8 /dev/vg1/lv0

9. To administer Red Hat Clusters with Conga, run luci and ricci as follows :
service luci start
service ricci start

10. On both systems, initialize the luci server using the luci_admin init command.
service luci stop
luci_admin init
This command create the ‘admin’ user and his password, for doing so follow on screen instruction, and check for an output as the following :
The admin password has been successfully set.
Generating SSL certificates…
The luci server has been successfully initialized
You must restart the luci server for changes to take effect, run the following to do it :

11. # service luci restart

12. Now start following cluster services

# service rgmanager start
# service cman start

13. Edit the fstab file as below.
/dev/vg1/lv0 /data gfs defaults,acl 0 0

14. Point your web browser to https://rhel-cluster-node1:8084 to access luci

15. As administrator of luci, select the cluster tab.
Click Create a New Cluster.
At the Cluster Name text box, enter cluster name “rhel-cluster.
Add the node name and password for each cluster node.
Click Submit. Clicking Submit causes the following actions:

a. Cluster software packages to be downloaded onto each cluster node.
b. Cluster software to be installed onto each cluster node.
c. Cluster configuration file to be created and propagated to each node in the cluster.
d. Starting the cluster.
A progress page shows the progress of those actions for each node in the cluster.
When the process of creating a new cluster is complete, a page is displayed providing a
configuration interface for the newly created cluster.

16. Managing your newly created cluster you can ad resources.
Add a resource, choose IP Address and use 192.168.234.200

17. Create a service named “cluster”, add the resource “IP Address” you had created before,
check “Automatically start this service”
check “Run exclusive”
choose “Recovery policy” as “Relocate”
Save the service.

18. If the service created give no errors, enable it, and try to start it on one cluster node.

19. The Cluster configuration file would be /etc/cluster/cluster.conf

Configure ISCSI storage with RHEL5

2009-03-06T15:05:00.000-08:00

In iSCSI (block level protocol on IP) parlance, the device where data is stored is called the target. This is usually a SAN or NAS device. The program or device on the server that handles communication with the iSCSI target is called the initiator. Red Hat ships a software-based initiator with RHEL.

1. Install iscsi-initiator-utils package.
# yum install iscsi-initiator-utils

2. Configure iSCSI by editing /etc/iscsi/iscsid.conf

node.session.auth.username = My_ISCSI_USR_NAME
node.session.auth.password = MyPassword
discovery.sendtargets.auth.username = My_ISCSI_USR_NAME
discovery.sendtargets.auth.password = MyPassword

3. Start iscsi service
# /etc/init.d/iscsi start

4. Discover Target storage (suppose with IP address 192.168.0.10)
# iscsiadm -m discovery -t sendtargets -p 192.168.0.10
#/etc/init.d/iscsi restart

5. Now new device should be available to system. do below
# fdisk -l

6. Partition Disk
#fdisk /dev/sdd

7. Create file system.
# mkfs.ext3 /dev/sdd1

8. Modify /etc/fstab for auto mounting of file system
/dev/sdd1 /mnt/iscsi ext3 _netdev 0 0

How to install Sun Grid Engine 6.2

2009-02-18T11:55:00.000-08:00

Sun Grid Engine, Enterprise Edition softwate provides advanced resource management and policy administration for UNIX environments that are composed of multiple shared resources.
Sun Grid Engine provides the user with the means to submit computationally demanding tasks to the Sun Grid Engine, Enterprise Edition system for transparent distribution of the associated workload. The user can submit batch jobs, interactive jobs, and parallel jobs to the Sun Grid Engine.
The Sun Grid Engine accepts jobs—users’ requests for computer resources—from the outside world, puts them in a holding area until they can be executed, sends them from the holding area to an execution device, manages them during execution, and logs the record of their execution when they are finished.

Four types of hosts are fundamental to the Sun Grid Engine.
1. Master
2. Execution
3. Administration
4. Submit
First two are installed and later two are added to configuration.

To Install master host.
[root@rhel1 6.2.u5]# ./inst_sge -m
Do you agree with that license? (y/n) [n] >> y
Grid Engine admin user account -- sgeadmin
Do you want to install Grid Engine as admin user >sgeadmin< (y/n) [y] >>

Installing Grid Engine as admin user >sgeadmin< Hit to continue >>

Checking $SGE_ROOT directory
----------------------------

The Grid Engine root directory is:

$SGE_ROOT = /common/sge/6.2.u5

If this directory is not correct (e.g. it may contain an automounter
prefix) enter the correct path to this directory or hit
to use default [/common/sge/6.2.u5] >>

Your $SGE_ROOT directory: /common/sge/6.2.u5

Hit to continue >>

Grid Engine TCP/IP communication service
----------------------------------------

The port for sge_qmaster is currently set as service.

sge_qmaster service set to port 536

Now you have the possibility to set/change the communication ports by using the
>shell environment< or you may configure it via a network service, configured in local >/etc/service<, >NIS< or >NIS+<, adding an entry in the form sge_qmaster /tcp

to your services database and make sure to use an unused port number.

How do you want to configure the Grid Engine communication ports?

Using the >shell environment<: [1] Using a network service like >/etc/service<, >NIS/NIS+<: [2] (default: 2) >>

Grid Engine TCP/IP service >sge_qmaster< ---------------------------------------- Using the service sge_qmaster for communication with Grid Engine. Hit to continue >>

Grid Engine TCP/IP communication service
----------------------------------------

The port for sge_execd is currently set as service.

sge_execd service set to port 537

Now you have the possibility to set/change the communication ports by using the
>shell environment< or you may configure it via a network service, configured in local >/etc/service<, >NIS< or >NIS+<, adding an entry in the form sge_execd /tcp

to your services database and make sure to use an unused port number.

How do you want to configure the Grid Engine communication ports?

Using the >shell environment<: [1] Using a network service like >/etc/service<, >NIS/NIS+<: [2] (default: 2) >>

Grid Engine TCP/IP communication service
-----------------------------------------

Using the service

sge_execd

for communication with Grid Engine.

Hit to continue >>

Grid Engine cells
-----------------

Grid Engine supports multiple cells.

If you are not planning to run multiple Grid Engine clusters or if you don't
know yet what is a Grid Engine cell it is safe to keep the default cell name

default

If you want to install multiple cells you can enter a cell name now.

The environment variable

$SGE_CELL=

will be set for all further Grid Engine commands.

Enter cell name [default] >> Rahway_SciComp

Using cell >Rahway_SciComp<. Hit to continue >>

Unique cluster name
-------------------

The cluster name uniquely identifies a specific Sun Grid Engine cluster.
The cluster name must be unique throughout your organization. The name
is not related to the SGE cell.

The cluster name must start with a letter ([A-Za-z]), followed by letters,
digits ([0-9]), dashes (-) or underscores (_).

Enter new cluster name or hit
to use default [p536] >> usrylxap1
creating directory: /common/sge/6.2.u5/Rahway_SciComp/common

Your $SGE_CLUSTER_NAME: usrylxap1

Hit to continue >>

Grid Engine qmaster spool directory
-----------------------------------

The qmaster spool directory is the place where the qmaster daemon stores
the configuration and the state of the queuing system.

The admin user >sgeadmin< must have read/write access to the qmaster spool directory. If you will install shadow master hosts or if you want to be able to start the qmaster daemon on other hosts (see the corresponding section in the Grid Engine Installation and Administration Manual for details) the account on the shadow master hosts also needs read/write access to this directory. Enter a qmaster spool directory [/common/sge/6.2.u5/Rahway_SciComp/spool/qmaster] >>

Using qmaster spool directory >/common/sge/6.2.u5/Rahway_SciComp/spool/qmaster<. Hit to continue >>
Windows Execution Host Support
------------------------------

Are you going to install Windows Execution Hosts? (y/n) [n] >> n

Verifying and setting file permissions
--------------------------------------

Did you install this version with >pkgadd< or did you already verify and set the file permissions of your distribution (enter: y) (y/n) [y] >>

We do not verify file permissions. Hit to continue >>

Select default Grid Engine hostname resolving method
----------------------------------------------------

Are all hosts of your cluster in one DNS domain? If this is
the case the hostnames

>hostA< and >hostA.foo.com< would be treated as equal, because the DNS domain name >foo.com< is ignored when comparing hostnames. Are all hosts of your cluster in a single DNS domain (y/n) [y] >>

Ignoring domain name when comparing hostnames.

Hit to continue >>

Grid Engine JMX MBean server
----------------------------

In order to use the SGE Inspect or the Service Domain Manager (SDM)
SGE adapter you need to configure a JMX server in qmaster. Qmaster
will then load a Java Virtual Machine through a shared library.
NOTE: Java 1.5 or later is required for the JMX MBean server.

Do you want to enable the JMX MBean server (y/n) [y] >> n

Making directories
------------------

creating directory: /common/sge/6.2.u5/Rahway_SciComp/spool/qmaster
creating directory: /common/sge/6.2.u5/Rahway_SciComp/spool/qmaster/job_scripts
Hit to continue >>

Setup spooling
--------------
Your SGE binaries are compiled to link the spooling libraries
during runtime (dynamically). So you can choose between Berkeley DB
spooling and Classic spooling method.
Please choose a spooling method (berkeleydb|classic) [berkeleydb] >>

The Berkeley DB spooling method provides two configurations!

Local spooling:
The Berkeley DB spools into a local directory on this host (qmaster host)
This setup is faster, but you can't setup a shadow master host

Berkeley DB Spooling Server:
If you want to setup a shadow master host, you need to use
Berkeley DB Spooling Server!
In this case you have to choose a host with a configured RPC service.
The qmaster host connects via RPC to the Berkeley DB. This setup is more
failsafe, but results in a clear potential security hole. RPC communication
(as used by Berkeley DB) can be easily compromised. Please only use this
alternative if your site is secure or if you are not concerned about
security. Check the installation guide for further advice on how to achieve
failsafety without compromising security.
Do you want to use a Berkeley DB Spooling Server? (y/n) [n] >> n

Hit to continue >>

Berkeley Database spooling parameters
-------------------------------------

Please enter the database directory now, even if you want to spool locally,
it is necessary to enter this database directory.

Default: [/common/sge/6.2.u5/Rahway_SciComp/spool/spooldb] >> /home/sge
The spooling directory already exists! Do you want to delete it? [n] >> y

creating directory: /home/sge
Dumping bootstrapping information
Initializing spooling database

Hit to continue >>

Grid Engine group id range
--------------------------

When jobs are started under the control of Grid Engine an additional group id
is set on platforms which do not support jobs. This is done to provide maximum
control for Grid Engine jobs.

This additional UNIX group id range must be unused group id's in your system.
Each job will be assigned a unique id during the time it is running.
Therefore you need to provide a range of id's which will be assigned
dynamically for jobs.

The range must be big enough to provide enough numbers for the maximum number
of Grid Engine jobs running at a single moment on a single host. E.g. a range
like >20000-20100< means, that Grid Engine will use the group ids from 20000-20100 and provides a range for 100 Grid Engine jobs at the same time on a single host. You can change at any time the group id range in your cluster configuration. Please enter a range [20000-20100] >>

Using >20000-20100< as gid range. Hit to continue >>

Grid Engine cluster configuration
---------------------------------

Please give the basic configuration parameters of your Grid Engine
installation:

The pathname of the spool directory of the execution hosts. User >sgeadmin< must have the right to create this directory and to write into it. Default: [/common/sge/6.2.u5/Rahway_SciComp/spool] >>

Grid Engine cluster configuration (continued)
---------------------------------------------

The email address of the administrator to whom problem reports are sent.

It is recommended to configure this parameter. You may use >none< if you do not wish to receive administrator mail. Please enter an email address in the form >user@foo.com<. Default: [none] >> abc@xyz.com

The following parameters for the cluster configuration were configured:

execd_spool_dir /common/sge/6.2.u5/Rahway_SciComp/spool
administrator_mail abc@xyz.com
Do you want to change the configuration parameters (y/n) [n] >>

Creating local configuration
----------------------------
Creating >act_qmaster< file Adding default complex attributes Adding default parallel environments (PE) Adding SGE default usersets Adding >sge_aliases< path aliases file Adding >qtask< qtcsh sample default request file Adding >sge_request< default submit options file Creating >sgemaster< script Creating >sgeexecd< script Creating settings files for >.profile/.cshrc< Hit to continue >>

qmaster startup script
----------------------

We can install the startup script that will
start qmaster at machine boot (y/n) [y] >> y

cp /common/sge/6.2.u5/Rahway_SciComp/common/sgemaster /etc/init.d/sgemaster.usrylxap1
/usr/lib/lsb/install_initd /etc/init.d/sgemaster.usrylxap1

Hit to continue >>

Grid Engine qmaster startup
---------------------------

Starting qmaster daemon. Please wait ...
starting sge_qmaster
Hit to continue >>

Adding Grid Engine hosts
------------------------

Please now add the list of hosts, where you will later install your execution
daemons. These hosts will be also added as valid submit hosts.

Please enter a blank separated list of your execution hosts. You may
press if the line is getting too long. Once you are finished
simply press without entering a name.

You also may prepare a file with the hostnames of the machines where you plan
to install Grid Engine. This may be convenient if you are installing Grid
Engine on many hosts.

Do you want to use a file which contains the list of hosts (y/n) [n] >> n

Adding admin and submit hosts
-----------------------------

Please enter a blank seperated list of hosts.

Stop by entering . You may repeat this step until you are
entering an empty list. You will see messages from Grid Engine
when the hosts are added.

Host(s): abc123.xyz.com abc124.xyz.com abc125.xyz.com abc126.xyz.com

Above hosts are added as administrative and submit hosts.
Do you want to add your shadow host(s) now? (y/n) [y] >> n

Creating the default queue and hostgroup
-----------------------------------------------------------

root@abc123.xyz.com added "@allhosts" to host group list
root@abc123.xyz.com added "all.q" to cluster queue list

Hit to continue >>
Scheduler Tuning
----------------

The details on the different options are described in the manual.

Configurations
--------------
1) Normal
Fixed interval scheduling, report limited scheduling information,
actual + assumed load

2) High
Fixed interval scheduling, report limited scheduling information,
actual load

3) Max
Immediate Scheduling, report no scheduling information,
actual load

Enter the number of your preferred configuration and hit !
Default configuration is [1] >>

We're configuring the scheduler with >Normal< settings! Do you agree? (y/n) [y] >> y

changed scheduler configuration

Using Grid Engine
-----------------

You should now enter the command:

source /common/sge/6.2.u5/Rahway_SciComp/common/settings.csh

if you are a csh/tcsh user or

# . /common/sge/6.2.u5/Rahway_SciComp/common/settings.sh

if you are a sh/ksh user.

This will set or expand the following environment variables:

- $SGE_ROOT (always necessary)
- $SGE_CELL (if you are using a cell other than >default<) - $SGE_CLUSTER_NAME (always necessary) - $SGE_QMASTER_PORT (if you haven't added the service >sge_qmaster<) - $SGE_EXECD_PORT (if you haven't added the service >sge_execd<) - $PATH/$path (to find the Grid Engine binaries) - $MANPATH (to access the manual pages) Hit to see where Grid Engine logs messages >>

Grid Engine messages
--------------------

Grid Engine messages can be found at:

/tmp/qmaster_messages (during qmaster startup)
/tmp/execd_messages (during execution daemon startup)

After startup the daemons log their messages in their spool directories.

Qmaster: /common/sge/6.2.u5/Rahway_SciComp/spool/qmaster/messages
Exec daemon: //messages

Grid Engine startup scripts
---------------------------

Grid Engine startup scripts can be found at:

/common/sge/6.2.u5/Rahway_SciComp/common/sgemaster (qmaster)
/common/sge/6.2.u5/Rahway_SciComp/common/sgeexecd (execd)

Do you want to see previous screen about using Grid Engine again (y/n) [n] >>

Your Grid Engine qmaster installation is now completed
------------------------------------------------------
lease now login to all hosts where you want to run an execution daemon
and start the execution host installation procedure.

If you want to run an execution daemon on this host, please do not forget
to make the execution host installation in this host as well.

All execution hosts must be administrative hosts during the installation.
All hosts which you added to the list of administrative hosts during this
installation procedure can now be installed.

You may verify your administrative hosts with the command

# qconf -sh

and you may add new administrative hosts with the command

# qconf -ah

Please hit >>
sge_qmaster successfully installed!

Execution host installation

[root@abc124.xyz.com.u5]# ./inst_sge -x

Welcome to the Grid Engine execution host installation
------------------------------------------------------

If you haven't installed the Grid Engine qmaster host yet, you must execute
this step (with >install_qmaster<) prior the execution host installation. For a sucessfull installation you need a running Grid Engine qmaster. It is also neccesary that this host is an administrative host. You can verify your current list of administrative hosts with the command: # qconf -sh You can add an administrative host with the command: # qconf -ah

The execution host installation will take approximately 5 minutes.

Hit to continue >>

Checking $SGE_ROOT directory
----------------------------

The Grid Engine root directory is:

$SGE_ROOT = /common/sge/6.2.u5

If this directory is not correct (e.g. it may contain an automounter
prefix) enter the correct path to this directory or hit
to use default [/common/sge/6.2.u5] >>

Your $SGE_ROOT directory: /common/sge/6.2.u5

Hit to continue >>

Grid Engine cells
-----------------

Please enter cell name which you used for the qmaster
installation or press to use [Rahway_SciComp] >>

Using cell: >Rahway_SciComp< Hit to continue >>

Grid Engine TCP/IP communication service
----------------------------------------

The port for sge_execd is currently set as service.

sge_execd service set to port 537

Hit to continue >>

Checking hostname resolving
---------------------------

This hostname is known at qmaster as an administrative host.

Hit to continue >>

Execd spool directory configuration
-----------------------------------

You defined a global spool directory when you installed the master host.
You can use that directory for spooling jobs from this execution host
or you can define a different spool directory for this execution host.

ATTENTION: For most operating systems, the spool directory does not have to
be located on a local disk. The spool directory can be located on a
network-accessible drive. However, using a local spool directory provides
better performance.

FOR WINDOWS USERS: On Windows systems, the spool directory MUST be located
on a local disk. If you install an execution daemon on a Windows system
without a local spool directory, the execution host is unusable.

The spool directory is currently set to:
<>

Do you want to configure a different spool directory
for this host (y/n) [n] >>

Creating local configuration
----------------------------
sgeadmin@abc124.xyz.com modified "abc124.xyz.com" in configuration list
Local configuration for host >abc124.xyz.com< created. Hit to continue >>

execd startup script
--------------------

We can install the startup script that will
start execd at machine boot (y/n) [y] >>

cp /common/sge/6.2.u5/Rahway_SciComp/common/sgeexecd /etc/init.d/sgeexecd.usrylxap1
/usr/lib/lsb/install_initd /etc/init.d/sgeexecd.usrylxap1

Hit to continue >>

Grid Engine execution daemon startup
------------------------------------

Starting execution daemon. Please wait ...
starting sge_execd

Hit to continue >>

Adding a queue for this host
----------------------------

We can now add a queue instance for this host:

- it is added to the >allhosts< hostgroup - the queue provides 4 slot(s) for jobs in all queues referencing the >allhosts< hostgroup You do not need to add this host now, but before running jobs on this host it must be added to at least one queue. Do you want to add a default queue instance for this host (y/n) [y] >>

root@abc124.xyz.com modified "@allhosts" in host group list
root@abc124.xyz.com modified "all.q" in cluster queue list

Hit to continue >>

Using Grid Engine
-----------------

You should now enter the command:

source /common/sge/6.2.u5/Rahway_SciComp/common/settings.csh

if you are a csh/tcsh user or

# . /common/sge/6.2.u5/Rahway_SciComp/common/settings.sh

if you are a sh/ksh user.

This will set or expand the following environment variables:

- $SGE_ROOT (always necessary)
- $SGE_CELL (if you are using a cell other than >default<) - $SGE_CLUSTER_NAME (always necessary) - $SGE_QMASTER_PORT (if you haven't added the service >sge_qmaster<) - $SGE_EXECD_PORT (if you haven't added the service >sge_execd<) - $PATH/$path (to find the Grid Engine binaries) - $MANPATH (to access the manual pages) Hit to see where Grid Engine logs messages >>

Grid Engine messages
--------------------

Grid Engine messages can be found at:

/tmp/qmaster_messages (during qmaster startup)
/tmp/execd_messages (during execution daemon startup)

After startup the daemons log their messages in their spool directories.

Qmaster: /common/sge/6.2.u5/Rahway_SciComp/spool/qmaster/messages
Exec daemon: //messages

Grid Engine startup scripts
---------------------------

Grid Engine startup scripts can be found at:

/common/sge/6.2.u5/Rahway_SciComp/common/sgemaster (qmaster)
/common/sge/6.2.u5/Rahway_SciComp/common/sgeexecd (execd)

Do you want to see previous screen about using Grid Engine again (y/n) [n] >>

Your execution daemon installation is now completed.

How to create Raw Device in RHEL4 and RHEL5

2009-01-02T14:29:00.000-08:00

On RHEL4, raw devices were setup easily using the simple and coherent file /etc/sysconfig/rawdevices, which included an internal example.

On RHEL5 this is not the case, and customizing in a rather less documented method the udev subsystem is required.

1. Add to /etc/udev/rules.d/60-raw.rules:

ACTION==”add”, KERNEL==”sdb1″, RUN+=”/bin/raw /dev/raw/raw1 %N”

2. To set permission (optional, but required for Oracle RAC!), create a new /etc/udev/rules.d/99-raw-perms.rules containing lines such as:

KERNEL==”raw[1-2]“, MODE=”0640″, GROUP=”oinstall”, OWNER=”oracle”

Reverse migration in BlueArc NAS storage

2008-10-21T10:49:00.000-07:00

In BlueArc storage migration does not have parameters to limit migration based on size, which some times causes secondary storage to be so full that even a file residing there can not be opened. In that case a reverse migration is required.

reverse-migrate --file-system --pattern *.* /dir1/dir2/dir3/rspa.*

How to Manually Fail a Drive in BlueArc NAS storage

2008-10-02T14:43:00.000-07:00

Manually failing a drive is done through the use of SANtricity. SANtricity is started in Storage Management > Storage Utilities > Launch SANtricity. You will be prompted
twice for login/password combos. To enter the second login/password combo you need to
enter the login and then hit the “Enter” key to provide the password. When typing the
password, nothing will display on the screen. Hit the “Enter” key again to have the
login/password combo accepted. This will provide access to the storage enclosure using
SANtricity.
From alert message identify which enclosure, which tray and which disk in tray has to be failed.
From the menu select Advanced -> Recovery -> Fail Drive. Type “yes” to confirm the
manual fail.

How to do a backup of veritas configuration ?

2008-05-16T23:29:00.000-07:00

# vxconfigbackup -l /var/vxvm/backups

How to disable VxVM on a system?

2008-03-26T11:22:00.000-07:00

1. Remove mirror from each file system
# vxassist -g rootdg remove mirror swapvol
# vxassist -g rootdg remove mirror export
# vxassist -g rootdg remove mirror rootvol

2. Comment out Vxvm load module entries from /etc/system

3. go to /etc/vx/reconfig.d/state.d
# rm root-done
# touch install-db

4. Copy pre VM vfstab file to /etc/vfstab.
5. Reboot the system.

How to mirror root disk?

2008-02-18T09:25:00.000-08:00

1. Find out available disk name using vxdisk list

2. Do below
# ./vxmirror -g rootdg bootdisk bootmirror

Replace failed drives in SVM

2008-02-17T09:19:00.000-08:00

Suppose we have two internal disks in a system mirrored with SVM and which are being monitored by Nagios/BB/sitescope or a script. When a drive fails we get a message meta devices (drives) “Needs Maintenance”.

We can try reparing the drive if its not Production system.

# metareplace -e d10 c1t2d0s0

where d10 is mirror having d11 and d12 metadevices, d12 (c1t2d0s0) is on the disk which went bad.

If Resynching starts and completes without error disk is fine otherwise we have to replace disk. Below is the process to do that.

1. Detach all metadevices existing on failed disk.
# metadetach -f d10 d12
# metadetach -f d20 d22
# metadetach -f d30 d32

2. Remove the metadevices.

# metaclear d12 d22 d32

3. use metadb -d command to delete metadb existing on failed drive

4. Replace the disk

5. Copy the partition table form disk under SVM to new disk

6. use metadb to create meta database on new disk.

7. Create metadevices again on new disk.

8. Attach the metadevices to create mirror which will also sync them.

Important files in SVM

# more /etc/lvm/mddb.cf -> (DO NO EDIT) records the locations of state database replicas

# more /etc/lvm/md.cf -> (DO NO EDIT) contains auto generated config info for the default (unspecified or local) disk set

# /kernel/drv/md.conf -> (DO NO EDIT) contains the state database replica config info and is read by SVM at startup

# /etc/lvm/md.tab -> contains SVM config info that can be used to reconstruct your SVM config (Manually)