Gowri's Network & Sysblog

A Small Tec DIGG on IBM XIV

2009-02-26T12:58:00.014+05:30

Will IBM XIV be able to make massive change on the storage system wolrd? A small digg on it..

Step One: As usual type “ IBM XIV ” on Google or any other search engine you like.. (but remember it should be able to give you the answers.. oops.. no same side goals pls..)

you will find a list of sites which are discussing about IBM’s acquisition of XIV. It’s non of our business anyhow.. but if you look carefully within this mess, you can find the whole story about XIV.

Step Two: I just want to see some tec stuff before going through the whole damp story about XIV. Wah.. here you go..

There is only two things were eye catchy..

Revolutionary self-healing process allows re-building a 1 TB disk drive in 30 minutes or less

Mm.. that’s really fantastic.. even I haven’t seen a 146GB SAS (RAID5) doing that much faster..

Up to 24 Fiber Channel ports offering 4 Gbps, 2Gbps or 1 Gbps multi-mode and single-mode support

Stop.. stop.. how you gonna squeeze existing bunch of FC cables in 24 ports?

3D-tour they provide was not that fascinating.. but gave some rough idea on how is gonna look like..

I’m not gonna explain how XIV works with 180 SATA disks(each 1TB) with only 80TB usable space.. and how it can provide high-performance(with 7.2Krpm compared to SAS 10K &15K rpm) in low-cost (180 disks? Twice as normal RAID5 SAS disks).

A Quick matrix on what IBM XIV storage offers..

Total Modules	6	9	10	11	12	13	14	15
Useable Capacity (TB)	27	43	50	54	61	66	73	79
Interface Modules	3	6	6	6	6	6	6	6
Data Modules	3	3	4	5	6	7	8	9
Disk Drives	72	108	120	132	144	156	168	180
Fibre Channel Ports	8	16	16	20	20	24	24	24
iSCSI Ports	0	4	4	6	6	6	6	6
Cache Memory (GB)	48	72	80	88	96	104	112	120

Step Three: Landed on XIV storage native site which introduces Moshe Yanai to us as a “leading architect of modern data storage who played a pioneering role in the creation of EMC’s groundbreaking Symmetrix/DMX product line” (DMX..? here the cold war starts between EMC and IBM.. I’ll talk about this bit later..)

Demo they have given for XIV GUI was really impressive, but can’t stop feeling something missing out there in the black box.. will it blast or give us bunch of roses..?

Step Four: cold war begins with an EMC blogger’s screaming towards Moshe “NO! I don't know you! You are not my Dad!”

After finish many ice balls throwing towards IBM, he finish the blog post with celebrating the independence. “We are no longer that which Moshe made. We are today what we have made of ourselves.”

There are few other posts by his colleagues as well.. Reply [The Two Faces of EMC] to them by Tony Pearson , Senior Storage Consultant for the IBM System Storage product line. .

Step Five: Just be in focus on what we need to find out technically...(I’m telling this because after read Tony’s blogs your mind may be converted to something else.. sorry Tony) Here is something you should know (about xiv) by barry a. burke

Q BB (barry burke not Black Berry) : "What happens if a second drive fails before my XIV array has completed rebuilding the first failed drive?"

A BB: You lose ALL of the LUNs on that XIV array!

The reason he gives..

All LUNs are spread across all of the drives in an XIV array, so some part of every LUN is (probably) on every single drive in the array.

When a drive is lost, the remaining drives have to regenerate mirror copies of any data that was on the failed drive

The probability of a second drive failure during the rebuild of the first is non-zero

Which is a valid point in a way.. but hold on, let’s see what Tony says..

Step Six: once again dig into Tony’s blog for any answers…? Wah found The Truth about XIV ..

"For each disk in the XIV, the 1MB chucks are only copied to 168 of the 180 drives. In other words, there are 11 other drives that do not contain mirrored data with any drive you pick. If your two drive failure occurred within 5 minutes on those two drives within this set, the system can handle this just fine, no LUNS are impacted, and no need to recover any data."

It seems to be reasonable.. but who is gonna to test it in a real environment to find out the probability..? are you..?

There are Few questions which BB asked but not been answered by Tony yet…

PAYGO also requires the customer to purchase the remaining capacity within 12 months of installation. So it is More of a 12-month installment plan than pay-as-you-grow.

you are powering (8.4KW) and cooling all 180 drives for the whole duration, whether you're using the capacity or not. is it what you called Greener power usage..?

And finally do not forget my question on 24-FC Ports…

Nagios Plugins Installation Problems

2008-05-05T08:36:00.003+05:30

Sometimes we prefer immediate solutions for our problems, specially when we already know the root cause but forgot the solution or in a hurry to resolve the problem than going through the detailed explanation.

Here I listed some of the problems briefly which I faced when I tried to install some Nagios plugins ...

Nagios-plugins
Problem: redhat spopen problem
Solution : check nameserver added in /etc/resolve.conf

NRPE
Problem: CHECK_NRPE: Error - Could not complete SSL handshake
Solution : check nagios admin IP added in /etc/xinetd.d/nrpe (only_from)

Problem: Return code of 127 is out of bounds - plugin may be missing
Solution : check your command definitions and make sure the path to all executables is correct

Problem: DISK CRITICAL - /dev/hda1 does not exist
Solution : check hda1 is really exist

Problem: The requested URL /nagios/cgi-bin/statusmap.cgi was not found on this server.
Solution : install gd,jpeg,png and their delvels also..

Problem: Can't locate Net/SNMP.pm in @INC
You don't have Net::SNMP installed, see below
Solution : http://nagios.manubulon.com/faq.html

rrdtool
Problem: I could not find a working copy of freetype2
./configure --prefix=/usr/local/rrd/ CPPFLAGS="-I/usr/local/include/freetype2 -I/usr/local/include/libart-2.0" tclrrd.c:20:17: tcl.h: No such file or directory
Solution : up2date tcl-devel

NAGMIN
Problem: Login failed : Incorrect administration username or password
Solution : http://readlist.com/lists/lists.sourceforge.net/webadmin-list/0/124.html

ndomod
Problem: ndomod: Could not open data sink! I'll keep trying, but some output may get lost...
Still unable to connect to data sink. 0 items lost, 492 queued items to flush
Solution : Check path spelled correctly in configuration file
vi /usr/local/nagios/etc/ndomod.cfg
output=/usr/local/nagios/var/ndo.sock
buffer_file=/usr/local/nagios/var/ndomod.tmp

nagiosgrapher
Problem: Can't locate RRDs.pm in @INC
Solution :
cp /usr/local/nagios/rrdtool/lib/perl/5.8.5/i386-linux-thread-multi/RRDs.pm /usr/lib/perl5/5.8.5/i386-linux-thread-multi/
cp /usr/local/nagios/rrdtool/lib/perl/5.8.5/i386-linux-thread-multi/auto/RRDs/RRDs.so /usr/lib/perl5/5.8.5/i386-linux-thread-multi/

Problem: host map: lookup (a.com): deferred
This Sendmail error--also seen as "Transient parse error -- message queued for future delivery" or "Name server: domain: host not found"--indicates that Sendmail encountered a problem with one of its DNS resource record lookups of the destination domain.
check sendmail -v -d8.32 -qImessage_ID

Watch out Google, here comes SPH Search

2007-08-21T09:40:00.000+05:30

A new online search engine being developed by Singapore Press Holdings (SPH) aims to be the first port of call for people anywhere in the world who want content about Singapore.

AsiaMedia :: SINGAPORE: Watch out Google, here comes SPH Search

Snort, BASE and Squid (Step 2) - my scripts

2007-04-09T14:33:00.000+05:30

Now I setup everything to monitor our LAN. I got all virus-affected machine's IP. But how I can inform them?.

It is not easy task where our departments are not much close each other, and we have only two network assistants who have to look on this matter.

One way is to send the list of virus-affected machine’s IP’s to relevant departments. It is possible, but to whom have to send? And how long they will take to response that?
It is unpredictable.

These days people don’t care about viruses even it slows down their computer or it sends anonymous emails to others. They do care if it shuts down suddenly or network not accessible or any files had been vanished.

So we got an idea to block those IP’s on the proxy and give an alert message says, “You machine is possibly affected by virus. Until your machine is fully cleaned the Internet connection will be temporally un available”.

So if they want Internet connection, they have to cleanup their computer.

Here I explain how I did this.

First I have to get the list of IPs from the database.

Snort.php

'
" .mysql_error());
mysql_select_db($dbase) or die ("Database $dbase not exist.
" .mysql_error());

$query="select DISTINCT ip_src from acid_event where sig_name like '%Virus%'";
$result=mysql_query($query);
while ($row=mysql_fetch_array($result)){
echo $row[ip_src];
echo " \n";
#echo $ip;
}
mysql_free_result($result);
?>

Here I want to tell you that I had made a new rule to find virus affected machines and gave the alert as “Virus”. So it was easy for me to query by that alert name.

There is another problem, those IP stored in the database are not in usual dot format. It was an integer. I found a package call num2dot, which converts integer to normal dot format.

This is the script to get the list of IP and send them to proxy every 30 minutes.

ipcron

#!/bin/bash
cd ~/scripts/
/usr/bin/php snort.php>ip_int
#/usr/local/bin/num2dot ip_list
/usr/local/bin/num2dot ip_list
if [ -s "ip_list" ]; then
echo "255.255.255.254" >>ip_list
scp ip_list cache:/etc/squid/
else echo "ERROR"
fi

Add cron
crontab –e
*/30 * * * * ~/scripts/ipcron

In the squid.conf, I create an ACL call virus and tell it to deny machine, which has the source IP from my list.

### Virus affected Machines #########
acl virus src "/etc/squid/virus"
deny_info ERR_VIRUS virus
http_access deny virus

Create Error message

ERR_VIRUS

We found that your machine is flooding packets to the campus network.
The machine can be possibly infected by a virus or having adware/malware.

Please check with a virus guard or contact lab administrator to scan your machine.

Please unplug network cable from your machine till you clean your machine.

Until such time your Internet connection will be Temporarily disabled.

Please contact NOC(%w) if you feel this is incorrect.

That was cool.

Snort, BASE and Squid (Step 1)

2007-04-09T14:32:00.000+05:30

Can you believe that we didn’t have any virus detection system in our LAN, which has nearly 1500 client computers. Yes, that was unbearable truth until I started working with snort.

I start installing snort on our monitoring machine. That is the only machine we are running on Debian. I searched for a nice front-end for snort. Finally I came across BASE (really cool). I would like to share my adventurous experience with you.

Snort

First I install snort with mysql compartible
# apt-get install snort-mysql

when you are installing it will ask your HOME_NETwork ip range and aall other parameters required to configure snort.

But later you can modify them by editing /etc/snort/snort.conf file.

Start snort service
# /etc/init.d/snort start

It was started successfully, but I did not get any alerts. What went wrong?

Snort need two network cards. One card(say eth0) with an IP for administration purpose, and another(say eth1) to collect packets from the LAN. This should not have an IP address.

I connect eth1 to an Ethernet port on L3 switch where all department uplinks (fiber links) are connected.

Now I want to copy all VLAN packets to my monitoring port. Fortunately 3com core builder had a facility call Roving analysis.

What we have to do is just configure an empty port as analyzer port, and add all other ports, which wants to be monitored as monitor port. Done.

BASE

Next I install BASE. They have nice documentation for how to install snort on centos.
But here I mention some problem I faced and how I solve them.

1. Download BASE and adodb from
http://sourceforge.net/project/showfiles.php?group_id=103348
http://sourceforge.net/project/showfiles.php?group_id=42718

2. Un tar both in to your web server DOCUMENTROOT normaly /var/www/html
# tar –xzvf base-version.tar.gz /var/www/html/
# tar –xzvf adbodb.tar.gz /var/www/html/

3. Rename base-version
# mv base-version base

4. Start web service
# /etc/init.d/httpd start

5.Open a web browser and type
http://localhost/base

You will get base installation page. Follow the steps.

Adodb path : /var/www/html/adodb

6. Next you have to create database for snort to log alerts and give user privileges for user snort. (by default it logs in /var/log/snort/alert.log)

# mysqladmin createdatabase snort_db
# grant privillages for snort

On the installation page.
Database: mysql
Host : localhost
Database name : snort_db
User : root (if you have user snort you can use it. It is more secure)
Password : (if you don’t have password for this user just leave it blank)

7. Click on createDB

8. Add this line in to snort.conf
output database: alert, mysql, user=root dbname=snort_db

9. restart snort

Done.

GADisk

2007-01-23T16:33:00.000+05:30

It is Globally Accessible Disk. That means you can download using our bandwidth and store them for future reference and can access it from anywhere in the world ....
more on this visit http://gadisk.sourceforge.net/

MySQL Error

2006-09-28T13:34:00.001+05:30

MySQL

If you get this error after done everything

ERROR 1045 (28000): Access denied for user 'user'@'localhost' (using password: NO)

try
mysql> FLUSH PRIVILEGES;

or run this script in command line
mysql_fix_privilege_tables

phpWebSite

2006-02-27T10:41:00.000+06:00

phpWebSite provides a complete web site content management system.
Web-based administration allows for easy maintenance of interactive, community-driven web sites.

It is really cool, if you know php you can modify it according to your requirements.
Here are some usefull links for you,

phpWebsite download

phpWebSite Manual

phpwebsitedemo

php configurations

sample site

Devi(e)ls on Asterisk

2005-09-29T14:02:00.000+06:00

We got new Grandstream BT-100 phones. I wants to setup an asterisk server to make voip calles throughout our LAN.

So today I started intalling Asterisk on my own mechine (running on Ubuntu). if it works fine I have to implement it on a server with a final touch.

fine. first I collect the packeges needed to install Asterisk. Those are

termcap
Bison
CVS
Zaptel, libpri

first I download Zaptel , Libpri and asterisk through CVS

# cd /usr/src # export CVSROOT=:pserver:anoncvs@cvs.digium.com:/usr/cvsroot # cvs login - the password is anoncvs. # cvs checkout zaptel libpri asterisk

To get the current snapshot from the release branch of CVS, issue the following command:
# cvs checkout -r v1-0 zaptel libpri asterisk asterisk-addons asterisk-sounds

After you receive the latest code from CVS, issue the following commands as root to install Asterisk on your system:
# cd zaptel # make clean; make install # cd ../libpri # make clean; make install # cd ../asterisk # make clean; make install

while tries to install I got this error message

/usr/bin/ld: cannot find -lncurses

collect2: ld returned 1 exit status

make: *** [asterisk] Error 1

ld combines a number of object and archive files, relocates their data and ties up symbol references. Usually the last step in compiling a program is to run ld.

ld accepts Linker Command Language files written in a superset of AT&T's Link Editor Command Language syntax, to

provide explicit and total control over the linking process.

but my system had been installed ncurses, so i was confusing why i'm getting this error. after gothrough severel malining lists i found that i have to install ncurses developer version also. likewise I need zlib-devel and libssl-devel from Ubuntu site .

after installed all devils sorry devels I manage to install asterisk successfully.

Freshclam

2005-09-22T10:08:00.000+06:00

There is a daemon called freshclam comes with clamav for update the virus database automatically. We can schedule with cron to check for updates.

I configured all what we need and wrote a cron to run freshclam in early morning.

when I came next day morning, I got this error message in the mail box.

ERROR: Please edit the example config file freshclam.conf

I went through each and every line but couldn't figure out the error. So I put it on hold.

Today(one week later) I went through that config file again to locate the virus database. By accidentally I noticed that I didn't comment out the first line which has the word EXAMPLE.

shhhhh........ What a silly mistake I did.

AV(or squirm?) killed me

2005-09-14T14:05:00.000+06:00

oh no i'm getting this dam error message on cache.log

WARNING! Your cache is running out of filedescriptors

after spent half an hour in googling i got the result. that's because the kernel defaults only allow processes to use up to 1024 filedescriptors, and Squid picks up the limit at build time.

So I did these changes
* Edit /usr/include/bits/types.h to define __FD_SETSIZE to at least the amount of filedescriptors you'd like to support (Not required for Squid-2.5 and later).
* Before configuring Squid run "ulimit -HSn ####" (where #### is the number of filedescriptors you need to support). Be sure to run "make clean" before configure if you have already run configure as the script might otherwise have cached the prior result.
* Configure, build and install Squid as usual
* Make sure your script for starting Squid contains the above ulimit command to raise the filedescriptor limit. You may also need to allow a larger port span for outgoing connections (set in /proc/sys/net/ipv4/, like in "echo 1024 32768 > /proc/sys/net/ipv4/ip_local_port_range")
we have a word in tamil "when we tried to make a well there is a devil came out". it's really matches my sittuation now.

I restart squid, its working fine now without any error messages.

AV & myself

2005-09-13T11:06:00.000+06:00

I need a quality but freeware anti virus to install our proxy . I got a suggestion to use sophos first. I had tested it with my backup proxy; It doesn’t seems to be working with redirectors or http download requests on proxy (running on squid 2.4).

Then I tried clamav, but itself only not enough. So I installed squirm for redirect particular subnet traffic and check for some specific patterns contains .exe, .zip, .rar etc and for some banded sites. If the match found the request send through viralator.cgi. After viralator downloads that file in to the system and scan with clamav. If any virus found it returns a page virus found and discard the file immediately or else it will complete the request. This is exactly what i want.

After spend one day in a black hole I managed to configure successfully. I informed to my head he asked me to give its feedback after watching about 30min. so I got break for tea. When I backed to my seat I found somebody tries continuously to access some unknown sites that my system is not allowing them to go in but some download requests denied but more allowed. So it means my system works well!!??.

I assume that you have already got Apache, Squid installed and running, please refer to the install guide that comes with each of those programs.

Clamav

1) Get Clamav from http://www.clamav.net/stable.php
2) tar -zxvf clamav-x.tar.gz
3) cd clamav-x/
4) ./configure
5) groupadd clamav
6)useradd -g clamav -s /bin/false -c "Clam AntiVirus" clamav
7) make
8) make install

Try to scan recursively the source directory:
9) clamscan -r -l scan.txt clamav-x.yz

It should find some test files in the clamav-x.yz/test directory. The scan result will be saved in the scan.txt log file . To test clamd, start it and use clamdscan (or connect directly to its socket and run the SCAN command instead):

Squirm

1) Get Squirm from http://squirm.foote.com.au/
2) tar -zxvf squirm-x.tar.gz
3) cd squirm-x/regex
4) ./configure
5) make clean
6) make
7) cp -p regex.o regex.h .. (Note the ".." !)
8) cd ..
9) vi Makefile and make sure that the specific files will be installed with user=squid and group=squid (Default root).
10) make
11) make install
12) cd /usr/local/squirm/etc
13) cp squirm.local.dist squirm.local
14) cp squirm.patterns.dist squirm.patterns
15) chgrp squid squirm.local
16) chgrp squid squirm.patterns
17) vi squirm.local and set local IP addresses that squirm will apply to e.g:
127.0.0
10.0.0
192.168.0
18) vi /usr/local/squid/etc/squid.conf and add the following values:
redirect_program /usr/local/squirm/bin/squirm
redirect_children 10
19) service squid restart

Viralator

1) Get Viralator from http://viralator.loddington.com
2) cd /usr/src
3) mkdir viralator
4) mv viralator-x.zip /usr/src/viralator
5) cd /usr/src/viralator
6) unzip viralator-x.zip
7) cp viralator.cgi /usr/local/apache/cgi-bin
8) cd /usr/local/apache/cgi-bin/
9) chown viralator:viralator viralator.cgi
10) chmod 755 viralator.cgi
11) vi viralator.cgi and set the following values:
$downloads = "/usr/local/apache/htdocs/downloads"; # The directory
$downloadsdir = "/downloads/";
12) Also set your options for your virus scanner in viralator.cgi.
13) mkdir /usr/local/apache/htdocs/downloads
14) chown viralator:viralator /usr/local/apache/htdocs/downloads
15) chmod 755 /usr/local/apache/htdocs/downloads
16) mkdir /downloads
17) chown viralator:viralator /downloads
18) chmod 755 /downloads
19) vi /usr/local/squirm/etc/squirm.patterns and add the following (192.168.0.1 is the internal network interface in this example):
abortregexi (^http://192\.168\.0\.1/.*) #eg (^http://192\.168\.100\.1/.*)
abortregexi (^http://mail.linuxbox.co.za.*)
regexi (^.*\.zip$) http://192.168.0.1/cgi-bin/viralator.cgi?url=\1
regexi (^.*\.doc$) http://192.168.0.1/cgi-bin/viralator.cgi?url=\1
regexi (^.*\.exe$) http://192.168.0.1/cgi-bin/viralator.cgi?url=\1

sometimes viralator.cgi looks for viralator.conf file in /etc/viralator if it happens copy the viralator folder to /etc or just make a link

if you are running apache as suexec add following line on httpd.conf file

Apache

ServerAdmin email@localhost
DocumentRoot /var/www/html
ServerName localhost (your server name)
ErrorLog logs/clamav_error_log
TransferLog logs/clamav_access_log
ScriptAlias /cgi-bin/ /var/www/cgi-bin/
User viralator
Group viralator

20) service squid restart
21) service httpd restart

Try to download a zip file, if it does not work look in the Apache error logs or the squirm.error logs

Browse to http://viralator.loddington.com/ and try to download test virus zip file. It contains the eicar test virus signature. It is not a real virus and is harmless but it will set off most virus scanners.

Getting start (struggle)

2005-08-18T09:41:00.000+06:00

see I'm not fluent in english. i would like to talk in my mother tounge. why don't they think about us. I should ask them to include Tamil font also. but then also it's difficult to type in normal keyboard. so it's better to keep going in english for awhile with the help of a dictionary. The thing is I don't know whether others could be able to cauch me or not. anyway this is my own diary i can blog whatever i think. thanx for blogger dot com. see you with my funny blogs, until bye from Gowri the Great!!!!!!!!!