Oded Maimon - IT Blog

Oracle - The bad boy in town...

noreply@blogger.com (Oded Maimon) — Mon, 16 Aug 2010 11:29:00 +0000

Oracle has just become the new bad boy in town.. every day it becomes worse...

- Killing WebStack
- Killing OpenSolaris
- Changing the Solaris License to be 90 days trail instead of free use
- This new Oracle VS Google lawsuit and trying to get revenue from something that you have nothing to do with

What will happen to other project, mainly MySQL? OpenOffice? NetBeans? VirtualBox?

Apache and HTML5 WebSockets

noreply@blogger.com (Oded Maimon) — Sun, 15 Aug 2010 14:07:00 +0000

I was surprised to find out that there is no implementation for WebSockets support in Apache web server.
But there is a way to make it work.

pywebsocket project is an Apache "extension" written in python, so mod_python is required also, and with it Apache can support HTML5 WebSockets. it is very easy to install and configure.

I flowed the instructions in this URL: http://www.travisglines.com/web-coding/how-to-set-up-apache-to-serve-html5-websocket-applications-with-pywebsocket

Enjoy.

Membase

noreply@blogger.com (Oded Maimon) — Sun, 08 Aug 2010 19:00:00 +0000

Just wanted to mention a project that I hope will continue to progress and release new features, Membase is almost all the things that are missing in Memcache for a long time.

To check it go to membase.org to read more and download it.

Google App Engine and Geo Targeting by IP using MaxMind DB

noreply@blogger.com (Oded Maimon) — Mon, 02 Aug 2010 16:02:00 +0000

Hi,
I tried using my Java based GAE project to geo target my user using MaxMind but I found out that it is not possible because the current implementation of MaxMind Java API uses restricted classes by Google.

The only way I was able to do it is by using Python based GAE project and using this great piece of code for doing it: http://code.google.com/p/python-geoip/

Hope it helps anyone.

Add disk to vmware linux guest without rebooting

noreply@blogger.com (Oded Maimon) — Thu, 22 Jul 2010 05:38:00 +0000

Adding a new disk to a vmware linux guest OS is very easy, the next step is to make the guest OS recognize the disk, this is also very easy, just run:
echo "- - -" > /sys/class/scsi_host/host#/scan

where host# should change to the correct host, probably host0.
after running the command you should be able to see the new disk by running: fdisk -l

SQL Cursor and Google App Engine

noreply@blogger.com (Oded Maimon) — Sun, 11 Apr 2010 21:23:00 +0000

Google App Engine SDK 1.3.1 released a new important feature Datastore Query Cursors.
With this new feature we can bypass requests hard limits (1MB response size and 30 sec timeout), we can run a large query and split it into multiple request. This is how it can be done with JDO (it is also possible with low level API)

Create your query
Limit execution for a specific amount of rows
Execute the query
Process the results set
Keep the cursor in memcache

Here is a simple code that implements all steps from my java Servlet:

        final int MAX_REC = 2000;
        final String cacheKeyName = "ExportMyTable" + req.getRemoteAddr();

        PrintWriter prt = resp.getWriter();

        PersistenceManager pm = PMF.get().getPersistenceManager(); // PMF is my representation to get the PersistenceManager instance
        Query query = pm.newQuery(myTable.class);
        List results = null;
        String cursorString = null;
        Cursor cursor = null;
        Map extensionMap = new HashMap();

        Cache cache = null;

        try { // check if cursor already saved in the cache, if so, use it
            cache = CacheManager.getInstance().getCacheFactory().createCache(Collections.emptyMap());
            cursorString = (String) cache.get(cacheKeyName);
            if (cursorString != null && !cursorString.isEmpty()){
                // Cursor were found in the cache, set the query to start from that cursor
                cursor = Cursor.fromWebSafeString(cursorString);
                extensionMap.put(JDOCursorHelper.CURSOR_EXTENSION, cursor);
                query.setExtensions(extensionMap);
            }
        } catch (CacheException e) {
            prt.println(e.getStackTrace());
        }
        query.setRange(0, MAX_REC);

        //execute the query
        results = (List) query.execute();

        // Process the results
        for (myTable t : results){
            prt.println(t.toString());
        }

        // Get the cursor
        cursor = JDOCursorHelper.getCursor(results);

        // get the cursor as string
        cursorString = cursor.toWebSafeString();

        // remove the cursor from the cache if got less records then requested (the query ended)
        if (results.size()
            cache.remove(cacheKeyName);
            prt.println("-END-");
        } else {
            // keep current cursor location
            cache.put(cacheKeyName, cursorString);
        }

        query.closeAll();
        pm.close();

Enjoy...

Authenticating Google App Engine apps using curl

noreply@blogger.com (Oded Maimon) — Fri, 09 Apr 2010 16:16:00 +0000

I'm going to show here the exact way that i use to authenticate to my google app engine application using unix script and curl.

This is a 3 steps process:

Get the auth code from google (using ClientLogin)
Get the cookie from google using the auth code
Access the service that we want in google app

In the example we will use:

Application name is: MyFirstApp
Application url is: http://MyFirstApp.com
Our application service name is: getAllData
Application admin is: admin@gmail.com
Application admin password is: adminadmin

Step 1:
Getting auth key:

curl -f -s --output myAuthFile.txt -d Email=admin@gmail.com -d Passwd=adminadmin -d accountType=GOOGLE -d service=ah -d source=MyFirstApp https://www.google.com/accounts/ClientLogin

after the command completes we will have a file named: myAuthFile.txt with 3 lines, the line that starts with "Auth=" is the auth code line, we will use this code in the next step.

Step 2:
Getting application cookie:

curl -c cookiefile "http://MyFirstApp.com/_ah/login?auth=`cat myAuthFile.txt | grep ^Auth= | cut -d= -f2`" > /dev/null

after the command completes we will have a file named: cookiefile that will include our cookie, we will use the cookie in the next step.

Step 3:
Calling our application service, getAllData:

curl -f -s -H "Cookie: ACSID=`cat cookiefile | grep -v ^# | grep -v ^$|cut -f7`" http://MyFirstApp.com/getAllData

That's all!
Hope it will help anyone.

Windows 7 MBR got destoried by a trojan!

noreply@blogger.com (Oded Maimon) — Wed, 27 Jan 2010 23:49:00 +0000

This Link is a bless for anything related to fixing window 7 MBR!!!

http://neosmart.net/wiki/display/EBCD/Recovering+the+Vista+Bootloader+from+the+DVD

Strange SMF "Bug?"

noreply@blogger.com (Oded Maimon) — Tue, 20 Jan 2009 06:33:00 +0000

Yesterday I've installed apache 2 on Solaris 10 x86 (on two machines) from the Solaris CD:

pkgadd -d . SUNWapch2r
pkgadd -d . SUNWapch2u
pkgadd -d . SUNWapch2d

After the successful install I've tried to enable the service, but the service was not there :), The next step was to reimport the service to the SMF repository:

bash-3.00# svccfg -v import /var/svc/manifest/network/http-apache2.xml
svccfg: Scope "localhost" changed unexpectedly (service "network/http" added).
svccfg: Could not refresh svc:/network/http:apache2 (deleted).
svccfg: Successful import.

and again, the service was not there, searching the net for it, and I've found this: http://mail.opensolaris.org/pipermail/smf-discuss/2006-October/005565.html

"This happens when a buggy i.manifest is used. If your package has an
i.manifest file which uses SVCCFG_REPOSITORY, then that is your problem.
You can fix it by restarting svc.configd ("pkill configd" as root).
Then you should fix your package, or file a bug."

By reading it I've thought that it is a long shot, but it worked! "killing" configd and reimporting the service worked:

bash-3.00# svccfg -v import /var/svc/manifest/network/http-apache2.xml
svccfg: Taking "previous" snapshot for svc:/network/http:apache2.
svccfg: Upgrading properties of svc:/network/http according to instance "apache2".
svccfg: Taking "last-import" snapshot for svc:/network/http:apache2.
svccfg: Refreshed svc:/network/http:apache2.
svccfg: Successful import.

Regards

Fast and simple Solaris Zone creation

noreply@blogger.com (Oded Maimon) — Sun, 11 Jan 2009 10:25:00 +0000

- Run zonecfg, for the first zone configuration:
bash-3.00# zonecfg -z mynewzone
create -b
set zonepath=/zones/mynewzone
set autoboot=true
set ip-type=shared
add net
set address=192.168.0.122
set physical=nxge1
set defrouter=192.168.0.254
end

commit
exit

NOTE: we can also save the commands in a file and run zonecfg like this:
zonecfg -z mynewzone -f myzone.conf

- Create the zone directory and change the permissions:
mkdir /zones/mynewzone
chmod 700 /zones/mynewzone

The zone installation will fail if the zone directory will have the wrong permissions:

/zones/mynewzone must not be group readable.
/zones/mynewzone must not be group executable.
/zones/mynewzone must not be world readable.
/zones/mynewzone must not be world executable.
could not verify zonepath /zones/mynewzone because of the above errors.
zoneadm: zone mynewzone failed to verify

- Install the zone (takes time):
zoneadm -z mynewzone install

- Check the log:
grep -v "successfully installed" /zones/mynewzone/root/var/sadm/system/logs/install_log | grep -v ^$

- list the zones:
zoneadm list -cv

ID NAME STATUS PATH BRAND IP
0 global running / native shared
1 mynewzone running /zones/mynewzone native shared

- Boot the zone:
zoneadm -z mynewzone boot

- Now we need do some last standard Solaris configurations, we will login to the zone console and follow the questions for basic configuratio (language, terminal, network...):
zlogin -C mynewzone

If this step will be skipped many of the services won't start.

Done.

Sun Cluster 3.2 Globaldevices issue...

noreply@blogger.com (Oded Maimon) — Sun, 11 Jan 2009 09:16:00 +0000

Hi,
I had an issue with my Sun Cluster 3.2, for some reason only one node /global/.devices was mounted at the same time, because of that I couldn't switch the resources between nodes (the switch/remaster command hang) and in one node the cluster globaldevices service fail to start.

When the resources switch hang, there was no message in syslog, it just waited util it timedout and failed back the resource.

Also, when the server booted I could see this message:

mount: /dev/md/dsk/d6 is already mounted or /global/.devices/node@1 is busy
Trying to remount /global/.devices/node@1
mount: /dev/md/dsk/d6 is already mounted or /global/.devices/node@1 is busy

WARNING - Unable to mount one or more of the following filesystem(s):
/global/.devices/node@1
If this is not repaired, global devices will be unavailable.
Run mount manually (mount filesystem...).
After the problems are corrected, please clear the
maintenance flag on globaldevices by running the
following command:
/usr/sbin/svcadm clear svc:/system/cluster/globaldevices:default

The problem was that both nodes had the same physical device name /dev/md/dsk/d6 for /global/.devices , here is how my vfstab on each node before the fix:
Node1:
/dev/md/dsk/d6 /dev/md/rdsk/d6 /global/.devices/node@1 ufs 2 no global

Node2:
/dev/md/dsk/d6 /dev/md/rdsk/d6 /global/.devices/node@2 ufs 2 no global

To solve it all I had to do is rename the metadevice on both nodes using metarename and modify /etc/vfstab to include the new change:
Node1:
metarename d6 d601
Node2:
metarename d6 d602

after the change you can restart svc:/system/cluster/globaldevices:default on both nodes and it all works.

Add zfs dataset to a zone

noreply@blogger.com (Oded Maimon) — Wed, 07 Jan 2009 12:34:00 +0000

I think that the easiest method for adding (and after that, managing) a zfs pool/file system to a zone is by adding it as a dataset.

Adding the zfs as a dataset will allow the zone administrator to manage this pool/filesystem and use it as a normal zfs filesystem.

zonecfg -z myzone
zonecfg:zion> add dataset
zonecfg:zion:dataset> set name=zpool/mydata
zonecfg:zion:dataset> end

There are few properties that the zone admin won't be able to change and it could be that the zone admin will see more than you wants him to see. for more information about the delegation properites: http://docs.sun.com/app/docs/doc/819-5461/gbbsn?a=view

There are times that all you want to do is just share space between the global zone and the non-global zone, in this case, you can add the zfs file system as a generic filesystem:

# zfs set mountpoint=legacy zpool/mydata
# zonecfg -z myzone
zonecfg:zion> add fs
zonecfg:zion:fs> set type=zfs
zonecfg:zion:fs> set special=zpool/mydata
zonecfg:zion:fs> set dir=/data
zonecfg:zion:fs> end

Adding new shared disk to a running sun cluster 3.2

noreply@blogger.com (Oded Maimon) — Tue, 06 Jan 2009 13:34:00 +0000

Adding a new shared disk to a running Solaris system, is easy, just add the device, and wait few secs for the OS to recognize it.
After that step, we need the cluster to be aware of the new added disk, and create a new DID device, just run: "cldevice refresh" and then you will be able to see it in "cldevice list -v"

Annoying syslog.conf syntax issue

noreply@blogger.com (Oded Maimon) — Sun, 28 Dec 2008 11:09:00 +0000

Syslog will not work when using space as separators in syslog.conf file, you must use TAB as a separator.

Sun Cluster 3.2 with Mysql on Failover zones

noreply@blogger.com (Oded Maimon) — Fri, 26 Dec 2008 00:53:00 +0000

Click here to download a pdf format of this post.

I'm going to write it step by step, with almost no explanations, just steps and tips:
First of all what I've tried to create is:

Active/Passive MySQL cluster
Two physical nodes
Non-global zone on each node
MySQL is running in the zone
Private interconnect – Two interfaces on each server connected thru two Ethernet switches
One central storage, with one LUN for MySQL data and one LUN for quorum
Hosts names are: host1 & host2
Zones names are: zone1 & zone2
IPMP is used for public interfaces on physical nodes
MPXIO is used for HBA's multipathing
Using ZFS as MySQL data filesystem

Resource I've used:

So, here we go:

1. Preparations:

a. On the Ethernet switches - Disable spanning tree on the private interconnect ports

b. No need to configure IP's for the private interconnect interfaces

c. It is encouraged to create two separate VLAN's for the private interconnect interfaces

d. Default interconnect network configuration is: 172.16.0.0 with netmask 255.255.248.0, the default can be changed when configuring the cluster

e. If using Jumbo frames for the public network, also configure Jumbo frames for the private interconnect

f. Disable Solaris power management

* Edit /etc/power.conf file and change the line autopm default to autopm disable

* Run: pmconfig

g. /globaldevices file system, is a local filesystem, can be stored on local disks, with the size of 512MB

h. Packages needed for the cluster are: SUNWrsm, SUNWrsmo

2. Make sure that RPC is open for public network:
# svccfg
svc:/network/rpc/bind> select network/rpc/bind
svc:/network/rpc/bind> setprop config/local_only=false
svc:/network/rpc/bind> quit
# svcadm refresh network/rpc/bind:default
If TCP Wrapper is used, also add: rpcbind: ALL too /etc/hosts.allow file on both servers, if you don't wish to allow rpc for everyone, you can restrict it to both servers public and private ip's.

Tip: run tail –f on the messages file for both servers in different sessions, this will help you find the problems in no time

3. Create the zones on both servers

a. Define public IP for both zones

b. Configure /etc/hosts to include the public IP name

4. Install the cluster on host1 and host2 (can be done simultaneously)

a. Mount CD

b. Define DISPLAY environment variable to your X machine

c. Run ./installer

i. Install Cluster Core, Cluster Manager (not required) and MySQL agent

ii. Select to configure cluster after install

5. Change root .profile (or whatever you are using) to include /usr/sbin:/usr/cluster/bin in the PATH environment variable and /usr/cluster/man in the MANPATH environment variable

6. Create SSH key based authentication between the two physical servers for the root account (use RSA keys)

7. After the installation has completed, you need to run scinstall to configure the cluster

a. If TCP wrapper is enabled, add this line to /etc/hosts.allow:
sccheckd: localhost
it is required for sccheck utility to work properly.

b. Don't use Auto Quorum configuration, we will add it later

c. NOTE: both servers will be rebooted at the end of scinstall, without confirmation!!!

d. If you want a different private interconnect IP's/Subnet, you can change it in this step

8. After both servers rebooted, check the cluster status by running:
clnode status

9. Adding a quorum disk device

a. Use format to label the device

b. The cluster is using DID pseudo names for devices, to find a device DID name run:
cldevice list -v

c. Run clsetup and add a disk quorum device

d. If quorum added successfully, press yes for resting installmode

10. Check that the quorum is defined by running:
clquorum list

11. Check that the cluster installmode is disabled:
cluster show -t global | grep installmode

12. Enable automatic node reboot if all monitored disk paths fail:
clnode set -p reboot_on_path_failure=enabled +

a. Check that it has changed:
clnode show

13. Registering the cluster storage and network service
clresourcetype register SUNW.gds SUNW.HAStoragePlus

14. Create a new Resource Group that includes both zones
clresourcegroup create –n host1:zone1,host2:zone2 RG–MYSQL

15. Check that the resource group was added:
clresourcegroup status

16. Create the ZFS pool – zMysql

17. Before we will be able to add zMysql pool as a cluster resource, we need to export it because the cluster will change the pool devices to the DID location:
zpool export zMysql

18. Add the zMysql as a resource in RG-MYSQL:
clresource create -g RG-MYSQL -t SUNW.HAStoragePlus -p AffinityOn=TRUE –p \
Zpools=zMysql -p ZpoolsSearchDir=/dev/did/dsk RS-MYSQL-HAS

19. Check that zMysql was added as a resource to RG-MYSQL:
clresource list

20. Now we can import it back
zpool import zMysql

21. Add a new Virtual IP resource to our RG-MYSQL resource group:

a. Add the VIP to /etc/hosts on both physical servers and zones (we used vip-mysql)

b. Add the VIP to the resource group:
clreslogicalhostname create -g RG-MYSQL -h vip-mysql -N \
private@host1,private@host2 RS-VIP

c. private = our IPMP group name on each server, if not using IPMP, replace private with the NIC name, for example e1000g1

d. Check if VIP added as a resource:
clresource list

22. You can move the resource group to the other servers by running:
clresourcegroup switch -n host1:zone1 RG-MYSQL

23. You can move the resource group back by:
clresourcegroup remaster RG-MYSQL

24. Install MySQL on zone1 and create the databases on zMysql filesystem

25. Install MySQL software on zone2

26. Create mysql account and group on both zones with same uid and gid

27. Start the MySQL instance on zone1

28. Copy my.cnf to the zMySQL filesystem (/data)

29. Edit my.cnf file and add this line to [mysqld] section:
bind-address =

30. Create root@vip-mysql account and grant permissions:
# mysql
mysql> GRANT ALL ON *.* TO 'root'@'vip-mysql' IDENTIFIED BY 'mypassword';
mysql> UPDATE mysql.user SET grant_priv='Y' WHERE user='root' AND host='vip-mysql';

31. Now, the next step is to create the cluster database in our MySQL instance, this database is used for MySQL checks by the cluster

a. cp /opt/SUNWscmys/util/mysql_config /data/

b. chmod 400 /data/mysql_config

c. Edit the file /data/mysql_config to look like this:
MYSQL_BASE=/opt/mysql

MYSQL_USER=root

MYSQL_PASSWD=mypassword

MYSQL_HOST=vip-mysql

FMUSER=fmuser

FMPASS=fmuserNewPassword

MYSQL_SOCK=/tmp/vip-mysql.sock

MYSQL_NIC_HOSTNAME=vip-mysql

MYSQL_DATADIR=/data

d. Note: Don't use $ signs in the password, the cluster scripts will fail...(tried to put \$, but failed for account fmuser)

e. Select an appropriate password for fmuser account

f. Create the database:
ksh /opt/SUNWscmys/util/mysql_register -f /data/mysql_config

g. If the script fails, look at /tmp for more information: cat /tmp/.mysql.error

32. Next step is to register the MySQL as a resource in the cluster, for that we will need to edit one more file:

a. cp /opt/SUNWscmys/util/ha_mysql_config /data/

b. chmod 400 /data/ha_mysql_config

c. Edit the file /data/ha_mysql_config to look like this:
RS=RS-MYSQL-DB

RG=RG-MYSQL

#PORT=3306

LH=RS-VIP

HAS_RS=RS-MYSQL-HAS

# local zone specific options

ZONE=

ZONE_BT=

PROJECT=

# mysql specifications

BASEDIR=/opt/mysql

DATADIR=/data

MYSQLUSER=mysql

MYSQLHOST=vip-mysql

FMUSER=fmuser

FMPASS=fmuserNewPassword

LOGDIR=/data

CHECK=YES

d. Register MySQL as a resource:
ksh /opt/SUNWscmys/util/ha_mysql_register -f /data/ha_mysql_config

e. If MySQL is registered as a Solaris Service, disable it:
svcadm disable mysql

f. Enable RS-MYSQL-DB resource:
clresource enable RS-MYSQL-DB

g. Cluster MySQL resource is saving data to /tmp, check the result/errors in /tmp

Why corporates need to rule everything?

noreply@blogger.com (Oded Maimon) — Wed, 25 Jun 2008 11:54:00 +0000

Why large corporate need to control everything and have everything? Why Oracle can't create the best DB they can and just do that? Why they need to go to places that they don't know how to handle... Did anyone had a first successful install of one of oracle Non-DB products? Actually, did anyone had a smooth experience with the oracle java based DBCA tool?

Well, I think that Oracle should develop Databases only and outsource the tool development to Microsoft, and same thing with Microsoft, they should drop this stupid so called SQL Server “database” :)...

This is a call to all large IT corporate:
Stop buying stuff!!! You don't know how to integrate tools! You just make a lot of Mishmash (Mess!!!)... Please start working together and not buying any more.. We can't handle it, we don't have the time to learn all of it, we don't need the choice of selection between 15 different products that do the same and we don't have time to POC each of them…

Please!!
Oracle (Databases), Microsoft (GUI), CA (Can’t remember what you expertise is anymore), IBM (MF), HP (Servers), Sun (Solaris, Java & Servers), Symantec (Defrag utilities, all your other products suck! Let Veritas products go free to the open source world & release sygate back to the world).
Please!!!

Oracle Appliance

noreply@blogger.com (Oded Maimon) — Wed, 25 Jun 2008 11:47:00 +0000

It was just a matter of time till Oracle will create an appliance!!
Now, HP & Oracle created an appliance for Data warehouse that comes installed with ASM & RAC..
I guess it comes with Oracle Enterprise Linux also...

http://www.crn.com/software/208800507

Oded.

Quest VAS Test (Vintela Authentication Services)

noreply@blogger.com (Oded Maimon) — Sun, 19 Aug 2007 13:18:00 +0000

I've done a quick test Quest VAS (Vintela Authentication Services) as an implementation for Unix/Linux Authentication with Active Directory.

First impression is that this is a very nice tool that will make your life easier if you wish to implement authentication from different *nix like OS's with AD.

The installation is very easy. All we need to do is install the server side (2min) and than the client side (1 RPM for Authentication and 1 for Group Policy (if needed)).

What we have done is:

Install the server side on your domain controller

Create users and groups for use in *nix machines

On the Linux/Solaris machine:

Install the rpm/pkg
Check if the server can find the domain controller LDAP service by running:
dig SRV _ldap._tcp.
For example, if my domain name is uxdc.corp then run:
dig SRV _ldap._tcp.uxdc.corp
Synchronize the server time from the domain controller:
/opt/quest/bin/vastool timesync -d
Restart VAS daemon:
/etc/init.d/vasd restart
Restart Group Policy daemon:
/etc/init.d/vasgpd restart
Join the server to the domain:
/opt/quest/bin/vastool -u administrator join -f

For my opinion the advantages of VAS over the configuration of nss, ldap and pam are:

Easy to install and configure
Encrypted ldap using Kerberos
Same installation and configuration process for all *nix like OS's
Automatically generates the server keytab after joining the server to the domain
Allow users password change from Unix
Timesync solution included without the need to use ntp
Manipulating AD objects from the Unix command line
Server based user access control
Personality Management, the ability of users to impersonate to different personality
Central place for configuring sudo/profile (Management from AD)
Supports better encryption type - arcfour-hmac-md5
File distribution and file permissions capability from AD
The ability to create services keytabs from the Unix CLI

Links
Quest VAS site

Fair Scheduler support for Linux

noreply@blogger.com (Oded Maimon) — Wed, 15 Aug 2007 12:16:00 +0000

Just few days ago I wrote about my experience with Solaris FSS and now I found out that Linux kernel 2.6.24 will support CFS (Completely Fair Scheduler). This should be very interesting to test...

Actually CFS is merged in release 2.6.23 but depends on some unmerged containers, so we will have to wait for release 2.6.24..

Can't wait to test it.

Oded.

See Who's Editing Wikipedia - Diebold, the CIA, a Campaign

noreply@blogger.com (Oded Maimon) — Wed, 15 Aug 2007 09:06:00 +0000

"Wikipedia Scanner -- the brainchild of Cal Tech computation and neural-systems graduate student Virgil Griffith -- offers users a searchable database that ties millions of anonymous Wikipedia edits to organizations where those edits apparently originated, by cross-referencing the edits with data on who owns the associated block of internet IP addresses."

Read more here.

YAST for Redhat

noreply@blogger.com (Oded Maimon) — Tue, 14 Aug 2007 17:31:00 +0000

Oracle migrated YAST to work with RHEL/Centos/Oracle Enterprise Linux (all the same)...
The download is available here.

Oracle new filesystem??

noreply@blogger.com (Oded Maimon) — Mon, 13 Aug 2007 16:45:00 +0000

I've just found out that Oracle is developing a new file system for Linux, I really don't understand what they are doing...

They started with OCFS, than OCFS2 (every thing is OK till now), than ASM and telling every one that this is the best choice for them and not OCFS/OCFS2, and now BTRFS (Btree File System).

What next?

Oded.

Solaris 10 - Resource Management

noreply@blogger.com (Oded Maimon) — Mon, 13 Aug 2007 14:28:00 +0000

Today we've started testing Solaris 10 CPU resource management using FSS (Fair Share Scheduler) and Projects.

The problem we wanted to solve is with Oracle database, we have a DWH database and at night we have few things:

ETL
Database Exports
Users access

What we need is that the database exports won't interrupt the ETL process (The users don't do much at night), So we've decided to try Solaris Resource Management capability.

Before I'll show what we have done you need to know that the Oracle owner is "orauser"
and the Oracle SID is "orcl".

The first step is to create the needed projects, projects are used to distinguish different workloads from one another. We've created two projects:

user.orauser
oracle_low

There are three ways to create a project:

user.USERNAME - a project that is associated with a specific Unix user
group.GROUPNAME - a project that is associated with a specific Unix group
project name - a project that is not associated to any group or user by default and can be associated manually or by SMF (Service Management Facility)

So the first project we've created, user.orauser, will be assigned automatically for the user orauser. The second project, oracle_low, is a non associated project that we will use to lower the CPU workload for the database exports.

The commands to create a the projects are:
# projadd -c "Oracle default project" user.orauser
# projadd -c "Oracle low priority project" oracle_low

All the projects defined in /etc/projects file and can be viewed in this file or by running:
# projects -l

To check orauser default project run:
# id -p orauser
uid=60000(orauser) gid=300(dba) projid=100(user.orauser)

Now that we've created the projects, we need to change the system scheduling to use FSS:
# ps -cafe
# dispadmin -d FSS
# priocntl -s -c FSS -i class TS
# priocntl -s -c FSS -i pid 1
# ps -cafe

"ps -cafe" - shows all processes and there scheduler class, I will not explain what a class is, but you should know that TS class is what Solaris uses for all processes by default. The TS class can be manipulated by the nice and renice commands.
"dispadmin -d FSS" - sets the default scheduling class to be used on reboot to FSS
"priocntl -s -c FSS -i class TS" - changes all current process with TS class to FSS class
"priocntl -s -c FSS -i pid 1" - changes the /sbin/init process to FSS class

The next step is to add the CPU resource (cpu-shares) to each project, one thing we need to know about cpu-shares resource is that the value we set for it is the ratio and not percentages, it means that if project A have cpu-shares=10 and project B have cpu-shares=20 and both process of both projects need 100% CPU load the processes of project B will get total of 66% CPU and the processes of project A will get 33%. So, here is the way to add the CPU resource to our projects:
# prctl -n project.cpu-shares -r -v 30 -i project user.orauser
# prctl -n project.cpu-shares -r -v 10 -i project oracle_low
# projmod -sK “project.cpu-shares=(privileged,10,none)” oracle_low
# projmod -sK “project.cpu-shares=(privileged,30,none)” user.orauser

projmod - changes the project attributes
prctl - changes the resource control for running process

Use the "projects -l" command to view the changes we've just made.
# project -l user.orauser
# project -l oracle_low

Now, we changed the export processes to run like this:
# newtask -p oracle_low /u01/scripts/oracle_export.sh

And every thing is working perfectly!!! Fun Fun Joy Joy.

One more useful command is "prstat -J" that will show resources utilization by project.

Oded.

Linux - Auditing file access

noreply@blogger.com (Oded Maimon) — Sun, 12 Aug 2007 12:39:00 +0000

This link is a very good post on how to audit file changes with Linux auditd.

ksvalidator - validate kickstart file syntax

noreply@blogger.com (Oded Maimon) — Thu, 09 Aug 2007 09:41:00 +0000

Finally, in Redhat 5 there is a tool called ksvalidator and we can use it to validate the syntax of a kickstart file!!

ksvalidator comes with the system-config-kickstart RPM and can be used this way:
system-config-kickstart mykickstartfile.ks

Enjoy.