sum it up...

Setting up ActiveMQ for HA-Load Balance

2015-08-29T13:12:00.000-07:00

In a typical enterprise applications, we often need messaging and asynchronous processing.
To satisfy this need, we need a reliable as well as scalable messaging infrastructure. In currently available messaging infrastructures Apache ActiveMQ stands out in terms of features and simplicity.

Apache ActiveMQ comes with lot of features in built and also provides a way to configure or tweak as per the needs of an application.

In this post , we will explore how to enable network of activeMQ brokers so that we achieve HA(High Availability) as well as load balance between consumers & producers.

I carried out my experiment on local machine with ACtiveMQ 5.8.0, but this can be easily upgraded to latest versions of ActiveMQ viz. 5.10.0

To have network of brokers, we need multiple brokers. So, I changed tcp and admin ports of brokers so that I can run multiple brokers on single machine.

To get brief background on network of broker, please visit this link

In this post we will setup below topology, we will mix failover and NOB to get work done,

1. Producer1 is configured to send messages to broker3 with failover to broker2

2. Producer2 is configured to send messages to broker2 with failover to broker3

3. Broker3, Broker2 are networked with Broker1as below

4. Broker1 is connected with broker4 with NOB.

5. Make sure you enable "advisorySupport" on the broker, which is essential for transparent routing of messages across brokers.

Dry Run:

1. Producer1 sends messages to queue "input.q" on broker3, where there are no active consumers, but it see subscriptions from broker1

2. Broker1 and broker 4 are has consumers which are looking at "input.q".

3. When broker3 receives a message it forwards it to broker1, as its in networked and has active consumers for "input.q"

4. When broker1 receives a messages on "input.q", it gets load balanced between broker1 and broker4 as both has consumers looking for "input.q".

5. Whenever broker3 goes down, producer1 switches transparently to broker2, as its configured with failover.

6. I used prefetch size as 1, so that you can load balancing on consumers

Sample activemq configurations can be downloaded from here.

Java 7 New Features

2011-07-05T23:14:00.000-07:00

Following are few new features of Java 7

1. Introduction of WatchService? (NIO)

A watch service that watches registered objects for changes and events. For example a file manager may use a watch service to monitor a directory for changes so that it can update its display of the list of files when files are created or deleted.

2. Strings in Java Switch construct
Will be able to String case based switch constructt; prior to java 7 switch was only allowed with int & enums

3. Introduction of autoclosable interface and try with resource construct
Developers will get rid of closing resources in finally block. JDBC 4.1 has accomodated these constructs in their classes.
4. Underscores in Numbers
Increasing readability of numbers.
int phoneNumber = 555_555_1212;
long creditCardNumber = 1234_5678_9012_3456L;
long socialSecurityNumbers = 999_99_9999L;
float monetaryAmount = 12_345_132.12;
long hexBytes = 0xFF_EC_DE_5E;

5. Introduction of Closures
Much awaited & requested closures are part of JDK7 on experimental basis ,RI for it will be included in JDK8 TCK as per JSR

6 Addition of forkJoin for parallel processing
Combined JSR 166 into JDK

Spring - MongoDB (Document Oriented Database)

2011-07-04T11:05:00.000-07:00

We often hear following terms viz high scalability, elasticity , high availability..
In addition to this, we do face performance issues with humongous data , which brings our critical applications to deadly halt.

I have experienced such performance problems numerous times in my professional carrier (almost 6 years til now), we usually solved these problems with trade offs like minimizing data fetched / even gone ahead and moved on to more cores and disks etc or tuning and other stuff.

Having such a experience , I was always looking to try out thing called NOSQL databases, recently I got chance & time to try out MongoDB .

My first reaction to Mongo was ..."Its Kool !!".

I tried MongoDB along with spring , (spring-data mongodb), as always spring provides nice abstraction and spring style templates to interact with mongo.

Setting up MongoDB is very simple,
1. Download zip from http://www.mongodb.org/downloads appropriate for your platform.For me its mongodb-win32-i386-1.8.1.zip
2. Unzip the zip the downloaded zip to a directory, it will have a bin directory with mongo executables.
3. To get mongo working create "C:\data\db\" ,as mongo uses this location as default to store file system.
4. You are almost done, now go to bin directory and run mongod.exe , this will start up database server.
5. "help' command will help you after that, to explore more functions like creating database, switching, adding users, adding updating/ removing collections.

Now, will move to spring -mongo part,

Spring comes with MongoTemplate,
sample configurations , that I tried are as follows

The class MongoRepository is a implementation of below interface with mongotemplate instance injected in it

Constructor of MongoRepository initializes a mongoTemplate instance as follows

It works very smoothly..

@Override
   public void saveEntity(String collectionName, Entity object,com.dhananjay.ithinker.persister.Repository.Mode mode) {
       log.info("Saving record for in Collection Name "+collectionName);
       switch(mode){
           case INSERT:
               this.mongoTemplate.insert(collectionName,object);
               break;
           case UPDATE:
           {
               Map objectMap;
               try {
                   objectMap = org.apache.commons.beanutils.BeanUtils.describe(object);
                   Update update = new Update();
                   if(objectMap != null && !objectMap.isEmpty()){

                       for(Map.Entry entry : objectMap.entrySet()){
                           if(entry.getValue() != null)
                               //update.addToSet(entry.getKey(), entry.getValue());
                               update = update.set(entry.getKey(), entry.getValue());
                           this.mongoTemplate.updateFirst(collectionName,new Query(where("_id").is(object.getId())), update);
                       }
                   }

               } catch (IllegalAccessException e) {
                   throw new RuntimeException("Error while update ",e);
               } catch (InvocationTargetException e) {
                   throw new RuntimeException("Error while update ",e);
               } catch (NoSuchMethodException e) {
                   throw new RuntimeException("Error while update ",e);
               }
               break;
           }
           case DELETE:
               this.mongoTemplate.remove(collectionName,new Query(where("_id").is(object.getId())));
               break;
           default :
               throw new UnsupportedOperationException("This mode for DB operation is not yet supported "+mode);

       }

       log.info("Saving record for in Collection Name is completed"+collectionName);
   }

@Override
    public T getEntityByColumn(String column, String value,String collection, Class clazz) {
        return this.mongoTemplate.findOne(collection, new Query(where(column).is(value)),clazz);
    }

    @Override
    public T getEntityById(String collectionName, long id,Class clazz) {
        log.info("Fetching single record for id "+id +" Collection Name "+collectionName);
        return this.mongoTemplate.findOne(collectionName, new Query(where("_id").is(id)),clazz);
    }

MongoRepository

HUMONGOUS :)
Thanks,
Dhananjay

Spring Remoting Over JMS

2010-12-16T11:27:00.000-08:00

JMS is one of the robust and widely used application integration style.
In this era of building loosely coupled service component based enterprise application; we often need to communicate with other application..exchange data...invoke operations.

Such inter JVM communication is made simpler and more robust by Spring's remoting support over JMS.
Its as simple as invoking a local object method.

Spring does it very transparently, with the help of JmsInvokerServiceExporter ,SimpleMessageListenerContainer and JmsInvokerProxyFactoryBean.
You just need to aware of these 3 classes and off course JMS connectionFactory and queue /topic definition.
First will define JMS related beans such as connectionFactory and Queue as follows:

bean>

<constructor-arg value="salaryCalculatorQueue"/>

bean>

Now how to expose a pojo Calculator.java service for remoting over JMS using JMSInvokerServiceExporter

bean>

property>

bean>

Above JMSInvokerServiceExporter exposes Calculator service for remoting over JMS; here you can also specific method which you wish to expose for remoting.

Define a listener for the queue and provide service bean reference for listener as follows

bean>
On the client Side; there will be a JmsInvokerProxyFactoryBean definition which will invoke remote method / basically acts as event sender

<property name="connectionFactory" ref="connectionFactory">property>

<property name="queue" ref="salaryQueue">property>

bean>
The connection Factory and Queue bean definitions are shared by both the client and server configurations

Before running client / server main programs ..start up Active MQ server.

write a simple main program to start up server like below..just load server side bean definitions

ApplicationContext applicationContext = new ClassPathXmlApplicationContext("classpath:spring/remoting/jms/remoting-context-server.xml");

System.out.println("Started Server application...");

write simple client program..look up Calculator bean and invoke a method on bean as if your invoking a local object method.

ApplicationContext applicationContext = new ClassPathXmlApplicationContext("classpath:spring/remoting/jms/remoting-context-client.xml");

Calculator calculator = (Calculator)applicationContext.getBean("remoteCalculatorService");

Employee employee = new Employee(12,”Dhananjay”);

employee.setGrade(1);

long sal = calculator.calculateSalary(employee);

System.out.println("Salary Returned "+sal);
Simple isn't it?

Cheers,
Dhananjay

Spring Quartz Priority Job Queue

2010-12-10T10:54:00.000-08:00

Following post describes how to build a Priority Quartz Job queue.

Basic and most important factor in this is to define (Spring's SchedulerFactory bean )Quartz scheduler with single worker thread.

This can be defined in spring configuration as follows

<bean id="springScheduler"

                class="org.springframework.scheduling.quartz.SchedulerFactoryBean" destroy-method="destroy">
                <property name="applicationContextSchedulerContextKey" value="applicationContext">property>
                <property name="quartzProperties">
                        <props>
                                <prop key="org.quartz.threadPool.threadCount"> 1 prop>
                        props>
                property>bean>

As worker thread count is 1; this scheduler can run a one job at a time.

Now will write a Queue Job; which will be taking care of priority while executing individual jobs.

QueueJobBean.java is extension to spring's QuartzJobBean and internally maintains a TreeMap containing job names (spring bean names) with priorities.

<bean name="jobQueue" class="org.springframework.scheduling.quartz.JobDetailBean">

<property name="jobClass" value="scheduler.QueueJobBean" />
<property name="jobDataAsMap" ref="prioMap">property>
bean>

We can build a Priority Job map within spring bean configuration as below

<bean id="noOperation" class="scheduler.jobs.NoOpJob" >

    bean>

    <util:map id="prioMap" map-class="java.util.TreeMap">
        <entry key="1" value="noOperation"/>
        <entry key="2" value="noOperation"/>
        <entry key="3" value="noOperation"/>
    util:map>

Cheers,
Dhananjay

Spring - JMX

2010-09-18T09:05:00.000-07:00

Spring bean Definition for JMX Client

Spring Bean definition for MBean Server / MBeanExporter

A Simple Java Program demonstrates the usage...
TBD....

Performance Tips : Improving Web pages

2010-02-04T13:53:00.000-08:00

Though we see broadband connections every where nowadays, according to statistics only 6.51% of total Indian internet users use broadband connection. Story is not different in US too, where almost 25% of population still not using internet at all and nearly 60% have broadband connections while rest internet users still rely on dial up connections , for more details please visit
[http://www.internetworldstats.com/stats.htm]

As a application developer , what I can do to minimize pains of slow internet, is nothing but develop a application which is optimum on performance, loads quickly.

Below are few of tips for Web application developer , how to go about this

1.Adopt strict XHTML coding practices for faster download of webpages & parsing.

2. Don't use images to represent text [Unlike fonts , where you have little control over the way they appear in different browsers, images always appear exactly how you design them, But this is not an excuse to use images in place of text]

3.Cookies might be small files,but browsers will need to download them. Large cookies take longer to download, which in turn increase the time the browser takes to load pages.

4.Don't include(inline script) javascript code in web pages,externalize wherever possible.
Because externalized files gets cached by browser, whereas javascript coded inline in a web page gets loaded along with HTML.

5.If Possible avoid tables, If still you do, explicitly specify the widths and heights of the tablecells,rows & columns, otherwise , browser have to perform many operations to calculate how to display them, which may slow down web page loading

6.Instead of developing bulky and heavy web page with lot of contents,prefer small and less bulky pages. (Divide pages into small pages..)

7.Compress & minify Javascript files. Java script files can be quite impressive in size,which means there download times can exceed that of all other components combined.
One approach to compress javascript files is to use GNU zip (GZip) compression ,which is supported by most of the browsers.
Another approach is to minify the Javascript,Minification is the process where we remove unwanted characters like tabs,newlines,spaces,comments reducing file size.
JSMin and YUI Compressor.

8.Always use lower case divs and class names
When compression takes place , It will treat

and

as different tags.Hence its better we have them all in lower case.

9.Delay Script loading ,if possible
Put the javascript at the bottom of the web page.
Often browsers download no more than 2 parallel objects from same domain, If one of those objects is of javascipt code , the download of the other page components will halt until the script is downloaded
Hence give the javascript least priority to get downloaded.
Use either deffered loading or on demand loading of javascript.

10.Use Google gears when available to avoid repeatitive loading of contents.

11.Use the PNG (Portable Network Graphic) format images, PNG provides good quality with optimal file size.

12.Minimize ajax calls, keep them short and to the point

13.Use tools such as JSLint, Firebug, YSlow during development to catch hold of tricky performance issues.

Cheers,
Dhananjay

OPQ - Oracle Parallel Queries

2010-01-22T07:37:00.000-08:00

Let me clear one thing before you read out this post, 'I am not an oracle expert, I am just a Java developer'. [;)]
Recently I faced a problem , where in I had a materialized view based on a huge table [about a 500 Million].Due to some manual error this MV got disturbed.
We tried all means of getting this MV on track.
1. Fast refresh (It failed , suggesting complete / full refresh)
2. Complete Refresh (It was running for 2-2.5 days , locking all the base tables , so finally we killed it, as blocking base table was not at all feasible for production system)

It was very critical situation, as somehow we need to get that MV in place as this MV used to show up summary data graphically, on which various management decisions are based.
From business point of view ,its a very critical and we cant even think of building a small MV (for past 3-4 months and so on...)
I started exploring Oracle (I trust oracle and their products), from day 1 of this problem I was sure, there must be some way out of this.
Oracle must have taken care of this [:)]

While going through oracle 10g docs , I came across Oracle's Parallel Query (OPQ) feature, and that's it.
That's what I was looking for...
As my MV was calculating summary data, It's having a Full table scan of that huge table, which was causing processing delays..
I introduced parallelism in my MV definition as below

DROP MATERIALIZED VIEW MV_STAT;
CREATE MATERIALIZED VIEW MV_STAT
PARALLEL
BUILD IMMEDIATE
REFRESH FAST ON COMMIT
USING DEFAULT LOCAL ROLLBACK SEGMENT
DISABLE QUERY REWRITE
AS {SELECT QUERY CALCULATING SUM}

The word in bold , done the trick "Parallel".

After adding this "Golden Keyword" , MV got created in less than 30 minutes..
Problem solved..

When Oracle has to perform a legitimate, large, full-table scan, OPQ can make a dramatic difference in the response time.
Using OPQ, Oracle partitions the table into logical chunks.Once the table has been partitioned into pieces,
Oracle fires off parallel query slaves (aka factotum processes), and each slave simultaneously reads a piece of the large table.
Upon completion of all slave processes,Oracle passes the results back to a parallel query coordinator,
which will reassemble the data, perform a sort if required, and return the results back to the end user.
OPQ can give you almost infinite scalability.

For more details about this Please visit

http://download.oracle.com/docs/cd/B19306_01/server.102/b14223/usingpe.htm

Cheers,
Dhananjay

Enabling sitevsion Portal for debugging

2009-09-29T06:44:00.000-07:00

List of Modifications
Sitevision related
Path File name Changed Lines
${SITEVISION_HOME}/custom/conf/service debug.conf wrapper.java.additional.11=-Xrunjdwp:transport=dt_socket,server=y,
suspend=n,address=8001

This is java command line switch as explained below
-Xrunjdwp
This option loads the JPDA (Java Platform Debugger Architecture) reference implementation of JDWP (The Java Debug Wire Protocol (JDWP) is the protocol used for communication between a debugger and the Java virtual machine (VM) which it debugs (hereafter called the target VM). This library resides in the target VM and uses JVMDI and JNI to interact with it. It uses a transport and the JDWP protocol to communicate with a separate debugger application.
Operation
Format: -Xrunjdwp:[=],[=]...

Start sitevision in debug mode by running ${SITEVISION_HOME}/bin/debug.bat

Now, sitevision server is running in debug mode with socket for debugging available at 8001 port.

Eclipse settings

Create a debug configuration in eclipse

Create a remote java debugging configuration with values as

Name : MyPortal
Project : PortalProject
Connection Type : Standard (Socket Attach)
Host: localhost
Port : 8001 (same as that you specify in java command switch)

Happy debugging....

Java Design Patterns - Quick Reference

2009-07-25T06:57:00.000-07:00

1.Observer Pattern
Defines a one to many dependency between objects so that when one object changes state, all of its dependents are notified and updated automatically.
Java's built in Observer pattern
Subject(Object under observation) needs to implement "Observable"
This subject can notify its dependent by calling following methods sequentially
1.setChanged()
2.NotifyObserver() / notifyObservers()
Observer's needs to implement "Observer" interface and register / unregister to the observable as
addObserver(Observable)
removeObserver(Observable)

2.Decorator Pattern
Attaches additional responsibilities to an object dynamically.
Decorator provides a flexible alternative to subclassing for extending functionality.

Diagrammatically it is represented as

3.Factory Method Pattern
Defines an interface for creating an object but lets subclasses decide which class to instantiate.
(Factory method lets class defer instantiation to subclass)

Diagrammatically this can be represented as

4. Abstract Factory Pattern
Provides an interface for creating families of related / dependent objects without specifying their concrete classes.

Can be thought as Factory of Factory...
One of example of abstract Factory is
javax.xml.validation.SchemaFactory it has static factory method newInstance( ) which Lookup an implementation of the SchemaFactory that supports the specified schema language and return it.

5. Command Pattern
Encapsulates a request as an object thereby letting you parameterize other objects with different requests / queue .
You can provide functionality of undo too.
Typical client for the command uses as follows
{
Command cmd = new ConcreteCommand();
cmd.execute();
}

Following diagram will make it more clearer with action receiver .

6. Adapter Pattern
Converts the interface of a class into another interface , the client expects.
Adapter lets classes work together that couldn't otherwise because of incompatible interfaces.

7. Template Method Pattern
Defines the skeleton of an algorithm in a method, deferring some steps to sub classes.
Template method lets subclasses redefine certain steps of an algorithm without changing overall algorithm structure.
If you look @ Spring ,It has many templates defined
JMSTemplate, JDBCTemplate
JDBCTemplate simplifies the use of JDBC and helps to avoid common errors. It executes core JDBC workflow (get connection , cerate statement , set parameter , set connection, execute statement , extract results ), leaving application code to provide SQL and extract results.

8. Composite Pattern
Allows you to compose objects into tree structure to represent part-whole hierarchy.
Composite lets client treat individual objects and composition of objects uniformly.

Diagram :

9. State Pattern
Allows an object to alter its behavior when its internal state changes.
The object will appear to change its class .
Like a coffee vending machine changing state from working to bin full ,out of milk... states...

Diagrammatically it is represented as below

10. Proxy Pattern
Provide a surrogate / placeholder for another object to control access to it.
Use the proxy pattern to create a representative object that controls access to another object which may be remote / expensive to create / in need of securing
1. Remote Proxy
Controls access to remote object
2. Virtual Proxy
Controls access to resource which is expensive to create
3. Protection Proxy
Controls access to a resource based on access rights.

FTP Operations In Enterprise applications

2008-11-29T01:05:00.000-08:00

As far as my experience in enterprise application domain especially the application having integration kind of nature requires frequent FTP operation.
Recently me too had similar requirement,wherein I need to fetch a file from the remote server....
I was using Mule as integration bus for my application,though Mule provides in built support for FTP transport,It has 1 major drawback in case of FTP operations as listed below
1.Mule FTP Connector has polling nature and it deletes the file once its downloaded from FTP server.
Sometimes its allowed and perfectly acceptable to delete the downloaded file from remote FTP server but not always.
That lead me to apache commons NET project
This project consists of basic client side implementation of transport protocols such as SMTP,FTP,POP..
If you look at these API's these are one of the most simplest way to implement the FTPCleint

FTPClient ftpClient = new FTPClient();
ftpClient.connect("ftp.foo.com", "user01", "pass1234");
ftpClient.download("C:\\Temp\\", "README.txt");
// Eventually other operations here ...
ftpClient.disconnect();

Can you see ,how simple is this code...

Spring-Quartz Transactions

2008-09-27T12:04:00.000-07:00

Its been after a long time I got chance to sit back in summarize....Past 3 months were hectic in terms of work n there was always a race on ,against time.
Recently I came across a excellent feature of Spring framework,Just wanted to share with you all.
You must be aware of scheduling framework Quartz.
Quartz is a full-featured, open source job scheduling system that can be integrated with, or used along side virtually any J2EE or J2SE application.
I was working on a spring application ,all my business services are spring beans ,my transaction manager is "org.springframework.jdbc.datasource.DataSourceTransactionManager" which is local transaction manager,as my application is not in demand of JTA or any distributed transaction.
Now comes a trick ,when I come across following scenario
Suppose I have a business service which does following

Insert a record in DB
Schedules a job in Quartz
Updates some record in DB

Now what I required is

If update in step 3 fails ,I want to rollback steps 1,2
If Step 2 fails ,I want step 1 to rollback

As mine datasource is a Local transaction manager,It cant work across multiple resources and manage.That was enforcing me to use JTA or similar transaction manager ,which is actually overkill for my application.
As expcted ,Solution to this problem is provided by none other than spring,Thanks to Juergen Hoeller.
Spring comes with
org.springframework.scheduling.quartz.LocalDataSourceJobStore which is extension to Quartz JobStoreCMT
Now you just need to cnfigure your job store as below


org.quartz.jobStore.class =

org.springframework.scheduling.quartz.LocalDataSourceJobStore
Now ,Operations performed by this JobStore will properly participate in any kind of Spring-managed transaction, as it uses Spring's DataSourceUtils connection handling methods that are aware of a current transaction.
How easily spring solves this problem shows ,springs capability to be an enterprise application framework......

My thoughts on developement frameworks...

2008-06-20T10:59:00.000-07:00

I have been in the IT industry for past 2 years,which is really a short period.
But still some how I feel ,whatever I have gained so far,I can share...
So started writting....
I have been working on Java and related technlogies and frameworks,I worked on Spring,iBatis,bit of hibernate,struts along with some custom frameworks (Refres to frameworks developed by individual organizations to meet there technical requirement) n toolkits....
When we say its a framework ,its actually a collection of usage patterns combined to hide the complexity from animal called developer or the framework implementor or user.

Whenver these developers (me too still a developer :)) use these frameworks,
Usually framework are made for generic purpose ,so often we need write configuration file
either XML or properties file
The framework must be supported by IDE plugins (Such as Spring IDE for Eclipse for Spring) otherwise writting a XML configuraton according to framework expectation requires expertise and experience on that framework, a begineer will struggle like anything.
Second thing is the exceptions shown by frameworks ,are mostly misleading.... :( ,thats what my experience ,as in If you mis-spelled some attribute in some configuration file but it will show you all other things apart from it.. and now debugging a framework becomes a challenge....
In my career ,I worked on couple of custom framweorks too... the experience with them is also not that satisfactory..., as frameworks I used , which was simple ,but developers need to write more lines of code for developing a functionality which can be devloped in less number of lines without using that framework .
So, I feel a framework must developed keeping in mind that the implementor need to as less LOC possible .One truth is that as many LOC ,that many defects you will inject...
So My idea of good developement framework is..
1.Configurable via XML, with IDE plugin support
2.Good exception handlling,pin point the actual error .....
3.Minimal LOC for implementors or framework users
4.Ease of deugging or catching misconfiguration
5.Clear and comprehensive API documentation

Many of the industry proven developement frameworks do provide support for points 1,3 ,5 but points 2,4 are still very much required for good developement framework...
I have all this in mind from long period, finally was able to free up my mind..
Thats it....m summing up :)