NS.Infra

Profiling with Spring AOP

2014-02-01T16:47:00.001+05:30

You might be seeking for the option to profile (capturing method's execution time) your spring application. Spring provides different ways to profile the application. Profiling should be treated as a separate concern and spring AOP facilitates easy approach to separate this concern. With the help of spring AOP you can profile your methods without making any changes in actual classes.

You just need to perform pretty simple steps to configure your application for profiling using spring AOP:

In application context file, add below tag to enable the load time weaving

context:load-time-weaver

With this configuration, AspectJ's Load-time weaver is registered with current class loader. All classes

(which are matched with the pattern in below aspect class) are weaved at load time as per defined

aspect class file.

Create the aspect class for profiling

d This aspect is written to capture method's execution time. You can configure your required

classes (which need to be profiled) with @Around advice.

package com.xxx.profiling.aspect;

import org.apache.log4j.Logger;

import org.aspectj.lang.JoinPoint;

import org.aspectj.lang.ProceedingJoinPoint;

import org.aspectj.lang.annotation.Around;

import org.aspectj.lang.annotation.Aspect;

import org.aspectj.lang.annotation.Before;

import org.springframework.stereotype.Component;

/**

* Aspect class to profile the configured methods

@Component

@Aspect

public class ProfilingAspect {

/** Logger instance. */

private static final Logger LOGGER = Logger.getLogger(ProfilingAspect.class);

public LoggingAspect(){

LOGGER.info("Initializing Profiling Aspect... ");

}

public void profileMethods(ProceedingJoinPoint thisJoinPoint) throws Throwable {

StringBuffer logMessage = new StringBuffer();

logMessage.append(thisJoinPoint.getSignature().getName());

logMessage.append("(");

// Collect arguments value

Object[] args = thisJoinPoint.getArgs();

for (int i = 0; i < args.length; i++) {

logMessage.append(args[i]).append(",");

}

if (args.length > 0) {

logMessage.deleteCharAt(logMessage.length() - 1);

}

logMessage.append(") ");

// Calculate method's elapsed time

logMessage.append(" [Elapsed Time: " );

long start = System.currentTimeMillis();

thisJoinPoint.proceed();

long end = System.currentTimeMillis();

logMessage.append(end - start);

logMessage.append(" ms]");

LOGGER.debug("Profiling : "+thisJoinPoint.getTarget().getClass().getName() +"."+logMessage.toString());

}

// All public methods which are in services package will be weaved for profiling

@Around("execution(public * com.xxx.services..*.*(..))")

public void profileServiceMethods(ProceedingJoinPoint thisJoinPoint) throws Throwable {

profileMethods(thisJoinPoint);

}

// All public methods which are in dao package will be weaved for profiling

@Around("execution(public * com.xxx.dao..*.*(..))")

public void profileDAOMethods(ProceedingJoinPoint thisJoinPoint) throws Throwable {

profileMethods(thisJoinPoint);

}

Scan package in application context file to initialize this aspect class

Download the spring-agent.jar file from here

Configure JAVA VM parameter by providing spring agent jar path

-javaagent:C:/yourpath/spring-agent-2.5.6.jar

You are done with required configuration and implementation. Now it's time to see your execution statistics.

Start your application server and verify whether aspect class is initialized successfully. You should get below log message in application log file. If this log message comes, that means your aspect is initialized properly.

Initializing Profiling Aspect...

Whenever your service or data access class methods are executed you must see below logs.

Profiling :com.xxx.evolution.dao.MyDAO.getUserDetail(123) [ Elapsed Time : 246 ms ]

Profiling :com.xxx.evolution.services.MyService.getUserDetail(123) [ Elapsed Time : 247 ms ]

Apart from this approach you can also go for other options for profiling:

Locking Management in JPA

2014-01-25T13:47:00.001+05:30

Why Locking is required?

When two concurrent users try to update database row simultaneously, there are absolute chances of losing data integrity. Locking comes in picture to avoid simultaneous updates and ensure data integrity.

Types of Locking
There are two types of locking, Optimistic and Pessimistic. In this post optimistic locking is described with example.

Optimistic Locking: This locking is applied on transaction commit. If entity (database row) is fetched by two users simultaneously, where first user updates and commits the row and second user tries to update row with an old version. Second user will get an exception as row is already updated and version is changed. A version number is associated for entity object to achieve optimistic locking. Whenever entity object is changed, its version number is automatically increased by one. If any transaction is performed on older version of entity, transaction is failed and application throws optimistic locking exception.

Pessimistic Locking: Pessimistic locking explicitly restricts shared resources until the transaction completed. To get more detail about pessimistic locking refer this link .

How to Implement Optimistic Locking in JPA 2.0?

Let’s take an example; there are two Tables Company and Employee:

Table: Company
Company_Id	Company_Name	Address	Survey_Sequnce	Version
1	AirTel	xyz	110	20
2	Reliance	abc	230	40

Table: Employee
Employee_Id	Employee_Name	Company_Id	Employee_Survey_No
1	Narendra	1	111
2	Vinay	1	112
3	Ranu	2	231
4	Rinku	2	232

Whenever, employee joins the company, he is asked to complete the company specific survey and after completion of the survey, 'Employee_Survey_No' is generated based on SURVEY_SEQUENCE from company table. To generate the survey number below steps are performed:

Read the current sequence value specific to a company from company table (SURVEY_SEQUENCE)
Increment the sequence count by 1
Update the incremented value in company table (SURVEY_SEQUENCE)
Use this incremented value to log into employee table as 'Employee_Survey_No'

In this use case, there are absolute chances that two parallel surveys are performed for same company, where two employees get same survey sequence. This scenario can be handled with the help of optimistic locking in JPA to avoid sequence update collisions for same company.

Let’s implement optimistic locking in JPA. JPA facilitates easy approach to handle optimistic locking.

Define the company entity and annotate version column property with @Version to enable optimistic locking for entity.

@Entity

@Table(name = "Company", schema = "MyDB")

public class Company {

private Long companyId;

private String companyName;

private String address;

private Long surveySequence;

private Long version;

@Id

@SequenceGenerator(name = "company_seq", sequenceName = "MyDB.company_seq")

@GeneratedValue(generator = "company_seq")

@Column(name = "COMPANY_ID", unique = true, nullable = false)

public Long getCompanyId() {

return companyId;

}

public void setCompanyId(Long companyId) {

this.companyId = companyId;

}

@Column(name = "COMPANY_NAME")

public String getCompanyName() {

return companyName;

}

public void setCompanyName(String companyName) {

this.companyName = companyName;

}

@Column(name = "ADDRESS")

public String getAddress() {

return address;

}

public void setAddress(String address) {

this.address = address;

}

@Column(name = "SURVEY_SEQUENCE")

public Long getSurveySequence() {

return surveySequence;

}

public void setSurveySequence(Long surveySequence) {

this.surveySequence = surveySequence;

}

@Column(name = "VERSION_ID")

@Version

public Long getVersion() {

return version;

}

public void setVersion(Long version) {

this.version = version;

}

Now create the company DAO method to fetch the survey sequence, increment and update incremented value into company table. As we have annotated company table column with @Version, JPA will take care about parallel company entity updates. If any transaction is performed with older version of entity, transaction is failed and this method will throw ‘OptimisticLockingFailureException’.

@Repository("companyDAO")

public class CompanyDAO {

public Long getNextSurveySequence(Long companyId) {

//Fetch company entity based on companyId

Company company = entityManager.findById(Company.class, companyId);

// Get the current survey sequence

Long surveySeq = company.getSurveySequence();

// Increment current sequence by 1

Long nextSurveySeq = surveySeq + 1L;

// Set new sequence in company entity

Company.setSurveySequence(nextSurveySeq)

// Update company with new sequence

entityManager.merge(company);

return nextSurveySeq;

}

Now create the service method to fetch the next survey sequence and handle optimistic locking exception. With maximum try count approach, you can retry data base operation for failed transactions.

@Service("companyService")

public class CompanyService {

public Long getNextSequenceForOrgId(Long companyId) throws Exception {

Long surveySequence = null;

//If any transaction is performed on older version of entity,

//transaction is failed and application throws the ‘OptimisticLockException’

for (int maxTryCount = 1; maxTryCount <= 3; maxTryCount++)

{

try {

surveySequence = companyDao.getNextSurveySequence(companyId);

break;

} catch (OptimisticLockingFailureException e) {

if (maxTryCount == 3) {

throw new Exception(e);

}

return surveySequence;

}

I hope this post helps you understanding and implementing optimistic locking with JPA. If you would like to share your thoughts on it, you are most welcome to comment.

Liferay + Spring MVC Portlets + Free Text Search

2014-01-11T13:13:00.000+05:30

Liferay is one of the portal frameworks based on java. You can create portlets in Liferay using Spring MVC framework. The main purpose of this post is to provide the complete solution for free text search integration with Liferay Spring MVC portlets. If I say in short, this post may be the answer of below questions:

How to create a portlet in Liferay using Spring MVC
How to use Liferay free text search functionality in custom portlet

Liferay uses Apache Lucene to provide the free text search functionality. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java.

Example Portlets

I have created below Spring MVC portlets in Liferay to understand that how we can use free text search functionality in our web site.

1. Add Search Content Portlet: This portlet adds the content which needs to be searched. For demo purpose, I have used three fields to add the search contents: ID, Title and Description. If you see in below screen shots, I have added three hotel names as search content in terms of ID (hotel id), Title (hotel name) and Description (about hotel).

2. Search Portlet: This portlet provides search field from where you can search the content
which have been added using above portlet ‘Add Search Content Portlet’. For example if you want
to search keyword like ‘Kapila Resort’. It will search this text in all added contents and will return
the result with matching score (Ratings).

Source Code for Example Portlets

This section provides source code for example portlets. To create the spring MVC portlet in Liferay you need to consider below points:

● Adding configuration in portlet.xml, liferay-display.xml and liferay-portlet.xml

● View Render Servlet configuration in web.xml

● Spring context configuration for View Resolver and Controller

● Creating Value Objects

● Creating Controller

● Creating JSP

For more detail about creating Spring MVC portlets in Liferay please refer.

● To create ‘Add Search Content Portlet’, follow below steps:

1. Configure ‘applicationContext.xml’ file:

xml version='1.0' encoding='UTF-8'?>

<beans xmlns="http://www.springframework.org/schema/beans"

xmlns:aop="http://www.springframework.org/schema/aop"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xmlns:context="http://www.springframework.org/schema/context"

xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd

http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd">

<context:annotation-config />

<context:component-scan base-package="your.package.mvc" />

<bean id="messageSource"

class="org.springframework.context.support.ResourceBundleMessageSource">

<list>

<value>messages</value>

</list>

</property>

</bean>

<bean id="viewResolver"

class="org.springframework.web.servlet.view.InternalResourceViewResolver">

<property name="viewClass"

value="org.springframework.web.servlet.view.JstlView" />

</bean>

<bean id="parameterMappingInterceptor"

class="org.springframework.web.portlet.handler.ParameterMappingInterceptor"/>

</beans>

2. Add below configuration in ‘portlet.xml’ file:

<portlet-name>addSearchContent</portlet-name>

<portlet-class>org.springframework.web.portlet.DispatcherPortlet</portlet-class>

<init-param>

<name>contextConfigLocation</name>

<value>/WEB-INF/context/addSearchContent.xml</value>

</init-param>

<mime-type>text/html</mime-type>

<portlet-mode>view</portlet-mode>

</supports>

<resource-bundle>content.messages</resource-bundle>

<portlet-info>

<title>Add Search Contents Portlet</title>

</portlet-info>

</portlet>

3. Add below configuration in ‘liferay-portlet.xml’ file:

<portlet-name>addSearchContent</portlet-name>

</portlet>

4. Add below configuration in ‘liferay-display.xml’ file:

</category>

5. Create ‘addSearchContent.xml’ file with below contents:

xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xmlns:p="http://www.springframework.org/schema/p"

xmlns:context="http://www.springframework.org/schema/context"

xmlns:util="http://www.springframework.org/schema/util"

xsi:schemaLocation="

http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-beans-3.0.xsd

http://www.springframework.org/schema/context

http://www.springframework.org/schema/context/spring-context-3.0.xsd

http://www.springframework.org/schema/util

http://www.springframework.org/schema/util/spring-util-3.0.xsd

http://www.springframework.org/schema/mvc

http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

<map>

</entry>

</map>

</property>

</bean>

</beans>

6. Create value objects:

● SearchContentVO Object

package your.package.mvc.vo;

import java.util.List;

/**

* The Class SearchContentVO.

* @author nverma

public class SearchContentVO {

/** The id. */

private Long id;

/** The title. */

private String title;

/** The description. */

private String description;

/** The search contents. */

private List searchContents;

/**

* Gets the id.

* @return the id

public Long getId() {

return id;

}

/**

* Sets the id.

* @param id the new id

public void setId(Long id) {

this.id = id;

}

/**

* Gets the title.

* @return the title

public String getTitle() {

return title;

}

/**

* Sets the title.

* @param title the title to set

public void setTitle(String title) {

this.title = title;

}

/**

* Gets the description.

* @return the description

public String getDescription() {

return description;

}

/**

* Sets the description.

* @param description the description to set

public void setDescription(String description) {

this.description = description;

}

/**

* Gets the search contents.

* @return the search contents

public List getSearchContents() {

return searchContents;

}

/**

* Sets the search contents.

* @param searchContents the new search contents

public void setSearchContents(List searchContents) {

this.searchContents = searchContents;

}

● ‘SearchResult’ Object

package your.package.mvc.vo;

/**

* The Class SearchResult.

* @author nverma

public class SearchResult {

/** The id. */

private Long id;

/** The title. */

private String title;

/** The description. */

private String description;

/** The search score. */

private float searchScore;

/**

* Gets the id.

* @return the id

public Long getId() {

return id;

}

/**

* Sets the id.

* @param id the new id

public void setId(Long id) {

this.id = id;

}

/**

* Gets the title.

* @return the title

public String getTitle() {

return title;

}

/**

* Sets the title.

* @param title the title to set

public void setTitle(String title) {

this.title = title;

}

/**

* Gets the description.

* @return the description

public String getDescription() {

return description;

}

/**

* Sets the description.

* @param description the description to set

public void setDescription(String description) {

this.description = description;

}

/**

* Gets the search score.

* @return the searchScore

public float getSearchScore() {

return searchScore;

}

/**

* Sets the search score.

* @param searchScore the searchScore to set

public void setSearchScore(float searchScore) {

this.searchScore = searchScore;

}

7. Create controller ‘AddSearchContentController’:

package your.package.mvc.controller;

import java.util.ArrayList;

import java.util.List;

import javax.portlet.ActionRequest;

import javax.portlet.ActionResponse;

import javax.portlet.PortletRequest;

import javax.portlet.RenderResponse;

import org.apache.commons.logging.Log;

import org.apache.commons.logging.LogFactory;

import org.springframework.stereotype.Controller;

import org.springframework.web.bind.annotation.ModelAttribute;

import org.springframework.web.bind.annotation.RequestMapping;

import org.springframework.web.portlet.bind.annotation.ActionMapping;

import org.springframework.web.portlet.bind.annotation.RenderMapping;

import your.package.mvc.vo.SearchContentVO;

import your.package.mvc.vo.SearchResult;

import com.liferay.portal.kernel.search.BooleanQuery;

import com.liferay.portal.kernel.search.BooleanQueryFactoryUtil;

import com.liferay.portal.kernel.search.Document;

import com.liferay.portal.kernel.search.DocumentImpl;

import com.liferay.portal.kernel.search.Field;

import com.liferay.portal.kernel.search.Hits;

import com.liferay.portal.kernel.search.ParseException;

import com.liferay.portal.kernel.search.SearchEngineUtil;

import com.liferay.portal.kernel.search.SearchException;

/**

* Controller to add search contents.

* @author nverma

@Controller("addSearchContentController")

@RequestMapping(value = "VIEW")

public class AddSearchContentController extends AbstractBaseController {

/** The Constant LOG. */

private static final Log LOG = LogFactory

.getLog(AddSearchContentController.class);

/** The Constant MODEL_ATT. */

private static final String MODEL_ATT = "searchContentVO";

/**

* Inits the command object.

* @param portletRequest the portlet request

* @return the search content vo

@ModelAttribute(MODEL_ATT)

public SearchContentVO initCommandObject(final PortletRequest portletRequest) {

LOG.debug("Initializing command object 'SearchContentVO'...");

SearchContentVO searchContentVO = new SearchContentVO();

searchContentVO.setSearchContents(allItems());

return searchContentVO;

}

/**

* Render add search content.

* @param renderResponse the render response

* @return the string

@RenderMapping

public String renderAddSearchContent(final RenderResponse renderResponse) {

return "addSearchContent";

}

/**

* Adds the search contents.

* @param searchContentVO the search content vo

* @param actionRequest the action request

* @param actionResponse the action response

@ActionMapping(params = "action=addSearchContent")

public void addSearchContents(@ModelAttribute SearchContentVO searchContentVO,

final ActionRequest actionRequest, ActionResponse actionResponse) {

// Adding Search Document

Document document = new DocumentImpl();

document.addKeyword("Id", searchContentVO.getId());

document.addText(Field.TITLE, searchContentVO.getTitle());

document.addText("description", searchContentVO.getDescription());

document.addText("AllDocuments", "OnlyMyItems");

try {

LOG.info("Adding search content...");

SearchEngineUtil.addDocument(10132, document);

LOG.info("Added search content successfully.");

} catch (SearchException e) {

LOG.error("Failed to add search document." +e.getMessage());

}
searchContentVO.setSearchContents(allItems());

}

/**

* Delete search contents.

* @param searchContentVO the search content vo

* @param actionRequest the action request

* @param actionResponse the action response

@ActionMapping(params = "action=deleteSearchContent")

public void deleteSearchContents(@ModelAttribute SearchContentVO searchContentVO,

final ActionRequest actionRequest, ActionResponse actionResponse) {

try {

LOG.info("Deleting search content...["+searchContentVO.getId()+"]");

SearchEngineUtil.deleteDocument(10132, String.valueOf(searchContentVO.getId()));

LOG.info("Deleted search content successfully.");

} catch (SearchException e) {

LOG.error("Failed to deleted search document." +e.getMessage());

}

searchContentVO.setSearchContents(allItems());

}

/**

* All items.

* @return the list

public List allItems(){

List searchResultLst = new ArrayList();

Document[] resultDocs = null;

Hits hits = null;

BooleanQuery searchQuery = BooleanQueryFactoryUtil.create();

try {

searchQuery.addTerm("AllDocuments", "OnlyMyItems");

hits = SearchEngineUtil.search(10132, searchQuery, -1, -1);

resultDocs = hits.getDocs();

} catch (ParseException e) {

LOG.error("Failed to parse while searching. " + e.getMessage());

e.printStackTrace();

} catch (SearchException e) {

LOG.error("Failed to search. " + e.getMessage());

e.printStackTrace();

}

for (int i = 0; i < resultDocs.length; i++) {

LOG.debug("Searched Items: ");

SearchResult searchResult = new SearchResult();

LOG.debug("Document -> " + hits.doc(i).get(Field.UID));

String idStr = hits.doc(i).get("Id");

if(idStr==null || idStr.isEmpty()){

continue;

}

Long id = Long.parseLong(idStr);

LOG.debug("Id -> " + id);

searchResult.setId(id);

String title = hits.doc(i).get(Field.TITLE);

if(title==null || title.isEmpty()){

continue;

}

LOG.debug("Title -> " + title);

searchResult.setTitle(title);

String description = hits.doc(i).get("description");

LOG.debug("Description -> " + description);

searchResult.setDescription(description);

//Add search result into list

searchResultLst.add(searchResult);

}

return searchResultLst;

}

The API marked under arrow lines is responsible to fetch the search content from JSP and then add them into search content repository.

8. Create JSP with name ‘addSearchContent.jsp’:

<%@ taglib prefix="portlet" uri="http://java.sun.com/portlet_2_0"%>

<%@ taglib uri="http://liferay.com/tld/ui" prefix="liferay-ui" %>

<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core"%>

<%@ taglib prefix="fmt" uri="http://java.sun.com/jsp/jstl/fmt"%>

<%@ page contentType="text/html" isELIgnored="false"%>

<%@ taglib prefix="form" uri="http://www.springframework.org/tags/form"%>

<portlet:actionURL var="addSearchContent">

<portlet:param name="action" value="addSearchContent" />

</portlet:actionURL>

<c:if test="${searchContentVO != null}">

<table>

<tr>

<td>

<h1>Add Search Contents in Search Repository</h1>

</td>

</tr>

<tr>

<td>

<form:form id="search" commandName="searchContentVO" action="${addSearchContent}">

<tr>

<td width=80 align = "center"><b>Title</b></td>

<td width=80 align = "center"><b>Description</b></td> </tr>

<tr>

</tr>

</table>

<br/>

</form:form>

</td>

</td>

<td>

<form:form id="search2" commandName="searchContentVO" action="${addSearchContent}">

<tr>

</tr>

<tr>

</tr>

</table>

<br/>

</form:form>

</td>

</table>

<br/>

<h1>Added Search Contents in Search Repository</h1>

<td width=250 align = "center"><b>Title</b></td>

<td width=480 align = "center"><b>Description</b></td>

</tr>

<c:forEach var="searchContent" items="${searchContentVO.searchContents}">

<td width=80 align = "center">${searchContent.id}</td>

<td width=250 align = "center">${searchContent.title}</td>

<td width=480>${searchContent.description}</td>

</tr>

</c:forEach>

</table>

</c:if>

<br/>

● To create ‘Search Portlet’, follow below steps:

1. Add below configuration in ‘portlet.xml’ file:

<portlet-name>search</portlet-name>

<portlet-class>org.springframework.web.portlet.DispatcherPortlet</portlet-class>

<init-param>

<name>contextConfigLocation</name>

<value>/WEB-INF/context/search.xml</value>

</init-param>

<mime-type>text/html</mime-type>

<portlet-mode>view</portlet-mode>

</supports>

<resource-bundle>content.messages</resource-bundle>

<portlet-info>

<title>Search Portlet</title>

</portlet-info>

</portlet>

2. Add below configuration in ‘liferay-portlet.xml’ file:

<portlet-name>search</portlet-name>

</portlet>

3. Add below configuration in ‘liferay-display.xml’ file

</category>

4. Create ‘search.xml’ file with below contents

xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xmlns:p="http://www.springframework.org/schema/p"

xmlns:context="http://www.springframework.org/schema/context"

xmlns:util="http://www.springframework.org/schema/util"

xsi:schemaLocation="

http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-beans-3.0.xsd

http://www.springframework.org/schema/context

http://www.springframework.org/schema/context/spring-context-3.0.xsd

http://www.springframework.org/schema/util

http://www.springframework.org/schema/util/spring-util-3.0.xsd

http://www.springframework.org/schema/mvc

http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">

<beanclass="org.springframework.web.portlet.mvc.annotation.AnnotationMethodHandlerAdapter"/>

<map>

</entry>

</map>

</property>

</bean>

</beans>

5. Create Value object ‘SearchVO’:

package your.package.mvc.vo;

import java.util.List;

/**

* The Class SearchVO.

* @author nverma

public class SearchVO {

/** The no search result flag. */

private boolean noSearchResultFlag;

/** The search words. */

private String searchWords;

/** The hotel search result. */

private List searchResult;

/**

* Checks if is no search result flag.

* @return true, if is no search result flag

public boolean isNoSearchResultFlag() {

return noSearchResultFlag;

}

/**

* Sets the no search result flag.

* @param noSearchResultFlag the new no search result flag

public void setNoSearchResultFlag(boolean noSearchResultFlag) {

this.noSearchResultFlag = noSearchResultFlag;

}

/**

* Gets the search words.

* @return the search words

public String getSearchWords() {

return searchWords;

}

/**

* Sets the search words.

* @param searchWords the new search words

public void setSearchWords(String searchWords) {

this.searchWords = searchWords;

}

/**

* Gets the search result.

* @return the search result

public List getSearchResult() {

return searchResult;

}

/**

* Sets the search result.

* @param searchResult the new search result

public void setSearchResult(List searchResult) {

this.searchResult = searchResult;

}

6. Create controller ‘SearchController’:

package your.package.controller;

import java.util.ArrayList;

import java.util.List;

import javax.portlet.ActionRequest;

import javax.portlet.ActionResponse;

import javax.portlet.PortletRequest;

import javax.portlet.RenderResponse;

import com.liferay.portal.kernel.search.BooleanQuery;

import com.liferay.portal.kernel.search.BooleanQueryFactoryUtil;

import com.liferay.portal.kernel.search.Document;

import com.liferay.portal.kernel.search.Field;

import com.liferay.portal.kernel.search.Hits;

import com.liferay.portal.kernel.search.ParseException;

import com.liferay.portal.kernel.search.SearchEngineUtil;

import com.liferay.portal.kernel.search.SearchException;

import org.apache.commons.logging.Log;

import org.apache.commons.logging.LogFactory;

import org.springframework.stereotype.Controller;

import org.springframework.web.bind.annotation.ModelAttribute;

import org.springframework.web.bind.annotation.RequestMapping;

import org.springframework.web.portlet.bind.annotation.ActionMapping;

import org.springframework.web.portlet.bind.annotation.RenderMapping;

import your.package.mvc.vo.SearchResult;

import your.package.mvc.vo.SearchVO;

/**

* Controller for free text search.

* @author nverma

@Controller("searchController")

@RequestMapping(value = "VIEW")

public class SearchController extends AbstractBaseController {

/** The Constant LOG. */

private static final Log LOG = LogFactory

.getLog(SearchController.class);

/** The Constant MODEL_ATT. */

private static final String MODEL_ATT = "searchVO";

/**

* Inits the command object.

* @param portletRequest the portlet request

* @return the search vo

@ModelAttribute(MODEL_ATT)

public SearchVO initCommandObject(final PortletRequest portletRequest) {

LOG.debug("Initializing command object 'SearchVO'...");

return new SearchVO();

}

/**

* Render search.

* @param renderResponse the render response

* @return the string

@RenderMapping

public String renderSearch(final RenderResponse renderResponse) {

return "search";

}

/**

* Render search result.

* @param renderResponse the render response

* @return the string

@RenderMapping(params = "action=showSearchResult")

public String renderSearchResult(final RenderResponse renderResponse) {

return "search";

}

/**

* Search.

* @param searchVO the search vo

* @param actionRequest the action request

* @param actionResponse the action response

@ActionMapping(params = "action=search")

public void search(@ModelAttribute SearchVO searchVO,

final ActionRequest actionRequest, ActionResponse actionResponse) {

List searchResultLst = new ArrayList();

String searchKeyWords = searchVO.getSearchWords();

LOG.debug("Searing for input keywords [" + searchKeyWords + "]...");

Document[] resultDocs = null;

Hits hits = null;

BooleanQuery searchQuery = BooleanQueryFactoryUtil.create();

try {

searchQuery.addTerm(Field.TITLE, searchKeyWords);

searchQuery.addTerm("description", searchKeyWords);

hits = SearchEngineUtil.search(10132, searchQuery, -1, -1);

resultDocs = hits.getDocs();

} catch (ParseException e) {

LOG.error("Failed to parse while searching. " + e.getMessage());

e.printStackTrace();

} catch (SearchException e) {

LOG.error("Failed to search. " + e.getMessage());

e.printStackTrace();

}

if(hits == null || hits.getLength() == 0){

searchVO.setNoSearchResultFlag(true);

}

LOG.debug(" Search result count [" + hits.getLength() + "]");

for (int i = 0; i < resultDocs.length; i++) {

LOG.debug("Searched Items: ");

SearchResult searchResult = new SearchResult();

LOG.debug("Document -> " + hits.doc(i).get(Field.UID));

String onlyForHotel = hits.doc(i).get("AllDocuments");

if(!"OnlyMyItems".equals(onlyForHotel)){

continue;

}

String idStr = hits.doc(i).get("Id");

if(idStr==null || idStr.isEmpty()){

continue;

}

Long id = Long.parseLong(idStr);

LOG.debug("Id -> " + id);

searchResult.setId(id);

String title = hits.doc(i).get(Field.TITLE);

LOG.debug("Title -> " + title);

searchResult.setTitle(title);

String description = hits.doc(i).get("description");

LOG.debug("Description -> " + description);

searchResult.setDescription(description);

float score = hits.score(i);

LOG.debug("Result Score -> " + hits.score(i));

searchResult.setSearchScore(score);

//Add search result into list

searchResultLst.add(searchResult);

}

searchVO.setSearchResult(searchResultLst);

actionResponse.setRenderParameter("action", "showSearchResult");

LOG.debug("Searched successfully.");

}

The API marked under arrow lines is responsible to perform the search for given keywords and set the result into VO to render on JSP.

7. Create JSP ‘search.jsp’:

<%@ taglib prefix="portlet" uri="http://java.sun.com/portlet_2_0"%>

<%@ taglib uri="http://liferay.com/tld/ui" prefix="liferay-ui" %>

<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core"%>

<%@ taglib prefix="fmt" uri="http://java.sun.com/jsp/jstl/fmt"%>

<%@ page contentType="text/html" isELIgnored="false"%>

<%@ taglib prefix="form" uri="http://www.springframework.org/tags/form"%>

<portlet:actionURL var="searchUrl">

<portlet:param name="action" value="search" />

</portlet:actionURL>

<c:if test="${searchVO != null}">

<form:form id="search" commandName="searchVO" action="${searchUrl}">

<form:input path="searchWords" value = "Search Words" class="textfield disabledfield"/>

</form:form>

</c:if>

<h3>Search Result</h3>

<form:form id="searchItem" commandName="searchVO">

<td width=80 align = "center"><b>Search ID</b></td>

<td width=250 align = "center"><b>Title</b></td>

<td width=480 align = "center"><b>Description</b></td>

<td width=140 align = "center"><b>Ratings</b></td>

</tr>

<c:forEach var="searchResult" items="${searchVO.searchResult}">

<td width=80 align = "center"><a href="/search?searchId=${searchResult.id}">${searchResult.id}</a></td>

<td width=250 align = "center"><a href="/search?searchId=${searchResult.id}">${searchResult.title}</a></td>

<td width=480 align = "center"><a href="/search?searchId=${searchResult.id}">${searchResult.description}</a></td>

</tr>

</c:forEach>

</table>

</form:form>

After configuring all above source files you can build your application and then deploy on application server. Access your Liferay server, add a test page and drop both portlets on test page and test the search functionality by adding search content first and then searching keywords.

:) Happy Free Text Searching :)

See More:

Liferay, Apache Lucene, Spring MVC Portlet, Adding Search Capabilities to a Portlet

Coding Contest - Arranging numbers from 1 to 71 in a sequence such that sum of any 2 adjacent numbers in the sequence adds up to a prime number

2014-01-11T12:49:00.001+05:30

I have got a problem in coding contest. In this post, I would like to share the approach to solve this problem. I would definitely not say that I have invented something new and I am not trying to reinvent the wheel again. I just described the end to end approach that I followed to solve this problem.It was really a great brain storming activity.

Problem Statement

"Arrange numbers from 1 to 71 in a sequence such that sum of any 2 adjacent numbers in the sequence adds up to a prime number.”For e.g : 7,6,5,2,1,4,3 is a valid one sequence for arranging 1..7"

Theory: How to Solve?

Let's take an example, we want to find all series from 1 to 5 such that sum of any 2 adjacent numbers in the sequence adds up to a prime number. For this problem, there should be below no. of possible combinations.

1, 4, 3, 2, 5

3, 4, 1, 2, 5

5, 2, 1, 4, 3

5, 2, 3, 4, 1

Now, lets talk about the approach to get above combinations.

Firstly, for each number, check all possible numbers in series such that sum should be a prime number. Let's take number 1 and check which sum is prime number out of 1+2 = 3, 1+3=4, 1+4=5, 1+5=6. So, we have got 1+2 = 3 and 1+4=5 that means 2 and 4 are the numbers that can be connected to number 1. Now apply the same logic for all remaining numbers 2,3,4,5. You will get below combinations.

1 --> 2, 4

2 --> 1, 3, 5

3 --> 2, 4

4 --> 1, 3

5 --> 2

Secondly, create a graph that connects any two numbers that has a prime number sum. If you take number 1 that is connected to 2 and 4, the graph must represent link between connected numbers. In graph's terminology you can say each number is a vertex and each connected link is an edge.

Till now you are half done. You have graph that connects any two numbers that has a prime number sum.

Thirdly, you just need to find out the unique cycles in graph where you visit each vertex exactly once. This is kind of calculating Hamiltonian Paths (also refer this).

Below graph shows two cycles:

1, 4, 3, 2, 5

5, 2, 3, 4, 1

Below graph shows two cycles:

3, 4, 1, 2, 5

5, 2, 1, 4, 3

Theoretically you have achieved what you wanted to achieve. You have all 4 possible combinations for given problem from 1 to 5.

Now let's convert above theory into computer programming and ask computer to get combinations for any given number. Well, I am a Java Techie and I would prefer to convert this theory into java programming. If you are a Java Techie you have prepared solution in your hand. In case you are not, you have to struggle converting my program into your required language.

Programmatic Approach: How to Solve?

Before jumping into the programming, you have to think first how you can represent this graph in java. This is bit complex. But once you get into this; it will be easy to understand and apply.

I took help from here to represent the graph in two dimensional array. To get combinations from 1 to 5, graph can be represented in 2D array like below:

Action Points for Programming

Populate 2D array that connects any two numbers that has a prime number sum.
Implement recursive function to find out the path that visits each connected number exactly once
Print the path if path length equals to the provided last number

Solution in Java

Program

import java.io.File;

import java.util.ArrayList;

import java.util.Date;

import java.util.List;

import org.apache.log4j.Appender;

import org.apache.log4j.DailyRollingFileAppender;

import org.apache.log4j.FileAppender;

import org.apache.log4j.Level;

import org.apache.log4j.Logger;

import org.apache.log4j.PatternLayout;

/**

* Coding Contest

* Arrange numbers from 1 to n in a sequence such that sum of any 2 adjacent

* numbers in the sequence adds up to a prime number. For e.g : 7,6,5,2,1,4,3 is

* a valid one sequence for arranging 1..7

* @author NarendraVerma

public class PrimeSequenceSolution {

/** The pattern layout. */

private static PatternLayout patternLayout = new PatternLayout("%d{ISO8601}\t%m%n");

/** Logger instance. */

private static final Logger LOGGER = Logger

.getLogger(PrimeSequenceSolution.class);

/** Matrix that holds all prime numbers . */

public static int[][] inputArray = null;

/** Matrix[1][n] hat connects any two numbers that has a prime number sum. */

public static int inputArrayLength;

/** The total sequences count. */

public static int totalCount = 0;

/** Maximum Sequences to be generated. */

public static long maxSeq = 1000000000;

/** The start time. */

public static long startTime;

/** The logging flag. */

public static boolean loggingFlag = true;

/**

* Find prime sequences.

public static void findPrimeSequences() {

traverseVertexes(new ArrayList());

}

/**

* Populate 2D array that connects any two numbers that has a prime number sum

* For example: If last No is 5, matrix is populated like below

* 1 --> 2, 4

* 2 --> 1, 3, 5

* 3 --> 2, 4

* 4 --> 1, 3

* 5 --> 2

* 2D Array:

* 1 2 3 4 5

* 1 [0 1 0 1 0]

* 2 [1 0 1 0 1]

* 3 [0 1 0 1 0]

* 4 [1 0 1 0 0]

* 5 [0 1 0 0 0]

* @param lastNo the last no

public static void prepareInputMatrix(int lastNo) {

inputArray = new int[lastNo][lastNo];

for (int i = 1; i <= lastNo; i++) {

for (int j = 1; j <= lastNo; j++) {

if (i == j) {

inputArray[i - 1][j - 1] = 0;

} else {

inputArray[i - 1][j - 1] = isPrime(i + j);

}

/**

* Checks if given number is prime number

* @param number

* the number

* @return the int

public static int isPrime(int number) {

// check if number is a multiple of 2

if (number % 2 == 0)

return 0;

// if not, then check the odd numbers

for (int i = 3; i * i <= number; i += 2) {

if (number % i == 0)

return 0;

}

return 1;

}

/**

* The recursive function to find out the path that visits each connected number exactly once

* @param traversedPath

* the path so far

* @return the list

public static List traverseVertexes(List traversedPath) {

inputArrayLength = inputArray.length;

if (traversedPath.size() == inputArrayLength) {

printSequences(traversedPath);

return traversedPath;

} else if (traversedPath.size() == 0) {

for (int i = 0; i < inputArrayLength; i++) {

traversedPath.add(i);

traverseVertexes(traversedPath);

traversedPath.remove(traversedPath.size() - 1);

}

} else {

int currentNode = traversedPath.get(traversedPath.size() - 1);

for (int i = 0; i < inputArrayLength; i++) {

if (!traversedPath.contains(i) && inputArray[currentNode][i] != 0) {

System.out.println("I am here:" + traversedPath );

traversedPath.add(i);

traverseVertexes(traversedPath);

traversedPath.remove(traversedPath.size() - 1);

}

return traversedPath;

}

/**

* Prints the solution.

* @param pathSoFar

* the path so far

public static void printSequences(List pathSoFar) {

StringBuffer sb = new StringBuffer(" -- ");

sb.append(" [ SeqNo."+ (totalCount + 1 )+"] -- ");

for (Integer number : pathSoFar) {

sb.append(number + 1).append(", ");

}

if (loggingFlag)

LOGGER.info(sb.toString());

else

System.out.println(new Date() + " " + sb.toString());

if (maxSeq != 1000000000 && totalCount > maxSeq - 2) {

LOGGER.info("Maximum sequences [ " + maxSeq

+ " ] have been generated. Total-Elapsed-Time [ "

+ (System.currentTimeMillis() - startTime) + " msec ] ");

System.out.println("Maximum sequences [ "

+ maxSeq

+ " ] have been generated. Total-Elapsed-Time [ "

+ (System.currentTimeMillis() - startTime) + " msec ]");

System.exit(1);

}

++totalCount;

}

/**

* Creates the log file.

* @param lastNo the last no

* @return the file

public static File createLogFile(int lastNo){

String fileName = "271962_primeseq_"+ lastNo + ".log";

File logFile = new File(fileName);

//Remove existing log file if exists

if(logFile.exists()){

logFile.delete();

}

FileAppender fa = new FileAppender();

fa.setName("FileOut");

fa.setFile(fileName);

fa.setLayout(patternLayout);

fa.setThreshold(Level.DEBUG);

fa.setAppend(true);

fa.activateOptions();

LOGGER.setAdditivity(false);

LOGGER.addAppender(fa);

return new File(fileName);

}

/**

* The main method.

* @param args the arguments

* @throws Exception the exception

public static void main(String[] args) throws Exception{

int lastNo = 0;

try{

// Fetch second argument - The number for which the sequence to be generated

lastNo = Integer.parseInt(args[1]);

// Create log file

File logFile = createLogFile(lastNo);

startTime = System.currentTimeMillis();

// Fetch first argument - This is log level flag. If 'true' write to log

// file, if false print to the console

String logFlag = args[0];

if ("true".equals(logFlag)){

System.out.println("\nPrime sequences are logged into file ["+logFile.getAbsolutePath()+"]");

loggingFlag = true;

}

else

loggingFlag = false;

// Fetch third argument - Maximum Sequences to be generated

if(args.length == 3){

maxSeq = Integer.parseInt(args[2]);

}

catch(Exception e ){

String errorMessage = "\nPlease provide the valid arguments \n"
System.out.println(errorMessage);

System.exit(1);

}

// Populate 2D array

prepareInputMatrix(lastNo);

// Find the sequences

findPrimeSequences();

long end = System.currentTimeMillis();

long totalTimeSpend = end - startTime;

if(!loggingFlag){

LOGGER.info("Since logging flag (first argument of the program) is false, sequences are printed to the console");

}

LOGGER.info("Total-Elapsed-Time [ " + totalTimeSpend

+ " msec ] and No. of total sequences found [ "

+ totalCount + " ]");

System.out.println("\nCongratulations !! Sequences generated successfully.");

System.out.println("\nTotal sequences generated [ "+ totalCount + " ], Elapsed Time [ " + totalTimeSpend + " msec ]");

}

How to Execute Program?

You can use below command to execute above program.

java PrimeSequenceSolution

First Argument: If 'true' sequences are written in log file, if 'false' print to the console
Second Argument : The number for which the sequence to be generated
Third Argument: Maximum sequences to be generated. This is optional. Default value is 1000000000

Example:

java PrimeSequenceSolution true 5

java PrimeSequenceSolution false 7

java PrimeSequenceSolution false 15 100000

Execution Statistics

Below are the captured execution statistics for above program.

Conclusion

Above approach is one of the solutions for given problem. I am sure there can be other approaches to solve this problem. If you notice the execution statistics, execution time is drastically increased if last number is increased. What about if program gets executed to get combinations for last number say 70, 80, 100 etc. This program will definitely take much time to get combinations or it may prompt out of memory error. Generating the expected result can be one of the aspect to solve a problem but efficiency is most important aspect that must not be ignored. Though, this solution gives the expected result but it may not be considered as efficient as it should be. This execution statistics left me thinking about what can be the other approaches? How can I make this program efficient? Can I think about parallel processing? etc.Well, I still need to get answers of my own questions. If you know the better solution; your comments are greatly appreciated.

Additionally, I tried to find out the real world problems related to this mathematical problem. TSP is one of the most famous problems. This is just to get into the real world problems. isn't it?

Setup Jenkins Continuous and Automatic Build System

2013-12-15T14:32:00.001+05:30

Jenkins is an open source tool which provides continuous integration services for software development. If you want to get more detail about Jenkins and it's history I would suggest refer this link. This post will help you installing and configuring Jenkins and creating jobs to trigger maven builds. Setting up Jenkins is not a rocket science indeed. In this post, I would like to concise the installation and configuration steps.

Installing Jenkins

Though, there are different ways to install Jenkins. But, in this post I will be describing Jenkins installation with Apache Tomcat server. Hence, please make sure that you have Apache Tomcat.

Download Jenkins war file from here and copy war into tomcat's 'webapp' folder
Start the tomcat server and access Jenkins

Jenkins installation is done. Wow, so easy. Isn't it?

Configuring Jenkins

Now you have Jenkins running with you. It's time to configure the project specific environment for continuous and automatic build. This section describes about JDK, Maven and Mail Notification configuration.

Go to 'Manage Jenkins'

Go to 'Manage Jenkins > Configure System'

Creating Job for Automatic and Continuous Build

So far you installed and configured the Jenkins. It's time to create new job. Before creating new job ensure that your project is a maven project. Now, go to 'New Job' and fill the required information. I believe, below screen shot tells everything which is required to configure a job. You just need to configure below points at minimum:

Source Code Management
Build Triggers
Build
Notifications

Once you save the job you will see below screen

Now click on 'Build Now' and see the 'Build History'. You will get new entry in 'Build History'. To see the build logs just hover the mouse on this build and click on 'Console Output', you can see the build logs.

Congratulations! you are done with setting up continuous build system.If you want to see that how effectively it works, just check-in any file in your project and see whether your build is triggered automatically. In case, build is failed with latest checked-in changes you must get a notification mail saying build is failed.

Last but not least, you should agree with my point 'Setting up Jenkins is not a rocket science indeed'. Shouldn't you? :) :) :)

CodeReviewer - Free Peer Code Review Tool

2013-12-07T16:29:00.002+05:30

Peer code review is an important activity to find and fix mistakes which are overlooked during development. It helps improving both software quality as well as developers skills. Though, it’s a good process for quality improvement. But this process becomes tedious if you need to share files and send review comments through mails, you need to organize formal meetings and you need to communicate peers who are in different time zone. Here is the point, why to do this manually? Why to follow this tedious process? There are so many automated code review tools which ensure fast, easy and collaborative code review process.

My team used to follow this tedious process manually. By following this process almost for one year we realized that we should go for automated code review tools. I explored free tool for code review. After evaluating so many tools I found pretty interesting tool called CodeReviewer. I would say this is an ‘Awesome’ tool with rich documentation and great community support.

CodeReviewer is free code review tool from SmartBear. This tool provides basic features which are required in code review process like Subversion integration, user administration, event based email notification, real time chat with peers etc. If you want advanced feature you can go for commercial version ‘CodeReviewer Pro’. The yearly price for ‘CodeReviewer Pro’ subscription is around 257$ per enabled user.

'CodeReviewer' and 'CodeReviewer Pro' Features Comparison Matrix

Installation and Configuration Insights

Installing CodeReviewer is very easy task. You just download the windows setup from here and install on your system by following this link. Once installation is done follow link1 and link2 to configure CodeReviewer.

Environment Specific Configuration

Once CodeReviewer is ready to use, now it’s time to configure Server URL, integrate mail server, integrate Subversion and configure users.

Review Life Cycle

As far as CodeReviewer’s life cycle concern, there are 5 phases of review life cycle. Below diagram describes each and every phase and there processing order.

How to Use CodeReviewer

You can start review by creating new review request. Note that you can't create review as admin user. You need to log-in as different user. If you want to upload file from Subversion you can follow this. Once the review request is created and sent for review, reviewers need to perform review and communicate with other participants who are participating in this review.

Questions and Answers

Which database is used in CodeReviewer by default?

The server installer comes with support for the hypersonic embedded database.

Can I integrate my own database with CodeReviewer?

Yes. MySQL Server can be integrated with CodeReviewer. For Oracle and SQL Server you need to go for Pro. Refer this link for more detail.

Can I integrate my own Bug Tracking system with CodeReviewer?

Code Reviewer can be integrated with external issue-tracking systems such as TeamTrack, Bugzilla, JIRA, and FogBugz. Refer this link for more detail.

Can I take CodeReviewer Backup?

Yes. Admin user can take the CodeReviewer backup by following 'System Status > Server Backup'. Refer this link for more detail.

How to restore backup?

CodeReviewer backup can be restored. Refer this link for more detail.

Others

You might face below issue while configuring SVN Repository.

Install the SVN client from here and configure svn.exe path in 'svn Executable' text box.

At last I would suggest you to have a look into this video. This tells about how to use CodeReviewer in Eclipse IDE itself.

WOW, I have very nice experience with CodeReviewer. I liked this one. You will like this.Won't you?

Hudson Plugin for Eclipse

2013-12-01T11:57:00.001+05:30

If you are using Hudson as continuous integration server and you might feel lazy about accessing Hudson explicitly to check the build status or checking Hudson build status mails, there is an option to monitor Hudson build and perform build activities in Eclipse IDE itself. This post describes about installing, configuring and using the Hudson in Eclipse IDE.

Installing Hudson Plugin in Eclipse

Use below URL To install the Hudson Eclipse plugin using Eclipse Update Manager:

http://hudson-eclipse.googlecode.com/svn/trunk/hudson-update/

In case you face issue while installing plugin using update manager you can directly drop dk.contix.eclipse.hudson_x.x.x.jar in Eclipse's plugins directory and restart IDE.

Configuring Hudson

Once the installation is done, configure the Hudson URL from Preferences > Hudson to connect to Hudson server. Also remember to Check URL whether URL is valid or not. Your Hudson server must be running at this point of time.

How to Use Hudson in Eclipse?

In eclipse, go to Windows > Show View > Hudson. You will see below tab is added in your Eclipse. Also, When the plugin is running, a health icon is displayed at the bottom of the Eclipse window. The icon is red on build failure and green on success. Please note that due to limitations in Eclipse, the Hudson view must be active before the icon is displayed.

In case build is failed, you will get below pop-up in your eclipse IDE.

You can also perform below activities using this Eclipse Hudson plugin.For this, select any project listed in Hudson tab and right click.

Schedule New Build
View Console Output
Open Hudson in Browser

If you want to access the Hudson admin console in your Eclipse, go to 'Hudson' tab and double click on your project.

Limitations

If your Hudson server requires authentication and authorization for access, then you can not see the list of jobs in Eclipse. Because Eclipse Hudson plugin does not facilitates interface to provide Hudson web credentials. To overcome this limitation, you have to give read permission to Anonymous user for jobs in Hudson which need to be displayed in Eclipse.

Common Issues

Issue in configuring Hudson base URL which is SSL enabled

Solution: This issue is due to missing required certificates. Refer below links to generate the certificate. Once the certificate is generated you can copy those into $JAVA_HOME/lib/security for the JRE that eclipse is using for its run time. It will trust that cert and then you can access configured HTTPS URL successfully.

Generating Certificate, Related Issue, Tool to generate certificate

Once your certificate issue is resolved then you may face below issue

java.security.cert.CertificateException: No subject alternative names present

Solution: Configure HTTPS URL using DNS rather giving IP address.

While refreshing build status you get below pop-up

Solution: Ensure that your job's name in Hudson does not contain white spaces.

References

Below are the references for more detail about Hudson Eclipse plugin.

Automatic Sequence Diagram Generation

2013-10-20T22:38:00.002+05:30

As a developer or architect, you always need to draw some sequence diagrams to demonstrate or document your functionality. And of course, if you do this manually you have to spare much time for this activity. Just think about, what If sequence diagram is generated automatically and you get it free of cost. Your reaction would be ‘wow’ this is great. But the next question will be ‘how’. So, I am publishing this post to describe ‘how’.There are different open source tools which help to generate UML sequence diagrams. ‘Java Call Trace’ is such a tool that can generate UML sequence diagram based on reverse engineering.

Install ‘Java Call Trace’

Download the java call trace from here.
Extract its contents at your desired location.

How to Use ‘Java Call Trace’ to Generate Sequence Diagram

To generate sequence diagram using Java call trace you have to follow below steps:

Define your filter in \Calltracer\filters.txt

This will generate sequence diagram only for the classes which are defined under given package. For example: com.mycom.example

Capture call trace (as xml) of your application/program

For JDK 1.5 and above, add the following JVM options and execute your application/program.

Syntax:

-agentpath:[full-path-to-folder-of-dll]\calltracer5.dll=traceFile-[trace-file],filterFile-[filter-file],filterList-[filter|filter|...],outputType-[xml or text],usage-[controlled, uncontrolled] -Djava.library.path=[full-path-to-folder-of-dll] -Dcalltracerlib=calltracer5

Example

-agentpath:C:\calltracer\jvmti\calltracer5.dll=traceFile-C:\calltracer\call.trace,filterFile-C:\calltracer\filters.txt,outputType-xml,usage-uncontrolled -Djava.library.path=C:\calltracer\jvmti -Dcalltracerlib=calltracer5

Generate sequence diagram using call trace captured in first step

Once you have generated a XML output using the ‘calltracer’ tool you can use the ‘Calltrace2Seq tool’ to generate UML sequence diagram.

\Calltrace2Seq\src\Calltrace2Seq.java INPUTXMLFILE-C:\\test.xml OUTPUTFOLDER-C:\\ OUTPUTFILENAME-mysequencediagram

INPUTXMLFILE: This is the generated xml location.

OUTPUTFILENAME: This is the location where generated sequence diagram image is copied.

Note: The output generated by the ‘calltracer’ tool has some header information that will not allow it (the output) to be used directly as an input to the ‘Calltrace2Seq’ tool. You can remove the header from the output xml file and then use this file as input argument for ‘Calltrace2Seq’.

Generated Sample Sequence Diagram

Below diagram is generated using Java Call Trace. There are three classes ManagedBean, Service and DataAccess. Where ManagedBean is having main method which is calling Service's method and Service is calling DataAccess's method.

Java Call Trace Details

If you are interested to get more details about Java Call Trace you can refer below:

javacalltracer\readme.txt

javacalltracer\calltracer\readme.txt

javacalltracer\Calltrace2Seq\readme.txt

Audit Logging using Hibernate Interceptor in JPA 2.0

2013-10-14T13:01:00.000+05:30

If you are using JPA 2.0 with Hibernate and you want to do audit logging from middle-ware itself, I believe you landed up on the exact place where you should be. You can try audit logging in your local environment by following this post.

Required JPA/Hibernate Maven Dependencies

<groupId>org.hibernate.java-persistence</groupId>

</dependency>

<groupId>org.hibernate</groupId>

<artifactId>hibernate-entitymanager</artifactId>

<version>3.6.4.Final</version>

</dependency>

<groupId>org.springframework</groupId>

<artifactId>spring-orm</artifactId>

<version>3.0.5.RELEASE</version>

</dependency>

JPA Configuration in Spring Application Context File

For JPA/Hibernate, you need to configure entity manager, transaction, data source and JPA vendor in your ‘applicationContext.xml’ file.

<bean id="dataSource"

class="org.springframework.jdbc.datasource.DriverManagerDataSource"

p:driverClassName="oracle.jdbc.driver.OracleDriver"

p:url="jdbc:oracle:thin:@127.0.0.1:1521/mydbservice "

p:username="nvuser" p:password="nvpass" />

<bean id="entityManagerFactory"

class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"

p:persistenceUnitName="MyPersistentUnit"

p:persistenceXmlLocation="classpath*:persistence.xml"

p:dataSource-ref="dataSource" p:jpaVendorAdapter-ref="jpaAdapter">

</property>

</bean>

<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager"

p:entityManagerFactory-ref="entityManagerFactory" />

<bean id="jpaAdapter"

class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"

p:database="ORACLE" p:showSql="true" />

Persistence.xml Configuration

To enable the hibernate audit logging you need to configure a property called ‘hibernate.ejb.interceptor’ in persistence.xml file and the value of this property should be the class that extends the Hibernate ‘EmptyInterceptor’.

xml version=”1.0” encoding=”UTF-8”?>

<persistence version="2.0"

xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">

<persistence-unit name=" MyPersistentUnit ">

//JPA Entity classes are configured here

<class>com.mycom.entities.User </class>

<shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>

<property name="hibernate.ejb.interceptor"

value=" com.xxx.entities.baseentity.AuditLogInterceptor" />

</properties>

</persistence-unit>

</persistence>

Audit Logging Interceptor Implementation

Hibernate provides an interceptor called ‘EmptyInterceptor’ with following methods.

onSave() – This method is called when an entity is saved, but the entity is not saved into database yet.
onFlushDirty() – This method is called when an entity is updated, the entity is not update into database yet.
onDelete () – This method is called when an entity is deleted , the entity is not deleted into database yet.
preFlush() – This method is called before the saved, updated or deleted entities are not committed to the database.
postFlush() – This method is called after the saved, updated or deleted entities are committed to database. This is called after preFlush() method.

To perform audit logging follow below steps:

Create data base table to capture audit data:

Audit [ Row_ID, Old_Value, New_Value, Column_Changed, Table_Changed,

Transaction_Id, User_ID]

Create JPA entity say ‘AuditTrail’ for above table
Create class ‘AuditLogInterceptor’ by extending Hibernate ‘EmptyInterceptor’.
Override interceptor methods in ‘AuditLogInterceptor’
Capture required information into AuditTrail entity and save entity

public class AuditLogInterceptor extends EmptyInterceptor {  
    
  private List auditTrailList = new ArrayList();    
   . . . . . .
 
   // First Parameter: The entity which is being updated     
   // Second Parameter: The primary key of updated row 
   // Third Parameter:  Current states of updated entity    
   // Fourth Parameter: Previous states of updated entity  
   // Fifth Parameter: The variable names of updated entity
   // Sixth Parameter: The type of variables of updated entity
   @Override
   public boolean onFlushDirty(Object entity, Serializable id,
            Object[] currentState, Object[] previousState,
            String[] propertyNames, Type[] types) {                     

           // Get the table name from entity

           Class entityClass = entity.getClass();

            Table tableAnnotation = entityClass.getAnnotation(Table.class);            

          

           for (int i = 0; i < currentState.length; i++) {              

                // Track changes only for the states which are String or Number types           

                 if(!(previousState[i] instanceof String

                     || previousState[i] instanceof Long

                     || previousState[i] instanceof BigDecimal

                      || previousState[i] instanceof Integer

                     || previousState[i] instanceof Timestamp))

                 {

                          continue;

                 }                

                // Check whether column value is updated

                 if(previousState[i] != null && currentState[i] != null

                   &&  !(previousState[i].equals(currentState[i]))){

                         

                  AuditTrail auditTrail = new AuditTrail();               

                 // Set updated table name

                 auditTrail.setTableName(tableAnnotation.name());

                 // Set updated column name

                 Column col = null;

                 try {                                                  

                        String filedName = propertyNames[i];

                        if(filedName.contains("serialVersionUID")){

                                  continue;

                        }

                        Character firstChar =  filedName.charAt(0);

                        firstChar = Character.toUpperCase(firstChar);

                        String filedNameSubStr = filedName.substring(1);

                        String methodName = "get"+ firstChar+filedNameSubStr;                       

                        Class[] parameterTypes = new Class[]{};                           

                        Method ueMethod = entityClass.getDeclaredMethod(methodName, parameterTypes);

                        col = ueMethod.getAnnotation(Column.class);                         

                    } catch (Exception e) {

                                e.printStackTrace();

                    }

                   if(col != null){

                          auditTrail.setColumnIdentifier(col.name());       

                   }               

                // Set old value

                if(previousState[i] != null){

                          auditTrail.setOldValue(previousState[i] == null  ? null :

                         previousState[i].toString());    

                }               

                // Set new value

                if(currentState[i] != null){

                          auditTrail.setNewValue(currentState[i] == null ? null:

                         currentState[i].toString());

                }               

                // Set database operation

                auditTrail.setOperation("Update");               

                // Set row primary key value

                auditTrail.setRowIdentifier(id.toString());  


                auditTrailList.add(auditTrail);                         


              }           

            }                        
        return false;
    }        
    @Override  
    public void postFlush(Iterator iterator) throws CallbackException {            
           // Set unique transaction id in all AuditTrail Entities
           String transId = UUID.randomUUID().toString();     

           for (AuditTrail auditTrail: auditTrailList) {
               auditTrail.setTransactionId(transId);
           }   
          // Save ‘AuditTrail’ entity list into database using AuditDAO 
    }  
 }

Audit logging Sequence Flow

Issue Faced during Audit Logging Implementation

I faced an interesting issue during audit logging implementation in my application. Let me share that issue detail.

I had ‘AbstractDAO’ and 'AuditDAO' class extends this ‘AbstractDAO’. 'AuditDAO' class is configured as ‘@Repository’. ‘AbstractDAO’ has entity manager property which is initialized by Spring Container. I wanted to auto wire ‘AuditDAO’ in ‘AuditLogInterceptor’ to save ‘AuditTrail’ entity. But, this couldn’t happen. The reason was ‘AuditLogInterceptor’ was not defined as spring resource (like @Component or @Resources or @Repository). Hence's its property can’t be auto wired. Since, this interceptor is initialized by hibernate, I couldn't configure this as spring resource. Finally, I had to take the help of StackOverflow and got the idea to solve this problem. You can refer this issue and solution from here.

Splunk - A Solution for Centralized Logging

2013-09-29T22:18:00.000+05:30

If any issue is observed in production, there are two major aspects related to providing the solution. First ‘how quick you can analyze the root cause’ and second ‘how quick you can fix the issue’. Story starts from analyzing the root cause. Unless, you find the root cause, you can’t even think about providing solution for that. Can you?

Now let’s think about actual production environment. There may be multiple JVMs where application is deployed. For the developer or support team, it’s very tedious to go to each box, download the log files and start analyzing the log files to know the root cause of the problem.

To get rid of this tedious task, ‘Splunk’ can help you. Splunk is a product that turns machine data into valuable insights. Splunk can index application logs at centralized location and provides rich user interface on top of indexed logs. With the help of this user interface you can look for data patterns that you might be interested in. Splunk is an Agent-Sever based platform where agents are responsible to collect and index any machine data from various resources at real time.

Licensing Aspects

Splunk charges it’s customer on the basis of how may GB data per day collected/indexed. If you want to try this and you download this first time you will get all of the Enterprise features of Splunk for 60 days and you can index up to 500 megabytes of data per day.

Features

Fast data search and analysis
Facilitates custom dashboards
Graphical representation
Access Control
Monitor and Alert
Distributed Search
Reports

Do you want to play with Splunk?

If yes, you can follow pretty simple step-by-step instructions from here to install it. First try to install it as ‘Local System User’. Once you install and log-in to the Splunk Web you will get below page:

Click on ‘Add Data’ link.

Click on ‘A file or directory of files’ link and provide your log file location. Once you provide your log file location and save data successfully you will see below page:

Now ready to search and click on ‘Start Searching’ link. In search box you can provide your data pattern to search in the log files. You can save your search pattern and result with actions provided with ‘Save’ button. Also, you can create reports and alerts using ‘Create’ button.

Isn't it so easy and interesting? Of course yes. I was really impressed when I tried in my local environment and used different features.

Alternative

There are many open sources in market which also provides centralized logging. For more detail refer this link.

ESB vs. EAI frameworks

2013-09-05T11:37:00.000+05:30

Few months back we had a debate about using Camel vs. Enterprise bus in a new project. I was on the camel side , I found hard to have an ESB just for integration and service chaining. With this blog post, based upon my understanding I will try to summarize when to use what.

Background
An existing software that has different independent modules needs to be redesigned.

It is almost a decade old and hasn't been through many design changes.
Business services are created on top of those modules.
It does not run in a managed environment and neither it is Java based.

The problem is to re-design mentioned system using Java technologies.While creating solutions using Java, much of the non functional requirements such as scalability, security, life-cycle management etc are leveraged from a container. Container can be bus or an application server.

How should we model and host those modules (the building blocks) as services and integrate them using bus ?

I was not clear why to use a bus just for integration. For integration there are other technologies like Camel and Spring-integration which are light weight and open sourced.

After having few discussions, we concluded to using a bus rather than going with a integration framework. The main point of choosing a bus was to create business proxies on top of those services.
Business proxies shall cover pre and post handling, assemble service invocation, compensate transaction failure and provide coarse grained business interface. It would be easy to deploy new business services on an ESB hosting the building blocks.

Let us run through an example to get into more details. Suppose we have a travel portal to build that allows complete holiday booking. It provides holiday packages that include flight, hotel, taxi, sight-seeing etc. This site serves as an agent that does reservation on your behalf.

Now, getting more into its working details. For booking a package it shall use booking and payment services provided by individual airline, taxi firms. Lets assume it books the ticket that is provided by Lufthansa airlines and rents a cab on Hertz. So behind the scenes it makes call to booking services of Lufthansa's and Hertz's software systems.It needs to handle scenario's such as, if booking in Hertz's system fails then cancel the booking in Lufthansa's system or retry. If the customer decides to cancel his booking then the portal needs to initiate cancellation in both the system, make sure that the booking amount gets remitted to the customer's account etc. In this kind of system one can see that it is not just integration but one needs to deal with post/pre processing before calling the involved systems, a lot of cases needs to be handled based upon the result of invoking a system. It's just not chaining of service calls. Using an ESB in this case seems far more appropriate than an integration framework.

While on the other hand integration frameworks would concentrate more on messaging passing, mediation and message routing between services. An important attribute of an integration framework is that it needs to be lightweight since it is involved in message passing. Things mentioned above would be difficult to plug-in between service chaining. Apart from that, functionalities provided by a container would have to be taken care of.

Esper EPL Modules

2013-06-29T16:35:00.002+05:30

The last time our team worked on Esper for complex event processing, it was version 3.4.0. One of the requirement we envisaged was for EPL statements to be externalized into configuration files rather than keeping them in code. So, we came up with this XML configuration file where one can configure EPLs, provide certain attributes (epl-name, enable/disable etc.) and associate them with a listener.

Working on Esper (v4.9.0) again and I met "EPL Modules", a very useful feature which renders our earlier custom-xml-configurations for epl almost useless. Though this feature was introduced long back in v3.5.0 itself, it is recently that came to know about it.

From the esper docs, "An EPL module file is a plain text file in which EPL statements appear separated by the semicolon(;) character"

The EPL statements specified here are named using @Name annotation. In the code, once the module is deployed (using EPDeploymentAdmin interfaces), the statements are looked-up via these names and then, listeners are attached. The module file allows "import"ing any Java class depdencies (for Event-Type definition or user-defined functions). as well as import constructs like Event-type definition, named-windows etc. from other modules.

The current Esper versions also include support for Testing out EPLs as well as debugging (@Audit).

To me, introduction of EPL modules is a very useful feature; however, it would have been more useful, if there were provisions for configuring the association between statements and listeners too in these modules itself.

Storm for CEP

2013-06-24T18:55:00.002+05:30

More than a year back, during some research related to CEP, I came across Storm which was "touted" as being a CEP engine and it was very difficult to come to terms with these assertions. Storm and S4 had just entered the market; for me, having some prior experience with ESPER, there was no comparision between ESPER and Storm/S4.

Reviewing Storm a year later now (Release 0.8.2) and it seems very mature, popular as well as very equipped to be used as a CEP engine. Definitely, even now there is no support for something like the powerful array of succinct ESPER EPL statements and patterns. However, to achieve similar goals, the Storm based Trident (again by Nathan Marz) aims to provide higher level abstraction (akin to DSL) over the core Storm APIs and ease creating event processing topology.

From the Trident tutorial: it allows you to seamlessly intermix high throughput (millions of messages per second), stateful stream processing with low latency distributed querying. Trident has joins, aggregations, grouping, functions, and filters. Trident adds primitives for doing stateful, incremental processing on top of any database or persistence store. Trident has consistent, exactly-once semantics.

The "Tick" time event stream introduced in recent Storm version caught my eyes for implementing time-window based constructs and I was fortunate to stumble upon a nice elaboration of one such Tick based implementation by Michael Noll. In this elaboration, Michael talks about implementing a system to identify trending topics in real-time. However looks like this Tick stream is not yet exposed in Trident.

There is also a Storm UI which provides useful diagnostics on the cluster and topologies. Check the "Storm UI improvements" section in the 0.8.2 release notes.

Coming to interfacing event-publishers with Storm, Abhishek has put up a blog entry explaining the push and pull style Storm spouts and concludes that the preferred choice with Storm is having a Pull based spout. Also, having a queue system between the event producer and the consumer spout adds to the system reliability. Only caveat with queue design is that queue might become the single point of failure; a distributed queue implementation shall help here (kafka?). From this dzone entry, looks like Brian too is content using Kafka + Storm.

In conclusion, Storm was already geared to provide the primitives for doing realtime computation in a way that's horizontally scalable and does not lose data (has inherent support for parallelization, fault-tolerance, guaranteeing message processing); offers high message throughput rates. One thing that it was missing a while back is higher level abstractions, but with the advent of Trident and the continual work on Storm/Trident, looks like this drawback is going away too. And even without Trident, it shall be possible to draw-up complex topologies using Storm's spouts & bolts.

Storm Spout : Push vs. Pull

2013-06-20T10:46:00.002+05:30

Since the day one when we started working on Storm, I was mistaken on spouts modus operandi. I believed spouts can both pull and push data from the sources. But when I was about to implement a push styled spout, I stumbled with a few challenges.

I wanted to build a thrift base spout such that different event sources can push data on it. I found one such implementation storm-scribe (https://github.com/stormprocessor/storm-scribe) and on its wiki it quotes this to be a push styled. This made me believed that one such spout can be created. Moving ahead I had below challenges to solve:

A question which pondered was how shall I make my spout visible to different sources? Since on storm topology spout can reside on any of node (one or all based on configuration). How'll sources get to know the addresses of running spouts, so that they can push event in my storm topology.
Will they lookup Zookeeper to identify where spouts are running? If yes, then each push call is going to be a 2 stepped process. Thereby effecting my sources performance..

I was really keen to know how such ‘Push-Styled’ spout can be designed. I asked this question on stackoverflow (http://stackoverflow.com/questions/16938184/push-styled-storm-spout) to get an expert opinion. Below is the comment on my question

Spouts are designed and intended to poll, so you can't push to them. However, what many people do is use things like Redis, Thrift, or Kafka as services that you can push messages to and then your spout can poll them.

Aha !! Ok this is how it works, this comment enlightened me :) and I had an insight of how this works.

I meet Saurabh next morning and tell him how can our push styled spout be designed. A thrift service (@ known location, so that source can push onto it) where all pushed events are captured in a datastore/in-memory/broker, paired with spouts which periodically polls this service for new events to be processed. Saurabh further looked into scribe spout to verify this, he confirmed me that it behaves exactly in a similar fashion.

Below are a few comments (by Saurabh) which emphaize on points to be considered while designing a Spout (migrating these from comments section to main post):

It’s recommended to create connections from spouts to message emitters, rather than the other way around. If the machine on which a spout is running goes down, Storm will restart it on another machine, so it’s easier for the spout to locate the message emitter than for the message emitter to keep track of which machine the spout is on.
To connect to unknown devices using a coordinating system to maintain the device list. The coordinator detects changes to the list and creates and destroys connections. For example, when collecting log files from web servers, the list of web servers may change over time. When a web server is added, the coordinator detects the change and creates a new spout for it.
The advantage of using a queue system is that it can serve as middleware between the spouts and data source; in many cases, you can use the queue to be reliable using the capability of replay messages of many queue systems. This means you don’t need to know anything about message emitters, and the process of adding and removing emitters will be easier than with direct connection. The problem with this architecture is that the queue will be your point of failure, and you’ll be adding a new layer to your processing flow.
Usually the spout pulls data from a data source like mq, rather than send the data to the spout. it's more convenient because the address the data source is fixed, while the spout's address is not fixed -- it may be started on any machine in the cluster.

Conclusion is ‘Storm spout’s modus operandi is Pull and not Push’ :)

Cassandra: Schema Disagreement Problem

2013-06-06T00:39:00.000+05:30

Yesterday our Cassandra development cluster broke down, Mahendra reported that on executing any statement on cassandra-cli, it errs prompting a weird message 'schema disagreement error' on console. I googled, my usual way of being :) and found this FAQ on Cassandra wiki. This clearly describes the problem's reason:

Prior to Cassandra 1.1 and 1.2, Cassandra schema updates assume that schema changes are done one-at-a-time. If you make multiple changes at the same time, you can cause some nodes to end up with a different schema, than others. (Before 0.7.6, this can also be caused by cluster system clocks being substantially out of sync with each other.)

Since we have installed Cassandra's 1.1+ version, I ruled out the above reasons to cause any harm to my cluster. Per my discussion with Mahendra, he encountered this problem after two new column family were added to this cluster and the DDL commands were fired from different cluster nodes. I had no clues of what went wrong.

Anyways I proceeded with the fix suggested in the same FAQ:

Identified which schemas are in the minority and mark down those IPs. Login to each of those machines and cleaninly stop the Cassandra service/process, typically by running:

nodetool disablethrift
nodetool disablegossip
nodetool drain
'sudo service cassandra stop' or 'kill '.

Removed the Schema* and Migration* sstables inside of your system keyspace

I was expecting cluster to start cleanly after following these steps, BUT it didn't. Cluster stumbled back on the same issue, schema disagreement. I revisited the FAQ and read this statement again, "Before 0.7.6, this can also be caused by cluster system clocks being substantially out of sync with each other". I doubted on the cluster's system clocks and found them to be out of sync. Just to give it a try since I had no other option to move ahead, I synced the clock on all the cluster node and repeated above steps again. This time I was sure that it is not going to work, as the docs clearly says this was issue with 0.7.6 and previous version. But again to my surprise cluster started cleanly and I could again execute commands on it.

This experience has left me with doubts on the FAQ correctness, I shall get into insight of this issue in days to come and keep the blog updated with my findings.

Cassandra: Hector Guava issue

2013-04-25T17:41:00.002+05:30

We faced a weird issue with Hector, one of the API which we developed to read data from cassandra crashed when we tried integrating DAO layer with other app dependencies. We were shocked and had no clues on what went wrong. Though the code we developed was thoroughly tested. Then why this error..?? Below is the trace which was observed:

Caused by: java.lang.NullPointerException at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
at com.google.common.collect.ImmutableClassToInstanceMap.getInstance(ImmutableClassToInstanceMap.java:147)
at me.prettyprint.hector.api.beans.AbstractComposite.serializerForComparator(AbstractComposite.java:319)
at me.prettyprint.hector.api.beans.AbstractComposite.getSerializer(AbstractComposite.java:342)
at me.prettyprint.hector.api.beans.AbstractComposite.deserialize(AbstractComposite.java:710)
at me.prettyprint.cassandra.serializers.CompositeSerializer.fromByteBuffer(CompositeSerializer.java:35)
at me.prettyprint.cassandra.serializers.CompositeSerializer.fromByteBuffer(CompositeSerializer.java:17)
at me.prettyprint.cassandra.model.HColumnImpl.getName(HColumnImpl.java:111)

When analyzed closely what we found was, Google Guava API was causing this issue. One of the other module of our application which explicitly uses Guava made resloved this dependency to a higher version with maven. This latest version was not compatible to Hector and thereby this error. If you are still reading this blog means you facing this issue as well and here the fix check dependency or the compatibility of Hector API which you are using.

Cassandra Composite: Generic Insert

2013-04-18T10:13:00.002+05:30

Cassandra composites as discussed in Datastax blog influenced us to adapt composite modelling in one of our pilot project. We used Hector APIs as the client library for this assignment. Below is the column family example which uses composite comparator type.

CREATE COLUMN FAMILY composite_column_family
WITH comparator = 'CompositeType(UTF8Type,UTF8Type,DateType,UTF8Type)'
AND key_validation_class = UTF8Type

Below is the generic insert method which one can use to insert in any column family leveraging composite column feature. One important point to note here is the default_validation_class is defaulted to byte type. Since we are unaware of what type of data shall be inserted in this column family and also not all columns can be of same data type, this has been defaulted to byte type.

public void insert(

String rowKey, String columnFamilyName,

Map columns,

Map columnserializer,

Map compositeColumn,

Map compositeColumnSerializer

) {

// create the composite using composite columns.

Composite composite = new Composite();

for(Object key : compositeColumn.keySet()){

Serializer sz = compositeColumnSerializer.get(key);

if(sz == null){

sz = stringSerializer;

}

composite.addComponent(compositeColumn.get(key), sz);

}

Mutator mutator = getHectorTemplate().createMutator();

// add component columns and values

for(Entry entry : columns.entrySet()) {

if(entry.getKey() != null && entry.getValue() != null){

Serializer sz = columnserializer.get(entry.getKey());

if(sz == null){

sz = stringSerializer;

}

composite.addComponent(entry.getKey(), stringSerializer);

HColumn col = hectorTemplate.createColumn(

composite,

entry.getValue(),

new Date().getTime(),

new CompositeSerializer(),

);

mutator.addInsertion(rowKey, columnFamilyName, col);

composite.remove(composite.size()-1);

}

MutationResult result = mutator.execute();

}

XML Databases

2013-04-11T16:26:00.002+05:30

Prologue

Rich media applications (RIA) have led to a tremendous acceptance of web applications. And along with this, instead of HTML travelling back and forth, XMLs are interchanged to communicate the information. Before the advent of JSON, XML was considered as the de-facto interchange language of the web.
Further, with growing acceptance of web services in the community, XML interchange is becoming very popular style of data interchange. Both styles of web service - RESTful and SOAP – use this markup language to model business objects or SOAP request/response. This interchange can represent business objects, entities, value objects (JEE Pattern), view objects or aggregates. Web tier maps these XMLs into respective business entities using technologies like JAXB. These are persisted in the database via technologies like ORM, JPA or EJB.
XMLs when stored in normalized (multiple) tables in database leads to some common observed problems.

Query Performance: Since related data is spread across different tables, querying or updating information comes with performance cost. Data lookup queries are modeled using costly SQL joins. Updates too require data entry/updates in multiple tables at a time. Even these queries may require joins and sub queries.
Maintenance Cycle: Applications today are developed using an iterative/incremental approach. Requirements and design keeps maturing with time. Adding/updating new fields in data layer normally affect all tier of application. Below is the list of components which may change:

View objects: logic for request/response serialization/deserialization
Entity bean: mapping view to entity
DAO query: query to fetch/update data to/from database
ORM layer changes
Table: table structure to accommodate added or removed fields. Also indexes and constraints need updates

Above clearly reflects a maintenance cost incurred with this common application architecture.

In this article, we shall discuss a new way of modeling the persistence tier using XML databases instead of traditional style of using RDBMS. Also, we will see resolutions to the above problems by following this novel approach.

XML Databases

Excerpt from wiki: An XML database is a data persistence software system that allows data to be stored in XML format. These data can then be queried, exported and serialized into the desired format. XML databases are usually associated with document-oriented databases.
Two major classes of XML database exist:

XML-enabled: these may either map XML to traditional database structures (such as a relational database), accepting XML as input and rendering XML as output, or more recently support native XML types within the traditional database. This term implies that the database processes the XML itself (as opposed to relying on middleware).

Native XML (NXD): the internal model of such databases depends on XML and uses XML documents as the fundamental unit of storage, which are, however, not necessarily stored in the form of text files.

Though most of the RDBMS are XML-enabled, but using them extensively for XML data shall not be a right choice. Instead NXDs are the one which we shall be interested in, since NXDs are designed and optimized for XML data, they can perform extremely well on XMLs. In this blog further we will focus on Native XML databases.

XML databases features

XQuery: XQuery has emerged as the standard interface to query and manipulate data in XMLs. It is designed to be a language in which queries are concise and easily understood. And also flexible enough to query a broad spectrum of XML information sources, including both databases and documents. Thereby most of the NXDs today support XQuery at core. XQuery can be considered as SQL for NXDs.
Support for other XML standards: These databases are build on top of well know XML standards. Below is the list of different XML databases, listing different XML standards they adhere to.

Above table is extracted from wiki

Indexing support: These databases support indexing XML document information at different levels viz. element, attribute, xpath or text levels. Full-text indexes are enabled on W3C specs. Provided XML is transformed into different native data-structures per the vendor designed, (discussed in section later). These helps in accessing large XMLs (size in GBs) in fraction of seconds.
REST support: Most of these databases exposes REST APIs which in turn transforms to xqueries to access the database. Also, these databases support database management via these APIs.Accessing these databases.
Others: Client server architecture, ACID safe transactions, user management, XSLT support are some of other salient features which are extensively supported.

Storage structures

The most stunning feature of NXDs is a quick XML lookup. As discussed in the previous section, these databases transform XMLs into native data structures which enable a quick look up on XML documents. Below are the different XML storage model extensively used:

pre/post encoding
pre/size/level
pre/dist/size

MonetDB is based on pre/post encoding storage model (for further reference have a look at this whitepaper, http://doc.utwente.nl/63574/1/db-utwente-0000003685.pdf), whereas BaseX database uses pre/dist/size model for storing XMLs.

Below diagram represents how XMLs are stored using pre/dist/size encoding scheme. Pre specifies the preorder traversal index of each node. Size represents the total number of descendants of any node. Dist represents the distance of each node from its parent. If we consider node C, pre order index will be 4. Total number of descendants of C is 6 (D,D,D,x,xyzw,xy) and thereby size is 6. And difference of C and A(parent) pre index is (4-1) 3, and hence dist of C is 3.

With all these heuristics stored, algorithms can traverse tree up and down very quickly.

Size: speed up child and descendant traversals
Distance: allowing access to the parents and ancestors of a node in constant time.

Casestudy

We carried out a case study to see performance of these NXDs, specifically we used BaseX for experiments.

Sample Database

The sample database consisted of order line items XML for a SaaS based order management system. Data was aggregated in two different ways:

One XML per order: All order-lines grouped together and merged into a single XML for a given order.
One XML per customer: All order-lines merged into an order and further all orders for a given customer are merged into a single XML.

These was done to evaluate whether these databases performs well with only small number of large XMLs or are good with large number of small XMLs as well.

Also, data was partitioned by customers, i.e. separate document database created for different customers with full text indexes created on all these databases.

Read Performance Observations

Below are the observed performance for query operation:

Faster access to both small and large customers.
All nodes of XML are searchable with almost same speed.
Performance almost same for both the modeling styles ( One XML per customer vs. order)
Distinct columns lookup is faster compared to non-distinct ones.
The more specific is the criteria, faster is the lookup i.e. when searching working telephone number with wildcard %-123-% shall be slow WRT to %1-123-2%.

Write Performance Observations

Below is the observed performance for insert operations with indexing.

With indexing enabled performance of application degrades with size.
High performing incremental processing can be performed using two database approach discussed here (https://mailman.uni-konstanz.de/pipermail/basex-talk/2012-September/003889.html)

Conclusion

One challenge that XML DBs face is from the parallel rise of popularity of JSON as an interchange format as well as research inclination towards JSON Document DBs. However, XML Databases are emerging as an interesting alternative to RDBMS. With increasing number of application using XML as interchange, use XML DB for persistence seems to a right choice with following benefits. As observed from the above numbers, read with full text index is always faster, while the write operation will need additional tuning. One of the features that developers should look at when choosing a XML DB is the support for distributed mode; not all NXDs today support it.

Low maintenance cost

When compared to RDBMs backed application, XML database backed applications shall incur a very low maintenance cost. It’s only the XQuery which need to be change.

No serialization/deserialization

Request responses can be stored as is in the databases. All business logic is now implemented in XQuery and not on top of entities. This removes an overhead of using another layer for transformation of XML to beans.

Faster

RDBMs based application extensively uses SQL joins and sub queries to lookup data. But with NXDs internal data structure XML document stored can be looked up in no time.

Enables real time analytics

Fast access to full text indexed XML obviates the need for separate data-warehouse or storage/processing mechanism for analytics purposes. XQuery provides extensive support on top of these DBs to aggregate data and generate statistical reports.

Full Text Search

2013-03-31T10:03:00.001+05:30

Searching documents, an headache

Today in organizations different documents are created to track processes, knowledge sharing, tracking facts and figures. I being a part of IT based organization generally design documents, software release notes, requirement documents, technical information bulletins, user guides etc. and these are stored in a source control repository. Various groups in our organization refer these documents for doing there day today jobs. With time when this has blown up as a separate concern for us to manage these document and how shall everyone be made aware of each document so that at time of need they can refer this. I was wondering if I could provide a search interface over this repository of database. Every one who wish to refer may look up these documents by some key words and search the contents of these document. Atleast this will make life simpler for a wider mass of people in my org.

One way to search these documents (rather than remembering their locations/paths) is to search for related keywords with in the document repository using tools like ‘Grep’ or ‘Search in Files’ available in many text editors. These tools perform well with character files, but if the repository contains binary files, which contains meta-data as well, like PDF or Microsoft office documents, these tools hit their limitation.
Secondly, if someone wants to search the documents on the basis of meta-data like author-name, modified-dates etc., then these tools are helpless again.
And thirdly, such tools read documents line by line i.e. via serial scanning; serial access is inherently slower approach and takes substantial time to return results.

If the number of documents to search is potentially large, then the preferred approach is using a full text search engine.

Full text search refers to techniques for searching a single computer-stored document or a collection in a full text database. Full text search operates via two broad tasks viz. indexing and searching. In the indexing process, scanning of all documents is performed and a list of search terms is created; this list is often called an index. The indexer will make an entry in the index for each term or word found in a document and possibly its relative position within the document. One advantage of using indexer is at search time, search engine can ignore stop words, such as the English "the, of, is, are” etc., which are both too common and carry too little meaning to be useful for searching. Other thing which can be achieved by indexers is to employ language-specific stemming on the words being indexed, for example the words "writes", "wrote", or "written" can be stored in the index by a single word "write".

In the search process, when we query with some keywords that need to be searched, the keywords are only searched in the index rather than the text of the original documents, which allows it to return results much faster.

This blog further compares different text search engines and discuss when one shall be preferred over other. Our choice of implementation was Apache Solr so we will discuss about this in details.

Basics: Apache Solr

Before understanding Solr, it would be good to know about the core of Apache Solr which is Apache Lucene. Lucene is a open source information retrieval library, originally created in Java by Doug Cutting and is released under the Apache Software License. The core of Lucene's logical architecture is the idea of a document containing fields of text. Lucene itself cannot extract text and meta-data from PDFs, HTML, Microsoft Word, and OpenDocument documents etc. But these all can be indexed by extracting their textual information by other libraries like Apache-Tika. Lucene itself is just an indexing and search library and does not contain crawling and HTML parsing functionality.

Why Solr?

The table below summarizes the comparison among various open source text search and related libraries.

	Written in	Platform	License	Developer(s)	Description
Apache Solr	Java	Cross-platform	Apache License 2.0	Apache Software Foundation	Solr is an open source enterprise search platform from the Apache Lucene project. Its major features include powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling. Providing distributed search and index replication, Solr is highly scalable
Apache Nutch	Java	Cross-platform	Apache License 2.0	Apache Software Foundation	Nutch is an effort to build an open source web search engine based on Lucene and Java for the search and index component
Compass	Java	Cross-platform	Apache License 2.0	Shay Banon	Compass provides a simple API for working with Lucene (similar to API provided by ORM libraries). Compass integrates with ORM frameworks allowing for a single API to index all the content mapped by both the ORM and Compass.
ElasticSearch	Java	Cross-platform	Apache License 2.0	Shay Banon	ElasticSearch is a distributed, RESTful, free/open source search server based on Apache Lucene. It is developed by Shay Banon and is released under the Apache Software License. ElasticSearch is developed in Java
Sphinx	C++	Linux, Windows, Solaris, FreeBSD, NetBSD, Mac OS, AIX	GPLv2 or proprietary	Andrew Aksyonoff	Sphinx is a free software search engine designed with indexing database content in mind. It currently supports MySQL, PostgreSQL, and ODBC-compliant databases as data sources natively.
Xapian	C++	Cross-platform	GNU General Public License	xapian.org	Xapian is an open source probabilistic information retrieval library, designed to be a highly adaptable toolkit to allow developers to easily add advanced indexing and search facilities to their own applications.
Recoll	C++ and Python	Unix-like	GPL	Jean-François Dockes	Recoll is a desktop search tool that provides efficient full text search (from single-word to arbitrarily complex Boolean searches) in a friendly GUI, with minimum technical sophistication and few mandatory external dependencies.

In the non-Java category, Sphinx search engine does indexing of content in database. It supports MySQL, PostgreSQL and ODBC-compliant databases as data source. Xapian and Recoll are basically desktop search tool that provide full text search for generally small applications.

In the Java category, Compass is Lucene based Full Text Search Engine, but has become deprecated and restructured with name ElasticSearch by the founder and main developer Shay Banon. Apache Nutch is basically an open source Web crawler, and build to find Web page hyperlinks in an automated manner. Apache Solr is an open source enterprise search platform based on the Lucene project. There is a close competition between Apache Solr and ElasticSearch. Both frameworks have their own features.

Features of Apache Solr-

Advanced Full-Text Search Capabilities
Optimized for High Volume Web Traffic
Standards Based Open Interfaces - XML,JSON and HTTP
Comprehensive HTML Administration Interfaces
Server statistics exposed over JMX for monitoring
Scalability - Efficient Replication to other Solr Search Servers
Flexible and Adaptable with XML configuration
Extensible Plugin Architecture

Features of ElasticSearch-

ElasticSearch is schema less, just toss it a typed JSON document and it will automatically index it
Lucene based query string or using an extensive JSON based search query DSL.
Elastic Search is its distributed nature. Indices are broken down into shards, each shard with 0 or more replicas.
ElasticSearch provides the gateway module which help to restore to the latest state of the cluster when it got crashed.
Easily handle multi-tenancy and able to search across multiple indices.
Almost everything is configurable and pluggable. Moreover, each index can have its own settings

Why we consider Apache Solr over ElasticSearch, below points summarizes our rationale:

Only one main developer behind ElasticSearch, while Solr is supported by Apache Software Foundation.
No auto warming feature Supported by ElasticSearch.
ElasticSearch has no convenient wrapper for Java beans as the @Field annotation in SolrJ.
ElasticSearch use parent/child feature instead of Solr's results grouping.
ElasticSearch only support JSON but no XML
ElasticSearch’s common container deployment (as a WAR file) is in development plugin

But there are cases when ElasticSearch must be considered over Apache Solr:

ElasticSearch is distributed. No separate project required. Replicas are near real-time too, which is called "Push replication".
Rebalancing and routing are done automatically and behind the scenes.
ElasticSearch fully supports the near real-time search of Apache Lucene.
Handling multi-tenancy is not a special configuration, where with Solr a more advanced setup is necessary.
ElasticSearch introduces the concept of the Gateway, which makes full backups easier.

It totally depends on the use case i.e. in which scenario we want to use these technologies. Per statistics provided by ‘Ryan Sonnek’ here, Solr is better when indexing process is in idle state and only search is being performed. But when index is being updated with new content concurrently with searching then ElasticSearch is much better than Apache Solr.

In our use-case, the documents are downloaded from the repository every midnight and an update-job updates index of search engine only with newly downloaded document. Here, documents are getting updated daily or weekly or quarterly basis i.e. frequency of updating document is low. We chose Apache Solr because in this scenario, indexing happens as an offline activity rather than happening at the same time as the search.

JPA 2.0 + Hibernate: OrphanRemoval & CascadeType.REMOVE

2013-03-23T16:28:00.003+05:30

Here I am writing to cover up the difference between OrphanRemoval=true and CascadeType.REMOVE in JPA2.0 with hibernate.

Let say there is one to many relationship between User and Address and that can be defined in JPA as below:

public class User{     
    @OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY,

        mappedBy = "user")
    public Set getAddresses() {
       return this.addresses;
    }
}

public class Address{
    @ManyToOne(fetch = FetchType.LAZY)
    @JoinColumn(name="userid") 
    public User getUser() {
       return this.user;
    }
}

You can have more detail in this post about above entities.

If I want to fetch the user entity and corresponding addresses, I can use below piece of code:

EntityManagerFactory emf

  = Persistence.createEntityManagerFactory("StateCityService");
EntityManager entityManager= emf.createEntityManager();

//fetching user entity based on id

User user = entityManager.find(User.class, userId);

// Lazy loading of addresses 
Set addresses = user.getAddresses();

Now if I change any state of user or address entity and execute merge operation on user entity, then corresponding User's and Address table's column value will be updated.

entityManager.merge(user);

That means, If I remove couple of address items from the set which user entity holds and perform merge operation on user entity, corresponding address rows (which have been removed from the collection defined in user entity) should be deleted based on CascadeType.ALL(add/remove/update). What do you think? You might answer yes. Even I think so.

But not really, CascadeType.REMOVE dose not help you in this scenario.

Achieving same thing in my project I invested couple of ours and found the solution. I had to use OrphanRemoval=true with one to many relationship to achieve this.

public class User{     
    @OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY,

       orphanRemoval=true, mappedBy = "user")

 public Set getAddresses() {
       return this.addresses;
    }
}

'OrphanRemoval=true' Vs 'CascadeType.REMOVE'

The difference between the two settings is in the response to removing child objects from the collection pointed by parent entity.

If orphanRemoval=true is specified the removed address instance is automatically removed. If only cascade=CascadeType.REMOVE is specified no automatic action is taken since removing a relationship is not a remove operation.

Where 'OrphanRemoval=true' and 'CascadeType.REMOVE' are identical ?

When an User entity object is removed the remove operation is cascaded to the referenced Address entity object. In this case, orphanRemoval=true and cascade=CascadeType.REMOVE are identical.

Use this link to have more detail.

Testing Syslog functionality inside Graylog2

2013-03-18T18:43:00.000+05:30

Having installed Graylog2 for centralized logging, we quickly wanted to test it's syslog capability. Configuring syslog daemon to redirect its log on remote server is easy to setup, but requires administrator privileges.

To quickly verify whether syslog daemon inside Graylog2 is working or not there is an easy way out. Just take any of the log files present in syslog format present inside /var/log directory and use netcat to redirect the contents to graylog2 daemon listening for syslog messages.

Tried the following command, after copying the cron log (cron.1) into current working directory :

Note :

Our graylog2 was configured to listen on 33000 port.
Cron was configured to log using syslog

nc -u 127.0.0.1 33000 < cron.1

The above command sends the file over UDP on a given host and port combination.

To our surprise we did'nt see any logging being shown on the graylog2 web console. Tried this many times but could not figure out what was going wrong, since syslog messages are accepted over UDP protocol and it being connection-less further makes difficult to debug.
After giving up, posted the problem on stackoverflow. Few days went by and someone pointed out the mistake that we were doing. What nc did was that it transferred the complete log file to the syslog daemon, the daemon did receive the contents and when it went to parse the complete file content it failed. The error was obvious since the complete content at the graylog2 was being understood as one log statement.

Now we tried to only have one single log statement in the above file (cron.1) and test.
Again this did not work. We quickly checked the logs of graylog2 and again it failed to parse. Did a little googling and found a useful library syslog4j for logging syslog messages through java, wrote a test program using that and checked the console.

import org.productivity.java.syslog4j.*;

public class SyslogClient {
public static void main(String[] args)
{
SyslogIF syslog = Syslog.getInstance("udp");
syslog.getConfig().setHost("192.168.64.133");
syslog.getConfig().setPort(33000);
syslog.critical("Log Message at critical priority");
}
}

Voila it worked ! , the same library is also being used by graylog2 for syslog functionality.

The console showed the following log

Now realized that the message did not had the priority field included and hence graylog2 failed to parse.

Now tried the following command and it worked without any problem

echo "<10>Mar 31 22:44:09 This message was sent to local7.emerg (0)" > /dev/udp/192.168.64.133/33000

JPA 2.0 Criteria Query with Hibernate

2013-03-16T14:38:00.000+05:30

Criteria Query has been introduced in JPA 2.0. With the help of criteria queries you can write your queries in a type-safe way. Before criteria queries, developers had to write queries through the construction of object-based query definitions. There might be chances of incorrect syntax while building the queries. The criteria query API provides feature to create structured and nested queries with compile time safety. Getting into more theory about criteria query may not be a good idea as there are plethora of pages can be found on net for same. Let's have a simple example to understand query using criteria query. In this post, I used hibernate as vendor with JPA 2.0 to show the example of Criteria Query.

Before going into deep about criteria query, let's think about database tables and corresponding java classes which can be represented as entities:

Database Tables:

For example we have below two database tables:

State [stateId, stateName]
City [cityId, stateId(FK#), CityName]

We want to fetch list of cities for a particular state where city name should be started from 'M' and city list should be in ascending order. If we think about simple native SQL, it would be like below:

Select * from City c, State s where c.stateId = s.stateId and c.stateId = ? and c.cityName like "M%" order by c.cityName

JPA Entities:

Below are the two java classes configured as JPA entities to represent state and city tables. These can be generated using reverse engineering tool as well.

Java entity for STATE table

import javax.persistence.CascadeType;

import javax.persistence.Column;

import javax.persistence.Entity;

import javax.persistence.FetchType;

import javax.persistence.GeneratedValue;

import javax.persistence.GenerationType;

import javax.persistence.Id;

import javax.persistence.OneToMany;

import javax.persistence.Table;

import java.util.Date;

import java.util.Set;

import javax.persistence.SequenceGenerator;

@Entity

@Table(name="STATE")

@SequenceGenerator(sequenceName="STATE_SEQ",name="STATE_SEQ_GEN")

public class State {

private Long stateId;

private String stateName;

private Set citySet;

@Id

@Column(name="stateId")

@GeneratedValue(generator="STATE_SEQ_GEN",strategy=GenerationType.SEQUENCE)

public Long getStateId;() {

return stateId;

}

public void setId(long stateId) {

stateId = stateId;

}

@Column(name="stateName")

public String getStateName() {

return stateName;

}

public void setStateName(String stateName) {

this.stateName = stateName;

}

public void setCitySet(Set citySet)

this.citySet= citySet;

}

@OneToMany(fetch=FetchType.LAZY,cascade=CascadeType.ALL,mappedBy="state")

public Set getCitySet() {

return citySet;

}

Java entity for CITY table

import javax.persistence.Column;

import javax.persistence.Entity;

import javax.persistence.FetchType;

import javax.persistence.GeneratedValue;

import javax.persistence.GenerationType;

import javax.persistence.Id;

import javax.persistence.JoinColumn;

import javax.persistence.ManyToOne;

import javax.persistence.SequenceGenerator;

import javax.persistence.Table;

@Entity

@Table(name="CITY")

@SequenceGenerator(sequenceName="CITY_SEQ",name="CITY_SEQ_GEN")

public class City {

private Long cityId;

private String cityName;

private State state;

public void setId(long cityId) {

this.cityId = cityId;

}

@Id

@Column(name="CityId")

@GeneratedValue(generator="CITY_SEQ_GEN",strategy=GenerationType.SEQUENCE)

public Long getCityId() {

return cityId;

}

@Column(name="CityName")

public void setContent(String cityName) {

this.cityName = cityName;

}

public String getCityName() {

return cityName;

}

public void setState(State state) {

this.state = state;

}

@ManyToOne(fetch = FetchType.LAZY)

@JoinColumn(name = "STATEID", nullable = false)

public state getState() {

return state;

}

Value Objects (POJO):

Below are the two value objects. With the help of criteria query you can directly map your value objects from result data. You don't need to write the code to copy the result data from entity classes to your values objects. This is really an exciting feature in criteria query.

public class StateVO {

private Long stateId;

private String stateName;

private Set cityVOSet;

// All getter setters

}

public class CityVO {

private Long cityId;

private String cityName;

private StateVO stateVO;

public CityVO( Long cityId, String cityName){

this.cityId=cicityId;

this.cityName=cityName;

}

// All getter setters

}

DAO Implementation:

Now this is the time where we can use criteria query to fetch the data. We will expose a method which will take input argument as StateVO and will return list of CityVOs.

import javax.persistence.EntityManager;

import javax.persistence.EntityManagerFactory;

import javax.persistence.Persistence;

public class StateCityDAOImpl{

public List getCityList(StateVO searchStateVO) {

// Get the entity manager instance

EntityManagerFactory emf = Persistence.createEntityManagerFactory("StateCityService");

EntityManager entityManager= emf.createEntityManager();

// Get the criteria builder instance from entity manager

final CriteriaBuilder criteriaBuilder = entityManager.getCriteriaBuilder();

// Create criteria query and pass the value object which needs to be populated as result

final CriteriaQuery criteriaQuery = criteriaBuilder.createQuery(CityVO.class);

// Tell to criteria query which tables/entities you want to fetch

// To join the City and State tables, we need to write below code

final Root stateRoot = criteriaQuery.from(State.class);

final Root cityRoot = criteriaQuery.from(City.class);

// Time to define where clause in terms of Predicates

// This list will contain all Predicates (where clauses)

List criteriaList = new ArrayList();

// Note: Ensure that whatever string you are passing in root variables

// It should be matched with variables' name in entity classes

// [1] where condition: State.StateId = City.StateId

Predicate predicate1 = criteriaBuilder.equal(

cityRoot. get("state"). get("stateId"),

stateRoot. get("stateId"));

criteriaList.add(predicate1);

// [2] where condition: City.StateId = ?

if (searchStateVO.getStateId() != null) {

Predicate predicate2= criteriaBuilder.equal(

cityRoot. get("state"). get("stateId"),

searchStateVO.getStateId());

criteriaList.add(predicate2);

}

// [3] where condition: City.cityName like 'M%'

Predicate predicate3

= criteriaBuilder.like(criteriaBuilder.upper(cityRoot. get("cityName")),

"M%");

criteriaList.add(predicate3);

// This statement maps your CityVO with result data

// You have to have a custom constructor in CityVO (see above) to populate the result data

criteriaQuery.select(criteriaBuilder.construct(CityVO.class, cityRoot. get("cityId"),

cityRoot. get("cityName")));

// Pass the criteria list to the where method of criteria query

criteriaQuery.where(criteriaBuilder.and(criteriaList.toArray(new Predicate[0])));

// Order by clause based on city names

criteriaQuery.orderBy(criteriaBuilder.asc(cityRoot. get("cityName")));

// Here entity manager will create actual SQL query out of criteria query

final TypedQuery query = entityManager.createQuery(criteriaQuery);

// This code can be used to define the row range to fetch the result

/* if (CitySize != 0) {

query.setFirstResult(startIndex); // starting row index

query.setMaxResults(endIndex); // end row index

}*/

return query.getResultList();

}

Now we have CityVO list as answer in your hand. We can access the corresponding city name based on our aforementioned requirement.

Though, writing queries using criteria query may be little bit clumsy but once you get handy, you will enjoy the Criteria Query.

HDFS to Mongo migration

2013-02-05T09:53:00.000+05:30

One of our current assignment demanded to migrate data from HDFS to Mongo database. Data contained in HDFS was in JSON format and this was a plus since Mongo explicitly support JSON documents.

I started looking out for strategies how shall this migration be executed. The very first thought was to use Hadoop MapRed task to read from HDFS and insert it to Mongo. I was looking explicilty at Spring-Data and Spring-Data-Mongo framework to get this done.

Varun, one of my team member, suggested if MongoImport can be used to get this done. I though of giving it a try and started looking at different options available with this.

Linux pipes and redirection does the trick, without using --file parameter with mongoimport output from HDFS could be redirected to mongoimport command. The command looked as below:

hadoop dfs -text HDFSPath| mongoimport --host hostname --port configuredPort --collection collectionName --db databaseName

I am not sure whether this is the optimal way to get this done, but at least this has let us move forward for now and we are moving ahead with this.

Liferay + Spring MVC Portlet

2013-02-03T12:44:00.001+05:30

Liferay is one of the portal frameworks based on java. You can create portlets in Liferay using Spring MVC framework. This post may give you the answer of your one question "How to create a portlet in Liferay using Spring MVC ? ". You can setup Liferay from here .

Let's think about a user interface where search contents can be added into a search repository. Let's play with Liferay and Spring MVC Portlet to provide this user interface and feature. We can have a portlet named 'Add Search Content Portlet' as a user interface to add the search contents into a search repository.

Add Search Content Portlet: This portlet can have three fields to add the search contents: ID, Title and Description. If you see in below screen shots, I have added three hotel names as search content in terms of ID (hotel id), Title (hotel name) and Description (about hotel).

Steps for Portlet Implementation

To implement a portlet in Liferay using Spring MVC you need to follow below steps. Source code can be referred from document.

1. Configuring Spring context file, this is a core configuration in applicationContext.xml file where you need to define the message bundle, view resolver, portlet resolver etc. [refer (a) in document]

2. Adding configuration in portlet.xml, liferay-display.xml and liferay-portlet.xml. These are the liferay specific configuration files, where you need to define the portlet name and display name, portlet specific configuration file like addSearchContent.xml etc. [refer (b), (c) and (d) in document]

3. Configuring portlet specific configuration file like addSearchContent.xml, where you define the controller bean which serves the request for this portlet. [refer (e) in document]

4. Creating java value objects, these objects are used to map JSP filed values into java object. [refer (f) in document]

5. Defining the controller class, this is a controller class where actual request is served for this portlet. [refer (g) in document]

6. Creating JSP, where portlet UI is defined. [refer (h) in document]

After creating all above source files you can build your application and then deploy on application server. Access your Liferay server, add a test page and add your new portlet on test page. You can refer this link to add new portlet into your page.

Refer below links for more detail:

Liferay, Spring MVC Portlet

Sharing Client Time-Zone information with Server

2013-01-02T15:30:00.001+05:30

As it happens, there are many instances that one comes across while programming client side of a web-applications where one has to make a trade-off between the so called ‘best-practices’ and the current scope of change in the application. This is more evident when the changes are required to be made in large enterprise web-applications where the scope of making a change to the code is restricted by the maintenance cycles and release deadlines. Although not the most difficult one, but managing timezone information between the client (read browser) and server can be a tricky task. One way of doing it can be that the server sends across UTC time values to the client GUI and the client takes the responsibility to render that date in the client’s native timezone.

But, there are instances when there is an application that is built without the timezone perspective in mind. And in order to render correct dates on the client GUI the dates need to be handled and timezone adjusted by the server. This becomes more critical if the server itself is running with assumptions that the client is running in same timezone as server.

What is all this about?

This article aims at providing an approach (apart from many other possible approaches out there) to provide a client’s timezone specific details to the server so that the server can use it to further process requests. Please note that there are many assumptions to the applicability of this approach. Most important thing is that the server is just using this information to process data on per-request basis and not storing it anywhere for later usage. This approach is more of a ‘hack’ for quick fix to situations that are too big to handle conventionally.

Simply put, custom JavaScript can be used to collect timezone information that the client machine is running in and pass this information to the server using conventional methods (as form data etc.). And at the server end the Java SimpleTimeZone class can be used to create a custom or mimicking timezone object that roughly follows the same rules as the actual Java timezone object for the given client for a given year.

On the Server Side

SimpleTimeZone Class allows creating custom Java TimeZone Objects complete with DST rules and timezone offset values. To Quote Java (version-6) documentation summary:

“ SimpleTimeZone is a concrete subclass of TimeZone that represents a time zone for use with a Gregorian calendar. The class holds an offset from GMT, called raw offset, and start and end rules for a daylight saving time schedule. ”

Hence, creating a Java SimpleTimeZone object provided sufficient details are at hand, SimpleTimeZone can allow replication of actual client timezone behavior on the server-side.

The following table clearly lays out the details for a SimpleTimeZone object constructor that can be used. Following are also the parameters that shall be provided by client side JavaScript.

SimpleTimeZone Constructor Details

SimpleTimeZone Property

Description

`rawOffset`

Raw offset of the timezone (in milliseconds) as compared to GMT (e.g. IST has +0530 as offset that means it is 5 hours and 30 minutes ahead of GMT hence the rawOffset value should be +18000000). Please note that a negative value shall signify that the given timezone is ‘behind’ GMT.

`ID`

String Identifier for identifying the new timezone instance

`startMonth`

The month value in which the day-light savings starts (e.g. 3 for April)

`startDay`

Day-of-month on which day-light savings starts

`startDayOfWeek`

Day-of-week is negative of the value of the day of week on which the day-light savings starts (should be ‘0’ in case the exact startDay value is being used)

`startTime`

Time in milliseconds representing the hours of day when the day-light savings starts

`endMonth`

The month value in which the day-light savings starts (e.g. 10 for November)

`endDay`

Day-of-month on which the daylight savings ends

`endDayOfWeek`

Day-of-week is value of the day of week on which the day-light savings ends (should be ‘0’ in case the exact endDay value is being used)

`endTime`

Time in milliseconds representing the hours of day when the day-light savings starts

`dstSavings`

Day-Light Savings in milliseconds

All set, but there are a few points to be noted here. Firstly, the rawOffset: The value for rawOffset needs to be the exact time difference that the client’s timezone is from the GMT(or UTC). Also, the sign of the offset value determines which side of GMT the client timezone reside. Namely, a negative value signifies that the client timezone is behind the GMT timezone and a positive value signifies that the client timezone is as many hours ahead of GMT. (It is worth a note that JavaScript uses negative sign to denote timezones that are ahead of GMT and positive sign to denote timezone as behind GMT.)

Secondly, when using startDay and endDay values (since we are) the startDayOfWeek and endDayOfWeek should be zero. This will create a SimpleTimeZone object that shall mimic the actual client timezone object for that current year (i.e. when working in tandem with the JavaScript provided values mentioned below.)

Also, the ID attribute is a String identifier for the custom timezone object that is created. This can be any String value to differentiate it from other timezone objects.

The Client Side

On client side JavaScript code can find the details of the timezone information about the timezone. This JavaScript code works in tandem with the server end piece and provides the server end with all the needed information to create a SimpleTimeZone object. The following is a JavaScript algorithm to extract timezone information from the browser. (Since the JavaScript method date.getTimezoneOffset() yields just the numeric offset of the client’s timezone which might correspond to many different timezone following different rules. For example GMT-0700 can correspond to US/Mountain (which follows daylight savings) and at the same time it can also mean US/Arizona time (which does not follow daylight savings). Hence the simple timezoneOffset cannot provide directly useful information). This is the case also because the JavaScript date object’s String representation is in consistent across the various browsers.

Algorithm 1.1

`CalculateDstSwitchDates( year ){`

`1. Loop through every month of the current year`

      `1.1 Fetch the timezone OFFSET value for 1^st date of each month`

      `1.2 Capture when a OFFSET change occurs`

      `1.3 (occurs twice let us call them firstSwitchMonth and secondSwitchMonth)`

          `firstSwitchMonth – OFFSET INCREASES (DST turns ON)`

          `secondSwitchMonth – OFFSET DECREASES (DST turns OFF)`

   `End Loop`

`2. Go figure out date/time up to a minutes the DST adjustment occurs:`

      `2.1 Call FindDstSwitchDate(year, firstSwitchMonth) to get firstSwitchDate`

      `2.2 Call FindDstSwitchDate(year, secondSwitchMonth) to get secondSwitchDate`



`3. Fetch the Timezone offset of time between first and second switch. This shall be the standard timezone offset or the rawOffset in our case.`

`E.g. US eastern: standard time = between November (DST OFF) and March (DST ON)`

`4. Calculate the modulus of difference between offset calculated at firstSwitch and secondSwitch for the same year (or the hours shift during FirstSwitch or SecondSwitch). This shall become the dstSavings.`

`5. Put all the values firstSwitchDate, secondSwitchDate, rawOffset, dstSavings in an array and return this array`

`}`

`function FindDstSwitchDate(year, month){`



      `1. Set the starting date`



      `2. Loop to find the exact day a timezone adjust occurs`

            `2.1 Check if the timezone changed from one day to the next`

            `2.2 Back-up one day and grab the offset`

            `2.3 Count the minutes until a timezone change occurs`

                  `2.3.1 Determine the exact minute a timezone change occurs`

                  `2.3.2 Back-up a minute to get the date/time just before a timezone change occurs`

                `End Count`



      `3. Add a month (for display) since JavaScript counts months from 0 to 11`

      `4. Prepend preceding zeros to the month (single digit months may cause problems)`



      `5. Add the numeric values for day and year`

      `6. Use the above captured values of day, month, year, hours and minutes to create a Date object and return it`



`}`

`Please Note: You may find the source code for the above algorithm at [1]as mentioned in the References section. Also, one more point to be noted is that you shall need to handle cases where the client timezone does not follow DST Savings.`

The DST determination is split between two functions. The CalculateDstSwitchDates() function locates the initial date/time for each month before a timezone change occurs. This method internally calls the FindDstSwitchDate() method, which in turn determines the exact minute before a timezone change occurs.

To elaborate more on the code, let's first take a look at the CalculateDstSwitchDates() function. This function is very simple. Its only purpose is to get the date/time from 12 consecutive months and compare the timezone offset month to month. When the timezone changes, the month prior to the current date/time checked is captured. Typically, there are two times the timezone will change if the code is running on a browser running in a geographical location where DST is observed. The for loop completes once the month counter (variable i) reaches 12. With the current year and the two month numbers captured, the FindDstSwitchDate() function is called.

The FindDstSwitchDate() function may appear big, but it is actually very simple. The first part of the function starts checking the timezone offset day by day, beginning with the month and year supplied as function parameters. Once the timezone offset changes, the day prior to the timezone offset change is captured. The rest of the function counts minute by minute, beginning with the day captured, until the timezone offset changes. The minute prior to the actual date/time that the timezone offset changes is captured. Using the captured day and number of minutes, the function assembles a string to represent the date/time that is one minute prior to when a timezone offset change occurs. This function is called twice for locations where DST is observed. One date represents a minute before when you turn the clock back an hour, and the other date represents a minute before when you move the clock ahead by an hour.

Once the DST specific values for the client timezone have been accumulated using JavaScript, we can send these across to the corresponding server resource as POST data from where the processing can be picked up in the manner as mentioned above in the server end section.

After one has the TimeZone object at the server end one can simply use the SimpleDateFormat class to parse the date between various timezones.

References:

[1] http://www.codeproject.com/Articles/58728/JavaScript-code-to-determine-when-DayLight-Savings

[2] http://docs.oracle.com/javase/6/docs/api/java/util/SimpleTimeZone.html

SimpleTimeZone Property	Description
`rawOffset`	Raw offset of the timezone (in milliseconds) as compared to GMT (e.g. IST has +0530 as offset that means it is 5 hours and 30 minutes ahead of GMT hence the rawOffset value should be +18000000). Please note that a negative value shall signify that the given timezone is ‘behind’ GMT.
`ID`	String Identifier for identifying the new timezone instance
`startMonth`	The month value in which the day-light savings starts (e.g. 3 for April)
`startDay`	Day-of-month on which day-light savings starts
`startDayOfWeek`	Day-of-week is negative of the value of the day of week on which the day-light savings starts (should be ‘0’ in case the exact startDay value is being used)
`startTime`	Time in milliseconds representing the hours of day when the day-light savings starts
`endMonth`	The month value in which the day-light savings starts (e.g. 10 for November)
`endDay`	Day-of-month on which the daylight savings ends
`endDayOfWeek`	Day-of-week is value of the day of week on which the day-light savings ends (should be ‘0’ in case the exact endDay value is being used)
`endTime`	Time in milliseconds representing the hours of day when the day-light savings starts
`dstSavings`	Day-Light Savings in milliseconds