> My Space, My Voice ***

HTTP Caching with ESI

noreply@blogger.com (Nads) — Thu, 17 Mar 2016 20:00:00 GMT

In general, the idea behind caching is to improve web performance by delivering content more quickly and to reduce the load on the origin servers.

Caching static content, such as images, JavaScript and CSS files, and web content that rarely changes is a relatively straightforward process. Cache updates can be handled by regular timeouts or conditional gets etc.

Caching personalised content is generally not possible, because the server’s response to each request for the same resource is different. Techniques such as server side includes (SSI) and edge side includes (ESI) can help to assemble the page.

Lets take a deeper look at ESI and see how it works -
By definition, ESI is a markup language for edge level dynamic web content assembly. The purpose of ESI is to tackle the problem of web infrastructure scaling.

ESI is implemented by different content delivery networks such as Akamai, and by some caching proxy servers such as Varnish. Akamai also adds additional features to the version they support.

So how ESI is implemented?
ESI element tags are inserted into HTML or other text based content during creation. Instead of being displayed to viewers, these ESI tags are directives that instruct an ESI processor to take some action. The XML based ESI tags indicate to the edge-side processing agent the action that needs to be taken to complete the page's assembly.

One simple example of an ESI element is the include tag which is used to include content external to the page. An ESI include tag placed in-line within an HTML document would look like this:

<esi:include src="http://yoursite.com/page1.html" alt="http://beta.yoursite.com/page2.html" onerror="continue"> </esi:include>

In this case the ESI processor would retrieve the src URL, or failing that the alt URL. The ESI system is usually a caching proxy server so it may have a local copy of these files which it can insert without going back to the server. Alternatively, the whole page with the ESI tags may be cached, and only the ESI requests may be made to the origin server. This allows different caching times for different parts of the page, or different degrees of personalisation.

There are four main features in ESI -

Inclusion of page fragments
Variables which can be set from cookies or HTTP headers and then used in other ESI statements or written into markup
Conditions so that different markup can be used based on variables, for example if a cookie is set or not
Error handling for failover if an origin server is unavailable.

Which CDNs support ESI?

ESI is currently supported by CDNs like Akamai, Fastly, CloudFlare and caching proxy servers like Varnish and Squid. Although many do not implement the complete specification, Akamai also adds additional features to the version they support.

Reference and next steps -

Extending and Developing Cartridges in Endeca 11.x

noreply@blogger.com (Nads) — Wed, 24 Sep 2014 23:02:00 +0100

In this post I have tried to explore Cartridges and Endeca Assembler Application by examining how they work together in a "Hello World" example cartridge.

So without any further delays let’s first understand what is cartridge , cartridge template , cartridge handlers and the structure of a cartridge before making our own custom Cartridges. Further we will also take a close look at Endeca assembler application to understand what it does under the hood.

Cartridges and Cartridge Templates –

A cartridge is a content item with a specific role in your application; for example, a cartridge can map to a GUI component in the front-end application. The Assembler includes a number of cartridges that map to typical

GUI components – for example, a Breadcrumbs cartridge, a Search Box cartridge, and a Results List cartridge.

You can create other cartridges that map to other GUI components expected by your business users.

Every cartridge is defined by a template. A cartridge template defines:

· The structure and initial configuration for a content item.

· A set of configurable properties and the associated editors with which the business user can configure them.

Experience Manager instantiates each content item from its cartridge template. This includes any configuration

made by the business user, and results in a content item with instance configuration that is passed to the Assembler.

Cartridge Handlers -

A cartridge handler takes a content item as input, processes it, and returns a content item as output.

The input content item typically includes instance configuration, which consists of any properties specified by a business user using the Experience Manager or Rule Manager tool in Endeca Workbench. The content item

is typically initialized by layering configuration from other sources: your application may include default values, or URL parameters that represent end user selections in the front-end application.

A cartridge handler can optionally perform further processing, such as querying a search engine for data. When processing is finished, the handler returns a completed content item to the application.

Note: Not all cartridges require cartridge handlers. In the case of a content item with no associated cartridge handler, the Assembler returns the unmodified content item.

Cartridge structure -

The template contains two main sections: the <ContentItem> element and the <EditorPanel> element.

The content item is a core concept in Assembler applications that can represent both the configuration model for a cartridge and the response model that the Assembler returns to the client application. A content item is a map of properties, or key-value pairs. The <ContentItem> element in the template defines the prototypical content item and its properties, similar to a class or type definition.

In our example (explained below) template, we defined two string properties named message and messageColor and attached two simple string editors to those properties. The result looks like this in Experience Manager:

A brief note on Endeca Assembler Application -

The Endeca assembler application enables a WEB application to query the MDEX engine and retrieve the appropriate dynamic content based on user navigation state.

The assembler application provides a RESTful web service API that returns results either in JSON or XML format.

What happens at runtime?

The business user creates and configures instances of cartridges in Experience Manager based on a template. During cartridge development you need to create at least one instance of a cartridge for testing.

The Assembler retrieves this configuration at runtime and uses it to build the response model that it returns to the client application.

For any given cartridge, the default behavior is for the Assembler to do no processing on the configuration and simply return the configuration content item as a map of properties. That is, the response object is the same as the configuration object unless specific processing logic is defined in the Assembler for that cartridge.

Cartridge creation workflow –

The high-level workflow for creating a basic cartridge is:

1. Create a cartridge template and upload it to Endeca Workbench.

2. Use Experience Manager to create and configure an instance of the cartridge.

3. Add a renderer to the front-end application.

Step 2 is necessary during development in order to have a cartridge instance with which to test. However, once the cartridge is complete, the business user is typically responsible for creating and maintaining cartridge instances in Experience Manager.

In the below sections, we'll see each of these elements of the cartridge in detail.

Hello World cartridge example –

Here we will define a new cartridge and use Workbench to configure it to appear on a page.

Follow these steps to create and configure a basic "Hello World" cartridge.

1. Navigate to the templates directory of your application (Discovery in our case), and create a subdirectory named "HelloWorld." This directory name is the template ID for your template.

For example: D:\Endeca\apps\Discover\config\import\templates\HelloWorld

2. Create a cartridge template - copy the following into the contents of the file.

Save the file with the name template.xml in the HelloWorld directory which we just created in point 1 above.

<ContentTemplate xmlns="http://endeca.com/schema/content-template/2008"

xmlns:editors="editors" type="SecondaryContent">

<Description>A sample cartridge that can display a simple message.</Description>

<ThumbnailUrl>/ifcr/tools/xmgr/img/template_thumbnails/sidebar_content.jpg</ThumbnailUrl>

<Name>Hello cartridge</Name>

</Property>

</Property>

</ContentItem>

<editors:StringEditor propertyName="message" label="Message"/>

<editors:StringEditor propertyName="messageColor"

label="Color"/>

</BasicContentItemEditor>

</EditorPanel>

</ContentTemplate>

3.   Upload the template to Endeca Workbench –
·  Open a command prompt and navigate to the control directory of your deployed application, for example, D:\Endeca\apps\Discover\control
·  Run the set_templates command

4. Add the cartridge to a page –

· Open Endeca Workbench in a Web browser.

The default URL for Workbench is http://localhost:8006. The default Username is admin and the default Password is admin

· From the launch page, select Experience Manager

· In the tree on the left, select Search and Navigation Pages under the Content section, then select the Default Page

· In the Edit Pane on the right, select the right column section from the Content Tree in the bottom left

· Click Add.

The cartridge selector dialog displays

· Select the HelloWorld cartridge and click OK

· Select the new Hello cartridge from the Content Tree on the left and configure it as shown

· Click Save Changes in the upper right of the page.

5. Try to view the cartridge in the Discover Electronics application.

In a Web browser, navigate to http://localhost:8006/discover-authoring/

The error displays because we have not yet created a renderer for the Hello cartridge.

Scroll down to the bottom of the page and click the json link to view the serialized Assembler response model that represents the current page.

Oracle recommends that you use a browser or install a plugin that supports native JSON display.

Otherwise, you can download the JSON response as a file.

Alternatively, you can click the xml link to view the same response in XML. In this article, we use the

JSON format when examining the Assembler response.

The following shows the JSON representation of the page with most of the tree collapsed, highlighting the data for the cartridge that we just added.

{

"@type": "ResultsPageSlot",

"name": "Browse Page",

"contentCollection": "Search And Navigation Pages",

"ruleLimit": "1",

"contents": [

{

"@type": "ThreeColumnNavigationPage",

"name": "Default Page",

"title": "Discover Electronics",

"metaKeywords": "camera cameras electronics",

"metaDescription": "Endeca eBusiness reference application.",

"links": [ ],

"header": [ ... ],

"leftColumn": [ ... ],

"main": [ ... ],

"rightColumn": [

{ ... },

{

"@type": "Hello",

"name": "Hello cartridge",

"message": "Hello",

"messageColor": "#FF0000"

}

]

}

...

}

In the next section, we'll create a simple renderer that displays the message based on the values configured in Experience Manager.

Adding a basic renderer -

While there is no one way to write rendering code for an application, in this example we'll write a simple JSP renderer for our basic cartridge.

To write a basic "Hello, World" renderer:

1. Create a new JSP page (Hello.jsp) and type or copy the following:

<%@page language="java" pageEncoding="UTF-8" contentType="text/html;charset=UTF-8"%>

<%@include file="/WEB-INF/views/include.jsp"%>

<div style="font-size: 150%;

color: ${component.messageColor}">${component.message}

</div>

2. Save the above renderer to

D:\Endeca\ToolsAndFrameworks\11.1.0\reference\discover-electronics-authoring\WEB-INF\views\desktop\Hello\Hello.jsp

(You need to create “Hello” folder)

3. Refresh the Discover Electronics authoring application at

http://localhost:8006/discover-authoring/ to see the end result :-)

In future posts I would love to explore Optimizing Application URLs and Integrating with the Sitemap Generator. All the above concepts and much more is covered in Endeca Assembler Application Developer's Guide.

Endeca commerce guided search essentials

noreply@blogger.com (Nads) — Sat, 20 Sep 2014 22:42:00 +0100

Time to explore barebone concepts around Endeca commerce guided search v11.1. It’s vital to understand the key concepts before moving ahead with some of the complex areas of Endeca search. So lets dive in -

Endeca commerce key components -

Oracle Endeca Commerce is comprised of three major components. These components are:

Endeca ITL (Information Transformation Layer)

Endeca MDEX Engine

Endeca Application Tier

Endeca Information Transformation Layer (ITL)

Reads your raw source data and manipulates it into a set of Oracle Endeca MDEX Engine indexes.
The ITL consists of the Content Acquisition System (which includes the Endeca CAS Server and Console, the CAS API and the Endeca Web Crawler), and the Data Foundry (which includes data-manipulation programs such as Forge).

Endeca MDEX Engine

MDEX engine is the query engine that is the core of guided search. MDEX engine consists of the Indexer (Dgidx), Dgraph and Agraph.
The MDEX Engine loads the indexes generated by the indexing component of the Endeca Information Transformation Layer.
Although the Indexer (also known as Dgidx) is installed as part of the MDEX Engine package, in effect it is part of the ITL process.

Endeca Application Tier

After the indexes are loaded, the MDEX Engine receives queries from the Endeca Application Tier, executes them against the loaded indexes, and returns the results to the client application.
The Application Tier provides an interface to the MDEX Engine via the Endeca Assembler. The Assembler acts as a language-agnostic interface for aggregating and sending queries to the MDEX Engine, and executing any necessary post-processing on the results.

Note: The ITL components are run offline at intervals that are appropriate for your business requirements.

The MDEX Engine and Endeca Application Tier are both online processes; that is, they must remain running as long as you want clients to have access to your data set.

After knowing the main components now lets go through some of the key terminologies around guided search which we will encounter throughout our Endeca experience. Endeca records, dimensions, and properties store and organize product information, making it accessible to customers through your Guided Search applications.

Endeca Records -

Endeca records are the elements of your data set that users navigate to or search for.

Records are the fundamental units of data. Endeca records are based on traditional records in a source database. Source database records typically contain information such as the bottles of wine in a wine store, the customer records in a CRM application, or the mutual funds in a fund evaluator.
Source database records store this information in one or more key/value pairs, known as properties.
A single Endeca record could correspond to any number of source records. For example, suppose that four different source records refer to the same book in different formats: hardcover, paperback, large print, and audio.

Endeca Properties -

Endeca properties are the basic attributes of an endeca record.
Are usually generated from a record’s source properties, using source property mapping.
Consist of key/value pairs (property name/property value).
Can be searched and displayed.

Endeca properties often contain more specific information about a record than dimensions. For example, a Price Range dimension is useful for navigation — give me all the bottles of wine that cost between $10 and $20 dollars — but it is the exact price of each bottle that you want to see when looking at the individual records. A common implementation for this type of application uses a Price Range dimension for navigation and a Price property that is displayed when a bottle’s record has been located.

Note: Property, dimension, and dimension value names are case sensitive.

Endeca Queries -

Oracle Endeca Commerce uses two types of queries: navigation queries and keyword search queries.
Navigation queries return a set of records based on application-defined record characteristics (such as wine type or region in an online wine store), plus any follow-on query information.
Keyword search queries return a set of records or dimensions based on a user-defined keyword, plus any follow-on query information.

Navigation queries and keyword search queries are complementary. In fact, a keyword search query is a specialized form of navigation query, and the data structures for the results of the two queries are identical: a set of records and follow-on query information.

Users can execute a combination of navigation queries and keyword search queries to navigate to their desired record set in the way that works best for them.

Dimensions and dimension values -

Dimensions are logical categories that make it possible to organize your Endeca records into a hierarchical structure that customers can search.
A dimension is a collection of related dimension values, organized into a tree. The top-most dimension value in a dimension tree is known as the dimension root. A dimension root always has the same name as its dimension.
Each dimension value can have one or more child dimension values; a dimension value with child dimension values is known as a parent dimension value. A child dimension value can have only one parent dimension value. Dimension values that are children of the same parent dimension value are known as sibling dimension values. The dimension values that have no children are known as leaf dimension values.

Dimension hierarchy -

Dimension hierarchy gives you additional control over the logical structure used to organize your Endeca records.
As the term "dimension tree" implies, dimension values can have parent and child dimension values.
A dimension value that has sub-dimension values is the parent of those sub-dimension values. The sub-dimension values themselves are children or child dimension values.
Child dimension values of the same parent dimension at the same level of hierarchy are dimension value siblings.

The following figure illustrates a typical dimension :

Flat Dimensions -

Dimensions that have only one level of hierarchy beneath the dimension root are called flat dimensions

The one parent rule -

It is possible for a dimension value to be simultaneously a child of one dimension value and the parent of other dimension values.

Ancestors -

The ancestors of a given dimension value are all the dimension values between the given dimension value and the dimension root. The parent of the given dimension value is also one of its ancestors.

In the example below, Other and Fortified represent the ancestors for the Sherry dimension value.

Advantages of dimension hierarchy -

You can design dimension hierarchies in ways that reduce the number of follow-on queries that are presented to users as they navigate.

For example, in the flat dimension below, a navigation query on the Wine Type dimension value would return six possible refinement queries, one for each child.

A simple flat dimension such as the one shown in the preceding figure is small enough to navigate through quickly. Larger flat dimensions, however, present too many choices for follow-on queries to be easily navigable.

Organizing data as dimensions reduces this information overload and provides for an easier, more intuitive navigation experience. In the hierarchical example below, the Wine Type dimension value has only three possible refinement queries: Red, White, or Sparkling.

A second reason to use dimension hierarchy is that, by limiting the number of refinement queries, you decrease the amount of time it takes for the MDEX Engine to return its results. Returning query results that contain follow-on information for 20 refinement queries is faster than returning query results that contain information for 2000 refinement queries.

Note: The time that the MDEX Engine takes to process a large flat dimension can be lessened by requesting that the MDEX Engine return only the most important refinements.

After going through above concepts we should next look at Guided Navigation and Using Keyword Search.

Oracle ATG commerce integration with Endeca search

noreply@blogger.com (Nads) — Wed, 10 Sep 2014 01:10:00 +0100

Exploring new software and tool is always exciting and fun way to learn new stuff. With ATG Commerce v11 Oracle has replaced ATG Search with Endeca search. I have tried to list out all the steps involved in this integration and focused to highlight the issues/errors I faced with their resolutions. So lets dive in -

Installation requirements with download links –

a. Download and install Java 7 or higher, I am using jdk 7 update 45 , set the environment variable ( JAVA_HOME = D:\Java\jdk1.7.0_45 )

b. Install application server of your choice – I have used JBoss Enterprise Application Platform - Version 6.1.0.GA , you can Download from Jboss site.

c. Installing Oracle XE 11g - Download from Oracle website and accept all the default settings.

After you install Oracle XE then create following schemas which will be used at the time of ATG configuration via CIM.

create user crs_switching_a identified by crs_switching_a;

grant connect, resource, dba to crs_switching_a;

create user crs_switching_b identified by crs_switching_b;

grant connect, resource , dba to crs_switching_b;

create user crs_publishing identified by crs_publishing;

grant connect, resource,dba to crs_publishing;

create user crs_production identified by crs_production;

grant connect, resource,dba to crs_production;

commit;

You also need JDBC driver jar , this will be needed later when we do Database configuration in CIM. You can download the jar from Oracle site. I am using ojdbc6.jar , you can use higher versions as well.

d. Go to https://edelivery.oracle.com and download ATG frameworks and Endeca frameworks (version 11.1)

Downloads for ATG –

Oracle Commerce Platform 11.1

Oracle Commerce ReferenceStore 11.1

Downloads for Endeca search –

Oracle Endeca MDEX Engine 6.5.1

Oracle Endeca Platform Services 11.1.0

Oracle Endeca Tools and Frameworks 11.1.0

Oracle Endeca Content Acquisition System 11.1.0

Installing Endeca Commerce with Experience Manager -

Goto http://docs.oracle.com/cd/E51272_02/Common.110/pdf/GettingStarted.pdf and follow instructions given in chapter 3 and chapter 5 in this guide for Endeca installation and configuration.

Please note we are installing Endeca first as its not dependent on ATG installation. We want to make sure Endeca is working fine before moving to ATG installation and configuration.

You can avoid Developer studio installation as its not needed for this integration.

After a successful installation you can see following services installed. Make sure all 3 Endeca services are started as shown below.

You will also see following folders in your d:\Endeca folder after all Endeca related installation.

Time to validate our Endeca installation – If you have followed all the instructions then your Discover Electronics reference is ready to validate.

Visit - http://localhost:8006/discover/ and you should see the below page

Note: To see workbench screen goto http://localhost:8006/

Installing ATG commerce and reference st -
Now we need to install ATG platform first and then reference store . Visit this link http://docs.oracle.com/cd/E41069_01/CRS.11-0/ATGCRSInstall/html/s0101introduction01.html for step by step installation guide.

During installation setup will ask for Jboss and Java path – Specifiy your local folder path where you have installed them. Please note the port I have used here is 8180.

Just to mention, I have avoided using 8080 as Oracle XE is listening to this port so avoid using.

I am using following ports throughout all installations and configurations here.

8180 for Store

8280 for Publishing

8380 for SSO

Dont forget to set environment variable DYNAMO_HOME = D:\ATG\ATG11.1\home

Configuring ATG environment using CIM -

Once ATG platform and CRS installation is completed in step # 3 then we need to launch CIM. Open a command prompt , navigate to D:\ATG\ATG11.1\home\bin and run cim.bat

Here mainly you are going to configure following items – I suggest you follow CIM response given in any of the two links provided in the Reference section below. It has all the detailed CIM responses.

Set the Administrator Password

Product Selection – select 9 as it will select all essentials items to install. further in AddOns choose *** Single Sign On (SSO) ***

Database Configuration

Configure OPSS Security

Server Instance Configuration - After server instance configuration , make sure you also visit *** Deploy CRS Endeca Application *** option. Also make sure you don’t get any error in Deploy and Initialize tasks. If Initialize fails that means there is something wrong in your Endeca installation done in step #2. Refer below link directly only for Endeca application configuration in CIM - http://docs.oracle.com/cd/E41069_01/CRS.11-0/ATGCRSInstall/html/s0207creatingtheendecaapplication01.html

Application Assembly and Deployment

After all the above configurations we are left with following –

a. Start all the server instance like store , publishing and SSO. Use below commands

D:\jboss-eap-6.1\bin>standalone.bat --server-config=ATGProduction.xml -b 0.0.0.0

D:\jboss-eap-6.1\bin>standalone.bat --server-config=ATGPublishing.xml -b 0.0.0.0

D:\jboss-eap-6.1\bin>standalone.bat --server-config=ATGSSO.xml -b 0.0.0.0

b. You need to perform a full deployment in BCC – follow direct link for this task here - http://docs.oracle.com/cd/E41069_01/CRS.11-0/ATGCRSInstall/html/s0213configuringandrunningafulldeploy01.html

If you face issues in accessing BCC then check below troubleshoot section for help.

c. Ideally after successful full deployment the system should build the index. To verify goto http://localhost:8280/dyn/admin/nucleus/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin

and check the indexing job status. If its in PENDING or CANCELLED state then you need to manually invoke “Baseline Index”

d. Once index is completed we finally need to promote the commerce reference store content . please follow instructions here to invoke promote_content - http://docs.oracle.com/cd/E41069_01/CRS.11-0/ATGCRSInstall/html/s0215promotingthecommercereferencesto01.html

e. Its time now to finally hit the store URL and explore CRS store - http://localhost:8180/crs/storeus

Common errors troubleshoot –

If you get following error while accessing BCC then read solution –

Error – /atg/portal/framework/PortalObjectResolver No root community folder for portal 'default'

Solutions – Its due to missing BIZUI data, follow below url and see the last section about

accessing BCC. http://atgkid.blogspot.co.uk/2012_01_01_archive.html

You may encounter ORA-01658 error -

Error - ORA-01658: unable to create INITIAL extent for segment in tablespace LARGE_
Solution - Its due to the fact that Oracle failed to find sufficient contiguous space to allocate

INITIAL extent for segment being created. Visit this link for solution -

If you encounter any connection related error -

Error - Caused by: java.sql.SQLException: Listener refused the connection with the following
error: ORA-12519, TNS:no appropriate service handler found

Solution - I increased timeout setting in the pulishing server configuration, also make sure your

DB is up and running.

You may not be able to login to BCC until you login to SSO (http://localhost:8380/sso/login) . Make sure you run the SSO instance like you start your store or publishing servers.

References -

a. http://iam-saminda.blogspot.co.uk/2013/06/oracle-ecommerce-atg-102.html - One of the best written blog which details ATG 10 integration with Endeca search.

b. http://internetmarketing-readme.pricemaniacs.com/oracle-commerce-v11-step-by-step-cim-responses - Blog lists all the CIM response for ATG environment configuration using CIM.

Bonus -

> How to check the status of the Endeca Log Server -
Visit - http://localhost:15010/stats

If the Log Server is running, above URL returns a confirmation message containing the file name, number of log entries and number of errors.
If it is not running, you will see browser’s default error message.

OUTPUT -
Endeca log server is running.
log file: D:\Endeca\apps\Discover\logs\logserver_output\LogServer.2014_09_10.17_45_40
number of log entries: 9
number of errors: 0

> Anatomy of an Oracle Commerce Experience Manager Site

Emerging languages series - Scala

noreply@blogger.com (Nads) — Sun, 8 Dec 2013 15:00:00 GMT

Although my experience in Scala language is limited but in last couple of weeks have read a lot about Scala and its growth over recent years. Coming from Java background and recent demand for Scala has forced me to dig deeper to explore the potential and features of the language.

Scala overview –

Scala is an acronym for “Scalable Language”. Scala is a statically typed, object oriented functional language, merging two popular programming approaches into one

In other words we can say when Object-Oriented meets Functional then the result is known as Scala. It was built by Martin Odersky at EPFL, launched in 2003.

Scala runs on the JVM. Java and Scala classes can be freely mixed, no matter whether they reside in different projects or in the same.

Java libraries, frameworks and tools are all available. Build tools like ant or maven, IDEs like Eclipse, IntelliJ, or Netbeans, frameworks like Spring or Hibernate all work seamlessly with Scala. Scala runs on all common JVMs and also on Android.

Already used by Twitter, LinkedIn, Yammer, Foursquare, eBay, Amazon , Guardian newspaper and many others

Key features of Scala language – Scala has a set of features which differ from Java. Some of these are:

All types are objects

Type inference

Functions are objects

Anonymous function

Curried function

Closures

Traits

Domain specific language (DSL) support

Concurrency support inspired by Erlang

Scala comes with an interactive interpreter, the REPL (for Read, Evaluate, Print Loop). After installing Scala, you can start the REPL by entering the command scala in a command prompt or shell window. Visit the Scala download page here.

You can type statements and expressions at the prompt, and the REPL will evaluate these and print the results. The REPL is a useful tool for experimenting with Scala language features. The REPL understands a number of special commands that start with a colon. Type the command :help to see an overview. Type :quit to exit the REPL.

Compiling and running Scala code –

Before getting into any example please remember you can Compile Scala source files using the Scala compiler, scalac. When you use Scala as a compiled language your code must be organized a little differently than when using it as a scripting language. You’ll need to create a Scala object with a main method, as in the following example.

// HelloWorld.scala

object HelloWorld {

def main(args: Array[String]) {

println(“Hello World!”)

}

Save this in a source file named HelloWorld.scala. Compile it with: scalac HelloWorld.scala and run it with: scala HelloWorld. Note that in this last command, you supply the name of the object (HelloWorld) rather than the name of the source file (HelloWorld.scala) or the compiled class file (HelloWorld.class).

Scala features By Example –

Values and variables -

val name: Type = expression // use to define an immutable value

var name: Type = expression // use to define a mutable variable

val myVal : Int = 1;

var name = "Sajid"

Tuples -

Tuples are type-safe containers for multiple values. They useful if you want to return more than one value from a method. They are different from collections;

each element in a tuple has its own type, whereas in collections all elements in the collection usually have the same type.

// Defines a tuple with a String and an Int

val tup = (“Hello”, 123)

// The elements of a tuple are named _1, _2 etc.

println(tup._1)

println(tup._2)

// Method that returns a Tuple with two Ints

def div(a: Int, b: Int): (Int, Int) = (a / b, a % b)

// Call the method, use pattern matching to extract the values from the result, throw away the second value

val (x, _) = div(20, 7)

Methods

def name[TypeParams](ParamList1)(ParamList2)(...): ReturnType = {

// Statements and expressions

}

// Here you see return type is inferred and “return” keyword is optional

def add(x:Int, y:Int) = {

x + y

}

println(add(42,13))

Default parameters

Parameters may have a default value that is used if a value is not supplied when calling the method.

def greet(name: String = “there”) {

println(“Hello “ + name)

}

Return multiple variables - It is possible to return multiple variables using Tuple

def swap(x:String, y:String) = (y, x)

val (a,b) = swap("hello","world")

println(a, b)

Functions –

The difference between a function and a method is that a function is a value (an object) while a method is not. A function is an instance of one of the traits Function0, Function1 etc., depending on the number of parameters that the function takes.

def add(a:Int, b:Int):Int = a + b

val m:Int = add(1, 2)

println(m)

You can convert a method to a function by assigning it to a val or var. However, to avoid ambiguity, you have to either explicitly specify the type of the val or var, or put an underscore after the method name to indicate that you want to treat it as a function.

// A method

def succ1(x: Int) = x + 1

// Explicitly specify the type of succ2

val succ2: Int => Int = succ1

// Or put an underscore after the method name

val succ2 = succ1 _

Higher order functions –

def identity(x: Int) = x

def sqr(x: Int) = x * x

def cube(x: Int) = x * x * x

def sum(f: Int=>Int, a: Int, b: Int): Int =

if (a == b) f(a) else f(a) + sum(f, a + 1, b)

println(sum(identity, 1, 10))

println(sum(sqr, 1, 10))

println(sum(cube, 1, 10))

"sum" is now a "higher order" function, It's first parameter is a function which maps an Int to an Int

The type "Int" represents a simple Integer value. The type Int => Int represents a function which accepts an Int and returns an Int.

Anonymous functions –

A method that requires a function as a parameter the function's type is (Int,Int) => Int

def sum(f: Int=>Int, a: Int, b: Int): Int =

if (a == b) f(a) else f(a) + sum(f, a + 1, b)

println(sum(x=>x, 1, 3))

println(sum(x=>x*x, 1, 3))

println(sum(x=>x*x*x, 1, 3))

We can create "anonymous" functions on the fly. x => x*x is a function which takes an "x" and returns x*x

Curried functions –

Currying allows to turn a function that expects two arguments into a function that expects only one, and that function returns a function that expects the second argument. Creating basically a chain of functions.

def addA(x: Int, y: Int): Int =

x + y

def addB(x: Int):Int=>Int =

y => x + y

val a = addA(10, 20)

val b = addB(10)(20)

println(a)

println(b)

You can take any function of multiple arguments and curry it.

Traits –

trait Name[TypeParams] extends ClassOrTrait with Trait1 with Trait2 with ... {

// Constructor statements and trait members

}

A trait can extend one class and multiple other traits. A trait can have a constructor (you can put statements in the body of the trait that are executed as part of its constructor), but it cannot have constructor parameters.

Traits are like Java interfaces, except that a trait can be partially implemented. Like a Java interface, a trait cannot be directly instantiated even if it is fully implemented.

trait Car {

val brand: String

}

trait Shiny {

val shineRefraction: Int

}

One class can extend several traits using the with keyword:

class BMW extends Car with Shiny {

val brand = "BMW"

val shineRefraction = 12

}

Closure –

A "closure" is a function which carries with it references to the environment in which it was defined.

Alternately we can say that closure is a function, whose return value depends on the value of one or more variables declared outside this function. Consider the following piece of code with anonymous function

object Test {

def main(args: Array[String]) {

println( "muliplier(1) value = " + multiplier(1) )

println( "muliplier(2) value = " + multiplier(2) )

}

var factor = 3

val multiplier = (i:Int) => i * factor

}

Here factor has a reference to a variable outside the function but in the enclosing scope.

Functions are Objects – Yes you read it correct J

Probably it might be little difficult to digest it if you coming from Java background. Don’t worry, here comes a golden rule of thumb. The key to scala is that everything is an object. Since every value is an Object in Scala, Functions are Objects, too!

Every Function you’ll ever define in Scala, will become an instance of an Implementation which will feature a certain Function Trait. There is a whole bunch of that Function Traits, ranging from Function1 up to Function22. Wow, and why are there so many? Since Functions are Objects in Scala and Scala is a statically typed language, it has to provide an appropriate type for every Function which comes with a different number of arguments. If you define a Function with two arguments, the compiler picks Function2 as the underlying type. If you define a Function with seven arguments, you’ll end up with Function7. And since there is a maximum up to Function22, that means you can’t define a Function which will accept 23 arguments or more.

new Function1[Int, Int] {

def apply(x: Int): Int =

x + 1

}

Patten matching –

Scala has a built-in general pattern matching mechanism. It allows to match on any sort of data with a first-match policy.

Here is a small example which shows how to match against an integer value:

object MatchTest1 extends Application {

def matchTest(x: Int): String = x match {

case 1 => "one"

case 2 => "two"

case _ => "many"

}

println(matchTest(3))

}

The block with the case statements defines a function which maps integers to strings. The match keyword provides a convenient way of applying a function (like the pattern matching function above) to an object.

Here is a second example which matches a value against patterns of different types:

object MatchTest2 extends Application {

def matchTest(x: Any): Any = x match {

case 1 => "one"

case "two" => 2

case y: Int => "scala.Int"

}

println(matchTest("two"))

}

The first case matches if x refers to the integer value 1. The second case matches if x is equal to the string "two". The third case consists of a typed pattern; it matches against any integer and binds the selector value x to the variable y of type integer.

Finally some of the key benefits of Scala –

Scala's case classes and pattern-matching make for very clean code, for example, and Scala traits (like interfaces with implemented methods) can be a very powerful and simple way to "inherit" behaviour without the pain of OO inheritance.
Scala's powerful type inference also combines the benefits of a strictly typed language with some of the benefits of dynamic typing in terms of having to write less code yourself.
Maintainability e.g. Scala code is often much more concise than Java (with all that boilerplate) and fewer LOC often means fewer places to screw up and less maintenance in the longer term.
Access to great Scala libraries e.g. for Collections, as well as to Java's existing libraries.
Scalability e.g through the use of Actors for (asynchronous) concurrency instead of threads, or through greater use of immutable data, functional programming etc.
Scala's hybrid ability to take advantage of FP and/or OOP as required for the task in hand.

Most people are using Scala on the server, because that's where you can gain the benefit of its scalability. Obviously, you can mix Scala with Java on a JVM server, and use your preferred client-side technology for browser interfaces etc. Scala is also the basis for version 2 of the lightweight Play! framework for web applications (although my understanding is that version 2.x is not currently Java EE compatible). My impression is that there is a growing trend towards looking at using the right language for specific purposes rather than a monolithic "one size fits all" approach.

Twitter is a prominent user and promoter of Scala, while LinkedIn also uses Scala for some applications.

Resources

Download Scala - http://scala-lang.org/download/

Download Typesafe Reactive Platform for Scala - http://typesafe.com/platform/getstarted

Scala IDE for Eclipse - http://scala-ide.org/

Scala documentation - http://www.scala-lang.org/documentation/

Scala school - http://twitter.github.io/scala_school/

Scala tutorial - http://docs.scala-lang.org/tutorials/

Java 8 vs Scala: A Feature Comparison

Here’s a nicely written article on InfoQ which goes deep into this comparison and list out some of the key findings. Just to mention Java 8 is going to provide a variety of new language features, with the potential to fundamentally change the way we write applications. Especially the functional programming constructs, such as lambda expressions, can be considered a paradigm shift. This shift provides great new possibilities for more concise, compact and easy to understand code.

If you want to try them out then explore early builds of Java 8 on most platforms.

The best way to find out what a language can do for you is probably to try it out, and best way would be to sign up and learn about Functional Programming Principles In Scala from the language's creator Martin Odersky of EPFL at Coursera. Just check and register for any future date for free course.

Hybris commerce suite - Insight

noreply@blogger.com (Nads) — Sun, 24 Nov 2013 11:30:00 GMT

Recently started exploring Hybris commerce suite 5.0.4 and have some of the interesting facts to share. So let’s start with basics.

What is [y] Hybris ?

Hybris is one of the upcoming agile and comprehensive commerce suite. Hybris platform is simple and based on open standards like Spring, ZK, SOLR/Lucene, ANT, Groovy, Apache Commons. The hybris commerce suite is engineered for expansion, extension and scale.

How’s the [y] Hybris architecture ?

From business standpoint the hybris Commerce Suite is divided into individual packages, such as Commerce, Content, Channel and Orders. These packages are bundles of functionality assembled for a certain range of business functionality.

From a technical point of view, packages consist of individual modules (aka extensions).

Extensions written by hybris provide standardized functionality and are supported and maintained by hybris.

hybris Commerce Suite is run in a Java Virtual Machine on a Servlet Container or a J2EE-compliant application server (such as IBM Websphere or Oracle WebLogic) and connects to an external database (MySQL, Oracle DB, Microsoft SQL Server). Internal caching and persistence mechanisms allow the hybris Commerce Suite to run on a Servlet Container. A full-fledged J2EE-compliant application server can be used, but is not necessary.

What are the key Hybris commerce capabilities and features which comes out of the box ?

General Storefront Capabilities - Language selection , Country selection , Currency selection , SEO frienddly URLs

Product Search - Solr-based product search with Keyword Search , Keyword Autocomplete , Spelling suggestions , Faceted browsing and more

Merchandising

Personalization and Promotion

Catalog Navigation

Product Display

Cart

Checkout

Hosted Payment

Customer Account

Store Location

Content Management

Data Import / Export

Google Local / Google Shopping

Content Pages and Templates

Organization Management

Jirafe Customer Intelligence

Google Analytics

Quote Negotiation

Order Approval

Order Management (OMS)

Store Locator

Device Detection

Lets dig deeper into some of the other concepts which is essential to know –

Hybris Cockpit framework –

The hybris Cockpit framework offers a large number of highly configurable components which can be used to build a new cockpit for your customers needs, such as for example workflow and comments.

The hybris Cockpit framework is the foundation for all hybris Cockpits. It offers a large number of highly configurable components, which can be used to build a Graphical User Interface (GUI) to support high-level business use cases, allowing users to perform all their common tasks quickly and intuitively. The hybris Cockpit framework complements the functionality offered by the hMC, which provides lower-level control over all the data in the hybris system.

The hybris Cockpit framework is based on ZK, a rich internet application (RIA) framework that enables desktop-like GUIs within a web browser. The ZK Framework is also well supported by Spring. ZK delivers a rich set of user interface (UI) components, making it easy and quick for developers to deliver a rich front end experience.

The major features of the hybris Cockpit framework include:

Security infrastructure based on Spring security
Ready to use front end template
Base configuration as a starting point for new applications, based on Spring
Reusable front end components in addition to existing ZK components
Generic abstraction of persisted items in the front end layer via the Type Service
GenericSearch providers
Highly customizable front end
Automatically stored user interface configuration at application run time
Drag and drop functionality

Here’s the key Hybris cockpits listed with brief description -

Hybris Administration Console - The hybris Administration Console is an administration Web page provided by the hac extension. It provides functionality for administration, monitoring, and configuration of the hybris Commerce Suite.

Hybris Management Console (hMC) - hMC is the administration tool of the hybris Commerce Suite. As part of the hybris Commerce Suite, it runs in web browsers. Using the hMC, you can manage business objects of the hybris Commerce Suite such as products, customer data, orders.

Product Cockpit - The hybris Product Cockpit enables organizations to manage and structure product information and catalogs in high-volume and collaborative environments. Product managers can visualize their input from multiple perspectives and make product management processes more efficient

WCMS Cockpit - The hybris WCMS Cockpit enables you to manage data of a website. For efficient management of information on your online shop, you can use both the catalog perspective and live edit perspective. Each of them provides you with different ways of data management.

In the hybris WCMS Cockpit you can easily do the following:

· Create and update website pages and content components

· Check and change website pages status

· Synchronize catalog versions of your website

· Create collections of chosen pages

· Search for website pages, according to advanced criteria

Admin Cockpit - At present, the hybris Administration Cockpit is used for the manipulation of: Data Validation constraints and Instances of types

Customer service Cockpit - The hybris Customer Service Cockpit enables you to quickly and efficiently manage the customer and orders in the call-center environment.

Report Cockpit - The hybris Reporting Module adds Report Cockpit in which users can create and manage personal Dashboard containing rich graphical report widgets.

Print Cockpit - The hybris Print Cockpit enables you to create the publication structure and to manage the publication data for in an intuitive and user-friendly way. Linked with Adobe InDesign, it is a unique and powerful application.

Lets take closer look at some technical bits –

Adding a new page in Hybris –

Create a new page using the cmscockpit and with the available templates(we can also create new templates using the hmc as per the requirement).
Add components to the content slots available
Create a controller to handle the user requests for the newly created page.
Get the request mapping from the request, perform the business logic and return the model and view

Adding a new component in Hybris –

Create own item in items.xml file as below

jaloclass="de.hybris.platform.endeca.jalo.cms2.components.OurComponent">

//necessary attributes go here.....

Create a new controller OurComponentController.java

Note: The name of controller is important and should be created as follows: Component Type +"Controller" (in our case OurComponentController). Beside that we also have to create a representation of our component ourComponent.jsp

All the controllers are defined the way we define beans in Spring MVC in the spring-servlet.xml file as mentioned below

Perform the business logic and return the model and view.

Creating new extensions --

The hybris Commerce Suite comes with an extension generator system called extgen located in the ${HYBRIS_BIN_DIR}/platform/extgen/ directory. Using extgen, you can create new extensions based on extension templates.

The hybris Commerce Suite comes with the following extensions, which you can use as a template for new extensions:

yempty

ycockpit

yaddon

ybackoffice

yacceleratorstorefront

yacceleratortest

ycommercewebservices

yinstoreinitialdata

yacceleratorcore

yacceleratorfacades

yacceleratorinitialdata

Extgen prompts you to specify:

The extension's name.
The extension's Java package.
The extension template to use.

After you have provided the prompted values, extgen does the following:

Copying the extension template to a temporary directory.
Modifying the extension template to reflect the specified values.
Copying the new extension from the temporary directory to the directory specified by the extgen.extension.path property in the project.properties file.

To create a new extension we follow following steps -

Optionally: Modify default values for the extension generation process.
Create a new extension.
Reference the new extension in localextensions.xml file.
Rebuild the hybris Commerce Suite.
Update the hybris Commerce Suite.

Afterwords, you can start implementing your business logic in the new extension.

How to get started with Hybris ?

Best to register at https://wiki.hybris.com and get started. As of now they only allow certain partner companies to register so please check with your organization.

Once you registered at their wiki then you can access the product documentation.

I would highly recommend doing the Hybris core and commerce Trails. They are logically sequenced and it helps to understand overall product features and capabilities. Make sure you are good with Spring framework fundamentals , Annotations and ant commands.

Final thoughts ... No deny that Hybris has put great efforts to bring a light weight agile commerce framework which seems very promising. Best part is it cuts lot of development time to build some of the common features in a commerce site in no time, Their modular approach and open stack also gives you enough choice to pick and chose as per your need. It will be interesting to see how their future releases fair. I am yet to see how it scales and perform in a high volume and high transaction environments.

Emerging languages series - Groovy

noreply@blogger.com (Nads) — Sun, 10 Nov 2013 15:39:00 GMT

Our favorite Java is getting some serious competition in recent years , It seems there are more than 200 languages that target Java platform.
There is growing interest in languages like Scala, Groovy , Clojure , JRuby, Jython for their features and simplicity. Out of this lot we will take a close look at Groovy today as its getting wide attention in the market over recent years.

Why Groovy ?

Groovy is more or less a superset of Java
Builds upon the strengths of Java but has additional power features inspired by languages like Python, Ruby and Smalltalk
Could seamlessly switch back and forth between Java and Groovy
Considered simpler and easier to learn over Jython and JRuby
Groovy has Java-like syntax and hence the learning curve is not steep.
Simplifies testing by supporting unit testing and mocking out-of-the-box

Once you install Groovy, you could straightaway start programming in Groovy, just put your Groovy jar file along with your application and get the program working in any machine.

There are many online editors and IDE plugin available to compile and run your Groovy code.

So what is Groovy ?

James Strachan conceived the idea of Groovy language and started development in 2003.

Groovy is a dynamic language that dynamically compiles the code to the Java platform. It is popular as a scripting language and also widely used for unit testing Java code. Groovy is open source (Apache v2 license) and is backed by VMware.

The current stable release is Groovy 2.2. You can use Groovy in most common operating systems like Linux, Windows, and Mac.

Let’s see how Groovy looks better Java with few examples –

Hello World Example –

// Java

class HelloWorld {

public static void main(String []args) {

System.out.println("Hello world");

}

// Groovy

println("hello world")

More Detailed Examples –

Let’s take a look at collection (map) processing example. Lets create a map and do some processing in both Java and Groovy.

// Java

Map worldCapitals = new HashMap<>();

worldCapitals.put("England", "London");

worldCapitals.put("Japan", "Tokyo");

worldCapitals.put("France", "Paris");

worldCapitals.put("Italy", "Rome");

worldCapitals.put("USA", "Washington");

for(Map.Entry capital : worldCapitals.entrySet()) {

System.out.printf("%s => %s \n", capital.getKey(), capital.getValue());

}

// Groovy

def worldCapitals = [ "England" : "London",

"Japan" : "Tokyo",

"France" : "Paris",

"Italy" : "Rome",

"USA" : "Washington" ]

worldCapitals.each {

key, value ->

println "$key = $value"

}

In this case Groovy provides a simple array-like syntax for the Map data structure. See how use of each() method closure makes it simple to write the code.

We all know how we write custom code to sort a map by value in Java but the same can be achieved in a line with Groovy. See below -

def sortedMap = worldCapitals.sort { it.value }

sortedMap.each {

key, value ->

println "$key => $value"

}

In above Groovy code, we just passed the it.value (current iterator’s value) to the sort method. With the power of closures, the code became much simple and readable.

Groovy provides a higher-level abstraction to this functionality, and hence the code is much simpler.

Tip - The soon to be released Java 8 has also introduced concept of closures in form of "lambda expressions". It will be interesting to see how Java is incorporating these new age language features in their new release. I will write separate post to cover Java 8 features and capabilities soon.

XML Generation Example -

In general it’s very common to generate structured output like HTML , XML or JSON in Java code. So let’s see one example around this piece.

// java

PrintWriter pw = new PrintWriter(new FileWriter("/capitals.xml"))

pw.println("");

for(Map.Entry capital : worldCapitals.entrySet()) {

pw.printf("\t \n \t\t '%s' \n \t \n", capital.getKey(), capital.getValue());

}

pw.println("

");

// Groovy

def capitals = [ "England" : "London",

"Japan" : "Tokyo",

"France" : "Paris",

"Italy" : "Rome",

"USA" : "Washington" ]

xmlbuilder = new groovy.xml.MarkupBuilder()

xmlbuilder.worldcapitals {

capitals.each {

key, value ->

country(name:key) { capital(value) }

}

Now just see how simple it is to do the same thing with Groovy with lot of readability. Groovy supports builders (based on Builder design pattern) for creating HTML and XML files.

Take away from this comparison -

Traversing a data structure is easy in Groovy - you can use “closures” instead of plain loops and use each() method.
Groovy is high on meta-programming and regular expression handling capabilities.
Groovy builders are fabulous if you dealing with HTML , XML , JSON etc.
Better for productivity and code readability for sure
Being a dynamic language Groovy is an excellent candidate for creating DSLs (Domain Specific Languages)

We have just tried to explore few basic features of Groovy here. You should explore further programming improvements such as automatic check for nulls, helper methods, embedded expressions in strings, and operator overloading. One should also explore its meta-programming and regular expression handling capabilities.

You’ll find Grails web framework and Groovy Server Pages (GSP) interesting if you are a serious web-developer. If you are an admin, you may find Gradle build automation framework useful. Grails framework (that is similar to Ruby on Rails) is built on top of powerful and popular frameworks such as Spring and Hibernate.

Resources to learn further –

Groovy Documentation - http://groovy.codehaus.org/Documentation

Groovy Eclipse plugin - http://groovy.codehaus.org/Eclipse+Plugin

Programming Groovy 2 - Dynamic Productivity for the Java Developer, By Venkat Subramaniam

Groovy in Action, By Dierk Koenig, Andrew Glover, Paul King, Guillaume Laforge, Jon Skeet

If you are a developer, tester or script guru, you have to love Groovy. So keep grooving and have fun with it !!

What TDD won’t tell you and why BDD is equally important?

noreply@blogger.com (Nads) — Fri, 15 Feb 2013 23:48:00 GMT

Most of us think Test Driven Development is a cycle of test, implement and refactor. In reality its the freedom to start working toward your solution before you can map it all out.

It’s the freedom to make changes knowing your tests will alert you if you have violated a previous assumption that you have forgotten.

Since TDD forces you to write test first, it also forces you to think about how you want to use an API, because that is the very first thing you write down. This is a good thing and it leads to more usable APIs.

TDD also forces you to factor out dependencies in order to be able to mock them. Thats the other point where TDD encourages good design.

Now here comes the twist : There is more to software design than just nice to use APIs and good principles.TDD does almost nothing for you in order to find a good solution for your problem.

TDD won’t tell you following -

Can your domain model represent all the cases needed by the business?
Which of all the design patterns would help you? Is there any well known algorithm you could use to solve your problem?
Does the problem become trivial once you approach it in a functional way?
Is recursion a solution or a dead end?

Ultimately one has to understand the problem and know lots of possible approaches to a problem in order to design software well. There is just no way around it.

Once you have the solution in your mind, once you know the basic data structures and algorithm you going to use, TDD will help you to implement it in a clean, well factored way.

Having said enough about TDD I would like to now shift my focus on BDD (Behavior Driven Development) and its real value.

TDD is more “low level” talking about unit tests and integration tests and how to write code that tests real code while BDD is about specifications and behaviors.

In TDD we talk about Arrange Act Assert while in BDD we have Given When Then

There are clear cut differences between both. BDD is more “high level” dealing with specifications and was intended to be used by domain experts as well as developers

As a matter of fact one should write unit tests as well as specifications – because we need to test both.

TDD is described by a basic red-green-refactor cycle, I have come across this interesting checklist on dzone which is written in the form of questions we should ask ourselves while going through the different phases of the cycle.

Happy Testing :)

ATG DUST – A JUnit framework for ATG applications

noreply@blogger.com (Nads) — Sun, 6 Jan 2013 16:40:00 GMT

Well we all have been writing the classic JUnit tests from years now but it becomes challenging to write test cases when you dealing with framework like ATG.

So here comes ATG DUST - A framework for writing JUnit tests for applications built on the ATG Dynamo platform.

Its indeed makes our life lot simpler by providing some amazing features. The highlight of this framework is that it allows one to quickly write test classes that depends on Nucleus or ATG Repositories. By using ATG DUST one can drastically cut down on development time. It takes only a few seconds to start up a test with a repository, not like application server which takes good time to start up.

To get started go straight to first test page . This page will walk you through the process of running a basic test which starts Nucleus.

This tutorial walks you through the steps of writing a test which does not need to run in an application server. This is one of the underlying benefits of writing unit tests.

The test does not depend upon starting up an application server so turnaround time is also very quick.

Visit out-of-the-box-component-testing page which gives you a feel of simple test class. Try to open the java source (StartWithModulesTest.java) file attached there to get a fair idea how easy to use ATG DUST.

The best part I liked about ATG DUST is having the ability to load any specific ATG module and write a quick Repository test class. Go to the repository test page for a simple example.

The repository is started up against an in-memory Hypersonic Database.

DUST also support Formhandler test features with the help of utility (ServletTestUtils) class which can create a dummy Dynamo Request/Response pair.

Some of the key classes to look at –

atg.adapter.gsa.GSATest - A basic GSA test which has several utility methods.

atg.test.util.DBUtils - Utility code for getting a connection to a database. The most common method is getHSQLDBInMemoryDBConnection.

atg.test.util.FileUtil - A collection of utility methods for dealing with the file system.

atg.nucleus.NucleusTestUtils - This class contains some utility methods to make it faster to write a unit test that needs to resolve components against Nucleus.

Note on ATG version support – ATG DUST was developed using 9.0 but it work on most ATG versions back to 7.2 . I used ATG DUST version 1.2.2

Download ATG DUST

Oracle ATG Search 9.3 configuration on Local (Windows 7 , 64 bit) machine

noreply@blogger.com (Nads) — Thu, 11 Oct 2012 04:52:00 +0100

Recently installed ATG search 9.3 on local windows machine and configured search environment and index. It was a good fun as it gave quite some errors to fix before running my first index on local machine. I would try to give all the main steps and error faced in this process.

Lets first start with prerequisites –

You should have a working/running Store and BCC JBoss instances in your local system. I also assume that you have already installed Search 9.3 in your local ATG home.

Now let’s focus on search configuration. Kindly follow below steps in same sequence.

1) Delete any existing projects and Environments from Search Admin in BCC Admin screen.

2) Goto Store ACC and delete everything from SearchConfigurationRepository under Content. You may leave searchindex and logicalpartition items in it.

3) Set following remoteHost=localhost , remotePort=8860 in below components as per your local store.

C:\ATG\ATG9.1\home\servers\pub\localconfig\atg\commerce\search\refinement\RemoteCatalogRefineConfigAdapter.properties

C:\ATG\ATG9.1\home\servers\pub\localconfig\atg\commerce\search\config\RemoteCatalogRankConfigAdapter.properties

C:\ATG\ATG9.1\home\servers\pub\localconfig\atg\commerce\search\config\RemoteSearchUpdateAdapter.properties

4) Now goto BCC and create a new search project called production under Search Admin

and add content as below

ATG Repository - Local - /atg/commerce/search/ProductCatalogOutputConfig

5) Now goto "Language Customizations(Pre-Index)" under content option of new project(production)

a) Add Synonyms by using option "Select Custom Term Dictionaries"

b) Add languages like english , italian, french, german, japansese,korean using option "core language support"

6) Now add Post-Index Customizations options in the below sequence ONLY

Merchandising Search Config - Remote - localhost:8860 - /atg/commerce/search/config/CatalogRankConfigAdapter

Refine Config (Facet Set) - Remote - localhost:8860 - /atg/commerce/search/refinement/CatalogRefineConfigAdapter

Search Update Config (Auxiliary Data) - Remote - localhost:8860 - /atg/commerce/search/config/SearchUpdateAdapter

7) Check the Environment - You should see one default one just rename it to production

8) Now build the index and see if it works

If you get this below error at BCC server startup then do the following ---

ERROR [IndexDeploymentService] Swap failed for index 35000001; aborting swap and tearing down staging. Swap rollback Policy Description: All physical partitions must have at least one successfully initialized search engine

1) Add this in Store jboss server run.conf file and restart your store server.

JAVA_OPTS="$JAVA_OPTS -Datg.allowRemoveAllItems=true"

2) Now goto below component in your Store server dyn/admin and do a remove-all-items on all items under SearchConfigurationRepository Item Descriptor.

http://localhost:8180/dyn/admin/nucleus//atg/search/routing/repository/SearchConfigurationRepository/

swapcheck

searchEngine

searchMachine

searchEnvironmentHost

searchEnvironment

physicalPartition

searchIndex

logicalPartition

indexingActivitySummary

indexingCommandCount

deploymentHistory

3) Now do the whole setup again in BCC which we did in above steps thru 4 to 9.

Note -- You also need to increase the TransactionTimeout setting in jboss-service.xml under your store server's conf/ folder.

30000

Note -- You also need to increase heap size setting in JBOSS_OPTS variable in Jboss run.bat file

References –

Error ticket on community - https://forums.oracle.com/forums/thread.jspa?messageID=10640583#10640583

ATG search documentation - http://docs.oracle.com/cd/E23095_01/Search.93/index.html

Let me know if you face any issues with Search setup or if you know of any better way to configure search.

SQL Error: ORA-01591: lock held by in-doubt distributed transaction

noreply@blogger.com (Nads) — Tue, 4 Sep 2012 23:47:00 +0100

Well last week I came across this ORA-01591 while running alter command against one of the Oracle DB schema. It simply failed by giving following error.

SQL Error: ORA-01591: lock held by in-doubt distributed transaction 5.4.426183
01591. 00000 - "lock held by in-doubt distributed transaction %s"
*Cause:    Trying to access resource that is locked by a dead two-phase commit
           transaction that is in prepared state.
*Action:   DBA should query the pending_trans$ and related tables, and attempt
           to repair network connection(s) to coordinator and commit point.
           If timely repair is not possible, DBA should contact DBA at commit
           point if known or end user for correct outcome, or use heuristic
           default if given to issue a heuristic commit or abort command to
           finalize the local portion of the distributed transaction.

I did Google around and got some help to resolve this. Here are the steps I followed to fix this issue.

1) Connect to Oracle as sysdba by using this command -- sqlplus sys as sysdba

2) select LOCAL_TRAN_ID from dba_2pc_pending

This above select command will give you list of all pending transaction ids and one id will match with the id mentioned in the above error. you have to pick that id and delete it. Before attempting to delete (step 5) you need to first execute steps 3 and 4.

3) alter system enable distributed recovery;

Above statement enables distributed recovery

4) rollback force '5.4.426183';

commit;

Note - Use ROLLBACK statement with the FORCE option and a text string that indicates either the local or global transaction ID of the in-doubt transaction to commit.

5) execute dbms_transaction.purge_lost_db_entry('5.4.426183');

Once the above procedure executes successfully then you may try your original Alter/DDL command and restart your app server.

Purging Asset Versions in BCC using ATG dynamo PurgingService

noreply@blogger.com (Nads) — Fri, 10 Aug 2012 16:04:00 +0100

Recently faced couple of issues with running the Purge Service under BCC and had to do some tuning to finally make it work on large volume of versioned assets.

It is generally a good idea to periodically purge versioned repository data of old projects and asset versions. Over the period of time, the versioning system in Content Administration (CA) can accumulate a large number of asset versions and completed projects. As asset versions accumulate, they can strain storage capacity and system performance. It also becomes hard to take live data copy and replicate to other environments.

The length of a purge depends on the number of repository and file assets that need to be purged. A purge that has large number assets can be lengthy specially for the first time. Try scheduling multiple purges in this case.

Its also a good idea to take back up of all affected datastores and file systems before you start a purge.

Purge Service generates a Summary Metrics report before starting the purge activity and another one after the purge is completed.

The report basically has details like number of projects and asset versions removed, and the number of projects and asset versions that remain. It has a good amount of details as what is going to be purged and what will remain after that.

You might like to do few changes in your BCC server instance to run the purge service as its likely that purge will fail initially due to various reasons. Worth trying these -

(1) Purge operation executes in a transaction. If a purge has a large number of assets, you might need to raise your application server’s transaction timeout setting—for JBoss reset TransactionTimeout attribute (in /server/yourserver/conf/jboss-service.xml file).

(2) JVM memory setting in Jboss - It depends on volume of data you have. If you get any memory error while performing Purge activity then consider increasing memory by 1 GB and keep increasing till the time memory error goes off. In ideal condition going upto 6 GB would be good enough. (-Xms6144m -Xmx6144m) under Jboss/bin/run.bat on Windows.

(3) Resolving repository data conflict - Purge service might failed on ContentRepository data, to fix this try setting VersionManagerService.enableProtectivePurge as false and then run the purge service.

For more details read Oracle ATG CA docs here –

Effective use of robots.txt

noreply@blogger.com (Nads) — Wed, 18 Apr 2012 00:22:00 +0100

Off late I did some work on SEO and got an opprtunity to play around the robots.txt file to apply various rules. Would like to share the common understanding around it, feel free to provide your comments or share your experiences.

As part of sensible SEO practice its important to keep a firm grasp on managing exactly what information we don't want being crawled!
A robots.txt file restricts access to your site by search engine robots that crawl the web. These bots are automated and before they access pages of a site, they check to see if a robots.txt file exists that prevents them from accessing certain pages.
You need a robots.txt file only if your site includes content that you don't want search engines to index. If you want search engines to index everything in your site, you don't need a robots.txt file

The simplest robots.txt file uses two rules:
User-agent: the robot the following rule applies to
Disallow: the URL you want to block

These two lines are considered a single entry in the file. You can include as many entries as you want. You can include multiple Disallow lines and multiple user-agents in one entry.

Some example below -

User-agent: *
Disallow: /images/

User-Agent: Googlebot
Disallow: /archive/

The Disallow line lists the pages you want to block. You can list a specific URL or a pattern. The entry should begin with a forward slash (/).

To block the entire site, use a forward slash.

Disallow: /

To block a directory and everything in it, follow the directory name with a forward slash.

Disallow: /archive-directory/

To block a page, list the page.

Disallow: /checkout.jsp

To remove a specific image from Google Images, add the following:

User-agent: Googlebot-Image
Disallow: /images/logo.jpg

To remove all images on your site from Google Images:

User-agent: Googlebot-Image
Disallow: /

To block files of a specific file type (for example, .gif), use the following:

User-agent: Googlebot
Disallow: /*.gif$

To specify matching the end of a URL, use $. For instance, to block any URLs that end with .xls:

User-agent: Googlebot
Disallow: /*.xls$

We can restrict crawling where it's not needed with robots.txt
A "robots.txt" file tells search engines whether they can access and therefore crawl parts of your site. This file, which must be named "robots.txt", is placed in the root directory of your site. e.g - www.example.com/robots.txt

If you have a multi country site then each country should have its own robots.txt
To further read follow these links for generating and using robots.txt

robots.txt generator
Using robots.txt files
Caveats of each URL blocking method

Kindly note Google has a limit of only being able to process up to 500KB of your robots.txt file.

11 most commonly used FormHandlers in ATG

noreply@blogger.com (Nads) — Thu, 25 Nov 2010 15:55:00 GMT

Here are some of the commonly used FormHandlers, although ATG provides many out of the box FormHandlers and even we can write our own custom FormHandlers by extending GenericFormHandler class provided by ATG.

CartModifierFormHandler -- This formhandler is used to modify a ShoppingCart by adding items to it, deleting items from it, modifying the quantities of items in it, and preparing it for the checkout process.

ExpressCheckoutFormHandler -- The ExpressCheckoutFormHAndler is used to expedite the checking out of an Order. This supports creating a maximum of one Profile derived HardgoodShippingGroup and one Profile derived CreditCard, followed by committing the Order.

SaveOrderFormHandler -- The SaveOrderFormHandler is used to save the user's current Order based on a descriptive name that the user specifies. A new empty Order is then made the user's current shopping cart. If a descriptive name for the Order is not specified, then one is created based on the user's Locale and date and time.

ShippingGroupFormHandler -- The ShippingGroupFormHandler is used to associate ShippingGroups with the various Order pieces. This component is used during the Order checkout process, and any Order successfully processed by the ShippingGroupFormHandler is ready for the next checkout phase, which may be Payment.

PaymentGroupFormHandler -- The PaymentGroupFormHandler is used to associate PaymentGroups with the various Order pieces. This component is used during the Order checkout process, and any Order successfully processed by the PaymentGroupFormHandler is ready for the next checkout phase, which may be confirmation.

CommitOderFormHandler -- The CommitOrderFormHandler is used to submit the Order for final confirmation. This calls the OrderManager's processOrder method. If there are no errors with processing the Order, then the current Order in the user's ShoppingCart will be set to null and the submitted Order will become the ShoppingCart's last Order.

CancelOderFormHandler -- The CancelOrderFormHandler is used to cancel the user's current Order, which deletes the Order from the ShoppingCart.

RepositoryFromHandler -- Saves repository data to a database.

ProfileFormHandler -- Connects forms with user profiles stored in a profile repository.

SearchFormHandler -- Specifies properties available to a search engine.

SimpleSQLFormHandler -- Works with form data that is stored in a SQL database.

ATG's Data Anywhere Architecture

noreply@blogger.com (Nads) — Sat, 20 Nov 2010 15:30:00 GMT

Just read one of the white papers from ATG's site , liked the way they have explained about Data Anywhere Architecture. Thought to share the important points about it. Here we go :-)

Challenges --
All enterprise applications need to access and manipulate data in some shape or form. Common challenges that have been found in building scalable, object-oriented, data-centric applications are:

Object-to-relational mapping – Issues surrounding how a relational data representation can be appropriately mapped to an object oriented programming language in a way that does not impact simplicity or data integrity.

Data source insulation – Issues surrounding the fact that relational/SQL database may not be the only form of data that the application requires. Other data source types may include LDAP directories or XML file-based assets.

Data caching – Issues surrounding the appropriate use of data resources without inflicting the high volumes of data source ‘hits’ common in high-traffic Web applications. Caching issues include the integrity and cache invalidation of the data used throughout a distributed application.

Solutions --
ATG’s Data Anywhere Architecture (DAA) meets all of these challenges. DAA gives developers a single API, called the Repository API, for using data resources in their applications. Behind the Repository API, DAA insulates and abstracts application developers from the specifics of the data source, so much so that the nature of the underlying data source may completely change without major impact. For example, customer data may reside in a SQL/JDBC database today, but will move to an LDAP directory in the future. DAA could handle this without having to touch any code within the application. The fundamental construct in the DAA is a ‘Repository’. A Repository is a logical view of a data resource (or resources), and to a developer, manifests itself as a set of JavaBeans to be used within their application. Like everything else in an ATG application, Repositories are represented as Nucleus components.

The Repository is described in a Repository Definition XML file, which holds all appropriate information about the data’s physical location and how it is mapped to the logical view. The DAA consists of three primary Repository types for data access and manipulation.

SQL repository – A SQL Repository presents a logical view of data stored in a relational database, accessed through JDBC. The Repository definition file defines how the databases, tables, rows, and columns of a relational database are presented through the Repository API. It also defines the item caching strategy to be employed to optimize database read/write performance.

LDAP repository – An LDAP Repository presents a logical view of any data source that has an LDAP interface, typically used to store user data. The Repository definition file defines how the hierarchical structure and contents of an LDAP directory are presented through the Repository API.

Integration repository – In some cases, data sources may not be directly accessible, or information may be returned by an application rather than directly from a database or directory service. The Integration Repository presents a Repository API in front of some remote application processing. For example, an Integration Repository may be implemented to facilitate integration with SAP’s BAPI interface, or to retrieve information through the execution of a Web Service SOAP call. The Integration Repository is an open architecture into which these specific integration technologies can be plugged, while still presenting the same Repository API to the application developer. It also gives developer sophisticated data access and caching strategies to protect the application from remote latency and downtime.
In addition to the primary types of Repository mentioned so far, there are two types of ‘overlay’ repository types that can be used.

Secure repository – A Secure Repository introduces application level security and access control to the data being accessed and manipulated. Working with ATG’s Security Management Framework, varying levels of security can be defined on the Repository contents, all the way down to individual data properties. Access Control Lists (ACLs) are written to describe the different levels of access that are provided to ATG’s User Model, which itself provides a rich structure to model user, organizational hierarchies and roles.

Versioned repository – A Versioned Repository introduces a versioning mechanism to one of the other primary Repository types. It provides all of the required tools to maintain, version and roll-back versions of a Repositories contents. Any existing SQL Repository may be turned into a Versioned Repository through additional configuration files. The Versioned Repository architecture is heavily used by ATG’s Content Administration product, but the versioning features are open for any other type of application usage that may be customer specific. Versioned Repositories integrate closely with ATG’s workflow capabilities that reside in the ATG Adaptive Scenario Engine. A Composite Repository is the final construct that can be especially useful for building applications requiring access to data in multiple data sources and formats.

Composite repository – A Composite Repository represents an aggregate view over other repository types, or over other composite Repositories (although one should not create too many layers of Composite Repository). The most common use of a Composite Repository is where a businesses customer data is distributed over multiple SQL databases and an LDAP directory, but a Web application wants a ‘single view of the customer’ to reduce application complexity.

A Composite Repository provides some welcome simplicity.

To ensure scalability of Web site usage of SQL database, the DAA provides sophisticated caching and cache invalidation mechanisms for SQL Repositories.

DAA provides all of the necessary tools to manage and flush caches at the repository item level. There are also mechanisms for managing distributed caches

and cache invalidation via JMS or TCP/IP. All in all, ATG’s Data Anywhere Architecture provides a rich, robust, and highly scalable set of tools to facilitate the use of enterprise data resources, while providing a loose coupling approach between data source and application.

If you want to further explore features of ATG repository or may like to compare it with Hibernate then follow the below link which points to ATG community site - https://community.atg.com/docs/DOC-1894

ATG Dynamo Application Framework (DAF) and Nucleus

noreply@blogger.com (Nads) — Tue, 5 Oct 2010 19:21:00 +0100

Hope most of you must be knowing about Dynamo Application Framework (DAF) and Nucleus concepts in ATG . I have just tried to summarized the basic idea about it which I read mostly from ATG white papers. Hope it will be useful for someone who is new to the world of ATG. Any comments or suggestions are most welcome. Would try to write more practical examples about dynamo and commerce related topics.

The ATG Dynamo Application Framework is an application framework designed to help simplify the creation of Web applications. It provides a large number of the common services, components, and frameworks that an application developer needs when building highly scalable, feature-rich, enterprise Web applications. DAF provides three core ‘pillars’ to help a developer construct an application.

Component model – Any software application requires a component model that provides structure and coherence to an application. The component model used by DAF is JavaBeans, managed in a component container called the Nucleus (read about it in below section).

Data access model – All recent Web applications require information access and an ability to manipulate data. DAF’s data access and manipulation model is called the Data Anywhere Architecture.

Messaging model – Responsive applications require a messaging architecture that allows events to be fired and appropriate actions to execute on the occurrence of those events elsewhere in the system. The JMS messaging model implemented by DAF is managed by a service called the Patch Bay.

Although not classified as a ‘main pillar,’ the user interface (UI) programming model by which these previously mentioned elements can be used is also important, and an area where ATG has innovated ahead of the general market. DAF uses JavaBeans as the primary, lightweight, component model. These JavaBean components are configured and linked together by .properties files within Nucleus. The DAF application framework can also be run on all major J2EE application servers (JBoss, WebLogic, WebSphere etc.).

Nucleus is a ‘light weight’, yet feature-rich component model. It adheres to the “Inversion of Control” design pattern, whereby software components are discrete entities coupled together by the Nucleus container, rather than through direct reference. The services and structure provided by Nucleus makes building Java applications much simpler than when starting with the base set of Java and J2EE services. It promotes good interface-based programming principles and helps application developers take a modular approach, resulting in more modularized, maintainable, and understandable applications.

Nucleus is DAF’s component namespace for building applications from JavaBeans. Nucleus allows the assembly of applications through simple configuration files that specify what components are used by the application, what properties they should have, and how components hook together. Nucleus itself provides no application-specific functionality, since it is only a container in which components live, discover, and interact with each other. It is the collection of components that make up the functionality of an overall application. Nucleus organizes these application components into a hierarchical namespace.

A lot of what makes Nucleus special is encapsulated in the following core areas:

Component creation and administration – Nucleus provides a simple way to write new components. It is a simple process to take any Java object and allow it to act as a component in Nucleus. Nucleus takes on the task of creating and initializing components. A very useful aspect of Nucleus is that applications don’t need to contain code to create instances of components. Instead, components can be created and administered through configuration files that specify the component and the initial values of its properties. If needed, administrators can alter the properties of ‘live’ components within the application. The component instances are then initialized automatically at start up time, rather than programmatically created. Nucleus employs a ‘lazy instantiation’ policy for creating components. One component is only created at the point it is referenced by another.

Component layering and combination – Nucleus provides a convenient way to modify and extend component properties by organizing configuration files into layers. This layering allows application developers to add new components or override the settings of existing ones without modifying the configuration files shipped by ATG. Nucleus automatically combines the layers at application start-up. These layers are organized into ‘modules’ so the associated Java class files can be maintained with the configuration files, simplifying application maintenance and upgrade.

Component scoping – To further increase its usefulness as a component model for Web applications, Nucleus makes it very easy for application developers to set the scope of their components. The scope can be set to ‘global’, ‘session’, or ‘request’,. Nucleus takes care of how the components are managed so that developers do not have to do any specific coding.

Nucleus includes a large number of out-of-the-box generalized service components that can be used in many ways within an application. Service components include TCP Request Services, Scheduler Services, ID Generation Services, Resource Pools, Queues, Email Senders and Listeners, and many more. Each Nucleus service has a unique Nucleus name. For example, the default javax.sql.DataSource component is located at /atg/dynamo/service/jdbc/JTDataSource

It is clear that emerging models typically tackle one or the other element of DAF but not everything. Hibernate tackles the data access problem, Struts tackles the UI development problem, etc. It is often left to the application developer to figure out how these different initiatives are used together.

playing around equals() and hashCode() methods of Object class

noreply@blogger.com (Nads) — Fri, 16 Apr 2010 07:42:00 +0100

Why, when & how of overriding equals() and hashCode() methods of Object class

As we know the Object class has five non final methods namely equals, hashCode, toString, clone, and finalize.
I believe they were primarily designed to be overridden according to specific needs.
I am just trying to summarize my understanding around these two methods. When should we override these and how we should implement these etc. Any comments/discussion/feedback would be highly appreciated on above topics.

Overriding the equals method -
Ideally you should override the equals method when you want to specify the rules of logical equality of objects. Two objects are logically equal if they have the same values for the same uniqueness attributes.

We need to implement an equivalence relation between non null object references. The rules to override equals method can be found on sun's site. They basically talk about Symmetry , Reflectivity Consistency & Transitivity among the objects.

The rule for the null reference is that any non-null reference value x, x.equals(null) must return false.

Object class already provides an implementation of the equals method. It is
public boolean equals(Object obj) {
return (this == obj);
}

The method above simply tests for equality of object references. This is not always the desired behavior, particularly when comparing Strings. That's why the String class provides its own implementation of the equals method.

For example suppose we have a Student class which has a constructor like this.
public Student(String name, String id, int age) { …}

Now if you create two student objects with the same attributes, you'd want those two objects to be the same student. If you write a code like below then you will get false as result.

Student student1 = new Student("sajid", "456789", 28);
Student student2 = new Student("sajid", "456789", 28);
System.out.println(student1.equals(student2));

Result is false because student1 and student2 are different references and the equals method being used here (from the object class) is comparing references. So here we need to override the equals method to check for our uniqueness attributes for our comparisons to work.

Caution: we should not do a mistake of overloading the equals method instead of overriding it. So the argument to the equals method must be an Object.

See the difference between below two statements. You will understand it.
public boolean equals(Object obj)
public boolean equals(Student student)

A good implementation of equals should start with symmetry test. Means x.equals(x) should always return true. If this it self is failing then no need to proceed further.

Another important point would be to test the instance of the object passed to equals() method. People might pass a String object instead of Object in the argument. Here instanceof operator helps us to check this.

instanceof also eliminates the trivial cases when the object passed is null because it returns false if the first argument is null.

So whole implementation might look like this …

public boolean equals(Object obj) {
if (this == obj) {
return true;
}

if ( !(obj instanceof Student)) {
return false;
}

Student student = (Student)obj;
return age == student.getAge() && name.equals(student.getName())
&& id.equals(student.getId());

}

Now System.out.println(student1.equals(student2)); will print true.

Food for thought -
Note that we deliberately compared the ages (integers) first. The && operator has short circuit behavior, meaning that if the age comparison fails the rest of the comparison is abandoned and false is returned. It is therefore a performance advantage to have the cheaper (memory wise) tests first and the more memory demanding tests last.

Is there any situation when we should not override equals() ?
Yes. When the references check is sufficient. This is when each instance of the class is unique. Other situation could be when parent class already has implemented the desired behavior then we need not bother.

Now whenever we override the equals method, we must also override the hashCode method.
So let's move ahead to hashing …

Overriding the hashCode method -
Why hashcode ? Simple the hashCode method is supported for the benefit of hash based collections. Basically this hash code value is used by hash based collections such as Hashtable, HashMap, HashSet, etc. for storing, retrieving, sorting, and other data structure operations.

The contract says that If two objects are equal according to the equals(Object) method, then calling the hashCode method on each of the two objects must produce the same integer result. It also says that the hashCode method must consistently return the same integer, provided no information used in equals comparisons on the object is modified.

It further says that It is not required that if two objects are unequal according to the equals method, then calling the hashCode method on each of the two objects must produce distinct integer results.

So equal objects must have equal hashCodes. An easy way to ensure that this condition is always satisfied is to use the same attributes used in determining equality in determining the hashCode. Now we should realize why it is important to override hashCode every time we override equals.

The story of hashtable and buckets .. (Some useful history) -
we can think a hash table as a group of buckets. When you add a key-value pair, the key's hashCode is used to determine which bucket to put the mapping.
Similarly when you call the get method with a key on the hash table, the key's hashCode is used to determine in which bucket the mapping was stored. This is the bucket that is searched (sequentially) for the mapping.
If you have two "equal" objects but with different hashCodes, then the hash table will see them as different objects and put them in different buckets. Similarly you can only retrieve an object from a hash table by passing an object with the same hashCode as the object you are trying to retrieve. If no matching hashCode is found, null is returned.
So let's say it again, "Equal objects must have equal hashCodes".

The best hashcode approach -
We should try to make all unequal objects have unequal hashCodes. This means each mapping is stored in its own bucket. This is the optimal case for the hash table and results in linear search times because only the correct bucket needs to be searched for. Once the correct bucket is found, the search is complete. That's why the API docs said

"However, the programmer should be aware that producing distinct integer results for unequal objects may improve the performance of hash tables."

How do I implement hashCode ?
Well we want it to be linked to the equals method in some way and so it must use the same attributes as the equals method.

The hashCode method signature in Object class is -
public native int hashCode();

The key thing to note here is that the method returns an integer. This means that we should try to get an integer representation of all the attributes that were used to determine equality in the equals method. The trick is that we should get this integer representation in a way that ensures that we always get the same int value for the same attribute value.
Once we have the integers, it's up to us to find a way of combining them into one integer that represents the hashCode for our object.

Whatever algorithm we use, we must make sure that the result is always an integer and will be the same integer returned for equal objects.
So how do we determine the hashCodes for the attributes themselves?

For the individual attributes values, you can use the following popular approach
(Source: http://bytes.com/topic/java/insights/723476-overriding-equals-hashcode-methods )

For boolean variables use 0 if it's true and 1 if it's false.

Converting byte, char or short to int is easy. Just cast to int. The result is always the same for the same value of the attribute.

A long is bigger than an int. You can use (int)value^(value >>> 32) . This is the method used by the java.lang.Long class.

If the field is a float, use Float.floatToIntBits(value).

If the field is a double, use Double.doubleToLongBits(value), and then hash the resulting long using the method above for long type.

If the field is an object reference and this class's equals method compares the field by recursively invoking equals, then recursively invoke hashCode on the field as well.

If the value of the field is null, return 0 (or some other constant, 0 is more common but you might want to distinguish it from the boolean case).

Finally, if the field is an array, go through each element and compute each element's hashCode value. Use the sum of the hashCodes as the hashCode for the array attribute.

Let's look at some ways of doing it.
A common approach is to choose a multiplier, say p, and then compute an int value by applying the following formula
hashCode = multiplier * hashCode + attribute's hashCode for all the attributes.

For three atributes (a1, a2, a3), the hashCode would be computed in the following steps
hashCode = multiplier * hashCode + a1's hashCode //step 1
hashCode = multiplier * hashCode + a2's hashCode //step 2
hashCode = multiplier * hashCode + a3's hashCode //step 3

Now putting it all together our Student class will look like this. .

public class Student {
String id;
String name;
int age;
private volatile int hashCode = 0;

public Student(String name , String id, int age) {
this.name = name;
this.id = id;
this.age = age;
}

String getName() {
return name;
}

int getAge() {
return age;
}

String getId() {
return id;
}

public boolean equals(Object obj) {
if(this == obj) {
return true;
}
if (!(obj instanceof Student)) {
return false;
}
Student student = (Student)obj;
return age == student.getAge() && name.equals(student.getName())
&& id.equals(student.getId());

}

public int hashCode () {
final int multiplier = 23; // we should use any prime number here like 23 or 31 etc
if (hashCode == 0) {
int code = 133;
code = multiplier * code + age;
code = multiplier * code + id.hashCode();
code = multiplier * code + name.hashCode();
hashCode = code;
}
return hashCode;
}

}

To test we can use a main() method in the above class like this -

public static void main(String args[]){
Student student1 = new Student("sajid", "456789", 28);
Student student2 = new Student("sajid", "456789", 28);
System.out.println(student1.equals(student2)); // prints true
}

Another way to implement hashCode would be
Similar to how we implemented hashCode our hash code calculation must involve all attributes of the class that contribute to the equality comparison of the class. For Car class example, the following would work:
public int hashCode()
{
int hash = 7;
hash = 31 * hash +
(null == this.licensePlate ? 0 : this.licensePlate.hashCode());
hash = 31 * hash +
(null == this.vinNumber ? 0 : this.vinNumber.hashCode());

return hash;
}

Java provides an implementation of hashCode for built-in classes such as String and other wrapper classes. If you have an int data type that is part of your equals comparison you can simply add, subtract, multiply, or divide it into your hash code value.
Several APIs demand that the user must implement the hashCode() method. The reason is that these APIs are using hash based containers (like HashMap) to have a fast means of managing lots of objects (always comparing objects using equals() would need endless time).

I will try to further dig into the internals of hashing and difference between different hash based collections and the best performance strategies around it.

Lets come forward

noreply@blogger.com (Nads) — Wed, 27 Jan 2010 12:36:00 GMT

I don't know but it really gives me eternal happiness to help people around and i feel that's the way to live.
we should always be concerned about others around. in recent times i have come across quite some statements , calls , adds , interview which attracts me to come forward for such good cause.

I came across -
CHILDLINE (1098) -- Country's first toll-free tele-helpline for street children in distress. just dail 1098.
WorldVision -- Here you can sponsor a child across the world , you can donate money and much more.

I am trying to spread the word here and would like to get involved with them. i think your contribution is important no matter big or small.
I also feel that these organizations need to come forward and bring greater transparency to common people. there is help available but most of the time people dont know about it.
I request you all to share your experiences and association if any with any such organizations/NGOs.

The brand Sapient

noreply@blogger.com (Nads) — Wed, 20 Jan 2010 10:32:00 GMT

Its been more than a year now at Sapient , i still remember few people used to tell me don't change company ,its not a good time(at end of 2008) , few told that you cant even survive 6 months at sapient due to slogging etc.. but i feel i made a right move and joined the right company.
I can say its my kind of place. some of the things I really enjoy here is the open culture and unlimited freedom.
Sapient is very unique in its own way. we follow different kind of processes here to make things work better.

Some of the terms i learned/heard being at sapient (call it sapient lingo)--
Retrofit
Caveats
Sync-up
Gated
Head-on
Triage

I wanna thank Rupinder for providing me an opportunity to join Sapient in the worst of economic conditions way back in Oct,2008.
i am lucky indeed :-)

Why just 3 Idiots? Why not the entire nation?

noreply@blogger.com (Nads) — Mon, 28 Dec 2009 18:20:00 GMT

I truly enjoyed the latest flick of Aamir, the preacher. It’s interesting to see Bollywood coming up with good messages and this time for better education system. Its second tryst by Aamir, the first one was taare zamine per, where he tried to show the challenges around a student’s life and now again in 3 idiots he has brought some good points about the process we follow in colleges.
Although this movie is loosely based on Chetan bhagat’s 5 point someone but it has got its own charm and an entertaining way to put across the strong message. I believe it’s a good eye opener for all Indians including students, parents, teachers and honorable education minister. It a must watch for one and all. If you think you will go and get tickets at show time then you are wrong. Go with an advance booking. By the way I managed to get ticket in black :-)

Signed Applet and how to make a signed jar

noreply@blogger.com (Nads) — Tue, 22 Dec 2009 12:05:00 GMT

A signed Applet is a trusted Applet (applet will reside inside a jar). In order to get access to the file system all jar files in an application must be signed. By default for security reasons, Java applets are contained within a sandbox.

This means that the applets can’t do anything, which might be threatening to the user’s machine (e.g. reading, writing or deleting local files, putting up message windows, or querying various system parameters).

Early browsers had no provisions for Java applets to reach outside of the sandbox. Recent browsers, however have provisions to give “trusted” applets the ability to work outside the sandbox. For this power to be granted to one of your applets, the applet’s code must be digitally signed with your unforgettable digital ID, and then the user must state that he trusts applets signed with your ID. The mistrusted applet can request to have privileges outside the sand box but will have to request the user for privileges every time it executes. But with the trusted applet the user can choose to remember their answer to the request, which means they won’t be asked again.

Let’s see what it takes to create your own signed applets. There are few tools given by Sun to create your own signed applets. It mainly comprise of three main steps.

(1) First step is generating a keypair with the keytool.

D:/j2sdk1.5/bin/keytool -genkey -alias green -keypass greenpass

green is the alias for the keypair, and greenpass is the password to access that alias(or keypair). The keypair
will be stored in the file .keystore in your homedirectory unless you specify otherwise.
You'll be asked a lot of questions, you may lie or just hit enter!

(2) Create a self-signed certificate.

D:/j2sdk1.5/bin/keytool -selfcert -alias green

(3) After the keypair has been generated the jar file can be signed with jarsigner.

jarsigner YourJarFileName alias

d:/j2sdk1.5/bin/jarsigner Test.jar green
It will ask for both passwords specified while generating the keys.

The jar file has now been signed. Now you can distribute this jar file to anyone or it can run on any system with the privilege to access your file system.

Note: this signature is only valid for 6 months. After that it will expire so you probably need to sign it again after 6 months.

New features in Java 6.0 (Mustang)

noreply@blogger.com (Nads) — Tue, 3 Nov 2009 14:10:00 GMT

In this Java release SUN has not done much changes at language level rather major enhancement is done in core, XML and desktop sections.

Any feature or enhancement in Java is encapsulated in the form of a JSR (Java Specification Request) which details the need for a specific functionality to be available in the Java Platform that can be used by Applications. These JSR's will be reviewed and released by Java Expert Groups (JEG).

Following the major list of features (JSRs') that comes with the Java 6.0.

Java Compiler API (JSR 199)
JDBC 4.0 (JSR 221)
Scripting in the Java Platform (JSR 223)
Pluggable Annotation Processing API (JSR 269)
Common Annotations (JSR 250)
Java API for XML Based Web Services - 2.0 (JSR 224)
JAXB 2.0 (JSR 222)
Web Services Metadata (JSR 181)
Streaming API for XML (JSR 173)
XML Digital Signature (JSR 105)
Java Class File Specification Update (JSR 202)

I will try to explore and explain each of these JSR's in my future articles.
The latest java 6.0 can be downloaded from here. I am really excited about scripting framework, scripting API and new JDBC 4.0 API in this release.

List of top 10 features in Java SE 6.0 adapted from SUN's site

Web Services
You get first-class support for writing XML web service client applications (death of Apache Axis?). No more messing with the plumbing (unless you really want to). You can also expose your APIs as .NET interoperable web services with a simple annotation.
For developers who want to handle XML directly Mustang adds new parsing and XML to Java object-mapping APIs, previously only available in Java EE platform implementations or the Java Web Services Pack.

Scripting
You can now mix in JavaScript technology with your Java technology source code, useful for prototyping. Also useful when you have teams with a variety of skill sets.

Database
The final Mustang development kit will co-bundle the all-Java JDBC database, Java DB based on Apache Derby. This should enable developers (new to Java) to get a jumpstart on java development.
Developers will get the updated JDBC 4.0, which focuses on ease of use. It contains many feature additions like support for XML as an SQL datatype and better integration of Binary Large OBjects (BLOBs) and Character Large OBjects (CLOBs) into the APIs. Additional features that improve ease of use include removal of some JDBC boilerplate and some of the new annotations that make SQL strings embed better into your JDBC application - like decorating your getAllUsers() method with an @Query(sql="select * from user") annotation, and that being all you need. More annotation love for you.

More Desktop APIs
GUI developers get a large number of new tricks to play like the ever popular yet newly incorporated SwingWorker utility to help you with threading in GUI apps, JTable sorting and filtering, and a new facility for quick splash screens to quiet impatient users.

Monitoring and Management
The really big deal here is that you don't need do anything special to the startup to be able to attach on demand with any of the monitoring and management tools in the Java SE platform. Mustang adds yet more diagnostic information, and we co-bundled the infamous memory-heap analysis tool Jhat for forensic explorations of those core dumps.

Compiler Access
Really aimed at people who create tools for Java development and for frameworks like JavaServer Pages (JSP) or Personal Home Page construction kit (PHP) engines that need to generate a bunch of classes on demand, the compiler API opens up programmatic access to javac for in-process compilation of dynamically generated Java code. Finally you don't have to save your code as a Java files and invoke javac to generate a classfile, a clumsy procedure at best.

Pluggable Annotations
Java tool and framework vendors (and you) can define annotations and have core API support for plugging in and executing the processors that do the heavy lifting. It seamlessly integrates your custom annotations .

Desktop Deployment
Better platform look-and-feel in Swing technology, LCD text rendering (more clarity on LCD monitors like Vista), and snappier GUI performance overall. Java applications can integrate better with the native platform with things like new access to the platform's System Tray and Start menu. At long last, Mustang unifies the Java Plug-in technology and Java WebStart engines, which just makes sense. Installation of the Java WebStart application got a much needed makeover.

Security
Mustang simplified the job of security administrators by providing various new ways to access platform-native security services, such as native Public Key Infrastructure (PKI) and cryptographic services on Microsoft Windows for secure authentication and communication, Java Generic Security Services (Java GSS) and Kerberos services for authentication, and access to LDAP servers for authenticating users.

Quality, Compatibility, Stability
Sun has around 80,000 test cases and several million lines of code testing conformance. People have been downloading (and testing) snapshots of Mustang for the last 15 months and filing bugs. So even before beta several quality and regression issues were fixed. Performance is claimed to be better than J2SE 5.

LAKSHYA program by HCL for freshers

noreply@blogger.com (Nads) — Fri, 30 Oct 2009 07:41:00 GMT

HCL has introduced a unique program called LAKSHYA for 2009 and 2010 BE and MCA freshers where by they are proving an assured placement after the completion of this program. other features include live industry project wi-fi campus.

To register visit click here. see the details below in their advertisement.

One thing i don't appreciate about this program is that students need to buy HCL laptops.
i don't know why people have hidden agenda behind good work. I feel HCL is doing this program to boost their own revenues rather than helping the fresh talents :-)

Stock details on Gtalk by Zibika.com

noreply@blogger.com (Nads) — Wed, 26 Aug 2009 13:53:00 +0100

It's really cool to see Zibika stockbot, a chat-based stock search service that allows users to check the current stock prices on BSE and NSE through Google Talk.
You just need to add market.zibika@gmail.com as a friend in GTalk.
Its so easy to use it, now no need to login to different sites to check stock prices. They have few very simple commands like

Sensex --> gives you sensex current points
Nifty --> gives you nifty current points
BSE jindal steel --> gives you current price of jindal steel on BSE
NSE axis bank --> gives you current price of axis bank on NSE

Type BSE or NSE followed by the company name.

I think stockbot is a good idea, since many people check stock updates on their mobile, it becomes easy and convenient. Secondly if you are traveling then you don't need to open your laptop and check for the updates.

See the screen below.

TATA second career for women professionals

noreply@blogger.com (Nads) — Fri, 14 Aug 2009 06:47:00 +0100

Tata group, a well know name and pride of nation. Truly Tata’s are the creative thinkers.You name it they have it. From steel to cars, from software to hardware they are into everything. The brand is so huge that it has impact on our daily lives. The good thing is that they into education as well. Their new initiative is really a big leap towards women empowerment. Yes I am talking about second career program for women by Tata group.

This programme is currently open only to women who have work experience of at least 4 continuous years or more in specific domain areas.Currently SCIP II is open only to candidates who are currently residing in Mumbai and Bangalore. SCIP II is extension of SCIP which once again focuses on women professionals. This program is intended to spread over 6 months.

There is no placement guarantee at the end of the project.However, the women professionals have the option of exploring full-time employment on mutually acceptable terms with the respective Group Company.

They are looking for people from various fields like Marketing , Sales , Advertising / Communications, Human Resource Management , Finance / Accounting, Legal , Manufacturing , Engineering , IT , Corporate Planning, Hospitality , Retail , CSR , Other Corporate Functions.

Main highlights of the program

Attractive project stipend up to Rs. 4 lakhs
Flexi-time ‘Live’ business projects in Mumbai & Bangalore
Project presentation to Tata Team
Management Development Programme conducted by Tata Management
Training Centre
HR helpline during the programme
Work certificate

Last date for registration 20^th Aug. Visit the FAQ section for more details on their site and spread the word