Amila Paranawithana's Blog

Securing API access in AdroitLogic Management API

2016-11-30T01:20:00.002-08:00

Securing the API access with JWT based Authentication

Management server REST API requests can be secured with JWT authentication. By default authentication is disabled and you can enable it by changing the ''console.auth.jwt.on" property to "true" in management.properties file.

Following diagram shows the main authentication flow of application.

1. Receive username, password from web interface on user login

2. Authenticate the user from shiro realm (Supported file based user store realm)

3. Successful authentication by the ldap server

4. Create a JWT token including the following claims

- username, user_permissions, user_ldap_groups, issuer (configurable), expiration_date (configurable)

5. Signs this JWS token with the specified Message Authentication Code (MAC) signer using a shared key

6. Encrypt the token with Base64 with a hashkey
7. Save the encrypted token over username in in-memory cache(This is to avoid disk access on later for token validation)

8. Return the encrypted token to the user

After user get a valid token at the first login, he can use the same token in next logins until the token expires. Following diagram shows the authentication flow when user request come with a JWT token.

1. Request is filtered by JWTAuthorizationFilter.

2. JWT token extracted from AUTHORIZATION header

3. Verify the token whether the token is expired and similar to the token issued by the application earlier. (first compares with token saved in the cache)

4. Decrypt the token
5. Token values are fed to AuthDataContainer object (these information is used over the application for authorization purpose and this is valid only for a one session)

6. Authentication success returned to the user

Following table explains how the token request and a sample api call (get request) is done.

Authentication	Request	Response
Token request	POST /management/auth HTTP/1.0 Connection: close Content-Length: 42 Content-Type: application/json; charset=UTF-8 Host: localhost:8011 User-Agent: <> {"username":"admin","password":"password"}	HTTP/1.1 200 OK Set-Cookie: JSESSIONID=1qj32rt2mppjfjhnah8zre3st;Path=/ Expires: Thu, 01 Jan 1970 00:00:00 GMT Set-Cookie: rememberMe=deleteMe; Path=/; Max-Age=0; Expires=Tue, 29-Nov-2016 06:55:02 GMT Content-Type: application/json Content-Length: 982 Server: Jetty(9.2.3.v20140905) {"msg":"bDd6RllXSX……..."}
API call (GET)	GET /management/project HTTP/1.0 Connection: close Authorization: bDd6RllXSX……... Host: localhost:8011 User-Agent: <>	HTTP/1.1 200 OK Set-Cookie: JSESSIONID=17n23j1souoiraqsompt9jah5;Path=/ Expires: Thu, 01 Jan 1970 00:00:00 GMT Set-Cookie: rememberMe=deleteMe; Path=/; Max-Age=0; Expires=Tue, 29-Nov-2016 09:16:21 GMT Content-Type: application/json Content-Length: 164 Server: Jetty(9.2.3.v20140905) <response body...>

Security related properties in management.properties file

Property	Description
console.auth.jwt.on	‘True’ or ‘false’ to enable or disable api security
jwt.lifetime	JWT token lifetime in milliseconds
console.auth.url	Authentication request URL
console.auth.unauthorized	Path to direct for unauthorized requests
shiro.users	Users and password file location. Each user need to be added in a new line with the format <user><SPACE><password>
shiro.permissions	Users and permission file location. Each user permission need be added in a new line with the format ‘<username><SPACE><comma separated permissions>’
jwt.encryption.key	JWT token encryption key
jwt.shared.key	JWT shared key

FIX Connector in AdroitLogic - ProjectX

2016-11-28T01:54:00.007-08:00

AdroitLogic ProjectX supports FIX message transferring with new FIX connector introducing with ProjectX-Beta1 release.

FIX connector configuration for the ESB with UltraStudio.

Create a new project with adding the FIX connector to connector list.

Configuring the project.xpl

Add the following resources which will be referred project spesific configuration file .xcml

Resource	Type	Resource class type
fileMsgStore		org.adroitlogic.x.transport.fix.util.FileMessageStoreFactoryImpl
fileLogFactory		org.adroitlogic.x.transport.fix.util.FileLogFactoryImpl
SessionBANZAI1		org.adroitlogic.x.transport.fix.FIXSession
senderOtherSettings		java.util.Map
EndpointOtherSettings		java.util.Map
configuredSessions		java.util.List
otherAcceptorSettingss		java.util.Map

The following is the final .xpml file which will be created while configuration is done through the user interface.

<x:project id="StudioProj2" name="StudioProj2" version="1.0-SNAPSHOT"

xmlns="http://www.springframework.org/schema/beans"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:x="http://www.adroitlogic.org/x/x-project"

xsi:schemaLocation="http://www.adroitlogic.org/x/x-project http://schemas.adroitlogic.org/x/x-project-1.0.0.xsd

http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.1.xsd">

<x:description>A sample project that demonstrates the projects concept</x:description>

<x:flows>

<x:flow id="test" file="test.xcml"/>

</x:flows>

<x:resources>

<x:resource id="fileMsgStore">

</bean>

</x:resource>

<x:resource id="fileLogFactory">

</bean>

</x:resource>

<x:resource id="SessionBANZAI1">

</bean>

</x:resource>

<x:resource id="senderOtherSettings">

<map>

</map>

</x:resource>

<x:resource id="EndpointOtherSettings">

<map>

</map>

</x:resource>

<x:resource id="configuredSessions">

<list>

</list>

</x:resource>

<x:resource id="otherAcceptorSettingss">

<map>

</map>

</x:resource>

</x:resources>

</x:project>

Configuring a simple message flow path

This post explains a simple message flow to mediate fix message transaction with a fix-listener to listen to incoming messages over FIX protocol and a fix-sender to send-out the fix messages to outside receivers over fix protocol.

Configuring FIX Ingress connector

IngressConnector is the message entry point for fix messages. You can define the configuration in the ESB fix message acceptor and other fix ingress connector configuration through the tool user interface.

Configuring FIX Egress connector

EgressConnector is the message out port of ESB for fix messages. You can define the configuration in the ESB fix message initiator and other fix egress connector configuration through the tool user interface.

Following is the final .xcml file that will be created after configuring the above

<beans xmlns="http://www.springframework.org/schema/beans"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.1.xsd">

</bean>

</bean>

</beans>

Testing the message flow

Download QuickFIX/J and run the sample they have provided(Banzai-Executor) with modifying configuration as bellow.Here we are sending a message from Banzai(initiator to port 12000) to UltraESB(working as an acceptor listening in port 12000) over session FIX.4.2. After receiving the message from Banzai UltraESB(working as an initiator) will forward the message to Executor(acceptor listening in port 9879)

Initiator configuration for Banzai (banzai.cfg)

[default]

FileStorePath=examples/target/data/banzai

ConnectionType=initiator

TargetCompID=UESB

SocketConnectHost=localhost

StartTime=00:00:00

EndTime=23:00:00

HeartBtInt=30

ReconnectInterval=5

SocketConnectPort=12000

[session]

BeginString=FIX.4.2

SenderCompID=BANZAI

Acceptor configuration for Executor(executor.cfg)

[default]

FileStorePath=examples/target/data/executor

ConnectionType=acceptor

StartTime=00:00:00

EndTime=23:00:00

HeartBtInt=30

ValidOrderTypes=1,2,F

SenderCompID=EXEC

TargetCompID=UESB

UseDataDictionary=Y

DefaultMarketPrice=12.30

[session]

BeginString=FIX.4.2

SocketAcceptPort=9879

Following is the UI provided by QuickFIX/J sample, you can send a message with this UI over session FIX.4.2:BANZAI->UESB.

Welcome to CSE

2016-09-13T05:27:00.001-07:00

Welcome to CSE from Amila Paranawithana

Why we need to create indexes for foreign keys in Oracle

2016-08-19T03:14:00.004-07:00

This post is about a small POC I did to check the performance and the effect of indexes on foreign keys in a database. Usually databases like Jet Engine (Microsoft Access Files) Firebird and MySQL automatically creates indexes for foreign keys but in Oracle it is not.

Indexes on foreign keys is designed to avoid child table locking issue as if there is no index on the child table's foreign key, table-level locking may occur when a parent row is updated.

Lets do a quick demo on this.

Setting up the environment

I will be creating a small USERS table as the parent table and a BOOK table as the child table which has foreign key from the USERS table.

Lets create two tables USERS and BOOKS

CREATE TABLE APS_ESB.USERS (ID NUMBER, INDEX_NO NUMBER, NAME VARCHAR2(30));

CREATE TABLE APS_ESB.BOOKS (ID NUMBER, NAME VARCHAR2(30), USER_ID NUMBER, BOOK_CODE NUMBER);

Populate data to the USERS table

INSERT INTO APS_ESB.USERS (ID, INDEX_NO, NAME) VALUES (1,01,'AMILA1');

INSERT INTO APS_ESB.USERS (ID, INDEX_NO, NAME) VALUES (2,02,'AMILA2');

INSERT INTO APS_ESB.USERS (ID, INDEX_NO, NAME) VALUES (3,03,'AMILA3');

INSERT INTO APS_ESB.USERS (ID, INDEX_NO, NAME) VALUES (4,04,'AMILA4');

INSERT INTO APS_ESB.USERS (ID, INDEX_NO, NAME) VALUES (5,05,'AMILA5');

Populate data to BOOKS table with large number of records

-- inserting 1000000 rows to the table

INSERT INTO APS_ESB.BOOKS SELECT rownum, 'BLAH', 1, mod(rownum,2)+1 FROM dual CONNECT BY level <= 2000000;

COMMIT;

Add primary key and foreign key constraints

ALTER TABLE APS_ESB.USERS ADD PRIMARY KEY (ID);

ALTER TABLE APS_ESB.BOOKS ADD CONSTRAINT USERS_FK FOREIGN KEY (USER_ID) REFERENCES APS_ESB.USERS(ID);

Lets play with INDEX

Now there are no any indexes created so far. Lets try to delete data from the very small USERS (parent) table.

SET AUTOTRACE ON; and collect the statistics to compare.

DELETE APS_ESB.USERS WHERE ID=3; // query to delete a row from users.

Execution Plan

----------------------------------------------------------

Plan hash value: 880004690

----------------------------------------------------------------------------------

----------------------------------------------------------------------------------

| 0 | DELETE STATEMENT | | 1 | 13 | 0 (0)| 00:00:01 |

|* 2 | INDEX UNIQUE SCAN| SYS_C008544 | 1 | 13 | 0 (0)| 00:00:01 |

----------------------------------------------------------------------------------

Predicate Information (identified by operation id):

---------------------------------------------------

2 - access("ID"=3)

Statistics

----------------------------------------------------------

36 recursive calls

9 db block gets

2981 consistent gets

0 physical reads

636 redo size

839 bytes sent via SQL*Net to client

783 bytes received via SQL*Net from client

3 SQL*Net roundtrips to/from client

3 sorts (memory)

0 sorts (disk)

1 rows processed

Now lets create a INDEX to the foreign key in child table and check the statistics again.

CREATE INDEX BOOKS_IDX ON BOOKS(USER_ID); // create an index

Execution Plan

----------------------------------------------------------

Plan hash value: 880004690

----------------------------------------------------------------------------------

----------------------------------------------------------------------------------

| 0 | DELETE STATEMENT | | 1 | 13 | 0 (0)| 00:00:01 |

|* 2 | INDEX UNIQUE SCAN| SYS_C008544 | 1 | 13 | 0 (0)| 00:00:01 |

----------------------------------------------------------------------------------

Predicate Information (identified by operation id):

---------------------------------------------------

2 - access("ID"=3)

Statistics

----------------------------------------------------------

1 recursive calls

6 db block gets

1 consistent gets

2 physical reads

516 redo size

839 bytes sent via SQL*Net to client

783 bytes received via SQL*Net from client

3 SQL*Net roundtrips to/from client

1 sorts (memory)

0 sorts (disk)

1 rows processed

	consistent gets - Number of times a consistent read was requested for a block.
Without INDEX	2981
With INDEX	6

Explanation

We can only successfully delete a row in parent table if there are no corresponding FKs based on this parent row. When there is no associated index, Oracle perform this check on the large child table by performing an expensive, slow, Full Table Scan (FTS) because Oracle needs to ensure while it runs the FTS looking for any FKs associated with the deleted parent row.

When we create a INDEX , we can see a drastic reduction in the consistent gets from 2981 to 6. This means now Oracle does not perform any expensive FTS of the child table so the child table locking issues will not occur.

Docker for dummies

2016-08-02T04:17:00.004-07:00

1. Install docker. Use docker documentation.

2. Write a the docker file - refer
This is script we define what are the commands to run when initiating the container. This includes, some software instillations, environment variable setting, resource copying and main running program defining. It use a specific language and following are some of the meanings of the language syntax.

Meanings of commands
FROM - base image
RUN - to run commands in the terminal
ADD - Copy files
ENV - Setting environment variables
EXPOSE - ports to open
ENTRYPOINT - main program to run

eg: Following is a sample docker file which creates a container with java and tomcat installed and tomcat server started.

FROM ubuntu:14.04

MAINTAINER Dimuthu Upeksha <dimuthu@adroitlogic.com>

RUN apt-get update && apt-get install -y unzip

# Installing java

ADD jdk-7u75-linux-x64.tar.gz /opt

#RUN tar -xzf /tmp/jdk-7u75-linux-x64.tar.gz -C /opt

RUN ln -s /opt/jdk1.7.0_75 /opt/jdk

ADD UnlimitedJCEPolicyJDK7.zip /tmp/UnlimitedJCEPolicyJDK7.zip

RUN unzip /tmp/UnlimitedJCEPolicyJDK7.zip -d /opt

RUN cp /opt/UnlimitedJCEPolicy/*.jar /opt/jdk/jre/lib/security/

# Copying ultraesb binaries

ADD tomcat.tar.gz /opt

ADD start-tomcat.sh /opt/start-tomcat.sh

RUN rm -rf /tmp/*

ENV JAVA_HOME /opt/jdk

ENV PATH /opt/jdk/bin:$PATH

#ports to open in the container

EXPOSE 1-65535

ENTRYPOINT /bin/bash -c "sh /opt/start-tomcat.sh && tail -100f /opt/apache-tomcat-7.0.29/logs/catalina.out"

3. Build docker image

sudo docker build -t <image_name> // usual pattern, name/app_name:version

eg : sudo docker build -t amila/ultraesb:v1

4. Run docker image

sudo docker run -it<if u want a interactive console> --name <container name> <image_name>

eg : sudo docker run -it --name ultraesb dimuthuupe/ultraesb:v1 /bin/bash

sudo docker run -privileged -t -i dimuthuupe/ultraesb:v1 /bin/bash
After running this to check the created docker container use command 'docker ps' .
You can use the same image to create many containers with different names.

You can log in to the created container with command
docker exec -it <container_name> /bin/bash

5. Port mapping

sudo docker run -t -i -p 8080:8080 dimuthuupe/ultraesb:v1 /bin/bash

Writing grails code to be compatible with both MySQL and Oracle database

2016-06-28T00:16:00.001-07:00

Domain classes

1. [constrants] - If the id is auto generated , add the generator: "assigned"

constrain to id field

2. [mappings] - Make sure to add the table name in the mapping

3. [mappings] - If the id is auto generated type, add the generator: "increment" mapping

4. [mappings] - Must add the column name mapping. Character case is not important.

eg : name column: "name" can map to a 'NAME' column in oracle.

5. [mapping] - If there are any join tables, add the relevant mapping.
6. When giving names for the column names, remember that the maximum character length for oracle is 30.

Services

Usually database transactions are done in the service layer. Few things to be remembered when writing your code to be compatible with both databases are listed below.

1. If you are creating new database entries, first create the entry without setting foreign dependencies, then make sure to save the object first and then set the other foreign dependencies to the saved and again obtained object.

Ref - Change Schemas online - http://www.sqlines.com/online
Ref - MySQL to Oracle Migration - http://www.sqlines.com/mysql-to-oracle

User management in AdroitLogic IPS (Integration Platform as a Service)

2016-03-21T04:38:00.003-07:00

Adroitlogic Integration Platform as a Service (IPS) is the solution to deploy UltraESB as a service in a private/public cloud environment. This post describe about how the user management aspect is addressed in the current solution.

JWT based Authentication

Authentication of the application happens based on JWT tokens. Following diagram shows the main authentication flow of application.

Authentication flow

1. Receive username, password from web interface on user login

2. Authenticate the user from shiro realm (Supported LDAP realm)

3. Successful authentication by the ldap server

4. Create a JWT token including the following claims

- username, user_permissions, user_ldap_groups, issuer (configurable), expiration_date (configurable)

5. Signs this JWS token with the specified Message Authentication Code (MAC) signer using a shared key

6. Encrypt the token with Base64 with a hashkey
7. Save the encrypted token over username in redis cache(This is to avoid disk access on later for token validation)

8. Return the encrypted token to the user

Authentication with token

1. Request is filtered by JWTAuthorizationFilter.

2. JWT token extracted from AUTHORIZATION header

3. Verify the token whether the token is expired and similar to the token issued by the application earlier. (first compares with token saved in the cache)

4. Decrypt the token
5. Token values are fed to AuthDataContainer object (these information is used over the application for authorization purpose and this is valid only for a one session)

6. Authentication success returned to the user

LDAP Groups, ROLEs and Permissions

Every project defined in the IPS has collaborators/users and each of these users belongs to a specific LDAP GROUP. Each LDAP group has a set of ROLES which is defined by the admin. Permissions that belongs to each ROLE are defined by the admin. The following diagram shows the the relationship between the Clusters, Projects, LDAP_Groups, Roles and Permissions.

Permission model

Fine grained authorization with Apache Shiro

Each action of the application has a defined permission. Any user needs to access any resource/ perform any action on a resource need to have that particular permission defined in his ROLE.

Create a ROLE

Admin of the application can create new ROLEs as required and assign the permissions as below. Permissions are categorized in to 8 groups for the convenience. Users will get the union of the permissions belongs to his ROLEs.

Create a new ROLE

List Roles

Create a LDAP_GROUP

LDAP group represents a group created in real LDAP server. The name of the LDAP group should match the group name in real LDAP server. Also you need to define what are the ROLEs belongs to this group so that at user login user will be given these roles according to their LDAP group.

Create a new LDAP Group

List existing LDAP groups

Secure WebSphere Application Server JMS queue with LDAP authentication

2015-11-29T21:57:00.003-08:00

This post explains how to secure a JMS queue created on IBM WebSphere application server with LDAP authentication. This post assumes that WebSphere application server is already installed and will describe only on how to create a JMS queue with enabling LDAP authentication with using the WebSphere user interface.

1. Create a new SI Bus
    Service Integration --> Buses --> [New]
   Give a name for new bus (SampleBUS) and keep the “Bus Security” option checked {Next}
   Keep the “Require clients use SSL protected transports” option checked {Next}

2. “Create a new security domain” for the bus {Next}

By giving a name (ReachBUSDomain). To select the security mechanism to LDAP , select “Customize the user realm for this domain.”, then select “Standalone LDAP registry” from the combo box. {Next}

Then in the coming form fill up the information to configure the LDAP registry.

3. To add security to the JMS queue you created earlier.

Go to Service Integration --> Buses --> SampleBUS --> Security and configure the values as shown in the image below.

4. Add a permitted transport for the bus

Service Integration --> Buses > SampleBUS > Security for SampleBUS > Permitted transports > [New]

Select transport chain name as “InboundBasicMessage” {OK}

5. Create a bus member and a queue

Service Integration > Buses > ReachBUS > Bus Members > [Add]

Select “Server” with “adroitlogicNode1:server1” {Next}

Select “File Store” {Next}

Configure file store settings (default) {Next}

Tune performance parameters (default) {Next}

{Finish}

Go into the bus member “adroitlogicNode1:server1” Server and check the status of the message engine “adroitlogicNode1:server1-ReachBUS”

If it is unavailable, Save changes and go again into the bus member and check.

6 . Create a queue type destination in bus

Service Integration > Buses > ReachBUS > Destinations > [New]

Select destination type “Queue” {Next}

Give an identifier (ReachSTQ) & description for the queue {Next}

Assign queue to bus member “Node=adroitlogicNode1:Server=server1” {Next}

{Finish}

7 . Create a new queue connection factory

Resources > JMS > Queue connection factories

Keep scope selected as “Node=adroitlogicNode1:Server=server1”

[New]

Select “Default Messaging Provider” {OK}

Name: RCF.Test, JNDI Name: jms/RCF.Test, Provider endpoints: trojan2: 7276

Select bus name “ReachBUS”

{OK}

8. Create the queue

Resources > JMS > Queues

Keep scope selected as “Node=adroitlogicNode1:Server=server1”

[New]

Select “Default Messaging Provider” {OK}

Name: ReachSTQ, JNDI Name: ReachSTQ

Select bus name “ReachBUS”

Select queue name “ReachSTQ”

{OK}

9. Configure BUS authorization

Service Integration > Buses > ReachBUS > Security > Users and groups in the bus connector role > [New]

Select resource as “ReachBUS” > “Users” {Next}

Select users to grant access {Next}

{Finish}

10. Configure BUS destination (queue) authorization

Service Integration > Buses > ReachBUS > Security > Manage destination access roles > ReachSTQ [Add]

Select resource as “ReachSTQ” > “Users” {Next}

Select users to grant access. {Next}

Select the role types to be granted to the users. See Appendix to know more about Messaging Roles {Next}

11. Create and apply activation specification

Resources > JMS > Activation Specifications > [New]

Select “Default messaging provider” {Next}

Name:ReachActicationSpec, JNDI Name: ReachActicationSpec, Destination Type: Queue, Destination JNDI Name: ReachSTQ, Bus Name: ReachBUS {OK}

.startServer server1 -profileName AppSrv01

12. Save changes and restart the WebSphere Application Server

AdroiltLogic - Integration Monitor

2015-09-20T23:07:00.002-07:00

Adroitlogic - Integration Monitor (AIM) is an independent Web Application that allows easy management and monitoring of your Enterprise integrated with UltraESB.

This is a paid solution that any UltraESB customer can use. This solution can connect to UltraESB nodes via JMX (must have JMX enabled in ESB nodes) so can be hosted in a same or remote node such as data-center, developer workstation or can be started as a demand process.

Main features provided are,

Metrics

AIM provide ESB node wise and cluster space wise metrics with nice graphs. Under this message specific metrics such as received & sent messages, work manager tasks, file cache usage, error count, and total latency... and some system status statistics such as CPU usage, heap usage, thread usage, file descriptor usage, ESB worker manager tasks, ESB file cache usage can be monitored with graphs where the user can filter details by date range, get min/max/avg values for the selected time period. Refer this post to see more on statistics.

Log Management

AIM provide useful features for monitoring and management of system logs ,message logs via UI. It allow viewing details of Log4J loggers, log appenders and the tail end of the log file while allowing entries to be filtered easily on the severity level, change log level and change additivity.

Cluster Management

Easy management and monitoring of the ESB cluster is also possible with AMI. Viewing the details of nodes categorized under cluster-sapce/cluster - nodeGroup , view statistics, check the current status of nodes and viewing the cluster command history are the main features allowed.

ESB Proxy services, Sequences, Endpoints and Addresses Management.

AIM provides a graphical interface to view details and manage proxy services , sequences , endpoints and addresses defined in the server. These can be viewed categorized in cluster-space and deployment units and also can perform cluster- space wise operations (start/stop/debug-on/debug-off) on these artifacts via the UI.

Troubleshoot Reporter

User can select an active ESB node and then select a list of tasks to be executed on the node such as Patch Library Archival, Detailed Log Sampling, configuration directory archival, JVM heap dump, JVM thread dump, log archival, system information, UltraESB information and custom library archival.

Audit Logs

A log on actions performed by user and a detailed log on received messages can be viewed/downloaded with the Audit Log feature. Also it provides the capability to filter audit logs on user preferred fields.

Sending and receving messages with Adroitlogic AS2 Gateway

2015-03-19T22:54:00.002-07:00

This post describe how you can use Adroitlogic AS2 Gateway for transferring messages over AS2 Protocol.

To demonstrate this I am using the free and open source AS2 communication software product "mendelson AS2" from mendelson-e-commerce GmbH.

1. First register the AS2 Gateway
2. Create a station. This will act as one end of AS2 stations.
3. Install mendelson

- Download mendelson
- Follow following commands to run mendelson
>> mkdir mendelson
>> cd mendelson
>> unzip /tmp/install_mendelson_opensource_as2_1.1b47.zip
>> chmod +x *.sh
>> ./mendelson_as2_start.sh

4. By default there is a partner 'mypartner' created in mendelson, I will change it as 'mendelson' in this demo. And using one of default keys 'key1' in mendelson as keys for this station. First we will see how to send AS2 Messages over http

5. Export the public certificate for 'key1' of partner 'mendelson' from Export-->'Export Certificate(for your trading partner)'

6.Create a partner in AS2 Gateway for mendelson with giving above exported key as 'Partner Certificate' and giving mendelson as 'Trading Partner Name' and 'AS2 Identifier'

7. Create a partner in mendelson to receive messages from AS2 Gateway

steps
- Go to your station in AS2 Gateway and view partner configuration of your station, and download 'Partner Certificate' as Pem or Dem

- Go to mendelson and add the above certificate to mendelson certificate store
- Go to mendelson and create a new partner to receive messages from AS2Gateway
Misc -->Name = AS2 Identifier shown in AS2Gateway partner config
Security --> Partner Certificate(encryption/signature) = certificate you added above
Send --> Receipt URL = Partner URL shown in AS2 Gateway partner config

Now you are ready to send a AS2 message from mendelson station to your station host in AS2 Gateway. You can go to File --> Send to partner and from the pop up, select the partner you set up (adroitlogic.com) and send your message.

Sending AS2 over HTTPS

Similar to above create a partner in mendelson to send over https. For that HTTPS url (https://as2gateway.org:8443/service/as2-receiver ) as the Send -->receipt URL in your mendelson partner

Additionally we need to tell msndelson to trust the certificate of partner from AS2Gateway. For that simply you can type 'https://as2gateway.org:8443/service/as2-receiver ' in your browser and download the certificate and add that certificate in mendelson SSL certificate store.

With that setting you can send AS2 Messages over HTTPS from your mendelson station to partner in AS2Gateway.

Identifying Performance Deviations in Thread Pools

2015-03-18T21:42:00.001-07:00

Presentation slides for research paper 'Identifying Performance Deviations in Thread Pools' by Mark D. Syer, Bram Adams and Ahmed E. Hassan
Software Analysis and Intelligence Lab (SAIL),School of Computing, Queen's University, Canada.

FIX(Financial Information eXchange) protocol supporting Transport in UltraESB

2014-08-03T03:11:00.003-07:00

UltraESB supports FIX Message transfer and is available with V2.2.0. This is implemented with the QuickFixJ open source FIX implementation.

As shown in the in the below diagram, instead of directly talking to the FIX service , UltraESB acts as the proxy for all the fix messages going to the back-end fix endpoint.

Sample Configuration

The configuration for this use case consists of a proxy service identified as "fix-proxy", exposed on the FIX transport and forwards the incoming message into the target back-end service configured using the in destination of the proxy service.

FIX Transport Listener configuration

Fix Transport listener work as the fix message acceptor in UltraESB. Configuration properties you define in the transport listener <property> tags are used as [Default] properties for creating socket acceptor in QuickFix/j. All the messages receive through this transport listener will actually accept fix messages with socket acceptor created with using default properties same to properties given in transport-listener. There are essential properties need to set in order to create a QuickFIX/j socket acceptor, and other properties can be configured in the map named 'otherSettings'.

If you want to create several sessions through same acceptor those [Session] properties are configured within the proxy service under transport properties.

eg: If you have 2 sessions, one with FIX version 4.2 and other with FIX version 4.3, in proxy service define properties as following.

Property name suffixes are only to indicate the settings belong to a same session.
Property name prefix must be 'ultra.fix.session.' + property_name. To find allowed properties, refer 'FIX Transport'

FIX Transport Sender configuration

Fix Transport sender act as the initiator in UltraESB which initiate a session with a external fix endpoint and send fix messages it received. Configuration properties you define in the transport sender <property> tags are used as [Default] properties for creating socket initiator in QuickFix/j. All the messages send via this transport sender will actually send fix messages with socket initiator created with using default properties same to properties given in transport-sender. There are essential properties need to set in order to create a QuickFIX/j socket initiator, and other properties can be configured in the map named 'otherSettings'.

FIX Endpoint

This defines the final destination that the fix-message should receive.
There are three mandatory parameters (BeginString, SenderCompID and TargetCompID) need to create a new session. These three parameters must be given in the endpoint address in the format - fix://<host>:<port>?BeginString=<>&SenderCompID=<>&TargetCompID=< > . Other optional properties can be configured as <u:property.. in endpoint with property name , ultra.fix.endpoint.session.+ property_id.

property_id

property_id should be given in the same syntax defined in QuickFIX/J configuration. Refer QuickFIX/J configuration to find the exact IDs.

New initiator is created for each endpoint you define with using endpoint properties you define as [Session] settings and transport-sender properties as [Default] settings.

Make sure you don't define values for same property within the transport sender and , endpoint.

Setting Log factory

In QuickFIX/J logging can be done file base(fileLogFactory) or by saving to database(jdbcLogFactory). In this sample transport-listener and transport-sender configured to use file base logs (acc-fileLogFactory and ini-fileLogFactory). Following is how to configure fileLogfactory beans. Give the location of where to save the logs in fileLogPath property, here it is configured to '/tmp/fixTest'.

To persist loggin information to a database, define a 'dataSource' and 'fix-jdbcLogFactory' by setting required database information and give reference to 'logFactory' property in FIX transport listener and FIX transport sender. Sample configuration is included in sample-550 which you can un-comment and use.

Setting Message Factory

The message store is used by a fix session to store and retrieve messages for resend purposes. QhickFIX/J allow to store messages in file base(FileMessageStoreFactory) or in a database(JDBCMessageStoreFactory). In this sample transport-listener and transport-sender configured to use file base messageStores (acc-fileMessageStore and ini-fileMessageStore). Following is how to configure JDBCMessageStoreFactory beans. Give the location of where to save the messages in 'messageStoreFactory' property, here it is configured to '/tmp/fixTest'.

To persist messages in a database, define a 'dataSource' and 'fix-jdbcMessageStore' by setting required database information and giving reference to 'messageStoreFactory' property in FIX transport listener and FIX transport sender. Sample configuration is included in sample-550 which you can un-comment and use.

Sending sample FIX message through UltraESB

To issue a fix message you may refer the unit test FIXMessageExchangeTest.java.
Following is how to send a sample fix message through a external client, mediate message with UltraESB and forwarding to the destination fix endpont. To send and receive a sample fix message you can use samples provided by QuickFIX/J, BANZAI and EXECUTOR

1. Since UltraESB do not ship required jars to FIX transport, you need to add following jars in to 'lib/custom'

quickfixj-all-1.5.3.jar
mina-core-1.1.7.jar

2. Start the UltraESB sample configuration 550 via the ToolBox or on the command line as follows.

$ cd /opt/ultraesb-2.2.0/bin

$ ./ultraesb.sh -sample 550


Proper starting of FIX transport listener and FIX transport sender will log as follow. 



2014-04-09 11:45:05,348 [-] [main] [system] [000000I]  INFO FIXTransportSender Started FIX Transport sender : fix-sender...
2014-04-09 11:45:05,358 [-] [main] [system] [000000I]  INFO SimpleQueueWorkManager Started Work Manager : default
2014-04-09 11:45:05,359 [-] [main] [system] [000000I]  INFO ServerManager Initializing transport listeners
2014-04-09 11:45:05,364 [-] [main] [system] [000000I]  INFO ServerManager Starting Proxy Services, Endpoints and Sequences
2014-04-09 11:45:05,371 [-] [main] [system] [000000I]  INFO Address Started Address : address of endpoint : fix-endpoint
2014-04-09 11:45:05,371 [-] [main] [system] [000000I]  INFO fix-endpoint Endpoint : fix-endpoint started
2014-04-09 11:45:05,376 [-] [main] [system] [000000I]  INFO Address Started Address : address of endpoint : mediation.response
2014-04-09 11:45:05,376 [-] [main] [system] [000000I]  INFO Endpoint Endpoint : mediation.response started
2014-04-09 11:45:07,391 [-] [main] [system] [000000I]  INFO fix-proxy-inSequence Sequence : fix-proxy-inSequence started
2014-04-09 11:45:07,394 [-] [main] [system] [000000I]  INFO Address Started Address : address of endpoint : fix-proxy-outDestination
2014-04-09 11:45:07,394 [-] [main] [system] [000000I]  INFO fix-proxy-outDestination Endpoint : fix-proxy-outDestination started
2014-04-09 11:45:07,396 [-] [main] [system] [000000I]  INFO fix-proxy Proxy service : fix-proxy started
2014-04-09 11:45:07,396 [-] [main] [system] [000000I]  INFO ServerManager UltraESB root deployment unit started successfully
2014-04-09 11:45:07,396 [-] [main] [system] [000000I]  INFO ServerManager Starting transport listeners
2014-04-09 11:45:11,142 [-] [main] [system] [000000I]  INFO FIXSessionFactory Created new socket acceptor for transport listener: fix-lsnr
2014-04-09 11:45:11,234 [-] [main] [system] [000000I]  INFO FIXTransportListener Socket acceptor for fix transport listener: fix-lsnr started
2014-04-09 11:45:11,234 [-] [main] [system] [000000I]  INFO FIXTransportListener Started FIX Listener : fix-lsnr

3. Download QuickFIX/J and run the sample they have provided(Banzai-Executor) with modifying configuration as bellow.Here we are sending a message from Banzai(initiator to port 12000) to UltraESB(working as an acceptor listening in port 12000) over session FIX.4.2. After receiving the message from Banzai UltraESB(working as an initiator) will forward the message to Executor(acceptor listening in port 9879)

Initiator configuration for Banzai (banzai.cfg)

Acceptor configuration for Executor(executor.cfg)

Following is the UI provided by QuickFIX/J sample, you can send a message with this UI over session FIX.4.2:BANZAI->UESB.

Banzai console output

// initiating session and sending order message
<20140409-06:24:38, FIX.4.2:BANZAI->UESB, event> (Session FIX.4.2:BANZAI->UESB schedule is daily, 00:00:00-UTC - 23:00:00-UTC)
<20140409-06:24:38, FIX.4.2:BANZAI->UESB, event> (Session state is not current; resetting FIX.4.2:BANZAI->UESB)
<20140409-06:24:38, FIX.4.2:BANZAI->UESB, event> (Created session: FIX.4.2:BANZAI->UESB)
<20140409-06:24:39, FIX.4.2:BANZAI->UESB, outgoing> (8=FIX.4.29=6535=A34=149=BANZAI52=20140409-06:24:39.48856=UESB98=0108=3010=243)
<20140409-06:24:39, FIX.4.2:BANZAI->UESB, event> (Initiated logon request)
<20140409-06:24:39, FIX.4.2:BANZAI->UESB, incoming> (8=FIX.4.29=6535=A34=149=UESB52=20140409-06:24:39.56456=BANZAI98=0108=3010=238)
<20140409-06:24:39, FIX.4.2:BANZAI->UESB, event> (Received logon)
<20140409-06:25:09, FIX.4.2:BANZAI->UESB, outgoing> (8=FIX.4.29=12935=D34=249=BANZAI52=20140409-06:25:09.24456=UESB11=139702470918721=138=1040=154=155=dell59=060=20140409-06:25:09.24210=206)
<20140409-06:25:10, FIX.4.2:BANZAI->UESB, incoming> (8=FIX.4.29=5335=034=249=UESB52=20140409-06:25:10.15656=BANZAI10=181)

// receiving acknowledgement from Executor via UltraESB
<20140409-06:25:10, FIX.4.2:BANZAI->UESB, incoming> (8=FIX.4.29=12435=834=349=UESB52=20140409-06:25:10.68756=BANZAI6=011=139702470918714=017=120=037=139=054=155=dell150=2151=010=061)
<20140409-06:25:10, FIX.4.2:BANZAI->UESB, incoming> (8=FIX.4.29=14835=834=449=UESB52=20140409-06:25:10.68856=BANZAI6=12.311=139702470918714=1017=220=031=12.332=1037=238=1039=254=155=dell150=2151=010=130)

UltraESB console output

// receiving message from Banzai and sending to Executor
2014-04-09 11:54:39,537 [-] [SocketAcceptorIoProcessor-0.0] [system] [000000I]  INFO FIXTransportListener FIX session with SessionID: FIX.4.2:UESB->BANZAI created for fix listener: fix-lsnr
2014-04-09 11:54:39,578 [-] [QF/J Session dispatcher: FIX.4.2:UESB->BANZAI] [system] [000000I]  INFO FIXTransportListener FIX session with SessionID: FIX.4.2:UESB->BANZAI logged in via fix listener: fix-lsnr
2014-04-09 11:55:09,539 [-] [QF/J Session dispatcher: FIX.4.2:UESB->BANZAI] [system] [000000I]  INFO FIXSessionFactory Started initiator for endpoint address: FIX.4.2:UESB->EXEC with session ID: fix://localhost:9879?BeginString=FIX.4.2&SenderCompID=UESB&TargetCompID=EXEC 
2014-04-09 11:55:09,540 [-] [QF/J Session dispatcher: FIX.4.2:UESB->BANZAI] [system] [000000I]  INFO FIXTransportSender Fix message sent to target over session: FIX.4.2:UESB->EXEC

// receiving acknowledgement from Executor and sending back to Banzai
2014-04-09 11:55:10,687 [-] [QF/J Session dispatcher: FIX.4.2:UESB->EXEC] [system] [000000I]  INFO FIXTransportSender Acknowledgement message received from:UESB sent to message sender: BANZAI
2014-04-09 11:55:10,689 [-] [QF/J Session dispatcher: FIX.4.2:UESB->EXEC] [system] [000000I]  INFO FIXTransportSender Acknowledgement message received from:UESB sent to message sender: BANZAI

Executor console output

// receiving order message from UESB 
<20140409-07:08:34, FIX.4.2:EXEC->UESB, incoming> (8=FIX.4.29=15435=D34=143=Y49=UESB52=20140409-07:08:34.55056=EXEC122=20140409-07:08:3311=139702731319321=138=1040=154=155=dell59=060=20140409-07:08:33.21610=109)
<20140409-07:08:34, FIX.4.2:EXEC->UESB, incoming> (8=FIX.4.29=8935=434=243=Y49=UESB52=20140409-07:08:34.55256=EXEC122=20140409-07:08:3436=3123=Y10=125)
<20140409-07:08:34, FIX.4.2:EXEC->UESB, event> (ResendRequest for messages FROM 1 TO 1 has been satisfied.)

// sending acknowledgement to UESB 
<20140409-07:08:34, FIX.4.2:EXEC->UESB, outgoing> (8=FIX.4.29=12235=834=349=EXEC52=20140409-07:08:34.59056=UESB6=011=139702731319314=017=120=037=139=054=155=dell150=2151=010=163)
<20140409-07:08:34, FIX.4.2:EXEC->UESB, outgoing> (8=FIX.4.29=14635=834=449=EXEC52=20140409-07:08:34.59256=UESB6=12.311=139702731319314=1017=220=031=12.332=1037=238=1039=254=155=dell150=2151=010=233)<20140409-07:08:34, FIX.4.2:EXEC->UESB, incoming> (8=FIX.4.29=15435=D34=143=Y49=UESB52=20140409-07:08:34.55056=EXEC122=20140409-07:08:3311=139702731319321=138=1040=154=155=dell59=060=20140409-07:08:33.21610=109)
<20140409-07:08:34, FIX.4.2:EXEC->UESB, incoming> (8=FIX.4.29=8935=434=243=Y49=UESB52=20140409-07:08:34.55256=EXEC122=20140409-07:08:3436=3123=Y10=125)
<20140409-07:08:34, FIX.4.2:EXEC->UESB, event> (ResendRequest for messages FROM 1 TO 1 has been satisfied.)

<20140409-07:08:34, FIX.4.2:EXEC->UESB, outgoing> (8=FIX.4.29=12235=834=349=EXEC52=20140409-07:08:34.59056=UESB6=011=139702731319314=017=120=037=139=054=155=dell150=2151=010=163)
<20140409-07:08:34, FIX.4.2:EXEC->UESB, outgoing> (8=FIX.4.29=14635=834=449=EXEC52=20140409-07:08:34.59256=UESB6=12.311=139702731319314=1017=220=031=12.332=1037=238=1039=254=155=dell150=2151=010=233)

Introduction to Junit4

2014-08-01T04:44:00.004-07:00

Testing with Junit4 from Amila Paranawithana

Oauth2 in API Director - Autherization Code and Implicit

2014-06-10T23:39:00.001-07:00

End User Registration

End Users need to register in API Director Autherization Server

Authorization Code Grant Type

This grant type is most appropriate for server-side web applications. After the re-source owner has authorized access to their data, they are redirected back to the web application with an authorization code as a query parameter in the URL. This code must be exchanged for an access token by the client application. This ex-change is done server-to-server and requires both the client_id and client_secret, preventing even the resource owner from obtaining the access token.

1. Create an Access Key of 'web application' type

2. You will get client_id and secret.

3. Make a autherization request to APID autherization server.

http://localhost:8080/uaa/oauth/authorize?response_type=code&client_id=zapTechs.app1.sandBox.MyWebApp&redirect_uri=http://geomaster.com/

If the end user is not logged in will be redirected to login. If there is no cookie user's browser is redirected to /login, and will eventually come back to /oauth/authorize

Cookie: JSESSIONID=ADHGFKHDSJGFGF; Path / - the authentication cookie for the client with UAA.

4. User log in with username and password of his account in APID Autherization server.

5. User is asked to grant permission to access for his information in APID Autherization server account to web-application. Select the permissions you need to allow access to and click 'OK'. This means that this web application can access the information of your account in Autherization server.

6.Will get a redirect response where the location is the URL defined in redirect_uri param in request. You receive a code which can be used to do the token request.

HTTP/1.1 302 Found

Location: http://geomaster.com/?code=4jCNR6

7. Token request : (from toolbox)

GET:
http://localhost:8080/uaa/oauth/token?grant_type=authorization_code&code=4jCNR6 &redirect_uri=http://geomaster.com/

Request Header: Basic Auth

username: zapTechs.app1.sandBox.MyWebApp

password : 210277d6-103b-44e8-931d-86ac2ffd2e7d

8. Will receive a API access token

{"access_token":"eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiI0NDUwMjM3My05ZDJkLTRiNGEtOTViNS03OGM5ZDFlNTczYjEiLCJlbmRfdXNlcl9lbWFpbCI6bnVsbCwic3ViIjoiMjI5YTFiZTEtZTE5Ny00NTUxLWEyNTctZDFmMDlkMGU0NWYzIiwic2NvcGUiOlsic2NpbS51c2VyaWRzIiwicGFzc3dvcmQud3JpdGUiLCJvcGVuaWQiLCJjbG91ZF9jb250cm9sbGVyLndyaXRlIiwiY2xvdWRfY29udHJvbGxlci5yZWFkIl0sImNsaWVudF9pZCI6InphcFRlY2hzLmFwcDEuc2FuZEJveC5tYWxha2FJRCIsImNpZCI6InphcFRlY2hzLmFwcDEuc2FuZEJveC5tYWxha2FJRCIsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJ1c2VyX2lkIjoiMjI5YTFiZTEtZTE5Ny00NTUxLWEyNTctZDFmMDlkMGU0NWYzIiwidXNlcl9uYW1lIjoibWFsYWthZyIsImVtYWlsIjoibWFsYWthQGFkcm9pdGxvZ2ljLmNvbSIsImlhdCI6MTQwMjQ2MTg4MSwiZXhwIjoxNDAyNTA1MDgxLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvdWFhL29hdXRoL3Rva2VuIiwiYXVkIjpbInNjaW0iLCJvcGVuaWQiLCJjbG91ZF9jb250cm9sbGVyIiwicGFzc3dvcmQiXX0.2W14LtA4eRv_qt_msDriSMZ1LPIIAmlmnpGadHSMKxI","token_type":"bearer","refresh_token":"eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJlZWJjNzkzYy04YjJjLTRhMjAtOWMyMC1iN2ZiZGNmNmZjZWUiLCJzdWIiOiIyMjlhMWJlMS1lMTk3LTQ1NTEtYTI1Ny1kMWYwOWQwZTQ1ZjMiLCJzY29wZSI6WyJzY2ltLnVzZXJpZHMiLCJwYXNzd29yZC53cml0ZSIsIm9wZW5pZCIsImNsb3VkX2NvbnRyb2xsZXIud3JpdGUiLCJjbG91ZF9jb250cm9sbGVyLnJlYWQiXSwiaWF0IjoxNDAyNDYxODgxLCJleHAiOjE0MDUwNTM4ODEsImNpZCI6InphcFRlY2hzLmFwcDEuc2FuZEJveC5tYWxha2FJRCIsImlzcyI6Imh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC91YWEvb2F1dGgvdG9rZW4iLCJncmFudF90eXBlIjoiYXV0aG9yaXphdGlvbl9jb2RlIiwidXNlcl9uYW1lIjoibWFsYWthZyIsImF1ZCI6WyJzY2ltLnVzZXJpZHMiLCJwYXNzd29yZC53cml0ZSIsIm9wZW5pZCIsImNsb3VkX2NvbnRyb2xsZXIud3JpdGUiLCJjbG91ZF9jb250cm9sbGVyLnJlYWQiXX0.VfhH6X5AnyAYC-ZLwlyGaYh3C7uOIERiI8k9xp67nK8","expires_in":43199,"scope":"scim.userids password.write openid cloud_controller.write cloud_controller.read","end_user_email":null,"jti":"44502373-9d2d-4b4a-95b5-78c9d1e573b1"}

Implicit Grant Type

The implicit grant is the most simplistic of all flows, and is optimized for clientside web applications running in a browser. The resource owner grants access to the application, and a new access token is immediately minted and passed back to the application using a #hash fragment in the URL. The application can immediately extract the access token from the hash fragment (using JavaScript) and make API requests. This grant type does not require the intermediary “authorization code”.

1. Create an Access Key of 'Service account' type

2. You will get client_id and secret.

3. Make a token request

http://localhost:8080/uaa/oauth/authorize?response_type=token&client_id=zapTechs.app1.sandBox.malakaID&redirect_uri=http://amilaparanawithana.blogspot.com/

4. If the end user is not logged in will be redirected to login. If there is no cookie user's browser is redirected to /login, and will eventually come back to /oauth/authorize

5. User log in with username and password of his account in APID Autherization server.

6. User is asked to grant permission to access for his information in APID Autherization server account to web-application. Select the permissions you need to allow access to and click 'OK'. This means that this web application can access the information of your account in Autherization server.

7. Will get a redirect response where the location is the URI defined in redirect_uri param above with the API access token

http://amilaparanawithana.blogspot.com/#access_token=eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiIwOGMxYzRmYS03OThkLTQwYTAtYThhYy1jM2IxMzMzNTFlNzYiLCJlbmRfdXNlcl9lbWFpbCI6bnVsbCwic3ViIjoiMjhhMTQ2MGYtNWEzMC00ZDIyLTgxMmEtYzE2Y2NhNTBmMDE1Iiwic2NvcGUiOlsic2NpbS51c2VyaWRzIiwicGFzc3dvcmQud3JpdGUiLCJvcGVuaWQiLCJjbG91ZF9jb250cm9sbGVyLndyaXRlIiwiY2xvdWRfY29udHJvbGxlci5yZWFkIl0sImNsaWVudF9pZCI6InphcFRlY2hzLmFwcDEuc2FuZEJveC5TZXJ2aWNlSUQiLCJjaWQiOiJ6YXBUZWNocy5hcHAxLnNhbmRCb3guU2VydmljZUlEIiwidXNlcl9pZCI6IjI4YTE0NjBmLTVhMzAtNGQyMi04MTJhLWMxNmNjYTUwZjAxNSIsInVzZXJfbmFtZSI6ImFtaWxhcGFyYSIsImVtYWlsIjoiYW1pbGExMjA0QGdtYWlsLmNvbSIsImlhdCI6MTQwMjQ2ODMwMCwiZXhwIjoxNDAyNTExNTAwLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvdWFhL29hdXRoL3Rva2VuIiwiYXVkIjpbInNjaW0iLCJvcGVuaWQiLCJjbG91ZF9jb250cm9sbGVyIiwicGFzc3dvcmQiXX0.z_ULKCjlZdl0xAvRYAepMihBurw2NKCURvEMnkf_dJs&token_type=bearer&expires_in=43199&scope=scim.userids%20password.write%20openid%20cloud_controller.write%20cloud_controller.read&jti=08c1c4fa-798d-40a0-a8ac-c3b133351e7 6

Reference - https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-APIs.rst

Access APIs secured with OAuth2 in Adroitlogic API Director, from your Facebook authentication enabled mobile application

2014-03-31T10:18:00.002-07:00

Adroitlogic API Director supports securing your API services exposed through API Directer with OAuth2. It also supports getting end user information with connecting to facebook,google or twitter. The following post describes how OAuth2 'client credential' method is used in API Director to secure API service provider's services and how end user information is exposed with user permission and limitations by letting them to log via facebook(or google/twitter)

Main roles

APID_Publisher Create API, define permissions for API access with circles and tiers,publish API
APID_Consumer Create applications in APID (apid_application), subscribe for APIs, and build end_user web/mobile applications using subscribed APIs
End_user Consume end_user_applications built by APID Consumer.

API access flow

APID_Publisher publish an API in API Director.
APID_Consumer Creates an apid_application in APID.

a. Gets a Client_ID

b. Gets a Client_Secret

3. APID_Consumer subscribe to an API via APID_Application.
c. APID_Consumer subscribe to an API via APID_Application.

APID_Consumer creates an end_user_application(web/mobile). You can enable authentication to application through some other social media like Facebook or Google. For enabling authentication through facebook follow ‘Enabling facebook authentication’ section.

Following is the flow of a mobile application with facebook login and access APIs via API Director.

User click on facebook login button.
User redirected to facebook for loggin.
URL:https://m.facebook.com/login.php?client_id=326057350868420&scope=user_likes,user_status,email&redirect_uri=fbconnect
On successful login user redirected to facebook oauth dialog.
https://m.facebook.com/dialog/oauth?redirect_uri=fbconnect&scope=user_likes,user_status,email&client_id=326057350868420
User allow access for his facebook information and facebook redirect to mobile application with the facebook_access_token
fbconnect://success#access_token=CAAEojCSTLcQBAP5su0XO7U8j5H47YZCb9v9d0ab
b9RtG4kkl
Mobile application make a request to the APIDirector authorization server requesting for a API_access_token
URL :http://localhost:8182/uaa/oauth/token?
grant_type=client_credentials&token_type=fb&social_token=CAAEojCSTLcQBADVEiFKi8BY8Sq1PrhrMQDHPfr0kMnbq1hQfIAZAdPMzSHHmUCg6o...
Basic Authenntication User_name = client_id, password = client_secret
API Director authorization server validate facebook_access_token from facebook and retriew user information
If the user does not already exist in APIDirector , provisions a new user and persist in APID database .
APIDirector authorization server returns an API_access_token which
end_user_application can use for subsequent API calls. This token is issued only for that particular end_user to user through that particular end_user_application.
End_user_application make API call using API access url. API_access_token is sent in a custom header
Autherization: Bearer $API_access_token
On API request call APID_oauth2filter checks for the API_access_token and validate it from APID_autherization_server.
If the token is valid, let the end_user_application to access API.

Enabling Facebook authentication

To add facebook login to your application you can follow developer’s guide for facebook login. For a complete guide for creating android application with facebook login you can follow Getting Started with the Facebook SDK for Android.

Create a facebook app on the Facebook App Dashboard entering your app's basic information.
Create your web/mobile application with facebook login adding your facebook app_id in your project.
Set permissions (eg: "user_likes", "user_status","email") to your login button. Since API Director is persisting end_user details with using email as the unique key application developer must include ‘email’ permission in this permission list.
Invoke On successful login Facebook OAuth dialog is invoked.
If the end_user allow permission to access his information from facebook he will click on ‘OK.’

Extract elements with JSONPath in UltraESB JSONSupport

2014-03-26T11:10:00.000-07:00

UltraESB supports accessing contents from a JSON payload with JSON path. JSON path is a simple way to extract parts of a given document with faster results and less processing overhead

Find API at JSONSupport - extractUsingJSONPath

JSON paylaod

{

"OutterNode": {

"InnerNode1": [

{ "id": 55555,

"string_id": "55555A",

"node_value": 25,

"object": {"aa":1},

"valueNull": null,

"booleanVal":true,

"valueArray":[ 1, 2, [ [], [{"inner_id": 5000}] ] ]

{ " id": 66666,

"string_id": "66666B",

"node_value": 15,

"object": {"bb":2},

"valueNull": null,

"booleanVal":false,

"valueArray":[ 1, 2, [ [], [{"inner_id": 6000}] ] ]

}

"InnerNode2": {

"name": "second_innerNode",

"value": 200.15

}

Expression	Result
$.OutterNode.InnerNode1[0].string_id	"55555A"
$.OutterNode.InnerNode1[0].node_value	25
$.OutterNode.InnerNode1[0].booleanVal	true
$.OutterNode.InnerNode1[1].booleanVal	false
$.OutterNode.InnerNode1[1].valueNull	null
$.OutterNode.InnerNode1[*].id	[55555,66666]
$.OutterNode.InnerNode1[*].string_id	["55555A","66666B"]
$.OutterNode.InnerNode1[?(@.node_value > 16)].id	[55555]
$.OutterNode.InnerNode1[?(@.node_value > 16)].id	[55555]

JSON payload as an Array

[

{

"a": 1,

"b": 2,

"value": 40,

"string_val":"aa",

"nullElement":null,

{

"a": 4,

"b": 5,

"value": 35,

"node_value": 15,

"string_val":"bb",

"booleanElement":false

}

]

Expression	Result
$.[0].string_val	"aa"
$.[1].a	4
$.[0].booleanElement	true
$.[1].booleanElement	false
$.[0].nullElement	null
$.[*].a	[1,4]
$.[*].string_val	["aa","bb"]
$.[?(@.value > 35)].a	[1]

Find more information on JSONPath here
JSON Path tester online

Delete all tables from database

2014-02-23T11:49:00.003-08:00

If you want to delete all the tables from a database there are two methods.

Drop the database and re-create

This is the easiest way. You can drop the database with 'Drop database <database>;' and create again with 'create database <database>'

2. Using a script to delete all the tables.

If you do not have permission to create new databases(eg: you do not have root permission) you can not use the above method. So in that case you use the following script(deleteTabels.sh) and run with command 'sh deleteTabels.sh <host> <database> <user> <password>'. This will create all the tables ignoring the foreign keys

host=$1
dbName=$2
user=$3
password=$4

if [ -z "$1" ]
then
host="localhost"
fi

# drop all the tables in the database
for i in `mysql -u$user -p$password $dbName -e "show tables" | grep -v Tables_in` ; do echo $i && mysql -u$user -p$password $dbName -e "SET FOREIGN_KEY_CHECKS = 0; drop table $i ; SET FOREIGN_KEY_CHECKS = 1" ; done

OAuth 2 + API security

2013-11-16T10:58:00.002-08:00

Writing a JAX-WS web service with maven structured and deployed on tomcat

2013-10-07T11:03:00.003-07:00

This post describes how to write a sample JAX-WS web service. Here maven is used to build the project and tomcat to deploy the web service.

1. Create a new maven project. Add JAX-WS dependency and build.

2. Write the WebService class HelloWebService.java in src/main/java/com.webservices.demo/ and add your web methods. Use @WebService and @WebMethod annotations for your service class and methods.

     3. Create sun-jaxws.xml in /main/webapp/WEB-INF and include endpoint details,
         name- endpoint name
         implementation - path of your webService class
         url-pattern - url pattern you want to call the service

4. Create web.xml in /main/webapp/WEB-INF
Following is the final project structure

5. Build the project with maven (mvn clean install)

6. Deploy it with tomcat by coping the helloWebService.war created inside target directory to /apache-tomcat-x.x.x/webapps folder and run tomcat

Your web service can be accessed by url http://localhost:8182/HelloService/helloWebService
WSDL with url http://localhost:8182/HelloService/helloWebService?wsdl
(assuming tomcat run in localhost:8182)

Following is how this service is tested by sending soap messages with soapUI

Apache Shiro annotation based security for grails controller actions

2013-08-02T02:28:00.000-07:00

Apache Shiro plugging for Grails which is used to integrate Apache Shiro security framework to grails applications provides easy authentication and access control via roles and permissions. Installation of this plugging can be found in following blogs.

In above blogs autherization is done by using securityFilter where all the nonAuthenticatedActions and authenticatedActions are given in static arrays.

Apache Shiro provides annotation based authentication and authorization(link). Following are the available shiro annotaions.

org.apache.shiro.authz.annotation.RequiresAuthentication
org.apache.shiro.authz.annotation.RequiresGuest
org.apache.shiro.authz.annotation.RequiresPermissions
org.apache.shiro.authz.annotation.RequiresRoles
org.apache.shiro.authz.annotation.RequiresUser

Shiro grails plugging also provides annotations based security. But currently only Grails services and other Spring beans can be annotated and this feature isn't available in controllers.

Following is how I tried to add annotation based security for grails controllers.

First lets write a custom java annotaion 'RolePermission' in src/java/. Here as I have defined Target 'ElementType.METHOD, ElementType.TYPE' which allows you to add this annotaion to both controller class and methods. Here I assume that there are 3 roles in this application, ROLE_CONSUMER, ROLE_ADMIN and ROLE_SUPER_USER

 
import java.lang.annotation.*;

@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface RolePermission {
    String[] roles();
}

Then add this annotaion to your controllers

Then we need to read these controller action permissions by annotations at the bootStrap and create static array authenticatedActions which holds [controller: <controllerName>, action: <actionName>, roles: roles] objects. This array defines what actions of what controllers are allowed by which roles.

Call the init() method of ControllerPermissionReader class at the BootStrap

To filter and authorize access to controller actions we can write a grails filter , 'SecurityFilters' which basically filter requests to controller actions based on user roles by looking at the static arrays nonAuthenticatedActions and authenticatedActions we created.

Spring Collections (List, Set, Map, and Properties)

2013-07-20T12:03:00.001-07:00

There are four collection types supported by spring which are List(<list/>), Set(<set/>), Map(<map/>) and Properties(<props/>). This post will explain the usage of it with a simple sample.

Lets create a simple spring bean class with all four collections included.

CollectionHome class

To insert values to collections of spring bean we have to configure that in spring-config.xml. The following shows how to insert values to each collection in your bean configuration xml file.

Lists

Can insert values, other beans by giving reference id or beans defined on place

Sets

Maps

Can insert values, other beans by giving reference id or beans defined on place for values of map

Properties

spring-config.xml

Testing

Results

List: [1, com.spring.collections.demo.Entity@6855a338, com.spring.collections.demo.Entity@c68c26c]

Set : [1, com.spring.collections.demo.Entity@6855a338, com.spring.collections.demo.Entity@4e4ee70b]

Map : {Key 1=1, Key 2=com.spring.collections.demo.Entity@6855a338, Key 3=com.spring.collections.demo.Entity@31b446d1}

Props : {post=spring collections, blog=http://amilaparanawithana.blogspot.com/}

Kanthaka: Big Data Caller Detail Record (CDR) Analyzer for Near Real Time Telecom Promotions

2013-07-14T05:18:00.002-07:00

This is the research paper written during my final year research project 'Kanthaka: Big Data Caller Detail Record (CDR) Analyzer for Near Real Time Telecom Promotions' as an undergraduate at Department of Computer Science and Engineering, University of Moratuwa, Srilanka. It was a project of 4 students P. Jayawardhana, D. Perera, M.Kumar and myself.

This was presented and published at Intelligent Systems Modelling & Simulation (ISMS), 2013 4th International Conference which was held on 29-31 Jan. 2013 at Bangkok, Thailand .

It is now available at IEEE Xplore Digital Library(link)

Abstract

With the competitiveness that is growing in the telecom industry, the operators seek for ways to attract and keep the subscribers in their network. Giving away attractive promotions to their subscribers is a powerful and commonly used approach in that context. With the technical limitations and the scales of the rising subscriber base, these promotions have been limited to a very narrow pattern. In order to gain a competitive advantage, the operators need new technologies and methodologies to support more attractive newer patterns of promotions, despite the challenges involved in complex analyzing procedures. In this research study, we have come up with a system to select eligible users for a particular predefined promotion, analyzing the Caller Detail Records, while being scalable for a rapidly escalating subscriber base.

Interceptors - UltraESB

2013-07-07T11:29:00.000-07:00

In UltraESB the Interceptors are used to mediate the message and perform any action you need to do such as auditing, loggin, throttling, time measuring . The main difference between an interceptor and a filter in UltraESB is,

Filters - are global and will be applied to all the coming messages. Filter execution happens with the transport dispatcher threads.

Interceptors - can be invoked in particular places in message flow. Interceptor invocation happens in a worker thread.

The place you want to invoke the interceptor can be specified by a class level annotation.Following are the types of interceptors based on the places you can invoke it.

onRequestInterceptors - (@OnRequest)
onResponseInterceptors - (@OnResponse)
beforeSequenceInterceptors - (@BeforeSequence)
afterSequenceInterceptors - (@AfterSequence)
beforeEndpointInterceptors - (@BeforeEndpoint)
afterEndpointInterceptors - (@AfterEndpoint)

<InterceptionHandler> - The interception handler will scan the classpath and find all the AbstractMessageInterceptor implementations and categorize them based on the class level annotations specifying any MessageInterceptorCriteria.

MessageInterceptionHandler - MessageInterceptionHandler extends InterceptionHandler. This will invoke the interceptors you have specified with class level annotations as mentioned above on the messaging events (onRequest, onResponse, beforeSequence, afterSequence, beforeEndpoint, afterEndpoin).

<InterceptorHolder> - Helps the ordering of execution of the interceptors. This is usually specified in the interceptor annotation {@link MessageInterceptorHolder}.

MessageInterceptorHolder - MessageInterceptorHolder extends InterceptorHolder. Methods of this are invoked by different types of interceptors mentioned above

Following is a sample structure of a interceptor. Lets assume that you want to throttle/ limit your requests based on Tiers.

@OnRequest
public class TierLimitInterceptor extends AbstractMessageInterceptor {

    @Override
    protected boolean intercept(MessageEvent event) {

        Message message = event.getMessage();
        
        // your throttling code
        ......
         
        if (!tierThrottle.isAllowed(<>, message)) {
                sendResponse(503, "You have exceeded the tier rate limit, please try again later", message, null);
            }
        
        return true;
    }
}

Additional information on setting up ultraesb for custom Interceptors you have written

In ultra-root.xml configuration file under "ultra-config" bean define the property extentionPakages with adding your Interceptor class package

<property name="extensionPackages">
            <list>
                <value>org.adroitlogic.ultraesb.apid</value>
            </list>
        </property>

WADL support for UltraESB

2013-07-07T10:23:00.002-07:00

Grails searchable pluggin to your maven structured project

2013-04-11T23:16:00.000-07:00

Grails searchable pluggin gives you capability to add search functionality to your grails project. With this pluggin you can index your domain classes. It is built on the Compass Search Engine Framework and Apache Lucene.

The main advantage I experienced while using lucene indexing for searching is, it is simpler and faster than doing %LIKE% query on relational database. The underline method of tokenizing words and indexing with lucene can be found from here. Here I explain sample codes I used in my maven structured grails project.

Maven dependencies you need.

I assume There is a Domain class ‘Book’ and a Controller ‘BookController’ and a method ‘search’ in BookController class.

This is a part of the list.gsp where the books are listed.

Domain class

Controller class

Service class

helpful link